2024-09-21T06:29:48.174 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T06:29:48.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:29:48.283 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913559' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi114.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGzd4feLN9hBqeSDSNK+p/I3v3lNj/lfE9hpf7H1DQ1JGELq906sef325s7HcELTfa1pwRseJOtM4NQkMtS7Iek= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMc0uFkO9pCm3Gd9uPRtySdbORsgM55Sok4EQ8uJfMWfMCRiLZoYGBgAa4Q2iLKSJ9goOVSQHw1QxPNOFixAy9c= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDkq9Y8C0ijEb8uxI/J+yv3Qn/ruNTDRWueIdfe8ivW7C3cQYcvBkS1Kr1MmsjYP2FI83UkuI7g5QQbUMmD3EIk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T06:29:48.283 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T06:29:48.284 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T06:29:48.284 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T06:29:48.285 INFO:teuthology.task.internal:Checking packages... 2024-09-21T06:29:48.306 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T06:29:48.306 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T06:29:48.306 INFO:teuthology.packaging:ref: None 2024-09-21T06:29:48.306 INFO:teuthology.packaging:tag: None 2024-09-21T06:29:48.306 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:29:48.306 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:29:48.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T06:29:48.448 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T06:29:48.449 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T06:29:48.460 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T06:29:48.460 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T06:29:48.511 INFO:teuthology.task.internal:Saving configuration 2024-09-21T06:29:48.533 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T06:29:48.547 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T06:29:48.569 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi114.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:24:48.685814', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGzd4feLN9hBqeSDSNK+p/I3v3lNj/lfE9hpf7H1DQ1JGELq906sef325s7HcELTfa1pwRseJOtM4NQkMtS7Iek='} 2024-09-21T06:29:48.588 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:24:48.683447', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMc0uFkO9pCm3Gd9uPRtySdbORsgM55Sok4EQ8uJfMWfMCRiLZoYGBgAa4Q2iLKSJ9goOVSQHw1QxPNOFixAy9c='} 2024-09-21T06:29:48.611 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 06:24:48.686953', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDkq9Y8C0ijEb8uxI/J+yv3Qn/ruNTDRWueIdfe8ivW7C3cQYcvBkS1Kr1MmsjYP2FI83UkuI7g5QQbUMmD3EIk='} 2024-09-21T06:29:48.611 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T06:29:48.622 INFO:teuthology.task.internal:roles: ubuntu@smithi114.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T06:29:48.623 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T06:29:48.623 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T06:29:48.623 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T06:29:48.724 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcc4c134160>, signals=[15]) 2024-09-21T06:29:48.724 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T06:29:48.734 INFO:teuthology.task.internal:Opening connections... 2024-09-21T06:29:48.735 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:29:48.736 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:29:48.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-21T06:29:48.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:29:48.887 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T06:29:48.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:29:48.971 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T06:29:49.004 DEBUG:teuthology.orchestra.run.smithi114:> uname -m 2024-09-21T06:29:49.033 INFO:teuthology.orchestra.run.smithi114.stdout:x86_64 2024-09-21T06:29:49.033 DEBUG:teuthology.orchestra.run.smithi114:> cat /etc/os-release 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:NAME="CentOS Stream" 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION="9" 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:ID="centos" 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:VERSION_ID="9" 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:29:49.095 INFO:teuthology.orchestra.run.smithi114.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:ANSI_COLOR="0;31" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:29:49.096 INFO:teuthology.orchestra.run.smithi114.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:29:49.097 INFO:teuthology.lock.ops:Updating smithi114.front.sepia.ceph.com on lock server 2024-09-21T06:29:49.118 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-21T06:29:49.137 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-21T06:29:49.138 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-21T06:29:49.197 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:29:49.198 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:29:49.199 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:29:49.199 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:29:49.199 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-21T06:29:49.227 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-21T06:29:49.246 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-21T06:29:49.247 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-21T06:29:49.306 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T06:29:49.307 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-21T06:29:49.307 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T06:29:49.307 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T06:29:49.307 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T06:29:49.307 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-21T06:29:49.325 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T06:29:49.340 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T06:29:49.367 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T06:29:49.367 DEBUG:teuthology.orchestra.run.smithi114:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:29:49.370 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:29:49.373 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-21T06:29:49.390 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T06:29:49.415 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T06:29:49.415 DEBUG:teuthology.orchestra.run.smithi114:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:29:49.429 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:29:49.431 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-21T06:29:49.575 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T06:29:49.647 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T06:29:49.662 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T06:29:49.662 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-21T06:29:49.662 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-21T06:29:49.662 DEBUG:teuthology.orchestra.run.smithi114:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:29:49.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:49.680 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T06:29:49.847 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:29:49.847 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T06:29:49.847 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-21T06:29:49.865 INFO:teuthology.orchestra.run.smithi114.stdout:5.14.0-503.el9.x86_64 2024-09-21T06:29:49.865 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:29:49.865 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T06:29:49.866 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum upgrade -y linux-firmware 2024-09-21T06:29:52.127 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-21T06:29:53.895 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-21T06:29:58.262 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-21T06:30:00.246 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-21T06:30:01.275 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-21T06:30:06.931 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T06:30:08.926 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: Package Architecture Version Repository Size 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-21T06:30:08.945 INFO:teuthology.orchestra.run.smithi114.stdout:================================================================================ 2024-09-21T06:30:08.946 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 1 Package 2024-09-21T06:30:08.946 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:30:08.946 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 388 M 2024-09-21T06:30:08.946 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-21T06:30:40.171 INFO:teuthology.orchestra.run.smithi114.stdout:linux-firmware-20240905-146.el9.noarch.rpm 12 MB/s | 388 MB 00:31 2024-09-21T06:30:40.172 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:30:40.172 INFO:teuthology.orchestra.run.smithi114.stdout:Total 12 MB/s | 388 MB 00:31 2024-09-21T06:30:43.287 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-21T06:30:43.623 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-21T06:30:43.623 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-21T06:30:46.906 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-21T06:30:46.907 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-21T06:30:56.292 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-21T06:30:56.980 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:30:57.061 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:30:57.556 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:30:57.556 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T06:30:58.345 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T06:30:58.345 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:30:58.345 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-09-21T06:30:58.345 INFO:teuthology.orchestra.run.smithi114.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T06:30:58.345 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:30:58.345 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-21T06:30:58.667 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T06:30:58.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T06:30:58.808 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T06:30:58.808 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T06:30:58.811 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi114.front.sepia.ceph.com to complete... 2024-09-21T06:31:05.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21 06:31:05 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T06:31:05.153 INFO:teuthology.orchestra.run.smithi114.stderr:FINISHED --2024-09-21 06:31:05-- 2024-09-21T06:31:05.153 INFO:teuthology.orchestra.run.smithi114.stderr:Total wall clock time: 6.3s 2024-09-21T06:31:05.153 INFO:teuthology.orchestra.run.smithi114.stderr:Downloaded: 1 files, 343M in 6.0s (57.1 MB/s) 2024-09-21T06:31:05.155 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T06:31:05.155 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T06:31:07.210 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying... ######################################## 2024-09-21T06:31:07.274 INFO:teuthology.orchestra.run.smithi114.stdout:Preparing... ######################################## 2024-09-21T06:31:07.333 INFO:teuthology.orchestra.run.smithi114.stdout:Updating / installing... 2024-09-21T06:31:15.342 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T06:31:55.290 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi114.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T06:31:55.291 DEBUG:teuthology.orchestra.run.smithi114:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T06:31:55.372 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T06:31:55.373 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T06:31:55.374 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T06:31:55.375 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T06:31:55.376 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T06:31:55.377 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T06:31:55.378 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T06:31:55.379 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T06:31:55.380 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T06:31:55.381 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T06:31:55.382 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T06:31:55.383 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T06:31:55.384 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T06:31:55.385 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T06:31:55.386 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T06:31:55.387 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T06:31:55.388 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T06:31:55.389 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T06:31:55.390 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T06:31:55.391 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T06:31:55.392 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T06:31:55.393 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T06:31:55.394 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T06:31:55.395 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T06:31:55.396 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T06:31:55.397 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T06:31:55.398 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T06:31:55.399 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T06:31:55.400 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T06:31:55.401 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T06:31:55.402 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T06:31:55.403 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T06:31:55.404 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T06:31:55.405 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T06:31:55.406 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T06:31:55.407 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T06:31:55.408 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T06:31:55.409 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T06:31:55.410 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T06:31:55.411 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T06:31:55.412 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T06:31:55.413 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T06:31:55.414 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T06:31:55.415 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T06:31:55.416 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T06:31:55.417 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T06:31:55.418 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T06:31:55.419 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T06:31:55.420 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T06:31:55.421 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T06:31:55.422 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T06:31:55.423 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T06:31:55.424 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T06:31:55.425 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T06:31:55.426 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T06:31:55.427 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T06:31:55.428 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T06:31:55.429 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T06:31:55.430 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T06:31:55.431 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T06:31:55.432 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T06:31:55.433 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T06:31:55.434 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T06:31:55.435 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T06:31:55.436 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T06:31:55.437 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T06:31:55.438 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T06:31:55.439 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T06:31:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T06:31:55.441 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T06:31:55.442 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T06:31:55.443 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T06:31:55.444 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T06:31:55.445 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T06:31:55.446 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T06:31:55.447 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T06:31:55.448 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T06:31:55.449 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T06:31:55.450 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T06:31:55.451 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T06:31:55.452 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T06:31:55.453 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T06:31:55.454 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T06:31:55.455 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T06:31:55.456 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T06:31:55.457 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T06:31:55.458 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T06:31:55.459 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T06:31:55.460 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T06:31:55.461 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T06:31:55.462 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T06:31:55.463 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T06:31:55.464 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T06:31:55.465 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T06:31:55.466 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T06:31:55.467 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T06:31:55.468 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T06:31:55.469 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T06:31:55.470 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T06:31:55.471 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T06:31:55.472 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T06:31:55.473 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T06:31:55.474 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T06:31:55.475 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T06:31:55.476 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T06:31:55.477 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T06:31:55.478 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T06:31:55.479 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T06:31:55.480 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T06:31:55.481 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T06:31:55.482 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T06:31:55.483 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T06:31:55.484 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T06:31:55.485 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T06:31:55.486 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T06:31:55.487 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T06:31:55.488 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T06:31:55.489 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T06:31:55.490 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T06:31:55.491 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T06:31:55.492 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T06:31:55.493 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T06:31:55.494 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T06:31:55.495 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T06:31:55.496 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T06:31:55.497 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T06:31:55.498 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T06:31:55.499 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T06:31:55.500 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T06:31:55.501 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T06:31:55.502 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T06:31:55.503 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T06:31:55.504 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T06:31:55.505 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T06:31:55.506 INFO:teuthology.orchestra.run.smithi114.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:31:55.510 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.510 DEBUG:teuthology.orchestra.run.smithi114:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T06:31:55.533 INFO:teuthology.orchestra.run.smithi114.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T06:31:55.533 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:31:55.533 INFO:teuthology.orchestra.run.smithi114.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T06:31:55.533 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:for file in vmlinuz System.map config; do 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: fi 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:done 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-09-21T06:31:55.534 INFO:teuthology.orchestra.run.smithi114.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T06:31:55.535 INFO:teuthology.orchestra.run.smithi114.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T06:31:55.535 INFO:teuthology.orchestra.run.smithi114.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.535 INFO:teuthology.orchestra.run.smithi114.stdout:fi 2024-09-21T06:31:55.535 DEBUG:teuthology.orchestra.run.smithi114:> sudo rpm -qi grub2-tools 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Name : grub2-tools 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Epoch : 1 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Version : 2.06 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Release : 82.el9 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Architecture: x86_64 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Group : Unspecified 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:Size : 8274670 2024-09-21T06:31:55.609 INFO:teuthology.orchestra.run.smithi114.stdout:License : GPLv3+ 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Packager : builder@centos.org 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Vendor : CentOS 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Summary : Support tools for GRUB. 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:Description : 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:hardware devices. 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:31:55.610 INFO:teuthology.orchestra.run.smithi114.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T06:31:55.611 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T06:31:55.612 INFO:teuthology.task.kernel:Updating grub on smithi114 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:55.612 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T06:31:56.166 INFO:teuthology.orchestra.run.smithi114.stderr:Generating grub configuration file ... 2024-09-21T06:31:57.521 INFO:teuthology.orchestra.run.smithi114.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T06:31:57.553 INFO:teuthology.orchestra.run.smithi114.stderr:done 2024-09-21T06:31:57.556 DEBUG:teuthology.orchestra.run.smithi114:> mktemp 2024-09-21T06:31:57.595 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/tmp.rBi7uD2qOG 2024-09-21T06:31:57.595 DEBUG:teuthology.orchestra.run.smithi114:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rBi7uD2qOG 2024-09-21T06:31:57.672 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0666 /tmp/tmp.rBi7uD2qOG 2024-09-21T06:31:57.817 DEBUG:teuthology.orchestra.remote:smithi114:/tmp/tmp.rBi7uD2qOG is 6KB 2024-09-21T06:31:57.838 DEBUG:teuthology.orchestra.run.smithi114:> rm -fr /tmp/tmp.rBi7uD2qOG 2024-09-21T06:31:57.852 DEBUG:teuthology.orchestra.run.smithi114:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T06:31:57.915 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T06:31:57.915 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T06:31:57.915 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T06:31:57.915 INFO:teuthology.orchestra.run.smithi114.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T06:31:57.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:31:58.066 DEBUG:teuthology.orchestra.run.smithi114:> sudo shutdown -r now 2024-09-21T06:32:28.068 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T06:32:28.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:32:28.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:32:46.579 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-21T06:32:55.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:32:55.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:32:58.643 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-21T06:33:10.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:33:10.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:33:29.071 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.114 2024-09-21T06:33:44.071 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:33:44.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:33:44.461 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-21T06:33:45.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:33:45.038 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T06:33:45.038 DEBUG:teuthology.orchestra.run.smithi114:> uname -r 2024-09-21T06:33:45.072 INFO:teuthology.orchestra.run.smithi114.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:33:45.072 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T06:33:45.072 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T06:33:45.072 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T06:33:46.073 DEBUG:teuthology.parallel:result is None 2024-09-21T06:33:46.074 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T06:33:46.080 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T06:33:46.080 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:33:46.084 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:33:46.087 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T06:33:46.115 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T06:33:46.121 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T06:33:46.152 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T06:33:46.152 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:33:46.159 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:33:46.161 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T06:33:46.273 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T06:33:46.279 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T06:33:46.280 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:33:46.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:46.297 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T06:33:46.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:46.314 DEBUG:teuthology.orchestra.run.smithi114:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:33:46.318 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:33:46.340 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T06:33:46.372 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:33:46.387 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:33:46.423 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:33:46.440 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:33:46.470 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:33:46.514 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T06:33:46.517 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T06:33:46.524 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T06:33:46.525 DEBUG:teuthology.orchestra.run.smithi114:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:33:46.528 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:33:46.531 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T06:33:46.584 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T06:33:46.592 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T06:33:46.593 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:33:46.597 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:33:46.600 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T06:33:46.628 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:33:46.732 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:33:46.804 DEBUG:teuthology.orchestra.run.smithi114:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:33:46.869 DEBUG:teuthology.orchestra.run.smithi114:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:33:46.939 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:33:46.939 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:33:47.017 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:33:47.063 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:33:47.123 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:33:47.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:33:47.209 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:33:47.210 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:33:47.275 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:33:47.320 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T06:33:47.374 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:33:47.455 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T06:33:47.500 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:33:47.500 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T06:33:47.567 DEBUG:teuthology.orchestra.run.smithi114:> sudo service rsyslog restart 2024-09-21T06:33:47.571 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-21T06:33:47.573 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-21T06:33:47.607 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:33:47.646 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:33:47.664 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:33:48.120 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T06:33:48.129 INFO:teuthology.task.internal:Starting timer... 2024-09-21T06:33:48.129 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T06:33:48.180 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T06:33:48.213 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T06:33:48.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo service auditd rotate 2024-09-21T06:33:48.285 INFO:teuthology.orchestra.run.smithi114.stdout:Rotating logs: 2024-09-21T06:33:48.289 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-21T06:33:48.335 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-21T06:33:48.337 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-21T06:33:48.424 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-21T06:33:48.427 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T06:33:48.427 DEBUG:teuthology.orchestra.run.smithi114:> /usr/sbin/getenforce 2024-09-21T06:33:48.475 INFO:teuthology.orchestra.run.smithi114.stdout:Permissive 2024-09-21T06:33:48.476 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-21T06:33:48.505 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-21T06:33:48.505 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-21T06:33:48.538 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-21T06:33:48.538 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi114.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-09-21T06:33:48.539 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:33:48.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:48.597 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:33:48.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:48.626 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:33:48.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:33:48.652 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T06:33:48.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:33:48.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:33:48.877 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-21T06:33:48.905 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T06:33:48.915 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T06:33:48.918 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T06:33:49.039 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T06:33:49.049 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T06:33:49.050 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi114.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T06:38:29.247 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi114.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-09-21T06:38:29.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:38:29.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi114.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:38:29.338 DEBUG:teuthology.orchestra.run.smithi114:> true 2024-09-21T06:38:29.447 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi114.front.sepia.ceph.com' 2024-09-21T06:38:29.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-21T06:38:29.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:38:29.521 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-21T06:38:29.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-21T06:38:29.603 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T06:38:29.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T06:38:29.677 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-21T06:38:29.722 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-21T06:38:29.723 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T06:38:29.732 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T06:38:29.733 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:38:29.733 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:38:29.737 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:38:29.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:38:29.739 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T06:38:29.739 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:38:29.766 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:38:29.780 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:38:29.791 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:38:29.792 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T06:38:29.804 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-21T06:38:29.805 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:38:29.815 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-21T06:38:29.821 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T06:38:29.829 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:38:29.829 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-21T06:38:29.840 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-21T06:38:29.843 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:38:29.854 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:38:29.868 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:38:29.872 INFO:teuthology.orchestra.run.smithi114.stderr:sudo: ntpd: command not found 2024-09-21T06:38:29.896 INFO:teuthology.orchestra.run.smithi114.stdout:506 Cannot talk to daemon 2024-09-21T06:38:29.900 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:38:29.903 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:38:29.903 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-21T06:38:29.903 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.903 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.904 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.904 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.925 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T06:38:29.929 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:38:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:38:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-21T06:38:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.933 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:29.955 INFO:teuthology.orchestra.run.smithi114.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T06:38:30.052 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:38:30.059 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:38:30.059 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-21T06:38:30.059 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:30.059 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:30.059 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:30.059 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:38:30.060 INFO:teuthology.run_tasks:Running task install... 2024-09-21T06:38:30.069 DEBUG:teuthology.task.install:project ceph 2024-09-21T06:38:30.069 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T06:38:30.069 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T06:38:30.069 INFO:teuthology.task.install:Using flavor: default 2024-09-21T06:38:30.074 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T06:38:30.074 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T06:38:30.075 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:38:30.075 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:38:30.076 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:38:30.077 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:38:30.077 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T06:38:30.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:38:30.229 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:38:30.230 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:38:30.233 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:38:30.233 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:38:30.236 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T06:38:30.236 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T06:38:30.342 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:38:30.342 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:38:30.342 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:38:30.348 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:38:30.349 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:38:30.349 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:38:30.359 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T06:38:30.359 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:38:30.359 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T06:38:30.375 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:38:30.375 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:38:30.387 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:38:30.387 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:38:30.399 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T06:38:30.399 DEBUG:teuthology.orchestra.run.smithi114:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T06:38:30.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:38:30.454 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:38:30.496 DEBUG:teuthology.orchestra.run.smithi114:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T06:38:30.516 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:38:30.528 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:38:30.574 DEBUG:teuthology.orchestra.run.smithi114:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T06:38:30.580 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-09-21T06:38:30.581 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-09-21T06:38:30.592 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-09-21T06:38:30.594 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-09-21T06:38:30.629 INFO:teuthology.orchestra.run.smithi114.stdout:check_obsoletes = 1 2024-09-21T06:38:30.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum clean all 2024-09-21T06:38:30.806 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-09-21T06:38:30.817 INFO:teuthology.orchestra.run.smithi193.stdout:45 files removed 2024-09-21T06:38:30.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:38:30.844 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:38:30.910 INFO:teuthology.orchestra.run.smithi114.stdout:45 files removed 2024-09-21T06:38:30.942 DEBUG:teuthology.orchestra.run.smithi114:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T06:38:31.389 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-21T06:38:31.393 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-21T06:38:31.626 INFO:teuthology.orchestra.run.smithi114.stdout:ceph packages for x86_64 294 kB/s | 84 kB 00:00 2024-09-21T06:38:31.676 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-21T06:38:31.681 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T06:38:31.935 INFO:teuthology.orchestra.run.smithi114.stdout:ceph noarch packages 92 kB/s | 24 kB 00:00 2024-09-21T06:38:31.952 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T06:38:31.956 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T06:38:32.219 INFO:teuthology.orchestra.run.smithi114.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T06:38:32.461 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-21T06:38:32.557 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.3 MB 00:00 2024-09-21T06:38:34.296 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-21T06:38:34.641 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-21T06:38:36.336 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.3 MB 00:04 2024-09-21T06:38:38.307 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-21T06:38:40.200 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-21T06:38:40.313 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-21T06:38:41.751 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-21T06:38:41.797 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-21T06:38:42.876 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-21T06:38:43.022 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-21T06:38:43.279 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T06:38:45.030 INFO:teuthology.orchestra.run.smithi114.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-21T06:38:45.976 INFO:teuthology.orchestra.run.smithi114.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T06:38:48.315 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-21T06:38:48.377 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-21T06:38:50.451 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:38:50.452 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:38:50.463 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:38:50.464 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:38:50.465 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:38:50.466 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:38:50.467 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:38:50.470 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:38:50.471 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:38:50.523 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:38:50.524 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:38:50.535 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-21T06:38:50.538 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:38:50.539 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:38:50.540 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:38:50.541 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:38:50.543 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:38:50.545 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:38:50.547 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:38:50.555 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T06:38:50.555 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-09-21T06:38:50.555 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:38:50.556 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:38:50.557 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:38:50.558 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:38:50.559 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:38:50.560 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:38:50.561 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:38:50.562 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:38:50.563 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:38:50.564 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:38:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:38:50.566 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout:Install 149 Packages 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 8 Packages 2024-09-21T06:38:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:38:50.568 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 228 M 2024-09-21T06:38:50.568 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-21T06:38:50.610 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-21T06:38:50.641 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-09-21T06:38:50.641 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-21T06:38:50.641 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-09-21T06:38:50.641 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:38:50.642 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:38:50.643 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:38:50.644 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:38:50.645 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:38:50.646 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:38:50.647 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:38:50.648 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:38:50.649 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:38:50.650 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:38:50.651 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:38:50.652 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:38:50.653 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout:Install 149 Packages 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 8 Packages 2024-09-21T06:38:50.654 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:38:50.655 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 228 M 2024-09-21T06:38:50.655 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-21T06:38:51.184 INFO:teuthology.orchestra.run.smithi193.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-09-21T06:38:51.370 INFO:teuthology.orchestra.run.smithi183.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-09-21T06:38:51.409 INFO:teuthology.orchestra.run.smithi193.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 873 kB 00:00 2024-09-21T06:38:51.484 INFO:teuthology.orchestra.run.smithi193.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.9 MB/s | 148 kB 00:00 2024-09-21T06:38:51.568 INFO:teuthology.orchestra.run.smithi193.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T06:38:51.677 INFO:teuthology.orchestra.run.smithi193.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 14 MB/s | 1.5 MB 00:00 2024-09-21T06:38:51.727 INFO:teuthology.orchestra.run.smithi193.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 9.4 MB/s | 2.3 MB 00:00 2024-09-21T06:38:51.863 INFO:teuthology.orchestra.run.smithi183.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 873 kB 00:00 2024-09-21T06:38:51.979 INFO:teuthology.orchestra.run.smithi183.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.2 MB/s | 148 kB 00:00 2024-09-21T06:38:51.994 INFO:teuthology.orchestra.run.smithi193.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 16 MB/s | 5.0 MB 00:00 2024-09-21T06:38:52.202 INFO:teuthology.orchestra.run.smithi193.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T06:38:52.261 INFO:teuthology.orchestra.run.smithi193.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 432 kB/s | 25 kB 00:00 2024-09-21T06:38:52.305 INFO:teuthology.orchestra.run.smithi183.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 7.0 MB/s | 2.3 MB 00:00 2024-09-21T06:38:52.415 INFO:teuthology.orchestra.run.smithi114.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T06:38:52.688 INFO:teuthology.orchestra.run.smithi183.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-09-21T06:38:52.824 INFO:teuthology.orchestra.run.smithi193.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 13 MB/s | 10 MB 00:00 2024-09-21T06:38:52.883 INFO:teuthology.orchestra.run.smithi193.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T06:38:52.949 INFO:teuthology.orchestra.run.smithi193.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 406 kB/s | 27 kB 00:00 2024-09-21T06:38:53.033 INFO:teuthology.orchestra.run.smithi193.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 380 kB/s | 32 kB 00:00 2024-09-21T06:38:53.133 INFO:teuthology.orchestra.run.smithi193.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T06:38:53.255 INFO:teuthology.orchestra.run.smithi183.stdout:(6/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 8.8 MB/s | 5.0 MB 00:00 2024-09-21T06:38:53.300 INFO:teuthology.orchestra.run.smithi193.stdout:(15/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 11 MB/s | 18 MB 00:01 2024-09-21T06:38:53.351 INFO:teuthology.orchestra.run.smithi193.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 730 kB/s | 158 kB 00:00 2024-09-21T06:38:53.376 INFO:teuthology.orchestra.run.smithi193.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.6 MB/s | 127 kB 00:00 2024-09-21T06:38:53.410 INFO:teuthology.orchestra.run.smithi193.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.4 MB/s | 507 kB 00:00 2024-09-21T06:38:53.477 INFO:teuthology.orchestra.run.smithi193.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 679 kB/s | 45 kB 00:00 2024-09-21T06:38:53.560 INFO:teuthology.orchestra.run.smithi193.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T06:38:53.644 INFO:teuthology.orchestra.run.smithi193.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 1.9 MB/s | 161 kB 00:00 2024-09-21T06:38:53.736 INFO:teuthology.orchestra.run.smithi193.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 3.4 MB/s | 321 kB 00:00 2024-09-21T06:38:53.747 INFO:teuthology.orchestra.run.smithi183.stdout:(7/157): ceph-base-19.3.0-5120.g18432218.el9.x8 2.2 MB/s | 5.5 MB 00:02 2024-09-21T06:38:53.861 INFO:teuthology.orchestra.run.smithi193.stdout:(23/157): librgw2-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.3 MB 00:00 2024-09-21T06:38:53.895 INFO:teuthology.orchestra.run.smithi193.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 1.9 MB/s | 307 kB 00:00 2024-09-21T06:38:54.189 INFO:teuthology.orchestra.run.smithi183.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 19 MB/s | 18 MB 00:00 2024-09-21T06:38:54.231 INFO:teuthology.orchestra.run.smithi183.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 610 kB/s | 25 kB 00:00 2024-09-21T06:38:54.254 INFO:teuthology.orchestra.run.smithi193.stdout:(25/157): ceph-test-19.3.0-5120.g18432218.el9.x 20 MB/s | 40 MB 00:01 2024-09-21T06:38:54.279 INFO:teuthology.orchestra.run.smithi193.stdout:(26/157): python3-rgw-19.3.0-5120.g18432218.el9 238 kB/s | 99 kB 00:00 2024-09-21T06:38:54.304 INFO:teuthology.orchestra.run.smithi193.stdout:(27/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 211 kB/s | 86 kB 00:00 2024-09-21T06:38:54.355 INFO:teuthology.orchestra.run.smithi193.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 170 kB 00:00 2024-09-21T06:38:54.388 INFO:teuthology.orchestra.run.smithi193.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 410 kB/s | 34 kB 00:00 2024-09-21T06:38:54.464 INFO:teuthology.orchestra.run.smithi193.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 15 MB/s | 3.1 MB 00:00 2024-09-21T06:38:54.489 INFO:teuthology.orchestra.run.smithi193.stdout:(31/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.1 MB/s | 154 kB 00:00 2024-09-21T06:38:54.769 INFO:teuthology.orchestra.run.smithi114.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:38:54.770 INFO:teuthology.orchestra.run.smithi114.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T06:38:54.784 INFO:teuthology.orchestra.run.smithi114.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T06:38:54.785 INFO:teuthology.orchestra.run.smithi114.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T06:38:54.786 INFO:teuthology.orchestra.run.smithi114.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T06:38:54.787 INFO:teuthology.orchestra.run.smithi114.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T06:38:54.788 INFO:teuthology.orchestra.run.smithi114.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T06:38:54.791 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T06:38:54.792 INFO:teuthology.orchestra.run.smithi114.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T06:38:54.858 INFO:teuthology.orchestra.run.smithi114.stdout:Dependencies resolved. 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: Package Arch Version Repository Size 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout:Installing: 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:38:54.884 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T06:38:54.885 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T06:38:54.886 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrading: 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T06:38:54.887 INFO:teuthology.orchestra.run.smithi114.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout:Installing dependencies: 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T06:38:54.888 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T06:38:54.889 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T06:38:54.890 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T06:38:54.891 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T06:38:54.892 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T06:38:54.893 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T06:38:54.894 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T06:38:54.894 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T06:38:54.894 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T06:38:54.894 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T06:38:54.894 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T06:38:54.894 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T06:38:54.895 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T06:38:54.896 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T06:38:54.897 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T06:38:54.898 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T06:38:54.898 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T06:38:54.898 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T06:38:54.898 INFO:teuthology.orchestra.run.smithi114.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T06:38:54.898 INFO:teuthology.orchestra.run.smithi114.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T06:38:54.898 INFO:teuthology.orchestra.run.smithi114.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout:Installing weak dependencies: 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction Summary 2024-09-21T06:38:54.899 INFO:teuthology.orchestra.run.smithi114.stdout:======================================================================================= 2024-09-21T06:38:54.900 INFO:teuthology.orchestra.run.smithi114.stdout:Install 149 Packages 2024-09-21T06:38:54.900 INFO:teuthology.orchestra.run.smithi114.stdout:Upgrade 8 Packages 2024-09-21T06:38:54.900 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:38:54.901 INFO:teuthology.orchestra.run.smithi114.stdout:Total download size: 228 M 2024-09-21T06:38:54.901 INFO:teuthology.orchestra.run.smithi114.stdout:Downloading Packages: 2024-09-21T06:38:54.931 INFO:teuthology.orchestra.run.smithi193.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 16 MB/s | 7.4 MB 00:00 2024-09-21T06:38:54.998 INFO:teuthology.orchestra.run.smithi193.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 742 kB/s | 49 kB 00:00 2024-09-21T06:38:55.040 INFO:teuthology.orchestra.run.smithi193.stdout:(34/157): ceph-mgr-modules-core-19.3.0-5120.g18 530 kB/s | 292 kB 00:00 2024-09-21T06:38:55.065 INFO:teuthology.orchestra.run.smithi193.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 246 kB/s | 16 kB 00:00 2024-09-21T06:38:55.115 INFO:teuthology.orchestra.run.smithi193.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T06:38:55.157 INFO:teuthology.orchestra.run.smithi193.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 8.3 MB/s | 784 kB 00:00 2024-09-21T06:38:55.191 INFO:teuthology.orchestra.run.smithi193.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 244 kB/s | 18 kB 00:00 2024-09-21T06:38:55.291 INFO:teuthology.orchestra.run.smithi193.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 597 kB/s | 80 kB 00:00 2024-09-21T06:38:55.383 INFO:teuthology.orchestra.run.smithi193.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-21T06:38:55.417 INFO:teuthology.orchestra.run.smithi193.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-21T06:38:55.442 INFO:teuthology.orchestra.run.smithi193.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-21T06:38:55.493 INFO:teuthology.orchestra.run.smithi193.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-21T06:38:55.518 INFO:teuthology.orchestra.run.smithi193.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-21T06:38:55.543 INFO:teuthology.orchestra.run.smithi193.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T06:38:55.569 INFO:teuthology.orchestra.run.smithi193.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:38:55.594 INFO:teuthology.orchestra.run.smithi193.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T06:38:55.650 INFO:teuthology.orchestra.run.smithi114.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-09-21T06:38:55.745 INFO:teuthology.orchestra.run.smithi193.stdout:(48/157): ceph-mgr-dashboard-19.3.0-5120.g18432 10 MB/s | 14 MB 00:01 2024-09-21T06:38:55.821 INFO:teuthology.orchestra.run.smithi114.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 5.1 MB/s | 873 kB 00:00 2024-09-21T06:38:55.839 INFO:teuthology.orchestra.run.smithi183.stdout:(10/157): ceph-test-19.3.0-5120.g18432218.el9.x 25 MB/s | 40 MB 00:01 2024-09-21T06:38:55.853 INFO:teuthology.orchestra.run.smithi193.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-21T06:38:55.878 INFO:teuthology.orchestra.run.smithi193.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 802 kB/s | 107 kB 00:00 2024-09-21T06:38:55.912 INFO:teuthology.orchestra.run.smithi193.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-21T06:38:55.917 INFO:teuthology.orchestra.run.smithi114.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.5 MB/s | 148 kB 00:00 2024-09-21T06:38:55.937 INFO:teuthology.orchestra.run.smithi193.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-21T06:38:55.948 INFO:teuthology.orchestra.run.smithi183.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.0 MB/s | 226 kB 00:00 2024-09-21T06:38:55.979 INFO:teuthology.orchestra.run.smithi193.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-21T06:38:56.004 INFO:teuthology.orchestra.run.smithi193.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 354 kB/s | 33 kB 00:00 2024-09-21T06:38:56.023 INFO:teuthology.orchestra.run.smithi183.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 361 kB/s | 27 kB 00:00 2024-09-21T06:38:56.042 INFO:teuthology.orchestra.run.smithi114.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T06:38:56.055 INFO:teuthology.orchestra.run.smithi193.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 241 kB/s | 18 kB 00:00 2024-09-21T06:38:56.073 INFO:teuthology.orchestra.run.smithi183.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 638 kB/s | 32 kB 00:00 2024-09-21T06:38:56.097 INFO:teuthology.orchestra.run.smithi193.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-21T06:38:56.139 INFO:teuthology.orchestra.run.smithi193.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T06:38:56.164 INFO:teuthology.orchestra.run.smithi193.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-21T06:38:56.179 INFO:teuthology.orchestra.run.smithi114.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.7 MB/s | 2.3 MB 00:00 2024-09-21T06:38:56.182 INFO:teuthology.orchestra.run.smithi183.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T06:38:56.226 INFO:teuthology.orchestra.run.smithi114.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-09-21T06:38:56.231 INFO:teuthology.orchestra.run.smithi193.stdout:(59/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-09-21T06:38:56.256 INFO:teuthology.orchestra.run.smithi193.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-09-21T06:38:56.257 INFO:teuthology.orchestra.run.smithi183.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T06:38:56.282 INFO:teuthology.orchestra.run.smithi193.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T06:38:56.324 INFO:teuthology.orchestra.run.smithi193.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-21T06:38:56.357 INFO:teuthology.orchestra.run.smithi193.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-21T06:38:56.382 INFO:teuthology.orchestra.run.smithi193.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-21T06:38:56.409 INFO:teuthology.orchestra.run.smithi193.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-21T06:38:56.433 INFO:teuthology.orchestra.run.smithi193.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T06:38:56.467 INFO:teuthology.orchestra.run.smithi193.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 457 kB/s | 38 kB 00:00 2024-09-21T06:38:56.584 INFO:teuthology.orchestra.run.smithi193.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 28 MB/s | 4.9 MB 00:00 2024-09-21T06:38:56.585 INFO:teuthology.orchestra.run.smithi114.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.0 MB 00:00 2024-09-21T06:38:56.626 INFO:teuthology.orchestra.run.smithi193.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-21T06:38:56.660 INFO:teuthology.orchestra.run.smithi193.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-21T06:38:56.718 INFO:teuthology.orchestra.run.smithi193.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-21T06:38:56.724 INFO:teuthology.orchestra.run.smithi183.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 271 kB/s | 127 kB 00:00 2024-09-21T06:38:56.743 INFO:teuthology.orchestra.run.smithi193.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T06:38:56.784 INFO:teuthology.orchestra.run.smithi183.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 8.3 MB/s | 507 kB 00:00 2024-09-21T06:38:56.806 INFO:teuthology.orchestra.run.smithi114.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 16 MB/s | 20 MB 00:01 2024-09-21T06:38:56.835 INFO:teuthology.orchestra.run.smithi193.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-09-21T06:38:56.860 INFO:teuthology.orchestra.run.smithi114.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 473 kB/s | 25 kB 00:00 2024-09-21T06:38:56.861 INFO:teuthology.orchestra.run.smithi193.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-09-21T06:38:56.925 INFO:teuthology.orchestra.run.smithi193.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-09-21T06:38:56.953 INFO:teuthology.orchestra.run.smithi193.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 668 kB/s | 61 kB 00:00 2024-09-21T06:38:56.995 INFO:teuthology.orchestra.run.smithi193.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 3.4 MB/s | 245 kB 00:00 2024-09-21T06:38:57.020 INFO:teuthology.orchestra.run.smithi193.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-21T06:38:57.096 INFO:teuthology.orchestra.run.smithi193.stdout:(79/157): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-09-21T06:38:57.121 INFO:teuthology.orchestra.run.smithi193.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 204 kB/s | 26 kB 00:00 2024-09-21T06:38:57.146 INFO:teuthology.orchestra.run.smithi193.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-21T06:38:57.168 INFO:teuthology.orchestra.run.smithi183.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 14 MB/s | 5.3 MB 00:00 2024-09-21T06:38:57.179 INFO:teuthology.orchestra.run.smithi193.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T06:38:57.221 INFO:teuthology.orchestra.run.smithi193.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-09-21T06:38:57.226 INFO:teuthology.orchestra.run.smithi183.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 777 kB/s | 45 kB 00:00 2024-09-21T06:38:57.255 INFO:teuthology.orchestra.run.smithi193.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 250 kB 00:00 2024-09-21T06:38:57.277 INFO:teuthology.orchestra.run.smithi183.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T06:38:57.289 INFO:teuthology.orchestra.run.smithi193.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-21T06:38:57.343 INFO:teuthology.orchestra.run.smithi183.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T06:38:57.352 INFO:teuthology.orchestra.run.smithi114.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 16 MB/s | 18 MB 00:01 2024-09-21T06:38:57.372 INFO:teuthology.orchestra.run.smithi193.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-21T06:38:57.394 INFO:teuthology.orchestra.run.smithi183.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T06:38:57.398 INFO:teuthology.orchestra.run.smithi193.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-09-21T06:38:57.402 INFO:teuthology.orchestra.run.smithi114.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.5 MB/s | 226 kB 00:00 2024-09-21T06:38:57.423 INFO:teuthology.orchestra.run.smithi193.stdout:(88/157): python3-packaging-20.9-5.el9.noarch.r 460 kB/s | 77 kB 00:00 2024-09-21T06:38:57.444 INFO:teuthology.orchestra.run.smithi183.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T06:38:57.448 INFO:teuthology.orchestra.run.smithi193.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-21T06:38:57.482 INFO:teuthology.orchestra.run.smithi193.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-21T06:38:57.495 INFO:teuthology.orchestra.run.smithi183.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T06:38:57.507 INFO:teuthology.orchestra.run.smithi193.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-09-21T06:38:57.519 INFO:teuthology.orchestra.run.smithi114.stdout:(12/157): ceph-radosgw-19.3.0-5120.g18432218.el 11 MB/s | 10 MB 00:00 2024-09-21T06:38:57.532 INFO:teuthology.orchestra.run.smithi193.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-09-21T06:38:57.544 INFO:teuthology.orchestra.run.smithi114.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 191 kB/s | 27 kB 00:00 2024-09-21T06:38:57.545 INFO:teuthology.orchestra.run.smithi183.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T06:38:57.558 INFO:teuthology.orchestra.run.smithi193.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-09-21T06:38:57.570 INFO:teuthology.orchestra.run.smithi114.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 634 kB/s | 32 kB 00:00 2024-09-21T06:38:57.600 INFO:teuthology.orchestra.run.smithi193.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-09-21T06:38:57.612 INFO:teuthology.orchestra.run.smithi114.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T06:38:57.625 INFO:teuthology.orchestra.run.smithi193.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-09-21T06:38:57.637 INFO:teuthology.orchestra.run.smithi114.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T06:38:57.659 INFO:teuthology.orchestra.run.smithi193.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T06:38:57.670 INFO:teuthology.orchestra.run.smithi114.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T06:38:57.676 INFO:teuthology.orchestra.run.smithi193.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-21T06:38:57.703 INFO:teuthology.orchestra.run.smithi183.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 20 MB/s | 3.1 MB 00:00 2024-09-21T06:38:57.712 INFO:teuthology.orchestra.run.smithi114.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 6.6 MB/s | 507 kB 00:00 2024-09-21T06:38:57.735 INFO:teuthology.orchestra.run.smithi193.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-09-21T06:38:57.754 INFO:teuthology.orchestra.run.smithi183.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T06:38:57.768 INFO:teuthology.orchestra.run.smithi193.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-21T06:38:57.781 INFO:teuthology.orchestra.run.smithi114.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 678 kB/s | 45 kB 00:00 2024-09-21T06:38:57.829 INFO:teuthology.orchestra.run.smithi183.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 457 kB/s | 34 kB 00:00 2024-09-21T06:38:57.880 INFO:teuthology.orchestra.run.smithi114.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.5 MB/s | 148 kB 00:00 2024-09-21T06:38:57.896 INFO:teuthology.orchestra.run.smithi183.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.3 MB/s | 154 kB 00:00 2024-09-21T06:38:57.935 INFO:teuthology.orchestra.run.smithi193.stdout:(100/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-21T06:38:57.961 INFO:teuthology.orchestra.run.smithi193.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 256 kB/s | 58 kB 00:00 2024-09-21T06:38:57.980 INFO:teuthology.orchestra.run.smithi114.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.3 MB 00:00 2024-09-21T06:38:57.986 INFO:teuthology.orchestra.run.smithi193.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 441 kB/s | 96 kB 00:00 2024-09-21T06:38:58.005 INFO:teuthology.orchestra.run.smithi114.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T06:38:58.011 INFO:teuthology.orchestra.run.smithi193.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-09-21T06:38:58.036 INFO:teuthology.orchestra.run.smithi193.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 593 kB/s | 45 kB 00:00 2024-09-21T06:38:58.047 INFO:teuthology.orchestra.run.smithi114.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T06:38:58.073 INFO:teuthology.orchestra.run.smithi114.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T06:38:58.095 INFO:teuthology.orchestra.run.smithi193.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T06:38:58.115 INFO:teuthology.orchestra.run.smithi114.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.4 MB/s | 99 kB 00:00 2024-09-21T06:38:58.120 INFO:teuthology.orchestra.run.smithi193.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T06:38:58.137 INFO:teuthology.orchestra.run.smithi193.stdout:(107/157): lua-devel-5.4.4-4.el9.x86_64.rpm 147 kB/s | 22 kB 00:00 2024-09-21T06:38:58.140 INFO:teuthology.orchestra.run.smithi114.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.3 MB/s | 86 kB 00:00 2024-09-21T06:38:58.154 INFO:teuthology.orchestra.run.smithi193.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 579 kB/s | 19 kB 00:00 2024-09-21T06:38:58.179 INFO:teuthology.orchestra.run.smithi193.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.4 MB/s | 26 kB 00:00 2024-09-21T06:38:58.205 INFO:teuthology.orchestra.run.smithi193.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-21T06:38:58.230 INFO:teuthology.orchestra.run.smithi193.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T06:38:58.240 INFO:teuthology.orchestra.run.smithi114.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 1.7 MB/s | 170 kB 00:00 2024-09-21T06:38:58.255 INFO:teuthology.orchestra.run.smithi193.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T06:38:58.289 INFO:teuthology.orchestra.run.smithi193.stdout:(113/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-21T06:38:58.339 INFO:teuthology.orchestra.run.smithi193.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-21T06:38:58.364 INFO:teuthology.orchestra.run.smithi193.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.9 MB/s | 43 kB 00:00 2024-09-21T06:38:58.365 INFO:teuthology.orchestra.run.smithi114.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 12 MB/s | 3.1 MB 00:00 2024-09-21T06:38:58.382 INFO:teuthology.orchestra.run.smithi114.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 241 kB/s | 34 kB 00:00 2024-09-21T06:38:58.390 INFO:teuthology.orchestra.run.smithi193.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T06:38:58.415 INFO:teuthology.orchestra.run.smithi193.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-21T06:38:58.441 INFO:teuthology.orchestra.run.smithi114.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.0 MB/s | 154 kB 00:00 2024-09-21T06:38:58.449 INFO:teuthology.orchestra.run.smithi193.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-21T06:38:58.483 INFO:teuthology.orchestra.run.smithi193.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-09-21T06:38:58.524 INFO:teuthology.orchestra.run.smithi193.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-09-21T06:38:58.558 INFO:teuthology.orchestra.run.smithi193.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 6.4 MB/s | 220 kB 00:00 2024-09-21T06:38:58.584 INFO:teuthology.orchestra.run.smithi193.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-21T06:38:58.617 INFO:teuthology.orchestra.run.smithi193.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-21T06:38:58.643 INFO:teuthology.orchestra.run.smithi193.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T06:38:58.668 INFO:teuthology.orchestra.run.smithi193.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-21T06:38:58.755 INFO:teuthology.orchestra.run.smithi183.stdout:(30/157): ceph-mgr-dashboard-19.3.0-5120.g18432 17 MB/s | 14 MB 00:00 2024-09-21T06:38:58.827 INFO:teuthology.orchestra.run.smithi193.stdout:(126/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-21T06:38:58.852 INFO:teuthology.orchestra.run.smithi193.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 126 kB/s | 23 kB 00:00 2024-09-21T06:38:58.877 INFO:teuthology.orchestra.run.smithi193.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 392 kB/s | 19 kB 00:00 2024-09-21T06:38:58.903 INFO:teuthology.orchestra.run.smithi193.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-21T06:38:58.928 INFO:teuthology.orchestra.run.smithi193.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:38:58.962 INFO:teuthology.orchestra.run.smithi193.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T06:38:58.983 INFO:teuthology.orchestra.run.smithi114.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T06:38:58.987 INFO:teuthology.orchestra.run.smithi193.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:38:59.005 INFO:teuthology.orchestra.run.smithi183.stdout:(31/157): ceph-radosgw-19.3.0-5120.g18432218.el 2.0 MB/s | 10 MB 00:05 2024-09-21T06:38:59.037 INFO:teuthology.orchestra.run.smithi193.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-09-21T06:38:59.062 INFO:teuthology.orchestra.run.smithi193.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-21T06:38:59.096 INFO:teuthology.orchestra.run.smithi193.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-21T06:38:59.122 INFO:teuthology.orchestra.run.smithi193.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-21T06:38:59.130 INFO:teuthology.orchestra.run.smithi183.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.3 MB/s | 292 kB 00:00 2024-09-21T06:38:59.147 INFO:teuthology.orchestra.run.smithi193.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-21T06:38:59.172 INFO:teuthology.orchestra.run.smithi193.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-09-21T06:38:59.197 INFO:teuthology.orchestra.run.smithi193.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-21T06:38:59.223 INFO:teuthology.orchestra.run.smithi193.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-21T06:38:59.239 INFO:teuthology.orchestra.run.smithi183.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 15 MB/s | 7.4 MB 00:00 2024-09-21T06:38:59.248 INFO:teuthology.orchestra.run.smithi193.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T06:38:59.264 INFO:teuthology.orchestra.run.smithi183.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 369 kB/s | 49 kB 00:00 2024-09-21T06:38:59.273 INFO:teuthology.orchestra.run.smithi193.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T06:38:59.289 INFO:teuthology.orchestra.run.smithi183.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T06:38:59.299 INFO:teuthology.orchestra.run.smithi193.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T06:38:59.332 INFO:teuthology.orchestra.run.smithi193.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-21T06:38:59.357 INFO:teuthology.orchestra.run.smithi183.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 3.0 MB/s | 285 kB 00:00 2024-09-21T06:38:59.357 INFO:teuthology.orchestra.run.smithi193.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T06:38:59.358 INFO:teuthology.orchestra.run.smithi114.stdout:(32/157): ceph-test-19.3.0-5120.g18432218.el9.x 16 MB/s | 40 MB 00:02 2024-09-21T06:38:59.374 INFO:teuthology.orchestra.run.smithi193.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-21T06:38:59.382 INFO:teuthology.orchestra.run.smithi183.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 8.3 MB/s | 784 kB 00:00 2024-09-21T06:38:59.392 INFO:teuthology.orchestra.run.smithi114.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 714 kB/s | 292 kB 00:00 2024-09-21T06:38:59.399 INFO:teuthology.orchestra.run.smithi183.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 440 kB/s | 18 kB 00:00 2024-09-21T06:38:59.399 INFO:teuthology.orchestra.run.smithi193.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T06:38:59.418 INFO:teuthology.orchestra.run.smithi114.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 848 kB/s | 49 kB 00:00 2024-09-21T06:38:59.425 INFO:teuthology.orchestra.run.smithi193.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T06:38:59.443 INFO:teuthology.orchestra.run.smithi114.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T06:38:59.468 INFO:teuthology.orchestra.run.smithi114.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T06:38:59.475 INFO:teuthology.orchestra.run.smithi193.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T06:38:59.510 INFO:teuthology.orchestra.run.smithi114.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 444 kB/s | 18 kB 00:00 2024-09-21T06:38:59.532 INFO:teuthology.orchestra.run.smithi183.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-09-21T06:38:59.552 INFO:teuthology.orchestra.run.smithi114.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.1 MB/s | 784 kB 00:00 2024-09-21T06:38:59.616 INFO:teuthology.orchestra.run.smithi183.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-21T06:38:59.635 INFO:teuthology.orchestra.run.smithi114.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 639 kB/s | 80 kB 00:00 2024-09-21T06:38:59.659 INFO:teuthology.orchestra.run.smithi193.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-21T06:38:59.735 INFO:teuthology.orchestra.run.smithi193.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 10 MB/s | 3.2 MB 00:00 2024-09-21T06:38:59.776 INFO:teuthology.orchestra.run.smithi193.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 16 MB/s | 1.8 MB 00:00 2024-09-21T06:38:59.808 INFO:teuthology.orchestra.run.smithi183.stdout:(41/157): ceph-common-19.3.0-5120.g18432218.el9 2.3 MB/s | 20 MB 00:08 2024-09-21T06:38:59.811 INFO:teuthology.orchestra.run.smithi114.stdout:(40/157): ceph-mgr-dashboard-19.3.0-5120.g18432 9.9 MB/s | 14 MB 00:01 2024-09-21T06:38:59.818 INFO:teuthology.orchestra.run.smithi193.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 490 kB/s | 41 kB 00:00 2024-09-21T06:38:59.836 INFO:teuthology.orchestra.run.smithi114.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 955 kB/s | 191 kB 00:00 2024-09-21T06:38:59.842 INFO:teuthology.orchestra.run.smithi183.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 801 kB 00:00 2024-09-21T06:38:59.844 INFO:teuthology.orchestra.run.smithi193.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.6 MB/s | 249 kB 00:00 2024-09-21T06:38:59.867 INFO:teuthology.orchestra.run.smithi183.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-21T06:38:59.894 INFO:teuthology.orchestra.run.smithi193.stdout:(155/157): librados2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 3.4 MB 00:00 2024-09-21T06:38:59.894 INFO:teuthology.orchestra.run.smithi114.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-21T06:38:59.909 INFO:teuthology.orchestra.run.smithi183.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T06:38:59.928 INFO:teuthology.orchestra.run.smithi114.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 3.2 MB/s | 106 kB 00:00 2024-09-21T06:38:59.934 INFO:teuthology.orchestra.run.smithi183.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-21T06:38:59.952 INFO:teuthology.orchestra.run.smithi193.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-21T06:38:59.953 INFO:teuthology.orchestra.run.smithi114.stdout:(44/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-21T06:38:59.959 INFO:teuthology.orchestra.run.smithi183.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T06:38:59.987 INFO:teuthology.orchestra.run.smithi114.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-21T06:38:59.993 INFO:teuthology.orchestra.run.smithi183.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-21T06:39:00.012 INFO:teuthology.orchestra.run.smithi114.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-21T06:39:00.028 INFO:teuthology.orchestra.run.smithi193.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 6.5 MB/s | 1.4 MB 00:00 2024-09-21T06:39:00.034 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:39:00.034 INFO:teuthology.orchestra.run.smithi193.stdout:Total 24 MB/s | 228 MB 00:09 2024-09-21T06:39:00.046 INFO:teuthology.orchestra.run.smithi114.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T06:39:00.071 INFO:teuthology.orchestra.run.smithi114.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T06:39:00.085 INFO:teuthology.orchestra.run.smithi183.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-21T06:39:00.127 INFO:teuthology.orchestra.run.smithi183.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-21T06:39:00.163 INFO:teuthology.orchestra.run.smithi114.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-09-21T06:39:00.169 INFO:teuthology.orchestra.run.smithi183.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-09-21T06:39:00.186 INFO:teuthology.orchestra.run.smithi183.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 369 kB/s | 22 kB 00:00 2024-09-21T06:39:00.196 INFO:teuthology.orchestra.run.smithi114.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 855 kB/s | 107 kB 00:00 2024-09-21T06:39:00.211 INFO:teuthology.orchestra.run.smithi183.stdout:(52/157): boost-program-options-1.75.0-8.el9.x8 849 kB/s | 107 kB 00:00 2024-09-21T06:39:00.245 INFO:teuthology.orchestra.run.smithi183.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-21T06:39:00.268 INFO:teuthology.orchestra.run.smithi114.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-21T06:39:00.270 INFO:teuthology.orchestra.run.smithi183.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-21T06:39:00.297 INFO:teuthology.orchestra.run.smithi114.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 162 kB/s | 22 kB 00:00 2024-09-21T06:39:00.312 INFO:teuthology.orchestra.run.smithi183.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-21T06:39:00.322 INFO:teuthology.orchestra.run.smithi114.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-21T06:39:00.337 INFO:teuthology.orchestra.run.smithi183.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-09-21T06:39:00.348 INFO:teuthology.orchestra.run.smithi114.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 412 kB/s | 33 kB 00:00 2024-09-21T06:39:00.379 INFO:teuthology.orchestra.run.smithi183.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T06:39:00.392 INFO:teuthology.orchestra.run.smithi114.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 264 kB/s | 18 kB 00:00 2024-09-21T06:39:00.454 INFO:teuthology.orchestra.run.smithi183.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-21T06:39:00.467 INFO:teuthology.orchestra.run.smithi114.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-21T06:39:00.480 INFO:teuthology.orchestra.run.smithi183.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-21T06:39:00.492 INFO:teuthology.orchestra.run.smithi114.stdout:(57/157): libaio-devel-0.3.111-13.el9.x86_64.rp 77 kB/s | 11 kB 00:00 2024-09-21T06:39:00.518 INFO:teuthology.orchestra.run.smithi114.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.3 MB/s | 171 kB 00:00 2024-09-21T06:39:00.530 INFO:teuthology.orchestra.run.smithi183.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-09-21T06:39:00.543 INFO:teuthology.orchestra.run.smithi114.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-21T06:39:00.564 INFO:teuthology.orchestra.run.smithi183.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-21T06:39:00.568 INFO:teuthology.orchestra.run.smithi114.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-21T06:39:00.597 INFO:teuthology.orchestra.run.smithi183.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T06:39:00.602 INFO:teuthology.orchestra.run.smithi114.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T06:39:00.623 INFO:teuthology.orchestra.run.smithi183.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 266 kB/s | 24 kB 00:00 2024-09-21T06:39:00.627 INFO:teuthology.orchestra.run.smithi114.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T06:39:00.648 INFO:teuthology.orchestra.run.smithi183.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-21T06:39:00.669 INFO:teuthology.orchestra.run.smithi114.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-21T06:39:00.673 INFO:teuthology.orchestra.run.smithi183.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T06:39:00.694 INFO:teuthology.orchestra.run.smithi114.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-09-21T06:39:00.698 INFO:teuthology.orchestra.run.smithi183.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T06:39:00.719 INFO:teuthology.orchestra.run.smithi114.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T06:39:00.724 INFO:teuthology.orchestra.run.smithi183.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 509 kB/s | 38 kB 00:00 2024-09-21T06:39:00.745 INFO:teuthology.orchestra.run.smithi114.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T06:39:00.770 INFO:teuthology.orchestra.run.smithi114.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-21T06:39:00.816 INFO:teuthology.orchestra.run.smithi183.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-21T06:39:00.858 INFO:teuthology.orchestra.run.smithi183.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi114.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-21T06:39:00.908 INFO:teuthology.orchestra.run.smithi183.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-21T06:39:00.925 INFO:teuthology.orchestra.run.smithi183.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T06:39:00.937 INFO:teuthology.orchestra.run.smithi114.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-21T06:39:01.004 INFO:teuthology.orchestra.run.smithi114.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-21T06:39:01.029 INFO:teuthology.orchestra.run.smithi114.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-21T06:39:01.034 INFO:teuthology.orchestra.run.smithi183.stdout:(72/157): postgresql-server-13.16-1.el9.x86_64. 43 MB/s | 5.8 MB 00:00 2024-09-21T06:39:01.063 INFO:teuthology.orchestra.run.smithi114.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-21T06:39:01.100 INFO:teuthology.orchestra.run.smithi183.stdout:(73/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-21T06:39:01.113 INFO:teuthology.orchestra.run.smithi114.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-21T06:39:01.126 INFO:teuthology.orchestra.run.smithi183.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 955 kB/s | 87 kB 00:00 2024-09-21T06:39:01.138 INFO:teuthology.orchestra.run.smithi114.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-21T06:39:01.159 INFO:teuthology.orchestra.run.smithi183.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-09-21T06:39:01.176 INFO:teuthology.orchestra.run.smithi183.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T06:39:01.222 INFO:teuthology.orchestra.run.smithi114.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-09-21T06:39:01.239 INFO:teuthology.orchestra.run.smithi114.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 610 kB/s | 61 kB 00:00 2024-09-21T06:39:01.243 INFO:teuthology.orchestra.run.smithi183.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-21T06:39:01.269 INFO:teuthology.orchestra.run.smithi183.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-21T06:39:01.283 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-21T06:39:01.285 INFO:teuthology.orchestra.run.smithi183.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-09-21T06:39:01.339 INFO:teuthology.orchestra.run.smithi114.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-09-21T06:39:01.361 INFO:teuthology.orchestra.run.smithi183.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-09-21T06:39:01.364 INFO:teuthology.orchestra.run.smithi114.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-21T06:39:01.378 INFO:teuthology.orchestra.run.smithi183.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-21T06:39:01.406 INFO:teuthology.orchestra.run.smithi114.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-21T06:39:01.414 INFO:teuthology.orchestra.run.smithi183.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-21T06:39:01.429 INFO:teuthology.orchestra.run.smithi183.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 516 kB/s | 35 kB 00:00 2024-09-21T06:39:01.431 INFO:teuthology.orchestra.run.smithi114.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-09-21T06:39:01.436 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-21T06:39:01.436 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-21T06:39:01.474 INFO:teuthology.orchestra.run.smithi114.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 285 kB/s | 19 kB 00:00 2024-09-21T06:39:01.499 INFO:teuthology.orchestra.run.smithi114.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-21T06:39:01.512 INFO:teuthology.orchestra.run.smithi183.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-09-21T06:39:01.527 INFO:teuthology.orchestra.run.smithi114.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 363 kB/s | 35 kB 00:00 2024-09-21T06:39:01.537 INFO:teuthology.orchestra.run.smithi183.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-21T06:39:01.558 INFO:teuthology.orchestra.run.smithi114.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-21T06:39:01.562 INFO:teuthology.orchestra.run.smithi183.stdout:(86/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-21T06:39:01.588 INFO:teuthology.orchestra.run.smithi183.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-21T06:39:01.591 INFO:teuthology.orchestra.run.smithi114.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 222 kB 00:00 2024-09-21T06:39:01.613 INFO:teuthology.orchestra.run.smithi183.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 557 kB/s | 42 kB 00:00 2024-09-21T06:39:01.617 INFO:teuthology.orchestra.run.smithi114.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T06:39:01.639 INFO:teuthology.orchestra.run.smithi183.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-21T06:39:01.664 INFO:teuthology.orchestra.run.smithi183.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-21T06:39:01.689 INFO:teuthology.orchestra.run.smithi183.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-21T06:39:01.692 INFO:teuthology.orchestra.run.smithi114.stdout:(87/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-21T06:39:01.715 INFO:teuthology.orchestra.run.smithi183.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-21T06:39:01.717 INFO:teuthology.orchestra.run.smithi114.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-21T06:39:01.740 INFO:teuthology.orchestra.run.smithi183.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-21T06:39:01.743 INFO:teuthology.orchestra.run.smithi114.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-21T06:39:01.768 INFO:teuthology.orchestra.run.smithi114.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-21T06:39:01.782 INFO:teuthology.orchestra.run.smithi183.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-09-21T06:39:01.793 INFO:teuthology.orchestra.run.smithi114.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T06:39:01.807 INFO:teuthology.orchestra.run.smithi183.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 662 kB/s | 44 kB 00:00 2024-09-21T06:39:01.818 INFO:teuthology.orchestra.run.smithi114.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-21T06:39:01.843 INFO:teuthology.orchestra.run.smithi114.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-21T06:39:01.849 INFO:teuthology.orchestra.run.smithi183.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T06:39:01.874 INFO:teuthology.orchestra.run.smithi183.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 535 kB/s | 36 kB 00:00 2024-09-21T06:39:01.886 INFO:teuthology.orchestra.run.smithi114.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 621 kB/s | 42 kB 00:00 2024-09-21T06:39:01.911 INFO:teuthology.orchestra.run.smithi114.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 659 kB/s | 44 kB 00:00 2024-09-21T06:39:01.924 INFO:teuthology.orchestra.run.smithi183.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-09-21T06:39:01.950 INFO:teuthology.orchestra.run.smithi183.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.9 MB/s | 304 kB 00:00 2024-09-21T06:39:02.000 INFO:teuthology.orchestra.run.smithi114.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-09-21T06:39:02.017 INFO:teuthology.orchestra.run.smithi183.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 630 kB/s | 58 kB 00:00 2024-09-21T06:39:02.028 INFO:teuthology.orchestra.run.smithi114.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-09-21T06:39:02.042 INFO:teuthology.orchestra.run.smithi183.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.0 MB/s | 96 kB 00:00 2024-09-21T06:39:02.078 INFO:teuthology.orchestra.run.smithi114.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 611 kB/s | 47 kB 00:00 2024-09-21T06:39:02.103 INFO:teuthology.orchestra.run.smithi114.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-21T06:39:02.192 INFO:teuthology.orchestra.run.smithi183.stdout:(102/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-21T06:39:02.217 INFO:teuthology.orchestra.run.smithi183.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 256 kB/s | 45 kB 00:00 2024-09-21T06:39:02.243 INFO:teuthology.orchestra.run.smithi183.stdout:(104/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 282 kB/s | 64 kB 00:00 2024-09-21T06:39:02.271 INFO:teuthology.orchestra.run.smithi114.stdout:(100/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-21T06:39:02.293 INFO:teuthology.orchestra.run.smithi183.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T06:39:02.296 INFO:teuthology.orchestra.run.smithi114.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 58 kB 00:00 2024-09-21T06:39:02.321 INFO:teuthology.orchestra.run.smithi114.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 440 kB/s | 96 kB 00:00 2024-09-21T06:39:02.327 INFO:teuthology.orchestra.run.smithi183.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T06:39:02.346 INFO:teuthology.orchestra.run.smithi114.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-09-21T06:39:02.352 INFO:teuthology.orchestra.run.smithi183.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-21T06:39:02.371 INFO:teuthology.orchestra.run.smithi114.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-09-21T06:39:02.430 INFO:teuthology.orchestra.run.smithi114.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-21T06:39:02.452 INFO:teuthology.orchestra.run.smithi183.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-21T06:39:02.472 INFO:teuthology.orchestra.run.smithi114.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.3 MB/s | 548 kB 00:00 2024-09-21T06:39:02.494 INFO:teuthology.orchestra.run.smithi183.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 947 kB/s | 26 kB 00:00 2024-09-21T06:39:02.497 INFO:teuthology.orchestra.run.smithi114.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 856 kB/s | 19 kB 00:00 2024-09-21T06:39:02.519 INFO:teuthology.orchestra.run.smithi183.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T06:39:02.531 INFO:teuthology.orchestra.run.smithi114.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.0 MB/s | 308 kB 00:00 2024-09-21T06:39:02.544 INFO:teuthology.orchestra.run.smithi183.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T06:39:02.578 INFO:teuthology.orchestra.run.smithi183.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-09-21T06:39:02.640 INFO:teuthology.orchestra.run.smithi114.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-09-21T06:39:02.645 INFO:teuthology.orchestra.run.smithi183.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-21T06:39:02.679 INFO:teuthology.orchestra.run.smithi183.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T06:39:02.681 INFO:teuthology.orchestra.run.smithi114.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 174 kB/s | 26 kB 00:00 2024-09-21T06:39:02.696 INFO:teuthology.orchestra.run.smithi183.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.5 MB/s | 43 kB 00:00 2024-09-21T06:39:02.713 INFO:teuthology.orchestra.run.smithi183.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-21T06:39:02.715 INFO:teuthology.orchestra.run.smithi114.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-09-21T06:39:02.730 INFO:teuthology.orchestra.run.smithi183.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 854 kB/s | 14 kB 00:00 2024-09-21T06:39:02.740 INFO:teuthology.orchestra.run.smithi114.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 494 kB/s | 49 kB 00:00 2024-09-21T06:39:02.764 INFO:teuthology.orchestra.run.smithi183.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-21T06:39:02.765 INFO:teuthology.orchestra.run.smithi114.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-21T06:39:02.805 INFO:teuthology.orchestra.run.smithi183.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-21T06:39:02.823 INFO:teuthology.orchestra.run.smithi183.stdout:(120/157): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-09-21T06:39:02.824 INFO:teuthology.orchestra.run.smithi114.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-21T06:39:02.850 INFO:teuthology.orchestra.run.smithi114.stdout:(115/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-21T06:39:02.856 INFO:teuthology.orchestra.run.smithi183.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-09-21T06:39:02.875 INFO:teuthology.orchestra.run.smithi114.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 864 kB/s | 43 kB 00:00 2024-09-21T06:39:02.900 INFO:teuthology.orchestra.run.smithi114.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-09-21T06:39:02.923 INFO:teuthology.orchestra.run.smithi183.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 2.1 MB/s | 220 kB 00:00 2024-09-21T06:39:02.925 INFO:teuthology.orchestra.run.smithi114.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-21T06:39:02.959 INFO:teuthology.orchestra.run.smithi114.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T06:39:02.973 INFO:teuthology.orchestra.run.smithi183.stdout:(123/157): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-09-21T06:39:02.990 INFO:teuthology.orchestra.run.smithi183.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-21T06:39:02.992 INFO:teuthology.orchestra.run.smithi114.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-09-21T06:39:03.027 INFO:teuthology.orchestra.run.smithi114.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-21T06:39:03.033 INFO:teuthology.orchestra.run.smithi183.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-09-21T06:39:03.049 INFO:teuthology.orchestra.run.smithi183.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-09-21T06:39:03.052 INFO:teuthology.orchestra.run.smithi114.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T06:39:03.066 INFO:teuthology.orchestra.run.smithi183.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 714 kB/s | 23 kB 00:00 2024-09-21T06:39:03.083 INFO:teuthology.orchestra.run.smithi183.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 577 kB/s | 19 kB 00:00 2024-09-21T06:39:03.085 INFO:teuthology.orchestra.run.smithi114.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T06:39:03.100 INFO:teuthology.orchestra.run.smithi183.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-09-21T06:39:03.110 INFO:teuthology.orchestra.run.smithi114.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T06:39:03.125 INFO:teuthology.orchestra.run.smithi183.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:39:03.151 INFO:teuthology.orchestra.run.smithi183.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:39:03.152 INFO:teuthology.orchestra.run.smithi114.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-09-21T06:39:03.176 INFO:teuthology.orchestra.run.smithi183.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:39:03.178 INFO:teuthology.orchestra.run.smithi114.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 939 kB/s | 23 kB 00:00 2024-09-21T06:39:03.209 INFO:teuthology.orchestra.run.smithi183.stdout:(133/157): protobuf-compiler-3.14.0-13.el9.x86_ 870 kB/s | 863 kB 00:00 2024-09-21T06:39:03.245 INFO:teuthology.orchestra.run.smithi114.stdout:(127/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-09-21T06:39:03.260 INFO:teuthology.orchestra.run.smithi183.stdout:(134/157): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-21T06:39:03.263 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-21T06:39:03.263 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-21T06:39:03.270 INFO:teuthology.orchestra.run.smithi114.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-09-21T06:39:03.285 INFO:teuthology.orchestra.run.smithi183.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-21T06:39:03.295 INFO:teuthology.orchestra.run.smithi114.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 801 kB/s | 20 kB 00:00 2024-09-21T06:39:03.310 INFO:teuthology.orchestra.run.smithi183.stdout:(136/157): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-09-21T06:39:03.320 INFO:teuthology.orchestra.run.smithi114.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T06:39:03.336 INFO:teuthology.orchestra.run.smithi183.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T06:39:03.345 INFO:teuthology.orchestra.run.smithi114.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T06:39:03.361 INFO:teuthology.orchestra.run.smithi183.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-09-21T06:39:03.371 INFO:teuthology.orchestra.run.smithi114.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T06:39:03.386 INFO:teuthology.orchestra.run.smithi183.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-21T06:39:03.404 INFO:teuthology.orchestra.run.smithi114.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.1 MB/s | 272 kB 00:00 2024-09-21T06:39:03.411 INFO:teuthology.orchestra.run.smithi183.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T06:39:03.430 INFO:teuthology.orchestra.run.smithi114.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-09-21T06:39:03.436 INFO:teuthology.orchestra.run.smithi183.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-09-21T06:39:03.455 INFO:teuthology.orchestra.run.smithi114.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-21T06:39:03.462 INFO:teuthology.orchestra.run.smithi183.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T06:39:03.487 INFO:teuthology.orchestra.run.smithi183.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T06:39:03.489 INFO:teuthology.orchestra.run.smithi114.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-21T06:39:03.514 INFO:teuthology.orchestra.run.smithi183.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T06:39:03.514 INFO:teuthology.orchestra.run.smithi114.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-09-21T06:39:03.538 INFO:teuthology.orchestra.run.smithi183.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-21T06:39:03.539 INFO:teuthology.orchestra.run.smithi114.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-21T06:39:03.563 INFO:teuthology.orchestra.run.smithi183.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-21T06:39:03.581 INFO:teuthology.orchestra.run.smithi114.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-21T06:39:03.597 INFO:teuthology.orchestra.run.smithi183.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T06:39:03.607 INFO:teuthology.orchestra.run.smithi114.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 879 kB/s | 59 kB 00:00 2024-09-21T06:39:03.622 INFO:teuthology.orchestra.run.smithi183.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T06:39:03.632 INFO:teuthology.orchestra.run.smithi114.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 707 kB/s | 36 kB 00:00 2024-09-21T06:39:03.658 INFO:teuthology.orchestra.run.smithi114.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-21T06:39:03.673 INFO:teuthology.orchestra.run.smithi183.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T06:39:03.683 INFO:teuthology.orchestra.run.smithi114.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-21T06:39:03.725 INFO:teuthology.orchestra.run.smithi114.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-21T06:39:03.750 INFO:teuthology.orchestra.run.smithi114.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-21T06:39:03.775 INFO:teuthology.orchestra.run.smithi114.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-09-21T06:39:03.806 INFO:teuthology.orchestra.run.smithi183.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 17 MB/s | 2.3 MB 00:00 2024-09-21T06:39:03.809 INFO:teuthology.orchestra.run.smithi114.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T06:39:03.834 INFO:teuthology.orchestra.run.smithi114.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-21T06:39:03.873 INFO:teuthology.orchestra.run.smithi183.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-21T06:39:03.885 INFO:teuthology.orchestra.run.smithi114.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T06:39:03.932 INFO:teuthology.orchestra.run.smithi183.stdout:(152/157): librados2-19.3.0-5120.g18432218.el9. 10 MB/s | 3.4 MB 00:00 2024-09-21T06:39:03.957 INFO:teuthology.orchestra.run.smithi183.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 490 kB/s | 41 kB 00:00 2024-09-21T06:39:03.991 INFO:teuthology.orchestra.run.smithi183.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T06:39:04.033 INFO:teuthology.orchestra.run.smithi183.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-21T06:39:04.093 INFO:teuthology.orchestra.run.smithi114.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-09-21T06:39:04.108 INFO:teuthology.orchestra.run.smithi183.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-09-21T06:39:04.160 INFO:teuthology.orchestra.run.smithi114.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.7 MB/s | 3.4 MB 00:00 2024-09-21T06:39:04.211 INFO:teuthology.orchestra.run.smithi114.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-09-21T06:39:04.236 INFO:teuthology.orchestra.run.smithi114.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 545 kB/s | 41 kB 00:00 2024-09-21T06:39:04.261 INFO:teuthology.orchestra.run.smithi114.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-21T06:39:04.336 INFO:teuthology.orchestra.run.smithi114.stdout:(155/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.3 MB/s | 3.2 MB 00:00 2024-09-21T06:39:04.395 INFO:teuthology.orchestra.run.smithi114.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-21T06:39:04.437 INFO:teuthology.orchestra.run.smithi114.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 6.7 MB/s | 1.4 MB 00:00 2024-09-21T06:39:04.443 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:39:04.443 INFO:teuthology.orchestra.run.smithi114.stdout:Total 24 MB/s | 228 MB 00:09 2024-09-21T06:39:04.483 INFO:teuthology.orchestra.run.smithi183.stdout:(157/157): librbd1-19.3.0-5120.g18432218.el9.x8 3.7 MB/s | 3.2 MB 00:00 2024-09-21T06:39:04.489 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:39:04.489 INFO:teuthology.orchestra.run.smithi183.stdout:Total 16 MB/s | 228 MB 00:13 2024-09-21T06:39:05.733 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-21T06:39:05.881 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-21T06:39:05.881 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-21T06:39:06.534 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction check 2024-09-21T06:39:06.714 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction check succeeded. 2024-09-21T06:39:06.714 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction test 2024-09-21T06:39:07.685 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-21T06:39:07.687 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-21T06:39:09.082 INFO:teuthology.orchestra.run.smithi114.stdout:Transaction test succeeded. 2024-09-21T06:39:09.082 INFO:teuthology.orchestra.run.smithi114.stdout:Running transaction 2024-09-21T06:39:09.678 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-21T06:39:09.798 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:39:09.937 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:39:10.090 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:39:10.124 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:39:10.149 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:39:10.200 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:39:10.534 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:39:10.667 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:39:10.793 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:39:10.827 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:39:10.918 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:39:10.948 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:39:10.977 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:39:11.128 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:39:11.188 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:39:11.216 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:39:11.347 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:39:11.378 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:39:11.438 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:39:11.460 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:39:11.479 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:39:11.669 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:39:11.829 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:39:11.966 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:39:12.060 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:12.147 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:12.383 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:12.495 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:39:12.535 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:39:12.578 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:39:12.642 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:39:12.687 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:39:12.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:39:12.771 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:39:12.812 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:39:12.848 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:39:12.916 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:39:12.970 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:39:13.001 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:39:13.037 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:39:13.117 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:39:13.149 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:39:13.191 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:39:13.220 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:39:13.288 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:39:13.343 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:39:13.417 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:39:13.547 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:39:13.607 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:39:13.936 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-21T06:39:14.061 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:39:14.123 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:39:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:39:14.211 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:39:14.250 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:39:14.301 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:39:14.339 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:39:14.380 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:39:14.404 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:39:14.437 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:39:14.462 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:39:14.514 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:39:14.548 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:39:14.601 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:39:14.691 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:39:14.757 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:39:14.838 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:39:14.870 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:39:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:39:14.926 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:39:14.972 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:39:14.977 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:39:15.014 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:39:15.057 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:39:15.098 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:39:15.131 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:39:15.133 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:39:15.223 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:39:15.252 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:39:15.282 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:39:15.432 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:39:15.480 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:39:15.483 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:39:15.512 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:39:15.576 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:39:15.618 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:39:15.625 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:39:15.646 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:39:15.708 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:39:15.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:39:15.730 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:39:15.754 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:39:15.759 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:39:15.833 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:39:15.858 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:39:15.883 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:39:15.940 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:39:15.940 INFO:teuthology.orchestra.run.smithi114.stdout: Preparing : 1/1 2024-09-21T06:39:15.956 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:39:16.064 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:39:16.076 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:39:16.083 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T06:39:16.109 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:39:16.137 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T06:39:16.138 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:39:16.171 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:39:16.190 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:39:16.199 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T06:39:16.222 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:39:16.244 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T06:39:16.259 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:39:16.275 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T06:39:16.289 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:39:16.301 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:16.324 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:39:16.332 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T06:39:16.357 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:39:16.399 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:39:16.400 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:16.434 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:39:16.519 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:39:16.563 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:39:16.603 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:39:16.609 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:16.713 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:39:16.717 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T06:39:16.723 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:39:16.762 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:39:16.763 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:39:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:39:16.835 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:39:16.862 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:39:16.915 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:39:16.972 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:39:16.984 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:39:17.015 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:39:17.040 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:39:17.048 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:39:17.077 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:39:17.145 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:39:17.156 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:39:17.182 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:39:17.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:39:17.222 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T06:39:17.229 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:39:17.265 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:39:17.355 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:39:17.401 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:39:17.445 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:39:17.446 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T06:39:17.474 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:39:17.489 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:39:17.496 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T06:39:17.523 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:39:17.535 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:39:17.543 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:39:17.577 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:39:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:39:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:39:17.668 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:39:17.674 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:39:17.691 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:39:17.732 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:39:17.774 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T06:39:17.777 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:39:17.798 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:39:17.800 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:39:17.842 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T06:39:17.849 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:39:17.855 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:39:17.930 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:39:18.076 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:39:18.122 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:39:18.153 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:39:18.230 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:39:18.259 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T06:39:18.401 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:39:18.457 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T06:39:18.504 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:39:18.533 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:39:18.567 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:39:18.571 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:18.614 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:39:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:39:18.675 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:18.820 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:39:18.866 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:39:18.885 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:39:18.984 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:39:18.985 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:39:18.987 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T06:39:19.049 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:39:19.121 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:39:19.122 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T06:39:19.154 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:39:19.155 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T06:39:19.190 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:39:19.199 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T06:39:19.236 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:39:19.273 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:39:19.280 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T06:39:19.316 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:39:19.365 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T06:39:19.408 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T06:39:19.461 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T06:39:19.501 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T06:39:19.513 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:39:19.534 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T06:39:19.622 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T06:39:19.677 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T06:39:19.689 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:39:19.713 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T06:39:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:39:19.751 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T06:39:19.793 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:39:19.836 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T06:39:19.875 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T06:39:19.894 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:39:19.927 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:39:19.935 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T06:39:19.963 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T06:39:19.966 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:39:19.992 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:39:20.017 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:39:20.043 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T06:39:20.090 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:39:20.099 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T06:39:20.190 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T06:39:20.206 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:39:20.209 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:39:20.258 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:39:20.292 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:39:20.293 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:39:20.320 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:39:20.328 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:39:20.346 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T06:39:20.353 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:39:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:39:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:39:20.400 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T06:39:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:39:20.480 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:39:20.514 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:39:20.555 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:39:20.590 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:39:20.675 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:39:20.678 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:39:20.728 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:39:20.760 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:39:20.818 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:39:20.861 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:39:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:39:21.020 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T06:39:21.025 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:39:21.054 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:39:21.091 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:39:21.134 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T06:39:21.171 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T06:39:21.216 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T06:39:21.262 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T06:39:21.315 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T06:39:21.486 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T06:39:21.597 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:39:21.626 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:39:21.627 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:39:21.674 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:39:21.699 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:39:21.708 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:39:21.738 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:39:21.814 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:39:21.828 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T06:39:21.883 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:39:21.933 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:39:21.936 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T06:39:22.009 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T06:39:22.021 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:39:22.052 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:39:22.093 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T06:39:22.134 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T06:39:22.170 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T06:39:22.229 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T06:39:22.235 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:39:22.258 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:39:22.269 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T06:39:22.314 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T06:39:22.335 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:39:22.374 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T06:39:22.560 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T06:39:22.607 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T06:39:22.649 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T06:39:22.766 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T06:39:22.806 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T06:39:22.844 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T06:39:22.874 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T06:39:22.898 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T06:39:22.975 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:39:23.021 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T06:39:23.090 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:39:23.175 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T06:39:23.223 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T06:39:23.260 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T06:39:23.294 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T06:39:23.335 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T06:39:23.374 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T06:39:23.411 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T06:39:23.446 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T06:39:23.479 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T06:39:23.521 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T06:39:23.588 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T06:39:23.686 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T06:39:23.746 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T06:39:23.783 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T06:39:23.903 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T06:39:23.970 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T06:39:24.020 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:24.051 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:24.302 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:39:24.380 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:39:24.407 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:39:24.441 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:39:24.615 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:24.646 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:39:24.728 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:39:24.728 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:39:24.728 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:39:24.728 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:24.764 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:39:24.765 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:39:24.812 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:39:24.842 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T06:39:24.894 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:39:24.907 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T06:39:24.925 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:39:24.941 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T06:39:24.977 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:39:25.057 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T06:39:25.123 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T06:39:25.177 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T06:39:25.281 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T06:39:25.326 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T06:39:25.475 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:39:25.506 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:39:25.557 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T06:39:25.577 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:39:25.589 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T06:39:25.599 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:39:25.684 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T06:39:26.471 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T06:39:26.610 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T06:39:27.872 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:27.904 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:28.265 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T06:39:28.361 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T06:39:28.397 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T06:39:28.446 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T06:39:28.472 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:28.508 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:39:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:39:28.591 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:39:28.591 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:39:28.591 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:28.626 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:39:28.805 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T06:39:28.858 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:39:28.972 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T06:39:29.006 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T06:39:29.054 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T06:39:29.595 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T06:39:29.636 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:39:29.727 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T06:39:29.755 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T06:39:32.238 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:32.358 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:32.929 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T06:39:32.971 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:39:33.098 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T06:39:33.098 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T06:39:33.098 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T06:39:33.098 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:33.164 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-09-21T06:39:36.476 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-09-21T06:39:36.477 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:36.662 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:39:36.681 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:39:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:36.681 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:39:36.681 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:38.274 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:39:38.301 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:39:38.301 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:38.301 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:39:38.301 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:39:38.301 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:39:38.302 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:38.532 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:39:38.556 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:39:38.557 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:38.557 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:39:38.557 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:39:38.557 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:39:38.557 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:39.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:39:39.048 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:39:39.049 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:39.049 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:39:39.049 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:39:39.049 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:39:39.049 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:39.156 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:39:39.211 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:39:39.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:39:39.461 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:39:39.510 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:39:39.588 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:39:39.676 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:39:39.832 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:39:39.868 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:39:39.908 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:39:39.945 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:39:40.397 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:39:40.427 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:39:40.438 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-09-21T06:39:40.439 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:40.459 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:39:40.525 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:39:40.550 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:39:40.612 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:39:40.631 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:39:40.631 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:40.631 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:39:40.631 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:42.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:39:42.225 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:39:42.225 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:42.225 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:39:42.226 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:39:42.226 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:39:42.226 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:42.449 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:39:42.472 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:39:42.472 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:42.472 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:39:42.472 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:39:42.473 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:39:42.473 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:42.941 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:39:42.965 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:39:42.965 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:42.965 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:39:42.965 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:39:42.965 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:39:42.965 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:43.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:39:43.120 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:39:43.175 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:39:43.367 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:39:43.417 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:39:43.495 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:39:43.583 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:39:43.722 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:39:43.747 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:39:43.759 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:39:43.791 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:39:43.806 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:39:43.827 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:39:44.219 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:39:44.251 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:39:44.291 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:39:44.357 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:39:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:39:44.774 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:39:44.815 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:39:44.929 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:39:45.098 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:39:45.126 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:39:45.150 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:39:45.150 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:45.150 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:39:45.150 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:39:45.150 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:39:45.150 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:45.201 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:39:45.214 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:39:45.313 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:39:45.367 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:39:46.328 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:39:46.329 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T06:39:46.329 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /sys 2024-09-21T06:39:46.329 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /proc 2024-09-21T06:39:46.329 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /mnt 2024-09-21T06:39:46.330 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /var/tmp 2024-09-21T06:39:46.330 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /home 2024-09-21T06:39:46.330 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /root 2024-09-21T06:39:46.330 INFO:teuthology.orchestra.run.smithi114.stdout:skipping the directory /tmp 2024-09-21T06:39:46.330 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:46.354 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:39:46.378 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:39:46.378 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:46.378 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:39:46.378 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:39:46.379 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:39:46.379 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:46.430 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:39:46.453 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:39:46.453 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:46.454 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:39:46.454 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:46.514 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:39:46.537 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:39:46.537 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:46.538 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:39:46.538 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:39:46.538 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:39:46.538 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:46.541 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:39:46.578 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T06:39:46.578 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:46.578 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T06:39:46.578 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:46.865 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:39:46.888 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:39:46.888 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:46.889 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:39:46.889 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:39:46.889 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:39:46.889 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:39:47.551 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:39:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:39:48.323 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:39:48.381 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T06:39:48.381 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:48.381 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T06:39:48.381 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:39:48.381 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T06:39:48.382 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:48.577 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:39:48.605 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:39:48.623 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:39:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T06:39:48.674 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:48.674 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T06:39:48.674 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:39:48.674 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T06:39:48.674 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:48.719 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:39:48.897 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:39:48.932 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:39:48.957 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:39:48.957 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:48.957 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:39:48.957 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:39:48.957 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:39:48.957 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:49.000 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:39:49.012 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:39:49.110 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:39:49.164 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:39:49.190 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:39:49.241 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T06:39:49.241 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:49.241 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T06:39:49.241 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:39:49.242 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T06:39:49.242 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:49.365 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T06:39:49.425 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T06:39:49.490 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T06:39:49.738 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T06:39:49.803 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T06:39:49.889 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T06:39:50.003 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T06:39:50.128 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:39:50.152 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:39:50.176 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:39:50.176 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:50.176 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:39:50.176 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:39:50.176 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:39:50.176 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:50.198 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T06:39:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:39:50.251 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:39:50.252 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:50.252 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:39:50.252 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:50.312 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:39:50.336 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:39:50.336 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:50.336 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:39:50.336 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:39:50.336 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:39:50.336 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:50.385 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T06:39:50.425 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T06:39:50.451 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T06:39:50.663 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:39:50.686 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:39:50.686 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:50.686 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:39:50.686 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:39:50.686 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:39:50.686 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:39:50.870 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:39:50.911 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:39:50.951 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:39:50.964 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:39:50.983 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:39:51.000 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T06:39:51.010 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:39:51.034 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T06:39:51.115 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T06:39:51.148 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:39:51.191 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:39:51.233 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:39:51.263 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:39:51.301 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:39:51.356 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:39:51.966 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:39:52.004 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:39:52.024 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:39:52.858 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:39:52.943 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:39:52.983 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:39:53.074 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:39:53.113 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:39:53.147 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:39:53.173 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:39:53.174 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:39:53.257 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:39:53.257 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:39:53.341 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:39:53.367 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:39:53.372 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:39:53.372 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:39:53.551 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:39:53.552 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:39:53.644 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:39:53.707 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:39:54.642 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:39:54.700 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:39:54.740 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:39:54.767 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T06:39:54.772 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:39:54.799 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:39:54.846 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:39:54.980 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:39:55.038 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:39:55.068 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:39:55.098 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:39:55.169 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:39:55.748 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:39:55.792 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:39:55.813 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:39:55.925 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T06:39:55.962 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:39:56.113 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T06:39:56.307 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T06:39:56.339 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:39:56.391 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T06:39:56.392 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:56.392 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T06:39:56.392 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:39:56.392 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T06:39:56.392 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:56.473 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:39:56.500 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T06:39:56.608 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T06:39:56.629 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:39:56.676 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T06:39:56.713 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:39:56.754 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:39:56.834 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:39:56.892 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:39:56.917 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:39:56.944 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:39:56.944 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:39:57.027 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:39:57.027 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:39:57.111 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:39:57.136 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:39:57.142 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:39:57.142 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:39:57.322 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:39:57.322 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:39:57.414 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:39:57.476 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:39:57.728 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T06:39:57.768 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:39:57.819 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T06:39:57.819 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:57.819 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T06:39:57.819 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:39:57.819 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T06:39:57.819 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:57.868 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:39:57.918 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T06:39:57.918 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:57.918 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T06:39:57.918 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:57.969 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:39:58.019 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T06:39:58.020 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:58.020 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T06:39:58.020 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:39:58.020 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T06:39:58.020 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:39:58.386 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:39:58.438 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T06:39:58.438 INFO:teuthology.orchestra.run.smithi114.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T06:39:58.438 INFO:teuthology.orchestra.run.smithi114.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T06:39:58.438 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:39:58.438 INFO:teuthology.orchestra.run.smithi114.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T06:39:58.438 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:40:00.237 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:40:00.238 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:40:00.239 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:40:00.240 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:40:00.241 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:40:00.242 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:40:00.243 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:40:00.244 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:40:00.245 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:40:00.246 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:40:00.247 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:40:00.248 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:40:02.760 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T06:40:02.825 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T06:40:02.868 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T06:40:02.897 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T06:40:02.932 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T06:40:03.023 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:03.023 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:40:03.023 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-09-21T06:40:03.023 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:40:03.024 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.025 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:40:03.026 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:40:03.027 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:03.028 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:40:03.029 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:40:03.030 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:40:03.031 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:40:03.032 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:40:03.033 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:40:03.034 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:40:03.035 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:40:03.035 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:40:03.035 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-21T06:40:03.188 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T06:40:03.254 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T06:40:03.283 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T06:40:03.315 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T06:40:03.384 INFO:teuthology.orchestra.run.smithi114.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T06:40:03.470 DEBUG:teuthology.parallel:result is None 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:40:03.904 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:40:03.905 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:40:03.906 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:40:03.906 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:40:03.906 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:40:03.906 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:40:03.906 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:40:03.906 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:40:03.917 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:40:03.918 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:40:03.919 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:40:03.920 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:40:03.921 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:40:03.922 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:40:03.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:40:03.924 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:40:03.925 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:40:03.926 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:40:04.164 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:40:04.215 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:40:04.257 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T06:40:05.540 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T06:40:05.631 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T06:40:05.681 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T06:40:05.810 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T06:40:05.850 INFO:teuthology.orchestra.run.smithi114.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T06:40:05.875 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T06:40:05.920 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T06:40:05.920 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:40:06.036 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T06:40:06.036 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:40:06.145 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T06:40:06.169 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T06:40:06.185 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:40:06.185 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:40:06.438 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T06:40:06.438 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:40:06.547 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.600 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:40:06.601 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.602 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:40:06.603 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:40:06.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:40:06.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:40:06.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:40:06.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:06.608 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:40:06.609 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-21T06:40:06.621 INFO:teuthology.orchestra.run.smithi114.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:06.840 INFO:teuthology.orchestra.run.smithi114.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:06.840 INFO:teuthology.orchestra.run.smithi114.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T06:40:06.840 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:40:07.024 DEBUG:teuthology.parallel:result is None 2024-09-21T06:40:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T06:40:12.902 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T06:40:12.903 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T06:40:12.904 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T06:40:12.915 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T06:40:12.916 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T06:40:12.917 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T06:40:12.918 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T06:40:12.919 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T06:40:12.920 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T06:40:12.921 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T06:40:12.922 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T06:40:12.923 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T06:40:12.924 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T06:40:12.924 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T06:40:12.924 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T06:40:12.924 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T06:40:12.924 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T06:40:15.183 INFO:teuthology.orchestra.run.smithi114.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T06:40:15.183 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:40:15.183 INFO:teuthology.orchestra.run.smithi114.stdout:Upgraded: 2024-09-21T06:40:15.183 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout:Installed: 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.184 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.185 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T06:40:15.186 INFO:teuthology.orchestra.run.smithi114.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.187 INFO:teuthology.orchestra.run.smithi114.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T06:40:15.188 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T06:40:15.189 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T06:40:15.190 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T06:40:15.191 INFO:teuthology.orchestra.run.smithi114.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T06:40:15.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T06:40:15.192 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T06:40:15.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T06:40:15.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T06:40:15.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T06:40:15.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T06:40:15.193 INFO:teuthology.orchestra.run.smithi114.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T06:40:15.194 INFO:teuthology.orchestra.run.smithi114.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T06:40:15.195 INFO:teuthology.orchestra.run.smithi114.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T06:40:15.196 INFO:teuthology.orchestra.run.smithi114.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T06:40:15.196 INFO:teuthology.orchestra.run.smithi114.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T06:40:15.196 INFO:teuthology.orchestra.run.smithi114.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T06:40:15.196 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:40:15.196 INFO:teuthology.orchestra.run.smithi114.stdout:Complete! 2024-09-21T06:40:15.823 DEBUG:teuthology.parallel:result is None 2024-09-21T06:40:15.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:40:15.974 DEBUG:teuthology.orchestra.run.smithi114:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:40:16.018 INFO:teuthology.orchestra.run.smithi114.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:40:16.019 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:40:16.019 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:40:16.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:40:16.168 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:40:16.194 INFO:teuthology.orchestra.run.smithi183.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:40:16.194 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:40:16.194 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:40:16.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:40:16.337 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T06:40:16.363 INFO:teuthology.orchestra.run.smithi193.stdout:19.3.0-5120.g18432218.el9 2024-09-21T06:40:16.364 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T06:40:16.364 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T06:40:16.365 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T06:40:16.366 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:40:16.366 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:40:16.424 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:40:16.424 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:40:16.457 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:40:16.457 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T06:40:16.489 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T06:40:16.489 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:40:16.489 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:40:16.547 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:40:16.644 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:40:16.644 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:40:16.673 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:40:16.743 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:40:16.743 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T06:40:16.769 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T06:40:16.834 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T06:40:16.834 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:40:16.834 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:40:16.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:40:16.984 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:40:16.984 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:40:17.014 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:40:17.083 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:40:17.084 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T06:40:17.109 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T06:40:17.174 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T06:40:17.174 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:40:17.175 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:40:17.230 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:40:17.326 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:40:17.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:40:17.356 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:40:17.423 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:40:17.423 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T06:40:17.450 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T06:40:17.514 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T06:40:17.600 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T06:40:17.600 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:40:17.601 INFO:tasks.cephadm:Cluster fsid is 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:40:17.601 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T06:40:17.601 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.114', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.193'} 2024-09-21T06:40:17.601 INFO:tasks.cephadm:First mon is mon.a on smithi114 2024-09-21T06:40:17.601 INFO:tasks.cephadm:First mgr is x 2024-09-21T06:40:17.601 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T06:40:17.601 DEBUG:teuthology.orchestra.run.smithi114:> sudo hostname $(hostname -s) 2024-09-21T06:40:17.659 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-21T06:40:17.688 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-21T06:40:17.712 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T06:40:17.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:40:17.858 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T06:40:18.000 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:40:18.001 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:40:18.001 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T06:40:18.001 DEBUG:teuthology.orchestra.run.smithi114:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:18.675 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:40 /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:18.675 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:19.325 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:40 /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:19.325 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:19.946 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 06:40 /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:19.946 DEBUG:teuthology.orchestra.run.smithi114:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:19.984 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:20.004 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T06:40:20.040 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T06:40:20.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:40:20.046 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:40:20.048 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T06:40:21.638 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:40:21.649 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:40:21.993 INFO:teuthology.orchestra.run.smithi114.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout: "repo_digests": [ 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T06:40:54.920 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:41:05.867 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-21T06:41:05.867 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:41:05.868 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:41:05.868 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-21T06:41:05.868 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:41:05.868 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-21T06:41:05.868 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-21T06:41:17.504 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T06:41:17.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo mkdir -p /etc/ceph 2024-09-21T06:41:17.594 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-21T06:41:17.628 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-21T06:41:17.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 777 /etc/ceph 2024-09-21T06:41:17.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-21T06:41:17.742 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-21T06:41:17.767 INFO:tasks.cephadm:Writing seed config... 2024-09-21T06:41:17.768 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T06:41:17.768 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T06:41:17.769 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T06:41:17.770 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T06:41:17.770 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T06:41:17.770 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T06:41:17.770 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-21T06:41:17.770 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:41:17.770 DEBUG:teuthology.orchestra.run.smithi114:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T06:41:17.806 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5d75913a-77e4-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T06:41:17.806 DEBUG:teuthology.orchestra.run.smithi114:mon.a> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T06:41:17.850 DEBUG:teuthology.orchestra.run.smithi114:mgr.x> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T06:41:17.895 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T06:41:17.895 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.114 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:41:18.150 INFO:teuthology.orchestra.run.smithi114.stdout:-------------------------------------------------------------------------------- 2024-09-21T06:41:18.151 INFO:teuthology.orchestra.run.smithi114.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.114', '--skip-admin-label'] 2024-09-21T06:41:18.151 INFO:teuthology.orchestra.run.smithi114.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T06:41:18.151 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying podman|docker is present... 2024-09-21T06:41:18.224 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:41:18.225 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying lvm2 is present... 2024-09-21T06:41:18.225 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying time synchronization is in place... 2024-09-21T06:41:18.239 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:41:18.239 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:41:18.251 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:41:18.251 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-21T06:41:18.265 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout enabled 2024-09-21T06:41:18.277 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout active 2024-09-21T06:41:18.278 INFO:teuthology.orchestra.run.smithi114.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:41:18.278 INFO:teuthology.orchestra.run.smithi114.stdout:Repeating the final host check... 2024-09-21T06:41:18.321 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout 5.2.2 2024-09-21T06:41:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T06:41:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl is present 2024-09-21T06:41:18.322 INFO:teuthology.orchestra.run.smithi114.stdout:lvcreate is present 2024-09-21T06:41:18.335 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T06:41:18.335 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T06:41:18.347 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T06:41:18.347 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-21T06:41:18.360 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout enabled 2024-09-21T06:41:18.373 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout active 2024-09-21T06:41:18.373 INFO:teuthology.orchestra.run.smithi114.stdout:Unit chronyd.service is enabled and running 2024-09-21T06:41:18.373 INFO:teuthology.orchestra.run.smithi114.stdout:Host looks OK 2024-09-21T06:41:18.373 INFO:teuthology.orchestra.run.smithi114.stdout:Cluster fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:18.374 INFO:teuthology.orchestra.run.smithi114.stdout:Acquiring lock 140206452845200 on /run/cephadm/5d75913a-77e4-11ef-baf5-efdc52797490.lock 2024-09-21T06:41:18.374 INFO:teuthology.orchestra.run.smithi114.stdout:Lock 140206452845200 acquired on /run/cephadm/5d75913a-77e4-11ef-baf5-efdc52797490.lock 2024-09-21T06:41:18.374 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying IP 172.21.15.114 port 3300 ... 2024-09-21T06:41:18.374 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying IP 172.21.15.114 port 6789 ... 2024-09-21T06:41:18.375 INFO:teuthology.orchestra.run.smithi114.stdout:Base mon IP(s) is [172.21.15.114:3300, 172.21.15.114:6789], mon addrv is [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-09-21T06:41:18.381 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.114 metric 100 2024-09-21T06:41:18.381 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.114 metric 100 2024-09-21T06:41:18.388 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T06:41:18.388 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T06:41:18.388 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T06:41:18.393 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T06:41:18.393 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T06:41:18.393 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:41:18.393 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T06:41:18.394 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link noprefixroute 2024-09-21T06:41:18.394 INFO:teuthology.orchestra.run.smithi114.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T06:41:18.395 INFO:teuthology.orchestra.run.smithi114.stdout:Mon IP `172.21.15.114` is in CIDR network `172.21.0.0/20` 2024-09-21T06:41:18.395 INFO:teuthology.orchestra.run.smithi114.stdout:Mon IP `172.21.15.114` is in CIDR network `172.21.0.0/20` 2024-09-21T06:41:18.395 INFO:teuthology.orchestra.run.smithi114.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T06:41:18.395 INFO:teuthology.orchestra.run.smithi114.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T06:41:18.396 INFO:teuthology.orchestra.run.smithi114.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:41:18.904 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:41:18.904 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T06:41:18.904 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T06:41:18.905 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T06:41:18.905 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T06:41:18.905 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T06:41:18.905 INFO:teuthology.orchestra.run.smithi114.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T06:41:20.036 INFO:teuthology.orchestra.run.smithi114.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:41:20.036 INFO:teuthology.orchestra.run.smithi114.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T06:41:20.036 INFO:teuthology.orchestra.run.smithi114.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T06:41:21.105 INFO:teuthology.orchestra.run.smithi114.stdout:stat: stdout 167 167 2024-09-21T06:41:21.105 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial keys... 2024-09-21T06:41:22.160 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQCRau5mFRr0HxAAwJr5eW27lj27whNrI6qW9Q== 2024-09-21T06:41:23.211 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQCSau5m3aCMIxAAgoZ45Ti97BdWuPhg6XQn+Q== 2024-09-21T06:41:24.285 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph-authtool: stdout AQCTau5m+e3ZKRAA+N6fbGOLMCk2p5ekzMx+jw== 2024-09-21T06:41:24.286 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial monmap... 2024-09-21T06:41:25.420 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:41:25.420 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T06:41:25.420 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:25.420 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:41:25.421 INFO:teuthology.orchestra.run.smithi114.stdout:monmaptool for a [v2:172.21.15.114:3300,v1:172.21.15.114:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T06:41:25.421 INFO:teuthology.orchestra.run.smithi114.stdout:setting min_mon_release = quincy 2024-09-21T06:41:25.421 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: set fsid to 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:25.421 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T06:41:25.421 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:41:25.421 INFO:teuthology.orchestra.run.smithi114.stdout:Creating mon... 2024-09-21T06:41:26.551 INFO:teuthology.orchestra.run.smithi114.stdout:create mon.a on 2024-09-21T06:41:26.873 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T06:41:27.095 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T06:41:27.349 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5d75913a-77e4-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-5d75913a-77e4-11ef-baf5-efdc52797490.target. 2024-09-21T06:41:27.349 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5d75913a-77e4-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-5d75913a-77e4-11ef-baf5-efdc52797490.target. 2024-09-21T06:41:27.702 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:41:27.702 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T06:41:27.979 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5d75913a-77e4-11ef-baf5-efdc52797490.target.wants/ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-5d75913a-77e4-11ef-baf5-efdc52797490@.service. 2024-09-21T06:41:28.650 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:28 smithi114 podman[30277]: 2024-09-21 06:41:28.538863369 +0000 UTC m=+0.336320170 container start dee50341be697c0d274c58f87e9ac11525c88f14b4b8fffb237f63c4612b1222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T06:41:28.651 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:28 smithi114 bash[30277]: dee50341be697c0d274c58f87e9ac11525c88f14b4b8fffb237f63c4612b1222 2024-09-21T06:41:28.680 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:41:28.680 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-21T06:41:28.693 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:41:28.693 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-21T06:41:28.693 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-21T06:41:28.693 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:41:28.694 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mon to start... 2024-09-21T06:41:28.694 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mon... 2024-09-21T06:41:28.941 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:28 smithi114 systemd[1]: Started Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:41:28.941 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:28 smithi114 ceph-mon[30295]: mkfs 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:28.941 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:28 smithi114 ceph-mon[30295]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:41:29.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:41:29.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: monmap epoch 1 2024-09-21T06:41:29.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:29.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: last_changed 2024-09-21T06:41:24.747593+0000 2024-09-21T06:41:29.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:41:29.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: min_mon_release 19 (squid) 2024-09-21T06:41:29.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: election_strategy: 1 2024-09-21T06:41:29.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:41:29.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: fsmap 2024-09-21T06:41:29.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:41:29.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: mgrmap e1: no daemons active 2024-09-21T06:41:29.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:29 smithi114 ceph-mon[30295]: from='client.? 172.21.15.114:0/3188483506' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T06:41:29.962 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout id: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout services: 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.66724s) 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout data: 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T06:41:29.963 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:29.964 INFO:teuthology.orchestra.run.smithi114.stdout:mon is available 2024-09-21T06:41:29.964 INFO:teuthology.orchestra.run.smithi114.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T06:41:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:30 smithi114 ceph-mon[30295]: from='client.? 172.21.15.114:0/1652344688' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:41:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:30 smithi114 ceph-mon[30295]: from='client.? 172.21.15.114:0/1652344688' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T06:41:31.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:31.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:41:31.231 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout fsid = 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:31.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:41:31.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-09-21T06:41:31.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:41:31.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:41:31.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:41:31.232 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:41:31.233 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:41:31.234 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:41:31.234 INFO:teuthology.orchestra.run.smithi114.stdout:Generating new minimal ceph.conf... 2024-09-21T06:41:32.178 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:31 smithi114 ceph-mon[30295]: from='client.? 172.21.15.114:0/4261342023' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:41:32.442 INFO:teuthology.orchestra.run.smithi114.stdout:Restarting the monitor... 2024-09-21T06:41:32.801 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:32 smithi114 systemd[1]: Stopping Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:41:33.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a[30291]: 2024-09-21T06:41:32.798+0000 7f00b4800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:41:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a[30291]: 2024-09-21T06:41:32.798+0000 7f00b4800640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T06:41:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:32 smithi114 podman[30536]: 2024-09-21 06:41:32.881286837 +0000 UTC m=+0.211747166 container died dee50341be697c0d274c58f87e9ac11525c88f14b4b8fffb237f63c4612b1222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:41:33.456 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 podman[30536]: 2024-09-21 06:41:33.287795963 +0000 UTC m=+0.618256294 container cleanup dee50341be697c0d274c58f87e9ac11525c88f14b4b8fffb237f63c4612b1222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:41:33.456 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 bash[30536]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:41:33.708 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 podman[30549]: 2024-09-21 06:41:33.455322698 +0000 UTC m=+0.561747425 container remove dee50341be697c0d274c58f87e9ac11525c88f14b4b8fffb237f63c4612b1222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T06:41:34.024 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T06:41:34.024 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 systemd[1]: Stopped Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:41:34.024 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service: Consumed 1.275s CPU time. 2024-09-21T06:41:34.024 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:33 smithi114 systemd[1]: Starting Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:41:34.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 podman[30635]: 2024-09-21 06:41:34.022848823 +0000 UTC m=+0.110913347 container create 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T06:41:34.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 podman[30635]: 2024-09-21 06:41:33.938546346 +0000 UTC m=+0.026610878 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:41:34.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 podman[30635]: 2024-09-21 06:41:34.233681936 +0000 UTC m=+0.321746486 container init 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T06:41:34.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 podman[30635]: 2024-09-21 06:41:34.240309307 +0000 UTC m=+0.328373841 container start 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-21T06:41:34.339 INFO:teuthology.orchestra.run.smithi114.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T06:41:34.606 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 bash[30635]: 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 2024-09-21T06:41:34.606 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 systemd[1]: Started Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: pidfile_write: ignore empty --pid-file 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: load: jerasure load: lrc 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Git sha 0 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: DB SUMMARY 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: DB Session ID: EQWNCTSTXU5TLXIL6P5B 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 93744 ; 2024-09-21T06:41:34.607 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.env: 0x560b8ccc3d60 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.info_log: 0x560bd072d320 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.statistics: (nil) 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.use_fsync: 0 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:41:34.608 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:41:34.609 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.db_log_dir: 2024-09-21T06:41:34.610 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.wal_dir: 2024-09-21T06:41:34.610 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:41:34.610 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:41:34.610 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:41:34.610 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:41:34.610 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.write_buffer_manager: 0x560bd0731d60 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.unordered_write: 0 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:41:34.611 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.row_cache: None 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.wal_filter: None 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.wal_compression: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:41:34.612 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:41:34.613 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_open_files: -1 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Compression algorithms supported: 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kZSTD supported: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:41:34.614 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.merge_operator: 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_filter: None 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:41:34.615 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560bd072cf60) 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:41:34.616 INFO:journalctl@ceph.mon.a.smithi114.stdout: index_type: 0 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: data_block_index_type: 0 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: index_shortening: 1 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: checksum: 4 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: no_block_cache: 0 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache: 0x560bcc6f1610 2024-09-21T06:41:34.617 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache_options: 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: capacity : 536870912 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: num_shard_bits : 4 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: strict_capacity_limit : 0 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_cache_compressed: (nil) 2024-09-21T06:41:34.618 INFO:journalctl@ceph.mon.a.smithi114.stdout: persistent_cache: (nil) 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_size: 4096 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_size_deviation: 10 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_restart_interval: 16 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: index_block_restart_interval: 1 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: metadata_block_size: 4096 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: partition_filters: 0 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: use_delta_encoding: 1 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: filter_policy: bloomfilter 2024-09-21T06:41:34.619 INFO:journalctl@ceph.mon.a.smithi114.stdout: whole_key_filtering: 1 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: verify_compression: 0 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: format_version: 5 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: enable_index_compression: 1 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: block_align: 0 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: max_auto_readahead_size: 262144 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: prepopulate_block_cache: 0 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:41:34.620 INFO:journalctl@ceph.mon.a.smithi114.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression: NoCompression 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.num_levels: 7 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:41:34.621 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:41:34.622 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:41:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:41:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:41:34.625 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:41:34.626 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.ttl: 2592000 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:41:34.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3aeb8d6c-72c0-40b2-a3c3-237461fe3e1b 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726900894347603, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726900894349645, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90013, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 279, "table_properties": {"data_size": 88083, "index_size": 250, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11333, "raw_average_key_size": 47, "raw_value_size": 81688, "raw_average_value_size": 343, "num_data_blocks": 11, "num_entries": 238, "num_filter_entries": 238, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726900894, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3aeb8d6c-72c0-40b2-a3c3-237461fe3e1b", "db_session_id": "EQWNCTSTXU5TLXIL6P5B", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726900894349864, "job": 1, "event": "recovery_finished"} 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560bd0752a80 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: DB pointer 0x560bd14f4000 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** DB Stats ** 2024-09-21T06:41:34.628 INFO:journalctl@ceph.mon.a.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** Compaction Stats [default] ** 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: L0 2/0 89.76 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Sum 2/0 89.76 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 63.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-21T06:41:34.629 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** Compaction Stats [default] ** 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 63.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: Cumulative compaction: 0.00 GB write, 12.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:41:34.630 INFO:journalctl@ceph.mon.a.smithi114.stdout: Interval compaction: 0.00 GB write, 12.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: Block cache BinnedLRUCache@0x560bcc6f1610#2 capacity: 512.00 MB usage: 1.28 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.44 KB,8.34465e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: starting mon.a rank 0 at public addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] at bind addrs [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???) e1 preinit fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).mds e1 new map 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).mds e1 print_map 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: e1 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: btime 2024-09-21T06:41:28:662644+0000 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:41:34.631 INFO:journalctl@ceph.mon.a.smithi114.stdout: legacy client fscid: -1 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout: 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout: No filesystems configured 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: monmap epoch 1 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: last_changed 2024-09-21T06:41:24.747593+0000 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: min_mon_release 19 (squid) 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: election_strategy: 1 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:41:34.632 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: fsmap 2024-09-21T06:41:34.633 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T06:41:34.633 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:34 smithi114 ceph-mon[30654]: mgrmap e1: no daemons active 2024-09-21T06:41:35.863 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T06:41:35.890 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:41:35.891 INFO:teuthology.orchestra.run.smithi114.stdout:Creating mgr... 2024-09-21T06:41:35.891 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T06:41:35.891 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T06:41:36.237 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:41:36.238 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T06:41:36.467 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5d75913a-77e4-11ef-baf5-efdc52797490.target.wants/ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-5d75913a-77e4-11ef-baf5-efdc52797490@.service. 2024-09-21T06:41:36.501 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/166336635' entity='client.admin' 2024-09-21T06:41:36.809 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:36 smithi114 systemd[1]: Starting Ceph mgr.x for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:41:37.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:36 smithi114 podman[30859]: 2024-09-21 06:41:36.711823797 +0000 UTC m=+0.027356175 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T06:41:37.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:36 smithi114 podman[30859]: 2024-09-21 06:41:36.80891718 +0000 UTC m=+0.124449543 container create 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T06:41:37.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:37 smithi114 podman[30859]: 2024-09-21 06:41:37.0031015 +0000 UTC m=+0.318633878 container init 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:41:37.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:37 smithi114 podman[30859]: 2024-09-21 06:41:37.009210172 +0000 UTC m=+0.324742529 container start 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:41:37.126 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:41:37.126 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-21T06:41:37.141 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:41:37.141 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-21T06:41:37.141 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-21T06:41:37.141 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T06:41:37.162 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:41:37.162 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-21T06:41:37.176 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:41:37.176 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-21T06:41:37.176 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-21T06:41:37.176 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T06:41:37.176 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr to start... 2024-09-21T06:41:37.176 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr... 2024-09-21T06:41:37.363 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:37 smithi114 bash[30859]: 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 2024-09-21T06:41:37.364 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:37 smithi114 systemd[1]: Started Ceph mgr.x for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:41:38.236 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:37 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/789854642' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:41:38.569 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.570 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:41:38.571 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:41:38.574 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:41:28:662644+0000", 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:41:38.575 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:41:28.663590+0000", 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:41:38.576 INFO:teuthology.orchestra.run.smithi114.stdout:mgr not available, waiting (1/15)... 2024-09-21T06:41:41.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1840465370' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:41:41.903 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:41.903 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:41:41.903 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:41:41.903 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:41.904 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:41:41.905 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:41:41.908 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.908 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:41:41.908 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:41:41.908 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:41:28:662644+0000", 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:41:41.909 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:41:28.663590+0000", 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:41:41.910 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:41:41.911 INFO:teuthology.orchestra.run.smithi114.stdout:mgr not available, waiting (2/15)... 2024-09-21T06:41:43.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: Activating manager daemon x 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: mgrmap e2: x(active, starting, since 0.00511162s) 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: Manager daemon x is now available 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' 2024-09-21T06:41:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:43 smithi114 ceph-mon[30654]: from='mgr.14100 172.21.15.114:0/1385005105' entity='mgr.x' 2024-09-21T06:41:45.394 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:45 smithi114 ceph-mon[30654]: mgrmap e3: x(active, since 1.01033s) 2024-09-21T06:41:45.394 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:45 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3378927392' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T06:41:45.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:45.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:41:45.403 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 0 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T06:41:45.404 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T06:41:45.405 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T06:41:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T06:41:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T06:41:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T06:41:45.408 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T06:41:28:662644+0000", 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.409 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ], 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T06:41:28.663590+0000", 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T06:41:45.410 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout }, 2024-09-21T06:41:45.411 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T06:41:45.411 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:41:45.411 INFO:teuthology.orchestra.run.smithi114.stdout:mgr is available 2024-09-21T06:41:46.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:46 smithi114 ceph-mon[30654]: mgrmap e4: x(active, since 2s) 2024-09-21T06:41:46.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:46.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T06:41:46.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout fsid = 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:41:46.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T06:41:46.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.114:3300,v1:172.21.15.114:6789] 2024-09-21T06:41:46.953 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T06:41:46.954 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling cephadm module... 2024-09-21T06:41:47.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:47 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/649812646' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T06:41:48.462 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:48 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/371179735' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T06:41:48.462 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:48 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ignoring --setuser ceph since I am not root 2024-09-21T06:41:48.462 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:48 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ignoring --setgroup ceph since I am not root 2024-09-21T06:41:49.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/371179735' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T06:41:49.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:49 smithi114 ceph-mon[30654]: mgrmap e5: x(active, since 5s) 2024-09-21T06:41:50.251 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:50 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2547930689' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:41:50.257 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for the mgr to restart... 2024-09-21T06:41:50.258 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr epoch 5... 2024-09-21T06:41:54.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: Active manager daemon x restarted 2024-09-21T06:41:54.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: Activating manager daemon x 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: mgrmap e6: x(active, starting, since 0.00483649s) 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:41:54.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:54 smithi114 ceph-mon[30654]: Manager daemon x is now available 2024-09-21T06:41:56.033 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:41:56.033 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:41:56.033 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: Found migration_current of "None". Setting to last migration. 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: mgrmap e7: x(active, since 1.00969s) 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:41:55.573+0000 7f5dd5c00640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:41:55] ENGINE Error in HTTPServer.serve 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Traceback (most recent call last): 2024-09-21T06:41:56.034 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._connections.run(self.expiration_interval) 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._run(expiration_interval) 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: s = self.context.wrap_socket( 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: return self.sslsocket_class._create( 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:41:56.035 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self.do_handshake() 2024-09-21T06:41:56.036 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:41:56.036 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._sslobj.do_handshake() 2024-09-21T06:41:56.036 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:41:56.036 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:41:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:41:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:41:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T06:41:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:41:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:41:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:mgr epoch 5 is available 2024-09-21T06:41:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:Verifying orchestrator module is enabled... 2024-09-21T06:41:57.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:56 smithi114 ceph-mon[30654]: [21/Sep/2024:06:41:55] ENGINE Bus STARTING 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:56 smithi114 ceph-mon[30654]: [21/Sep/2024:06:41:55] ENGINE Serving on http://172.21.15.114:8765 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:56 smithi114 ceph-mon[30654]: [21/Sep/2024:06:41:55] ENGINE Serving on https://172.21.15.114:7150 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:56 smithi114 ceph-mon[30654]: [21/Sep/2024:06:41:55] ENGINE Bus STARTED 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:56 smithi114 ceph-mon[30654]: [21/Sep/2024:06:41:55] ENGINE Error in HTTPServer.serve 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: Traceback (most recent call last): 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._run(expiration_interval) 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:41:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: s = self.context.wrap_socket( 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: return self.sslsocket_class._create( 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: self.do_handshake() 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._sslobj.do_handshake() 2024-09-21T06:41:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:41:58.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:57 smithi114 ceph-mon[30654]: mgrmap e8: x(active, since 2s) 2024-09-21T06:41:58.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:57 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2170650908' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T06:41:58.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T06:41:58.438 INFO:teuthology.orchestra.run.smithi114.stdout:Setting orchestrator backend to cephadm... 2024-09-21T06:41:59.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:58 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2170650908' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T06:41:59.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:41:58 smithi114 ceph-mon[30654]: mgrmap e9: x(active, since 3s) 2024-09-21T06:42:00.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:00 smithi114 ceph-mon[30654]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:00.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:00 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:00 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T06:42:01.300 INFO:teuthology.orchestra.run.smithi114.stdout:Generating ssh key... 2024-09-21T06:42:01.556 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:01 smithi114 ceph-mon[30654]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:02.458 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Generating public/private rsa key pair. 2024-09-21T06:42:02.458 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Your identification has been saved in /tmp/tmpmbclxkl4/key 2024-09-21T06:42:02.458 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Your public key has been saved in /tmp/tmpmbclxkl4/key.pub 2024-09-21T06:42:02.458 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: The key fingerprint is: 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: SHA256:MKYwebqvjrR9hZq8K89adpLKBmKePX44EQCvLdzvkYE ceph-5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: The key's randomart image is: 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: +---[RSA 3072]----+ 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |o | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: | o . | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: | * . + | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |.o.*.o o | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |o.oEo.. S | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |oo oo.o. | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |=.====. | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |o*BX+o. | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: |o*BOO. | 2024-09-21T06:42:02.459 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: +----[SHA256]-----+ 2024-09-21T06:42:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:03 smithi114 ceph-mon[30654]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:03 smithi114 ceph-mon[30654]: Generating ssh key... 2024-09-21T06:42:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:03 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:03 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:04.119 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:42:04.120 INFO:teuthology.orchestra.run.smithi114.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:42:04.120 INFO:teuthology.orchestra.run.smithi114.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T06:42:04.131 INFO:teuthology.orchestra.run.smithi114.stdout:Adding host smithi114... 2024-09-21T06:42:04.428 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:04 smithi114 ceph-mon[30654]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:05.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:05 smithi114 ceph-mon[30654]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi114", "addr": "172.21.15.114", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:06.381 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:06 smithi114 ceph-mon[30654]: Deploying cephadm binary to smithi114 2024-09-21T06:42:09.219 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Added host 'smithi114' with addr '172.21.15.114' 2024-09-21T06:42:09.219 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying unmanaged mon service... 2024-09-21T06:42:09.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:09 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:09 smithi114 ceph-mon[30654]: Added host smithi114 2024-09-21T06:42:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:09 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:11.453 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:11 smithi114 ceph-mon[30654]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:11.453 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:11 smithi114 ceph-mon[30654]: Saving service mon spec with placement count:5 2024-09-21T06:42:11.453 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:11 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:11.785 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T06:42:11.785 INFO:teuthology.orchestra.run.smithi114.stdout:Deploying unmanaged mgr service... 2024-09-21T06:42:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:12 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:12 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:13.998 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:13 smithi114 ceph-mon[30654]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:13.998 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:13 smithi114 ceph-mon[30654]: Saving service mgr spec with placement count:2 2024-09-21T06:42:13.998 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:13 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:13.998 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:13 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:14.003 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T06:42:15.431 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:15 smithi114 ceph-mon[30654]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:15.431 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:15 smithi114 ceph-mon[30654]: from='mgr.14118 172.21.15.114:0/53552524' entity='mgr.x' 2024-09-21T06:42:15.431 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:15 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2530861697' entity='client.admin' 2024-09-21T06:42:17.374 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:17 smithi114 ceph-mon[30654]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:17.375 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:17 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2948691323' entity='client.admin' 2024-09-21T06:42:17.379 INFO:teuthology.orchestra.run.smithi114.stdout:Enabling the dashboard module... 2024-09-21T06:42:19.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:19 smithi114 ceph-mon[30654]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:19.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:19 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2809724932' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T06:42:19.566 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ignoring --setuser ceph since I am not root 2024-09-21T06:42:19.566 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ignoring --setgroup ceph since I am not root 2024-09-21T06:42:20.460 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:20 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2809724932' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T06:42:20.460 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:20 smithi114 ceph-mon[30654]: mgrmap e10: x(active, since 25s) 2024-09-21T06:42:21.431 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:21 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1756604445' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for the mgr to restart... 2024-09-21T06:42:21.438 INFO:teuthology.orchestra.run.smithi114.stdout:Waiting for mgr epoch 10... 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: Active manager daemon x restarted 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: Activating manager daemon x 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: mgrmap e11: x(active, starting, since 0.00481987s) 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:42:25.670 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:42:25.671 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:42:25.671 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:25 smithi114 ceph-mon[30654]: Manager daemon x is now available 2024-09-21T06:42:26.656 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:26.656 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:42:26.656 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:42:26.656 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-mon[30654]: mgrmap e12: x(active, since 1.00894s) 2024-09-21T06:42:27.063 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:42:26.652+0000 7f5b51600640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:42:26] ENGINE Error in HTTPServer.serve 2024-09-21T06:42:27.063 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Traceback (most recent call last): 2024-09-21T06:42:27.063 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:42:27.063 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._connections.run(self.expiration_interval) 2024-09-21T06:42:27.063 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:42:27.063 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._run(expiration_interval) 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: s = self.context.wrap_socket( 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: return self.sslsocket_class._create( 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self.do_handshake() 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._sslobj.do_handshake() 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:42:27.064 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:42:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:42:27.068 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout { 2024-09-21T06:42:27.068 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-21T06:42:27.068 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T06:42:27.068 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout } 2024-09-21T06:42:27.069 INFO:teuthology.orchestra.run.smithi114.stdout:mgr epoch 10 is available 2024-09-21T06:42:27.069 INFO:teuthology.orchestra.run.smithi114.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T06:42:27.719 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:27 smithi114 ceph-mon[30654]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T06:42:27.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:27 smithi114 ceph-mon[30654]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T06:42:28.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: [21/Sep/2024:06:42:26] ENGINE Bus STARTING 2024-09-21T06:42:28.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: [21/Sep/2024:06:42:26] ENGINE Serving on http://172.21.15.114:8765 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: [21/Sep/2024:06:42:26] ENGINE Serving on https://172.21.15.114:7150 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: [21/Sep/2024:06:42:26] ENGINE Bus STARTED 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: [21/Sep/2024:06:42:26] ENGINE Error in HTTPServer.serve 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: Traceback (most recent call last): 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._run(expiration_interval) 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:42:28.566 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: s = self.context.wrap_socket( 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: return self.sslsocket_class._create( 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: self.do_handshake() 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._sslobj.do_handshake() 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: mgrmap e13: x(active, since 2s) 2024-09-21T06:42:28.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:28 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:29.522 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:29 smithi114 ceph-mon[30654]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:29.522 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:29 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:29.527 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE/DCCAuSgAwIBAgIUTH36f8rj8shKqAf5QPJDgWLnr9YwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTA2NDIyOFoXDTM0\nMDkyMjA2NDIyOFowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjExNDCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAO2m4ckdLrvJZrt/E9xVtBmn08mnVdMfV1GF\n/7SFSfk6lxZZrs/Ei7tE0maSAIoc2Jc3Xz+UBwFAffB0oY96otCxVMOLSq/DEvGX\nX5wGnjxZAvAkeJ7BzL7SBaBMxIzwNmHxO0CJdMKjkIGhAerUJ5lLydzyFKGW/6GC\nsYc7z5QcpxFqVHebXCt+pE0XsL+wLZelgltB67RxyGZnX2QF/dmai9VG+mh/ZLc2\nEv/quHmYvMkYvnb77wul2pGyfkDT9YQk6s6Uq6XIwZlpBHCbXVx27oxSph7t9t+0\n8d2WijjudxsGqXjJ8ITqcCPQrvI7s05DRKRw7MyFmS9xNQS1H6BdkxphIfKHpICn\nHRU+PCIoSLEPSeVmzbp3C2a8+Nw5oZGD40hhgb1bZHyzZdli3zmmVqpaPoHGhJt8\nmlO4hkJdkBuK0kiMeu+hkP1Kel5IUsQI2B++CHHfeyHNlPSmhhvG+5QVk7NQ0NY9\n+7dWnGkJUBO7bm9oY6XjhTZtZrp6JykeY+xMLFYQTr030E3DzDnmysHpohB+c/vo\nU76ESXf8r/WYDkzwHO+8Ld9qCjmE9f4s+V/MoerWdA9+dy0KiqTrhKHThVqJWXQs\nNB6sI2Jzyxd/e15zIG/kEnDSMyqVdHvF5KeY0Of9IhihTssm5ZPR1J1hahfKdtlG\ni9u/2Sx7AgMBAAGjPzA9MC0GA1UdEQQmMCSCCXNtaXRoaTExNIIRZGFzaGJvYXJk\nX3NlcnZlcnOHBKwVD3IwDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAgEA\nZ8fwXiuFaN2+9g9xcw/4rRhPYImneTtCmfFiTXuez/IHfBvTyFwlGqWdkSsUWbi9\njfvPtpmDG/gB/KD/iArktbwxjxgiweGqwF2M/ALxWspVMrGxbg7uRJ/TIxxWU6bG\navUpyGZ9wHboU9//TsAxOB0WjU8/9Q8Dp37nyvUt2S83cfKOzUbk0Atkmt4Vw+6X\npF8ZKZeZlfmEkr6R1nRBNX8wHTNeVTdSL8DPUhJQuciJ/ItxzW1Ei3haFpQY98Sa\nMlj5dLuCQgTtmzhMmx6AgxX2xuFpnZilPTLIksgkNoeYUFiYJYWBs7TGDyH0yV//\nrKHgHH4f7X7N6VRDOIvlOMmtYoVyAt55fF9wk8vPKqzGiyV9yihROM/f3ITDPVe6\nNMi7BMNV7aGaD/T060H/Oq6ULmaBwiQ3kF0E9v5PF4DiBlhNafJauCfAfQAtOpUq\ni+8CnxAvPKEEGPac1QKcpdAcS9TdLozaKH9zIqn8H2ERt35c0MxIjElp/+QfIrWa\nMkEsQYucXGAC7IYyK/MfxJl8QwpMGdTl5VG3ZO8LOO1HREK6c7l4ym7sP9/DeLGI\nN6JpFMZZ8LYTr+tB8XqIl2cPbfPE9gtIVND84ZvGkVCmsfKU9tKLJgvKlqayVZeW\nUVBdaDjnJZvi2lf6oL/6pgK3XEVevNP+mLk/W3O5gmE=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEA7abhyR0uu8lmu38T3FW0GafTyadV0x9XUYX/tIVJ+TqXFlmu\nz8SLu0TSZpIAihzYlzdfP5QHAUB98HShj3qi0LFUw4tKr8MS8ZdfnAaePFkC8CR4\nnsHMvtIFoEzEjPA2YfE7QIl0wqOQgaEB6tQnmUvJ3PIUoZb/oYKxhzvPlBynEWpU\nd5tcK36kTRewv7Atl6WCW0HrtHHIZmdfZAX92ZqL1Ub6aH9ktzYS/+q4eZi8yRi+\ndvvvC6XakbJ+QNP1hCTqzpSrpcjBmWkEcJtdXHbujFKmHu3237Tx3ZaKOO53Gwap\neMnwhOpwI9Cu8juzTkNEpHDszIWZL3E1BLUfoF2TGmEh8oekgKcdFT48IihIsQ9J\n5WbNuncLZrz43DmhkYPjSGGBvVtkfLNl2WLfOaZWqlo+gcaEm3yaU7iGQl2QG4rS\nSIx676GQ/Up6XkhSxAjYH74Icd97Ic2U9KaGG8b7lBWTs1DQ1j37t1acaQlQE7tu\nb2hjpeOFNm1munonKR5j7EwsVhBOvTfQTcPMOebKwemiEH5z++hTvoRJd/yv9ZgO\nTPAc77wt32oKOYT1/iz5X8yh6tZ0D353LQqKpOuEodOFWolZdCw0HqwjYnPLF397\nXnMgb+QScNIzKpV0e8Xkp5jQ5/0iGKFOyyblk9HUnWFqF8p22UaL27/ZLHsCAwEA\nAQKCAgAL+ISAshDJEmzeKUQHAhAnh1h33LnKKoF7pfF1QTjdLieEejGqiIl1/jz6\nQATOmtkJdkISJ0gvtspr4swX40MGmnAdG3aVzOXhjSZFQItsRHA38suv9NjLNzQO\nWN+WWSvmxL/J0UK0N935vEem8ax+9cjYxsmQQYAQks8+HclSehurmRIL0PkUaHV+\nkUtsvxiaKL2cutYpyYjL07guhyNmzc3uXnwS6HEvzgWophpIZw+ssiuMkxsoQzXz\nxyxsjTYIQ9MfYxtppzz37R4f9/YncrGIbZ9YM9S5fFlUjUEwJfQ87xnOmklxwIde\niSP5fVVbrgVr9744fSBJZKfu2/AYSyajxhVBFPRPAKSXtV09f3YdNWFCvayG5uCm\nr2fUZNKkGGHg69G6MTDTzpjiybjbNEsRQc/L5TgeBOUlehiroKqdgLEjr9bVk/2u\nHf1HQarp614GZ5x9f1qq3qUUW4oOBWeERi5005StzSZ2NrGTP/zfsVtgMfSNpcfS\nIVFcfSpKHTlu0yFoObyINSMME/YEXsQz2VMqEEP9c7ZNbrb6vp3BvlMXALlV0SzP\neRY7/7NeilCwKT9MW4EWZCSVsTMsYaAEvRgIXbVieC4GRXWf6JGw9BnB9IB3igKN\nnRefm1DhhQ3YxjVTfV+dQHJAzX3kno1Ve927K9WtOG9Wf2U7SQKCAQEA/z9IRUkP\nKCHrAG3sp6Ol6tnLPLEBjFyniNVO8+YZ45buNVxgwh3p8MyA9RiqAoQwAHY6pDE6\nxXrAJS8B0BOEePn6xnu8Ye0IYjaZH6vQJpsPao92+jikIBylFYFMnTnhmL+0inS6\nC/Bqqf8xEhyn2zraV75CcL25bVRBaFVy51rSYp5bNwWLNCBJZlu7TJmzsXIIVMb4\n55CKg8Ascxr8m02CjFovENi6OUb8L55vEosdJwNR9dCLAZsCGBvhvV+dGYFT8BR9\nW1WwLbvDuEKiCS2MQqWezBhSAro9odvqxUR7vZ/6ovYNX7Hq8/dQEwWK3WSI+ogd\nLQ5qkdbMrjXxKQKCAQEA7lpQlfU01ZlPQP9/SmiLl4KEaNLcaelEm423uXjUC0hB\npU47b+FMVWH4m8cMWt1F3ZZIiGJ5WDpLCTY44Tzaqb9Htg7jqxN8mzBnRNhpHpbr\n8/3yN+w0LNQmQRiAwwUZC3akGaZfW8qwopwXO23o8crar1tsmJFJsAt03EPKnXpw\nY1L/iEr3iuqqpQoHp0hrPQSqoBy7iDOpqhQsXHL11bGtCFsoAbKR72vBVanIuFZp\n1IGhRsZkpE6PIh2ujd3jth2+8fkbSkmY1VgZVcch0/hTTXqCOhuxyuZp/cYaJHHZ\n9gq1aKucljE5rfLk6YFxd+lmutM3TeTXcG8xauqxAwKCAQAfLVahh+P/De2LduHS\nNC3++q1Vc3Vx9JtOHmfvaKq+3t+nyunSrB64F/KkfWe+uPC4Dbjp901rc6vmntBE\n3y1uLXKRLen7uB0vCeY4NR9Op+ABLxbkR2jct0L+1nVKOxiwNLJTuyBCdxJAQ3h1\n8uQZ5h1nXEj3RmVQ0lrO6KkJU1GhVCBmHxo6urrcctX07E1Dnl8gI9GhT0lsAFwt\njx1NxvDsPdPD5gURG/ePxXVhuXZNHp2zPc7M2mcuFLHFFNm2lNJw37/UGSpU27tz\nVUg/gL3CIkK8QTg7Nw5yJ8+MWuoWZmUR5KYLi0rck9fHCwVEN5R3rYqKx1kDCU6k\n9NfJAoIBAQCdRIYri9FCppd+gZaRmA2S+JE5QPgiAcI9CPMSs8DleNWnpsjc90EQ\njUYmMt3aDmoM3R4ary71ofDksRwysZ5tpkJAw+5AcOkk1HkvGcOyZU5mfUaW04lR\nkuFyziIXi4OM5sk8C2ScAk8t2+QQWGL9NyW30pEC0cpXGBgRRcGoyqLhRmBIo35V\n8KSLpOSL4eKwLiYO4yZxpfTqvcjPbyL5zvQmsU7iX9qeqPC1/wLrbkkB6E5rjnfV\nWehjUEZeYOvgvBvDOORizdDD5A9DY9LUM+NzGcRXZvKWuBVzVoTlSyHPkB1GZQU6\nrFwshDm6QKiHwD8CQl8ZTEdCbnui2hvVAoIBAFdlfWrK2bII/Zvgf7Ri1cxIJZe3\n8dIMLl7fr9wzTWMHbfIJFNa2FkUMeDEDpE+GNi109fKAgnYN0yS06gCsTx4RgrKI\nnYxpTYohj+23MbHIisOq+lYf+IQuFRHGjKW3BclzxVGz8YVT2UlyaaFCV4dnkgWZ\niHT8cEfIujsXirGms+pDKnsjdcUzq5SAMtpMKHdnKnnoQ4yLDYgYgDOYjeRL5Goq\nDK4A76JUA0LWDLTMSXznMzqHj2gdW6mqi1diJ2GBZEUnjKRhLqESEJCB0Va+/jHe\nuca5tB7mAjbj8lGRgwfZcLKO93d9lKtUbnIwcPmNQ46r2NEjSw12AfnzkAY=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T06:42:31.391 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:31 smithi114 ceph-mon[30654]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:31.391 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:31 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:31.800 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T06:42:33.197 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T06:42:33.197 INFO:teuthology.orchestra.run.smithi114.stdout:Creating initial admin user... 2024-09-21T06:42:33.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:33 smithi114 ceph-mon[30654]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:33.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:33 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:34.839 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$RM1hikHxJz/S2OHtQQOYWuI86pvNXJUhdGgSACt4f5tKihUIFLjKu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726900954, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T06:42:34.839 INFO:teuthology.orchestra.run.smithi114.stdout:Fetching dashboard port number... 2024-09-21T06:42:35.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:35 smithi114 ceph-mon[30654]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:35.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:35 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:36.193 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T06:42:36.211 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T06:42:36.211 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout disabled 2024-09-21T06:42:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T06:42:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:systemctl: stdout inactive 2024-09-21T06:42:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:firewalld.service is not enabled 2024-09-21T06:42:36.225 INFO:teuthology.orchestra.run.smithi114.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T06:42:36.227 INFO:teuthology.orchestra.run.smithi114.stdout:Ceph Dashboard is now available at: 2024-09-21T06:42:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:36.227 INFO:teuthology.orchestra.run.smithi114.stdout: URL: https://smithi114.front.sepia.ceph.com:8443/ 2024-09-21T06:42:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: User: admin 2024-09-21T06:42:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: Password: vmww76iu2d 2024-09-21T06:42:36.228 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:36.228 INFO:teuthology.orchestra.run.smithi114.stdout:Saving cluster configuration to /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config directory 2024-09-21T06:42:36.486 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2072220506' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T06:42:37.707 INFO:teuthology.orchestra.run.smithi114.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T06:42:37.707 INFO:teuthology.orchestra.run.smithi114.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T06:42:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:42:37.707 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: ceph telemetry on 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout:For more information see: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:37.708 INFO:teuthology.orchestra.run.smithi114.stdout:Bootstrap complete. 2024-09-21T06:42:37.764 INFO:tasks.cephadm:Fetching config... 2024-09-21T06:42:37.764 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:42:37.764 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T06:42:37.800 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T06:42:37.800 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:42:37.800 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T06:42:37.878 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T06:42:37.879 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:42:37.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T06:42:37.974 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T06:42:37.975 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:42:37.975 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T06:42:38.049 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T06:42:38.049 DEBUG:teuthology.orchestra.run.smithi114:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:42:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:42:38.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:42:38.235 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:42:38.247 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T06:42:38.268 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:38 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/909379014' entity='client.admin' 2024-09-21T06:42:38.290 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDA6LMKARQ9Ml4+SXHo/nGMZuKA4hi8AjX4fLfK1WtJPBhvBuufefUJ1yyMwvp3yTwopqecjSoLrITGFmXE31iWjK4xhPJBOr+Q1XPpkZc30gOdJ/Uun21UMoOGUGsAaUwEXIQt8bla6FORtqLhl1whFaJB29ThY7gL7yLCbq216+hXNwondGT+4sP0W95HGejVRxJSMzwWZPOaVi3OPDB3PNwvYiCY/u4WuBMMIllEgc1RJ9Boh3FOXLCkwUjv9gR4kUIrsd0V+7rQ2rLHxORSmYCcjUimong2sNESH2RHXNen7c9HTSSelzOX+zYTM/tCKpODDJV/SXJfm+VK6yD/OCNQQf/zzarvuI1+BTClAcY3RRYkXDtEaMv6yza6Tfstb9XO4tEQjCDLrOQcev8XgwSP8kmZ42RqYpPgZKN76qWo2TuE6FPQ4UxEMkwQAnN53gZoU0MnSOi12CH+tKjPw/oLI0aGdf89at5LKY3IyhccBfDmPXNEOJD88e4IREk= ceph-5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:42:38.302 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T06:42:38.665 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:39.554 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:39 smithi114 ceph-mon[30654]: mgrmap e14: x(active, since 12s) 2024-09-21T06:42:40.276 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T06:42:40.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T06:42:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:40 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1137992618' entity='client.admin' 2024-09-21T06:42:40.828 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:41.839 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:41.839 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:42:41.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:41 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:42:42.608 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-21T06:42:42.608 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:42:42.608 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:42:42.628 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:42:42.628 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:42:42.687 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-21T06:42:42.687 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch host add smithi183 2024-09-21T06:42:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:42 smithi114 ceph-mon[30654]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:42.865 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:42 smithi114 ceph-mon[30654]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:42:42.865 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:42 smithi114 ceph-mon[30654]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:42:42.865 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:42 smithi114 ceph-mon[30654]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:42:43.434 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:43.953 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:43 smithi114 ceph-mon[30654]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:42:43.953 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:43 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:43.953 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:43 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:43.953 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:43 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:45.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:44 smithi114 ceph-mon[30654]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:46.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:45 smithi114 ceph-mon[30654]: Deploying cephadm binary to smithi183 2024-09-21T06:42:46.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:45 smithi114 ceph-mon[30654]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:47.282 INFO:teuthology.orchestra.run.smithi114.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-21T06:42:48.020 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:42:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:48.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:48 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:48.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:48 smithi114 ceph-mon[30654]: Added host smithi183 2024-09-21T06:42:48.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:48 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:48.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:48 smithi114 ceph-mon[30654]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:48.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:48 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:48.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:48 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:49.106 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:49.106 INFO:teuthology.orchestra.run.smithi114.stdout:[{"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-21T06:42:49.687 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:49 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:49.688 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:49 smithi114 ceph-mon[30654]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:42:49.688 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:49 smithi114 ceph-mon[30654]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:49.724 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-21T06:42:49.724 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:42:49.724 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-21T06:42:49.743 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:42:49.743 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:42:49.802 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-21T06:42:49.802 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch host add smithi193 2024-09-21T06:42:50.157 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:52 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:53 smithi114 ceph-mon[30654]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:42:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:53 smithi114 ceph-mon[30654]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:42:53.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:53 smithi114 ceph-mon[30654]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:42:53.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:53 smithi114 ceph-mon[30654]: Deploying cephadm binary to smithi193 2024-09-21T06:42:54.046 INFO:teuthology.orchestra.run.smithi114.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-21T06:42:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:54 smithi114 ceph-mon[30654]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:54 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:54 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:42:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:54 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:54.939 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T06:42:55.296 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:55.589 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:55 smithi114 ceph-mon[30654]: Added host smithi193 2024-09-21T06:42:55.589 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:55 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:56.081 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:42:56.081 INFO:teuthology.orchestra.run.smithi114.stdout:[{"addr": "172.21.15.114", "hostname": "smithi114", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-21T06:42:56.710 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:56 smithi114 ceph-mon[30654]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:56.710 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:56 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:56.749 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T06:42:56.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T06:42:57.119 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:42:57.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:57 smithi114 ceph-mon[30654]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:42:58.373 INFO:teuthology.orchestra.run.smithi114.stderr:adjusted tunables profile to default 2024-09-21T06:42:58.699 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2201899389' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:42:58.700 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:58 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:42:59.063 INFO:tasks.cephadm:Adding mon.a on smithi114 2024-09-21T06:42:59.063 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-09-21T06:42:59.064 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-09-21T06:42:59.064 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi114:172.21.15.114=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c' 2024-09-21T06:42:59.298 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:42:59.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2201899389' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:42:59 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:42:59.929 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-09-21T06:43:00.496 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T06:43:00.498 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T06:43:00.500 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:43:00.500 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:43:00.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:00 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:01 smithi114 ceph-mon[30654]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi114:172.21.15.114=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:43:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:01 smithi114 ceph-mon[30654]: Saving service mon spec with placement smithi114:172.21.15.114=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-21T06:43:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:01 smithi114 ceph-mon[30654]: Deploying daemon mon.c on smithi193 2024-09-21T06:43:02.054 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:02.707 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:43:02.707 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","modified":"2024-09-21T06:41:24.747593Z","created":"2024-09-21T06:41:24.747593Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T06:43:02.707 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-21T06:43:02.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:02 smithi114 ceph-mon[30654]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:03 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/2546625905' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:04.233 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 podman[34582]: 2024-09-21 06:43:04.134900436 +0000 UTC m=+0.015739630 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:43:04.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 podman[34582]: 2024-09-21 06:43:04.430270365 +0000 UTC m=+0.311109558 container init 5930fe7c63e7ff4829d3af8df98e1b76d0e57c11641eea8cec5cf0b7f6195e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-21T06:43:04.501 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 podman[34582]: 2024-09-21 06:43:04.433446365 +0000 UTC m=+0.314285560 container start 5930fe7c63e7ff4829d3af8df98e1b76d0e57c11641eea8cec5cf0b7f6195e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pidfile_write: ignore empty --pid-file 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: load: jerasure load: lrc 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: RocksDB version: 7.9.2 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Git sha 0 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: DB SUMMARY 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: DB Session ID: L0ZYCDWCQ5E3LATQMERD 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: CURRENT file: CURRENT 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T06:43:04.502 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.error_if_exists: 0 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.create_if_missing: 0 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.paranoid_checks: 1 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.env: 0x55ab131c2d60 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.info_log: 0x55ab184dd360 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T06:43:04.503 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.statistics: (nil) 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.use_fsync: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_log_file_size: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.allow_fallocate: 1 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.use_direct_reads: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.db_log_dir: 2024-09-21T06:43:04.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.wal_dir: 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.write_buffer_manager: 0x55ab184e1d60 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T06:43:04.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.unordered_write: 0 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.row_cache: None 2024-09-21T06:43:04.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.wal_filter: None 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.two_write_queues: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.wal_compression: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.atomic_flush: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.log_readahead_size: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T06:43:04.508 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_background_jobs: 2 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_background_compactions: -1 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_subcompactions: 1 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T06:43:04.509 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_open_files: -1 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_background_flushes: -1 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Compression algorithms supported: 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kZSTD supported: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kXpressCompression supported: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kBZip2Compression supported: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kLZ4Compression supported: 1 2024-09-21T06:43:04.510 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kZlibCompression supported: 1 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: kSnappyCompression supported: 1 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.merge_operator: 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_filter: None 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T06:43:04.511 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ab184dcfa0) 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x55ab144a1610 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-09-21T06:43:04.512 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-09-21T06:43:04.513 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression: NoCompression 2024-09-21T06:43:04.514 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.num_levels: 7 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:43:04.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T06:43:04.516 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T06:43:04.517 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T06:43:04.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.inplace_update_support: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.bloom_locality: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.max_successive_merges: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.ttl: 2592000 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T06:43:04.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.enable_blob_files: false 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.min_blob_size: 0 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 149abae9-7ca6-4568-806d-95f7b683cbbf 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726900984470979, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T06:43:04.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726900984471402, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726900984, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "149abae9-7ca6-4568-806d-95f7b683cbbf", "db_session_id": "L0ZYCDWCQ5E3LATQMERD", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726900984471496, "job": 1, "event": "recovery_finished"} 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ab18500a80 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: DB pointer 0x55ab192a6000 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T06:43:04.521 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-21T06:43:04.522 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-21T06:43:04.523 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T06:43:04.523 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-21T06:43:04.523 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T06:43:04.523 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T06:43:04.523 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T06:43:04.523 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T06:43:04.525 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T06:43:04.525 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.34 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x55ab144a1610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(???) e0 preinit fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T06:43:04.526 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout: btime 2024-09-21T06:41:28:662644+0000 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T06:43:04.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Deploying cephadm binary to smithi183 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Added host smithi183 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:43:04.528 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.529 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Deploying cephadm binary to smithi193 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Added host smithi193 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.530 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2201899389' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2201899389' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.531 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi114:172.21.15.114=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Saving service mon spec with placement smithi114:172.21.15.114=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: Deploying daemon mon.c on smithi193 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.532 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/2546625905' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:04.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.533 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 ceph-mon[34601]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T06:43:04.656 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:43:04.656 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:43:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:04 smithi114 ceph-mon[30654]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:04.908 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:43:04.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 bash[34582]: 5930fe7c63e7ff4829d3af8df98e1b76d0e57c11641eea8cec5cf0b7f6195e0e 2024-09-21T06:43:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:04 smithi193 systemd[1]: Started Ceph mon.c for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:43:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: mon.a calling monitor election 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: mon.c calling monitor election 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:43:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: monmap epoch 2 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: last_changed 2024-09-21T06:43:04.506968+0000 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: min_mon_release 19 (squid) 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: election_strategy: 1 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: fsmap 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: mgrmap e14: x(active, since 44s) 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: overall HEALTH_OK 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:09.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:09.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:09.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:09.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:09 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:10.012 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:43:10.013 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","modified":"2024-09-21T06:43:04.506968Z","created":"2024-09-21T06:41:24.747593Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:43:10.013 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: mon.a calling monitor election 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:10.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: mon.c calling monitor election 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: monmap epoch 2 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: last_changed 2024-09-21T06:43:04.506968+0000 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: min_mon_release 19 (squid) 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: election_strategy: 1 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:10.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: fsmap 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: mgrmap e14: x(active, since 44s) 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: overall HEALTH_OK 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:10.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:09 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:10.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:10 smithi193 ceph-mon[34601]: Deploying daemon mon.b on smithi183 2024-09-21T06:43:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:10 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/3704590747' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:10 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:11.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:10 smithi114 ceph-mon[30654]: Deploying daemon mon.b on smithi183 2024-09-21T06:43:11.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:10 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/3704590747' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:11.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:10 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:11.598 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:43:11.599 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:43:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:11 smithi114 ceph-mon[30654]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:11.816 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:43:11.505+0000 7f5b98a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T06:43:11.848 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:43:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:11 smithi193 ceph-mon[34601]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:12.554 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:43:12.554 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","modified":"2024-09-21T06:43:04.506968Z","created":"2024-09-21T06:41:24.747593Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:43:12.554 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-21T06:43:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:12 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/3874037032' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:13.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:12 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/3874037032' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:13.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:13 smithi183 systemd[1]: Started Ceph mon.b for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:43:14.139 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T06:43:14.139 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T06:43:14.391 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: mon.c calling monitor election 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: mon.a calling monitor election 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: monmap epoch 3 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: last_changed 2024-09-21T06:43:13.243849+0000 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: min_mon_release 19 (squid) 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: election_strategy: 1 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: fsmap 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:18.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: mgrmap e14: x(active, since 52s) 2024-09-21T06:43:18.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: overall HEALTH_OK 2024-09-21T06:43:18.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:18 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: mon.c calling monitor election 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: mon.a calling monitor election 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:18.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: monmap epoch 3 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: last_changed 2024-09-21T06:43:13.243849+0000 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: min_mon_release 19 (squid) 2024-09-21T06:43:18.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: election_strategy: 1 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: fsmap 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: mgrmap e14: x(active, since 52s) 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: overall HEALTH_OK 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.664 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:18 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:18.680 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:43:18.681 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","modified":"2024-09-21T06:43:13.243849Z","created":"2024-09-21T06:41:24.747593Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:3300","nonce":0},{"type":"v1","addr":"172.21.15.114:6789","nonce":0}]},"addr":"172.21.15.114:6789/0","public_addr":"172.21.15.114:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T06:43:18.681 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-09-21T06:43:19.285 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T06:43:19.286 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T06:43:19.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:19 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:19.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:19 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/512585729' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:19.628 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:19 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:19.658 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:19.790 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:19 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:19.790 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:19 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/512585729' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:19.791 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:19 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.474 INFO:teuthology.orchestra.run.smithi114.stdout:# minimal ceph.conf for 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:20.474 INFO:teuthology.orchestra.run.smithi114.stdout:[global] 2024-09-21T06:43:20.474 INFO:teuthology.orchestra.run.smithi114.stdout: fsid = 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:20.474 INFO:teuthology.orchestra.run.smithi114.stdout: mon_host = [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: mon.c calling monitor election 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: mon.a calling monitor election 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: monmap epoch 3 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: last_changed 2024-09-21T06:43:13.243849+0000 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: min_mon_release 19 (squid) 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: election_strategy: 1 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: fsmap 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: mgrmap e14: x(active, since 52s) 2024-09-21T06:43:20.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: overall HEALTH_OK 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/512585729' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T06:43:20.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:20 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:21.219 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T06:43:21.219 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:43:21.220 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:43:21.277 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:43:21.277 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:21.374 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:43:21.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:43:21.407 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:43:21.407 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:21.472 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: mon.b calling monitor election 2024-09-21T06:43:21.473 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:43:21.473 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:21.473 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:21.473 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.473 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.473 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: mon.b calling monitor election 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: mon.c calling monitor election 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: mon.a calling monitor election 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: monmap epoch 3 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: last_changed 2024-09-21T06:43:13.243849+0000 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: min_mon_release 19 (squid) 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: election_strategy: 1 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: fsmap 2024-09-21T06:43:21.474 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: mgrmap e14: x(active, since 54s) 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: overall HEALTH_OK 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/23392018' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:21.475 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:21 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.476 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:43:21.476 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T06:43:21.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: mon.b calling monitor election 2024-09-21T06:43:21.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:43:21.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:21.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:21.504 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: mon.b calling monitor election 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: mon.c calling monitor election 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: mon.a calling monitor election 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: monmap epoch 3 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: last_changed 2024-09-21T06:43:13.243849+0000 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: min_mon_release 19 (squid) 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: election_strategy: 1 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:21.505 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: fsmap 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: mgrmap e14: x(active, since 54s) 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: overall HEALTH_OK 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/23392018' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:21.506 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:21.507 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:21.507 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:21 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.508 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:43:21.508 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:21.511 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: mon.b calling monitor election 2024-09-21T06:43:21.511 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:43:21.511 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:21.511 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:21.511 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: mon.b calling monitor election 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: mon.c calling monitor election 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: mon.a calling monitor election 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: monmap epoch 3 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: last_changed 2024-09-21T06:43:13.243849+0000 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: created 2024-09-21T06:41:24.747593+0000 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: min_mon_release 19 (squid) 2024-09-21T06:43:21.512 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: election_strategy: 1 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: 0: [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] mon.a 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: fsmap 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: mgrmap e14: x(active, since 54s) 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: overall HEALTH_OK 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/23392018' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.513 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.514 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.514 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.514 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:21.514 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:21.514 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:21.514 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:21 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:21.576 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T06:43:21.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T06:43:22.246 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:22 smithi114 ceph-mon[30654]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:43:22.247 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:22 smithi114 ceph-mon[30654]: Reconfiguring daemon mon.a on smithi114 2024-09-21T06:43:22.247 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:22 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:22.247 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:22 smithi114 ceph-mon[30654]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:22.352 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:22.537 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:22 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:43:22.242+0000 7f5b98a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T06:43:22.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:22 smithi183 ceph-mon[34219]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:43:22.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:22 smithi183 ceph-mon[34219]: Reconfiguring daemon mon.a on smithi114 2024-09-21T06:43:22.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:22 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:22.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:22 smithi183 ceph-mon[34219]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:22 smithi193 ceph-mon[34601]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T06:43:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:22 smithi193 ceph-mon[34601]: Reconfiguring daemon mon.a on smithi114 2024-09-21T06:43:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:22 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:22 smithi193 ceph-mon[34601]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: Reconfiguring daemon mon.b on smithi183 2024-09-21T06:43:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:23 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1243392313' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:43:23.684 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:23.684 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:23.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:43:23.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:23.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:23.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:23.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: Reconfiguring daemon mon.b on smithi183 2024-09-21T06:43:23.685 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:23 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1243392313' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: from='mgr.14152 172.21.15.114:0/1696609365' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: Reconfiguring daemon mon.b on smithi183 2024-09-21T06:43:23.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1243392313' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T06:43:23.803 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ignoring --setuser ceph since I am not root 2024-09-21T06:43:23.803 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ignoring --setgroup ceph since I am not root 2024-09-21T06:43:24.149 INFO:tasks.cephadm:Adding mgr.x on smithi114 2024-09-21T06:43:24.150 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-09-21T06:43:24.150 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-09-21T06:43:24.150 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi114=x;smithi183=y;smithi193=z' 2024-09-21T06:43:24.394 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:43:24.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:24 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1243392313' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:43:24.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:24 smithi193 ceph-mon[34601]: mgrmap e15: x(active, since 58s) 2024-09-21T06:43:24.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:24 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1243392313' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:43:24.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:24 smithi114 ceph-mon[30654]: mgrmap e15: x(active, since 58s) 2024-09-21T06:43:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:24 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1243392313' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T06:43:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:24 smithi183 ceph-mon[34219]: mgrmap e15: x(active, since 58s) 2024-09-21T06:43:29.801 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: Active manager daemon x restarted 2024-09-21T06:43:29.801 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: Activating manager daemon x 2024-09-21T06:43:29.801 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:43:29.801 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: mgrmap e16: x(active, starting, since 0.00607975s) 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:43:29.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:29 smithi114 ceph-mon[30654]: Manager daemon x is now available 2024-09-21T06:43:29.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: Active manager daemon x restarted 2024-09-21T06:43:29.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: Activating manager daemon x 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: mgrmap e16: x(active, starting, since 0.00607975s) 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:43:29.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:29 smithi183 ceph-mon[34219]: Manager daemon x is now available 2024-09-21T06:43:29.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: Active manager daemon x restarted 2024-09-21T06:43:29.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: Activating manager daemon x 2024-09-21T06:43:29.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T06:43:29.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: mgrmap e16: x(active, starting, since 0.00607975s) 2024-09-21T06:43:29.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:43:29.874 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:43:29.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:43:29.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T06:43:29.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T06:43:29.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T06:43:29.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T06:43:29.875 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:29 smithi193 ceph-mon[34601]: Manager daemon x is now available 2024-09-21T06:43:30.411 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-09-21T06:43:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:43:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:43:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:30 smithi193 ceph-mon[34601]: mgrmap e17: x(active, since 1.01195s) 2024-09-21T06:43:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:30.741 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:30.741 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:43:30.741 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:43:30.741 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-mon[30654]: mgrmap e17: x(active, since 1.01195s) 2024-09-21T06:43:30.741 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:30.769 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:30.769 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T06:43:30.769 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T06:43:30.769 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:30 smithi183 ceph-mon[34219]: mgrmap e17: x(active, since 1.01195s) 2024-09-21T06:43:30.769 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:43:30.851+0000 7f54bae00640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:06:43:30] ENGINE Error in HTTPServer.serve 2024-09-21T06:43:31.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Traceback (most recent call last): 2024-09-21T06:43:31.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._connections.run(self.expiration_interval) 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._run(expiration_interval) 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: s = self.context.wrap_socket( 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: return self.sslsocket_class._create( 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self.do_handshake() 2024-09-21T06:43:31.067 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:43:31.068 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self._sslobj.do_handshake() 2024-09-21T06:43:31.068 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:43:31.068 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:43:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: Saving service mgr spec with placement smithi114=x;smithi183=y;smithi193=z;count:3 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.654 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.680 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T06:43:31.682 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T06:43:31.684 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T06:43:31.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T06:43:31.689 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T06:43:31.690 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T06:43:31.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: Saving service mgr spec with placement smithi114=x;smithi183=y;smithi193=z;count:3 2024-09-21T06:43:31.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.703 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.733 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T06:43:31.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T06:43:31.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: Saving service mgr spec with placement smithi114=x;smithi183=y;smithi193=z;count:3 2024-09-21T06:43:31.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.736 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.736 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.736 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:31.738 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T06:43:31.738 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T06:43:31.785 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T06:43:31.785 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T06:43:31.832 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T06:43:31.832 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T06:43:31.879 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T06:43:31.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T06:43:31.934 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T06:43:31.934 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T06:43:31.982 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T06:43:31.982 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T06:43:32.036 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T06:43:32.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T06:43:32.088 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T06:43:32.088 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T06:43:32.139 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T06:43:32.139 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T06:43:32.215 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T06:43:32.215 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T06:43:32.275 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T06:43:32.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T06:43:32.329 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T06:43:32.329 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T06:43:32.384 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T06:43:32.384 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T06:43:32.441 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T06:43:32.441 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T06:43:32.455 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:32.478 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T06:43:32.478 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T06:43:32.541 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T06:43:32.542 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T06:43:32.597 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T06:43:32.597 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T06:43:32.665 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T06:43:32.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T06:43:32.721 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T06:43:32.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T06:43:32.743 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:32.760 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T06:43:32.760 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T06:43:32.789 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:32.813 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T06:43:32.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: [21/Sep/2024:06:43:30] ENGINE Bus STARTING 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: [21/Sep/2024:06:43:30] ENGINE Serving on http://172.21.15.114:8765 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: [21/Sep/2024:06:43:30] ENGINE Serving on https://172.21.15.114:7150 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: [21/Sep/2024:06:43:30] ENGINE Bus STARTED 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: [21/Sep/2024:06:43:30] ENGINE Error in HTTPServer.serve 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: Traceback (most recent call last): 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._run(expiration_interval) 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:43:32.819 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: s = self.context.wrap_socket( 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: return self.sslsocket_class._create( 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: self.do_handshake() 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: self._sslobj.do_handshake() 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:32.820 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:32 smithi114 ceph-mon[30654]: mgrmap e18: x(active, since 2s) 2024-09-21T06:43:32.836 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T06:43:32.836 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T06:43:32.837 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: [21/Sep/2024:06:43:30] ENGINE Bus STARTING 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: [21/Sep/2024:06:43:30] ENGINE Serving on http://172.21.15.114:8765 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: [21/Sep/2024:06:43:30] ENGINE Serving on https://172.21.15.114:7150 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: [21/Sep/2024:06:43:30] ENGINE Bus STARTED 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: [21/Sep/2024:06:43:30] ENGINE Error in HTTPServer.serve 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: Traceback (most recent call last): 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: self._run(expiration_interval) 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:43:32.865 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: s = self.context.wrap_socket( 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: return self.sslsocket_class._create( 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: self.do_handshake() 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: self._sslobj.do_handshake() 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:32.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:32 smithi183 ceph-mon[34219]: mgrmap e18: x(active, since 2s) 2024-09-21T06:43:32.889 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T06:43:32.889 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: [21/Sep/2024:06:43:30] ENGINE Bus STARTING 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: [21/Sep/2024:06:43:30] ENGINE Serving on http://172.21.15.114:8765 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: [21/Sep/2024:06:43:30] ENGINE Serving on https://172.21.15.114:7150 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: [21/Sep/2024:06:43:30] ENGINE Bus STARTED 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: [21/Sep/2024:06:43:30] ENGINE Error in HTTPServer.serve 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout: Traceback (most recent call last): 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T06:43:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._run(expiration_interval) 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: s = self.context.wrap_socket( 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: return self.sslsocket_class._create( 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: self.do_handshake() 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: self._sslobj.do_handshake() 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T06:43:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:32 smithi193 ceph-mon[34601]: mgrmap e18: x(active, since 2s) 2024-09-21T06:43:32.941 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T06:43:32.941 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T06:43:32.993 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T06:43:32.993 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T06:43:33.063 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T06:43:33.063 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T06:43:33.118 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T06:43:33.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T06:43:33.182 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T06:43:33.182 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi114') 2024-09-21T06:43:33.761 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:35.358 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2713496722' entity='client.admin' 2024-09-21T06:43:35.358 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.358 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.359 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2713496722' entity='client.admin' 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2713496722' entity='client.admin' 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi114", "name": "osd_memory_target"} : dispatch 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: Updating smithi114:/etc/ceph/ceph.conf 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:35.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:35.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 -- 172.21.15.114:0/679377109 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3ee403fb70 msgr2=0x7f3ee4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 --2- 172.21.15.114:0/679377109 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3ee403fb70 0x7f3ee4042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3efc108f20 tx=0x7f3ee8005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 -- 172.21.15.114:0/679377109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc1039b0 msgr2=0x7f3efc19c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 --2- 172.21.15.114:0/679377109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc1039b0 0x7f3efc19c360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0035040 tx=0x7f3ef0033f70 comp rx=0 tx=0).stop 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 -- 172.21.15.114:0/679377109 shutdown_connections 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 --2- 172.21.15.114:0/679377109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3efc109c40 0x7f3efc1a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 --2- 172.21.15.114:0/679377109 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3efc1092f0 0x7f3efc19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 --2- 172.21.15.114:0/679377109 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3ee403fb70 0x7f3ee4042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 --2- 172.21.15.114:0/679377109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3efc1039b0 0x7f3efc19c360 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.072+0000 7f3f02400640 1 -- 172.21.15.114:0/679377109 >> 172.21.15.114:0/679377109 conn(0x7f3efc0ff440 msgr2=0x7f3efc10e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.073+0000 7f3f02400640 1 -- 172.21.15.114:0/679377109 shutdown_connections 2024-09-21T06:43:36.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:36.073+0000 7f3f02400640 1 -- 172.21.15.114:0/679377109 wait complete. 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:43:36.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:36.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:36.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: Deploying daemon mgr.z on smithi193 2024-09-21T06:43:36.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1923640195' entity='client.admin' 2024-09-21T06:43:36.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:36 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.860 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: Deploying daemon mgr.z on smithi193 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1923640195' entity='client.admin' 2024-09-21T06:43:36.861 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:36 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi114:/etc/ceph/ceph.client.admin.keyring 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Updating smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: Deploying daemon mgr.z on smithi193 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1923640195' entity='client.admin' 2024-09-21T06:43:36.867 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:36 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:37.412 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 systemd[1]: Starting Ceph mgr.z for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:43:37.754 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 podman[36443]: 2024-09-21 06:43:37.398775507 +0000 UTC m=+0.015810819 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:43:37.754 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 podman[36443]: 2024-09-21 06:43:37.513323693 +0000 UTC m=+0.130359044 container create f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T06:43:37.754 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 podman[36443]: 2024-09-21 06:43:37.715403734 +0000 UTC m=+0.332439039 container init f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:43:37.755 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 podman[36443]: 2024-09-21 06:43:37.71836663 +0000 UTC m=+0.335401937 container start f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:43:37.755 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 ceph-mgr[36462]: -- 172.21.15.193:0/1303516550 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b026a4b0e0 con 0x55b026a69a80 2024-09-21T06:43:37.994 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi114') 2024-09-21T06:43:37.995 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi114') 2024-09-21T06:43:38.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 -- 172.21.15.114:0/1634706394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0106a10 msgr2=0x7f5ef0106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:38.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 --2- 172.21.15.114:0/1634706394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0106a10 0x7f5ef0106e20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009900 tx=0x7f5ed802a480 comp rx=0 tx=0).stop 2024-09-21T06:43:38.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 -- 172.21.15.114:0/1634706394 shutdown_connections 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 --2- 172.21.15.114:0/1634706394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ef01073b0 0x7f5ef010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 --2- 172.21.15.114:0/1634706394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0106a10 0x7f5ef0106e20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 --2- 172.21.15.114:0/1634706394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef0105980 0x7f5ef0105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 -- 172.21.15.114:0/1634706394 >> 172.21.15.114:0/1634706394 conn(0x7f5ef01012f0 msgr2=0x7f5ef01037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 -- 172.21.15.114:0/1634706394 shutdown_connections 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.812+0000 7f5ef7200640 1 -- 172.21.15.114:0/1634706394 wait complete. 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 Processor -- start 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 -- start start 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 0x7f5ef019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ef0106a10 0x7f5ef019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:38.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef01073b0 0x7f5ef019d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef01114e0 con 0x7f5ef0105980 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111360 con 0x7f5ef0106a10 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.813+0000 7f5ef7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0111660 con 0x7f5ef01073b0 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 0x7f5ef019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 0x7f5ef019c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49770/0 (socket says 172.21.15.114:49770) 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 -- 172.21.15.114:0/2193526323 learned_addr learned my addr 172.21.15.114:0/2193526323 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5ef5200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef01073b0 0x7f5ef019d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5ee7400640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ef0106a10 0x7f5ef019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 -- 172.21.15.114:0/2193526323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef01073b0 msgr2=0x7f5ef019d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef01073b0 0x7f5ef019d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 -- 172.21.15.114:0/2193526323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ef0106a10 msgr2=0x7f5ef019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ef0106a10 0x7f5ef019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 -- 172.21.15.114:0/2193526323 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef019db40 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5ef5200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef01073b0 0x7f5ef019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.814+0000 7f5eefe00640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 0x7f5ef019c110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e9f0 tx=0x7f5ee000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.815+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000cd80 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.815+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ee00054b0 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.815+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef019de30 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.815+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0010430 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.815+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a4530 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.816+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef010cb60 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.816+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7f5ee0005020 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.817+0000 7f5eed600640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5ed403fb70 0x7f5ed4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.817+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5ee0054b10 con 0x7f5ef0105980 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.817+0000 7f5ee7400640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5ed403fb70 0x7f5ed4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:38.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.818+0000 7f5ee7400640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5ed403fb70 0x7f5ed4042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000c00 tx=0x7f5ed8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:37.822+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ee001fe10 con 0x7f5ef0105980 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.021+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5ef002e640 con 0x7f5ef0105980 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.025+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f5ee0021440 con 0x7f5ef0105980 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.026+0000 7f5eed600640 1 -- 172.21.15.114:0/2193526323 <== mon.0 v2:172.21.15.114:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee000fca0 con 0x7f5ef0105980 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5ed403fb70 msgr2=0x7f5ed4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5ed403fb70 0x7f5ed4042080 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000c00 tx=0x7f5ed8031400 comp rx=0 tx=0).stop 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 msgr2=0x7f5ef019c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 0x7f5ef019c110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e9f0 tx=0x7f5ee000edf0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 shutdown_connections 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ef01073b0 0x7f5ef019d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5ed403fb70 0x7f5ed4042080 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ef0106a10 0x7f5ef019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 --2- 172.21.15.114:0/2193526323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5ef0105980 0x7f5ef019c110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:38.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 >> 172.21.15.114:0/2193526323 conn(0x7f5ef01012f0 msgr2=0x7f5ef0109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:38.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 shutdown_connections 2024-09-21T06:43:38.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:38.028+0000 7f5ef7200640 1 -- 172.21.15.114:0/2193526323 wait complete. 2024-09-21T06:43:38.162 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 bash[36443]: f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 2024-09-21T06:43:38.162 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 06:43:37 smithi193 systemd[1]: Started Ceph mgr.z for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:43:38.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi114') 2024-09-21T06:43:38.762 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi114') 2024-09-21T06:43:38.978 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: Deploying daemon mgr.y on smithi183 2024-09-21T06:43:38.979 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:38 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2193526323' entity='client.admin' 2024-09-21T06:43:38.980 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi114') 2024-09-21T06:43:39.052 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:39.116 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:39.116 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:43:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: Deploying daemon mgr.y on smithi183 2024-09-21T06:43:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:38 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2193526323' entity='client.admin' 2024-09-21T06:43:39.118 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.141 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.141 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:39.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:39.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:39.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:39.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T06:43:39.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:39.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:39.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: Deploying daemon mgr.y on smithi183 2024-09-21T06:43:39.165 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:38 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2193526323' entity='client.admin' 2024-09-21T06:43:39.229 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.765 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:39 smithi183 systemd[1]: Starting Ceph mgr.y for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:43:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:39.900 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:40.047 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:39 smithi183 podman[35723]: 2024-09-21 06:43:39.66616061 +0000 UTC m=+0.015869427 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:43:40.047 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:39 smithi183 podman[35723]: 2024-09-21 06:43:39.768724276 +0000 UTC m=+0.118433156 container create 09d6116226142296e4d5eda55146a25ff22081e64b1b3e0799c14e771d426a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:43:40.047 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:39 smithi183 podman[35723]: 2024-09-21 06:43:39.962542823 +0000 UTC m=+0.312251637 container init 09d6116226142296e4d5eda55146a25ff22081e64b1b3e0799c14e771d426a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:43:40.047 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:39 smithi183 podman[35723]: 2024-09-21 06:43:39.965575364 +0000 UTC m=+0.315284185 container start 09d6116226142296e4d5eda55146a25ff22081e64b1b3e0799c14e771d426a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T06:43:40.048 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:40 smithi183 ceph-mgr[35742]: -- 172.21.15.183:0/2756976011 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ddd41e30e0 con 0x55ddd4201a80 2024-09-21T06:43:40.048 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:40 smithi183 bash[35723]: 09d6116226142296e4d5eda55146a25ff22081e64b1b3e0799c14e771d426a5e 2024-09-21T06:43:40.333 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 06:43:40 smithi183 systemd[1]: Started Ceph mgr.y for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 -- 172.21.15.114:0/1942689731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 msgr2=0x7fd0fc10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 --2- 172.21.15.114:0/1942689731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc10bad0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009900 tx=0x7fd0e802a410 comp rx=0 tx=0).stop 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 -- 172.21.15.114:0/1942689731 shutdown_connections 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 --2- 172.21.15.114:0/1942689731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc10c010 0x7fd0fc112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 --2- 172.21.15.114:0/1942689731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc10bad0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 --2- 172.21.15.114:0/1942689731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 -- 172.21.15.114:0/1942689731 >> 172.21.15.114:0/1942689731 conn(0x7fd0fc101810 msgr2=0x7fd0fc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:40.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 -- 172.21.15.114:0/1942689731 shutdown_connections 2024-09-21T06:43:40.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.532+0000 7fd104600640 1 -- 172.21.15.114:0/1942689731 wait complete. 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.533+0000 7fd104600640 1 Processor -- start 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.533+0000 7fd104600640 1 -- start start 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd104600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd104600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd104600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc10c010 0x7fd0fc1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd104600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc115cd0 con 0x7fd0fc10b6c0 2024-09-21T06:43:40.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd104600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc115b50 con 0x7fd0fc10c010 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd104600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc115e50 con 0x7fd0fc105d80 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47486/0 (socket says 172.21.15.114:47486) 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 -- 172.21.15.114:0/3304468211 learned_addr learned my addr 172.21.15.114:0/3304468211 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd102600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc10c010 0x7fd0fc1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 -- 172.21.15.114:0/3304468211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc10c010 msgr2=0x7fd0fc1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc10c010 0x7fd0fc1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 -- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 msgr2=0x7fd0fc19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101c00640 1 -- 172.21.15.114:0/3304468211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc1a2630 con 0x7fd0fc105d80 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.534+0000 7fd101200640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:40.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.535+0000 7fd101c00640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc19c6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000e9f0 tx=0x7fd0f000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:40.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.535+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000cd80 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.535+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc1a2920 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.535+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f00054d0 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.535+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0010690 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.535+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc1a9080 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.536+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc106190 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.536+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7fd0f0004e90 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.537+0000 7fd0fa000640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e403fb70 0x7fd0e4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.537+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd0f0054bc0 con 0x7fd0fc105d80 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.537+0000 7fd101200640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e403fb70 0x7fd0e4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:40.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.538+0000 7fd101200640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e403fb70 0x7fd0e4042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009230 tx=0x7fd0e8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:40.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.543+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd0f0013190 con 0x7fd0fc105d80 2024-09-21T06:43:40.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.752+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fd0fc02e640 con 0x7fd0fc105d80 2024-09-21T06:43:40.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.759+0000 7fd0fa000640 1 -- 172.21.15.114:0/3304468211 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v15)=0 v15) ==== 119+0+0 (secure 0 0 0) 0x7fd0f0011dc0 con 0x7fd0fc105d80 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e403fb70 msgr2=0x7fd0e4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e403fb70 0x7fd0e4042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009230 tx=0x7fd0e8031400 comp rx=0 tx=0).stop 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 msgr2=0x7fd0fc19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc19c6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000e9f0 tx=0x7fd0f000edf0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 shutdown_connections 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e403fb70 0x7fd0e4042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc10c010 0x7fd0fc1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd0fc10b6c0 0x7fd0fc19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 --2- 172.21.15.114:0/3304468211 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0fc105d80 0x7fd0fc19c6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 >> 172.21.15.114:0/3304468211 conn(0x7fd0fc101810 msgr2=0x7fd0fc103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:40.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 shutdown_connections 2024-09-21T06:43:40.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:40.762+0000 7fd104600640 1 -- 172.21.15.114:0/3304468211 wait complete. 2024-09-21T06:43:41.264 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.264 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.274 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.279 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.283 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:41.288 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:41 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:41.292 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.304 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.309 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.313 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.320 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.320 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.320 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.332 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:41.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 -- 172.21.15.114:0/1860373771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc10c020 msgr2=0x7f42fc112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 --2- 172.21.15.114:0/1860373771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc10c020 0x7f42fc112850 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009900 tx=0x7f42f402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 -- 172.21.15.114:0/1860373771 shutdown_connections 2024-09-21T06:43:41.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 --2- 172.21.15.114:0/1860373771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc10c020 0x7f42fc112850 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 --2- 172.21.15.114:0/1860373771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42fc10b6d0 0x7f42fc10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 --2- 172.21.15.114:0/1860373771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc105d90 0x7f42fc1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 -- 172.21.15.114:0/1860373771 >> 172.21.15.114:0/1860373771 conn(0x7f42fc101860 msgr2=0x7f42fc103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 -- 172.21.15.114:0/1860373771 shutdown_connections 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.033+0000 7f4303600640 1 -- 172.21.15.114:0/1860373771 wait complete. 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.034+0000 7f4303600640 1 Processor -- start 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.034+0000 7f4303600640 1 -- start start 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4303600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 0x7f42fc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4303600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc10b6d0 0x7f42fc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4303600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42fc10c020 0x7f42fc1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4303600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc115d30 con 0x7f42fc105d90 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4303600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc115bb0 con 0x7f42fc10c020 2024-09-21T06:43:41.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4303600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42fc115eb0 con 0x7f42fc10b6d0 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4300c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 0x7f42fc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4300c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 0x7f42fc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49816/0 (socket says 172.21.15.114:49816) 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4300c00640 1 -- 172.21.15.114:0/2326993409 learned_addr learned my addr 172.21.15.114:0/2326993409 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f4301600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42fc10c020 0x7f42fc1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.035+0000 7f42fbe00640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc10b6d0 0x7f42fc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4300c00640 1 -- 172.21.15.114:0/2326993409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc10b6d0 msgr2=0x7f42fc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4300c00640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc10b6d0 0x7f42fc19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4300c00640 1 -- 172.21.15.114:0/2326993409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42fc10c020 msgr2=0x7f42fc1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4300c00640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42fc10c020 0x7f42fc1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4300c00640 1 -- 172.21.15.114:0/2326993409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42fc1a25b0 con 0x7f42fc105d90 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4300c00640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 0x7f42fc19c640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42f000db80 tx=0x7f42f000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:41.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0017070 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f0003b70 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42fc1a28a0 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f00106c0 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.036+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42fc1a9000 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.037+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42fc1061a0 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.038+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7f42f0004020 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.038+0000 7f42f9600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42e403fb70 0x7f42e4042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.038+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f42f00546b0 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.039+0000 7f42fbe00640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42e403fb70 0x7f42e4042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.039+0000 7f42fbe00640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42e403fb70 0x7f42e4042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42fc1050b0 tx=0x7f42e800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.043+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f42f0018270 con 0x7f42fc105d90 2024-09-21T06:43:41.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.237+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f42fc02e640 con 0x7f42fc105d90 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.243+0000 7f42f9600640 1 -- 172.21.15.114:0/2326993409 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7f42f0004310 con 0x7f42fc105d90 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42e403fb70 msgr2=0x7f42e4042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42e403fb70 0x7f42e4042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f42fc1050b0 tx=0x7f42e800b390 comp rx=0 tx=0).stop 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 msgr2=0x7f42fc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 0x7f42fc19c640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42f000db80 tx=0x7f42f000b630 comp rx=0 tx=0).stop 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 shutdown_connections 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42e403fb70 0x7f42e4042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42fc10c020 0x7f42fc1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc10b6d0 0x7f42fc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 --2- 172.21.15.114:0/2326993409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42fc105d90 0x7f42fc19c640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 >> 172.21.15.114:0/2326993409 conn(0x7f42fc101860 msgr2=0x7f42fc108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.245+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 shutdown_connections 2024-09-21T06:43:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.246+0000 7f4303600640 1 -- 172.21.15.114:0/2326993409 wait complete. 2024-09-21T06:43:41.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:41.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:41 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:41.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.431+0000 7fc0a2e00640 1 -- 172.21.15.114:0/120784368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10a3c0 msgr2=0x7fc09c10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.431+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/120784368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10a3c0 0x7fc09c10a7d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc088009900 tx=0x7fc08802a440 comp rx=0 tx=0).stop 2024-09-21T06:43:41.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 -- 172.21.15.114:0/120784368 shutdown_connections 2024-09-21T06:43:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/120784368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc09c11cbb0 0x7fc09c11efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/120784368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10adf0 0x7fc09c111cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/120784368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10a3c0 0x7fc09c10a7d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 -- 172.21.15.114:0/120784368 >> 172.21.15.114:0/120784368 conn(0x7fc09c06ee50 msgr2=0x7fc09c06f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:41.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 -- 172.21.15.114:0/120784368 shutdown_connections 2024-09-21T06:43:41.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.435+0000 7fc0a2e00640 1 -- 172.21.15.114:0/120784368 wait complete. 2024-09-21T06:43:41.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.437+0000 7fc0a2e00640 1 Processor -- start 2024-09-21T06:43:41.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.437+0000 7fc0a2e00640 1 -- start start 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.438+0000 7fc0a2e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 0x7fc09c1a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.438+0000 7fc0a2e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10adf0 0x7fc09c1a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.438+0000 7fc0a2e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc09c11cbb0 0x7fc09c1ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.438+0000 7fc0a2e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c1227b0 con 0x7fc09c10a3c0 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.438+0000 7fc0a1a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 0x7fc09c1a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.439+0000 7fc0a1a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 0x7fc09c1a91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49824/0 (socket says 172.21.15.114:49824) 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.439+0000 7fc0a1a00640 1 -- 172.21.15.114:0/3217608177 learned_addr learned my addr 172.21.15.114:0/3217608177 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.439+0000 7fc0a1000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10adf0 0x7fc09c1a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.439+0000 7fc0a2400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc09c11cbb0 0x7fc09c1ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.439+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c122630 con 0x7fc09c11cbb0 2024-09-21T06:43:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.439+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c122930 con 0x7fc09c10adf0 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.440+0000 7fc0a1000640 1 -- 172.21.15.114:0/3217608177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc09c11cbb0 msgr2=0x7fc09c1ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.440+0000 7fc0a1000640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc09c11cbb0 0x7fc09c1ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.440+0000 7fc0a1000640 1 -- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 msgr2=0x7fc09c1a91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.440+0000 7fc0a1000640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 0x7fc09c1a91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.440+0000 7fc0a1000640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c1af050 con 0x7fc09c10adf0 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.440+0000 7fc0a1a00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 0x7fc09c1a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.441+0000 7fc0a1000640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10adf0 0x7fc09c1a9740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc09c06c910 tx=0x7fc09000cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.443+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090007bb0 con 0x7fc09c10adf0 2024-09-21T06:43:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.443+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc090007d50 con 0x7fc09c10adf0 2024-09-21T06:43:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.443+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0900186f0 con 0x7fc09c10adf0 2024-09-21T06:43:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.444+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c1af340 con 0x7fc09c10adf0 2024-09-21T06:43:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.445+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c1b5b60 con 0x7fc09c10adf0 2024-09-21T06:43:41.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.447+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7fc090018890 con 0x7fc09c10adf0 2024-09-21T06:43:41.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.448+0000 7fc09a000640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc08403fbf0 0x7fc084042100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:41.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.448+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc09000f450 con 0x7fc09c10adf0 2024-09-21T06:43:41.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.449+0000 7fc0a1a00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc08403fbf0 0x7fc084042100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:41.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.450+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09c10a7d0 con 0x7fc09c10adf0 2024-09-21T06:43:41.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.452+0000 7fc0a1a00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc08403fbf0 0x7fc084042100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc088000c00 tx=0x7fc0880094c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:41.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.460+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc090014030 con 0x7fc09c10adf0 2024-09-21T06:43:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:43:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:41 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.668+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fc09c1b5d30 con 0x7fc09c10adf0 2024-09-21T06:43:41.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.675+0000 7fc09a000640 1 -- 172.21.15.114:0/3217608177 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v17)=0 v17) ==== 127+0+0 (secure 0 0 0) 0x7fc090047070 con 0x7fc09c10adf0 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc08403fbf0 msgr2=0x7fc084042100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc08403fbf0 0x7fc084042100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc088000c00 tx=0x7fc0880094c0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10adf0 msgr2=0x7fc09c1a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10adf0 0x7fc09c1a9740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc09c06c910 tx=0x7fc09000cdd0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 shutdown_connections 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc08403fbf0 0x7fc084042100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc09c11cbb0 0x7fc09c1ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc09c10adf0 0x7fc09c1a9740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 --2- 172.21.15.114:0/3217608177 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc09c10a3c0 0x7fc09c1a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:41.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 >> 172.21.15.114:0/3217608177 conn(0x7fc09c06ee50 msgr2=0x7fc09c10b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:41.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 shutdown_connections 2024-09-21T06:43:41.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:41.678+0000 7fc0a2e00640 1 -- 172.21.15.114:0/3217608177 wait complete. 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 -- 172.21.15.114:0/3100869431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4103990 msgr2=0x7f70d4103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 --2- 172.21.15.114:0/3100869431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4103990 0x7f70d4103da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009900 tx=0x7f70c402a410 comp rx=0 tx=0).stop 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 -- 172.21.15.114:0/3100869431 shutdown_connections 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 --2- 172.21.15.114:0/3100869431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4109c20 0x7f70d4110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 --2- 172.21.15.114:0/3100869431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d41096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 --2- 172.21.15.114:0/3100869431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4103990 0x7f70d4103da0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 -- 172.21.15.114:0/3100869431 >> 172.21.15.114:0/3100869431 conn(0x7f70d40ff3e0 msgr2=0x7f70d4101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 -- 172.21.15.114:0/3100869431 shutdown_connections 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.098+0000 7f70d8e00640 1 -- 172.21.15.114:0/3100869431 wait complete. 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 Processor -- start 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 -- start start 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 0x7f70d419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4109c20 0x7f70d41a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f70d41138e0 con 0x7f70d4109c20 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4113760 con 0x7f70d4103990 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d8e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70d4113a60 con 0x7f70d41092d0 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 0x7f70d419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4109c20 0x7f70d41a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47536/0 (socket says 172.21.15.114:47536) 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 0x7f70d419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58300/0 (socket says 172.21.15.114:58300) 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.100+0000 7f70d2a00640 1 -- 172.21.15.114:0/1007761417 learned_addr learned my addr 172.21.15.114:0/1007761417 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d2a00640 1 -- 172.21.15.114:0/1007761417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 msgr2=0x7f70d419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d2a00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 0x7f70d419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d2a00640 1 -- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4109c20 msgr2=0x7f70d41a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d2a00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4109c20 0x7f70d41a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d2a00640 1 -- 172.21.15.114:0/1007761417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d41a24c0 con 0x7f70d41092d0 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d3e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4109c20 0x7f70d41a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d3400640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 0x7f70d419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:42.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d2a00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d419caf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70c000e9c0 tx=0x7f70c000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c000ccc0 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c00054d0 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d41a27b0 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.101+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0010430 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.102+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d41a8f10 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.103+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d4103da0 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.103+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7f70c00105d0 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.103+0000 7f70cbe00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70b803fb70 0x7f70b8042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.103+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f70c0054930 con 0x7f70d41092d0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.104+0000 7f70d3400640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70b803fb70 0x7f70b8042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.104+0000 7f70d3400640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70b803fb70 0x7f70b8042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70c4031440 tx=0x7f70c4032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:42.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.113+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70c0016330 con 0x7f70d41092d0 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.319+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f70d41a9170 con 0x7f70d41092d0 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.327+0000 7f70cbe00640 1 -- 172.21.15.114:0/1007761417 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v18)=0 v18) ==== 116+0+0 (secure 0 0 0) 0x7f70c00046c0 con 0x7f70d41092d0 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70b803fb70 msgr2=0x7f70b8042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70b803fb70 0x7f70b8042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f70c4031440 tx=0x7f70c4032ed0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 msgr2=0x7f70d419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d419caf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70c000e9c0 tx=0x7f70c000ee80 comp rx=0 tx=0).stop 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 shutdown_connections 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70d4109c20 0x7f70d41a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d41092d0 0x7f70d419caf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70b803fb70 0x7f70b8042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 --2- 172.21.15.114:0/1007761417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70d4103990 0x7f70d419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 >> 172.21.15.114:0/1007761417 conn(0x7f70d40ff3e0 msgr2=0x7f70d41089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:42.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 shutdown_connections 2024-09-21T06:43:42.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.329+0000 7f70d8e00640 1 -- 172.21.15.114:0/1007761417 wait complete. 2024-09-21T06:43:42.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2326993409' entity='client.admin' 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:42.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:42.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: Reconfiguring daemon mgr.x on smithi114 2024-09-21T06:43:42.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:42 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:42.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2326993409' entity='client.admin' 2024-09-21T06:43:42.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: Reconfiguring daemon mgr.x on smithi114 2024-09-21T06:43:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:42 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2326993409' entity='client.admin' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T06:43:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:42.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: Reconfiguring daemon mgr.x on smithi114 2024-09-21T06:43:42.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:42 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:42.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 -- 172.21.15.114:0/453533165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520103990 msgr2=0x7f6520103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 --2- 172.21.15.114:0/453533165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520103990 0x7f6520103da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6514009900 tx=0x7f651402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 -- 172.21.15.114:0/453533165 shutdown_connections 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 --2- 172.21.15.114:0/453533165 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6520109c20 0x7f6520110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 --2- 172.21.15.114:0/453533165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65201092d0 0x7f65201096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 --2- 172.21.15.114:0/453533165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6520103990 0x7f6520103da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 -- 172.21.15.114:0/453533165 >> 172.21.15.114:0/453533165 conn(0x7f65200ff3e0 msgr2=0x7f6520101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 -- 172.21.15.114:0/453533165 shutdown_connections 2024-09-21T06:43:42.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.773+0000 7f6526800640 1 -- 172.21.15.114:0/453533165 wait complete. 2024-09-21T06:43:42.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 Processor -- start 2024-09-21T06:43:42.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 -- start start 2024-09-21T06:43:42.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6520103990 0x7f652019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 0x7f652019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6520109c20 0x7f652019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6520113930 con 0x7f6520103990 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65201137b0 con 0x7f65201092d0 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.774+0000 7f6526800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6520113ab0 con 0x7f6520109c20 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 0x7f652019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6525400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6520103990 0x7f652019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 0x7f652019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58304/0 (socket says 172.21.15.114:58304) 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 -- 172.21.15.114:0/1620807977 learned_addr learned my addr 172.21.15.114:0/1620807977 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:42.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6525e00640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6520109c20 0x7f652019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 -- 172.21.15.114:0/1620807977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6520109c20 msgr2=0x7f652019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6520109c20 0x7f652019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 -- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6520103990 msgr2=0x7f652019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6520103990 0x7f652019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 -- 172.21.15.114:0/1620807977 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65201a02b0 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6525e00640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6520109c20 0x7f652019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.775+0000 7f6524a00640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 0x7f652019a8e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e9c0 tx=0x7f650c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.776+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c00ccc0 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.776+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f650c0054d0 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.776+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c010430 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.776+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65201a05a0 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.776+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65201a6d00 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.777+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6520103da0 con 0x7f65201092d0 2024-09-21T06:43:42.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.777+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7f650c004f40 con 0x7f65201092d0 2024-09-21T06:43:42.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.778+0000 7f651e000640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f650803fb70 0x7f6508042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:42.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.778+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f650c054a40 con 0x7f65201092d0 2024-09-21T06:43:42.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.778+0000 7f6525400640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f650803fb70 0x7f6508042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:42.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.778+0000 7f6525400640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f650803fb70 0x7f6508042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6514005ab0 tx=0x7f6514005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:42.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.783+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f650c013190 con 0x7f65201092d0 2024-09-21T06:43:42.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.981+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f652002e640 con 0x7f65201092d0 2024-09-21T06:43:42.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.987+0000 7f651e000640 1 -- 172.21.15.114:0/1620807977 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v19)=0 v19) ==== 138+0+0 (secure 0 0 0) 0x7f650c021440 con 0x7f65201092d0 2024-09-21T06:43:42.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f650803fb70 msgr2=0x7f6508042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f650803fb70 0x7f6508042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6514005ab0 tx=0x7f6514005a20 comp rx=0 tx=0).stop 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 msgr2=0x7f652019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 0x7f652019a8e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e9c0 tx=0x7f650c00edc0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 shutdown_connections 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6520109c20 0x7f652019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f650803fb70 0x7f6508042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65201092d0 0x7f652019a8e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 --2- 172.21.15.114:0/1620807977 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6520103990 0x7f652019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 >> 172.21.15.114:0/1620807977 conn(0x7f65200ff3e0 msgr2=0x7f6520101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 shutdown_connections 2024-09-21T06:43:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:42.990+0000 7f6526800640 1 -- 172.21.15.114:0/1620807977 wait complete. 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 -- 172.21.15.114:0/4178449795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 msgr2=0x7f91ac112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 --2- 172.21.15.114:0/4178449795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 0x7f91ac112850 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009ca0 tx=0x7f91a402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 -- 172.21.15.114:0/4178449795 shutdown_connections 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 --2- 172.21.15.114:0/4178449795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 0x7f91ac112850 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 --2- 172.21.15.114:0/4178449795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac10b6d0 0x7f91ac10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 --2- 172.21.15.114:0/4178449795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac105d90 0x7f91ac1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 -- 172.21.15.114:0/4178449795 >> 172.21.15.114:0/4178449795 conn(0x7f91ac101860 msgr2=0x7f91ac103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 -- 172.21.15.114:0/4178449795 shutdown_connections 2024-09-21T06:43:43.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.005+0000 7f91b1600640 1 -- 172.21.15.114:0/4178449795 wait complete. 2024-09-21T06:43:43.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 Processor -- start 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 -- start start 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 0x7f91ac19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac10b6d0 0x7f91ac19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 0x7f91ac1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac115d30 con 0x7f91ac10b6d0 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac115bb0 con 0x7f91ac105d90 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.006+0000 7f91b1600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac115eb0 con 0x7f91ac10c020 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 0x7f91ac19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 0x7f91ac19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58322/0 (socket says 172.21.15.114:58322) 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 -- 172.21.15.114:0/628494829 learned_addr learned my addr 172.21.15.114:0/628494829 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aa000640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac10b6d0 0x7f91ac19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91ab400640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 0x7f91ac1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 -- 172.21.15.114:0/628494829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 msgr2=0x7f91ac1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 0x7f91ac1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 -- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac10b6d0 msgr2=0x7f91ac19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac10b6d0 0x7f91ac19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 -- 172.21.15.114:0/628494829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac1a25f0 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.007+0000 7f91aaa00640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 0x7f91ac19c680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f919c00ea60 tx=0x7f919c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.008+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00cde0 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.008+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac1a28e0 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.008+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c0035f0 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.008+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c010690 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.008+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac1a9040 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.009+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 51916+0+0 (secure 0 0 0) 0x7f919c010830 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.009+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ac1061a0 con 0x7f91ac105d90 2024-09-21T06:43:43.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.009+0000 7f91a3400640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f919003fb70 0x7f9190042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.010+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f919c0549d0 con 0x7f91ac105d90 2024-09-21T06:43:43.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.010+0000 7f91aa000640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f919003fb70 0x7f9190042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.010+0000 7f91aa000640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f919003fb70 0x7f9190042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91ac105110 tx=0x7f9198005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:43.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.015+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f919c0173d0 con 0x7f91ac105d90 2024-09-21T06:43:43.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.220+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f91ac02e640 con 0x7f91ac105d90 2024-09-21T06:43:43.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.221+0000 7f91a3400640 1 -- 172.21.15.114:0/628494829 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) ==== 106+0+0 (secure 0 0 0) 0x7f919c0046c0 con 0x7f91ac105d90 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f919003fb70 msgr2=0x7f9190042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f919003fb70 0x7f9190042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f91ac105110 tx=0x7f9198005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 msgr2=0x7f91ac19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 0x7f91ac19c680 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f919c00ea60 tx=0x7f919c00ef20 comp rx=0 tx=0).stop 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 shutdown_connections 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91ac10c020 0x7f91ac1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91ac10b6d0 0x7f91ac19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f919003fb70 0x7f9190042080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 --2- 172.21.15.114:0/628494829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91ac105d90 0x7f91ac19c680 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 >> 172.21.15.114:0/628494829 conn(0x7f91ac101860 msgr2=0x7f91ac10acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.223+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 shutdown_connections 2024-09-21T06:43:43.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.224+0000 7f91b1600640 1 -- 172.21.15.114:0/628494829 wait complete. 2024-09-21T06:43:43.502 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:43.502 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: Standby manager daemon z started 2024-09-21T06:43:43.502 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:43:43.503 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:43:43.503 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:43:43.503 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:43:43.503 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:43 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: Standby manager daemon z started 2024-09-21T06:43:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:43:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:43:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:43:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:43:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:43 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: Standby manager daemon z started 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.193:0/2514346077' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:43:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:43 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 -- 172.21.15.114:0/883607475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204103110 msgr2=0x7f8204107640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 --2- 172.21.15.114:0/883607475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204103110 0x7f8204107640 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009930 tx=0x7f81f002a420 comp rx=0 tx=0).stop 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 -- 172.21.15.114:0/883607475 shutdown_connections 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 --2- 172.21.15.114:0/883607475 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8204107c00 0x7f82041080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 --2- 172.21.15.114:0/883607475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204103110 0x7f8204107640 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 --2- 172.21.15.114:0/883607475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82041027c0 0x7f8204102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 -- 172.21.15.114:0/883607475 >> 172.21.15.114:0/883607475 conn(0x7f8204079740 msgr2=0x7f8204079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 -- 172.21.15.114:0/883607475 shutdown_connections 2024-09-21T06:43:43.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.678+0000 7f820ac00640 1 -- 172.21.15.114:0/883607475 wait complete. 2024-09-21T06:43:43.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.679+0000 7f820ac00640 1 Processor -- start 2024-09-21T06:43:43.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.679+0000 7f820ac00640 1 -- start start 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f820ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 0x7f82041a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f820ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8204103110 0x7f82041a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f820ac00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8204107c00 0x7f82041a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f820ac00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f820411a0b0 con 0x7f8204107c00 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f820ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8204119f30 con 0x7f82041027c0 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f820ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f820411a230 con 0x7f8204103110 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f8203e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 0x7f82041a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f8203e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 0x7f82041a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58346/0 (socket says 172.21.15.114:58346) 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f8203e00640 1 -- 172.21.15.114:0/2187095233 learned_addr learned my addr 172.21.15.114:0/2187095233 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f8208c00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8204107c00 0x7f82041a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.680+0000 7f8203400640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8204103110 0x7f82041a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203e00640 1 -- 172.21.15.114:0/2187095233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8204103110 msgr2=0x7f82041a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203e00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8204103110 0x7f82041a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203e00640 1 -- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8204107c00 msgr2=0x7f82041a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203e00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8204107c00 0x7f82041a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203e00640 1 -- 172.21.15.114:0/2187095233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82041a6a70 con 0x7f82041027c0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203400640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8204103110 0x7f82041a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8203e00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 0x7f82041a0b00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81f400d770 tx=0x7f81f400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4017070 con 0x7f82041027c0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f4009c30 con 0x7f82041027c0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003cc0 con 0x7f82041027c0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82041a6d60 con 0x7f82041027c0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.681+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82041ad4c0 con 0x7f82041027c0 2024-09-21T06:43:43.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.682+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820406c240 con 0x7f82041027c0 2024-09-21T06:43:43.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.683+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 103195+0+0 (secure 0 0 0) 0x7f81f4003e60 con 0x7f82041027c0 2024-09-21T06:43:43.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.684+0000 7f8200c00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81e807b020 0x7f81e807d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:43.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.685+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f81f409c7e0 con 0x7f82041027c0 2024-09-21T06:43:43.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.685+0000 7f8203400640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81e807b020 0x7f81e807d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:43.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.685+0000 7f8203400640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81e807b020 0x7f81e807d530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81f002a940 tx=0x7f81f0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:43.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.689+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81f4009de0 con 0x7f82041027c0 2024-09-21T06:43:43.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.909+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f82041a1c30 con 0x7f82041027c0 2024-09-21T06:43:43.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.916+0000 7f8200c00640 1 -- 172.21.15.114:0/2187095233 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7f81f405ee20 con 0x7f82041027c0 2024-09-21T06:43:43.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81e807b020 msgr2=0x7f81e807d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81e807b020 0x7f81e807d530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81f002a940 tx=0x7f81f0031400 comp rx=0 tx=0).stop 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 msgr2=0x7f82041a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 0x7f82041a0b00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81f400d770 tx=0x7f81f400dc30 comp rx=0 tx=0).stop 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 shutdown_connections 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8204107c00 0x7f82041a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8204103110 0x7f82041a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81e807b020 0x7f81e807d530 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 --2- 172.21.15.114:0/2187095233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82041027c0 0x7f82041a0b00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 >> 172.21.15.114:0/2187095233 conn(0x7f8204079740 msgr2=0x7f8204105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 shutdown_connections 2024-09-21T06:43:43.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:43.919+0000 7f820ac00640 1 -- 172.21.15.114:0/2187095233 wait complete. 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.106+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2372456015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b10076080 msgr2=0x7f6b1010fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.106+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2372456015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b10076080 0x7f6b1010fa50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009860 tx=0x7f6af802a340 comp rx=0 tx=0).stop 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.106+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2372456015 shutdown_connections 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.107+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2372456015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b1010ff90 0x7f6b101123b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.107+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2372456015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b10076080 0x7f6b1010fa50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.107+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2372456015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b10075730 0x7f6b10075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.107+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2372456015 >> 172.21.15.114:0/2372456015 conn(0x7f6b100ff400 msgr2=0x7f6b10101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.107+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2372456015 shutdown_connections 2024-09-21T06:43:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.107+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2372456015 wait complete. 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 Processor -- start 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 -- start start 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 0x7f6b1019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b10076080 0x7f6b1019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b1010ff90 0x7f6b101a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10115a90 con 0x7f6b10076080 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10115910 con 0x7f6b10075730 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.108+0000 7f6b0fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b10115c10 con 0x7f6b1010ff90 2024-09-21T06:43:44.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 0x7f6b1019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b07e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b10076080 0x7f6b1019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 0x7f6b1019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58374/0 (socket says 172.21.15.114:58374) 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 -- 172.21.15.114:0/2354520269 learned_addr learned my addr 172.21.15.114:0/2354520269 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0f400640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b1010ff90 0x7f6b101a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 -- 172.21.15.114:0/2354520269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b1010ff90 msgr2=0x7f6b101a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b1010ff90 0x7f6b101a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 -- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b10076080 msgr2=0x7f6b1019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b10076080 0x7f6b1019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0ea00640 1 -- 172.21.15.114:0/2354520269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b101a2490 con 0x7f6b10075730 2024-09-21T06:43:44.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.109+0000 7f6b0f400640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b1010ff90 0x7f6b101a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:44.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.110+0000 7f6b0ea00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 0x7f6b1019c520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000e9c0 tx=0x7f6b0000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:44.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.110+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000ccc0 con 0x7f6b10075730 2024-09-21T06:43:44.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.110+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b000054d0 con 0x7f6b10075730 2024-09-21T06:43:44.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.110+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b101a2780 con 0x7f6b10075730 2024-09-21T06:43:44.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.110+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00010430 con 0x7f6b10075730 2024-09-21T06:43:44.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.110+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b101a8ee0 con 0x7f6b10075730 2024-09-21T06:43:44.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.111+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b10075b40 con 0x7f6b10075730 2024-09-21T06:43:44.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.112+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 103195+0+0 (secure 0 0 0) 0x7f6b00005020 con 0x7f6b10075730 2024-09-21T06:43:44.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.113+0000 7f6b07400640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6af407b020 0x7f6af407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.114+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6b0009d2f0 con 0x7f6b10075730 2024-09-21T06:43:44.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.114+0000 7f6b07e00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6af407b020 0x7f6af407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.114+0000 7f6b07e00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6af407b020 0x7f6af407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6af8035040 tx=0x7f6af8030340 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:44.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.118+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b00065880 con 0x7f6b10075730 2024-09-21T06:43:44.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.309+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f6b101a91c0 con 0x7f6b10075730 2024-09-21T06:43:44.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.310+0000 7f6b07400640 1 -- 172.21.15.114:0/2354520269 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v20)=0 v20) ==== 108+0+0 (secure 0 0 0) 0x7f6b00069c60 con 0x7f6b10075730 2024-09-21T06:43:44.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6af407b020 msgr2=0x7f6af407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6af407b020 0x7f6af407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6af8035040 tx=0x7f6af8030340 comp rx=0 tx=0).stop 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 msgr2=0x7f6b1019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 0x7f6b1019c520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000e9c0 tx=0x7f6b0000ee80 comp rx=0 tx=0).stop 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 shutdown_connections 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b1010ff90 0x7f6b101a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b10076080 0x7f6b1019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6af407b020 0x7f6af407d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 --2- 172.21.15.114:0/2354520269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b10075730 0x7f6b1019c520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 >> 172.21.15.114:0/2354520269 conn(0x7f6b100ff400 msgr2=0x7f6b10108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 shutdown_connections 2024-09-21T06:43:44.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.313+0000 7f6b0fe00640 1 -- 172.21.15.114:0/2354520269 wait complete. 2024-09-21T06:43:44.490 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:44 smithi114 ceph-mon[30654]: mgrmap e19: x(active, since 13s), standbys: z 2024-09-21T06:43:44.491 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:43:44.491 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:44 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:44 smithi183 ceph-mon[34219]: mgrmap e19: x(active, since 13s), standbys: z 2024-09-21T06:43:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:43:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:44 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:44.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:44 smithi193 ceph-mon[34601]: mgrmap e19: x(active, since 13s), standbys: z 2024-09-21T06:43:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T06:43:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:44 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 -- 172.21.15.114:0/2939148508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 msgr2=0x7f354c112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 --2- 172.21.15.114:0/2939148508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 0x7f354c112670 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3544009ca0 tx=0x7f354402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 -- 172.21.15.114:0/2939148508 shutdown_connections 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 --2- 172.21.15.114:0/2939148508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 0x7f354c112670 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 --2- 172.21.15.114:0/2939148508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c10b4f0 0x7f354c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 --2- 172.21.15.114:0/2939148508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c105bb0 0x7f354c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 -- 172.21.15.114:0/2939148508 >> 172.21.15.114:0/2939148508 conn(0x7f354c101680 msgr2=0x7f354c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 -- 172.21.15.114:0/2939148508 shutdown_connections 2024-09-21T06:43:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.970+0000 7f354be00640 1 -- 172.21.15.114:0/2939148508 wait complete. 2024-09-21T06:43:44.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.971+0000 7f354be00640 1 Processor -- start 2024-09-21T06:43:44.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.971+0000 7f354be00640 1 -- start start 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 0x7f354c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c10b4f0 0x7f354c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 0x7f354c1a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f354c115b00 con 0x7f354c10be40 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f354c115980 con 0x7f354c105bb0 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f354c115c80 con 0x7f354c10b4f0 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 0x7f354c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 0x7f354c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56526/0 (socket says 172.21.15.114:56526) 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 -- 172.21.15.114:0/3839521856 learned_addr learned my addr 172.21.15.114:0/3839521856 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354b400640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 0x7f354c1a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f3542000640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c10b4f0 0x7f354c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 -- 172.21.15.114:0/3839521856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c10b4f0 msgr2=0x7f354c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c10b4f0 0x7f354c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 -- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 msgr2=0x7f354c1a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 0x7f354c1a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f354aa00640 1 -- 172.21.15.114:0/3839521856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c1a2520 con 0x7f354c105bb0 2024-09-21T06:43:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.972+0000 7f3542000640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c10b4f0 0x7f354c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.973+0000 7f354aa00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 0x7f354c19c5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f353c00e9f0 tx=0x7f353c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.973+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c00cd80 con 0x7f354c105bb0 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.973+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f353c0054d0 con 0x7f354c105bb0 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.973+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c1a2810 con 0x7f354c105bb0 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.973+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c010690 con 0x7f354c105bb0 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.973+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c1a8f70 con 0x7f354c105bb0 2024-09-21T06:43:44.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.974+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c105fc0 con 0x7f354c105bb0 2024-09-21T06:43:44.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.975+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 103195+0+0 (secure 0 0 0) 0x7f353c004e90 con 0x7f354c105bb0 2024-09-21T06:43:44.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.976+0000 7f3543e00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f353007b020 0x7f353007d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:44.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.976+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f353c09d2f0 con 0x7f354c105bb0 2024-09-21T06:43:44.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.976+0000 7f3542000640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f353007b020 0x7f353007d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:44.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.977+0000 7f3542000640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f353007b020 0x7f353007d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f354c104d80 tx=0x7f3534005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:44.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:44.981+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f353c065940 con 0x7f354c105bb0 2024-09-21T06:43:45.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.178+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f354c10dc90 con 0x7f354c105bb0 2024-09-21T06:43:45.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.185+0000 7f3543e00640 1 -- 172.21.15.114:0/3839521856 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v21)=0 v21) ==== 111+0+0 (secure 0 0 0) 0x7f353c069d20 con 0x7f354c105bb0 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f353007b020 msgr2=0x7f353007d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f353007b020 0x7f353007d530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f354c104d80 tx=0x7f3534005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 msgr2=0x7f354c19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 0x7f354c19c5b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f353c00e9f0 tx=0x7f353c00edf0 comp rx=0 tx=0).stop 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 shutdown_connections 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f354c10be40 0x7f354c1a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354c10b4f0 0x7f354c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f353007b020 0x7f353007d530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 --2- 172.21.15.114:0/3839521856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f354c105bb0 0x7f354c19c5b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.188+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 >> 172.21.15.114:0/3839521856 conn(0x7f354c101680 msgr2=0x7f354c108aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.189+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 shutdown_connections 2024-09-21T06:43:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:45.189+0000 7f354be00640 1 -- 172.21.15.114:0/3839521856 wait complete. 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: Standby manager daemon y started 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:43:45.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:45 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: Standby manager daemon y started 2024-09-21T06:43:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:43:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:43:46.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:43:46.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:43:46.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:45 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:46.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: Standby manager daemon y started 2024-09-21T06:43:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T06:43:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T06:43:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T06:43:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: from='mgr.? 172.21.15.183:0/1115787946' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T06:43:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:45 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:46.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:46 smithi114 ceph-mon[30654]: mgrmap e20: x(active, since 16s), standbys: z, y 2024-09-21T06:43:46.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:43:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:46 smithi183 ceph-mon[34219]: mgrmap e20: x(active, since 16s), standbys: z, y 2024-09-21T06:43:47.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:43:47.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:46 smithi193 ceph-mon[34601]: mgrmap e20: x(active, since 16s), standbys: z, y 2024-09-21T06:43:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T06:43:47.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 -- 172.21.15.114:0/4214803535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136810de10 msgr2=0x7f1368110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 --2- 172.21.15.114:0/4214803535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136810de10 0x7f1368110300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1364009ca0 tx=0x7f136402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 -- 172.21.15.114:0/4214803535 shutdown_connections 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 --2- 172.21.15.114:0/4214803535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f136810de10 0x7f1368110300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 --2- 172.21.15.114:0/4214803535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1368105300 0x7f136810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 --2- 172.21.15.114:0/4214803535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f1368104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 -- 172.21.15.114:0/4214803535 >> 172.21.15.114:0/4214803535 conn(0x7f1368079730 msgr2=0x7f1368079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 -- 172.21.15.114:0/4214803535 shutdown_connections 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.659+0000 7f1370800640 1 -- 172.21.15.114:0/4214803535 wait complete. 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.660+0000 7f1370800640 1 Processor -- start 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 -- start start 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f13681a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1368105300 0x7f13681a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f136810de10 0x7f13681ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1368113ae0 con 0x7f1368105300 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1368113960 con 0x7f13681049b0 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.661+0000 7f1370800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1368113c60 con 0x7f136810de10 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f136810de10 0x7f13681ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f13681a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f13681a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56546/0 (socket says 172.21.15.114:56546) 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 -- 172.21.15.114:0/1590816088 learned_addr learned my addr 172.21.15.114:0/1590816088 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136d400640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1368105300 0x7f13681a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 -- 172.21.15.114:0/1590816088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f136810de10 msgr2=0x7f13681ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f136810de10 0x7f13681ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 -- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1368105300 msgr2=0x7f13681a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1368105300 0x7f13681a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136de00640 1 -- 172.21.15.114:0/1590816088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681a25e0 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136d400640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1368105300 0x7f13681a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.662+0000 7f136e800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f136810de10 0x7f13681ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.663+0000 7f136de00640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f13681a17c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f135c00e9c0 tx=0x7f135c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.663+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c00cd80 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.663+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f135c0035f0 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.663+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c010690 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.663+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681acea0 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.663+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681ad300 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.664+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368104dc0 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.666+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f135c004000 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.668+0000 7f1362a00640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f13500b6550 0x7f13500b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.668+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f135c0e5fd0 con 0x7f13681049b0 2024-09-21T06:43:47.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.668+0000 7f136d400640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f13500b6550 0x7f13500b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.669+0000 7f136d400640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f13500b6550 0x7f13500b8a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f136810fb00 tx=0x7f1358005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:47.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.670+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f135c0ae3c0 con 0x7f13681049b0 2024-09-21T06:43:47.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 -- 172.21.15.114:0/3884250589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 msgr2=0x7fa420103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 --2- 172.21.15.114:0/3884250589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa420103dc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa410009860 tx=0x7fa41002a340 comp rx=0 tx=0).stop 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 -- 172.21.15.114:0/3884250589 shutdown_connections 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 --2- 172.21.15.114:0/3884250589 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa420109c40 0x7fa420110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 --2- 172.21.15.114:0/3884250589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4201092f0 0x7fa420109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 --2- 172.21.15.114:0/3884250589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa420103dc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 -- 172.21.15.114:0/3884250589 >> 172.21.15.114:0/3884250589 conn(0x7fa4200ff440 msgr2=0x7fa420101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 -- 172.21.15.114:0/3884250589 shutdown_connections 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.770+0000 7fa425c00640 1 -- 172.21.15.114:0/3884250589 wait complete. 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.771+0000 7fa425c00640 1 Processor -- start 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.771+0000 7fa425c00640 1 -- start start 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa42019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4201092f0 0x7fa42019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420109c40 0x7fa4201a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa420113900 con 0x7fa4201092f0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa420113780 con 0x7fa4201039b0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa420113a80 con 0x7fa420109c40 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa42019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4201092f0 0x7fa42019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa42019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56562/0 (socket says 172.21.15.114:56562) 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 -- 172.21.15.114:0/1118359288 learned_addr learned my addr 172.21.15.114:0/1118359288 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa425200640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420109c40 0x7fa4201a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 -- 172.21.15.114:0/1118359288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420109c40 msgr2=0x7fa4201a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420109c40 0x7fa4201a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 -- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4201092f0 msgr2=0x7fa42019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4201092f0 0x7fa42019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.772+0000 7fa41fe00640 1 -- 172.21.15.114:0/1118359288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4201a24c0 con 0x7fa4201039b0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.773+0000 7fa41fe00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa42019c550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4100303f0 tx=0x7fa410032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.773+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410040020 con 0x7fa4201039b0 2024-09-21T06:43:47.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.773+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa410030dd0 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.773+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4201a2750 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.773+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa410037040 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.773+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4201a8f10 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.774+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa420104c30 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.777+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fa410043020 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.778+0000 7fa41cc00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa4040b6550 0x7fa4040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.778+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa4101053c0 con 0x7fa4201039b0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.779+0000 7fa41f400640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa4040b6550 0x7fa4040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.779+0000 7fa41f400640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa4040b6550 0x7fa4040b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa420105a60 tx=0x7fa40c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:47.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.780+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa41004a020 con 0x7fa4201039b0 2024-09-21T06:43:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.875+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f13681ad590 con 0x7f13681049b0 2024-09-21T06:43:47.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.881+0000 7f1362a00640 1 -- 172.21.15.114:0/1590816088 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v22)=0 v22) ==== 125+0+0 (secure 0 0 0) 0x7f135c0b27a0 con 0x7f13681049b0 2024-09-21T06:43:47.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.884+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f13500b6550 msgr2=0x7f13500b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.884+0000 7f1370800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f13500b6550 0x7f13500b8a60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f136810fb00 tx=0x7f1358005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 msgr2=0x7f13681a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f13681a17c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f135c00e9c0 tx=0x7f135c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 shutdown_connections 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f136810de10 0x7f13681ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1368105300 0x7f13681a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f13500b6550 0x7f13500b8a60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 --2- 172.21.15.114:0/1590816088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13681049b0 0x7f13681a17c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 >> 172.21.15.114:0/1590816088 conn(0x7f1368079730 msgr2=0x7f1368103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 shutdown_connections 2024-09-21T06:43:47.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.885+0000 7f1370800640 1 -- 172.21.15.114:0/1590816088 wait complete. 2024-09-21T06:43:47.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.989+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa420103dc0 con 0x7fa4201039b0 2024-09-21T06:43:47.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:47.997+0000 7fa41cc00640 1 -- 172.21.15.114:0/1118359288 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v23)=0 v23) ==== 133+0+0 (secure 0 0 0) 0x7fa420103dc0 con 0x7fa4201039b0 2024-09-21T06:43:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa4040b6550 msgr2=0x7fa4040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa4040b6550 0x7fa4040b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa420105a60 tx=0x7fa40c005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 msgr2=0x7fa42019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa42019c550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa4100303f0 tx=0x7fa410032da0 comp rx=0 tx=0).stop 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 shutdown_connections 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa420109c40 0x7fa4201a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa4201092f0 0x7fa42019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa4040b6550 0x7fa4040b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 --2- 172.21.15.114:0/1118359288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4201039b0 0x7fa42019c550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 >> 172.21.15.114:0/1118359288 conn(0x7fa4200ff440 msgr2=0x7fa4201089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 shutdown_connections 2024-09-21T06:43:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.000+0000 7fa425c00640 1 -- 172.21.15.114:0/1118359288 wait complete. 2024-09-21T06:43:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:48 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:48 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:48 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/123189910' entity='client.admin' 2024-09-21T06:43:49.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:48 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:49.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:48 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:49.204 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:48 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/123189910' entity='client.admin' 2024-09-21T06:43:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:48 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:48 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:48 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/123189910' entity='client.admin' 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 -- 172.21.15.114:0/764952776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 msgr2=0x7fa038105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 --2- 172.21.15.114:0/764952776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa038105fc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa028009900 tx=0x7fa02802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 -- 172.21.15.114:0/764952776 shutdown_connections 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 --2- 172.21.15.114:0/764952776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03810be40 0x7fa038112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 --2- 172.21.15.114:0/764952776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa03810b4f0 0x7fa03810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 --2- 172.21.15.114:0/764952776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa038105fc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 -- 172.21.15.114:0/764952776 >> 172.21.15.114:0/764952776 conn(0x7fa038101680 msgr2=0x7fa038103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 -- 172.21.15.114:0/764952776 shutdown_connections 2024-09-21T06:43:49.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.388+0000 7fa03e400640 1 -- 172.21.15.114:0/764952776 wait complete. 2024-09-21T06:43:49.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.389+0000 7fa03e400640 1 Processor -- start 2024-09-21T06:43:49.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.389+0000 7fa03e400640 1 -- start start 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa03819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03810b4f0 0x7fa03819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa03810be40 0x7fa0381a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa038115b00 con 0x7fa038105bb0 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa038115980 con 0x7fa03810b4f0 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa038115c80 con 0x7fa03810be40 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa03819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa03819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51110/0 (socket says 172.21.15.114:51110) 2024-09-21T06:43:49.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 -- 172.21.15.114:0/2137687849 learned_addr learned my addr 172.21.15.114:0/2137687849 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa02fe00640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03810b4f0 0x7fa03819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03da00640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa03810be40 0x7fa0381a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 -- 172.21.15.114:0/2137687849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa03810be40 msgr2=0x7fa0381a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa03810be40 0x7fa0381a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 -- 172.21.15.114:0/2137687849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03810b4f0 msgr2=0x7fa03819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03810b4f0 0x7fa03819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.390+0000 7fa03d000640 1 -- 172.21.15.114:0/2137687849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0381a24a0 con 0x7fa038105bb0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.391+0000 7fa03d000640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa03819c530 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa028005940 tx=0x7fa028004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.391+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02803f030 con 0x7fa038105bb0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.391+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa028003aa0 con 0x7fa038105bb0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.391+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa028035070 con 0x7fa038105bb0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.391+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0381a2730 con 0x7fa038105bb0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.391+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0381a8ef0 con 0x7fa038105bb0 2024-09-21T06:43:49.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.392+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa038105fc0 con 0x7fa038105bb0 2024-09-21T06:43:49.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.396+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fa028042030 con 0x7fa038105bb0 2024-09-21T06:43:49.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.397+0000 7fa036000640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa01c0b64d0 0x7fa01c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:49.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.398+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa028105fe0 con 0x7fa038105bb0 2024-09-21T06:43:49.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.398+0000 7fa02fe00640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa01c0b64d0 0x7fa01c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:49.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.398+0000 7fa02fe00640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa01c0b64d0 0x7fa01c0b89e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa03810bdb0 tx=0x7fa02000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:49.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.400+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa0280d27f0 con 0x7fa038105bb0 2024-09-21T06:43:49.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.612+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fa03819d660 con 0x7fa038105bb0 2024-09-21T06:43:49.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.617+0000 7fa036000640 1 -- 172.21.15.114:0/2137687849 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v27)=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fa03819d660 con 0x7fa038105bb0 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa01c0b64d0 msgr2=0x7fa01c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa01c0b64d0 0x7fa01c0b89e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa03810bdb0 tx=0x7fa02000b390 comp rx=0 tx=0).stop 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 msgr2=0x7fa03819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa03819c530 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa028005940 tx=0x7fa028004270 comp rx=0 tx=0).stop 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 shutdown_connections 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa03810be40 0x7fa0381a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa01c0b64d0 0x7fa01c0b89e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa03810b4f0 0x7fa03819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 --2- 172.21.15.114:0/2137687849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa038105bb0 0x7fa03819c530 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:49.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 >> 172.21.15.114:0/2137687849 conn(0x7fa038101680 msgr2=0x7fa038103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:49.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 shutdown_connections 2024-09-21T06:43:49.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.619+0000 7fa03e400640 1 -- 172.21.15.114:0/2137687849 wait complete. 2024-09-21T06:43:50.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 -- 172.21.15.114:0/2768388026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810c010 msgr2=0x7f7118112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:50.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 --2- 172.21.15.114:0/2768388026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810c010 0x7f7118112840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7110009ca0 tx=0x7f711002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 -- 172.21.15.114:0/2768388026 shutdown_connections 2024-09-21T06:43:50.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 --2- 172.21.15.114:0/2768388026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810c010 0x7f7118112840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 --2- 172.21.15.114:0/2768388026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711810b6c0 0x7f711810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 --2- 172.21.15.114:0/2768388026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7118105d80 0x7f7118106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 -- 172.21.15.114:0/2768388026 >> 172.21.15.114:0/2768388026 conn(0x7f7118101810 msgr2=0x7f7118103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 -- 172.21.15.114:0/2768388026 shutdown_connections 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.397+0000 7f711fe00640 1 -- 172.21.15.114:0/2768388026 wait complete. 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.398+0000 7f711fe00640 1 Processor -- start 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 -- start start 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7118105d80 0x7f711819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 0x7f711819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810c010 0x7f71181a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7118115cd0 con 0x7f711810b6c0 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7118115b50 con 0x7f7118105d80 2024-09-21T06:43:50.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7118115e50 con 0x7f711810c010 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 0x7f711819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 0x7f711819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51068/0 (socket says 172.21.15.114:51068) 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711ca00640 1 -- 172.21.15.114:0/123189910 learned_addr learned my addr 172.21.15.114:0/123189910 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711d400640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7118105d80 0x7f711819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.399+0000 7f711de00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810c010 0x7f71181a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711ca00640 1 -- 172.21.15.114:0/123189910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810c010 msgr2=0x7f71181a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711ca00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810c010 0x7f71181a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711ca00640 1 -- 172.21.15.114:0/123189910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7118105d80 msgr2=0x7f711819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711ca00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7118105d80 0x7f711819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711ca00640 1 -- 172.21.15.114:0/123189910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181a25d0 con 0x7f711810b6c0 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711d400640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7118105d80 0x7f711819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711ca00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 0x7f711819cc00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9f0 tx=0x7f710400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:50.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710400ccc0 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71040108e0 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a28c0 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104018860 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.400+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a9020 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.401+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7118106190 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.404+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f7104010430 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.405+0000 7f7116000640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71000b64d0 0x7f71000b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.405+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f71040e5f20 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.406+0000 7f711d400640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71000b64d0 0x7f71000b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.406+0000 7f711d400640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71000b64d0 0x7f71000b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f710c009fc0 tx=0x7f710c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.407+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f71040b26f0 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.607+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f711810de00 con 0x7f711810b6c0 2024-09-21T06:43:50.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.611+0000 7f7116000640 1 -- 172.21.15.114:0/123189910 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v24)=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7f71040b7630 con 0x7f711810b6c0 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71000b64d0 msgr2=0x7f71000b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71000b64d0 0x7f71000b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f710c009fc0 tx=0x7f710c005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 msgr2=0x7f711819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 0x7f711819cc00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9f0 tx=0x7f710400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 shutdown_connections 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711810c010 0x7f71181a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711810b6c0 0x7f711819cc00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71000b64d0 0x7f71000b89e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 --2- 172.21.15.114:0/123189910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7118105d80 0x7f711819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 >> 172.21.15.114:0/123189910 conn(0x7f7118101810 msgr2=0x7f711810ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 shutdown_connections 2024-09-21T06:43:50.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.614+0000 7f711fe00640 1 -- 172.21.15.114:0/123189910 wait complete. 2024-09-21T06:43:50.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3191152034' entity='client.admin' 2024-09-21T06:43:50.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4002030390' entity='client.admin' 2024-09-21T06:43:50.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:49 smithi114 ceph-mon[30654]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:50.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2137687849' entity='client.admin' 2024-09-21T06:43:50.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3191152034' entity='client.admin' 2024-09-21T06:43:50.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4002030390' entity='client.admin' 2024-09-21T06:43:50.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:49 smithi183 ceph-mon[34219]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:50.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2137687849' entity='client.admin' 2024-09-21T06:43:50.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3191152034' entity='client.admin' 2024-09-21T06:43:50.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4002030390' entity='client.admin' 2024-09-21T06:43:50.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:49 smithi193 ceph-mon[34601]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:50.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2137687849' entity='client.admin' 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 -- 172.21.15.114:0/1028395244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0110140 msgr2=0x7f69d0112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 --2- 172.21.15.114:0/1028395244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0110140 0x7f69d0112560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f69c8009ca0 tx=0x7f69c802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 -- 172.21.15.114:0/1028395244 shutdown_connections 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 --2- 172.21.15.114:0/1028395244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0110140 0x7f69d0112560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 --2- 172.21.15.114:0/1028395244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d0076080 0x7f69d010fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 --2- 172.21.15.114:0/1028395244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d0075730 0x7f69d0075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 -- 172.21.15.114:0/1028395244 >> 172.21.15.114:0/1028395244 conn(0x7f69d007b850 msgr2=0x7f69d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:51.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 -- 172.21.15.114:0/1028395244 shutdown_connections 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.033+0000 7f69d5c00640 1 -- 172.21.15.114:0/1028395244 wait complete. 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 Processor -- start 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 -- start start 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 0x7f69d019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d0076080 0x7f69d019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d0110140 0x7f69d01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0115cf0 con 0x7f69d0075730 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0115b70 con 0x7f69d0110140 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.034+0000 7f69d5c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0115e70 con 0x7f69d0076080 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 0x7f69d019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 0x7f69d019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51094/0 (socket says 172.21.15.114:51094) 2024-09-21T06:43:51.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 -- 172.21.15.114:0/4002030390 learned_addr learned my addr 172.21.15.114:0/4002030390 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cf400640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d0110140 0x7f69d01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69ce000640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d0076080 0x7f69d019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 -- 172.21.15.114:0/4002030390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d0076080 msgr2=0x7f69d019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d0076080 0x7f69d019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 -- 172.21.15.114:0/4002030390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d0110140 msgr2=0x7f69d01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d0110140 0x7f69d01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 -- 172.21.15.114:0/4002030390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d01a2630 con 0x7f69d0075730 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.035+0000 7f69cea00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 0x7f69d019c6c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69c000ea60 tx=0x7f69c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.036+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c000cde0 con 0x7f69d0075730 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.036+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69c00035f0 con 0x7f69d0075730 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.036+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0010690 con 0x7f69d0075730 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.036+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d01a2920 con 0x7f69d0075730 2024-09-21T06:43:51.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.036+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01a9080 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.037+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69d0075b40 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.039+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f69c00037a0 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.041+0000 7f69c7400640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f69b40b6550 0x7f69b40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.041+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f69c00e5c10 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.041+0000 7f69ce000640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f69b40b6550 0x7f69b40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.042+0000 7f69ce000640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f69b40b6550 0x7f69b40b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69d0111d90 tx=0x7f69bc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.042+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f69c0056710 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.239+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f69d019d7f0 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.243+0000 7f69c7400640 1 -- 172.21.15.114:0/4002030390 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v26)=0 v26) ==== 119+0+0 (secure 0 0 0) 0x7f69c00ae000 con 0x7f69d0075730 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f69b40b6550 msgr2=0x7f69b40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f69b40b6550 0x7f69b40b8a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69d0111d90 tx=0x7f69bc005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 msgr2=0x7f69d019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:51.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 0x7f69d019c6c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69c000ea60 tx=0x7f69c000ef20 comp rx=0 tx=0).stop 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 shutdown_connections 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f69b40b6550 0x7f69b40b8a60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69d0110140 0x7f69d01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d0076080 0x7f69d019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 --2- 172.21.15.114:0/4002030390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f69d0075730 0x7f69d019c6c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 >> 172.21.15.114:0/4002030390 conn(0x7f69d007b850 msgr2=0x7f69d0100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 shutdown_connections 2024-09-21T06:43:51.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:49.246+0000 7f69d5c00640 1 -- 172.21.15.114:0/4002030390 wait complete. 2024-09-21T06:43:51.814 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/374661773' entity='client.admin' 2024-09-21T06:43:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:51 smithi114 ceph-mon[30654]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:52.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/374661773' entity='client.admin' 2024-09-21T06:43:52.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:51 smithi183 ceph-mon[34219]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/374661773' entity='client.admin' 2024-09-21T06:43:52.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:51 smithi193 ceph-mon[34601]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:52.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.707+0000 7efe15400640 1 -- 172.21.15.114:0/1106962820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10075dd0 msgr2=0x7efe100761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.707+0000 7efe15400640 1 --2- 172.21.15.114:0/1106962820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10075dd0 0x7efe100761e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009900 tx=0x7efdf802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 -- 172.21.15.114:0/1106962820 shutdown_connections 2024-09-21T06:43:52.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 --2- 172.21.15.114:0/1106962820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10114280 0x7efe10116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 --2- 172.21.15.114:0/1106962820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10075dd0 0x7efe100761e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 --2- 172.21.15.114:0/1106962820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe10077380 0x7efe100757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 -- 172.21.15.114:0/1106962820 >> 172.21.15.114:0/1106962820 conn(0x7efe100ff440 msgr2=0x7efe10101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 -- 172.21.15.114:0/1106962820 shutdown_connections 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.708+0000 7efe15400640 1 -- 172.21.15.114:0/1106962820 wait complete. 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 Processor -- start 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 -- start start 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10075dd0 0x7efe101a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe10077380 0x7efe101a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 0x7efe101a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe10119ec0 con 0x7efe10114280 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe10119d40 con 0x7efe10075dd0 2024-09-21T06:43:52.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.709+0000 7efe15400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe1011a040 con 0x7efe10077380 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 0x7efe101a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 0x7efe101a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51086/0 (socket says 172.21.15.114:51086) 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 -- 172.21.15.114:0/3191152034 learned_addr learned my addr 172.21.15.114:0/3191152034 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe0fe00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10075dd0 0x7efe101a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe07e00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe10077380 0x7efe101a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 -- 172.21.15.114:0/3191152034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe10077380 msgr2=0x7efe101a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe10077380 0x7efe101a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 -- 172.21.15.114:0/3191152034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10075dd0 msgr2=0x7efe101a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10075dd0 0x7efe101a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 -- 172.21.15.114:0/3191152034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe101a68a0 con 0x7efe10114280 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe0fe00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10075dd0 0x7efe101a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe14a00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 0x7efe101a6170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efe0800eb10 tx=0x7efe0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:52.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.710+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004b20 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.711+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe08004cc0 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.711+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe101a6b90 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.711+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08010670 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.711+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe101ad2f0 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.712+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe100761e0 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.714+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7efe08005020 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.715+0000 7efe0d600640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efdf40b64d0 0x7efdf40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.716+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7efe080e5d50 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.716+0000 7efe0fe00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efdf40b64d0 0x7efdf40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.716+0000 7efe0fe00640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efdf40b64d0 0x7efdf40b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efe000095f0 tx=0x7efe00005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.719+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efe080ae140 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.925+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7efe10112a20 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.932+0000 7efe0d600640 1 -- 172.21.15.114:0/3191152034 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v25)=0 v25) ==== 122+0+0 (secure 0 0 0) 0x7efe080b2520 con 0x7efe10114280 2024-09-21T06:43:52.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efdf40b64d0 msgr2=0x7efdf40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efdf40b64d0 0x7efdf40b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efe000095f0 tx=0x7efe00005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 msgr2=0x7efe101a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 0x7efe101a6170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efe0800eb10 tx=0x7efe0800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 shutdown_connections 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe10114280 0x7efe101a6170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe10077380 0x7efe101a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efdf40b64d0 0x7efdf40b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 --2- 172.21.15.114:0/3191152034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe10075dd0 0x7efe101a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 >> 172.21.15.114:0/3191152034 conn(0x7efe100ff440 msgr2=0x7efe10110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 shutdown_connections 2024-09-21T06:43:52.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:48.935+0000 7efe15400640 1 -- 172.21.15.114:0/3191152034 wait complete. 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 -- 172.21.15.114:0/2827773102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4110220 msgr2=0x7fecd4112640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 --2- 172.21.15.114:0/2827773102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4110220 0x7fecd4112640 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feccc009ca0 tx=0x7feccc02a950 comp rx=0 tx=0).stop 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 -- 172.21.15.114:0/2827773102 shutdown_connections 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 --2- 172.21.15.114:0/2827773102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4110220 0x7fecd4112640 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 --2- 172.21.15.114:0/2827773102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd4102fc0 0x7fecd410fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 --2- 172.21.15.114:0/2827773102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecd4102670 0x7fecd4102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 -- 172.21.15.114:0/2827773102 >> 172.21.15.114:0/2827773102 conn(0x7fecd4079740 msgr2=0x7fecd4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 -- 172.21.15.114:0/2827773102 shutdown_connections 2024-09-21T06:43:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.720+0000 7fecd8c00640 1 -- 172.21.15.114:0/2827773102 wait complete. 2024-09-21T06:43:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.721+0000 7fecd8c00640 1 Processor -- start 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.721+0000 7fecd8c00640 1 -- start start 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 0x7fecd419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd4102fc0 0x7fecd419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecd4110220 0x7fecd41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd8c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4115ce0 con 0x7fecd4102670 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd8c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4115b60 con 0x7fecd4110220 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd8c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fecd4115e60 con 0x7fecd4102fc0 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 0x7fecd419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 0x7fecd419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51152/0 (socket says 172.21.15.114:51152) 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd1e00640 1 -- 172.21.15.114:0/3359400135 learned_addr learned my addr 172.21.15.114:0/3359400135 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7feccbe00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd4102fc0 0x7fecd419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.722+0000 7fecd2800640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecd4110220 0x7fecd41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd1e00640 1 -- 172.21.15.114:0/3359400135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd4102fc0 msgr2=0x7fecd419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd1e00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd4102fc0 0x7fecd419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd1e00640 1 -- 172.21.15.114:0/3359400135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecd4110220 msgr2=0x7fecd41a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd1e00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecd4110220 0x7fecd41a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd1e00640 1 -- 172.21.15.114:0/3359400135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd41a25a0 con 0x7fecd4102670 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd1e00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 0x7fecd419c630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecc400e9f0 tx=0x7fecc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc400cd80 con 0x7fecd4102670 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc40108e0 con 0x7fecd4102670 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd41a2890 con 0x7fecd4102670 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4018980 con 0x7fecd4102670 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.723+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd41a8ff0 con 0x7fecd4102670 2024-09-21T06:43:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.725+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd406c1c0 con 0x7fecd4102670 2024-09-21T06:43:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.727+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fecc4010430 con 0x7fecd4102670 2024-09-21T06:43:52.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.728+0000 7feccaa00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb80b64d0 0x7fecb80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.728+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fecc40e60a0 con 0x7fecd4102670 2024-09-21T06:43:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.729+0000 7feccbe00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb80b64d0 0x7fecb80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.729+0000 7feccbe00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb80b64d0 0x7fecb80b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fecd4103ef0 tx=0x7fecbc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.730+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fecc4026020 con 0x7fecd4102670 2024-09-21T06:43:52.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.933+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fecd4102a80 con 0x7fecd4102670 2024-09-21T06:43:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.938+0000 7feccaa00640 1 -- 172.21.15.114:0/3359400135 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7fecc40b2910 con 0x7fecd4102670 2024-09-21T06:43:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb80b64d0 msgr2=0x7fecb80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb80b64d0 0x7fecb80b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fecd4103ef0 tx=0x7fecbc005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 msgr2=0x7fecd419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 0x7fecd419c630 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecc400e9f0 tx=0x7fecc400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 shutdown_connections 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb80b64d0 0x7fecb80b89e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecd4110220 0x7fecd41a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fecd4102fc0 0x7fecd419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 --2- 172.21.15.114:0/3359400135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fecd4102670 0x7fecd419c630 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 >> 172.21.15.114:0/3359400135 conn(0x7fecd4079740 msgr2=0x7fecd41114c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.940+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 shutdown_connections 2024-09-21T06:43:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:52.941+0000 7fecd8c00640 1 -- 172.21.15.114:0/3359400135 wait complete. 2024-09-21T06:43:54.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:53 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3359400135' entity='client.admin' 2024-09-21T06:43:54.263 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:53 smithi114 ceph-mon[30654]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:54.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 -- 172.21.15.114:0/3908498659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481101a0 msgr2=0x7fcf481125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3908498659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481101a0 0x7fcf481125c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009ca0 tx=0x7fcf4002a950 comp rx=0 tx=0).stop 2024-09-21T06:43:54.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 -- 172.21.15.114:0/3908498659 shutdown_connections 2024-09-21T06:43:54.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3908498659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481101a0 0x7fcf481125c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3908498659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf48102f40 0x7fcf4810fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3908498659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf481025f0 0x7fcf48102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 -- 172.21.15.114:0/3908498659 >> 172.21.15.114:0/3908498659 conn(0x7fcf48079740 msgr2=0x7fcf48079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:54.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 -- 172.21.15.114:0/3908498659 shutdown_connections 2024-09-21T06:43:54.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.347+0000 7fcf4d000640 1 -- 172.21.15.114:0/3908498659 wait complete. 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.348+0000 7fcf4d000640 1 Processor -- start 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 -- start start 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481025f0 0x7fcf4819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 0x7fcf4819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf481101a0 0x7fcf481a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48115ce0 con 0x7fcf481025f0 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48115b60 con 0x7fcf48102f40 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf4d000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48115e60 con 0x7fcf481101a0 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf45e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481025f0 0x7fcf4819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf45400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 0x7fcf4819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf45e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481025f0 0x7fcf4819c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51168/0 (socket says 172.21.15.114:51168) 2024-09-21T06:43:54.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf45e00640 1 -- 172.21.15.114:0/3194828027 learned_addr learned my addr 172.21.15.114:0/3194828027 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:54.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf45400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 0x7fcf4819cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56692/0 (socket says 172.21.15.114:56692) 2024-09-21T06:43:54.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.349+0000 7fcf46800640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf481101a0 0x7fcf481a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf45400640 1 -- 172.21.15.114:0/3194828027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf481101a0 msgr2=0x7fcf481a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf45400640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf481101a0 0x7fcf481a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf45400640 1 -- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481025f0 msgr2=0x7fcf4819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf45400640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481025f0 0x7fcf4819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf45400640 1 -- 172.21.15.114:0/3194828027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf481a2560 con 0x7fcf48102f40 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf46800640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf481101a0 0x7fcf481a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf45400640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 0x7fcf4819cb90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf34005fd0 tx=0x7fcf3400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34009c30 con 0x7fcf48102f40 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf34009dd0 con 0x7fcf48102f40 2024-09-21T06:43:54.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34010460 con 0x7fcf48102f40 2024-09-21T06:43:54.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.350+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf481a2850 con 0x7fcf48102f40 2024-09-21T06:43:54.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.351+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf481a8fb0 con 0x7fcf48102f40 2024-09-21T06:43:54.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.352+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4806c240 con 0x7fcf48102f40 2024-09-21T06:43:54.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.354+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fcf34003e80 con 0x7fcf48102f40 2024-09-21T06:43:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.355+0000 7fcf3ea00640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcf2c0b6550 0x7fcf2c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.355+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcf340e56a0 con 0x7fcf48102f40 2024-09-21T06:43:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.355+0000 7fcf45e00640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcf2c0b6550 0x7fcf2c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.356+0000 7fcf45e00640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcf2c0b6550 0x7fcf2c0b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf380095f0 tx=0x7fcf38005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:54.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.359+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf34017020 con 0x7fcf48102f40 2024-09-21T06:43:54.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:53 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3359400135' entity='client.admin' 2024-09-21T06:43:54.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:53 smithi183 ceph-mon[34219]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:54.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:53 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3359400135' entity='client.admin' 2024-09-21T06:43:54.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:53 smithi193 ceph-mon[34601]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:54.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.552+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fcf481a9260 con 0x7fcf48102f40 2024-09-21T06:43:54.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.559+0000 7fcf3ea00640 1 -- 172.21.15.114:0/3194828027 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7fcf340ada90 con 0x7fcf48102f40 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcf2c0b6550 msgr2=0x7fcf2c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcf2c0b6550 0x7fcf2c0b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcf380095f0 tx=0x7fcf38005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 msgr2=0x7fcf4819cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 0x7fcf4819cb90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf34005fd0 tx=0x7fcf3400dde0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 shutdown_connections 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf481101a0 0x7fcf481a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcf2c0b6550 0x7fcf2c0b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf48102f40 0x7fcf4819cb90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 --2- 172.21.15.114:0/3194828027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf481025f0 0x7fcf4819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 >> 172.21.15.114:0/3194828027 conn(0x7fcf48079740 msgr2=0x7fcf48109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:54.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 shutdown_connections 2024-09-21T06:43:54.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:54.561+0000 7fcf4d000640 1 -- 172.21.15.114:0/3194828027 wait complete. 2024-09-21T06:43:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:54.712 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:43:54.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 -- 172.21.15.114:0/1489037142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8141039b0 msgr2=0x7fb814103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 --2- 172.21.15.114:0/1489037142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8141039b0 0x7fb814103dc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb804009860 tx=0x7fb80402a340 comp rx=0 tx=0).stop 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 -- 172.21.15.114:0/1489037142 shutdown_connections 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 --2- 172.21.15.114:0/1489037142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb814109c40 0x7fb814110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 --2- 172.21.15.114:0/1489037142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141092f0 0x7fb814109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 --2- 172.21.15.114:0/1489037142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8141039b0 0x7fb814103dc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 -- 172.21.15.114:0/1489037142 >> 172.21.15.114:0/1489037142 conn(0x7fb8140ff440 msgr2=0x7fb814101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 -- 172.21.15.114:0/1489037142 shutdown_connections 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.545+0000 7fb818a00640 1 -- 172.21.15.114:0/1489037142 wait complete. 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.546+0000 7fb818a00640 1 Processor -- start 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 -- start start 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 0x7fb81419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8141092f0 0x7fb81419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb814109c40 0x7fb81419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb814113900 con 0x7fb8141039b0 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb814113780 con 0x7fb814109c40 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb818a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb814113a80 con 0x7fb8141092f0 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb812800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 0x7fb81419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb812800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 0x7fb81419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51134/0 (socket says 172.21.15.114:51134) 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb812800640 1 -- 172.21.15.114:0/374661773 learned_addr learned my addr 172.21.15.114:0/374661773 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.547+0000 7fb813200640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb814109c40 0x7fb81419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb811e00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8141092f0 0x7fb81419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb812800640 1 -- 172.21.15.114:0/374661773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8141092f0 msgr2=0x7fb81419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb812800640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8141092f0 0x7fb81419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb812800640 1 -- 172.21.15.114:0/374661773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb814109c40 msgr2=0x7fb81419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb812800640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb814109c40 0x7fb81419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb812800640 1 -- 172.21.15.114:0/374661773 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8141a02a0 con 0x7fb8141039b0 2024-09-21T06:43:54.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb812800640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 0x7fb81419a330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb8040303f0 tx=0x7fb804032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.548+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804040020 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.549+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804030450 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.549+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8141a0530 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.549+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804037040 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.549+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8141a6cf0 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.550+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb814103dc0 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.552+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fb804043020 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.554+0000 7fb80b400640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7f80b64d0 0x7fb7f80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.554+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb804105530 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.554+0000 7fb811e00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7f80b64d0 0x7fb7f80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.555+0000 7fb811e00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7f80b64d0 0x7fb7f80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb814108ed0 tx=0x7fb800009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.558+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb80404a020 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.751+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fb81410ba10 con 0x7fb8141039b0 2024-09-21T06:43:54.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.756+0000 7fb80b400640 1 -- 172.21.15.114:0/374661773 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v28)=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fb8040cd920 con 0x7fb8141039b0 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.758+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7f80b64d0 msgr2=0x7fb7f80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.758+0000 7fb818a00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7f80b64d0 0x7fb7f80b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb814108ed0 tx=0x7fb800009260 comp rx=0 tx=0).stop 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.758+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 msgr2=0x7fb81419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.758+0000 7fb818a00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 0x7fb81419a330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb8040303f0 tx=0x7fb804032eb0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 shutdown_connections 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7f80b64d0 0x7fb7f80b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb814109c40 0x7fb81419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8141092f0 0x7fb81419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 --2- 172.21.15.114:0/374661773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb8141039b0 0x7fb81419a330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 >> 172.21.15.114:0/374661773 conn(0x7fb8140ff440 msgr2=0x7fb814101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 shutdown_connections 2024-09-21T06:43:54.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:50.759+0000 7fb818a00640 1 -- 172.21.15.114:0/374661773 wait complete. 2024-09-21T06:43:55.710 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:55 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:55 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:55 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:56.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.526+0000 7f5988c00640 1 -- 172.21.15.114:0/614588121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984103b80 msgr2=0x7f5984103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.526+0000 7f5988c00640 1 --2- 172.21.15.114:0/614588121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984103b80 0x7f5984103f90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5974009900 tx=0x7f597402a410 comp rx=0 tx=0).stop 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 -- 172.21.15.114:0/614588121 shutdown_connections 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 --2- 172.21.15.114:0/614588121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5984109e10 0x7f5984110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 --2- 172.21.15.114:0/614588121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59841094c0 0x7f59841098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 --2- 172.21.15.114:0/614588121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984103b80 0x7f5984103f90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 -- 172.21.15.114:0/614588121 >> 172.21.15.114:0/614588121 conn(0x7f59840ff590 msgr2=0x7f59841019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 -- 172.21.15.114:0/614588121 shutdown_connections 2024-09-21T06:43:56.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.527+0000 7f5988c00640 1 -- 172.21.15.114:0/614588121 wait complete. 2024-09-21T06:43:56.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.528+0000 7f5988c00640 1 Processor -- start 2024-09-21T06:43:56.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.528+0000 7f5988c00640 1 -- start start 2024-09-21T06:43:56.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5988c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5984103b80 0x7f598419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5988c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 0x7f598419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5988c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984109e10 0x7f598419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5988c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5984113ad0 con 0x7f5984109e10 2024-09-21T06:43:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5988c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5984113950 con 0x7f59841094c0 2024-09-21T06:43:56.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5988c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5984113c50 con 0x7f5984103b80 2024-09-21T06:43:56.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5981e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5984103b80 0x7f598419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:56.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5981400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 0x7f598419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:56.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5981400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 0x7f598419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56080/0 (socket says 172.21.15.114:56080) 2024-09-21T06:43:56.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5981400640 1 -- 172.21.15.114:0/1351608697 learned_addr learned my addr 172.21.15.114:0/1351608697 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.529+0000 7f5982800640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984109e10 0x7f598419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5981400640 1 -- 172.21.15.114:0/1351608697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5984103b80 msgr2=0x7f598419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5981400640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5984103b80 0x7f598419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5981400640 1 -- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984109e10 msgr2=0x7f598419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5981400640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984109e10 0x7f598419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5981400640 1 -- 172.21.15.114:0/1351608697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59841a04d0 con 0x7f59841094c0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5981400640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 0x7f598419ab00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f597000e9c0 tx=0x7f597000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597000ccc0 con 0x7f59841094c0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59700054d0 con 0x7f59841094c0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59841a07c0 con 0x7f59841094c0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5970010430 con 0x7f59841094c0 2024-09-21T06:43:56.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.530+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841a6f20 con 0x7f59841094c0 2024-09-21T06:43:56.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.531+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5984103f90 con 0x7f59841094c0 2024-09-21T06:43:56.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.534+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f5970005020 con 0x7f59841094c0 2024-09-21T06:43:56.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.535+0000 7f597aa00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f59680b64d0 0x7f59680b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:56.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.535+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f59700e5b50 con 0x7f59841094c0 2024-09-21T06:43:56.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.535+0000 7f5981e00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f59680b64d0 0x7f59680b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:56.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.536+0000 7f5981e00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f59680b64d0 0x7f59680b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5974031440 tx=0x7f5974009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:56.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.539+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f59700adf40 con 0x7f59841094c0 2024-09-21T06:43:56.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.737+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f59841a7200 con 0x7f59841094c0 2024-09-21T06:43:56.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.743+0000 7f597aa00640 1 -- 172.21.15.114:0/1351608697 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v31)=0 v31) ==== 120+0+0 (secure 0 0 0) 0x7f59700b2320 con 0x7f59841094c0 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f59680b64d0 msgr2=0x7f59680b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f59680b64d0 0x7f59680b89e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5974031440 tx=0x7f5974009560 comp rx=0 tx=0).stop 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 msgr2=0x7f598419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 0x7f598419ab00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f597000e9c0 tx=0x7f597000ee80 comp rx=0 tx=0).stop 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 shutdown_connections 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5984109e10 0x7f598419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f59680b64d0 0x7f59680b89e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59841094c0 0x7f598419ab00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 --2- 172.21.15.114:0/1351608697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5984103b80 0x7f598419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 >> 172.21.15.114:0/1351608697 conn(0x7f59840ff590 msgr2=0x7f5984107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 shutdown_connections 2024-09-21T06:43:56.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:56.746+0000 7f5988c00640 1 -- 172.21.15.114:0/1351608697 wait complete. 2024-09-21T06:43:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:56 smithi114 ceph-mon[30654]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:56 smithi183 ceph-mon[34219]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:56.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:56 smithi193 ceph-mon[34601]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:58.016 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:57 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:58.016 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:57 smithi114 ceph-mon[30654]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:58.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:57 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:58.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:57 smithi183 ceph-mon[34219]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:57 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:57 smithi193 ceph-mon[34601]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:43:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 -- 172.21.15.114:0/3097680595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c41047d0 msgr2=0x7f50c4104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3097680595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c41047d0 0x7f50c4104be0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009900 tx=0x7f50b402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 -- 172.21.15.114:0/3097680595 shutdown_connections 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3097680595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c410dc30 0x7f50c4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3097680595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c4105120 0x7f50c410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3097680595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c41047d0 0x7f50c4104be0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 -- 172.21.15.114:0/3097680595 >> 172.21.15.114:0/3097680595 conn(0x7f50c40fe2d0 msgr2=0x7f50c4100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.813+0000 7f50c9c00640 1 -- 172.21.15.114:0/3097680595 shutdown_connections 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.814+0000 7f50c9c00640 1 -- 172.21.15.114:0/3097680595 wait complete. 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 Processor -- start 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 -- start start 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 0x7f50c419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c4105120 0x7f50c419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c410dc30 0x7f50c419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4113900 con 0x7f50c410dc30 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4113780 con 0x7f50c4105120 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50c4113a80 con 0x7f50c41047d0 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c410dc30 0x7f50c419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 0x7f50c419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c4105120 0x7f50c419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 0x7f50c419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45336/0 (socket says 172.21.15.114:45336) 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50c3e00640 1 -- 172.21.15.114:0/3764624688 learned_addr learned my addr 172.21.15.114:0/3764624688 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.815+0000 7f50bbe00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c4105120 0x7f50c419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56120/0 (socket says 172.21.15.114:56120) 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c3e00640 1 -- 172.21.15.114:0/3764624688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c4105120 msgr2=0x7f50c419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c3e00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c4105120 0x7f50c419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c3e00640 1 -- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c410dc30 msgr2=0x7f50c419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c3e00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c410dc30 0x7f50c419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c3e00640 1 -- 172.21.15.114:0/3764624688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c41a02a0 con 0x7f50c41047d0 2024-09-21T06:43:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c3e00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 0x7f50c419a330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005ab0 tx=0x7f50b4003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b403f070 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b4031d60 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c41a0530 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.816+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4035030 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.817+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c41a6cf0 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.818+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c4104be0 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.820+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f50b4042070 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.821+0000 7f50c1600640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50a80b6550 0x7f50a80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.821+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f50b4105eb0 con 0x7f50c41047d0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.821+0000 7f50bbe00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50a80b6550 0x7f50a80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.822+0000 7f50bbe00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50a80b6550 0x7f50a80b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f50c41101e0 tx=0x7f50ac00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.823+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50b40d2680 con 0x7f50c41047d0 2024-09-21T06:43:59.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.025+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f50c41a6f20 con 0x7f50c41047d0 2024-09-21T06:43:59.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.031+0000 7f50c1600640 1 -- 172.21.15.114:0/3764624688 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7f50b40d75c0 con 0x7f50c41047d0 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50a80b6550 msgr2=0x7f50a80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50a80b6550 0x7f50a80b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f50c41101e0 tx=0x7f50ac00b390 comp rx=0 tx=0).stop 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 msgr2=0x7f50c419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 0x7f50c419a330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005ab0 tx=0x7f50b4003cd0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 shutdown_connections 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f50c410dc30 0x7f50c419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50a80b6550 0x7f50a80b8a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50c4105120 0x7f50c419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 --2- 172.21.15.114:0/3764624688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50c41047d0 0x7f50c419a330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 >> 172.21.15.114:0/3764624688 conn(0x7f50c40fe2d0 msgr2=0x7f50c4107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.034+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 shutdown_connections 2024-09-21T06:43:59.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.035+0000 7f50c9c00640 1 -- 172.21.15.114:0/3764624688 wait complete. 2024-09-21T06:43:59.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/686894741' entity='client.admin' 2024-09-21T06:43:59.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:59.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:59.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:59.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:59.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:59.518 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:43:59 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 -- 172.21.15.114:0/2371352084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410b6d0 msgr2=0x7f5cb410bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2371352084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410b6d0 0x7f5cb410bae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009900 tx=0x7f5ca402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 -- 172.21.15.114:0/2371352084 shutdown_connections 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2371352084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb410c020 0x7f5cb4112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2371352084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410b6d0 0x7f5cb410bae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2371352084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cb4105d90 0x7f5cb41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 -- 172.21.15.114:0/2371352084 >> 172.21.15.114:0/2371352084 conn(0x7f5cb41017c0 msgr2=0x7f5cb4103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 -- 172.21.15.114:0/2371352084 shutdown_connections 2024-09-21T06:43:59.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.597+0000 7f5cbc800640 1 -- 172.21.15.114:0/2371352084 wait complete. 2024-09-21T06:43:59.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.598+0000 7f5cbc800640 1 Processor -- start 2024-09-21T06:43:59.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.598+0000 7f5cbc800640 1 -- start start 2024-09-21T06:43:59.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.598+0000 7f5cbc800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cb4105d90 0x7f5cb419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:59.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cbc800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb410b6d0 0x7f5cb419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:59.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cbc800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 0x7f5cb41a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:59.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cbc800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4115d30 con 0x7f5cb410c020 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cbc800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4115bb0 con 0x7f5cb410b6d0 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cbc800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4115eb0 con 0x7f5cb4105d90 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 0x7f5cb41a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 0x7f5cb41a1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38482/0 (socket says 172.21.15.114:38482) 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 -- 172.21.15.114:0/2532841520 learned_addr learned my addr 172.21.15.114:0/2532841520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cb9e00640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cb4105d90 0x7f5cb419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cb9400640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb410b6d0 0x7f5cb419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 -- 172.21.15.114:0/2532841520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cb4105d90 msgr2=0x7f5cb419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cb4105d90 0x7f5cb419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 -- 172.21.15.114:0/2532841520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb410b6d0 msgr2=0x7f5cb419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb410b6d0 0x7f5cb419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.599+0000 7f5cba800640 1 -- 172.21.15.114:0/2532841520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb41a2710 con 0x7f5cb410c020 2024-09-21T06:43:59.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.600+0000 7f5cba800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 0x7f5cb41a1fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000efd0 tx=0x7f5cb000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.600+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0003b70 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.600+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb0003d10 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.600+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00108a0 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.600+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb41a2a00 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.600+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb41a9160 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.601+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb41061a0 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.604+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f5cb0004b40 con 0x7f5cb410c020 2024-09-21T06:43:59.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.605+0000 7f5caea00640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c9c0b6550 0x7f5c9c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:43:59.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.605+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5cb00e5ef0 con 0x7f5cb410c020 2024-09-21T06:43:59.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.605+0000 7f5cb9e00640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c9c0b6550 0x7f5c9c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:43:59.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.606+0000 7f5cb9e00640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c9c0b6550 0x7f5c9c0b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009bd0 tx=0x7f5ca8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:43:59.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.607+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5cb00ae2e0 con 0x7f5cb410c020 2024-09-21T06:43:59.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/686894741' entity='client.admin' 2024-09-21T06:43:59.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:43:59 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/686894741' entity='client.admin' 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:43:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:43:59 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:43:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.824+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f5cb402e640 con 0x7f5cb410c020 2024-09-21T06:43:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.829+0000 7f5caea00640 1 -- 172.21.15.114:0/2532841520 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v34)=0 v34) ==== 121+0+0 (secure 0 0 0) 0x7f5cb00b26c0 con 0x7f5cb410c020 2024-09-21T06:43:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c9c0b6550 msgr2=0x7f5c9c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c9c0b6550 0x7f5c9c0b8a60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009bd0 tx=0x7f5ca8005f90 comp rx=0 tx=0).stop 2024-09-21T06:43:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 msgr2=0x7f5cb41a1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 0x7f5cb41a1fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000efd0 tx=0x7f5cb000c6a0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 shutdown_connections 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cb410c020 0x7f5cb41a1fe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c9c0b6550 0x7f5c9c0b8a60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb410b6d0 0x7f5cb419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 --2- 172.21.15.114:0/2532841520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cb4105d90 0x7f5cb419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:43:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 >> 172.21.15.114:0/2532841520 conn(0x7f5cb41017c0 msgr2=0x7f5cb410ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:43:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 shutdown_connections 2024-09-21T06:43:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:59.831+0000 7f5cbc800640 1 -- 172.21.15.114:0/2532841520 wait complete. 2024-09-21T06:44:00.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 -- 172.21.15.114:0/2104278390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c109c20 msgr2=0x7faf1c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 --2- 172.21.15.114:0/2104278390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c109c20 0x7faf1c110450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf14009ca0 tx=0x7faf1402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 -- 172.21.15.114:0/2104278390 shutdown_connections 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 --2- 172.21.15.114:0/2104278390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c109c20 0x7faf1c110450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 --2- 172.21.15.114:0/2104278390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf1c1092d0 0x7faf1c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 --2- 172.21.15.114:0/2104278390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c103990 0x7faf1c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 -- 172.21.15.114:0/2104278390 >> 172.21.15.114:0/2104278390 conn(0x7faf1c0ff3e0 msgr2=0x7faf1c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.085+0000 7faf1be00640 1 -- 172.21.15.114:0/2104278390 shutdown_connections 2024-09-21T06:44:00.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.086+0000 7faf1be00640 1 -- 172.21.15.114:0/2104278390 wait complete. 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 Processor -- start 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 -- start start 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 0x7faf1c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf1c1092d0 0x7faf1c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c109c20 0x7faf1c1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c113930 con 0x7faf1c103990 2024-09-21T06:44:00.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c1137b0 con 0x7faf1c109c20 2024-09-21T06:44:00.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c113ab0 con 0x7faf1c1092d0 2024-09-21T06:44:00.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.087+0000 7faf1aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 0x7faf1c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:00.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 0x7faf1c19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38498/0 (socket says 172.21.15.114:38498) 2024-09-21T06:44:00.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c109c20 0x7faf1c1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:00.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 -- 172.21.15.114:0/1137985848 learned_addr learned my addr 172.21.15.114:0/1137985848 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:44:00.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1a000640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf1c1092d0 0x7faf1c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 -- 172.21.15.114:0/1137985848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf1c1092d0 msgr2=0x7faf1c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf1c1092d0 0x7faf1c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 -- 172.21.15.114:0/1137985848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c109c20 msgr2=0x7faf1c1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c109c20 0x7faf1c1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 -- 172.21.15.114:0/1137985848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c1a24b0 con 0x7faf1c103990 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1b400640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c109c20 0x7faf1c1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.088+0000 7faf1aa00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 0x7faf1c19c540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e9f0 tx=0x7faf0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.089+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00cd80 con 0x7faf1c103990 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.089+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c0108e0 con 0x7faf1c103990 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.089+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c018980 con 0x7faf1c103990 2024-09-21T06:44:00.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.089+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c1a27a0 con 0x7faf1c103990 2024-09-21T06:44:00.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.089+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c1a8f00 con 0x7faf1c103990 2024-09-21T06:44:00.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.091+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1c103da0 con 0x7faf1c103990 2024-09-21T06:44:00.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.092+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7faf0c010430 con 0x7faf1c103990 2024-09-21T06:44:00.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.094+0000 7faf13400640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faf000b6550 0x7faf000b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:00.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.094+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7faf0c0e59e0 con 0x7faf1c103990 2024-09-21T06:44:00.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.094+0000 7faf1a000640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faf000b6550 0x7faf000b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:00.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.095+0000 7faf1a000640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faf000b6550 0x7faf000b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf1c109b90 tx=0x7faf08005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:00.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.098+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faf0c0b2250 con 0x7faf1c103990 2024-09-21T06:44:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.291+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7faf1c19d670 con 0x7faf1c103990 2024-09-21T06:44:00.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.296+0000 7faf13400640 1 -- 172.21.15.114:0/1137985848 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v35)=0 v35) ==== 120+0+0 (secure 0 0 0) 0x7faf0c0b7190 con 0x7faf1c103990 2024-09-21T06:44:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.298+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faf000b6550 msgr2=0x7faf000b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.298+0000 7faf1be00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faf000b6550 0x7faf000b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf1c109b90 tx=0x7faf08005f90 comp rx=0 tx=0).stop 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 msgr2=0x7faf1c19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 0x7faf1c19c540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00e9f0 tx=0x7faf0c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 shutdown_connections 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faf000b6550 0x7faf000b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf1c109c20 0x7faf1c1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf1c1092d0 0x7faf1c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 --2- 172.21.15.114:0/1137985848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faf1c103990 0x7faf1c19c540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 >> 172.21.15.114:0/1137985848 conn(0x7faf1c0ff3e0 msgr2=0x7faf1c108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 shutdown_connections 2024-09-21T06:44:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.299+0000 7faf1be00640 1 -- 172.21.15.114:0/1137985848 wait complete. 2024-09-21T06:44:00.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:44:00.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: Reconfiguring daemon mon.b on smithi183 2024-09-21T06:44:00.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2532841520' entity='client.admin' 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:00.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:00 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1137985848' entity='client.admin' 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: Reconfiguring daemon mon.b on smithi183 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2532841520' entity='client.admin' 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:00.642 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:00 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1137985848' entity='client.admin' 2024-09-21T06:44:00.678 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi114') 2024-09-21T06:44:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: Reconfiguring mon.b (monmap changed)... 2024-09-21T06:44:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: Reconfiguring daemon mon.b on smithi183 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2532841520' entity='client.admin' 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:00 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1137985848' entity='client.admin' 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 -- 172.21.15.114:0/969025048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01039b0 msgr2=0x7f53d0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 --2- 172.21.15.114:0/969025048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01039b0 0x7f53d0103dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009900 tx=0x7f53c002a410 comp rx=0 tx=0).stop 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 -- 172.21.15.114:0/969025048 shutdown_connections 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 --2- 172.21.15.114:0/969025048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d0109c40 0x7f53d0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 --2- 172.21.15.114:0/969025048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53d01092f0 0x7f53d0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 --2- 172.21.15.114:0/969025048 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01039b0 0x7f53d0103dc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 -- 172.21.15.114:0/969025048 >> 172.21.15.114:0/969025048 conn(0x7f53d00ff440 msgr2=0x7f53d0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 -- 172.21.15.114:0/969025048 shutdown_connections 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.194+0000 7f53cf200640 1 -- 172.21.15.114:0/969025048 wait complete. 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.195+0000 7f53cf200640 1 Processor -- start 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 -- start start 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d01039b0 0x7f53d019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 0x7f53d019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53d0109c40 0x7f53d01a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0113900 con 0x7f53d01092f0 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0113780 con 0x7f53d0109c40 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cf200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0113a80 con 0x7f53d01039b0 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 0x7f53d019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 0x7f53d019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38438/0 (socket says 172.21.15.114:38438) 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cd400640 1 -- 172.21.15.114:0/686894741 learned_addr learned my addr 172.21.15.114:0/686894741 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53cde00640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d01039b0 0x7f53d019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.196+0000 7f53ce800640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53d0109c40 0x7f53d01a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cd400640 1 -- 172.21.15.114:0/686894741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d01039b0 msgr2=0x7f53d019c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cd400640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d01039b0 0x7f53d019c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cd400640 1 -- 172.21.15.114:0/686894741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53d0109c40 msgr2=0x7f53d01a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cd400640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53d0109c40 0x7f53d01a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cd400640 1 -- 172.21.15.114:0/686894741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d01a24a0 con 0x7f53d01092f0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cd400640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 0x7f53d019cad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00e9f0 tx=0x7f53bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc00ccc0 con 0x7f53d01092f0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc0108e0 con 0x7f53d01092f0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d01a2790 con 0x7f53d01092f0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc018860 con 0x7f53d01092f0 2024-09-21T06:44:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.197+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01a8ef0 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.198+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d0103dc0 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.201+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f53bc010430 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.203+0000 7f53c6a00640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53b40b64d0 0x7f53b40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.203+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f53bc0e5f20 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.203+0000 7f53cde00640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53b40b64d0 0x7f53b40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.204+0000 7f53cde00640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53b40b64d0 0x7f53b40b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53c0031440 tx=0x7f53c0009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.204+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53bc0b26f0 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.400+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f53d01a91d0 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.406+0000 7f53c6a00640 1 -- 172.21.15.114:0/686894741 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v32)=0 v32) ==== 116+0+0 (secure 0 0 0) 0x7f53bc0b7630 con 0x7f53d01092f0 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53b40b64d0 msgr2=0x7f53b40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53b40b64d0 0x7f53b40b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53c0031440 tx=0x7f53c0009560 comp rx=0 tx=0).stop 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 msgr2=0x7f53d019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 0x7f53d019cad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f53bc00e9f0 tx=0x7f53bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 shutdown_connections 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53b40b64d0 0x7f53b40b89e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53d0109c40 0x7f53d01a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53d01092f0 0x7f53d019cad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 --2- 172.21.15.114:0/686894741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53d01039b0 0x7f53d019c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.408+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 >> 172.21.15.114:0/686894741 conn(0x7f53d00ff440 msgr2=0x7f53d01089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.409+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 shutdown_connections 2024-09-21T06:44:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:43:58.409+0000 7f53cf200640 1 -- 172.21.15.114:0/686894741 wait complete. 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: Reconfiguring daemon mon.c on smithi193 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:01.779 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:44:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: Reconfiguring daemon mon.c on smithi193 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: Reconfiguring mon.c (monmap changed)... 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: Reconfiguring daemon mon.c on smithi193 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:02.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 -- 172.21.15.114:0/945099963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141039b0 msgr2=0x7f2414103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 --2- 172.21.15.114:0/945099963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141039b0 0x7f2414103dc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2404009860 tx=0x7f240402a340 comp rx=0 tx=0).stop 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 -- 172.21.15.114:0/945099963 shutdown_connections 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 --2- 172.21.15.114:0/945099963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2414109c40 0x7f2414110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 --2- 172.21.15.114:0/945099963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141092f0 0x7f2414109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 --2- 172.21.15.114:0/945099963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141039b0 0x7f2414103dc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 -- 172.21.15.114:0/945099963 >> 172.21.15.114:0/945099963 conn(0x7f24140ff440 msgr2=0x7f2414101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 -- 172.21.15.114:0/945099963 shutdown_connections 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.327+0000 7f2419a00640 1 -- 172.21.15.114:0/945099963 wait complete. 2024-09-21T06:44:02.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.329+0000 7f2419a00640 1 Processor -- start 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.329+0000 7f2419a00640 1 -- start start 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.329+0000 7f2419a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141039b0 0x7f241419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.329+0000 7f2419a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 0x7f241419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.329+0000 7f2419a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2414109c40 0x7f241419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2419a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2414113900 con 0x7f2414109c40 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2419a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2414113780 con 0x7f24141039b0 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2419a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2414113a80 con 0x7f24141092f0 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 0x7f241419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 0x7f241419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45370/0 (socket says 172.21.15.114:45370) 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413400640 1 -- 172.21.15.114:0/1010722786 learned_addr learned my addr 172.21.15.114:0/1010722786 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2419000640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2414109c40 0x7f241419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413e00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141039b0 0x7f241419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413e00640 1 -- 172.21.15.114:0/1010722786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 msgr2=0x7f241419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413e00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 0x7f241419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413e00640 1 -- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2414109c40 msgr2=0x7f241419fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413e00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2414109c40 0x7f241419fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413e00640 1 -- 172.21.15.114:0/1010722786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24141a0280 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.330+0000 7f2413400640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 0x7f241419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.331+0000 7f2413e00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141039b0 0x7f241419a310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2404035040 tx=0x7f2404032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.331+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404040020 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.331+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2404030e10 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.331+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404037040 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.331+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24141a0510 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.331+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24141a6cd0 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.333+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2414103dc0 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.334+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f2404043020 con 0x7f24141039b0 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.336+0000 7f2410c00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f23f80b6550 0x7f23f80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.336+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f24041051d0 con 0x7f24141039b0 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.336+0000 7f2413400640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f23f80b6550 0x7f23f80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.337+0000 7f2413400640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f23f80b6550 0x7f23f80b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2414108e80 tx=0x7f2400005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.340+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f240404a020 con 0x7f24141039b0 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.536+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f241410ba10 con 0x7f24141039b0 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.542+0000 7f2410c00640 1 -- 172.21.15.114:0/1010722786 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v36)=0 v36) ==== 129+0+0 (secure 0 0 0) 0x7f24040cd5c0 con 0x7f24141039b0 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f23f80b6550 msgr2=0x7f23f80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f23f80b6550 0x7f23f80b8a60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2414108e80 tx=0x7f2400005f90 comp rx=0 tx=0).stop 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141039b0 msgr2=0x7f241419a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141039b0 0x7f241419a310 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2404035040 tx=0x7f2404032d50 comp rx=0 tx=0).stop 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 shutdown_connections 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2414109c40 0x7f241419fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24141092f0 0x7f241419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f23f80b6550 0x7f23f80b8a60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 --2- 172.21.15.114:0/1010722786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24141039b0 0x7f241419a310 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:02.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 >> 172.21.15.114:0/1010722786 conn(0x7f24140ff440 msgr2=0x7f2414101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:02.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 shutdown_connections 2024-09-21T06:44:02.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:00.545+0000 7f2419a00640 1 -- 172.21.15.114:0/1010722786 wait complete. 2024-09-21T06:44:02.703 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:02 smithi114 ceph-mon[30654]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:02.704 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:02 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' 2024-09-21T06:44:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:02 smithi183 ceph-mon[34219]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:02 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' 2024-09-21T06:44:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:02 smithi193 ceph-mon[34601]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:02 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' 2024-09-21T06:44:04.215 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi114') 2024-09-21T06:44:04.215 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi114') 2024-09-21T06:44:04.215 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi114') 2024-09-21T06:44:04.215 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi114') 2024-09-21T06:44:04.215 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi114') 2024-09-21T06:44:04.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 -- 172.21.15.114:0/475913108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c64105d90 msgr2=0x7f8c641061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:04.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 --2- 172.21.15.114:0/475913108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c64105d90 0x7f8c641061a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009900 tx=0x7f8c5402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 -- 172.21.15.114:0/475913108 shutdown_connections 2024-09-21T06:44:04.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 --2- 172.21.15.114:0/475913108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 0x7f8c64112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 --2- 172.21.15.114:0/475913108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c6410b6d0 0x7f8c6410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 --2- 172.21.15.114:0/475913108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c64105d90 0x7f8c641061a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 -- 172.21.15.114:0/475913108 >> 172.21.15.114:0/475913108 conn(0x7f8c64101860 msgr2=0x7f8c64103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.776+0000 7f8c69000640 1 -- 172.21.15.114:0/475913108 shutdown_connections 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.777+0000 7f8c69000640 1 -- 172.21.15.114:0/475913108 wait complete. 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 Processor -- start 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 -- start start 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c64105d90 0x7f8c6419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c6410b6d0 0x7f8c6419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 0x7f8c641a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64115d30 con 0x7f8c6410c020 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64115bb0 con 0x7f8c6410b6d0 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.778+0000 7f8c69000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64115eb0 con 0x7f8c64105d90 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c62800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 0x7f8c641a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:04.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c62800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 0x7f8c641a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38528/0 (socket says 172.21.15.114:38528) 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c64105d90 0x7f8c6419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c62800640 1 -- 172.21.15.114:0/1088198147 learned_addr learned my addr 172.21.15.114:0/1088198147 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61400640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c6410b6d0 0x7f8c6419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 -- 172.21.15.114:0/1088198147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c6410b6d0 msgr2=0x7f8c6419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c6410b6d0 0x7f8c6419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 -- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 msgr2=0x7f8c641a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 0x7f8c641a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 -- 172.21.15.114:0/1088198147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641a2670 con 0x7f8c64105d90 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.779+0000 7f8c61e00640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c64105d90 0x7f8c6419c700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c5402a9e0 tx=0x7f8c54003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.780+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5403f070 con 0x7f8c64105d90 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.780+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c54031b50 con 0x7f8c64105d90 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.780+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c641a2900 con 0x7f8c64105d90 2024-09-21T06:44:04.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.780+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54035030 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.780+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a90c0 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.781+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c641061a0 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.783+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f8c54042070 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.785+0000 7f8c5aa00640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c480b6550 0x7f8c480b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.786+0000 7f8c61400640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c480b6550 0x7f8c480b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.786+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8c54105930 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.786+0000 7f8c61400640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c480b6550 0x7f8c480b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c500095f0 tx=0x7f8c50005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.788+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8c54031d00 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.988+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f8c6410ddf0 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.996+0000 7f8c5aa00640 1 -- 172.21.15.114:0/1088198147 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v37)=0 v37) ==== 118+0+0 (secure 0 0 0) 0x7f8c540d2100 con 0x7f8c64105d90 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.998+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c480b6550 msgr2=0x7f8c480b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.998+0000 7f8c69000640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c480b6550 0x7f8c480b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c500095f0 tx=0x7f8c50005f90 comp rx=0 tx=0).stop 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.998+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c64105d90 msgr2=0x7f8c6419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:04.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.998+0000 7f8c69000640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c64105d90 0x7f8c6419c700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c5402a9e0 tx=0x7f8c54003b90 comp rx=0 tx=0).stop 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 shutdown_connections 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c6410c020 0x7f8c641a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c480b6550 0x7f8c480b8a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c6410b6d0 0x7f8c6419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 --2- 172.21.15.114:0/1088198147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c64105d90 0x7f8c6419c700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 >> 172.21.15.114:0/1088198147 conn(0x7f8c64101860 msgr2=0x7f8c64103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 shutdown_connections 2024-09-21T06:44:04.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:01.999+0000 7f8c69000640 1 -- 172.21.15.114:0/1088198147 wait complete. 2024-09-21T06:44:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:04 smithi114 ceph-mon[30654]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:04 smithi183 ceph-mon[34219]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:04.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:04 smithi193 ceph-mon[34601]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:05.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi114') 2024-09-21T06:44:05.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi114') 2024-09-21T06:44:05.207 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi114') 2024-09-21T06:44:06.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:06 smithi114 ceph-mon[30654]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:06 smithi183 ceph-mon[34219]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:06 smithi193 ceph-mon[34601]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:06.929 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi114') 2024-09-21T06:44:06.929 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.316 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '5d75913a-77e4-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi114') 2024-09-21T06:44:07.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T06:44:07.677 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:08.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 -- 172.21.15.114:0/2589319509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 msgr2=0x7fe120106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:08.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 --2- 172.21.15.114:0/2589319509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe120106190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe110009900 tx=0x7fe11002a410 comp rx=0 tx=0).stop 2024-09-21T06:44:08.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 -- 172.21.15.114:0/2589319509 shutdown_connections 2024-09-21T06:44:08.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 --2- 172.21.15.114:0/2589319509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12010c010 0x7fe120112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 --2- 172.21.15.114:0/2589319509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12010b6c0 0x7fe12010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 --2- 172.21.15.114:0/2589319509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe120106190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 -- 172.21.15.114:0/2589319509 >> 172.21.15.114:0/2589319509 conn(0x7fe120101810 msgr2=0x7fe120103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:08.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 -- 172.21.15.114:0/2589319509 shutdown_connections 2024-09-21T06:44:08.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.219+0000 7fe126000640 1 -- 172.21.15.114:0/2589319509 wait complete. 2024-09-21T06:44:08.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.220+0000 7fe126000640 1 Processor -- start 2024-09-21T06:44:08.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 -- start start 2024-09-21T06:44:08.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe12019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:08.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12010b6c0 0x7fe12019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12010c010 0x7fe1201a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe120115cd0 con 0x7fe120105d80 2024-09-21T06:44:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe120115b50 con 0x7fe12010b6c0 2024-09-21T06:44:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe126000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe120115e50 con 0x7fe12010c010 2024-09-21T06:44:08.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe11f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe12019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe11f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe12019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58470/0 (socket says 172.21.15.114:58470) 2024-09-21T06:44:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe11f400640 1 -- 172.21.15.114:0/2334616440 learned_addr learned my addr 172.21.15.114:0/2334616440 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:44:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.221+0000 7fe11fe00640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12010c010 0x7fe1201a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe116a00640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12010b6c0 0x7fe12019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11f400640 1 -- 172.21.15.114:0/2334616440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12010c010 msgr2=0x7fe1201a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:08.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11f400640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12010c010 0x7fe1201a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11f400640 1 -- 172.21.15.114:0/2334616440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12010b6c0 msgr2=0x7fe12019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11f400640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12010b6c0 0x7fe12019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11f400640 1 -- 172.21.15.114:0/2334616440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1201a2630 con 0x7fe120105d80 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11f400640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe12019c6c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe110005940 tx=0x7fe1100040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.222+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11003f030 con 0x7fe120105d80 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.223+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1100044c0 con 0x7fe120105d80 2024-09-21T06:44:08.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.223+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110035070 con 0x7fe120105d80 2024-09-21T06:44:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.223+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1201a28c0 con 0x7fe120105d80 2024-09-21T06:44:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.223+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201a9080 con 0x7fe120105d80 2024-09-21T06:44:08.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.224+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe120107000 con 0x7fe120105d80 2024-09-21T06:44:08.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.226+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fe110042030 con 0x7fe120105d80 2024-09-21T06:44:08.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.227+0000 7fe11cc00640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1040b6550 0x7fe1040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:08.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.227+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe110105d10 con 0x7fe120105d80 2024-09-21T06:44:08.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.228+0000 7fe116a00640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1040b6550 0x7fe1040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:08.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.228+0000 7fe116a00640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1040b6550 0x7fe1040b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe12010b150 tx=0x7fe108005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:08.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.230+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1100d24e0 con 0x7fe120105d80 2024-09-21T06:44:08.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.417+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe120106190 con 0x7fe120105d80 2024-09-21T06:44:08.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.418+0000 7fe11cc00640 1 -- 172.21.15.114:0/2334616440 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+21721 (secure 0 0 0) 0x7fe1100c9030 con 0x7fe120105d80 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:global dev auth_debug true 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:global basic log_to_file true 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:global basic log_to_journald false 2024-09-21T06:44:08.421 INFO:teuthology.orchestra.run.smithi114.stdout:global basic log_to_stderr false 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_allow_pool_delete true 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global dev ms_die_on_bug true 2024-09-21T06:44:08.422 INFO:teuthology.orchestra.run.smithi114.stdout:global dev ms_die_on_old_message true 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:global dev ms_die_on_skipped_message false 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced debug_mon 20/20 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced debug_ms 1/1 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced debug_paxos 20/20 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T06:44:08.423 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T06:44:08.424 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced client_mount_timeout 30 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced debug_client 20/20 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced debug_mgr 20/20 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced debug_ms 1/1 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T06:44:08.425 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev bdev_debug_aio true 2024-09-21T06:44:08.426 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced debug_bluefs 20/20 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced debug_bluestore 20/20 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced debug_ms 1/1 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced debug_osd 20/20 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_op_order true 2024-09-21T06:44:08.427 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_shutdown true 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_max_omap_entries_per_request 10000 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T06:44:08.428 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced debug_mds 20/20 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced debug_ms 2/2 2024-09-21T06:44:08.429 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced mds_bal_split_size 100 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds dev mds_debug_frag true 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds dev mds_debug_scatterstat true 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds dev mds_verify_scatter true 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T06:44:08.430 INFO:teuthology.orchestra.run.smithi114.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client advanced client_mount_timeout 600 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client advanced debug_client 20/20 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client advanced debug_ms 1/1 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T06:44:08.431 INFO:teuthology.orchestra.run.smithi114.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T06:44:08.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1040b6550 msgr2=0x7fe1040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:08.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1040b6550 0x7fe1040b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe12010b150 tx=0x7fe108005f90 comp rx=0 tx=0).stop 2024-09-21T06:44:08.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 msgr2=0x7fe12019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:08.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe12019c6c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe110005940 tx=0x7fe1100040c0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 shutdown_connections 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe12010c010 0x7fe1201a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1040b6550 0x7fe1040b8a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe12010b6c0 0x7fe12019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 --2- 172.21.15.114:0/2334616440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe120105d80 0x7fe12019c6c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.421+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 >> 172.21.15.114:0/2334616440 conn(0x7fe120101810 msgr2=0x7fe120103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.422+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 shutdown_connections 2024-09-21T06:44:08.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:08.422+0000 7fe126000640 1 -- 172.21.15.114:0/2334616440 wait complete. 2024-09-21T06:44:08.790 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:08 smithi114 ceph-mon[30654]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:08.790 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:08 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2334616440' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:44:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:08 smithi183 ceph-mon[34219]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:08 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2334616440' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:44:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:08 smithi193 ceph-mon[34601]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:08 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2334616440' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T06:44:09.097 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T06:44:09.097 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:44:09.098 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:44:09.133 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:44:09.133 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_1 2024-09-21T06:44:09.208 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 06:42:42.425633157 +0000 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 06:38:05.712228155 +0000 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 06:38:05.712228155 +0000 2024-09-21T06:44:09.209 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 06:38:05.712228155 +0000 2024-09-21T06:44:09.209 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:44:09.302 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T06:44:09.302 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T06:44:09.302 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000362021 s, 1.4 MB/s 2024-09-21T06:44:09.304 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:44:09.340 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_2 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 657 Links: 1 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 06:42:42.466632246 +0000 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 06:38:06.171217790 +0000 2024-09-21T06:44:09.416 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 06:38:06.171217790 +0000 2024-09-21T06:44:09.417 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 06:38:06.171217790 +0000 2024-09-21T06:44:09.417 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:44:09.509 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T06:44:09.509 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T06:44:09.509 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000270661 s, 1.9 MB/s 2024-09-21T06:44:09.512 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:44:09.549 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_3 2024-09-21T06:44:09.624 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:44:09.624 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:09.624 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-09-21T06:44:09.625 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:09.625 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:09.625 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 06:42:42.505631380 +0000 2024-09-21T06:44:09.625 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 06:38:06.619207674 +0000 2024-09-21T06:44:09.625 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 06:38:06.619207674 +0000 2024-09-21T06:44:09.625 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 06:38:06.619207674 +0000 2024-09-21T06:44:09.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:44:09.716 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T06:44:09.716 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T06:44:09.716 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000254825 s, 2.0 MB/s 2024-09-21T06:44:09.719 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:44:09.755 DEBUG:teuthology.orchestra.run.smithi114:> stat /dev/vg_nvme/lv_4 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout:Access: 2024-09-21 06:42:42.546630469 +0000 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout:Modify: 2024-09-21 06:38:07.117196429 +0000 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout:Change: 2024-09-21 06:38:07.117196429 +0000 2024-09-21T06:44:09.830 INFO:teuthology.orchestra.run.smithi114.stdout: Birth: 2024-09-21 06:38:07.117196429 +0000 2024-09-21T06:44:09.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:44:09.922 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records in 2024-09-21T06:44:09.922 INFO:teuthology.orchestra.run.smithi114.stderr:1+0 records out 2024-09-21T06:44:09.922 INFO:teuthology.orchestra.run.smithi114.stderr:512 bytes copied, 0.000256204 s, 2.0 MB/s 2024-09-21T06:44:09.925 DEBUG:teuthology.orchestra.run.smithi114:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:44:09.961 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-21T06:44:09.961 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:44:09.981 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:44:09.981 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-21 06:43:39.496344499 +0000 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-21 06:37:40.656219748 +0000 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-21 06:37:40.656219748 +0000 2024-09-21T06:44:10.042 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-21 06:37:40.656219748 +0000 2024-09-21T06:44:10.043 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:44:10.113 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-21T06:44:10.113 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-21T06:44:10.113 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000287875 s, 1.8 MB/s 2024-09-21T06:44:10.114 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:44:10.173 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-21T06:44:10.230 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-21 06:43:39.496344499 +0000 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-21 06:37:40.966211030 +0000 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-21 06:37:40.966211030 +0000 2024-09-21T06:44:10.231 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-21 06:37:40.966211030 +0000 2024-09-21T06:44:10.231 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:44:10.296 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-21T06:44:10.296 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-21T06:44:10.296 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000229159 s, 2.2 MB/s 2024-09-21T06:44:10.297 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:44:10.355 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-21 06:43:39.497344471 +0000 2024-09-21T06:44:10.412 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-21 06:37:41.286202030 +0000 2024-09-21T06:44:10.413 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-21 06:37:41.286202030 +0000 2024-09-21T06:44:10.413 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-21 06:37:41.286202030 +0000 2024-09-21T06:44:10.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:44:10.478 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-21T06:44:10.478 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-21T06:44:10.478 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000218229 s, 2.3 MB/s 2024-09-21T06:44:10.479 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:44:10.536 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-21T06:44:10.592 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:44:10.592 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:10.592 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T06:44:10.592 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:10.592 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:10.592 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-21 06:43:39.497344471 +0000 2024-09-21T06:44:10.593 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-21 06:37:41.616192750 +0000 2024-09-21T06:44:10.593 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-21 06:37:41.616192750 +0000 2024-09-21T06:44:10.593 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-21 06:37:41.616192750 +0000 2024-09-21T06:44:10.593 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:44:10.639 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:10 smithi183 ceph-mon[34219]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-21T06:44:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-21T06:44:10.657 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000212824 s, 2.4 MB/s 2024-09-21T06:44:10.658 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:44:10.715 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-21T06:44:10.716 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T06:44:10.735 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T06:44:10.735 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-21T06:44:10.792 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T06:44:10.792 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:10.792 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T06:44:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 06:43:37.231536332 +0000 2024-09-21T06:44:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 06:37:34.346553393 +0000 2024-09-21T06:44:10.793 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 06:37:34.346553393 +0000 2024-09-21T06:44:10.793 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 06:37:34.346553393 +0000 2024-09-21T06:44:10.793 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T06:44:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:10 smithi114 ceph-mon[30654]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:10.856 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:10 smithi193 ceph-mon[34601]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:10.859 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T06:44:10.859 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T06:44:10.859 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000208177 s, 2.5 MB/s 2024-09-21T06:44:10.860 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T06:44:10.917 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-21T06:44:10.973 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T06:44:10.973 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:10.973 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T06:44:10.973 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:10.973 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:10.974 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 06:43:37.232536317 +0000 2024-09-21T06:44:10.974 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 06:37:34.676548204 +0000 2024-09-21T06:44:10.974 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 06:37:34.676548204 +0000 2024-09-21T06:44:10.974 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 06:37:34.676548204 +0000 2024-09-21T06:44:10.974 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T06:44:11.038 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T06:44:11.038 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T06:44:11.038 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000211891 s, 2.4 MB/s 2024-09-21T06:44:11.039 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T06:44:11.095 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 06:43:37.232536317 +0000 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 06:37:35.017542843 +0000 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 06:37:35.017542843 +0000 2024-09-21T06:44:11.151 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 06:37:35.017542843 +0000 2024-09-21T06:44:11.152 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T06:44:11.215 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T06:44:11.215 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T06:44:11.215 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000178187 s, 2.9 MB/s 2024-09-21T06:44:11.217 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T06:44:11.273 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-21 06:43:37.232536317 +0000 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-21 06:37:35.343537717 +0000 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-21 06:37:35.343537717 +0000 2024-09-21T06:44:11.329 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-21 06:37:35.343537717 +0000 2024-09-21T06:44:11.329 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T06:44:11.393 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-21T06:44:11.394 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-21T06:44:11.394 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000214617 s, 2.4 MB/s 2024-09-21T06:44:11.395 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T06:44:11.451 INFO:tasks.cephadm:Deploying osd.0 on smithi114 with /dev/vg_nvme/lv_4... 2024-09-21T06:44:11.452 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:44:11.817 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:12 smithi114 ceph-mon[30654]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:12 smithi183 ceph-mon[34219]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:12 smithi193 ceph-mon[34601]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:14.755 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:14 smithi114 ceph-mon[30654]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:14.762 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:44:14.793 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_4 2024-09-21T06:44:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:14 smithi183 ceph-mon[34219]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:14 smithi193 ceph-mon[34601]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:15.151 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:44:15.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 -- 172.21.15.114:0/206151968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c10faf0 msgr2=0x7f706c10ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 --2- 172.21.15.114:0/206151968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c10faf0 0x7f706c10ff00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7054009900 tx=0x7f705402a410 comp rx=0 tx=0).stop 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 -- 172.21.15.114:0/206151968 shutdown_connections 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 --2- 172.21.15.114:0/206151968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f706c110440 0x7f706c116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 --2- 172.21.15.114:0/206151968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c10faf0 0x7f706c10ff00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 --2- 172.21.15.114:0/206151968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c10a1b0 0x7f706c10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 -- 172.21.15.114:0/206151968 >> 172.21.15.114:0/206151968 conn(0x7f706c0780e0 msgr2=0x7f706c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 -- 172.21.15.114:0/206151968 shutdown_connections 2024-09-21T06:44:15.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.724+0000 7f7073c00640 1 -- 172.21.15.114:0/206151968 wait complete. 2024-09-21T06:44:15.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.725+0000 7f7073c00640 1 Processor -- start 2024-09-21T06:44:15.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 -- start start 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f706c10a1b0 0x7f706c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c10faf0 0x7f706c1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 0x7f706c1a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f706c11a0b0 con 0x7f706c110440 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f706c119f30 con 0x7f706c10a1b0 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7073c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f706c11a230 con 0x7f706c10faf0 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7071c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 0x7f706c1a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7071c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 0x7f706c1a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35558/0 (socket says 172.21.15.114:35558) 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7071c00640 1 -- 172.21.15.114:0/813705171 learned_addr learned my addr 172.21.15.114:0/813705171 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.726+0000 7f7071200640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f706c10a1b0 0x7f706c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7071c00640 1 -- 172.21.15.114:0/813705171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c10faf0 msgr2=0x7f706c1a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7071c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c10faf0 0x7f706c1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:15.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7071c00640 1 -- 172.21.15.114:0/813705171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f706c10a1b0 msgr2=0x7f706c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:15.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7071c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f706c10a1b0 0x7f706c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:15.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7071c00640 1 -- 172.21.15.114:0/813705171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c1a6ad0 con 0x7f706c110440 2024-09-21T06:44:15.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7071c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 0x7f706c1a63a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f706400eb10 tx=0x7f706400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:15.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064004b20 con 0x7f706c110440 2024-09-21T06:44:15.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7064004cc0 con 0x7f706c110440 2024-09-21T06:44:15.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.727+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c1a6dc0 con 0x7f706c110440 2024-09-21T06:44:15.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.728+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064010670 con 0x7f706c110440 2024-09-21T06:44:15.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.728+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c1ad520 con 0x7f706c110440 2024-09-21T06:44:15.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.729+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70480052c0 con 0x7f706c110440 2024-09-21T06:44:15.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.733+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7f7064010810 con 0x7f706c110440 2024-09-21T06:44:15.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.734+0000 7f706a000640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70500b64d0 0x7f70500b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:15.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.734+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f70640e5d90 con 0x7f706c110440 2024-09-21T06:44:15.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.734+0000 7f7071200640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70500b64d0 0x7f70500b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:15.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.735+0000 7f7071200640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70500b64d0 0x7f70500b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f705c009bd0 tx=0x7f705c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:15.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.736+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7064056710 con 0x7f706c110440 2024-09-21T06:44:15.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:15.935+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7048000bf0 con 0x7f70500b64d0 2024-09-21T06:44:16.702 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:16 smithi114 ceph-mon[30654]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:16.703 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:44:16.703 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:44:16.703 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:16 smithi183 ceph-mon[34219]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:44:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:44:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:16 smithi193 ceph-mon[34601]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:44:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:44:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:17.795 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:17 smithi114 ceph-mon[30654]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:44:17.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:17 smithi183 ceph-mon[34219]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:44:17.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:17 smithi193 ceph-mon[34601]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:44:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:18 smithi183 ceph-mon[34219]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:18 smithi193 ceph-mon[34601]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:19.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:18 smithi114 ceph-mon[30654]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:19.846 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:19 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/791324315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "89d9168b-a941-4432-a508-6ab83fc6148c"} : dispatch 2024-09-21T06:44:19.847 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:19 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/791324315' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89d9168b-a941-4432-a508-6ab83fc6148c"}]': finished 2024-09-21T06:44:19.847 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:19 smithi114 ceph-mon[30654]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:44:19.847 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:19 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:19.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:19 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/791324315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "89d9168b-a941-4432-a508-6ab83fc6148c"} : dispatch 2024-09-21T06:44:19.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:19 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/791324315' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89d9168b-a941-4432-a508-6ab83fc6148c"}]': finished 2024-09-21T06:44:19.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:19 smithi183 ceph-mon[34219]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:44:19.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:19 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:19.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:19 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/791324315' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "89d9168b-a941-4432-a508-6ab83fc6148c"} : dispatch 2024-09-21T06:44:19.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:19 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/791324315' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89d9168b-a941-4432-a508-6ab83fc6148c"}]': finished 2024-09-21T06:44:19.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:19 smithi193 ceph-mon[34601]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T06:44:19.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:19 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:20 smithi183 ceph-mon[34219]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:20 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/607463204' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:44:20.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:20 smithi193 ceph-mon[34601]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:20.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:20 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/607463204' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:44:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:20 smithi114 ceph-mon[30654]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:20 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/607463204' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:44:22.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:22 smithi183 ceph-mon[34219]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:22.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:22 smithi193 ceph-mon[34601]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:23.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:22 smithi114 ceph-mon[30654]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:24.688 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:24 smithi114 ceph-mon[30654]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:24 smithi183 ceph-mon[34219]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:24.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:24 smithi193 ceph-mon[34601]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:26.603 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:26 smithi114 ceph-mon[30654]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:26 smithi183 ceph-mon[34219]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:26 smithi193 ceph-mon[34601]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:27 smithi114 ceph-mon[30654]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:27 smithi193 ceph-mon[34601]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:28.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:27 smithi183 ceph-mon[34219]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:44:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:28 smithi193 ceph-mon[34601]: Deploying daemon osd.0 on smithi114 2024-09-21T06:44:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:44:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:28 smithi114 ceph-mon[30654]: Deploying daemon osd.0 on smithi114 2024-09-21T06:44:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T06:44:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:28 smithi183 ceph-mon[34219]: Deploying daemon osd.0 on smithi114 2024-09-21T06:44:29.780 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:29 smithi114 ceph-mon[30654]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:29.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:29 smithi193 ceph-mon[34601]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:30.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:29 smithi183 ceph-mon[34219]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:32.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:32 smithi114 ceph-mon[30654]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:32 smithi183 ceph-mon[34219]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:32 smithi193 ceph-mon[34601]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:34 smithi114 ceph-mon[30654]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:34 smithi183 ceph-mon[34219]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:34 smithi193 ceph-mon[34601]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:36 smithi114 ceph-mon[30654]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:36 smithi183 ceph-mon[34219]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:36 smithi193 ceph-mon[34601]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:38.809 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:38 smithi114 ceph-mon[30654]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:38 smithi183 ceph-mon[34219]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:38 smithi193 ceph-mon[34601]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:39 smithi193 ceph-mon[34601]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:39 smithi114 ceph-mon[30654]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:39 smithi183 ceph-mon[34219]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:40.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:40.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:41.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.775+0000 7f706a000640 1 -- 172.21.15.114:0/813705171 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7048000bf0 con 0x7f70500b64d0 2024-09-21T06:44:41.779 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 0 on host 'smithi114' 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70500b64d0 msgr2=0x7f70500b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70500b64d0 0x7f70500b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f705c009bd0 tx=0x7f705c005f90 comp rx=0 tx=0).stop 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 msgr2=0x7f706c1a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 0x7f706c1a63a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f706400eb10 tx=0x7f706400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 shutdown_connections 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f706c110440 0x7f706c1a63a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f706c10faf0 0x7f706c1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70500b64d0 0x7f70500b89e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 --2- 172.21.15.114:0/813705171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f706c10a1b0 0x7f706c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:41.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 >> 172.21.15.114:0/813705171 conn(0x7f706c0780e0 msgr2=0x7f706c108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:41.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 shutdown_connections 2024-09-21T06:44:41.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:44:41.778+0000 7f7073c00640 1 -- 172.21.15.114:0/813705171 wait complete. 2024-09-21T06:44:42.500 DEBUG:teuthology.orchestra.run.smithi114:osd.0> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T06:44:42.503 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-09-21T06:44:42.504 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:44:42.749 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:42 smithi114 ceph-mon[30654]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:42.750 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:42.750 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:42.750 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:42.750 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:42 smithi114 ceph-mon[30654]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:44:42.755 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:44:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:42 smithi183 ceph-mon[34219]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:42 smithi183 ceph-mon[34219]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:44:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:42 smithi193 ceph-mon[34601]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:44:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:42 smithi193 ceph-mon[34601]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:43 smithi183 ceph-mon[34219]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:43 smithi183 ceph-mon[34219]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:43 smithi183 ceph-mon[34219]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:44.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:43 smithi183 ceph-mon[34219]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:44.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:43 smithi114 ceph-mon[30654]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:44:44.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:43 smithi114 ceph-mon[30654]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:44:44.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:43 smithi114 ceph-mon[30654]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:44:44.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:44.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:43 smithi114 ceph-mon[30654]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:43 smithi193 ceph-mon[34601]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T06:44:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:43 smithi193 ceph-mon[34601]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T06:44:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:43 smithi193 ceph-mon[34601]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:44:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:43 smithi193 ceph-mon[34601]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:44.785 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0[47237]: 2024-09-21T06:44:44.611+0000 7fe98aa00640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T06:44:44.785 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0[47237]: 2024-09-21T06:44:44.634+0000 7fe980000640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:44:45.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-mon[30654]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:44:45.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-mon[30654]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:44:45.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:45.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:45.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:44 smithi114 ceph-mon[30654]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' 2024-09-21T06:44:45.080 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:44 smithi183 ceph-mon[34219]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:44:45.080 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:44 smithi183 ceph-mon[34219]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:44:45.081 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:45.081 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:45.081 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:44 smithi183 ceph-mon[34219]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' 2024-09-21T06:44:45.084 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:44:45.110 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-09-21T06:44:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:44 smithi193 ceph-mon[34601]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:44:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:44 smithi193 ceph-mon[34601]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T06:44:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:44 smithi193 ceph-mon[34601]: from='osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446]' entity='osd.0' 2024-09-21T06:44:45.356 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:44:45.800 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.794+0000 7fd6ba818640 1 -- 172.21.15.183:0/4177754131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 msgr2=0x7fd6b4104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.794+0000 7fd6ba818640 1 --2- 172.21.15.183:0/4177754131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b4104dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009900 tx=0x7fd6a402a410 comp rx=0 tx=0).stop 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 -- 172.21.15.183:0/4177754131 shutdown_connections 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 --2- 172.21.15.183:0/4177754131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b410de10 0x7fd6b4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 --2- 172.21.15.183:0/4177754131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6b4105300 0x7fd6b410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 --2- 172.21.15.183:0/4177754131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b4104dc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 -- 172.21.15.183:0/4177754131 >> 172.21.15.183:0/4177754131 conn(0x7fd6b4079730 msgr2=0x7fd6b4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 -- 172.21.15.183:0/4177754131 shutdown_connections 2024-09-21T06:44:45.801 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 -- 172.21.15.183:0/4177754131 wait complete. 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.795+0000 7fd6ba818640 1 Processor -- start 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 -- start start 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6b4105300 0x7fd6b419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b410de10 0x7fd6b419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4113b40 con 0x7fd6b4105300 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b41139c0 con 0x7fd6b41049b0 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6ba818640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b4113cc0 con 0x7fd6b410de10 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6b3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6b3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:57468/0 (socket says 172.21.15.183:57468) 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.796+0000 7fd6b3fff640 1 -- 172.21.15.183:0/1920338956 learned_addr learned my addr 172.21.15.183:0/1920338956 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-21T06:44:45.802 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b3fff640 1 -- 172.21.15.183:0/1920338956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b410de10 msgr2=0x7fd6b419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b8db5640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b410de10 0x7fd6b419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b3fff640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b410de10 0x7fd6b419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b3fff640 1 -- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6b4105300 msgr2=0x7fd6b419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b3fff640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6b4105300 0x7fd6b419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b3fff640 1 -- 172.21.15.183:0/1920338956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b41a0520 con 0x7fd6b41049b0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b3fff640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b419a5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005ab0 tx=0x7fd6a4003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a403f070 con 0x7fd6b41049b0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6a4031d60 con 0x7fd6b41049b0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4035030 con 0x7fd6b41049b0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b41a07b0 con 0x7fd6b41049b0 2024-09-21T06:44:45.803 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.797+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b41a6f70 con 0x7fd6b41049b0 2024-09-21T06:44:45.804 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.798+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b4104dc0 con 0x7fd6b41049b0 2024-09-21T06:44:45.804 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.799+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fd6a4042070 con 0x7fd6b41049b0 2024-09-21T06:44:45.804 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.800+0000 7fd6b17fa640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd69c0b6550 0x7fd69c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:44:45.805 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.800+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7fd6a41061d0 con 0x7fd6b41049b0 2024-09-21T06:44:45.805 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.800+0000 7fd6b37fe640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd69c0b6550 0x7fd69c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:44:45.805 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.801+0000 7fd6b37fe640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd69c0b6550 0x7fd69c0b8a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6b410fca0 tx=0x7fd6a8006c40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:44:45.806 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.805+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6a40ce400 con 0x7fd6b41049b0 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] boot 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:44:46.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:46.002 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:44:45.999+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd6b4100c70 con 0x7fd69c0b6550 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] boot 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:44:46.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: osd.0 [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] boot 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T06:44:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T06:44:47.040 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:44:47.040 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:44:47.040 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:44:47.040 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi114 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi114 to 17089M 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:47.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:44:47.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:44:47.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi114 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi114 to 17089M 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi114 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:44:47.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi114 to 17089M 2024-09-21T06:44:47.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:47.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:47.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:44:47.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:44:48.013 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:47 smithi183 ceph-mon[34219]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:44:48.013 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:47 smithi183 ceph-mon[34219]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:48.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:47 smithi114 ceph-mon[30654]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:44:48.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:47 smithi114 ceph-mon[30654]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:47 smithi193 ceph-mon[34601]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T06:44:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:47 smithi193 ceph-mon[34601]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:48 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/1593809970' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"} : dispatch 2024-09-21T06:44:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:48 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"} : dispatch 2024-09-21T06:44:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:48 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"}]': finished 2024-09-21T06:44:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:48 smithi183 ceph-mon[34219]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:44:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:48 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:44:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:48 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/1593809970' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"} : dispatch 2024-09-21T06:44:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:48 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"} : dispatch 2024-09-21T06:44:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:48 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"}]': finished 2024-09-21T06:44:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:48 smithi114 ceph-mon[30654]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:44:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:48 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:44:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:48 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/1593809970' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"} : dispatch 2024-09-21T06:44:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:48 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"} : dispatch 2024-09-21T06:44:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:48 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9bba525-a4a1-4988-8288-4a9023f07724"}]': finished 2024-09-21T06:44:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:48 smithi193 ceph-mon[34601]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T06:44:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:48 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:44:50.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/2797965455' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:44:50.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:49 smithi183 ceph-mon[34219]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:50.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/2797965455' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:44:50.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:49 smithi193 ceph-mon[34601]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:50.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/2797965455' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:44:50.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:49 smithi114 ceph-mon[30654]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:52 smithi114 ceph-mon[30654]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:52 smithi183 ceph-mon[34219]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:52 smithi193 ceph-mon[34601]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:54.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:54 smithi183 ceph-mon[34219]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:54 smithi114 ceph-mon[30654]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:54 smithi193 ceph-mon[34601]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:56 smithi114 ceph-mon[30654]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:44:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:56.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:56 smithi183 ceph-mon[34219]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:56.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:44:56.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:56 smithi193 ceph-mon[34601]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T06:44:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:44:57.765 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:57 smithi183 ceph-mon[34219]: Deploying daemon osd.1 on smithi183 2024-09-21T06:44:57.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:57 smithi114 ceph-mon[30654]: Deploying daemon osd.1 on smithi183 2024-09-21T06:44:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:57 smithi193 ceph-mon[34601]: Deploying daemon osd.1 on smithi183 2024-09-21T06:44:58.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:44:58 smithi183 ceph-mon[34219]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:44:58 smithi114 ceph-mon[30654]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:44:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:44:58 smithi193 ceph-mon[34601]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:00.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:00 smithi114 ceph-mon[30654]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:00 smithi183 ceph-mon[34219]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:00 smithi193 ceph-mon[34601]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:02.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:02 smithi114 ceph-mon[30654]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:02 smithi183 ceph-mon[34219]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:02 smithi193 ceph-mon[34601]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:03.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:03.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:03.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:03.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:03.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:03.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:04 smithi114 ceph-mon[30654]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:04 smithi183 ceph-mon[34219]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:04 smithi193 ceph-mon[34601]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:06.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:05 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:05 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:05 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:05 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:05 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:05 smithi183 ceph-mon[34219]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:05 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:05 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:05 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:05 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:05 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:05 smithi193 ceph-mon[34601]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:05 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:05 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:05 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:05 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:05 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:05 smithi114 ceph-mon[30654]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:06.949 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.946+0000 7fd6b17fa640 1 -- 172.21.15.183:0/1920338956 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd6b4100c70 con 0x7fd69c0b6550 2024-09-21T06:45:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-09-21T06:45:06.951 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.948+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd69c0b6550 msgr2=0x7fd69c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:06.951 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.948+0000 7fd6ba818640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd69c0b6550 0x7fd69c0b8a60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd6b410fca0 tx=0x7fd6a8006c40 comp rx=0 tx=0).stop 2024-09-21T06:45:06.951 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.948+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 msgr2=0x7fd6b419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.948+0000 7fd6ba818640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b419a5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005ab0 tx=0x7fd6a4003cd0 comp rx=0 tx=0).stop 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 shutdown_connections 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b410de10 0x7fd6b419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6b4105300 0x7fd6b419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd69c0b6550 0x7fd69c0b8a60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 --2- 172.21.15.183:0/1920338956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6b41049b0 0x7fd6b419a5b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 >> 172.21.15.183:0/1920338956 conn(0x7fd6b4079730 msgr2=0x7fd6b40ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 shutdown_connections 2024-09-21T06:45:06.952 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:45:06.949+0000 7fd6ba818640 1 -- 172.21.15.183:0/1920338956 wait complete. 2024-09-21T06:45:07.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:07.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:07.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:07.326 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:07.326 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:07.326 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:07.672 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T06:45:07.674 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-09-21T06:45:07.674 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T06:45:07.925 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:45:08.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:07 smithi114 ceph-mon[30654]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:08.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:07 smithi193 ceph-mon[34601]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:08.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:07 smithi183 ceph-mon[34219]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:09.250 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:09 smithi193 ceph-mon[34601]: from='osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:45:09.250 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:09 smithi193 ceph-mon[34601]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:45:09.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:09 smithi183 ceph-mon[34219]: from='osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:45:09.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:09 smithi183 ceph-mon[34219]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:45:09.365 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:09 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1[38912]: 2024-09-21T06:45:09.080+0000 7fbf057e1640 -1 osd.1 0 log_to_monitors true 2024-09-21T06:45:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:09 smithi114 ceph-mon[30654]: from='osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:45:09.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:09 smithi114 ceph-mon[30654]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T06:45:10.290 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: from='osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:10.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:10 smithi193 ceph-mon[34601]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:45:10.314 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-09-21T06:45:10.323 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1[38912]: 2024-09-21T06:45:10.147+0000 7fbf01763640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T06:45:10.323 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1[38912]: 2024-09-21T06:45:10.153+0000 7fbefd561640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:45:10.323 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:45:10.323 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: from='osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:45:10.323 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:45:10.323 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:10.324 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:45:10.324 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:10.324 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:10 smithi183 ceph-mon[34219]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:45:10.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T06:45:10.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: from='osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:45:10.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T06:45:10.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:10.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:45:10.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:10.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:10 smithi114 ceph-mon[30654]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:45:10.566 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 -- 172.21.15.193:0/559657594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628102ae0 msgr2=0x7fd628102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 --2- 172.21.15.193:0/559657594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628102ae0 0x7fd628102ef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd620009900 tx=0x7fd62002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 -- 172.21.15.193:0/559657594 shutdown_connections 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 --2- 172.21.15.193:0/559657594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 0x7fd62810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 --2- 172.21.15.193:0/559657594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd628108420 0x7fd628108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 --2- 172.21.15.193:0/559657594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628102ae0 0x7fd628102ef0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 -- 172.21.15.193:0/559657594 >> 172.21.15.193:0/559657594 conn(0x7fd628079770 msgr2=0x7fd628079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 -- 172.21.15.193:0/559657594 shutdown_connections 2024-09-21T06:45:11.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.035+0000 7fd62d092640 1 -- 172.21.15.193:0/559657594 wait complete. 2024-09-21T06:45:11.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 Processor -- start 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 -- start start 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd628102ae0 0x7fd62819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 0x7fd62819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 0x7fd62819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd628112a30 con 0x7fd628108d70 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6281128b0 con 0x7fd628102ae0 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62d092640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd628112bb0 con 0x7fd628108420 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 0x7fd62819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 0x7fd62819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:60190/0 (socket says 172.21.15.193:60190) 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd62659c640 1 -- 172.21.15.193:0/213964894 learned_addr learned my addr 172.21.15.193:0/213964894 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.036+0000 7fd626d9d640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd628102ae0 0x7fd62819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62659c640 1 -- 172.21.15.193:0/213964894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd628102ae0 msgr2=0x7fd62819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:11.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62759e640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 0x7fd62819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62659c640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd628102ae0 0x7fd62819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62659c640 1 -- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 msgr2=0x7fd62819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62659c640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 0x7fd62819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62659c640 1 -- 172.21.15.193:0/213964894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6281a0360 con 0x7fd628108420 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62759e640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 0x7fd62819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62659c640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 0x7fd62819aa50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00cce0 tx=0x7fd61c00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00f450 con 0x7fd628108420 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6281a0650 con 0x7fd628108420 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd61c00faa0 con 0x7fd628108420 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c018c50 con 0x7fd628108420 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.037+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6281a6e70 con 0x7fd628108420 2024-09-21T06:45:11.041 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.038+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd628103d60 con 0x7fd628108420 2024-09-21T06:45:11.042 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.041+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fd61c00fc50 con 0x7fd628108420 2024-09-21T06:45:11.043 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.042+0000 7fd60ffff640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6100b6500 0x7fd6100b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:11.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.042+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7fd61c0e61e0 con 0x7fd628108420 2024-09-21T06:45:11.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.042+0000 7fd626d9d640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6100b6500 0x7fd6100b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:11.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.043+0000 7fd626d9d640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6100b6500 0x7fd6100b8a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd620031440 tx=0x7fd620005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:11.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.044+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd61c056710 con 0x7fd628108420 2024-09-21T06:45:11.243 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:11.242+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd628106230 con 0x7fd6100b6500 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi183 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:11.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:11.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi183 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.457 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:45:11.458 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-21T06:45:11.458 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.458 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:11.458 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:11.458 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.458 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T06:45:11.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi183 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:12.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:45:12.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:45:12.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] boot 2024-09-21T06:45:12.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:45:12.429 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:12.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:45:12.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:45:12.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:45:12.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:12.430 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:12 smithi193 ceph-mon[34601]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:45:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:45:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] boot 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:12.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:12 smithi114 ceph-mon[30654]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: osd.1 [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] boot 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:12 smithi183 ceph-mon[34219]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T06:45:13.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:13 smithi193 ceph-mon[34601]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:45:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:13 smithi114 ceph-mon[30654]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:45:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:13 smithi183 ceph-mon[34219]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/1408696775' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"} : dispatch 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"} : dispatch 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"}]': finished 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:14.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:14 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/288849688' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:45:14.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/1408696775' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"} : dispatch 2024-09-21T06:45:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"} : dispatch 2024-09-21T06:45:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"}]': finished 2024-09-21T06:45:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:45:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:14 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/288849688' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:45:14.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/1408696775' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"} : dispatch 2024-09-21T06:45:14.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"} : dispatch 2024-09-21T06:45:14.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bc9357c-f3a5-4370-954a-e1654bab5e75"}]': finished 2024-09-21T06:45:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: pgmap v55: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T06:45:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:14 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/288849688' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:45:16.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:16 smithi114 ceph-mon[30654]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:16 smithi183 ceph-mon[34219]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:16.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:16 smithi193 ceph-mon[34601]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:18.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:18 smithi193 ceph-mon[34601]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:18.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:18 smithi114 ceph-mon[30654]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:18 smithi183 ceph-mon[34219]: pgmap v58: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:20 smithi193 ceph-mon[34601]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:20.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:20 smithi114 ceph-mon[30654]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:20 smithi183 ceph-mon[34219]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:22.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:22 smithi193 ceph-mon[34601]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:22.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:22 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:45:22.492 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:22 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:22.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:22 smithi183 ceph-mon[34219]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:22.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:22 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:45:22.744 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:22 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:22 smithi114 ceph-mon[30654]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:22 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T06:45:22.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:22 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:23.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:23 smithi193 ceph-mon[34601]: Deploying daemon osd.2 on smithi193 2024-09-21T06:45:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:23 smithi114 ceph-mon[30654]: Deploying daemon osd.2 on smithi193 2024-09-21T06:45:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:23 smithi183 ceph-mon[34219]: Deploying daemon osd.2 on smithi193 2024-09-21T06:45:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:24 smithi193 ceph-mon[34601]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:24.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:24 smithi114 ceph-mon[30654]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:24 smithi183 ceph-mon[34219]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:26.768 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:26 smithi193 ceph-mon[34601]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:26 smithi114 ceph-mon[30654]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:26 smithi183 ceph-mon[34219]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:28.711 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:28 smithi193 ceph-mon[34601]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:28.712 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:28.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:28 smithi114 ceph-mon[30654]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:28.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:28.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:28.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:28 smithi183 ceph-mon[34219]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:28.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:28.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:30 smithi114 ceph-mon[30654]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:30 smithi183 ceph-mon[34219]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:30 smithi193 ceph-mon[34601]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:32.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.422+0000 7fd60ffff640 1 -- 172.21.15.193:0/213964894 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd628106230 con 0x7fd6100b6500 2024-09-21T06:45:32.424 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6100b6500 msgr2=0x7fd6100b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6100b6500 0x7fd6100b8a10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd620031440 tx=0x7fd620005d50 comp rx=0 tx=0).stop 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 msgr2=0x7fd62819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 0x7fd62819aa50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00cce0 tx=0x7fd61c00a620 comp rx=0 tx=0).stop 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 shutdown_connections 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd628108d70 0x7fd62819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd628108420 0x7fd62819aa50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6100b6500 0x7fd6100b8a10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 --2- 172.21.15.193:0/213964894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd628102ae0 0x7fd62819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 >> 172.21.15.193:0/213964894 conn(0x7fd628079770 msgr2=0x7fd628101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 shutdown_connections 2024-09-21T06:45:32.427 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:45:32.425+0000 7fd62d092640 1 -- 172.21.15.193:0/213964894 wait complete. 2024-09-21T06:45:32.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:32 smithi114 ceph-mon[30654]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:32.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:32.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:32 smithi193 ceph-mon[34601]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:32.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:32.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:32.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:32 smithi183 ceph-mon[34219]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:33.038 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T06:45:33.040 INFO:tasks.cephadm:Deploying osd.3 on smithi114 with /dev/vg_nvme/lv_3... 2024-09-21T06:45:33.040 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:45:33.400 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:45:34.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:34 smithi193 ceph-mon[34601]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:34.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:34 smithi193 ceph-mon[34601]: from='osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:45:34.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:34 smithi193 ceph-mon[34601]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:45:34.560 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2[39650]: 2024-09-21T06:45:34.226+0000 7f06bbd8e640 -1 osd.2 0 log_to_monitors true 2024-09-21T06:45:34.623 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:34 smithi114 ceph-mon[30654]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:34 smithi114 ceph-mon[30654]: from='osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:45:34.624 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:34 smithi114 ceph-mon[30654]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:45:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:34 smithi183 ceph-mon[34219]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:34 smithi183 ceph-mon[34219]: from='osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:45:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:34 smithi183 ceph-mon[34219]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T06:45:35.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:35 smithi193 ceph-mon[34601]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:45:35.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:35 smithi193 ceph-mon[34601]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:45:35.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:35 smithi193 ceph-mon[34601]: from='osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:45:35.625 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:35.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:35 smithi193 ceph-mon[34601]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:45:35.626 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:35 smithi193 ceph-mon[34601]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:45:35.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:35 smithi114 ceph-mon[30654]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:45:35.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:35 smithi114 ceph-mon[30654]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:45:35.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:35 smithi114 ceph-mon[30654]: from='osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:45:35.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:35.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:35 smithi114 ceph-mon[30654]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:45:35.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:35 smithi114 ceph-mon[30654]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:45:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:35 smithi183 ceph-mon[34219]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T06:45:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:35 smithi183 ceph-mon[34219]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T06:45:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:35 smithi183 ceph-mon[34219]: from='osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:45:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:35 smithi183 ceph-mon[34219]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:45:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:35 smithi183 ceph-mon[34219]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:45:36.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2[39650]: 2024-09-21T06:45:36.185+0000 7f06b7d10640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T06:45:36.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2[39650]: 2024-09-21T06:45:36.190+0000 7f06afb06640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:45:36.524 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:45:36.556 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_3 2024-09-21T06:45:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='osd.2 ' entity='osd.2' 2024-09-21T06:45:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='osd.2 ' entity='osd.2' 2024-09-21T06:45:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='osd.2 ' entity='osd.2' 2024-09-21T06:45:36.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:36.918 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:45:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 -- 172.21.15.114:0/168611284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec48105d80 msgr2=0x7fec48106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 --2- 172.21.15.114:0/168611284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec48105d80 0x7fec48106190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fec38009900 tx=0x7fec3802a410 comp rx=0 tx=0).stop 2024-09-21T06:45:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 -- 172.21.15.114:0/168611284 shutdown_connections 2024-09-21T06:45:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 --2- 172.21.15.114:0/168611284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec4810c010 0x7fec48112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 --2- 172.21.15.114:0/168611284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec4810b6c0 0x7fec4810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 --2- 172.21.15.114:0/168611284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec48105d80 0x7fec48106190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 -- 172.21.15.114:0/168611284 >> 172.21.15.114:0/168611284 conn(0x7fec48101810 msgr2=0x7fec48103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:45:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 -- 172.21.15.114:0/168611284 shutdown_connections 2024-09-21T06:45:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.520+0000 7fec4de00640 1 -- 172.21.15.114:0/168611284 wait complete. 2024-09-21T06:45:37.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.521+0000 7fec4de00640 1 Processor -- start 2024-09-21T06:45:37.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.521+0000 7fec4de00640 1 -- start start 2024-09-21T06:45:37.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec4de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 0x7fec4819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:37.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec4de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec4810b6c0 0x7fec4819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec4de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4810c010 0x7fec481a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec4de00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec48115d20 con 0x7fec4810c010 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec4de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec48115ba0 con 0x7fec48105d80 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec4de00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec48115ea0 con 0x7fec4810b6c0 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec47400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4810c010 0x7fec481a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec46a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 0x7fec4819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec3e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec4810b6c0 0x7fec4819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec46a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 0x7fec4819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:55490/0 (socket says 172.21.15.114:55490) 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec46a00640 1 -- 172.21.15.114:0/2231034342 learned_addr learned my addr 172.21.15.114:0/2231034342 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec46a00640 1 -- 172.21.15.114:0/2231034342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec4810b6c0 msgr2=0x7fec4819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec46a00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec4810b6c0 0x7fec4819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.522+0000 7fec46a00640 1 -- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4810c010 msgr2=0x7fec481a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:45:37.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec46a00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4810c010 0x7fec481a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:45:37.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec46a00640 1 -- 172.21.15.114:0/2231034342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec481a2660 con 0x7fec48105d80 2024-09-21T06:45:37.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec47400640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4810c010 0x7fec481a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:45:37.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec46a00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 0x7fec4819c6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fec38005ab0 tx=0x7fec38003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:37.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3803f070 con 0x7fec48105d80 2024-09-21T06:45:37.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec481a28f0 con 0x7fec48105d80 2024-09-21T06:45:37.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38031b60 con 0x7fec48105d80 2024-09-21T06:45:37.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38035030 con 0x7fec48105d80 2024-09-21T06:45:37.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.523+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481a90b0 con 0x7fec48105d80 2024-09-21T06:45:37.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.524+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec48106190 con 0x7fec48105d80 2024-09-21T06:45:37.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.527+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 154474+0+0 (secure 0 0 0) 0x7fec38042070 con 0x7fec48105d80 2024-09-21T06:45:37.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.528+0000 7fec4ca00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec2c0b64d0 0x7fec2c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:45:37.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.528+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3066+0+0 (secure 0 0 0) 0x7fec38106c10 con 0x7fec48105d80 2024-09-21T06:45:37.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.528+0000 7fec3e000640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec2c0b64d0 0x7fec2c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:45:37.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.529+0000 7fec3e000640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec2c0b64d0 0x7fec2c0b89e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec4810b120 tx=0x7fec30005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:45:37.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.530+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec380d2d40 con 0x7fec48105d80 2024-09-21T06:45:37.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:37.728+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fec4810de10 con 0x7fec2c0b64d0 2024-09-21T06:45:37.731 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:45:37.731 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:45:37.731 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi193 2024-09-21T06:45:37.731 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T06:45:37.731 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] boot 2024-09-21T06:45:37.731 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:45:37.732 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:45:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:45:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi193 2024-09-21T06:45:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T06:45:37.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] boot 2024-09-21T06:45:37.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:45:37.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi193 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi193 to 17088M 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: osd.2 [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] boot 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T06:45:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T06:45:38.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:38.563 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:45:38.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:45:38.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:45:38.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:38.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:45:38.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:45:38.564 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:38 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:45:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:45:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:45:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:45:38.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:38.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:45:38.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:45:38.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:38 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T06:45:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:38 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:45:39.315 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51596]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:45:39.315 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51596]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.315 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51596]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.316 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51596]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.589 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51599]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:45:39.589 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51599]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.590 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51599]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.590 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51599]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 ceph-mon[30654]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 ceph-mon[30654]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51623]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.590 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51623]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.656 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42442]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:45:39.656 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42442]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.656 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42442]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.656 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42442]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.657 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42445]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:45:39.657 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.657 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42445]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.657 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42445]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 ceph-mon[34219]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:45:39.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:45:39.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 ceph-mon[34219]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:45:39.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:39.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:39.839 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:45:39.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51628]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.840 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:39 smithi114 sudo[51628]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:45:39.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43320]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43320]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.913 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43323]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:45:39.913 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43323]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.913 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43323]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.913 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43323]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 ceph-mon[34601]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:45:39.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 ceph-mon[34601]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43326]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43326]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43326]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:39.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43329]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:45:39.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:39.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43329]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:39.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:39 smithi193 sudo[43329]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:40.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42448]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T06:45:40.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42448]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:40.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42448]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:40.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42448]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:40.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T06:45:40.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T06:45:40.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42451]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T06:45:40.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:39 smithi183 sudo[42451]: pam_unix(sudo:session): session closed for user root 2024-09-21T06:45:40.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:45:40.384+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fec380ca580 con 0x7fec48105d80 2024-09-21T06:45:40.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T06:45:40.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:40 smithi193 ceph-mon[34601]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T06:45:40.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:40 smithi114 ceph-mon[30654]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T06:45:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:45:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:45:40.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:45:40.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T06:45:40.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: osdmap e22: 3 total, 3 up, 3 in 2024-09-21T06:45:40.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:40 smithi183 ceph-mon[34219]: mgrmap e21: x(active, since 2m), standbys: z, y 2024-09-21T06:45:41.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/523078884' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36ee2356-5e1f-491f-bdf5-d9bd22956581"} : dispatch 2024-09-21T06:45:41.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/523078884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ee2356-5e1f-491f-bdf5-d9bd22956581"}]': finished 2024-09-21T06:45:41.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:41 smithi114 ceph-mon[30654]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:45:41.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:45:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/523078884' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36ee2356-5e1f-491f-bdf5-d9bd22956581"} : dispatch 2024-09-21T06:45:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/523078884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ee2356-5e1f-491f-bdf5-d9bd22956581"}]': finished 2024-09-21T06:45:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:41 smithi183 ceph-mon[34219]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:45:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:45:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/523078884' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36ee2356-5e1f-491f-bdf5-d9bd22956581"} : dispatch 2024-09-21T06:45:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/523078884' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36ee2356-5e1f-491f-bdf5-d9bd22956581"}]': finished 2024-09-21T06:45:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:41 smithi193 ceph-mon[34601]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T06:45:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:45:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:42 smithi114 ceph-mon[30654]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:42 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3215661314' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:45:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:42 smithi183 ceph-mon[34219]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:42 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3215661314' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:45:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:42 smithi193 ceph-mon[34601]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:42 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3215661314' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:45:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:44 smithi114 ceph-mon[30654]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:44 smithi183 ceph-mon[34219]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:44 smithi193 ceph-mon[34601]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:46.810 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:46 smithi114 ceph-mon[30654]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:46 smithi183 ceph-mon[34219]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:46 smithi193 ceph-mon[34601]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:48.725 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:48 smithi114 ceph-mon[30654]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:48 smithi183 ceph-mon[34219]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:48 smithi193 ceph-mon[34601]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:50.788 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:50 smithi114 ceph-mon[30654]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:50.788 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:50 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:45:50.788 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:50 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:50 smithi183 ceph-mon[34219]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:50 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:45:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:50 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:50 smithi193 ceph-mon[34601]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:50 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T06:45:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:50 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:45:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:51 smithi114 ceph-mon[30654]: Deploying daemon osd.3 on smithi114 2024-09-21T06:45:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:51 smithi183 ceph-mon[34219]: Deploying daemon osd.3 on smithi114 2024-09-21T06:45:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:51 smithi193 ceph-mon[34601]: Deploying daemon osd.3 on smithi114 2024-09-21T06:45:52.725 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:52 smithi114 ceph-mon[30654]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:52 smithi183 ceph-mon[34219]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:52 smithi193 ceph-mon[34601]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:54 smithi114 ceph-mon[30654]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:54.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:54 smithi183 ceph-mon[34219]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:54.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:54 smithi193 ceph-mon[34601]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:56 smithi114 ceph-mon[30654]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:56 smithi183 ceph-mon[34219]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:56.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:56 smithi193 ceph-mon[34601]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:58 smithi114 ceph-mon[30654]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:58 smithi183 ceph-mon[34219]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:58 smithi193 ceph-mon[34601]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:45:59.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:45:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:45:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:45:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:45:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:45:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:00.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:00 smithi114 ceph-mon[30654]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:00 smithi183 ceph-mon[34219]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:00.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:00 smithi193 ceph-mon[34601]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:02.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:02 smithi114 ceph-mon[30654]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:02 smithi183 ceph-mon[34219]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:02 smithi193 ceph-mon[34601]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.200 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 3 on host 'smithi114' 2024-09-21T06:46:04.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.198+0000 7fec4ca00640 1 -- 172.21.15.114:0/2231034342 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec4810de10 con 0x7fec2c0b64d0 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec2c0b64d0 msgr2=0x7fec2c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec2c0b64d0 0x7fec2c0b89e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec4810b120 tx=0x7fec30005f90 comp rx=0 tx=0).stop 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 msgr2=0x7fec4819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 0x7fec4819c6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fec38005ab0 tx=0x7fec38003c10 comp rx=0 tx=0).stop 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 shutdown_connections 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec4810c010 0x7fec481a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec4810b6c0 0x7fec4819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec2c0b64d0 0x7fec2c0b89e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:04.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 --2- 172.21.15.114:0/2231034342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec48105d80 0x7fec4819c6f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:04.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 >> 172.21.15.114:0/2231034342 conn(0x7fec48101810 msgr2=0x7fec48103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:04.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 shutdown_connections 2024-09-21T06:46:04.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:46:04.201+0000 7fec4de00640 1 -- 172.21.15.114:0/2231034342 wait complete. 2024-09-21T06:46:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:04 smithi114 ceph-mon[30654]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:04 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:04.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:04 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:04 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:04 smithi183 ceph-mon[34219]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:04 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:04 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:04 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:04 smithi193 ceph-mon[34601]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:04 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:04 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:04 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:04.973 DEBUG:teuthology.orchestra.run.smithi114:osd.3> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T06:46:04.977 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-09-21T06:46:04.977 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:46:05.233 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:46:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:05 smithi183 ceph-mon[34219]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:46:05.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:05 smithi114 ceph-mon[30654]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:46:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:05 smithi193 ceph-mon[34601]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T06:46:06.598 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-mon[30654]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:06.598 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-mon[30654]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:46:06.599 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-mon[30654]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:46:06.599 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-mon[30654]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:46:06.599 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:06.599 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T06:46:06.511+0000 7fd8b0000640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T06:46:06.599 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 06:46:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T06:46:06.517+0000 7fd8ab000640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:46:06.675 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:06 smithi183 ceph-mon[34219]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:06.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:06 smithi183 ceph-mon[34219]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:46:06.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:06 smithi183 ceph-mon[34219]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:46:06.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:06 smithi183 ceph-mon[34219]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:46:06.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:06 smithi193 ceph-mon[34601]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:06 smithi193 ceph-mon[34601]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T06:46:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:06 smithi193 ceph-mon[34601]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T06:46:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:06 smithi193 ceph-mon[34601]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:46:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:07 smithi183 ceph-mon[34219]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:46:07.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:07 smithi183 ceph-mon[34219]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:46:07.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:07 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.579 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:07 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.793 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:07 smithi114 ceph-mon[30654]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:46:07.793 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:07 smithi114 ceph-mon[30654]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:46:07.793 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:07 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.793 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:07 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:07 smithi193 ceph-mon[34601]: from='osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:46:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:07 smithi193 ceph-mon[34601]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T06:46:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:07 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:07 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:07.952 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:46:07.978 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-09-21T06:46:08.226 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:46:08.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:46:08.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:46:08.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:08.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:08.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] boot 2024-09-21T06:46:08.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:46:08.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:08 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:08.755 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 -- 172.21.15.183:0/1190777924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 msgr2=0x7f900c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:08.755 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 --2- 172.21.15.183:0/1190777924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c102f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc009900 tx=0x7f8ffc02a410 comp rx=0 tx=0).stop 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 -- 172.21.15.183:0/1190777924 shutdown_connections 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 --2- 172.21.15.183:0/1190777924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f900c108da0 0x7f900c10f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 --2- 172.21.15.183:0/1190777924 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f900c108450 0x7f900c108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 --2- 172.21.15.183:0/1190777924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c102f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 -- 172.21.15.183:0/1190777924 >> 172.21.15.183:0/1190777924 conn(0x7f900c0796d0 msgr2=0x7f900c079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 -- 172.21.15.183:0/1190777924 shutdown_connections 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.751+0000 7f9011f3d640 1 -- 172.21.15.183:0/1190777924 wait complete. 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.752+0000 7f9011f3d640 1 Processor -- start 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.752+0000 7f9011f3d640 1 -- start start 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f900c108450 0x7f900c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f900c108da0 0x7f900c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f900c113b80 con 0x7f900c108da0 2024-09-21T06:46:08.756 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f900c113a00 con 0x7f900c102b10 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f900c113d00 con 0x7f900c108450 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:60254/0 (socket says 172.21.15.183:60254) 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 -- 172.21.15.183:0/4210821083 learned_addr learned my addr 172.21.15.183:0/4210821083 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900affd640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f900c108450 0x7f900c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900bfff640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f900c108da0 0x7f900c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 -- 172.21.15.183:0/4210821083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f900c108450 msgr2=0x7f900c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f900c108450 0x7f900c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 -- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f900c108da0 msgr2=0x7f900c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f900c108da0 0x7f900c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 -- 172.21.15.183:0/4210821083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c1a0540 con 0x7f900c102b10 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f900b7fe640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c19a5d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc005940 tx=0x7f8ffc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc03f030 con 0x7f900c102b10 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ffc003cb0 con 0x7f900c102b10 2024-09-21T06:46:08.757 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.753+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c1a07d0 con 0x7f900c102b10 2024-09-21T06:46:08.758 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.754+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc035070 con 0x7f900c102b10 2024-09-21T06:46:08.758 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.754+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1a6f90 con 0x7f900c102b10 2024-09-21T06:46:08.758 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.755+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c102f20 con 0x7f900c102b10 2024-09-21T06:46:08.758 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.756+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f8ffc042030 con 0x7f900c102b10 2024-09-21T06:46:08.759 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.757+0000 7f9008ff9640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ff40b65e0 0x7f8ff40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:08.759 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.757+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3884+0+0 (secure 0 0 0) 0x7f8ffc106a80 con 0x7f900c102b10 2024-09-21T06:46:08.759 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.757+0000 7f900affd640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ff40b65e0 0x7f8ff40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:08.759 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.758+0000 7f900affd640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ff40b65e0 0x7f8ff40b8af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f900c105470 tx=0x7f900000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:08.763 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.761+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8ffc0ce460 con 0x7f900c102b10 2024-09-21T06:46:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:46:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:46:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:08.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:08.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] boot 2024-09-21T06:46:08.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:46:08.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:08 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: osd.3 [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] boot 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T06:46:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:08 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T06:46:08.950 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:08.948+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f900c063800 con 0x7f8ff40b65e0 2024-09-21T06:46:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi114 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:46:09.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:46:09.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi114 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:09.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:09.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:46:09.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:46:09.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi114 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi114 to 8544M 2024-09-21T06:46:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:46:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:46:09.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:10 smithi183 ceph-mon[34219]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:46:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:10 smithi183 ceph-mon[34219]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:10.733 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:10 smithi183 ceph-mon[34219]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T06:46:10.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:10 smithi193 ceph-mon[34601]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:46:10.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:10 smithi193 ceph-mon[34601]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:10.813 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:10 smithi193 ceph-mon[34601]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T06:46:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:10 smithi114 ceph-mon[30654]: from='client.24422 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:46:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:10 smithi114 ceph-mon[30654]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:10 smithi114 ceph-mon[30654]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T06:46:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:11 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/2473397991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"} : dispatch 2024-09-21T06:46:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:11 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"} : dispatch 2024-09-21T06:46:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:11 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"}]': finished 2024-09-21T06:46:11.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:11 smithi114 ceph-mon[30654]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:46:11.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:11.825 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:11 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/2473397991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"} : dispatch 2024-09-21T06:46:11.825 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:11 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"} : dispatch 2024-09-21T06:46:11.825 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:11 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"}]': finished 2024-09-21T06:46:11.825 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:11 smithi183 ceph-mon[34219]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:46:11.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:11 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/2473397991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"} : dispatch 2024-09-21T06:46:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:11 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"} : dispatch 2024-09-21T06:46:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:11 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7db874e-11e1-44a2-9f0f-3a1f8738356a"}]': finished 2024-09-21T06:46:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:11 smithi193 ceph-mon[34601]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T06:46:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:12 smithi114 ceph-mon[30654]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:12 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/127625671' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:46:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:12 smithi183 ceph-mon[34219]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:12 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/127625671' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:46:12.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:12 smithi193 ceph-mon[34601]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:12 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/127625671' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:46:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:14 smithi114 ceph-mon[30654]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T06:46:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:14 smithi183 ceph-mon[34219]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T06:46:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:14 smithi193 ceph-mon[34601]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T06:46:16.783 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:16 smithi183 ceph-mon[34219]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T06:46:16.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:16 smithi114 ceph-mon[30654]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T06:46:16.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:16 smithi193 ceph-mon[34601]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T06:46:18.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:18 smithi183 ceph-mon[34219]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:46:18.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:18 smithi114 ceph-mon[30654]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:46:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:18 smithi193 ceph-mon[34601]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:46:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:20 smithi183 ceph-mon[34219]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:46:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:46:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:20.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:20 smithi114 ceph-mon[30654]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:46:20.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:46:20.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:20.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:20 smithi193 ceph-mon[34601]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:46:20.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T06:46:20.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:21.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:21 smithi114 ceph-mon[30654]: Deploying daemon osd.4 on smithi183 2024-09-21T06:46:21.827 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:21 smithi183 ceph-mon[34219]: Deploying daemon osd.4 on smithi183 2024-09-21T06:46:21.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:21 smithi193 ceph-mon[34601]: Deploying daemon osd.4 on smithi183 2024-09-21T06:46:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:22 smithi114 ceph-mon[30654]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:46:22.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:22 smithi183 ceph-mon[34219]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:46:22.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:22 smithi193 ceph-mon[34601]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:46:24.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:24 smithi114 ceph-mon[30654]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:46:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:24 smithi183 ceph-mon[34219]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:46:24.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:24 smithi193 ceph-mon[34601]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:46:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:26 smithi114 ceph-mon[30654]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:26.834 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:26 smithi183 ceph-mon[34219]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:26.834 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:26.834 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:26.834 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:26 smithi193 ceph-mon[34601]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:28.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:28 smithi183 ceph-mon[34219]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:28.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:28 smithi114 ceph-mon[30654]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:28 smithi193 ceph-mon[34601]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:30.432 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.430+0000 7f9008ff9640 1 -- 172.21.15.183:0/4210821083 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f900c063800 con 0x7f8ff40b65e0 2024-09-21T06:46:30.432 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-09-21T06:46:30.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ff40b65e0 msgr2=0x7f8ff40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:30.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ff40b65e0 0x7f8ff40b8af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f900c105470 tx=0x7f900000a2d0 comp rx=0 tx=0).stop 2024-09-21T06:46:30.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 msgr2=0x7f900c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:30.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c19a5d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc005940 tx=0x7f8ffc0042f0 comp rx=0 tx=0).stop 2024-09-21T06:46:30.434 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 shutdown_connections 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f900c108da0 0x7f900c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f900c108450 0x7f900c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ff40b65e0 0x7f8ff40b8af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 --2- 172.21.15.183:0/4210821083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f900c102b10 0x7f900c19a5d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.432+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 >> 172.21.15.183:0/4210821083 conn(0x7f900c0796d0 msgr2=0x7f900c106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.433+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 shutdown_connections 2024-09-21T06:46:30.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:46:30.433+0000 7f9011f3d640 1 -- 172.21.15.183:0/4210821083 wait complete. 2024-09-21T06:46:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:30 smithi193 ceph-mon[34601]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:30.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:30.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:30.750 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.751 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:30 smithi183 ceph-mon[34219]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:30 smithi114 ceph-mon[30654]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:31.069 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T06:46:31.072 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-09-21T06:46:31.072 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T06:46:31.340 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:46:31.612 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:31.612 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:31.612 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:31.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:31.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:31.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:32.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:32 smithi114 ceph-mon[30654]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:32.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:32 smithi183 ceph-mon[34219]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:32.615 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 06:46:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T06:46:32.471+0000 7f104cdca640 -1 osd.4 0 log_to_monitors true 2024-09-21T06:46:32.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:32 smithi193 ceph-mon[34601]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:33 smithi193 ceph-mon[34601]: from='osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:46:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:33 smithi193 ceph-mon[34601]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:46:33.762 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:46:33.788 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-09-21T06:46:33.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:33 smithi114 ceph-mon[30654]: from='osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:46:33.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:33 smithi114 ceph-mon[30654]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:46:33.847 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:33 smithi183 ceph-mon[34219]: from='osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:46:33.847 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:33 smithi183 ceph-mon[34219]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T06:46:34.037 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:46:34.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.561+0000 7f9258ced640 1 -- 172.21.15.193:0/2322791360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109560 msgr2=0x7f9254109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 --2- 172.21.15.193:0/2322791360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109560 0x7f9254109970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f924802a410 comp rx=0 tx=0).stop 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 -- 172.21.15.193:0/2322791360 shutdown_connections 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 --2- 172.21.15.193:0/2322791360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254109eb0 0x7f92541106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 --2- 172.21.15.193:0/2322791360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109560 0x7f9254109970 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 --2- 172.21.15.193:0/2322791360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254102080 0x7f9254102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 -- 172.21.15.193:0/2322791360 >> 172.21.15.193:0/2322791360 conn(0x7f9254079ec0 msgr2=0x7f925407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:34.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 -- 172.21.15.193:0/2322791360 shutdown_connections 2024-09-21T06:46:34.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.562+0000 7f9258ced640 1 -- 172.21.15.193:0/2322791360 wait complete. 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 Processor -- start 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 -- start start 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254102080 0x7f925419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254109560 0x7f925419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 0x7f925419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9254113b70 con 0x7f9254102080 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92541139f0 con 0x7f9254109560 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9258ced640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9254113cf0 con 0x7f9254109eb0 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 0x7f925419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254109560 0x7f925419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:34.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 0x7f925419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:42156/0 (socket says 172.21.15.193:42156) 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9251d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254109560 0x7f925419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.193:39136/0 (socket says 172.21.15.193:39136) 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 -- 172.21.15.193:0/1927099318 learned_addr learned my addr 172.21.15.193:0/1927099318 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 -- 172.21.15.193:0/1927099318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254109560 msgr2=0x7f925419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254109560 0x7f925419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 -- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254102080 msgr2=0x7f925419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f925259c640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254102080 0x7f925419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254102080 0x7f925419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f9252d9d640 1 -- 172.21.15.193:0/1927099318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92541a04b0 con 0x7f9254109eb0 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.563+0000 7f925259c640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254102080 0x7f925419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.564+0000 7f9252d9d640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 0x7f925419fd80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f924c00cad0 tx=0x7f924c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.564+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c00ae20 con 0x7f9254109eb0 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.564+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92541a07a0 con 0x7f9254109eb0 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.564+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924c005020 con 0x7f9254109eb0 2024-09-21T06:46:34.568 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.564+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c00f6b0 con 0x7f9254109eb0 2024-09-21T06:46:34.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.564+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92541a6f00 con 0x7f9254109eb0 2024-09-21T06:46:34.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.565+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9254102490 con 0x7f9254109eb0 2024-09-21T06:46:34.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.566+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f924c0051d0 con 0x7f9254109eb0 2024-09-21T06:46:34.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.567+0000 7f92437fe640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f923c0b6560 0x7f923c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:46:34.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.567+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4036+0+0 (secure 0 0 0) 0x7f924c0e66f0 con 0x7f9254109eb0 2024-09-21T06:46:34.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.567+0000 7f925259c640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f923c0b6560 0x7f923c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:46:34.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.568+0000 7f925259c640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f923c0b6560 0x7f923c0b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9244009b40 tx=0x7f9244005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:46:34.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.570+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f924c014030 con 0x7f9254109eb0 2024-09-21T06:46:34.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:46:34.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:46:34.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:34.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:46:34.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:34.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:34 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.758 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:34.756+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9254063800 con 0x7f923c0b6560 2024-09-21T06:46:34.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:46:34.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:46:34.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:34.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:46:34.758 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:34.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:34 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T06:46:34.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:34.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:34 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:35.364 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T06:46:35.099+0000 7f1048d4c640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T06:46:35.364 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T06:46:35.105+0000 7f1040b42640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi183 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:46:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:35.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:35 smithi193 ceph-mon[34601]: from='osd.4 ' entity='osd.4' 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi183 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:35.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:35 smithi114 ceph-mon[30654]: from='osd.4 ' entity='osd.4' 2024-09-21T06:46:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi183 2024-09-21T06:46:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-21T06:46:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:46:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T06:46:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:46:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:46:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:35.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:35 smithi183 ceph-mon[34219]: from='osd.4 ' entity='osd.4' 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] boot 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:46:36.814 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:46:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:46:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:46:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] boot 2024-09-21T06:46:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:46:36.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:46:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:46:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: from='client.24451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:46:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T06:46:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: osd.4 [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] boot 2024-09-21T06:46:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T06:46:36.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T06:46:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:37 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/1178311038' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"} : dispatch 2024-09-21T06:46:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:37 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"} : dispatch 2024-09-21T06:46:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:37 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"}]': finished 2024-09-21T06:46:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:37 smithi193 ceph-mon[34601]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:46:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:46:37.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:37 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/1178311038' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"} : dispatch 2024-09-21T06:46:37.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:37 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"} : dispatch 2024-09-21T06:46:37.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:37 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"}]': finished 2024-09-21T06:46:37.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:37 smithi114 ceph-mon[30654]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:46:37.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:46:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:37 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/1178311038' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"} : dispatch 2024-09-21T06:46:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:37 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"} : dispatch 2024-09-21T06:46:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:37 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0798fc1c-9adf-42ec-b897-385b1be1bd71"}]': finished 2024-09-21T06:46:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:37 smithi183 ceph-mon[34219]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T06:46:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:46:38.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:38 smithi114 ceph-mon[30654]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:38.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:38 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/1491177411' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:46:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:38 smithi183 ceph-mon[34219]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:38 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/1491177411' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:46:38.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:38 smithi193 ceph-mon[34601]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:38 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/1491177411' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:46:40.713 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:40 smithi193 ceph-mon[34601]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:40 smithi114 ceph-mon[30654]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:40 smithi183 ceph-mon[34219]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:42.718 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:42 smithi193 ceph-mon[34601]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:42 smithi114 ceph-mon[30654]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:42 smithi183 ceph-mon[34219]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:44 smithi193 ceph-mon[34601]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:44 smithi114 ceph-mon[30654]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:44 smithi183 ceph-mon[34219]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:45.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:46:45.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:45.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:46:45.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:45.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T06:46:45.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:46.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:46 smithi193 ceph-mon[34601]: Deploying daemon osd.5 on smithi193 2024-09-21T06:46:46.745 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:46 smithi193 ceph-mon[34601]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:46.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:46 smithi114 ceph-mon[30654]: Deploying daemon osd.5 on smithi193 2024-09-21T06:46:46.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:46 smithi114 ceph-mon[30654]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:46 smithi183 ceph-mon[34219]: Deploying daemon osd.5 on smithi193 2024-09-21T06:46:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:46 smithi183 ceph-mon[34219]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:48 smithi114 ceph-mon[30654]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:48.829 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:48 smithi193 ceph-mon[34601]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:48 smithi183 ceph-mon[34219]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:50.759 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:50 smithi193 ceph-mon[34601]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:50.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:50 smithi114 ceph-mon[30654]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:50 smithi183 ceph-mon[34219]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:52 smithi114 ceph-mon[30654]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:52 smithi183 ceph-mon[34219]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:52 smithi193 ceph-mon[34601]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:54 smithi114 ceph-mon[30654]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:54 smithi183 ceph-mon[34219]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:54.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:54.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:54 smithi193 ceph-mon[34601]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:55.808 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.806+0000 7f92437fe640 1 -- 172.21.15.193:0/1927099318 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9254063800 con 0x7f923c0b6560 2024-09-21T06:46:55.808 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-09-21T06:46:55.810 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f923c0b6560 msgr2=0x7f923c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f923c0b6560 0x7f923c0b8a70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9244009b40 tx=0x7f9244005e50 comp rx=0 tx=0).stop 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 msgr2=0x7f925419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 0x7f925419fd80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f924c00cad0 tx=0x7f924c00cf90 comp rx=0 tx=0).stop 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 shutdown_connections 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9254109eb0 0x7f925419fd80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f923c0b6560 0x7f923c0b8a70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9254109560 0x7f925419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 --2- 172.21.15.193:0/1927099318 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9254102080 0x7f925419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 >> 172.21.15.193:0/1927099318 conn(0x7f9254079ec0 msgr2=0x7f92541055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 shutdown_connections 2024-09-21T06:46:55.811 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:46:55.809+0000 7f9258ced640 1 -- 172.21.15.193:0/1927099318 wait complete. 2024-09-21T06:46:56.426 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T06:46:56.429 INFO:tasks.cephadm:Deploying osd.6 on smithi114 with /dev/vg_nvme/lv_2... 2024-09-21T06:46:56.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:46:56.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:56 smithi193 ceph-mon[34601]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:56.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:56.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:56.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:56.806 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:46:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:56 smithi114 ceph-mon[30654]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:56 smithi183 ceph-mon[34219]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:46:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:58.412 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 06:46:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T06:46:58.049+0000 7feab6884640 -1 osd.5 0 log_to_monitors true 2024-09-21T06:46:58.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:58 smithi114 ceph-mon[30654]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:58.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:58 smithi114 ceph-mon[30654]: from='osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:46:58.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:58 smithi114 ceph-mon[30654]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:46:58.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:58 smithi193 ceph-mon[34601]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:58.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:58 smithi193 ceph-mon[34601]: from='osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:46:58.803 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:58 smithi193 ceph-mon[34601]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:46:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:58 smithi183 ceph-mon[34219]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:46:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:58 smithi183 ceph-mon[34219]: from='osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:46:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:58 smithi183 ceph-mon[34219]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T06:46:59.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:46:59.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:59.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:46:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:59.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:46:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T06:46:59.507+0000 7feab2806640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T06:46:59.912 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T06:46:59.514+0000 7feaae604640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:46:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T06:46:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T06:46:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:46:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:46:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:46:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:46:59.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:46:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:00.169 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:47:00.202 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_2 2024-09-21T06:47:00.569 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi193 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi193 2024-09-21T06:47:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T06:47:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:47:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:47:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:47:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi193 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi193 to 8544M 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 -- 172.21.15.114:0/398643062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7930105310 msgr2=0x7f793010d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 --2- 172.21.15.114:0/398643062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7930105310 0x7f793010d860 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7918009900 tx=0x7f791802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:47:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 -- 172.21.15.114:0/398643062 shutdown_connections 2024-09-21T06:47:01.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 --2- 172.21.15.114:0/398643062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f793010de20 0x7f7930110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 --2- 172.21.15.114:0/398643062 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7930105310 0x7f793010d860 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 --2- 172.21.15.114:0/398643062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 0x7f7930104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 -- 172.21.15.114:0/398643062 >> 172.21.15.114:0/398643062 conn(0x7f79300796e0 msgr2=0x7f7930079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 -- 172.21.15.114:0/398643062 shutdown_connections 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.159+0000 7f7937000640 1 -- 172.21.15.114:0/398643062 wait complete. 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.160+0000 7f7937000640 1 Processor -- start 2024-09-21T06:47:01.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.160+0000 7f7937000640 1 -- start start 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7937000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 0x7f7930102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7937000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7930105310 0x7f7930102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7937000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 0x7f7930103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7937000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7930113a40 con 0x7f793010de20 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7937000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79301138c0 con 0x7f7930105310 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7937000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7930113bc0 con 0x7f79301049c0 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7935000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 0x7f7930103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7935000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 0x7f7930103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45546/0 (socket says 172.21.15.114:45546) 2024-09-21T06:47:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7935000640 1 -- 172.21.15.114:0/1079919520 learned_addr learned my addr 172.21.15.114:0/1079919520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:47:01.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f792fe00640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 0x7f7930102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:01.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.161+0000 7f7927400640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7930105310 0x7f7930102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:01.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7935000640 1 -- 172.21.15.114:0/1079919520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 msgr2=0x7f7930102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:01.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7935000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 0x7f7930102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:01.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7935000640 1 -- 172.21.15.114:0/1079919520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7930105310 msgr2=0x7f7930102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:01.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7935000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7930105310 0x7f7930102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7935000640 1 -- 172.21.15.114:0/1079919520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79301a8730 con 0x7f793010de20 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f792fe00640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 0x7f7930102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7927400640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7930105310 0x7f7930102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7935000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 0x7f7930103180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f792800efd0 tx=0x7f792800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928003b70 con 0x7f793010de20 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7928003d10 con 0x7f793010de20 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79301a8a20 con 0x7f793010de20 2024-09-21T06:47:01.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79280108a0 con 0x7f793010de20 2024-09-21T06:47:01.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.162+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79301a8ee0 con 0x7f793010de20 2024-09-21T06:47:01.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.163+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793006c200 con 0x7f793010de20 2024-09-21T06:47:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.166+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f7928004b40 con 0x7f793010de20 2024-09-21T06:47:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.168+0000 7f792d600640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6560 0x7f79140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.169+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4781+0+0 (secure 0 0 0) 0x7f79280e6f60 con 0x7f793010de20 2024-09-21T06:47:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.169+0000 7f792fe00640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6560 0x7f79140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:01.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.169+0000 7f792fe00640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6560 0x7f79140b8a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7930104150 tx=0x7f7920005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:01.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.171+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79280ae520 con 0x7f793010de20 2024-09-21T06:47:01.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:01.368+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7930063800 con 0x7f79140b6560 2024-09-21T06:47:01.723 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:47:01.723 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:47:01.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] boot 2024-09-21T06:47:01.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:47:01.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:01.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:47:01.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:47:01.724 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] boot 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:47:01.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:47:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:47:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: osd.5 [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] boot 2024-09-21T06:47:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T06:47:01.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T06:47:01.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:47:01.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:47:01.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:02.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:02 smithi114 ceph-mon[30654]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:47:02.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:02 smithi114 ceph-mon[30654]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:47:02.720 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:02 smithi114 ceph-mon[30654]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:47:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:02 smithi183 ceph-mon[34219]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:47:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:02 smithi183 ceph-mon[34219]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:47:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:02 smithi183 ceph-mon[34219]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:47:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:02 smithi193 ceph-mon[34601]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:47:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:02 smithi193 ceph-mon[34601]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-21T06:47:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:02 smithi193 ceph-mon[34601]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T06:47:03.723 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:03 smithi114 ceph-mon[30654]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:47:03.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:03 smithi183 ceph-mon[34219]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:47:03.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:03 smithi193 ceph-mon[34601]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T06:47:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:04 smithi114 ceph-mon[30654]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:04 smithi183 ceph-mon[34219]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:04 smithi193 ceph-mon[34601]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:05.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:05 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1288841801' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d41daa7-6623-4431-9bbb-fe54c1329dc9"} : dispatch 2024-09-21T06:47:05.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:05 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1288841801' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d41daa7-6623-4431-9bbb-fe54c1329dc9"}]': finished 2024-09-21T06:47:05.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:05 smithi114 ceph-mon[30654]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:47:05.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:05 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:05.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:05 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2998879820' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:47:05.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:05 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1288841801' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d41daa7-6623-4431-9bbb-fe54c1329dc9"} : dispatch 2024-09-21T06:47:05.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:05 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1288841801' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d41daa7-6623-4431-9bbb-fe54c1329dc9"}]': finished 2024-09-21T06:47:05.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:05 smithi183 ceph-mon[34219]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:47:05.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:05 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:05.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:05 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2998879820' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:47:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:05 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1288841801' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6d41daa7-6623-4431-9bbb-fe54c1329dc9"} : dispatch 2024-09-21T06:47:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:05 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1288841801' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d41daa7-6623-4431-9bbb-fe54c1329dc9"}]': finished 2024-09-21T06:47:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:05 smithi193 ceph-mon[34601]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T06:47:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:05 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:05 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2998879820' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:47:06.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:06 smithi114 ceph-mon[30654]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:06 smithi183 ceph-mon[34219]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:06 smithi193 ceph-mon[34601]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:08 smithi114 ceph-mon[30654]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:47:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:08 smithi183 ceph-mon[34219]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:47:08.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:08 smithi193 ceph-mon[34601]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:47:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:10 smithi114 ceph-mon[30654]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T06:47:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:10 smithi183 ceph-mon[34219]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T06:47:10.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:10 smithi193 ceph-mon[34601]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T06:47:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:12 smithi114 ceph-mon[30654]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T06:47:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:12 smithi183 ceph-mon[34219]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T06:47:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:12 smithi193 ceph-mon[34601]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-21T06:47:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:14 smithi114 ceph-mon[30654]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:47:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:14 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:47:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:14 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:14 smithi183 ceph-mon[34219]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:47:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:14 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:47:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:14 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:14 smithi193 ceph-mon[34601]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:47:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:14 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T06:47:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:14 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:15.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:15 smithi114 ceph-mon[30654]: Deploying daemon osd.6 on smithi114 2024-09-21T06:47:15.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:15 smithi183 ceph-mon[34219]: Deploying daemon osd.6 on smithi114 2024-09-21T06:47:15.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:15 smithi193 ceph-mon[34601]: Deploying daemon osd.6 on smithi114 2024-09-21T06:47:16.653 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:16 smithi114 ceph-mon[30654]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T06:47:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:16 smithi183 ceph-mon[34219]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T06:47:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:16 smithi193 ceph-mon[34601]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-21T06:47:18.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:18 smithi114 ceph-mon[30654]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:47:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:18 smithi183 ceph-mon[34219]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:47:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:18 smithi193 ceph-mon[34601]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:47:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:20 smithi183 ceph-mon[34219]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:20.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:20 smithi193 ceph-mon[34601]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:20 smithi114 ceph-mon[30654]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:22.709 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:22 smithi114 ceph-mon[30654]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:22.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:22 smithi183 ceph-mon[34219]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:22.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:22 smithi193 ceph-mon[34601]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:23.845 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:23 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:23.845 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:23 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:23.845 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:23 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:23 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:23 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:23 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:23.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:23 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:23.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:23 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:23.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:23 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:24.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:24 smithi114 ceph-mon[30654]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:24 smithi183 ceph-mon[34219]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:24.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:24 smithi193 ceph-mon[34601]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:26 smithi183 ceph-mon[34219]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:26 smithi193 ceph-mon[34601]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:27.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:26 smithi114 ceph-mon[30654]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:27.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:27.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:27.842 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:27.842 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:27.842 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:28.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.348+0000 7f792d600640 1 -- 172.21.15.114:0/1079919520 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7930063800 con 0x7f79140b6560 2024-09-21T06:47:28.350 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 6 on host 'smithi114' 2024-09-21T06:47:28.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.350+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6560 msgr2=0x7f79140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:28.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.350+0000 7f7937000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6560 0x7f79140b8a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7930104150 tx=0x7f7920005f90 comp rx=0 tx=0).stop 2024-09-21T06:47:28.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.350+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 msgr2=0x7f7930103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:28.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.350+0000 7f7937000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 0x7f7930103180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f792800efd0 tx=0x7f792800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:47:28.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 shutdown_connections 2024-09-21T06:47:28.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f793010de20 0x7f7930103180 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:28.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6560 0x7f79140b8a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:28.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7930105310 0x7f7930102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:28.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 --2- 172.21.15.114:0/1079919520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79301049c0 0x7f7930102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:28.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 >> 172.21.15.114:0/1079919520 conn(0x7f79300796e0 msgr2=0x7f7930100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:28.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 shutdown_connections 2024-09-21T06:47:28.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:47:28.351+0000 7f7937000640 1 -- 172.21.15.114:0/1079919520 wait complete. 2024-09-21T06:47:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:28 smithi183 ceph-mon[34219]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:28 smithi193 ceph-mon[34601]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:28 smithi114 ceph-mon[30654]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:29.257 DEBUG:teuthology.orchestra.run.smithi114:osd.6> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T06:47:29.262 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-09-21T06:47:29.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:47:29.515 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:47:29.735 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:29 smithi114 ceph-mon[30654]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:47:29.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:29 smithi183 ceph-mon[34219]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:47:29.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:29 smithi193 ceph-mon[34601]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T06:47:30.859 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:30 smithi183 ceph-mon[34219]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:30.859 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:30 smithi183 ceph-mon[34219]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:47:30.860 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:30 smithi183 ceph-mon[34219]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:47:30.860 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:30 smithi183 ceph-mon[34219]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:47:30.860 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:30 smithi193 ceph-mon[34601]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:30 smithi193 ceph-mon[34601]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:47:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:30 smithi193 ceph-mon[34601]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:47:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:30 smithi193 ceph-mon[34601]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:47:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.021 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:30 smithi114 ceph-mon[30654]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:31.021 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:30 smithi114 ceph-mon[30654]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T06:47:31.021 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:30 smithi114 ceph-mon[30654]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T06:47:31.021 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:30 smithi114 ceph-mon[30654]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:47:31.021 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-mon[30654]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:47:31.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-mon[30654]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:47:31.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-mon[30654]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' 2024-09-21T06:47:31.817 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T06:47:31.366+0000 7fd762400640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T06:47:31.817 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 06:47:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T06:47:31.387+0000 7fd757a00640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:47:31.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:31 smithi183 ceph-mon[34219]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:47:31.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:31 smithi183 ceph-mon[34219]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:47:31.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:31 smithi183 ceph-mon[34219]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' 2024-09-21T06:47:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:31 smithi193 ceph-mon[34601]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:47:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:31 smithi193 ceph-mon[34601]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T06:47:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:31 smithi193 ceph-mon[34601]: from='osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039]' entity='osd.6' 2024-09-21T06:47:32.186 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:47:32.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-09-21T06:47:32.458 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:47:32.845 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:47:32.845 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:47:32.845 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:32.846 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:32.846 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] boot 2024-09-21T06:47:32.846 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:47:32.846 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] boot 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:47:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.936+0000 7f98c90e8640 1 -- 172.21.15.183:0/3836320476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c41094a0 msgr2=0x7f98c41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.936+0000 7f98c90e8640 1 --2- 172.21.15.183:0/3836320476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c41094a0 0x7f98c41098b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009900 tx=0x7f98b802a410 comp rx=0 tx=0).stop 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 -- 172.21.15.183:0/3836320476 shutdown_connections 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 --2- 172.21.15.183:0/3836320476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c4109df0 0x7f98c4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 --2- 172.21.15.183:0/3836320476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c41094a0 0x7f98c41098b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 --2- 172.21.15.183:0/3836320476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 0x7f98c4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 -- 172.21.15.183:0/3836320476 >> 172.21.15.183:0/3836320476 conn(0x7f98c407b850 msgr2=0x7f98c407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 -- 172.21.15.183:0/3836320476 shutdown_connections 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 -- 172.21.15.183:0/3836320476 wait complete. 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.937+0000 7f98c90e8640 1 Processor -- start 2024-09-21T06:47:32.962 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 -- start start 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 0x7f98c419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c41094a0 0x7f98c419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 0x7f98c419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4113ab0 con 0x7f98c41094a0 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4113930 con 0x7f98c4109df0 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c90e8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4113c30 con 0x7f98c4103b60 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 0x7f98c419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 0x7f98c419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:51488/0 (socket says 172.21.15.183:51488) 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 -- 172.21.15.183:0/95707482 learned_addr learned my addr 172.21.15.183:0/95707482 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c2d9d640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 0x7f98c419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c259c640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c41094a0 0x7f98c419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:32.963 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 -- 172.21.15.183:0/95707482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 msgr2=0x7f98c419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 0x7f98c419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 -- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c41094a0 msgr2=0x7f98c419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c41094a0 0x7f98c419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 -- 172.21.15.183:0/95707482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c41a0450 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c259c640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c41094a0 0x7f98c419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c2d9d640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 0x7f98c419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.938+0000 7f98c359e640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 0x7f98c419fd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009fd0 tx=0x7f98bc00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.939+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc017070 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.939+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c41a0740 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.939+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98bc003e20 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.939+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc010460 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.939+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c41a6ea0 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.940+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c4104de0 con 0x7f98c4109df0 2024-09-21T06:47:32.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.941+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f98bc003970 con 0x7f98c4109df0 2024-09-21T06:47:32.965 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.942+0000 7f98abfff640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98ac0b6560 0x7f98ac0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:32.965 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.942+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5180+0+0 (secure 0 0 0) 0x7f98bc0e6590 con 0x7f98c4109df0 2024-09-21T06:47:32.965 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.942+0000 7f98c2d9d640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98ac0b6560 0x7f98ac0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:32.965 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.943+0000 7f98c2d9d640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98ac0b6560 0x7f98ac0b8a70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009b40 tx=0x7f98b4005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:32.965 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:32.945+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98bc0ada60 con 0x7f98c4109df0 2024-09-21T06:47:33.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:47:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:47:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-21T06:47:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: osd.6 [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] boot 2024-09-21T06:47:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T06:47:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T06:47:33.136 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:33.135+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f98c4063800 con 0x7f98ac0b6560 2024-09-21T06:47:33.798 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:33.799 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:34.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:34.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:34.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:34 smithi183 ceph-mon[34219]: from='client.24494 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:47:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:34 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi114 2024-09-21T06:47:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:34 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-21T06:47:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:34 smithi183 ceph-mon[34219]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:34.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:34 smithi183 ceph-mon[34219]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:47:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:34 smithi193 ceph-mon[34601]: from='client.24494 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:47:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:34 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi114 2024-09-21T06:47:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:34 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-21T06:47:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:34 smithi193 ceph-mon[34601]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:34 smithi193 ceph-mon[34601]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:47:35.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:34 smithi114 ceph-mon[30654]: from='client.24494 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:47:35.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:34 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi114 2024-09-21T06:47:35.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:34 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi114 to 5696M 2024-09-21T06:47:35.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:34 smithi114 ceph-mon[30654]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:35.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:34 smithi114 ceph-mon[30654]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/3011393543' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"} : dispatch 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"} : dispatch 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"}]': finished 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:36 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/1560053957' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:47:37.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/3011393543' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"} : dispatch 2024-09-21T06:47:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"} : dispatch 2024-09-21T06:47:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"}]': finished 2024-09-21T06:47:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:47:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/1560053957' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:47:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/3011393543' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"} : dispatch 2024-09-21T06:47:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"} : dispatch 2024-09-21T06:47:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88b7c23c-2407-4f8e-895e-5349e4fb3083"}]': finished 2024-09-21T06:47:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T06:47:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:36 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/1560053957' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:47:38.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:37 smithi114 ceph-mon[30654]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:38.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:37 smithi183 ceph-mon[34219]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:38.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:37 smithi193 ceph-mon[34601]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:40.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:40 smithi183 ceph-mon[34219]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:40 smithi114 ceph-mon[30654]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:40.865 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:40 smithi193 ceph-mon[34601]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:42 smithi114 ceph-mon[30654]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:42 smithi183 ceph-mon[34219]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:42 smithi193 ceph-mon[34601]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:44.756 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:44 smithi183 ceph-mon[34219]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:44.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:47:44.757 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:44 smithi114 ceph-mon[30654]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:47:44.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:44 smithi193 ceph-mon[34601]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T06:47:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:45.802 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:45 smithi183 ceph-mon[34219]: Deploying daemon osd.7 on smithi183 2024-09-21T06:47:45.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:45 smithi114 ceph-mon[30654]: Deploying daemon osd.7 on smithi183 2024-09-21T06:47:45.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:45 smithi193 ceph-mon[34601]: Deploying daemon osd.7 on smithi183 2024-09-21T06:47:46.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:46 smithi114 ceph-mon[30654]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:46 smithi183 ceph-mon[34219]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:46 smithi193 ceph-mon[34601]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:48 smithi114 ceph-mon[30654]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:48 smithi183 ceph-mon[34219]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:48 smithi193 ceph-mon[34601]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:50 smithi183 ceph-mon[34219]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:50.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:50 smithi114 ceph-mon[30654]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:50 smithi193 ceph-mon[34601]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:51.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:51.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:51.668 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:52 smithi114 ceph-mon[30654]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:52 smithi183 ceph-mon[34219]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:52 smithi193 ceph-mon[34601]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:54 smithi114 ceph-mon[30654]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:54 smithi183 ceph-mon[34219]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.816 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:54.817 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:54.817 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:54 smithi193 ceph-mon[34601]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.188 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.187+0000 7f98abfff640 1 -- 172.21.15.183:0/95707482 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f98c4063800 con 0x7f98ac0b6560 2024-09-21T06:47:55.189 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98ac0b6560 msgr2=0x7f98ac0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98ac0b6560 0x7f98ac0b8a70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009b40 tx=0x7f98b4005e50 comp rx=0 tx=0).stop 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 msgr2=0x7f98c419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 0x7f98c419fd20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009fd0 tx=0x7f98bc00b260 comp rx=0 tx=0).stop 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 shutdown_connections 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98ac0b6560 0x7f98ac0b8a70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98c4109df0 0x7f98c419fd20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:55.191 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98c41094a0 0x7f98c419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:55.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 --2- 172.21.15.183:0/95707482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98c4103b60 0x7f98c419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:55.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 >> 172.21.15.183:0/95707482 conn(0x7f98c407b850 msgr2=0x7f98c4105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:55.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 shutdown_connections 2024-09-21T06:47:55.192 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:47:55.189+0000 7f98c90e8640 1 -- 172.21.15.183:0/95707482 wait complete. 2024-09-21T06:47:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:55 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:55 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:55 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:55 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:55 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:55 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:55 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:47:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:55 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:55 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:55.920 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T06:47:55.923 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-09-21T06:47:55.923 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T06:47:56.180 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:47:56.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:56 smithi193 ceph-mon[34601]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:56.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:56 smithi193 ceph-mon[34601]: from='osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:47:56.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:56 smithi193 ceph-mon[34601]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:47:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:56 smithi114 ceph-mon[30654]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:56 smithi114 ceph-mon[30654]: from='osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:47:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:56 smithi114 ceph-mon[30654]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:47:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:56 smithi183 ceph-mon[34219]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:56.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:56 smithi183 ceph-mon[34219]: from='osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:47:56.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:56 smithi183 ceph-mon[34219]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T06:47:56.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 06:47:56 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T06:47:56.481+0000 7fd05d872640 -1 osd.7 0 log_to_monitors true 2024-09-21T06:47:58.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-mon[34219]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:47:58.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-mon[34219]: from='osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:47:58.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-mon[34219]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:47:58.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:58.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-mon[34219]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:47:58.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-mon[34219]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:58.536 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T06:47:58.196+0000 7fd0597f4640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T06:47:58.536 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 06:47:58 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T06:47:58.207+0000 7fd0555f2640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:47:58.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:58 smithi114 ceph-mon[30654]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:47:58.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:58 smithi114 ceph-mon[30654]: from='osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:58 smithi114 ceph-mon[30654]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:58 smithi114 ceph-mon[30654]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:58 smithi114 ceph-mon[30654]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:58 smithi193 ceph-mon[34601]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:58 smithi193 ceph-mon[34601]: from='osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:58 smithi193 ceph-mon[34601]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:58.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:58 smithi193 ceph-mon[34601]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:47:58.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:58 smithi193 ceph-mon[34601]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-21T06:47:58.951 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:47:58.984 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-09-21T06:47:59.247 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.495 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] boot 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:47:59.496 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:47:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] boot 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:47:59.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:47:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: osd.7 [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] boot 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T06:47:59.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:47:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T06:47:59.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 -- 172.21.15.193:0/2107752996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d81031c0 msgr2=0x7ff2d81035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:59.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2107752996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d81031c0 0x7ff2d81035d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009ca0 tx=0x7ff2c802a8e0 comp rx=0 tx=0).stop 2024-09-21T06:47:59.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 -- 172.21.15.193:0/2107752996 shutdown_connections 2024-09-21T06:47:59.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2107752996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d8108f50 0x7ff2d810f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:59.751 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2107752996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d8108600 0x7ff2d8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2107752996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d81031c0 0x7ff2d81035d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 -- 172.21.15.193:0/2107752996 >> 172.21.15.193:0/2107752996 conn(0x7ff2d8079e30 msgr2=0x7ff2d807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 -- 172.21.15.193:0/2107752996 shutdown_connections 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.726+0000 7ff2e0253640 1 -- 172.21.15.193:0/2107752996 wait complete. 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 Processor -- start 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 -- start start 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d81031c0 0x7ff2d8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d8108600 0x7ff2d8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 0x7ff2d819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8112c70 con 0x7ff2d81031c0 2024-09-21T06:47:59.752 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8112af0 con 0x7ff2d8108600 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2e0253640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8112df0 con 0x7ff2d8108f50 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2de7f0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 0x7ff2d819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2de7f0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 0x7ff2d819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:35208/0 (socket says 172.21.15.193:35208) 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2de7f0640 1 -- 172.21.15.193:0/2445803353 learned_addr learned my addr 172.21.15.193:0/2445803353 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.727+0000 7ff2dd7ee640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d8108600 0x7ff2d8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2de7f0640 1 -- 172.21.15.193:0/2445803353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d8108600 msgr2=0x7ff2d8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2de7f0640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d8108600 0x7ff2d8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2de7f0640 1 -- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d81031c0 msgr2=0x7ff2d8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2ddfef640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d81031c0 0x7ff2d8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2de7f0640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d81031c0 0x7ff2d8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2de7f0640 1 -- 172.21.15.193:0/2445803353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d819f5f0 con 0x7ff2d8108f50 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2de7f0640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 0x7ff2d819eec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000d750 tx=0x7ff2d000db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0017070 con 0x7ff2d8108f50 2024-09-21T06:47:59.753 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00097c0 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004ba0 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d819f8e0 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.728+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81a6040 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.729+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d81035d0 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.731+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7ff2d0009c90 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.732+0000 7ff2d6ffd640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2c00b6560 0x7ff2c00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.732+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5612+0+0 (secure 0 0 0) 0x7ff2d00e6740 con 0x7ff2d8108f50 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.732+0000 7ff2ddfef640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2c00b6560 0x7ff2c00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.733+0000 7ff2ddfef640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2c00b6560 0x7ff2c00b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8004af0 tx=0x7ff2c8005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:47:59.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.735+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2d00ada60 con 0x7ff2d8108f50 2024-09-21T06:47:59.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:47:59.924+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff2d8063800 con 0x7ff2c00b6560 2024-09-21T06:48:00.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:48:00.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:48:00.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi183 2024-09-21T06:48:00.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-21T06:48:00.637 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:48:00.638 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:00.638 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:00.638 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:00.638 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:00.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:48:00.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi183 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:48:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:48:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi183 2024-09-21T06:48:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-21T06:48:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T06:48:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:00.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:01.649 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:01 smithi193 ceph-mon[34601]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:01.650 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:01 smithi193 ceph-mon[34601]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T06:48:01.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:01 smithi114 ceph-mon[30654]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:01.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:01 smithi114 ceph-mon[30654]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T06:48:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:01 smithi183 ceph-mon[34219]: from='client.24526 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:01 smithi183 ceph-mon[34219]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T06:48:02.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:02 smithi114 ceph-mon[30654]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:02.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:02 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/2833118922' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"} : dispatch 2024-09-21T06:48:02.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:02 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"} : dispatch 2024-09-21T06:48:02.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:02 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"}]': finished 2024-09-21T06:48:02.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:02 smithi114 ceph-mon[30654]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:48:02.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:02 smithi183 ceph-mon[34219]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:02 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/2833118922' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"} : dispatch 2024-09-21T06:48:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:02 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"} : dispatch 2024-09-21T06:48:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:02 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"}]': finished 2024-09-21T06:48:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:02 smithi183 ceph-mon[34219]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:48:02.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:02 smithi193 ceph-mon[34601]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:02 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/2833118922' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"} : dispatch 2024-09-21T06:48:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:02 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"} : dispatch 2024-09-21T06:48:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:02 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca4001c9-2aa5-4258-9263-2ed19f7dde20"}]': finished 2024-09-21T06:48:02.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:02 smithi193 ceph-mon[34601]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T06:48:02.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:03 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/3390883218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:48:03.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:03 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/3390883218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:48:03.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:03 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/3390883218' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:48:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:04 smithi114 ceph-mon[30654]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:04 smithi183 ceph-mon[34219]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:04.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:04 smithi193 ceph-mon[34601]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:06.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:06 smithi114 ceph-mon[30654]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:48:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:06 smithi183 ceph-mon[34219]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:48:06.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:06 smithi193 ceph-mon[34601]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:48:08.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:08 smithi193 ceph-mon[34601]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:48:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:08 smithi114 ceph-mon[30654]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:48:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:08 smithi183 ceph-mon[34219]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:48:10.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:10 smithi193 ceph-mon[34601]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T06:48:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:10 smithi114 ceph-mon[30654]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T06:48:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:10 smithi183 ceph-mon[34219]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T06:48:11.592 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:48:11.592 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:11 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:48:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:11 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:11.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T06:48:11.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:11 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:12.657 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:12 smithi193 ceph-mon[34601]: Deploying daemon osd.8 on smithi193 2024-09-21T06:48:12.657 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:12 smithi193 ceph-mon[34601]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:48:12.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:12 smithi114 ceph-mon[30654]: Deploying daemon osd.8 on smithi193 2024-09-21T06:48:12.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:12 smithi114 ceph-mon[30654]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:48:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:12 smithi183 ceph-mon[34219]: Deploying daemon osd.8 on smithi193 2024-09-21T06:48:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:12 smithi183 ceph-mon[34219]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:48:14.791 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:14 smithi193 ceph-mon[34601]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T06:48:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:14 smithi114 ceph-mon[30654]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T06:48:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:14 smithi183 ceph-mon[34219]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T06:48:16.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:16 smithi114 ceph-mon[30654]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:48:16.856 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:16 smithi193 ceph-mon[34601]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:48:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:16 smithi183 ceph-mon[34219]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:48:17.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:17 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:17.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:17 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:17.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:17 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:17.826 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:17 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:17.826 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:17 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:17.826 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:17 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:17.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:17 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:17.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:17 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:17.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:17 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:18.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:18 smithi114 ceph-mon[30654]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:18 smithi183 ceph-mon[34219]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:18 smithi193 ceph-mon[34601]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:20.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:20 smithi193 ceph-mon[34601]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:20.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:20.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:20.798 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:20 smithi114 ceph-mon[30654]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:20.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:20.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:20.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:20 smithi183 ceph-mon[34219]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:20.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:20.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:20.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:21.415 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.413+0000 7ff2d6ffd640 1 -- 172.21.15.193:0/2445803353 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff2d8063800 con 0x7ff2c00b6560 2024-09-21T06:48:21.416 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2c00b6560 msgr2=0x7ff2c00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2c00b6560 0x7ff2c00b8a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8004af0 tx=0x7ff2c8005fd0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 msgr2=0x7ff2d819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 0x7ff2d819eec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000d750 tx=0x7ff2d000db50 comp rx=0 tx=0).stop 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 shutdown_connections 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d8108f50 0x7ff2d819eec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2c00b6560 0x7ff2c00b8a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.418 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d8108600 0x7ff2d8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 --2- 172.21.15.193:0/2445803353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d81031c0 0x7ff2d8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 >> 172.21.15.193:0/2445803353 conn(0x7ff2d8079e30 msgr2=0x7ff2d81066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 shutdown_connections 2024-09-21T06:48:21.419 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:48:21.416+0000 7ff2e0253640 1 -- 172.21.15.193:0/2445803353 wait complete. 2024-09-21T06:48:21.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:21.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:21.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:21.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:21.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:21.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:21.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:21.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:21.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:22.099 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T06:48:22.102 INFO:tasks.cephadm:Deploying osd.9 on smithi114 with /dev/vg_nvme/lv_1... 2024-09-21T06:48:22.102 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:48:22.486 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:22 smithi193 ceph-mon[34601]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:22.777 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:22 smithi114 ceph-mon[30654]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:22.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:22 smithi183 ceph-mon[34219]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:23.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:23 smithi193 ceph-mon[34601]: from='osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:48:23.544 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:23 smithi193 ceph-mon[34601]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:48:23.544 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 06:48:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T06:48:23.493+0000 7f98e44a7640 -1 osd.8 0 log_to_monitors true 2024-09-21T06:48:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:23 smithi114 ceph-mon[30654]: from='osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:48:23.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:23 smithi114 ceph-mon[30654]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:48:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:23 smithi183 ceph-mon[34219]: from='osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:48:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:23 smithi183 ceph-mon[34219]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T06:48:24.612 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T06:48:24.520+0000 7f98e0429640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T06:48:24.612 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T06:48:24.530+0000 7f98dc227640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-mon[34601]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-mon[34601]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-mon[34601]: from='osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-mon[34601]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:24 smithi193 ceph-mon[34601]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:48:24.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:24 smithi114 ceph-mon[30654]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:24 smithi114 ceph-mon[30654]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:48:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:24 smithi114 ceph-mon[30654]: from='osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:48:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:24 smithi114 ceph-mon[30654]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:48:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:24 smithi114 ceph-mon[30654]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:48:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:24 smithi183 ceph-mon[34219]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:24 smithi183 ceph-mon[34219]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T06:48:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:24 smithi183 ceph-mon[34219]: from='osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:48:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:24 smithi183 ceph-mon[34219]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T06:48:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:24 smithi183 ceph-mon[34219]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:48:25.733 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:48:25.733 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:48:25.733 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.733 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.733 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.734 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.743 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:48:25.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi114:vg_nvme/lv_1 2024-09-21T06:48:25.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:48:25.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:25.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:26.140 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.771+0000 7f1a77e00640 1 -- 172.21.15.114:0/753170641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a78105bc0 msgr2=0x7f1a78105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.771+0000 7f1a77e00640 1 --2- 172.21.15.114:0/753170641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a78105bc0 0x7f1a78105fd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009900 tx=0x7f1a6802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 -- 172.21.15.114:0/753170641 shutdown_connections 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 --2- 172.21.15.114:0/753170641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a7810be50 0x7f1a78112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 --2- 172.21.15.114:0/753170641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810b500 0x7f1a7810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 --2- 172.21.15.114:0/753170641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a78105bc0 0x7f1a78105fd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 -- 172.21.15.114:0/753170641 >> 172.21.15.114:0/753170641 conn(0x7f1a78101670 msgr2=0x7f1a78103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 -- 172.21.15.114:0/753170641 shutdown_connections 2024-09-21T06:48:26.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.772+0000 7f1a77e00640 1 -- 172.21.15.114:0/753170641 wait complete. 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 Processor -- start 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 -- start start 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a78105bc0 0x7f1a7819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a7810b500 0x7f1a7819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 0x7f1a781a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78115b10 con 0x7f1a7810be50 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78115990 con 0x7f1a7810b500 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.773+0000 7f1a77e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78115c90 con 0x7f1a78105bc0 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 0x7f1a781a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:26.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 0x7f1a781a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38950/0 (socket says 172.21.15.114:38950) 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 -- 172.21.15.114:0/2753439523 learned_addr learned my addr 172.21.15.114:0/2753439523 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a76a00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a78105bc0 0x7f1a7819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a76000640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a7810b500 0x7f1a7819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 -- 172.21.15.114:0/2753439523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a78105bc0 msgr2=0x7f1a7819c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a78105bc0 0x7f1a7819c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 -- 172.21.15.114:0/2753439523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a7810b500 msgr2=0x7f1a7819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a7810b500 0x7f1a7819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 -- 172.21.15.114:0/2753439523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a781a2490 con 0x7f1a7810be50 2024-09-21T06:48:26.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.774+0000 7f1a77400640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 0x7f1a781a1d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000eb10 tx=0x7f1a7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:26.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.775+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70004b20 con 0x7f1a7810be50 2024-09-21T06:48:26.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.775+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a70004cc0 con 0x7f1a7810be50 2024-09-21T06:48:26.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.775+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a781a2780 con 0x7f1a7810be50 2024-09-21T06:48:26.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.775+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70010670 con 0x7f1a7810be50 2024-09-21T06:48:26.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.775+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a781a8ee0 con 0x7f1a7810be50 2024-09-21T06:48:26.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.776+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a78105fd0 con 0x7f1a7810be50 2024-09-21T06:48:26.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.778+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f1a70005000 con 0x7f1a7810be50 2024-09-21T06:48:26.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.780+0000 7f1a7ca00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b65e0 0x7f1a5c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:26.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.780+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6044+0+0 (secure 0 0 0) 0x7f1a700e7230 con 0x7f1a7810be50 2024-09-21T06:48:26.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.780+0000 7f1a76a00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b65e0 0x7f1a5c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:26.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.781+0000 7f1a76a00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b65e0 0x7f1a5c0b8af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a68031440 tx=0x7f1a68009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:26.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.782+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a70056710 con 0x7f1a7810be50 2024-09-21T06:48:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:48:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:48:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi193 2024-09-21T06:48:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T06:48:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] boot 2024-09-21T06:48:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:48:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:48:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:48:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi193 2024-09-21T06:48:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T06:48:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] boot 2024-09-21T06:48:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:48:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi193 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi193 to 5696M 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: osd.8 [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] boot 2024-09-21T06:48:26.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T06:48:26.915 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T06:48:26.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:26.981+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1a78063800 con 0x7f1a5c0b65e0 2024-09-21T06:48:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:27 smithi114 ceph-mon[30654]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:48:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:27.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:27 smithi183 ceph-mon[34219]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:48:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:27 smithi193 ceph-mon[34601]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T06:48:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:27.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:27.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:28.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:28 smithi114 ceph-mon[30654]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:28.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:28 smithi114 ceph-mon[30654]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:28.628 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:28 smithi114 ceph-mon[30654]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T06:48:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:28 smithi183 ceph-mon[34219]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:28 smithi183 ceph-mon[34219]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:28 smithi183 ceph-mon[34219]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T06:48:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:28 smithi193 ceph-mon[34601]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi114:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:28 smithi193 ceph-mon[34601]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:28 smithi193 ceph-mon[34601]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T06:48:30.717 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:30 smithi114 ceph-mon[30654]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T06:48:30.718 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:30 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1826460778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "76a2c023-5c26-46da-9de8-7d42a9b048df"} : dispatch 2024-09-21T06:48:30.718 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:30 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1826460778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a2c023-5c26-46da-9de8-7d42a9b048df"}]': finished 2024-09-21T06:48:30.718 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:30 smithi114 ceph-mon[30654]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:48:30.718 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:30 smithi183 ceph-mon[34219]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T06:48:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:30 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1826460778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "76a2c023-5c26-46da-9de8-7d42a9b048df"} : dispatch 2024-09-21T06:48:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:30 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1826460778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a2c023-5c26-46da-9de8-7d42a9b048df"}]': finished 2024-09-21T06:48:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:30 smithi183 ceph-mon[34219]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:48:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:30 smithi193 ceph-mon[34601]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-21T06:48:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:30 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1826460778' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "76a2c023-5c26-46da-9de8-7d42a9b048df"} : dispatch 2024-09-21T06:48:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:30 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1826460778' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a2c023-5c26-46da-9de8-7d42a9b048df"}]': finished 2024-09-21T06:48:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:30 smithi193 ceph-mon[34601]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T06:48:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:31 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3894105514' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:48:31.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:31 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3894105514' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:48:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:31 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3894105514' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:48:32.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:32 smithi114 ceph-mon[30654]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T06:48:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:32 smithi183 ceph-mon[34219]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T06:48:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:32 smithi193 ceph-mon[34601]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T06:48:34.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:34 smithi114 ceph-mon[30654]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T06:48:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:34 smithi183 ceph-mon[34219]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T06:48:34.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:34 smithi193 ceph-mon[34601]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T06:48:36.855 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:36 smithi114 ceph-mon[30654]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:48:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:36 smithi183 ceph-mon[34219]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:48:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:36 smithi193 ceph-mon[34601]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:48:38.683 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:38 smithi114 ceph-mon[30654]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:48:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:38 smithi183 ceph-mon[34219]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:48:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:38 smithi193 ceph-mon[34601]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T06:48:40.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:40 smithi114 ceph-mon[30654]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:40.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:48:40.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:40 smithi183 ceph-mon[34219]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:48:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:40 smithi193 ceph-mon[34601]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T06:48:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:41 smithi183 ceph-mon[34219]: Deploying daemon osd.9 on smithi114 2024-09-21T06:48:41.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:41 smithi193 ceph-mon[34601]: Deploying daemon osd.9 on smithi114 2024-09-21T06:48:42.048 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:41 smithi114 ceph-mon[30654]: Deploying daemon osd.9 on smithi114 2024-09-21T06:48:42.697 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:42 smithi114 ceph-mon[30654]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:42 smithi183 ceph-mon[34219]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:42 smithi193 ceph-mon[34601]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:44 smithi114 ceph-mon[30654]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:44 smithi183 ceph-mon[34219]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:44 smithi193 ceph-mon[34601]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:46.733 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:46 smithi114 ceph-mon[30654]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:46 smithi183 ceph-mon[34219]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:46 smithi193 ceph-mon[34601]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:48 smithi183 ceph-mon[34219]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:48 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:48 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:48 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:48.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:48 smithi193 ceph-mon[34601]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:48 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:48 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:48 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:48.990 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:48 smithi114 ceph-mon[30654]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:48.990 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:48 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:48.991 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:48 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:48.991 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:48 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:50 smithi183 ceph-mon[34219]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:50 smithi193 ceph-mon[34601]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:51.040 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:50 smithi114 ceph-mon[30654]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:52 smithi183 ceph-mon[34219]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:52.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:52.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:52.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:52.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:52 smithi193 ceph-mon[34601]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:52 smithi114 ceph-mon[30654]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:53.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:53.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:53.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.581+0000 7f1a7ca00640 1 -- 172.21.15.114:0/2753439523 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a78063800 con 0x7f1a5c0b65e0 2024-09-21T06:48:53.583 INFO:teuthology.orchestra.run.smithi114.stdout:Created osd(s) 9 on host 'smithi114' 2024-09-21T06:48:53.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b65e0 msgr2=0x7f1a5c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b65e0 0x7f1a5c0b8af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a68031440 tx=0x7f1a68009560 comp rx=0 tx=0).stop 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 msgr2=0x7f1a781a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 0x7f1a781a1d60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000eb10 tx=0x7f1a7000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 shutdown_connections 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810be50 0x7f1a781a1d60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b65e0 0x7f1a5c0b8af0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a7810b500 0x7f1a7819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 --2- 172.21.15.114:0/2753439523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a78105bc0 0x7f1a7819c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:53.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 >> 172.21.15.114:0/2753439523 conn(0x7f1a78101670 msgr2=0x7f1a78103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:53.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 shutdown_connections 2024-09-21T06:48:53.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:48:53.584+0000 7f1a77e00640 1 -- 172.21.15.114:0/2753439523 wait complete. 2024-09-21T06:48:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:53.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:48:53.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:53.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:54.521 DEBUG:teuthology.orchestra.run.smithi114:osd.9> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T06:48:54.526 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-09-21T06:48:54.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:48:54.784 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:48:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:54 smithi114 ceph-mon[30654]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:54 smithi114 ceph-mon[30654]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:48:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:54 smithi183 ceph-mon[34219]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:54 smithi183 ceph-mon[34219]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:48:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:54 smithi193 ceph-mon[34601]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:54 smithi193 ceph-mon[34601]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T06:48:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:48:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:48:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:48:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:55.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:48:55.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T06:48:55.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:55.816 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T06:48:55.622+0000 7f453ee00640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T06:48:55.816 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 06:48:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T06:48:55.636+0000 7f4539e00640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T06:48:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:55 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]} : dispatch 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: from='osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi114", "root=default"]}]': finished 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T06:48:55.952 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:55 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:56 smithi114 ceph-mon[30654]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:56 smithi183 ceph-mon[34219]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:56.946 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:56.947 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:56.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:56 smithi193 ceph-mon[34601]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:56.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:56.947 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:57.635 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-21T06:48:57.663 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-09-21T06:48:57.787 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:57 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:48:57.787 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:57 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:48:57.787 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:57 smithi114 ceph-mon[30654]: osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] boot 2024-09-21T06:48:57.787 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:57 smithi114 ceph-mon[30654]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:48:57.787 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:57 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:48:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:57 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:48:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:57 smithi193 ceph-mon[34601]: osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] boot 2024-09-21T06:48:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:57 smithi193 ceph-mon[34601]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:48:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:57.915 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:48:57.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:57 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:48:57.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:57 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:48:57.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:57 smithi183 ceph-mon[34219]: osd.9 [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] boot 2024-09-21T06:48:57.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:57 smithi183 ceph-mon[34219]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T06:48:57.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T06:48:58.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 -- 172.21.15.183:0/1596963562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 msgr2=0x7f66a01087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:58.435 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 --2- 172.21.15.183:0/1596963562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a01087d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6694009ca0 tx=0x7f669402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 -- 172.21.15.183:0/1596963562 shutdown_connections 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 --2- 172.21.15.183:0/1596963562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a0108d10 0x7f66a010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 --2- 172.21.15.183:0/1596963562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a01087d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 --2- 172.21.15.183:0/1596963562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a0102a80 0x7f66a0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 -- 172.21.15.183:0/1596963562 >> 172.21.15.183:0/1596963562 conn(0x7f66a0079740 msgr2=0x7f66a0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 -- 172.21.15.183:0/1596963562 shutdown_connections 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.422+0000 7f66a4a41640 1 -- 172.21.15.183:0/1596963562 wait complete. 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.423+0000 7f66a4a41640 1 Processor -- start 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.423+0000 7f66a4a41640 1 -- start start 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a0102a80 0x7f66a019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:58.436 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a0108d10 0x7f66a019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0113aa0 con 0x7f66a0102a80 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0113920 con 0x7f66a01083c0 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0113c20 con 0x7f66a0108d10 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59828/0 (socket says 172.21.15.183:59828) 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 -- 172.21.15.183:0/2716794427 learned_addr learned my addr 172.21.15.183:0/2716794427 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669ed9d640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a0108d10 0x7f66a019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 -- 172.21.15.183:0/2716794427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a0108d10 msgr2=0x7f66a019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a0108d10 0x7f66a019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 -- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a0102a80 msgr2=0x7f66a019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a0102a80 0x7f66a019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:48:58.437 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 -- 172.21.15.183:0/2716794427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a01a03e0 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f669dd9b640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a019aa10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6694006010 tx=0x7f66940319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694003bf0 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6694003d90 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a01a0670 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.424+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669402fb60 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.425+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01a6e30 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.426+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a0103d00 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.427+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f669402fd00 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.428+0000 7f668f7fe640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66880b6560 0x7f66880b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.428+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6476+0+0 (secure 0 0 0) 0x7f6694108210 con 0x7f66a01083c0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.428+0000 7f669e59c640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66880b6560 0x7f66880b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.429+0000 7f669e59c640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66880b6560 0x7f66880b8a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6698005fd0 tx=0x7f6698005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:48:58.438 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.431+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f66940cf1d0 con 0x7f66a01083c0 2024-09-21T06:48:58.626 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:48:58.625+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f66a0063800 con 0x7f66880b6560 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi114 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='client.24572 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:58.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:58 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:58.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:58.941 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi114 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:58.942 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='client.24572 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:58.943 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:58.943 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:58.943 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:58 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi114 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:48:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi114 to 4272M 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='client.24572 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:48:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:58 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:48:59.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:48:59 smithi193 ceph-mon[34601]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:00.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:48:59 smithi114 ceph-mon[30654]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:48:59 smithi183 ceph-mon[34219]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:01.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:01 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/707512638' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"} : dispatch 2024-09-21T06:49:01.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:01 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"} : dispatch 2024-09-21T06:49:01.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:01 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"}]': finished 2024-09-21T06:49:01.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:01 smithi183 ceph-mon[34219]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:49:01.754 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:01.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:01 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/707512638' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"} : dispatch 2024-09-21T06:49:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:01 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"} : dispatch 2024-09-21T06:49:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:01 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"}]': finished 2024-09-21T06:49:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:01 smithi114 ceph-mon[30654]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:49:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:01 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/707512638' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"} : dispatch 2024-09-21T06:49:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:01 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"} : dispatch 2024-09-21T06:49:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:01 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cf450ee-20ef-4871-afbf-3f9dc1d0c45c"}]': finished 2024-09-21T06:49:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:01 smithi193 ceph-mon[34601]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T06:49:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:02.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:02 smithi114 ceph-mon[30654]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:02.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:02 smithi114 ceph-mon[30654]: from='client.? 172.21.15.183:0/1107461336' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:49:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:02 smithi183 ceph-mon[34219]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:02 smithi183 ceph-mon[34219]: from='client.? 172.21.15.183:0/1107461336' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:49:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:02 smithi193 ceph-mon[34601]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:02 smithi193 ceph-mon[34601]: from='client.? 172.21.15.183:0/1107461336' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:49:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:04 smithi114 ceph-mon[30654]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:04 smithi183 ceph-mon[34219]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:04.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:04 smithi193 ceph-mon[34601]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:06.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:06 smithi114 ceph-mon[30654]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:06 smithi183 ceph-mon[34219]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:06.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:06 smithi193 ceph-mon[34601]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:08.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:08 smithi183 ceph-mon[34219]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:08 smithi114 ceph-mon[30654]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:08 smithi193 ceph-mon[34601]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:10 smithi114 ceph-mon[30654]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:10 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:49:10.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:10 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:10.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:10 smithi183 ceph-mon[34219]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:10.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:10 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:49:10.838 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:10 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:10 smithi193 ceph-mon[34601]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:10 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T06:49:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:10 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:11.805 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:11 smithi183 ceph-mon[34219]: Deploying daemon osd.10 on smithi183 2024-09-21T06:49:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:11 smithi114 ceph-mon[30654]: Deploying daemon osd.10 on smithi183 2024-09-21T06:49:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:11 smithi193 ceph-mon[34601]: Deploying daemon osd.10 on smithi183 2024-09-21T06:49:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:12 smithi114 ceph-mon[30654]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:12 smithi183 ceph-mon[34219]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:12.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:12 smithi193 ceph-mon[34601]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:14 smithi114 ceph-mon[30654]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:14.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:14 smithi183 ceph-mon[34219]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:14 smithi193 ceph-mon[34601]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:16.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:16 smithi114 ceph-mon[30654]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:16.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:16.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:16.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:16.819 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:16 smithi183 ceph-mon[34219]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:16.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:16.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:16.820 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:16 smithi193 ceph-mon[34601]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:18.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:18 smithi114 ceph-mon[30654]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:18 smithi183 ceph-mon[34219]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:18 smithi193 ceph-mon[34601]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:20.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.086+0000 7f668f7fe640 1 -- 172.21.15.183:0/2716794427 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f66a0063800 con 0x7f66880b6560 2024-09-21T06:49:20.088 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-09-21T06:49:20.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66880b6560 msgr2=0x7f66880b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:20.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66880b6560 0x7f66880b8a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6698005fd0 tx=0x7f6698005850 comp rx=0 tx=0).stop 2024-09-21T06:49:20.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 msgr2=0x7f66a019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:20.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a019aa10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6694006010 tx=0x7f66940319f0 comp rx=0 tx=0).stop 2024-09-21T06:49:20.090 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 shutdown_connections 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66a0108d10 0x7f66a019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66880b6560 0x7f66880b8a70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66a01083c0 0x7f66a019aa10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 --2- 172.21.15.183:0/2716794427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66a0102a80 0x7f66a019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.088+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 >> 172.21.15.183:0/2716794427 conn(0x7f66a0079740 msgr2=0x7f66a0105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.089+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 shutdown_connections 2024-09-21T06:49:20.091 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-21T06:49:20.089+0000 7f66a4a41640 1 -- 172.21.15.183:0/2716794427 wait complete. 2024-09-21T06:49:20.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:20.816 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T06:49:20.818 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-09-21T06:49:20.818 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T06:49:21.075 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:49:22.762 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:22 smithi193 ceph-mon[34601]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:22.762 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:22 smithi193 ceph-mon[34601]: from='osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:49:22.762 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:22 smithi193 ceph-mon[34601]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:49:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:22 smithi114 ceph-mon[30654]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:22 smithi114 ceph-mon[30654]: from='osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:49:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:22 smithi114 ceph-mon[30654]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:49:22.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:22 smithi183 ceph-mon[34219]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:22.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:22 smithi183 ceph-mon[34219]: from='osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:49:22.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:22 smithi183 ceph-mon[34219]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T06:49:22.865 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 06:49:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T06:49:22.470+0000 7f68eeb6d640 -1 osd.10 0 log_to_monitors true 2024-09-21T06:49:23.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:49:23.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: from='osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:49:23.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:49:23.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:23.766 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:49:23.767 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:49:23.767 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:49:23.767 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:23 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:23.769 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-21T06:49:23.794 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-09-21T06:49:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:49:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: from='osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:49:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:49:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:49:23.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:49:23.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:49:23.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:23 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:23.864 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T06:49:23.528+0000 7f68eaaef640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T06:49:23.864 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T06:49:23.542+0000 7f68e68ed640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:49:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T06:49:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: from='osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:49:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T06:49:23.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:23.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-21T06:49:23.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-21T06:49:23.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T06:49:23.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:23 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:24.053 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 -- 172.21.15.193:0/3465346438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530108cf0 msgr2=0x7fb53010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 --2- 172.21.15.193:0/3465346438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530108cf0 0x7fb53010f520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002a410 comp rx=0 tx=0).stop 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 -- 172.21.15.193:0/3465346438 shutdown_connections 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 --2- 172.21.15.193:0/3465346438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530108cf0 0x7fb53010f520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 --2- 172.21.15.193:0/3465346438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5301083a0 0x7fb5301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 --2- 172.21.15.193:0/3465346438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb530102a60 0x7fb530102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 -- 172.21.15.193:0/3465346438 >> 172.21.15.193:0/3465346438 conn(0x7fb530079730 msgr2=0x7fb530079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 -- 172.21.15.193:0/3465346438 shutdown_connections 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.588+0000 7fb5375fc640 1 -- 172.21.15.193:0/3465346438 wait complete. 2024-09-21T06:49:24.592 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 Processor -- start 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 -- start start 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 0x7fb530199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5301083a0 0x7fb530199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb530108cf0 0x7fb53019ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb5301129c0 con 0x7fb5301083a0 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb530112840 con 0x7fb530108cf0 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb5375fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb530112b40 con 0x7fb530102a60 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb535398640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 0x7fb530199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb535398640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 0x7fb530199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:60128/0 (socket says 172.21.15.193:60128) 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb535398640 1 -- 172.21.15.193:0/1102010184 learned_addr learned my addr 172.21.15.193:0/1102010184 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T06:49:24.593 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb535b99640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb530108cf0 0x7fb53019ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.589+0000 7fb535398640 1 -- 172.21.15.193:0/1102010184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb530108cf0 msgr2=0x7fb53019ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb535398640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb530108cf0 0x7fb53019ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb535398640 1 -- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5301083a0 msgr2=0x7fb530199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb535398640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5301083a0 0x7fb530199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb535398640 1 -- 172.21.15.193:0/1102010184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53019f3e0 con 0x7fb530102a60 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb535398640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 0x7fb530199470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00d990 tx=0x7fb52c00de50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c01b070 con 0x7fb530102a60 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb52c00edf0 con 0x7fb530102a60 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53019f6d0 con 0x7fb530102a60 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c014800 con 0x7fb530102a60 2024-09-21T06:49:24.594 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.590+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53006ae60 con 0x7fb530102a60 2024-09-21T06:49:24.595 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.591+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb530102e70 con 0x7fb530102a60 2024-09-21T06:49:24.595 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.592+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fb52c01e070 con 0x7fb530102a60 2024-09-21T06:49:24.595 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.593+0000 7fb5267fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb51c0b65e0 0x7fb51c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:24.595 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.594+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6942+0+0 (secure 0 0 0) 0x7fb52c0e8c40 con 0x7fb530102a60 2024-09-21T06:49:24.595 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.594+0000 7fb534b97640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb51c0b65e0 0x7fb51c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:24.596 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.594+0000 7fb534b97640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb51c0b65e0 0x7fb51c0b8af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb530107ad0 tx=0x7fb528009440 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:24.598 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.597+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb52c025070 con 0x7fb530102a60 2024-09-21T06:49:24.789 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:24.787+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb53006bb50 con 0x7fb51c0b65e0 2024-09-21T06:49:24.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:24.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:24.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.789 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] boot 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:49:24.790 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:24.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] boot 2024-09-21T06:49:24.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:49:24.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:24.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:24.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: osd.10 [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] boot 2024-09-21T06:49:24.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T06:49:24.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T06:49:25.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi183 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:25 smithi114 ceph-mon[30654]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:49:25.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi183 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:25.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:25 smithi183 ceph-mon[34219]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:49:25.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:49:25.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:49:25.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi183 2024-09-21T06:49:25.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-21T06:49:25.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T06:49:25.894 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T06:49:25.895 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:25.895 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:25 smithi193 ceph-mon[34601]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T06:49:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:26 smithi114 ceph-mon[30654]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:49:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:26 smithi114 ceph-mon[30654]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:26.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:26 smithi114 ceph-mon[30654]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T06:49:26.856 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:26 smithi193 ceph-mon[34601]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:49:26.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:26 smithi193 ceph-mon[34601]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:26.857 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:26 smithi193 ceph-mon[34601]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T06:49:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:26 smithi183 ceph-mon[34219]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:49:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:26 smithi183 ceph-mon[34219]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:26 smithi183 ceph-mon[34219]: osdmap e66: 11 total, 11 up, 11 in 2024-09-21T06:49:27.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:27 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/903864015' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"} : dispatch 2024-09-21T06:49:27.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:27 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"} : dispatch 2024-09-21T06:49:27.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:27 smithi193 ceph-mon[34601]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"}]': finished 2024-09-21T06:49:27.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:27 smithi193 ceph-mon[34601]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:49:27.809 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:27 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/903864015' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"} : dispatch 2024-09-21T06:49:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:27 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"} : dispatch 2024-09-21T06:49:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:27 smithi114 ceph-mon[30654]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"}]': finished 2024-09-21T06:49:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:27 smithi114 ceph-mon[30654]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:49:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:27 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/903864015' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"} : dispatch 2024-09-21T06:49:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:27 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"} : dispatch 2024-09-21T06:49:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:27 smithi183 ceph-mon[34219]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bdaf146-f4e6-4675-834d-1d130fa491ae"}]': finished 2024-09-21T06:49:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:27 smithi183 ceph-mon[34219]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T06:49:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:28.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:28 smithi114 ceph-mon[30654]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:28.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:28 smithi114 ceph-mon[30654]: from='client.? 172.21.15.193:0/1868599022' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:49:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:28 smithi183 ceph-mon[34219]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:28 smithi183 ceph-mon[34219]: from='client.? 172.21.15.193:0/1868599022' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:49:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:28 smithi193 ceph-mon[34601]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:28 smithi193 ceph-mon[34601]: from='client.? 172.21.15.193:0/1868599022' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T06:49:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:30 smithi114 ceph-mon[30654]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:49:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:30 smithi183 ceph-mon[34219]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:49:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:30 smithi193 ceph-mon[34601]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T06:49:32.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:32 smithi114 ceph-mon[30654]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:49:32.827 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:32 smithi193 ceph-mon[34601]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:49:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:32 smithi183 ceph-mon[34219]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T06:49:34.527 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:34 smithi193 ceph-mon[34601]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:49:34.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:34 smithi114 ceph-mon[30654]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:49:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:34 smithi183 ceph-mon[34219]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T06:49:36.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:36 smithi193 ceph-mon[34601]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T06:49:36.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:49:36.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:36 smithi114 ceph-mon[30654]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T06:49:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:49:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:36 smithi183 ceph-mon[34219]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T06:49:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T06:49:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:37.610 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:37 smithi193 ceph-mon[34601]: Deploying daemon osd.11 on smithi193 2024-09-21T06:49:37.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:37 smithi114 ceph-mon[30654]: Deploying daemon osd.11 on smithi193 2024-09-21T06:49:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:37 smithi183 ceph-mon[34219]: Deploying daemon osd.11 on smithi193 2024-09-21T06:49:38.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:38 smithi193 ceph-mon[34601]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:49:38.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:38 smithi114 ceph-mon[30654]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:49:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:38 smithi183 ceph-mon[34219]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-21T06:49:40.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:40 smithi193 ceph-mon[34601]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:49:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:40 smithi114 ceph-mon[30654]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:49:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:40 smithi183 ceph-mon[34219]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T06:49:42.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:42 smithi193 ceph-mon[34601]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:42.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:42.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:42.642 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:42 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:42 smithi114 ceph-mon[30654]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:42.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:42 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:42 smithi183 ceph-mon[34219]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:42 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:44 smithi193 ceph-mon[34601]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:44 smithi114 ceph-mon[30654]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:44.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:44.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:44 smithi183 ceph-mon[34219]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:45.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.598+0000 7fb5267fc640 1 -- 172.21.15.193:0/1102010184 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb53006bb50 con 0x7fb51c0b65e0 2024-09-21T06:49:45.600 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-09-21T06:49:45.602 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.600+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb51c0b65e0 msgr2=0x7fb51c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:45.602 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.600+0000 7fb5375fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb51c0b65e0 0x7fb51c0b8af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb530107ad0 tx=0x7fb528009440 comp rx=0 tx=0).stop 2024-09-21T06:49:45.602 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.600+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 msgr2=0x7fb530199470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:45.602 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.600+0000 7fb5375fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 0x7fb530199470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb52c00d990 tx=0x7fb52c00de50 comp rx=0 tx=0).stop 2024-09-21T06:49:45.602 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 shutdown_connections 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb51c0b65e0 0x7fb51c0b8af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb530108cf0 0x7fb53019ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5301083a0 0x7fb530199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 --2- 172.21.15.193:0/1102010184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb530102a60 0x7fb530199470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 >> 172.21.15.193:0/1102010184 conn(0x7fb530079730 msgr2=0x7fb530105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 shutdown_connections 2024-09-21T06:49:45.603 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:45.601+0000 7fb5375fc640 1 -- 172.21.15.193:0/1102010184 wait complete. 2024-09-21T06:49:46.288 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T06:49:46.290 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T06:49:46.290 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:49:46.592 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:46 smithi193 ceph-mon[34601]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:46.593 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:46.593 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:46.593 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:46.668 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:49:46.807 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:46 smithi114 ceph-mon[30654]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:46.807 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:46.807 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:46.807 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:46 smithi183 ceph-mon[34219]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:49:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:47.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 -- 172.21.15.114:0/2396950550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 msgr2=0x7f3858112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 --2- 172.21.15.114:0/2396950550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f3858112670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3850009ca0 tx=0x7f385002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 -- 172.21.15.114:0/2396950550 shutdown_connections 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 --2- 172.21.15.114:0/2396950550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f3858112670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 --2- 172.21.15.114:0/2396950550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f385810b4f0 0x7f385810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 --2- 172.21.15.114:0/2396950550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3858105bb0 0x7f3858105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.332+0000 7f3857e00640 1 -- 172.21.15.114:0/2396950550 >> 172.21.15.114:0/2396950550 conn(0x7f3858101680 msgr2=0x7f3858103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.333+0000 7f3857e00640 1 -- 172.21.15.114:0/2396950550 shutdown_connections 2024-09-21T06:49:47.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.333+0000 7f3857e00640 1 -- 172.21.15.114:0/2396950550 wait complete. 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 Processor -- start 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 -- start start 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3858105bb0 0x7f385819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f385810b4f0 0x7f385819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f38581a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3858115b00 con 0x7f385810be40 2024-09-21T06:49:47.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3858115980 con 0x7f385810b4f0 2024-09-21T06:49:47.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3858115c80 con 0x7f3858105bb0 2024-09-21T06:49:47.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.334+0000 7f3857400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f38581a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:47.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3858105bb0 0x7f385819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:47.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3857400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f38581a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42970/0 (socket says 172.21.15.114:42970) 2024-09-21T06:49:47.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3857400640 1 -- 172.21.15.114:0/16554798 learned_addr learned my addr 172.21.15.114:0/16554798 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:49:47.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856000640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f385810b4f0 0x7f385819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 -- 172.21.15.114:0/16554798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f385810b4f0 msgr2=0x7f385819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f385810b4f0 0x7f385819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 -- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 msgr2=0x7f38581a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f38581a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 -- 172.21.15.114:0/16554798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38581a2420 con 0x7f3858105bb0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.335+0000 7f3856a00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3858105bb0 0x7f385819c4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f384800e9f0 tx=0x7f384800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.336+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384800ccc0 con 0x7f3858105bb0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.336+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38480108e0 con 0x7f3858105bb0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.336+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38581a2710 con 0x7f3858105bb0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.336+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848018880 con 0x7f3858105bb0 2024-09-21T06:49:47.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.336+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38581a8e70 con 0x7f3858105bb0 2024-09-21T06:49:47.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.337+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3858105fc0 con 0x7f3858105bb0 2024-09-21T06:49:47.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.339+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f3848010430 con 0x7f3858105bb0 2024-09-21T06:49:47.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.341+0000 7f384f400640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f383c0b65e0 0x7f383c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:47.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.341+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7024+0+0 (secure 0 0 0) 0x7f38480e73a0 con 0x7f3858105bb0 2024-09-21T06:49:47.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.341+0000 7f3856000640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f383c0b65e0 0x7f383c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:47.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.342+0000 7f3856000640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f383c0b65e0 0x7f383c0b8af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3858104f60 tx=0x7f3844005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:47.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.343+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38480b24a0 con 0x7f3858105bb0 2024-09-21T06:49:47.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.532+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f385802e640 con 0x7f3858105bb0 2024-09-21T06:49:47.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.532+0000 7f384f400640 1 -- 172.21.15.114:0/16554798 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f38480b73e0 con 0x7f3858105bb0 2024-09-21T06:49:47.534 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:49:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.534+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f383c0b65e0 msgr2=0x7f383c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.534+0000 7f3857e00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f383c0b65e0 0x7f383c0b8af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3858104f60 tx=0x7f3844005f90 comp rx=0 tx=0).stop 2024-09-21T06:49:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.534+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3858105bb0 msgr2=0x7f385819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.534+0000 7f3857e00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3858105bb0 0x7f385819c4b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f384800e9f0 tx=0x7f384800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 shutdown_connections 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f385810be40 0x7f38581a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f383c0b65e0 0x7f383c0b8af0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f385810b4f0 0x7f385819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 --2- 172.21.15.114:0/16554798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3858105bb0 0x7f385819c4b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 >> 172.21.15.114:0/16554798 conn(0x7f3858101680 msgr2=0x7f385810ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 shutdown_connections 2024-09-21T06:49:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:47.535+0000 7f3857e00640 1 -- 172.21.15.114:0/16554798 wait complete. 2024-09-21T06:49:48.021 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 06:49:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T06:49:47.959+0000 7f8c376cd640 -1 osd.11 0 log_to_monitors true 2024-09-21T06:49:48.371 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726901364,"num_in_osds":12,"osd_in_since":1726901367,"num_remapped_pgs":0} 2024-09-21T06:49:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:48 smithi114 ceph-mon[30654]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:48 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/16554798' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:48 smithi114 ceph-mon[30654]: from='osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:49:48.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:48 smithi114 ceph-mon[30654]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:49:48.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:48 smithi193 ceph-mon[34601]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:48.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:48 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/16554798' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:48.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:48 smithi193 ceph-mon[34601]: from='osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:49:48.816 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:48 smithi193 ceph-mon[34601]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:49:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:48 smithi183 ceph-mon[34219]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:48 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/16554798' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:48 smithi183 ceph-mon[34219]: from='osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:49:48.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:48 smithi183 ceph-mon[34219]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T06:49:49.372 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:49:49.751 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:49:49.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:49:49.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:49 smithi114 ceph-mon[30654]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:49:49.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:49.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:49.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:49 smithi183 ceph-mon[34219]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:49:49.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:49 smithi193 ceph-mon[34601]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-21T06:49:50.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.369+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2682956876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f4109c40 msgr2=0x7fa3f4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.369+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2682956876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f4109c40 0x7fa3f4110470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009ca0 tx=0x7fa3ec02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2682956876 shutdown_connections 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2682956876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f4109c40 0x7fa3f4110470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2682956876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3f41092f0 0x7fa3f4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2682956876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 0x7fa3f4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2682956876 >> 172.21.15.114:0/2682956876 conn(0x7fa3f40ff440 msgr2=0x7fa3f4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2682956876 shutdown_connections 2024-09-21T06:49:50.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.370+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2682956876 wait complete. 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 Processor -- start 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 -- start start 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 0x7fa3f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f41092f0 0x7fa3f4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3f4109c40 0x7fa3f4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113900 con 0x7fa3f41092f0 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113780 con 0x7fa3f4109c40 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.371+0000 7fa3f9e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4113a80 con 0x7fa3f41039b0 2024-09-21T06:49:50.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f41092f0 0x7fa3f4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:50.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f41092f0 0x7fa3f4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42986/0 (socket says 172.21.15.114:42986) 2024-09-21T06:49:50.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f3e00640 1 -- 172.21.15.114:0/2280765600 learned_addr learned my addr 172.21.15.114:0/2280765600 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:49:50.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 0x7fa3f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f9400640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3f4109c40 0x7fa3f4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 -- 172.21.15.114:0/2280765600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3f4109c40 msgr2=0x7fa3f4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3f4109c40 0x7fa3f4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 -- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f41092f0 msgr2=0x7fa3f4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f41092f0 0x7fa3f4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 -- 172.21.15.114:0/2280765600 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41a8610 con 0x7fa3f41039b0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.372+0000 7fa3f8a00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 0x7fa3f4075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e9f0 tx=0x7fa3e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.373+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800ccc0 con 0x7fa3f41039b0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.373+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e8005020 con 0x7fa3f41039b0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.373+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f41a8900 con 0x7fa3f41039b0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.373+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004a00 con 0x7fa3f41039b0 2024-09-21T06:49:50.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.373+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a8df0 con 0x7fa3f41039b0 2024-09-21T06:49:50.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.374+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f4103dc0 con 0x7fa3f41039b0 2024-09-21T06:49:50.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.377+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fa3e8004ba0 con 0x7fa3f41039b0 2024-09-21T06:49:50.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.378+0000 7fa3f1600640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:50.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.379+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7062+0+0 (secure 0 0 0) 0x7fa3e80e6ed0 con 0x7fa3f41039b0 2024-09-21T06:49:50.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.379+0000 7fa3f3e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:50.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.379+0000 7fa3f3e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4102b40 tx=0x7fa3e000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:50.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.382+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3e80adba0 con 0x7fa3f41039b0 2024-09-21T06:49:50.521 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T06:49:50.178+0000 7f8c3364f640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T06:49:50.522 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T06:49:50.191+0000 7f8c2b445640 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T06:49:50.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.580+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa3f402e640 con 0x7fa3f41039b0 2024-09-21T06:49:50.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.581+0000 7fa3f1600640 1 -- 172.21.15.114:0/2280765600 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7fa3e80b1f80 con 0x7fa3f41039b0 2024-09-21T06:49:50.583 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3dc0b6560 msgr2=0x7fa3dc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4102b40 tx=0x7fa3e000b410 comp rx=0 tx=0).stop 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 msgr2=0x7fa3f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 0x7fa3f4075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e9f0 tx=0x7fa3e800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 shutdown_connections 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3dc0b6560 0x7fa3dc0b8a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3f4109c40 0x7fa3f4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3f41092f0 0x7fa3f4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 --2- 172.21.15.114:0/2280765600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f41039b0 0x7fa3f4075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 >> 172.21.15.114:0/2280765600 conn(0x7fa3f40ff440 msgr2=0x7fa3f41068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 shutdown_connections 2024-09-21T06:49:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:50.584+0000 7fa3f9e00640 1 -- 172.21.15.114:0/2280765600 wait complete. 2024-09-21T06:49:50.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: Detected new or changed devices on smithi193 2024-09-21T06:49:50.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T06:49:50.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:50.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T06:49:50.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:50.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:50.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:50 smithi114 ceph-mon[30654]: from='osd.11 ' entity='osd.11' 2024-09-21T06:49:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: Detected new or changed devices on smithi193 2024-09-21T06:49:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T06:49:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:50.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T06:49:50.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:50.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:50.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:50 smithi183 ceph-mon[34219]: from='osd.11 ' entity='osd.11' 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: Detected new or changed devices on smithi193 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi193 to 4272M 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: osdmap e69: 12 total, 11 up, 12 in 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:50 smithi193 ceph-mon[34601]: from='osd.11 ' entity='osd.11' 2024-09-21T06:49:51.387 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":69,"num_osds":12,"num_up_osds":11,"osd_up_since":1726901364,"num_in_osds":12,"osd_in_since":1726901367,"num_remapped_pgs":0} 2024-09-21T06:49:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: purged_snaps scrub starts 2024-09-21T06:49:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: purged_snaps scrub ok 2024-09-21T06:49:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2280765600' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] boot 2024-09-21T06:49:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:49:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: purged_snaps scrub starts 2024-09-21T06:49:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: purged_snaps scrub ok 2024-09-21T06:49:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2280765600' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] boot 2024-09-21T06:49:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:49:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: purged_snaps scrub starts 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: purged_snaps scrub ok 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2280765600' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: osd.11 [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] boot 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T06:49:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T06:49:52.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T06:49:52.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:52 smithi193 ceph-mon[34601]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:49:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:52 smithi114 ceph-mon[30654]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:52 smithi183 ceph-mon[34219]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-09-21T06:49:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 -- 172.21.15.114:0/1163811398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8078290 msgr2=0x7f74c8114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 --2- 172.21.15.114:0/1163811398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8078290 0x7f74c8114030 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 -- 172.21.15.114:0/1163811398 shutdown_connections 2024-09-21T06:49:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 --2- 172.21.15.114:0/1163811398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c8114570 0x7f74c8116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 --2- 172.21.15.114:0/1163811398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8078290 0x7f74c8114030 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 --2- 172.21.15.114:0/1163811398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c8077940 0x7f74c8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 -- 172.21.15.114:0/1163811398 >> 172.21.15.114:0/1163811398 conn(0x7f74c8101680 msgr2=0x7f74c8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 -- 172.21.15.114:0/1163811398 shutdown_connections 2024-09-21T06:49:53.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.360+0000 7f74c7200640 1 -- 172.21.15.114:0/1163811398 wait complete. 2024-09-21T06:49:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.361+0000 7f74c7200640 1 Processor -- start 2024-09-21T06:49:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.361+0000 7f74c7200640 1 -- start start 2024-09-21T06:49:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c8077940 0x7f74c81a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:53.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c8078290 0x7f74c81a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 0x7f74c81a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8119ed0 con 0x7f74c8114570 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74c8119d50 con 0x7f74c8077940 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74c811a050 con 0x7f74c8078290 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 0x7f74c81a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74bd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c8078290 0x7f74c81a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 0x7f74c81a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43006/0 (socket says 172.21.15.114:43006) 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c6800640 1 -- 172.21.15.114:0/2299535793 learned_addr learned my addr 172.21.15.114:0/2299535793 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.362+0000 7f74c5e00640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c8077940 0x7f74c81a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c6800640 1 -- 172.21.15.114:0/2299535793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c8078290 msgr2=0x7f74c81a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c6800640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c8078290 0x7f74c81a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c6800640 1 -- 172.21.15.114:0/2299535793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c8077940 msgr2=0x7f74c81a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:53.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c6800640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c8077940 0x7f74c81a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c6800640 1 -- 172.21.15.114:0/2299535793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c81a68b0 con 0x7f74c8114570 2024-09-21T06:49:53.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c6800640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 0x7f74c81a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74c000eb10 tx=0x7f74c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:53.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004b20 con 0x7f74c8114570 2024-09-21T06:49:53.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c0004cc0 con 0x7f74c8114570 2024-09-21T06:49:53.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0010670 con 0x7f74c8114570 2024-09-21T06:49:53.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c81a6ba0 con 0x7f74c8114570 2024-09-21T06:49:53.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.363+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c81ad300 con 0x7f74c8114570 2024-09-21T06:49:53.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.364+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c8077d50 con 0x7f74c8114570 2024-09-21T06:49:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.367+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f74c0005000 con 0x7f74c8114570 2024-09-21T06:49:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.368+0000 7f74bf400640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f74ac0b65e0 0x7f74ac0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.368+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f74c00e7740 con 0x7f74c8114570 2024-09-21T06:49:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.368+0000 7f74c5e00640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f74ac0b65e0 0x7f74ac0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.369+0000 7f74c5e00640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f74ac0b65e0 0x7f74ac0b8af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009bd0 tx=0x7f74b8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:53.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.371+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74c0056710 con 0x7f74c8114570 2024-09-21T06:49:53.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.560+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f74c81a1a70 con 0x7f74c8114570 2024-09-21T06:49:53.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.560+0000 7f74bf400640 1 -- 172.21.15.114:0/2299535793 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f74c81a1a70 con 0x7f74c8114570 2024-09-21T06:49:53.563 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:49:53.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f74ac0b65e0 msgr2=0x7f74ac0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f74ac0b65e0 0x7f74ac0b8af0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009bd0 tx=0x7f74b8005f90 comp rx=0 tx=0).stop 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 msgr2=0x7f74c81a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 0x7f74c81a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74c000eb10 tx=0x7f74c000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 shutdown_connections 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f74c8114570 0x7f74c81a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74c8078290 0x7f74c81a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f74ac0b65e0 0x7f74ac0b8af0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 --2- 172.21.15.114:0/2299535793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74c8077940 0x7f74c81a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 >> 172.21.15.114:0/2299535793 conn(0x7f74c8101680 msgr2=0x7f74c8116680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 shutdown_connections 2024-09-21T06:49:53.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:53.563+0000 7f74c7200640 1 -- 172.21.15.114:0/2299535793 wait complete. 2024-09-21T06:49:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:53 smithi114 ceph-mon[30654]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:49:53.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:53 smithi183 ceph-mon[34219]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:49:53.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:53 smithi193 ceph-mon[34601]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T06:49:54.297 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1726901391,"num_in_osds":12,"osd_in_since":1726901367,"num_remapped_pgs":0} 2024-09-21T06:49:54.298 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:49:54.663 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:49:54.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:54 smithi114 ceph-mon[30654]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:54.802 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:54 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2299535793' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:54 smithi183 ceph-mon[34219]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:54 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2299535793' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:54.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:54 smithi193 ceph-mon[34601]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:54 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2299535793' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T06:49:55.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 -- 172.21.15.114:0/2293368306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4114280 msgr2=0x7f70e4116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:55.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 --2- 172.21.15.114:0/2293368306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4114280 0x7f70e4116750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009ca0 tx=0x7f70dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 -- 172.21.15.114:0/2293368306 shutdown_connections 2024-09-21T06:49:55.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 --2- 172.21.15.114:0/2293368306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4114280 0x7f70e4116750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 --2- 172.21.15.114:0/2293368306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70e4075dd0 0x7f70e40761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 --2- 172.21.15.114:0/2293368306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70e4077380 0x7f70e40757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 -- 172.21.15.114:0/2293368306 >> 172.21.15.114:0/2293368306 conn(0x7f70e40ff440 msgr2=0x7f70e4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:55.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 -- 172.21.15.114:0/2293368306 shutdown_connections 2024-09-21T06:49:55.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.239+0000 7f70e8a00640 1 -- 172.21.15.114:0/2293368306 wait complete. 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.240+0000 7f70e8a00640 1 Processor -- start 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.240+0000 7f70e8a00640 1 -- start start 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 0x7f70e41a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70e4077380 0x7f70e41a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70e4114280 0x7f70e41a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e8a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4119ec0 con 0x7f70e4075dd0 2024-09-21T06:49:55.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4119d40 con 0x7f70e4114280 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f70e411a040 con 0x7f70e4077380 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 0x7f70e41a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 0x7f70e41a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40410/0 (socket says 172.21.15.114:40410) 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e3400640 1 -- 172.21.15.114:0/3145900057 learned_addr learned my addr 172.21.15.114:0/3145900057 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e2a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70e4077380 0x7f70e41a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.241+0000 7f70e3e00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70e4114280 0x7f70e41a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e3400640 1 -- 172.21.15.114:0/3145900057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70e4077380 msgr2=0x7f70e41a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e3400640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70e4077380 0x7f70e41a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e3400640 1 -- 172.21.15.114:0/3145900057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70e4114280 msgr2=0x7f70e41a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e3400640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70e4114280 0x7f70e41a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e3400640 1 -- 172.21.15.114:0/3145900057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e41a6800 con 0x7f70e4075dd0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e3400640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 0x7f70e41a0890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f70d400ea60 tx=0x7f70d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d400cde0 con 0x7f70e4075dd0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70d40035f0 con 0x7f70e4075dd0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e41a6af0 con 0x7f70e4075dd0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4010690 con 0x7f70e4075dd0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.242+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41ad250 con 0x7f70e4075dd0 2024-09-21T06:49:55.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.244+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e40761e0 con 0x7f70e4075dd0 2024-09-21T06:49:55.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.246+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f70d40037a0 con 0x7f70e4075dd0 2024-09-21T06:49:55.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.247+0000 7f70dbe00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70c80b6560 0x7f70c80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:55.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.248+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f70d40e74e0 con 0x7f70e4075dd0 2024-09-21T06:49:55.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.248+0000 7f70e2a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70c80b6560 0x7f70c80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:55.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.249+0000 7f70e2a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70c80b6560 0x7f70c80b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70e41161a0 tx=0x7f70d0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:55.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.250+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70d4056710 con 0x7f70e4075dd0 2024-09-21T06:49:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.436+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70e41a19c0 con 0x7f70e4075dd0 2024-09-21T06:49:55.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.437+0000 7f70dbe00640 1 -- 172.21.15.114:0/3145900057 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18848 (secure 0 0 0) 0x7f70d40ae0a0 con 0x7f70e4075dd0 2024-09-21T06:49:55.440 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:49:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":71,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:49:52.523513+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70c80b6560 msgr2=0x7f70c80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70c80b6560 0x7f70c80b8a70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70e41161a0 tx=0x7f70d0005f90 comp rx=0 tx=0).stop 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 msgr2=0x7f70e41a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 0x7f70e41a0890 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f70d400ea60 tx=0x7f70d400ef20 comp rx=0 tx=0).stop 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 shutdown_connections 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70c80b6560 0x7f70c80b8a70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70e4114280 0x7f70e41a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70e4077380 0x7f70e41a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 --2- 172.21.15.114:0/3145900057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70e4075dd0 0x7f70e41a0890 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 >> 172.21.15.114:0/3145900057 conn(0x7f70e40ff440 msgr2=0x7f70e4110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 shutdown_connections 2024-09-21T06:49:55.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:55.440+0000 7f70e8a00640 1 -- 172.21.15.114:0/3145900057 wait complete. 2024-09-21T06:49:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:55 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3145900057' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:49:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:55 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3145900057' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:49:55.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:55 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3145900057' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:49:56.240 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:45:37.711233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:49:56.241 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T06:49:56.605 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:49:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:56 smithi114 ceph-mon[30654]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:56 smithi183 ceph-mon[34219]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:56 smithi193 ceph-mon[34601]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:57.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 -- 172.21.15.114:0/2283109982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d8109c40 msgr2=0x7f75d8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 --2- 172.21.15.114:0/2283109982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d8109c40 0x7f75d8110470 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f75d0009ca0 tx=0x7f75d002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 -- 172.21.15.114:0/2283109982 shutdown_connections 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 --2- 172.21.15.114:0/2283109982 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d8109c40 0x7f75d8110470 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 --2- 172.21.15.114:0/2283109982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d81092f0 0x7f75d8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 --2- 172.21.15.114:0/2283109982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75d81039b0 0x7f75d8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.234+0000 7f75dee00640 1 -- 172.21.15.114:0/2283109982 >> 172.21.15.114:0/2283109982 conn(0x7f75d80ff440 msgr2=0x7f75d8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.235+0000 7f75dee00640 1 -- 172.21.15.114:0/2283109982 shutdown_connections 2024-09-21T06:49:57.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.235+0000 7f75dee00640 1 -- 172.21.15.114:0/2283109982 wait complete. 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 Processor -- start 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 -- start start 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 0x7f75d819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75d81092f0 0x7f75d819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d8109c40 0x7f75d81a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8113950 con 0x7f75d81039b0 2024-09-21T06:49:57.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.236+0000 7f75dee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81137d0 con 0x7f75d8109c40 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8113ad0 con 0x7f75d81092f0 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 0x7f75d819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 0x7f75d819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40426/0 (socket says 172.21.15.114:40426) 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 -- 172.21.15.114:0/95768898 learned_addr learned my addr 172.21.15.114:0/95768898 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dd000640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75d81092f0 0x7f75d819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75de400640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d8109c40 0x7f75d81a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 -- 172.21.15.114:0/95768898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75d81092f0 msgr2=0x7f75d819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:57.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75d81092f0 0x7f75d819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 -- 172.21.15.114:0/95768898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d8109c40 msgr2=0x7f75d81a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:57.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d8109c40 0x7f75d81a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.237+0000 7f75dda00640 1 -- 172.21.15.114:0/95768898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81a2430 con 0x7f75d81039b0 2024-09-21T06:49:57.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.238+0000 7f75dda00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 0x7f75d819c4c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00e9f0 tx=0x7f75cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:57.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.238+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc00cd80 con 0x7f75d81039b0 2024-09-21T06:49:57.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.238+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d81a2720 con 0x7f75d81039b0 2024-09-21T06:49:57.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.238+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc0108e0 con 0x7f75d81039b0 2024-09-21T06:49:57.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.238+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc018980 con 0x7f75d81039b0 2024-09-21T06:49:57.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.238+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81a8e80 con 0x7f75d81039b0 2024-09-21T06:49:57.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.239+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8103dc0 con 0x7f75d81039b0 2024-09-21T06:49:57.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.242+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f75cc010430 con 0x7f75d81039b0 2024-09-21T06:49:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.243+0000 7f75d6000640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75c00b6560 0x7f75c00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.243+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f75cc0e77f0 con 0x7f75d81039b0 2024-09-21T06:49:57.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.244+0000 7f75dd000640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75c00b6560 0x7f75c00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:57.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.244+0000 7f75dd000640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75c00b6560 0x7f75c00b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f75d8108ac0 tx=0x7f75c4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:57.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.245+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f75cc0b2790 con 0x7f75d81039b0 2024-09-21T06:49:57.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.431+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75d81a91f0 con 0x7f75d81039b0 2024-09-21T06:49:57.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.432+0000 7f75d6000640 1 -- 172.21.15.114:0/95768898 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f75cc0b76d0 con 0x7f75d81039b0 2024-09-21T06:49:57.434 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75c00b6560 msgr2=0x7f75c00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75c00b6560 0x7f75c00b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f75d8108ac0 tx=0x7f75c4005f90 comp rx=0 tx=0).stop 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 msgr2=0x7f75d819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 0x7f75d819c4c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00e9f0 tx=0x7f75cc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 shutdown_connections 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75c00b6560 0x7f75c00b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75d8109c40 0x7f75d81a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75d81092f0 0x7f75d819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 --2- 172.21.15.114:0/95768898 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75d81039b0 0x7f75d819c4c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:57.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.434+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 >> 172.21.15.114:0/95768898 conn(0x7f75d80ff440 msgr2=0x7f75d81068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:57.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.435+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 shutdown_connections 2024-09-21T06:49:57.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:49:57.435+0000 7f75dee00640 1 -- 172.21.15.114:0/95768898 wait complete. 2024-09-21T06:49:57.811 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:57 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/95768898' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:49:57.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:57 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/95768898' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:49:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:57 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/95768898' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:49:58.229 INFO:tasks.cephadm:Adding mds.a on smithi114 2024-09-21T06:49:58.229 INFO:tasks.cephadm:Adding mds.d on smithi114 2024-09-21T06:49:58.229 INFO:tasks.cephadm:Adding mds.g on smithi114 2024-09-21T06:49:58.229 INFO:tasks.cephadm:Adding mds.j on smithi114 2024-09-21T06:49:58.229 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-09-21T06:49:58.230 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-09-21T06:49:58.230 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-09-21T06:49:58.492 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:49:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:49:58 smithi114 ceph-mon[30654]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:49:58 smithi183 ceph-mon[34219]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:49:58 smithi193 ceph-mon[34601]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:49:59.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 -- 172.21.15.193:0/1971890689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c108db0 msgr2=0x7ff33c10f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:59.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 --2- 172.21.15.193:0/1971890689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c108db0 0x7ff33c10f5e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff334009ca0 tx=0x7ff33402a8e0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 -- 172.21.15.193:0/1971890689 shutdown_connections 2024-09-21T06:49:59.037 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 --2- 172.21.15.193:0/1971890689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c108db0 0x7ff33c10f5e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 --2- 172.21.15.193:0/1971890689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff33c108460 0x7ff33c108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 --2- 172.21.15.193:0/1971890689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c103180 0x7ff33c103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 -- 172.21.15.193:0/1971890689 >> 172.21.15.193:0/1971890689 conn(0x7ff33c077cc0 msgr2=0x7ff33c078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 -- 172.21.15.193:0/1971890689 shutdown_connections 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.024+0000 7ff3418db640 1 -- 172.21.15.193:0/1971890689 wait complete. 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 Processor -- start 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 -- start start 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 0x7ff33c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c108460 0x7ff33c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff33c108db0 0x7ff33c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112b30 con 0x7ff33c108460 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1129b0 con 0x7ff33c108db0 2024-09-21T06:49:59.038 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.025+0000 7ff3418db640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c112cb0 con 0x7ff33c103180 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 0x7ff33c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 0x7ff33c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:35196/0 (socket says 172.21.15.193:35196) 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 -- 172.21.15.193:0/1753971353 learned_addr learned my addr 172.21.15.193:0/1753971353 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33b7fe640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff33c108db0 0x7ff33c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff333fff640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c108460 0x7ff33c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 -- 172.21.15.193:0/1753971353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff33c108db0 msgr2=0x7ff33c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff33c108db0 0x7ff33c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 -- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c108460 msgr2=0x7ff33c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c108460 0x7ff33c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 -- 172.21.15.193:0/1753971353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c19f3d0 con 0x7ff33c103180 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff33affd640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 0x7ff33c199460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00b4a0 tx=0x7ff32c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c014070 con 0x7ff33c103180 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c19f6c0 con 0x7ff33c103180 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff32c010ac0 con 0x7ff33c103180 2024-09-21T06:49:59.039 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c017c30 con 0x7ff33c103180 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.026+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a5ea0 con 0x7ff33c103180 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.027+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c103590 con 0x7ff33c103180 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.029+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7ff32c017070 con 0x7ff33c103180 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.030+0000 7ff3408d9640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3280b6560 0x7ff3280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.030+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7ff32c0e6670 con 0x7ff33c103180 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.030+0000 7ff333fff640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3280b6560 0x7ff3280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.030+0000 7ff333fff640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3280b6560 0x7ff3280b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff33c108160 tx=0x7ff324008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:49:59.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.034+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff32c0ad2d0 con 0x7ff33c103180 2024-09-21T06:49:59.234 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.232+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7ff33c063800 con 0x7ff3280b6560 2024-09-21T06:49:59.243 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.241+0000 7ff3408d9640 1 -- 172.21.15.193:0/1753971353 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff33c063800 con 0x7ff3280b6560 2024-09-21T06:49:59.243 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-09-21T06:49:59.245 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3280b6560 msgr2=0x7ff3280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3280b6560 0x7ff3280b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff33c108160 tx=0x7ff324008040 comp rx=0 tx=0).stop 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 msgr2=0x7ff33c199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 0x7ff33c199460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00b4a0 tx=0x7ff32c00b960 comp rx=0 tx=0).stop 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 shutdown_connections 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3280b6560 0x7ff3280b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff33c108db0 0x7ff33c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff33c108460 0x7ff33c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 --2- 172.21.15.193:0/1753971353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff33c103180 0x7ff33c199460 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 >> 172.21.15.193:0/1753971353 conn(0x7ff33c077cc0 msgr2=0x7ff33c1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 shutdown_connections 2024-09-21T06:49:59.246 INFO:teuthology.orchestra.run.smithi193.stderr:2024-09-21T06:49:59.244+0000 7ff3418db640 1 -- 172.21.15.193:0/1753971353 wait complete. 2024-09-21T06:49:59.887 DEBUG:teuthology.orchestra.run.smithi114:mds.a> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T06:49:59.890 DEBUG:teuthology.orchestra.run.smithi114:mds.d> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T06:49:59.938 DEBUG:teuthology.orchestra.run.smithi114:mds.g> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T06:49:59.983 DEBUG:teuthology.orchestra.run.smithi114:mds.j> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T06:50:00.028 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T06:50:00.030 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T06:50:00.073 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T06:50:00.116 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T06:50:00.159 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T06:50:00.162 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T06:50:00.204 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T06:50:00.247 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T06:50:00.291 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T06:50:00.291 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:00.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: Saving service mds.all spec with placement smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:00.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:00.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:00.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:00.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:00.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:00 smithi183 ceph-mon[34219]: overall HEALTH_OK 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: Saving service mds.all spec with placement smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:00.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:00.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:00.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:00.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:00 smithi193 ceph-mon[34601]: overall HEALTH_OK 2024-09-21T06:50:00.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='client.24655 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:50:00.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: Saving service mds.all spec with placement smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T06:50:00.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:00.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:00 smithi114 ceph-mon[30654]: overall HEALTH_OK 2024-09-21T06:50:00.654 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 -- 172.21.15.114:0/1633377809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 msgr2=0x7f6ad0106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 --2- 172.21.15.114:0/1633377809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad0106190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009900 tx=0x7f6ac002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 -- 172.21.15.114:0/1633377809 shutdown_connections 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 --2- 172.21.15.114:0/1633377809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad010c010 0x7f6ad0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 --2- 172.21.15.114:0/1633377809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad010b6c0 0x7f6ad010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 --2- 172.21.15.114:0/1633377809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad0106190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 -- 172.21.15.114:0/1633377809 >> 172.21.15.114:0/1633377809 conn(0x7f6ad01017d0 msgr2=0x7f6ad0103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 -- 172.21.15.114:0/1633377809 shutdown_connections 2024-09-21T06:50:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.294+0000 7f6ad6400640 1 -- 172.21.15.114:0/1633377809 wait complete. 2024-09-21T06:50:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.295+0000 7f6ad6400640 1 Processor -- start 2024-09-21T06:50:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.295+0000 7f6ad6400640 1 -- start start 2024-09-21T06:50:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.295+0000 7f6ad6400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.295+0000 7f6ad6400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad010b6c0 0x7f6ad019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6ad6400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad010c010 0x7f6ad01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6ad6400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0115d20 con 0x7f6ad010c010 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6ad6400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0115ba0 con 0x7f6ad010b6c0 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6ad6400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad0115ea0 con 0x7f6ad0105d80 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad010c010 0x7f6ad01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34234/0 (socket says 172.21.15.114:34234) 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 -- 172.21.15.114:0/792091787 learned_addr learned my addr 172.21.15.114:0/792091787 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acea00640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad010b6c0 0x7f6ad019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 -- 172.21.15.114:0/792091787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad010b6c0 msgr2=0x7f6ad019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad010b6c0 0x7f6ad019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 -- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad010c010 msgr2=0x7f6ad01a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad010c010 0x7f6ad01a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.296+0000 7f6acf400640 1 -- 172.21.15.114:0/792091787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad01a2640 con 0x7f6ad0105d80 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.297+0000 7f6acf400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad019c6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0005ab0 tx=0x7f6ac0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.297+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac003f070 con 0x7f6ad0105d80 2024-09-21T06:50:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.297+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac0031b60 con 0x7f6ad0105d80 2024-09-21T06:50:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.297+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad01a28d0 con 0x7f6ad0105d80 2024-09-21T06:50:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.297+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0035030 con 0x7f6ad0105d80 2024-09-21T06:50:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.297+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad01a9090 con 0x7f6ad0105d80 2024-09-21T06:50:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.298+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad0106190 con 0x7f6ad0105d80 2024-09-21T06:50:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.300+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f6ac0042070 con 0x7f6ad0105d80 2024-09-21T06:50:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.302+0000 7f6ac7e00640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ab40b65e0 0x7f6ab40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.302+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f6ac0107950 con 0x7f6ad0105d80 2024-09-21T06:50:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.302+0000 7f6acea00640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ab40b65e0 0x7f6ab40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.303+0000 7f6acea00640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ab40b65e0 0x7f6ab40b8af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ad010b120 tx=0x7f6abc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.303+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ac00d28f0 con 0x7f6ad0105d80 2024-09-21T06:50:01.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.529+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad019d800 con 0x7f6ad0105d80 2024-09-21T06:50:01.531 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:01 smithi114 ceph-mon[30654]: Deploying daemon mds.l on smithi193 2024-09-21T06:50:01.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.529+0000 7f6ac7e00640 1 -- 172.21.15.114:0/792091787 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f6ad019d800 con 0x7f6ad0105d80 2024-09-21T06:50:01.532 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:01.532 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":1,"btime":"2024-09-21T06:41:28:662644+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T06:50:01.532 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 1 2024-09-21T06:50:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ab40b65e0 msgr2=0x7f6ab40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:01.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ab40b65e0 0x7f6ab40b8af0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ad010b120 tx=0x7f6abc005f90 comp rx=0 tx=0).stop 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 msgr2=0x7f6ad019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad019c6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0005ab0 tx=0x7f6ac0003c10 comp rx=0 tx=0).stop 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 shutdown_connections 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad010c010 0x7f6ad01a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ab40b65e0 0x7f6ab40b8af0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad010b6c0 0x7f6ad019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 --2- 172.21.15.114:0/792091787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad0105d80 0x7f6ad019c6d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 >> 172.21.15.114:0/792091787 conn(0x7f6ad01017d0 msgr2=0x7f6ad0103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 shutdown_connections 2024-09-21T06:50:01.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:01.532+0000 7f6ad6400640 1 -- 172.21.15.114:0/792091787 wait complete. 2024-09-21T06:50:01.567 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:01 smithi193 ceph-mon[34601]: Deploying daemon mds.l on smithi193 2024-09-21T06:50:01.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:01 smithi183 ceph-mon[34219]: Deploying daemon mds.l on smithi193 2024-09-21T06:50:01.912 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:01 smithi193 systemd[1]: Starting Ceph mds.l for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:02.226 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:01 smithi193 podman[63676]: 2024-09-21 06:50:01.860150949 +0000 UTC m=+0.015920150 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:02.226 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:01 smithi193 podman[63676]: 2024-09-21 06:50:01.978009306 +0000 UTC m=+0.133778494 container create e091ad4273b9875cd2b8f7afc24efafe3f2fa044809154e244b339ee3e3bd44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T06:50:02.226 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:02 smithi193 podman[63676]: 2024-09-21 06:50:02.189098688 +0000 UTC m=+0.344867878 container init e091ad4273b9875cd2b8f7afc24efafe3f2fa044809154e244b339ee3e3bd44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T06:50:02.226 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:02 smithi193 podman[63676]: 2024-09-21 06:50:02.191925376 +0000 UTC m=+0.347694566 container start e091ad4273b9875cd2b8f7afc24efafe3f2fa044809154e244b339ee3e3bd44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T06:50:02.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:02 smithi114 ceph-mon[30654]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:02.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:02 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/792091787' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:02 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6832/2086656361,v1:172.21.15.193:6833/2086656361] up:boot 2024-09-21T06:50:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:02 smithi114 ceph-mon[30654]: fsmap 1 up:standby 2024-09-21T06:50:02.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:02 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:50:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:02 smithi183 ceph-mon[34219]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:02 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/792091787' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:02 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6832/2086656361,v1:172.21.15.193:6833/2086656361] up:boot 2024-09-21T06:50:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:02 smithi183 ceph-mon[34219]: fsmap 1 up:standby 2024-09-21T06:50:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:02 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-mon[34601]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/792091787' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6832/2086656361,v1:172.21.15.193:6833/2086656361] up:boot 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-mon[34601]: fsmap 1 up:standby 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-mds[63695]: -- 172.21.15.193:0/3508747959 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bd4d6b860 con 0x559bd8f13a80 2024-09-21T06:50:02.662 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l[63691]: starting mds.l at 2024-09-21T06:50:02.663 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:02 smithi193 bash[63676]: e091ad4273b9875cd2b8f7afc24efafe3f2fa044809154e244b339ee3e3bd44f 2024-09-21T06:50:02.663 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 06:50:02 smithi193 systemd[1]: Started Ceph mds.l for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:03 smithi183 ceph-mon[34219]: Deploying daemon mds.k on smithi183 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:03.629 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:03 smithi114 ceph-mon[30654]: Deploying daemon mds.k on smithi183 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:03 smithi193 ceph-mon[34601]: Deploying daemon mds.k on smithi183 2024-09-21T06:50:04.292 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:04.570 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:04 smithi183 ceph-mon[34219]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:04.571 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 systemd[1]: Starting Ceph mds.k for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:04.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:04 smithi193 ceph-mon[34601]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:04.668 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:04.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:04 smithi114 ceph-mon[30654]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:04.825 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 podman[62247]: 2024-09-21 06:50:04.471840087 +0000 UTC m=+0.015635988 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:04.826 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 podman[62247]: 2024-09-21 06:50:04.7023876 +0000 UTC m=+0.246183497 container create 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T06:50:05.115 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 podman[62247]: 2024-09-21 06:50:04.947098082 +0000 UTC m=+0.490893983 container init 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-21T06:50:05.115 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 podman[62247]: 2024-09-21 06:50:04.950429054 +0000 UTC m=+0.494224955 container start 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:50:05.115 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 ceph-mds[62267]: -- 172.21.15.183:0/2321466900 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b43349860 con 0x562b474f1a80 2024-09-21T06:50:05.115 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:04 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k[62263]: starting mds.k at 2024-09-21T06:50:05.115 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:05 smithi183 bash[62247]: 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac 2024-09-21T06:50:05.115 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 06:50:05 smithi183 systemd[1]: Started Ceph mds.k for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 -- 172.21.15.114:0/3403424234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741092f0 msgr2=0x7f9874109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 --2- 172.21.15.114:0/3403424234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741092f0 0x7f9874109700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9860009860 tx=0x7f986002a340 comp rx=0 tx=0).stop 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 -- 172.21.15.114:0/3403424234 shutdown_connections 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 --2- 172.21.15.114:0/3403424234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9874109c40 0x7f9874110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 --2- 172.21.15.114:0/3403424234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741092f0 0x7f9874109700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 --2- 172.21.15.114:0/3403424234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f98741039b0 0x7f9874103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 -- 172.21.15.114:0/3403424234 >> 172.21.15.114:0/3403424234 conn(0x7f98740ff440 msgr2=0x7f9874101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:05.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 -- 172.21.15.114:0/3403424234 shutdown_connections 2024-09-21T06:50:05.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.344+0000 7f9879c00640 1 -- 172.21.15.114:0/3403424234 wait complete. 2024-09-21T06:50:05.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.345+0000 7f9879c00640 1 Processor -- start 2024-09-21T06:50:05.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 -- start start 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 0x7f9874075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741092f0 0x7f9874075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9874109c40 0x7f9874076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9874113900 con 0x7f9874109c40 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9874113780 con 0x7f98741039b0 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9879c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9874113a80 con 0x7f98741092f0 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9873e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 0x7f9874075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9873e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 0x7f9874075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46682/0 (socket says 172.21.15.114:46682) 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9873e00640 1 -- 172.21.15.114:0/3883228129 learned_addr learned my addr 172.21.15.114:0/3883228129 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.346+0000 7f9873400640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741092f0 0x7f9874075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:05.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9879200640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9874109c40 0x7f9874076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:05.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873e00640 1 -- 172.21.15.114:0/3883228129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741092f0 msgr2=0x7f9874075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873e00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741092f0 0x7f9874075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873e00640 1 -- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9874109c40 msgr2=0x7f9874076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873e00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9874109c40 0x7f9874076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873e00640 1 -- 172.21.15.114:0/3883228129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98741a8650 con 0x7f98741039b0 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873400640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741092f0 0x7f9874075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9873e00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 0x7f9874075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f986400e9c0 tx=0x7f986400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986400ccc0 con 0x7f98741039b0 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98640054d0 con 0x7f98741039b0 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864010430 con 0x7f98741039b0 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.347+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98741a8940 con 0x7f98741039b0 2024-09-21T06:50:05.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.348+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741a8e50 con 0x7f98741039b0 2024-09-21T06:50:05.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.349+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9874103dc0 con 0x7f98741039b0 2024-09-21T06:50:05.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.351+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f9864049d10 con 0x7f98741039b0 2024-09-21T06:50:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.353+0000 7f9870c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98580b6560 0x7f98580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.353+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f98640e7540 con 0x7f98741039b0 2024-09-21T06:50:05.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.353+0000 7f9873400640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98580b6560 0x7f98580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:05.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.354+0000 7f9873400640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98580b6560 0x7f98580b8a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9860035040 tx=0x7f9860030d10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:05.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.354+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98640ae100 con 0x7f98741039b0 2024-09-21T06:50:05.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.611+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9874076a10 con 0x7f98741039b0 2024-09-21T06:50:05.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.612+0000 7f9870c00640 1 -- 172.21.15.114:0/3883228129 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f9874076a10 con 0x7f98741039b0 2024-09-21T06:50:05.615 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:05.615 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":3,"btime":"2024-09-21T06:50:05:340280+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:50:05.615 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 3 2024-09-21T06:50:05.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98580b6560 msgr2=0x7f98580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:05.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98580b6560 0x7f98580b8a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9860035040 tx=0x7f9860030d10 comp rx=0 tx=0).stop 2024-09-21T06:50:05.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 msgr2=0x7f9874075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:05.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 0x7f9874075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f986400e9c0 tx=0x7f986400ee80 comp rx=0 tx=0).stop 2024-09-21T06:50:05.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 shutdown_connections 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9874109c40 0x7f9874076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98741092f0 0x7f9874075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98580b6560 0x7f98580b8a70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 --2- 172.21.15.114:0/3883228129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98741039b0 0x7f9874075710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 >> 172.21.15.114:0/3883228129 conn(0x7f98740ff440 msgr2=0x7f9874101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 shutdown_connections 2024-09-21T06:50:05.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:05.615+0000 7f9879c00640 1 -- 172.21.15.114:0/3883228129 wait complete. 2024-09-21T06:50:06.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:06.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:06.352 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:06.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: Deploying daemon mds.j on smithi114 2024-09-21T06:50:06.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6832/919969847,v1:172.21.15.183:6833/919969847] up:boot 2024-09-21T06:50:06.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: fsmap 2 up:standby 2024-09-21T06:50:06.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:50:06.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:06.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:06 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3883228129' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: Deploying daemon mds.j on smithi114 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6832/919969847,v1:172.21.15.183:6833/919969847] up:boot 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: fsmap 2 up:standby 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:06.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:06 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3883228129' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: Deploying daemon mds.j on smithi114 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6832/919969847,v1:172.21.15.183:6833/919969847] up:boot 2024-09-21T06:50:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: fsmap 2 up:standby 2024-09-21T06:50:06.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:50:06.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:06.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:06 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3883228129' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:08.483 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 systemd[1]: Starting Ceph mds.j for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:08.760 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:08 smithi114 ceph-mon[30654]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:08.760 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 podman[71273]: 2024-09-21 06:50:08.385998825 +0000 UTC m=+0.026896680 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:08.760 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 podman[71273]: 2024-09-21 06:50:08.516105568 +0000 UTC m=+0.157003412 container create 1704241ca44d3fe22f6f0f5af51f7e43772e177efd02ef9eaf5f11d2489f66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T06:50:08.760 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 podman[71273]: 2024-09-21 06:50:08.75913663 +0000 UTC m=+0.400034476 container init 1704241ca44d3fe22f6f0f5af51f7e43772e177efd02ef9eaf5f11d2489f66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T06:50:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:08 smithi183 ceph-mon[34219]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:08.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:08 smithi193 ceph-mon[34601]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:09.056 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:09.065 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 podman[71273]: 2024-09-21 06:50:08.765606519 +0000 UTC m=+0.406504374 container start 1704241ca44d3fe22f6f0f5af51f7e43772e177efd02ef9eaf5f11d2489f66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-21T06:50:09.066 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 ceph-mds[71292]: -- 172.21.15.114:0/3926563748 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e1c5d9860 con 0x561e20781180 2024-09-21T06:50:09.066 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 bash[71273]: 1704241ca44d3fe22f6f0f5af51f7e43772e177efd02ef9eaf5f11d2489f66a5 2024-09-21T06:50:09.066 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j[71288]: starting mds.j at 2024-09-21T06:50:09.066 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 06:50:08 smithi114 systemd[1]: Started Ceph mds.j for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:09.455 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:10.127 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: Deploying daemon mds.d on smithi114 2024-09-21T06:50:10.128 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:10.128 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6834/3420506424,v1:172.21.15.114:6835/3420506424] up:boot 2024-09-21T06:50:10.128 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: fsmap 3 up:standby 2024-09-21T06:50:10.128 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:09 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:50:10.282 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.282 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.282 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: Deploying daemon mds.d on smithi114 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6834/3420506424,v1:172.21.15.114:6835/3420506424] up:boot 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: fsmap 3 up:standby 2024-09-21T06:50:10.283 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:09 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:50:10.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: Deploying daemon mds.d on smithi114 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6834/3420506424,v1:172.21.15.114:6835/3420506424] up:boot 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: fsmap 3 up:standby 2024-09-21T06:50:10.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:09 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T06:50:10.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 -- 172.21.15.114:0/2139476744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 msgr2=0x7f1cd010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:10.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2139476744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 0x7f1cd010bae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009900 tx=0x7f1cb802a410 comp rx=0 tx=0).stop 2024-09-21T06:50:10.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 -- 172.21.15.114:0/2139476744 shutdown_connections 2024-09-21T06:50:10.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2139476744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd010c020 0x7f1cd0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2139476744 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 0x7f1cd010bae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2139476744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd0105d90 0x7f1cd01061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 -- 172.21.15.114:0/2139476744 >> 172.21.15.114:0/2139476744 conn(0x7f1cd01017c0 msgr2=0x7f1cd0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:10.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 -- 172.21.15.114:0/2139476744 shutdown_connections 2024-09-21T06:50:10.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.739+0000 7f1cd5000640 1 -- 172.21.15.114:0/2139476744 wait complete. 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.740+0000 7f1cd5000640 1 Processor -- start 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.740+0000 7f1cd5000640 1 -- start start 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cd5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 0x7f1cd019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cd5000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 0x7f1cd019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cd5000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd010c020 0x7f1cd01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cd5000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0115d30 con 0x7f1cd010b6d0 2024-09-21T06:50:10.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cd5000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0115bb0 con 0x7f1cd0105d90 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cd5000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0115eb0 con 0x7f1cd010c020 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cc5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 0x7f1cd019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 0x7f1cd019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1cce800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd010c020 0x7f1cd01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 0x7f1cd019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46716/0 (socket says 172.21.15.114:46716) 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1ccde00640 1 -- 172.21.15.114:0/2554782632 learned_addr learned my addr 172.21.15.114:0/2554782632 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.741+0000 7f1ccde00640 1 -- 172.21.15.114:0/2554782632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd010c020 msgr2=0x7f1cd01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1ccde00640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd010c020 0x7f1cd01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1ccde00640 1 -- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 msgr2=0x7f1cd019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1ccde00640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 0x7f1cd019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1ccde00640 1 -- 172.21.15.114:0/2554782632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd01a2630 con 0x7f1cd0105d90 2024-09-21T06:50:10.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1cce800640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd010c020 0x7f1cd01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1ccde00640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 0x7f1cd019c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ea60 tx=0x7f1cc000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000cde0 con 0x7f1cd0105d90 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd01a2920 con 0x7f1cd0105d90 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc00035f0 con 0x7f1cd0105d90 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0010690 con 0x7f1cd0105d90 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.742+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd01a9080 con 0x7f1cd0105d90 2024-09-21T06:50:10.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.743+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd01061a0 con 0x7f1cd0105d90 2024-09-21T06:50:10.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.746+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f1cc00037a0 con 0x7f1cd0105d90 2024-09-21T06:50:10.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.747+0000 7f1cc7400640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1cb40b6560 0x7f1cb40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:10.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.748+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f1cc00e74e0 con 0x7f1cd0105d90 2024-09-21T06:50:10.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.748+0000 7f1cc5400640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1cb40b6560 0x7f1cb40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:10.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.748+0000 7f1cc5400640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1cb40b6560 0x7f1cb40b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb802aa00 tx=0x7f1cb8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:10.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.749+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1cc0056710 con 0x7f1cd0105d90 2024-09-21T06:50:10.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.980+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cd019d7f0 con 0x7f1cd0105d90 2024-09-21T06:50:10.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.981+0000 7f1cc7400640 1 -- 172.21.15.114:0/2554782632 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f1cd019d7f0 con 0x7f1cd0105d90 2024-09-21T06:50:10.983 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:10.983 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":4,"btime":"2024-09-21T06:50:09:530388+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:50:10.983 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 4 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1cb40b6560 msgr2=0x7f1cb40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1cb40b6560 0x7f1cb40b8a70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cb802aa00 tx=0x7f1cb8005a20 comp rx=0 tx=0).stop 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 msgr2=0x7f1cd019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 0x7f1cd019c6c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000ea60 tx=0x7f1cc000ef20 comp rx=0 tx=0).stop 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 shutdown_connections 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1cd010c020 0x7f1cd01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1cd010b6d0 0x7f1cd019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1cb40b6560 0x7f1cb40b8a70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 --2- 172.21.15.114:0/2554782632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cd0105d90 0x7f1cd019c6c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 >> 172.21.15.114:0/2554782632 conn(0x7f1cd01017c0 msgr2=0x7f1cd010ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 shutdown_connections 2024-09-21T06:50:10.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:10.984+0000 7f1cd5000640 1 -- 172.21.15.114:0/2554782632 wait complete. 2024-09-21T06:50:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:11 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2554782632' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:11.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:11 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2554782632' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:11 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2554782632' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:12.684 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:12 smithi114 ceph-mon[30654]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:12 smithi183 ceph-mon[34219]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:12 smithi193 ceph-mon[34601]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:13.065 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 06:50:12 smithi114 systemd[1]: Started Ceph mds.d for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:14.125 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:14.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: Deploying daemon mds.e on smithi183 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6836/1280617419,v1:172.21.15.114:6837/1280617419] up:boot 2024-09-21T06:50:14.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: fsmap 4 up:standby 2024-09-21T06:50:14.164 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:13 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:50:14.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:14.181 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:14.182 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:14.182 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: Deploying daemon mds.e on smithi183 2024-09-21T06:50:14.182 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:14.182 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6836/1280617419,v1:172.21.15.114:6837/1280617419] up:boot 2024-09-21T06:50:14.182 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: fsmap 4 up:standby 2024-09-21T06:50:14.182 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:13 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:50:14.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:14.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:14.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:14.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:14.319 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: Deploying daemon mds.e on smithi183 2024-09-21T06:50:14.319 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:14.319 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6836/1280617419,v1:172.21.15.114:6837/1280617419] up:boot 2024-09-21T06:50:14.319 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: fsmap 4 up:standby 2024-09-21T06:50:14.319 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:13 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T06:50:14.510 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:15.114 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:14 smithi183 systemd[1]: Starting Ceph mds.e for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 -- 172.21.15.114:0/142151293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba81049d0 msgr2=0x7fbba8104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 --2- 172.21.15.114:0/142151293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba81049d0 0x7fbba8104de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009900 tx=0x7fbb9802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 -- 172.21.15.114:0/142151293 shutdown_connections 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 --2- 172.21.15.114:0/142151293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba810de30 0x7fbba8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 --2- 172.21.15.114:0/142151293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8105320 0x7fbba810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 --2- 172.21.15.114:0/142151293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba81049d0 0x7fbba8104de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 -- 172.21.15.114:0/142151293 >> 172.21.15.114:0/142151293 conn(0x7fbba8079740 msgr2=0x7fbba8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 -- 172.21.15.114:0/142151293 shutdown_connections 2024-09-21T06:50:15.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.167+0000 7fbbaec00640 1 -- 172.21.15.114:0/142151293 wait complete. 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.169+0000 7fbbaec00640 1 Processor -- start 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.169+0000 7fbbaec00640 1 -- start start 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.169+0000 7fbbaec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 0x7fbba819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.169+0000 7fbbaec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba8105320 0x7fbba819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.169+0000 7fbbaec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba810de30 0x7fbba819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbbaec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113b00 con 0x7fbba8105320 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbbaec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113980 con 0x7fbba81049d0 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbbaec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbba8113c80 con 0x7fbba810de30 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 0x7fbba819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 0x7fbba819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33708/0 (socket says 172.21.15.114:33708) 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 -- 172.21.15.114:0/3848505990 learned_addr learned my addr 172.21.15.114:0/3848505990 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:15.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbb9fe00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba8105320 0x7fbba819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbbacc00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba810de30 0x7fbba819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 -- 172.21.15.114:0/3848505990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba810de30 msgr2=0x7fbba819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba810de30 0x7fbba819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 -- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba8105320 msgr2=0x7fbba819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba8105320 0x7fbba819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 -- 172.21.15.114:0/3848505990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81a04a0 con 0x7fbba81049d0 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbbacc00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba810de30 0x7fbba819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:50:15.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.170+0000 7fbba7e00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 0x7fbba819a530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005ab0 tx=0x7fbb98003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.171+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9803f070 con 0x7fbba81049d0 2024-09-21T06:50:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.171+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb98031b50 con 0x7fbba81049d0 2024-09-21T06:50:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.171+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba81a0730 con 0x7fbba81049d0 2024-09-21T06:50:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.171+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98035030 con 0x7fbba81049d0 2024-09-21T06:50:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.171+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81025f0 con 0x7fbba81049d0 2024-09-21T06:50:15.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.172+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba806c240 con 0x7fbba81049d0 2024-09-21T06:50:15.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.174+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fbb98042070 con 0x7fbba81049d0 2024-09-21T06:50:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.176+0000 7fbba5600640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbb8c0b6560 0x7fbb8c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.176+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fbb981076a0 con 0x7fbba81049d0 2024-09-21T06:50:15.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.176+0000 7fbb9fe00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbb8c0b6560 0x7fbb8c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:15.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.176+0000 7fbb9fe00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbb8c0b6560 0x7fbb8c0b8a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbba810fc50 tx=0x7fbb90005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:15.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.180+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbb980ce260 con 0x7fbba81049d0 2024-09-21T06:50:15.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.406+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbba8102850 con 0x7fbba81049d0 2024-09-21T06:50:15.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.407+0000 7fbba5600640 1 -- 172.21.15.114:0/3848505990 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7fbb980d2640 con 0x7fbba81049d0 2024-09-21T06:50:15.409 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:15.409 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":5,"btime":"2024-09-21T06:50:13:490523+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T06:50:15.409 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 5 2024-09-21T06:50:15.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbb8c0b6560 msgr2=0x7fbb8c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:15.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbb8c0b6560 0x7fbb8c0b8a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbba810fc50 tx=0x7fbb90005f90 comp rx=0 tx=0).stop 2024-09-21T06:50:15.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 msgr2=0x7fbba819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:15.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 0x7fbba819a530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005ab0 tx=0x7fbb98003b90 comp rx=0 tx=0).stop 2024-09-21T06:50:15.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 shutdown_connections 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba810de30 0x7fbba819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbba8105320 0x7fbba819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbb8c0b6560 0x7fbb8c0b8a70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 --2- 172.21.15.114:0/3848505990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba81049d0 0x7fbba819a530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.409+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 >> 172.21.15.114:0/3848505990 conn(0x7fbba8079740 msgr2=0x7fbba80ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.410+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 shutdown_connections 2024-09-21T06:50:15.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:15.410+0000 7fbbaec00640 1 -- 172.21.15.114:0/3848505990 wait complete. 2024-09-21T06:50:15.444 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 podman[62528]: 2024-09-21 06:50:15.03859395 +0000 UTC m=+0.015988108 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:15.444 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 podman[62528]: 2024-09-21 06:50:15.16564342 +0000 UTC m=+0.143037574 container create 86959222bd3a838916a0cc93e5a88cea0daa4bbab4b69e0fb7b9f3991eb2a1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T06:50:15.445 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 podman[62528]: 2024-09-21 06:50:15.443500819 +0000 UTC m=+0.420894972 container init 86959222bd3a838916a0cc93e5a88cea0daa4bbab4b69e0fb7b9f3991eb2a1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T06:50:15.756 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:15 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3848505990' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:15.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:15 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3848505990' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:15.864 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 podman[62528]: 2024-09-21 06:50:15.446305981 +0000 UTC m=+0.423700135 container start 86959222bd3a838916a0cc93e5a88cea0daa4bbab4b69e0fb7b9f3991eb2a1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:15.864 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 ceph-mds[62547]: -- 172.21.15.183:0/3071290460 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593d810f860 con 0x5593dc2b7600 2024-09-21T06:50:15.865 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e[62543]: starting mds.e at 2024-09-21T06:50:15.865 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 bash[62528]: 86959222bd3a838916a0cc93e5a88cea0daa4bbab4b69e0fb7b9f3991eb2a1e4 2024-09-21T06:50:15.865 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 06:50:15 smithi183 systemd[1]: Started Ceph mds.e for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:15.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:15 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3848505990' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: Deploying daemon mds.f on smithi193 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6834/2729823804,v1:172.21.15.183:6835/2729823804] up:boot 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: fsmap 5 up:standby 2024-09-21T06:50:16.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:16 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: Deploying daemon mds.f on smithi193 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6834/2729823804,v1:172.21.15.183:6835/2729823804] up:boot 2024-09-21T06:50:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: fsmap 5 up:standby 2024-09-21T06:50:16.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:16 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:50:17.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: Deploying daemon mds.f on smithi193 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6834/2729823804,v1:172.21.15.183:6835/2729823804] up:boot 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: fsmap 5 up:standby 2024-09-21T06:50:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:16 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:50:17.849 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:17 smithi193 systemd[1]: Starting Ceph mds.f for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:18.104 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:17 smithi193 podman[63953]: 2024-09-21 06:50:17.750593595 +0000 UTC m=+0.015542373 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:18.104 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:17 smithi193 podman[63953]: 2024-09-21 06:50:17.866445103 +0000 UTC m=+0.131393915 container create bc6d2fcb8cb37a04b6e5011dfa36c9367a83cd927bbe172f9790dde1aa3f2ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T06:50:18.104 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:18 smithi193 podman[63953]: 2024-09-21 06:50:18.10251635 +0000 UTC m=+0.367465119 container init bc6d2fcb8cb37a04b6e5011dfa36c9367a83cd927bbe172f9790dde1aa3f2ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:50:18.153 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:18.412 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:18 smithi193 podman[63953]: 2024-09-21 06:50:18.10528632 +0000 UTC m=+0.370235090 container start bc6d2fcb8cb37a04b6e5011dfa36c9367a83cd927bbe172f9790dde1aa3f2ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T06:50:18.412 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mds[63972]: -- 172.21.15.193:0/432825900 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564be0f3f860 con 0x564be50e7600 2024-09-21T06:50:18.412 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f[63968]: starting mds.f at 2024-09-21T06:50:18.412 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:18 smithi193 bash[63953]: bc6d2fcb8cb37a04b6e5011dfa36c9367a83cd927bbe172f9790dde1aa3f2ee1 2024-09-21T06:50:18.412 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 06:50:18 smithi193 systemd[1]: Started Ceph mds.f for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:18.540 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:18.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:18 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:18.872 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:18 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:18 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 -- 172.21.15.114:0/1786804319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f508410b4f0 msgr2=0x7f508410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 --2- 172.21.15.114:0/1786804319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f508410b4f0 0x7f508410b900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5070009900 tx=0x7f507002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 -- 172.21.15.114:0/1786804319 shutdown_connections 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 --2- 172.21.15.114:0/1786804319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410be40 0x7f5084112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 --2- 172.21.15.114:0/1786804319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f508410b4f0 0x7f508410b900 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 --2- 172.21.15.114:0/1786804319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5084105bb0 0x7f5084105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 -- 172.21.15.114:0/1786804319 >> 172.21.15.114:0/1786804319 conn(0x7f5084101680 msgr2=0x7f5084103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:19.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 -- 172.21.15.114:0/1786804319 shutdown_connections 2024-09-21T06:50:19.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.232+0000 7f5083200640 1 -- 172.21.15.114:0/1786804319 wait complete. 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.233+0000 7f5083200640 1 Processor -- start 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.233+0000 7f5083200640 1 -- start start 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5083200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5084105bb0 0x7f508419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5083200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 0x7f508419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5083200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508410be40 0x7f50841a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5083200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5084115b00 con 0x7f5084105bb0 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5083200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5084115980 con 0x7f508410be40 2024-09-21T06:50:19.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5083200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5084115c80 con 0x7f508410b4f0 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5084105bb0 0x7f508419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 0x7f508419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 0x7f508419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52204/0 (socket says 172.21.15.114:52204) 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081400640 1 -- 172.21.15.114:0/1359698440 learned_addr learned my addr 172.21.15.114:0/1359698440 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5082800640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508410be40 0x7f50841a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081e00640 1 -- 172.21.15.114:0/1359698440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 msgr2=0x7f508419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081e00640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 0x7f508419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081e00640 1 -- 172.21.15.114:0/1359698440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508410be40 msgr2=0x7f50841a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081e00640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508410be40 0x7f50841a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081e00640 1 -- 172.21.15.114:0/1359698440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50841a2480 con 0x7f5084105bb0 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5081400640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 0x7f508419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.234+0000 7f5082800640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508410be40 0x7f50841a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:19.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.235+0000 7f5081e00640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5084105bb0 0x7f508419c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f507400e9f0 tx=0x7f507400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:19.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.235+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507400ccc0 con 0x7f5084105bb0 2024-09-21T06:50:19.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.235+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50740108e0 con 0x7f5084105bb0 2024-09-21T06:50:19.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.235+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5074018840 con 0x7f5084105bb0 2024-09-21T06:50:19.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.235+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50841a2770 con 0x7f5084105bb0 2024-09-21T06:50:19.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.235+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a8ed0 con 0x7f5084105bb0 2024-09-21T06:50:19.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.236+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084105fc0 con 0x7f5084105bb0 2024-09-21T06:50:19.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.238+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f5074010430 con 0x7f5084105bb0 2024-09-21T06:50:19.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.240+0000 7f507aa00640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50680b6560 0x7f50680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:19.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.240+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f50740e7770 con 0x7f5084105bb0 2024-09-21T06:50:19.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.240+0000 7f5081400640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50680b6560 0x7f50680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:19.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.241+0000 7f5081400640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50680b6560 0x7f50680b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f508410af80 tx=0x7f5070032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:19.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.242+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50740ae330 con 0x7f5084105bb0 2024-09-21T06:50:19.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.466+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f508419d640 con 0x7f5084105bb0 2024-09-21T06:50:19.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.467+0000 7f507aa00640 1 -- 172.21.15.114:0/1359698440 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7f508419d640 con 0x7f5084105bb0 2024-09-21T06:50:19.469 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:19.469 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":7,"btime":"2024-09-21T06:50:18:606284+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T06:50:19.469 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 7 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.470+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50680b6560 msgr2=0x7f50680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.470+0000 7f5083200640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50680b6560 0x7f50680b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f508410af80 tx=0x7f5070032ed0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5084105bb0 msgr2=0x7f508419c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5084105bb0 0x7f508419c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f507400e9f0 tx=0x7f507400edf0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 shutdown_connections 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50680b6560 0x7f50680b8a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f508410be40 0x7f50841a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f508410b4f0 0x7f508419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 --2- 172.21.15.114:0/1359698440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5084105bb0 0x7f508419c510 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:19.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 >> 172.21.15.114:0/1359698440 conn(0x7f5084101680 msgr2=0x7f5084103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:19.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 shutdown_connections 2024-09-21T06:50:19.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:19.471+0000 7f5083200640 1 -- 172.21.15.114:0/1359698440 wait complete. 2024-09-21T06:50:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:19 smithi193 ceph-mon[34601]: Deploying daemon mds.c on smithi193 2024-09-21T06:50:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:19 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6834/2057077471,v1:172.21.15.193:6835/2057077471] up:boot 2024-09-21T06:50:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:19 smithi193 ceph-mon[34601]: fsmap 6 up:standby 2024-09-21T06:50:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:19 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:50:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:19 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1359698440' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:19.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:19 smithi114 ceph-mon[30654]: Deploying daemon mds.c on smithi193 2024-09-21T06:50:19.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:19 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6834/2057077471,v1:172.21.15.193:6835/2057077471] up:boot 2024-09-21T06:50:19.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:19 smithi114 ceph-mon[30654]: fsmap 6 up:standby 2024-09-21T06:50:19.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:19 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:50:19.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:19 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1359698440' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:20.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:19 smithi183 ceph-mon[34219]: Deploying daemon mds.c on smithi193 2024-09-21T06:50:20.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:19 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6834/2057077471,v1:172.21.15.193:6835/2057077471] up:boot 2024-09-21T06:50:20.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:19 smithi183 ceph-mon[34219]: fsmap 6 up:standby 2024-09-21T06:50:20.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:19 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:50:20.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:19 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1359698440' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:20.412 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 systemd[1]: Starting Ceph mds.c for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:20.717 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:20 smithi193 ceph-mon[34601]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:20.717 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 podman[64222]: 2024-09-21 06:50:20.355024443 +0000 UTC m=+0.015930070 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:20.718 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 podman[64222]: 2024-09-21 06:50:20.47234796 +0000 UTC m=+0.133253571 container create a12693ffa40c1e6f8df24f40ef9d7e8325db36546b81b7e7cef8bc45aef111ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T06:50:20.718 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 podman[64222]: 2024-09-21 06:50:20.716728631 +0000 UTC m=+0.377634242 container init a12693ffa40c1e6f8df24f40ef9d7e8325db36546b81b7e7cef8bc45aef111ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:50:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:20 smithi183 ceph-mon[34219]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:21.004 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 podman[64222]: 2024-09-21 06:50:20.719656136 +0000 UTC m=+0.380561765 container start a12693ffa40c1e6f8df24f40ef9d7e8325db36546b81b7e7cef8bc45aef111ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:50:21.005 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 ceph-mds[64241]: -- 172.21.15.193:0/1264111996 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624eb979860 con 0x5624efb21180 2024-09-21T06:50:21.005 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c[64237]: starting mds.c at 2024-09-21T06:50:21.005 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 bash[64222]: a12693ffa40c1e6f8df24f40ef9d7e8325db36546b81b7e7cef8bc45aef111ba 2024-09-21T06:50:21.005 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 06:50:20 smithi193 systemd[1]: Started Ceph mds.c for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:20 smithi114 ceph-mon[30654]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:21.942 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: Deploying daemon mds.h on smithi183 2024-09-21T06:50:21.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:21.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] up:boot 2024-09-21T06:50:21.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: fsmap 7 up:standby 2024-09-21T06:50:21.943 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:21 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:50:22.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:22.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:22.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: Deploying daemon mds.h on smithi183 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] up:boot 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: fsmap 7 up:standby 2024-09-21T06:50:22.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:21 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: Deploying daemon mds.h on smithi183 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] up:boot 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: fsmap 7 up:standby 2024-09-21T06:50:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:21 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:50:22.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:22.864 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:22 smithi183 systemd[1]: Starting Ceph mds.h for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:23.293 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:22 smithi183 podman[62797]: 2024-09-21 06:50:22.886157753 +0000 UTC m=+0.015938730 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:23.293 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 podman[62797]: 2024-09-21 06:50:23.02254146 +0000 UTC m=+0.152322478 container create 2f8e30bc628063275d4d2edb5efa35ea92a48f1b3542c16f989e785241377686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T06:50:23.293 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 podman[62797]: 2024-09-21 06:50:23.291946444 +0000 UTC m=+0.421727406 container init 2f8e30bc628063275d4d2edb5efa35ea92a48f1b3542c16f989e785241377686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:50:23.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 -- 172.21.15.114:0/2923441382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c60109c40 msgr2=0x7f7c60110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:23.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2923441382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c60109c40 0x7f7c60110470 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009ca0 tx=0x7f7c5802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 -- 172.21.15.114:0/2923441382 shutdown_connections 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2923441382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c60109c40 0x7f7c60110470 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2923441382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601092f0 0x7f7c60109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2923441382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c601039b0 0x7f7c60103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 -- 172.21.15.114:0/2923441382 >> 172.21.15.114:0/2923441382 conn(0x7f7c600ff400 msgr2=0x7f7c60101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 -- 172.21.15.114:0/2923441382 shutdown_connections 2024-09-21T06:50:23.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.378+0000 7f7c64a00640 1 -- 172.21.15.114:0/2923441382 wait complete. 2024-09-21T06:50:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.379+0000 7f7c64a00640 1 Processor -- start 2024-09-21T06:50:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 -- start start 2024-09-21T06:50:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 0x7f7c6019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:23.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c601092f0 0x7f7c6019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c60109c40 0x7f7c601a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60113900 con 0x7f7c601092f0 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60113780 con 0x7f7c601039b0 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c64a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60113a80 con 0x7f7c60109c40 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 0x7f7c6019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c601092f0 0x7f7c6019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 0x7f7c6019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33744/0 (socket says 172.21.15.114:33744) 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c5f400640 1 -- 172.21.15.114:0/2737436190 learned_addr learned my addr 172.21.15.114:0/2737436190 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.380+0000 7f7c5fe00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c60109c40 0x7f7c601a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5ea00640 1 -- 172.21.15.114:0/2737436190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c60109c40 msgr2=0x7f7c601a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c60109c40 0x7f7c601a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5ea00640 1 -- 172.21.15.114:0/2737436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 msgr2=0x7f7c6019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 0x7f7c6019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5ea00640 1 -- 172.21.15.114:0/2737436190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c601a2420 con 0x7f7c601092f0 2024-09-21T06:50:23.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5f400640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 0x7f7c6019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:23.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c601092f0 0x7f7c6019ca50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c6006b930 tx=0x7f7c4c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:23.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.381+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00cc40 con 0x7f7c601092f0 2024-09-21T06:50:23.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.382+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4c00cde0 con 0x7f7c601092f0 2024-09-21T06:50:23.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.382+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c601a2710 con 0x7f7c601092f0 2024-09-21T06:50:23.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.382+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c601a8e70 con 0x7f7c601092f0 2024-09-21T06:50:23.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.382+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c018780 con 0x7f7c601092f0 2024-09-21T06:50:23.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.383+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c400052c0 con 0x7f7c601092f0 2024-09-21T06:50:23.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.385+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f7c4c018920 con 0x7f7c601092f0 2024-09-21T06:50:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.387+0000 7f7c57e00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c440b6560 0x7f7c440b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.387+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f7c4c0e7570 con 0x7f7c601092f0 2024-09-21T06:50:23.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.387+0000 7f7c5f400640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c440b6560 0x7f7c440b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:23.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.388+0000 7f7c5f400640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c440b6560 0x7f7c440b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005fd0 tx=0x7f7c5000b4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:23.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.389+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c4c0b2510 con 0x7f7c601092f0 2024-09-21T06:50:23.614 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 podman[62797]: 2024-09-21 06:50:23.294943028 +0000 UTC m=+0.424723993 container start 2f8e30bc628063275d4d2edb5efa35ea92a48f1b3542c16f989e785241377686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:50:23.614 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 ceph-mds[62816]: -- 172.21.15.183:0/1427293765 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c771b37860 con 0x55c775cdf180 2024-09-21T06:50:23.614 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h[62812]: starting mds.h at 2024-09-21T06:50:23.614 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 bash[62797]: 2f8e30bc628063275d4d2edb5efa35ea92a48f1b3542c16f989e785241377686 2024-09-21T06:50:23.615 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 06:50:23 smithi183 systemd[1]: Started Ceph mds.h for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:23.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.622+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c400055b0 con 0x7f7c601092f0 2024-09-21T06:50:23.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.623+0000 7f7c57e00640 1 -- 172.21.15.114:0/2737436190 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f7c4c0b7450 con 0x7f7c601092f0 2024-09-21T06:50:23.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:23.625 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":9,"btime":"2024-09-21T06:50:23:383849+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24725,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T06:50:23.626 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 9 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c440b6560 msgr2=0x7f7c440b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c440b6560 0x7f7c440b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c50005fd0 tx=0x7f7c5000b4a0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c601092f0 msgr2=0x7f7c6019ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c601092f0 0x7f7c6019ca50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c6006b930 tx=0x7f7c4c00ee50 comp rx=0 tx=0).stop 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 shutdown_connections 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c60109c40 0x7f7c601a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c601092f0 0x7f7c6019ca50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c440b6560 0x7f7c440b8a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 --2- 172.21.15.114:0/2737436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c601039b0 0x7f7c6019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 >> 172.21.15.114:0/2737436190 conn(0x7f7c600ff400 msgr2=0x7f7c601068e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 shutdown_connections 2024-09-21T06:50:23.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:23.626+0000 7f7c64a00640 1 -- 172.21.15.114:0/2737436190 wait complete. 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] up:boot 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: fsmap 8 up:standby 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:24.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:24.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:24.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: Deploying daemon mds.i on smithi193 2024-09-21T06:50:24.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:24.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:24 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2737436190' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:24.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] up:boot 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: fsmap 8 up:standby 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: Deploying daemon mds.i on smithi193 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:24 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2737436190' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] up:boot 2024-09-21T06:50:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: fsmap 8 up:standby 2024-09-21T06:50:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:50:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: Deploying daemon mds.i on smithi193 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:24.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:24 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2737436190' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:25.662 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 systemd[1]: Starting Ceph mds.i for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T06:50:25.939 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 podman[64494]: 2024-09-21 06:50:25.567796302 +0000 UTC m=+0.016323291 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:50:25.940 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 podman[64494]: 2024-09-21 06:50:25.694282582 +0000 UTC m=+0.142809558 container create 44628a12ac797a6731c5687c4c59ee11e584dc486b8528a84b8e437dd2c5e882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T06:50:25.940 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 podman[64494]: 2024-09-21 06:50:25.938721808 +0000 UTC m=+0.387248783 container init 44628a12ac797a6731c5687c4c59ee11e584dc486b8528a84b8e437dd2c5e882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:50:26.412 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 podman[64494]: 2024-09-21 06:50:25.941549337 +0000 UTC m=+0.390076311 container start 44628a12ac797a6731c5687c4c59ee11e584dc486b8528a84b8e437dd2c5e882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:50:26.412 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 ceph-mds[64513]: -- 172.21.15.193:0/4048600396 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55772ffbd860 con 0x557734165600 2024-09-21T06:50:26.412 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i[64509]: starting mds.i at 2024-09-21T06:50:26.412 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:26 smithi193 bash[64494]: 44628a12ac797a6731c5687c4c59ee11e584dc486b8528a84b8e437dd2c5e882 2024-09-21T06:50:26.412 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 06:50:26 smithi193 systemd[1]: Started Ceph mds.i for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:26.464 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:26.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:26 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:26.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:26 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:26 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:27.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:27 smithi114 ceph-mon[30654]: Deploying daemon mds.g on smithi114 2024-09-21T06:50:27.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:27 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] up:boot 2024-09-21T06:50:27.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:27 smithi114 ceph-mon[30654]: fsmap 9 up:standby 2024-09-21T06:50:27.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:50:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:27 smithi183 ceph-mon[34219]: Deploying daemon mds.g on smithi114 2024-09-21T06:50:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:27 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] up:boot 2024-09-21T06:50:27.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:27 smithi183 ceph-mon[34219]: fsmap 9 up:standby 2024-09-21T06:50:27.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:50:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:27 smithi193 ceph-mon[34601]: Deploying daemon mds.g on smithi114 2024-09-21T06:50:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:27 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] up:boot 2024-09-21T06:50:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:27 smithi193 ceph-mon[34601]: fsmap 9 up:standby 2024-09-21T06:50:27.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:50:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 -- 172.21.15.114:0/928305869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec0757b0 msgr2=0x7ff4ec075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/928305869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec0757b0 0x7ff4ec075bc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009900 tx=0x7ff4dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:50:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 -- 172.21.15.114:0/928305869 shutdown_connections 2024-09-21T06:50:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/928305869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/928305869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 0x7ff4ec10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/928305869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec0757b0 0x7ff4ec075bc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 -- 172.21.15.114:0/928305869 >> 172.21.15.114:0/928305869 conn(0x7ff4ec0ff590 msgr2=0x7ff4ec1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 -- 172.21.15.114:0/928305869 shutdown_connections 2024-09-21T06:50:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.611+0000 7ff4f1e00640 1 -- 172.21.15.114:0/928305869 wait complete. 2024-09-21T06:50:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.612+0000 7ff4f1e00640 1 Processor -- start 2024-09-21T06:50:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.612+0000 7ff4f1e00640 1 -- start start 2024-09-21T06:50:28.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4f1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4ec0757b0 0x7ff4ec19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:28.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4f1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 0x7ff4ec19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:28.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4f1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4f1e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec115c90 con 0x7ff4ec1100e0 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4f1e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec115b10 con 0x7ff4ec076100 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4f1e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec115e10 con 0x7ff4ec0757b0 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4eb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4eaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4ec0757b0 0x7ff4ec19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4eb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec1a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60042/0 (socket says 172.21.15.114:60042) 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4eb400640 1 -- 172.21.15.114:0/4095692572 learned_addr learned my addr 172.21.15.114:0/4095692572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.613+0000 7ff4ea000640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 0x7ff4ec19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4eb400640 1 -- 172.21.15.114:0/4095692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4ec0757b0 msgr2=0x7ff4ec19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4eb400640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4ec0757b0 0x7ff4ec19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4eb400640 1 -- 172.21.15.114:0/4095692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 msgr2=0x7ff4ec19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4eb400640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 0x7ff4ec19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4eb400640 1 -- 172.21.15.114:0/4095692572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec1a2610 con 0x7ff4ec1100e0 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4ea000640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 0x7ff4ec19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4eb400640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec1a1ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400eb10 tx=0x7ff4e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004b20 con 0x7ff4ec1100e0 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec1a2900 con 0x7ff4ec1100e0 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e4004cc0 con 0x7ff4ec1100e0 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.614+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4010690 con 0x7ff4ec1100e0 2024-09-21T06:50:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.615+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec1a9060 con 0x7ff4ec1100e0 2024-09-21T06:50:28.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.616+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec075bc0 con 0x7ff4ec1100e0 2024-09-21T06:50:28.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.618+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7ff4e4005000 con 0x7ff4ec1100e0 2024-09-21T06:50:28.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.620+0000 7ff4f0a00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4d00b6560 0x7ff4d00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:28.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.620+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7ff4e40e7770 con 0x7ff4ec1100e0 2024-09-21T06:50:28.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.620+0000 7ff4eaa00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4d00b6560 0x7ff4d00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:28.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.621+0000 7ff4eaa00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4d00b6560 0x7ff4d00b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc031440 tx=0x7ff4dc009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:28.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.626+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4e4056710 con 0x7ff4ec1100e0 2024-09-21T06:50:28.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:28 smithi114 ceph-mon[30654]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:28 smithi183 ceph-mon[34219]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:28.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.873+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4ec19d7d0 con 0x7ff4ec1100e0 2024-09-21T06:50:28.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.874+0000 7ff4f0a00640 1 -- 172.21.15.114:0/4095692572 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8058 (secure 0 0 0) 0x7ff4e40ae330 con 0x7ff4ec1100e0 2024-09-21T06:50:28.876 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:28.876 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":10,"btime":"2024-09-21T06:50:26:533099+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24725,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24742,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T06:50:28.876 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 10 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4d00b6560 msgr2=0x7ff4d00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4d00b6560 0x7ff4d00b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc031440 tx=0x7ff4dc009560 comp rx=0 tx=0).stop 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 msgr2=0x7ff4ec1a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec1a1ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400eb10 tx=0x7ff4e400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 shutdown_connections 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4ec1100e0 0x7ff4ec1a1ee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4d00b6560 0x7ff4d00b8a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec076100 0x7ff4ec19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 --2- 172.21.15.114:0/4095692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4ec0757b0 0x7ff4ec19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:28.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.876+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 >> 172.21.15.114:0/4095692572 conn(0x7ff4ec0ff590 msgr2=0x7ff4ec109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:28.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.877+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 shutdown_connections 2024-09-21T06:50:28.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:28.877+0000 7ff4f1e00640 1 -- 172.21.15.114:0/4095692572 wait complete. 2024-09-21T06:50:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:28 smithi193 ceph-mon[34601]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:29.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:29 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4095692572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:29.815 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 06:50:29 smithi114 systemd[1]: Started Ceph mds.g for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:29.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:29 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4095692572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:29 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4095692572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:30.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:31.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:31 smithi114 ceph-mon[30654]: Deploying daemon mds.a on smithi114 2024-09-21T06:50:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:31 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6838/244481807,v1:172.21.15.114:6839/244481807] up:boot 2024-09-21T06:50:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:31 smithi114 ceph-mon[30654]: fsmap 10 up:standby 2024-09-21T06:50:31.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:50:31.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:31 smithi183 ceph-mon[34219]: Deploying daemon mds.a on smithi114 2024-09-21T06:50:31.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:31 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6838/244481807,v1:172.21.15.114:6839/244481807] up:boot 2024-09-21T06:50:31.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:31 smithi183 ceph-mon[34219]: fsmap 10 up:standby 2024-09-21T06:50:31.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:50:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:31 smithi193 ceph-mon[34601]: Deploying daemon mds.a on smithi114 2024-09-21T06:50:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:31 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6838/244481807,v1:172.21.15.114:6839/244481807] up:boot 2024-09-21T06:50:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:31 smithi193 ceph-mon[34601]: fsmap 10 up:standby 2024-09-21T06:50:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:50:32.065 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:32.744 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:32 smithi114 ceph-mon[30654]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:32 smithi183 ceph-mon[34219]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:32 smithi193 ceph-mon[34601]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:32.970 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 -- 172.21.15.114:0/407821872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410b6f0 msgr2=0x7f27f410bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 --2- 172.21.15.114:0/407821872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410b6f0 0x7f27f410bb00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009900 tx=0x7f27dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 -- 172.21.15.114:0/407821872 shutdown_connections 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 --2- 172.21.15.114:0/407821872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27f410c040 0x7f27f4112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 --2- 172.21.15.114:0/407821872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410b6f0 0x7f27f410bb00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 --2- 172.21.15.114:0/407821872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f4105db0 0x7f27f41061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 -- 172.21.15.114:0/407821872 >> 172.21.15.114:0/407821872 conn(0x7f27f41017e0 msgr2=0x7f27f4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:33.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 -- 172.21.15.114:0/407821872 shutdown_connections 2024-09-21T06:50:33.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.601+0000 7f27f9600640 1 -- 172.21.15.114:0/407821872 wait complete. 2024-09-21T06:50:33.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.602+0000 7f27f9600640 1 Processor -- start 2024-09-21T06:50:33.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 -- start start 2024-09-21T06:50:33.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27f4105db0 0x7f27f419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:33.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 0x7f27f419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:33.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410c040 0x7f27f41a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:33.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4115d00 con 0x7f27f410c040 2024-09-21T06:50:33.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4115b80 con 0x7f27f410b6f0 2024-09-21T06:50:33.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f9600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4115e80 con 0x7f27f4105db0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27ea000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 0x7f27f419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27ea000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 0x7f27f419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40744/0 (socket says 172.21.15.114:40744) 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27ea000640 1 -- 172.21.15.114:0/3080969832 learned_addr learned my addr 172.21.15.114:0/3080969832 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f2a00640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27f4105db0 0x7f27f419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.603+0000 7f27f3400640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410c040 0x7f27f41a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.604+0000 7f27ea000640 1 -- 172.21.15.114:0/3080969832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27f4105db0 msgr2=0x7f27f419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.604+0000 7f27ea000640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27f4105db0 0x7f27f419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.604+0000 7f27ea000640 1 -- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410c040 msgr2=0x7f27f41a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.604+0000 7f27ea000640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410c040 0x7f27f41a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.604+0000 7f27ea000640 1 -- 172.21.15.114:0/3080969832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f41a26c0 con 0x7f27f410b6f0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.604+0000 7f27ea000640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 0x7f27f419ccf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006010 tx=0x7f27dc004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.605+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc03f030 con 0x7f27f410b6f0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.605+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f41a2950 con 0x7f27f410b6f0 2024-09-21T06:50:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.605+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27dc003aa0 con 0x7f27f410b6f0 2024-09-21T06:50:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.605+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc035070 con 0x7f27f410b6f0 2024-09-21T06:50:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.605+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f41a9110 con 0x7f27f410b6f0 2024-09-21T06:50:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.606+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f41061c0 con 0x7f27f410b6f0 2024-09-21T06:50:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.608+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f27dc042030 con 0x7f27f410b6f0 2024-09-21T06:50:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.610+0000 7f27ebe00640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27d80b6560 0x7f27d80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.610+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f27dc1074c0 con 0x7f27f410b6f0 2024-09-21T06:50:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.610+0000 7f27f2a00640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27d80b6560 0x7f27d80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:33.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.611+0000 7f27f2a00640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27d80b6560 0x7f27d80b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005d00 tx=0x7f27e400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.613+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27dc0d2460 con 0x7f27f410b6f0 2024-09-21T06:50:33.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.838+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27f410de30 con 0x7f27f410b6f0 2024-09-21T06:50:33.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.839+0000 7f27ebe00640 1 -- 172.21.15.114:0/3080969832 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7f27dc0d73a0 con 0x7f27f410b6f0 2024-09-21T06:50:33.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:33.841 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":12,"btime":"2024-09-21T06:50:33:572798+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14742,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24725,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24742,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24760,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-09-21T06:50:33.841 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 12 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27d80b6560 msgr2=0x7f27d80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27d80b6560 0x7f27d80b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005d00 tx=0x7f27e400b410 comp rx=0 tx=0).stop 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 msgr2=0x7f27f419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 0x7f27f419ccf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006010 tx=0x7f27dc004060 comp rx=0 tx=0).stop 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 shutdown_connections 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27f410c040 0x7f27f41a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27d80b6560 0x7f27d80b8a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.841+0000 7f27f9600640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f410b6f0 0x7f27f419ccf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.842+0000 7f27f9600640 1 --2- 172.21.15.114:0/3080969832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27f4105db0 0x7f27f419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.842+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 >> 172.21.15.114:0/3080969832 conn(0x7f27f41017e0 msgr2=0x7f27f410ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.842+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 shutdown_connections 2024-09-21T06:50:33.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:33.842+0000 7f27f9600640 1 -- 172.21.15.114:0/3080969832 wait complete. 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: Deploying daemon mds.b on smithi183 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] up:boot 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: fsmap 11 up:standby 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:50:34.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:33 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3080969832' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:34.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.219 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: Deploying daemon mds.b on smithi183 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:34.220 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] up:boot 2024-09-21T06:50:34.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: fsmap 11 up:standby 2024-09-21T06:50:34.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:50:34.221 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:33 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3080969832' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T06:50:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T06:50:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:34.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: Deploying daemon mds.b on smithi183 2024-09-21T06:50:34.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:34.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] up:boot 2024-09-21T06:50:34.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: fsmap 11 up:standby 2024-09-21T06:50:34.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:50:34.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:33 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3080969832' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:35.365 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 podman[63065]: 2024-09-21 06:50:35.29292794 +0000 UTC m=+0.139933859 container create 6ed66db88e2214c16687189ca1fc3c030ae7d863f33adcf3995b4c934630acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:50:35.864 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 podman[63065]: 2024-09-21 06:50:35.56264273 +0000 UTC m=+0.409648642 container init 6ed66db88e2214c16687189ca1fc3c030ae7d863f33adcf3995b4c934630acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T06:50:35.865 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 podman[63065]: 2024-09-21 06:50:35.565456991 +0000 UTC m=+0.412462916 container start 6ed66db88e2214c16687189ca1fc3c030ae7d863f33adcf3995b4c934630acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T06:50:35.865 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 ceph-mds[63084]: -- 172.21.15.183:0/881580367 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56315204b860 con 0x5631561f3180 2024-09-21T06:50:35.865 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: starting mds.b at 2024-09-21T06:50:35.865 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 bash[63065]: 6ed66db88e2214c16687189ca1fc3c030ae7d863f33adcf3995b4c934630acdd 2024-09-21T06:50:35.865 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:35 smithi183 systemd[1]: Started Ceph mds.b for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:36 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:36.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:50:36.811 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:36.811 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.811 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.811 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.811 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.812 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.812 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:36 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:36.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:36 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:37.315 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] up:boot 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: fsmap 12 up:standby 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:37 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] up:boot 2024-09-21T06:50:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: fsmap 12 up:standby 2024-09-21T06:50:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:50:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:37 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] up:boot 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: fsmap 12 up:standby 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:37 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 -- 172.21.15.114:0/1292422251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590114440 msgr2=0x7fb590116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 --2- 172.21.15.114:0/1292422251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590114440 0x7fb590116910 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb588009ca0 tx=0x7fb58802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 -- 172.21.15.114:0/1292422251 shutdown_connections 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 --2- 172.21.15.114:0/1292422251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590114440 0x7fb590116910 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 --2- 172.21.15.114:0/1292422251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590075dd0 0x7fb5900761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 --2- 172.21.15.114:0/1292422251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb590077380 0x7fb5900757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 -- 172.21.15.114:0/1292422251 >> 172.21.15.114:0/1292422251 conn(0x7fb5900ff5e0 msgr2=0x7fb590101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:37.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 -- 172.21.15.114:0/1292422251 shutdown_connections 2024-09-21T06:50:37.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.938+0000 7fb598600640 1 -- 172.21.15.114:0/1292422251 wait complete. 2024-09-21T06:50:37.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 Processor -- start 2024-09-21T06:50:37.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 -- start start 2024-09-21T06:50:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590075dd0 0x7fb5901a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 0x7fb5901a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb590114440 0x7fb5901a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb59011a070 con 0x7fb590075dd0 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb590119ef0 con 0x7fb590077380 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.940+0000 7fb598600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb59011a1f0 con 0x7fb590114440 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 0x7fb5901a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 0x7fb5901a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58484/0 (socket says 172.21.15.114:58484) 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 -- 172.21.15.114:0/3866352311 learned_addr learned my addr 172.21.15.114:0/3866352311 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595c00640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590075dd0 0x7fb5901a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb596600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb590114440 0x7fb5901a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 -- 172.21.15.114:0/3866352311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb590114440 msgr2=0x7fb5901a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb590114440 0x7fb5901a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 -- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590075dd0 msgr2=0x7fb5901a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590075dd0 0x7fb5901a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 -- 172.21.15.114:0/3866352311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5901a6950 con 0x7fb590077380 2024-09-21T06:50:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.941+0000 7fb595200640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 0x7fb5901a1040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e9c0 tx=0x7fb57c00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.942+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00ccc0 con 0x7fb590077380 2024-09-21T06:50:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.942+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5901a6c40 con 0x7fb590077380 2024-09-21T06:50:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.942+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c0054d0 con 0x7fb590077380 2024-09-21T06:50:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.942+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c010430 con 0x7fb590077380 2024-09-21T06:50:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.942+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901ad460 con 0x7fb590077380 2024-09-21T06:50:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.943+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5900761e0 con 0x7fb590077380 2024-09-21T06:50:37.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.946+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fb57c004f20 con 0x7fb590077380 2024-09-21T06:50:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.947+0000 7fb58e000640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5780b6590 0x7fb5780b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.948+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7fb57c0e7510 con 0x7fb590077380 2024-09-21T06:50:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.948+0000 7fb595c00640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5780b6590 0x7fb5780b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.948+0000 7fb595c00640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5780b6590 0x7fb5780b8aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb58400ab60 tx=0x7fb58400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:37.951+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb57c0ae0d0 con 0x7fb590077380 2024-09-21T06:50:38.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.201+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb59002e640 con 0x7fb590077380 2024-09-21T06:50:38.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.202+0000 7fb58e000640 1 -- 172.21.15.114:0/3866352311 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10536 (secure 0 0 0) 0x7fb57c0b24b0 con 0x7fb590077380 2024-09-21T06:50:38.204 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:38.204 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":13,"btime":"2024-09-21T06:50:36:535220+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14742,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24725,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24742,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24760,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24761,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T06:50:38.204 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 13 2024-09-21T06:50:38.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.204+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5780b6590 msgr2=0x7fb5780b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:38.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.204+0000 7fb598600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5780b6590 0x7fb5780b8aa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb58400ab60 tx=0x7fb58400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:50:38.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 msgr2=0x7fb5901a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:38.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 0x7fb5901a1040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e9c0 tx=0x7fb57c00edc0 comp rx=0 tx=0).stop 2024-09-21T06:50:38.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 shutdown_connections 2024-09-21T06:50:38.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb590114440 0x7fb5901a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:38.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5780b6590 0x7fb5780b8aa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:38.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb590077380 0x7fb5901a1040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:38.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 --2- 172.21.15.114:0/3866352311 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb590075dd0 0x7fb5901a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:38.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 >> 172.21.15.114:0/3866352311 conn(0x7fb5900ff5e0 msgr2=0x7fb590110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:38.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 shutdown_connections 2024-09-21T06:50:38.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:38.205+0000 7fb598600640 1 -- 172.21.15.114:0/3866352311 wait complete. 2024-09-21T06:50:38.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:38 smithi114 ceph-mon[30654]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:38.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:38 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3866352311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:38 smithi183 ceph-mon[34219]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:38.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:38 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3866352311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:38.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:38 smithi193 ceph-mon[34601]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:38 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3866352311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:50:39.041 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T06:50:39.430 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:40.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 -- 172.21.15.114:0/3356614681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 msgr2=0x7f81641087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:40.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 --2- 172.21.15.114:0/3356614681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f81641087c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8150009900 tx=0x7f815002a410 comp rx=0 tx=0).stop 2024-09-21T06:50:40.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 -- 172.21.15.114:0/3356614681 shutdown_connections 2024-09-21T06:50:40.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 --2- 172.21.15.114:0/3356614681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164108d00 0x7f816410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 --2- 172.21.15.114:0/3356614681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f81641087c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 --2- 172.21.15.114:0/3356614681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 0x7f8164102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 -- 172.21.15.114:0/3356614681 >> 172.21.15.114:0/3356614681 conn(0x7f8164079720 msgr2=0x7f8164079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:40.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 -- 172.21.15.114:0/3356614681 shutdown_connections 2024-09-21T06:50:40.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.107+0000 7f816b200640 1 -- 172.21.15.114:0/3356614681 wait complete. 2024-09-21T06:50:40.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.109+0000 7f816b200640 1 Processor -- start 2024-09-21T06:50:40.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.109+0000 7f816b200640 1 -- start start 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f816b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 0x7f816419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f816b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f816419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f816b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164108d00 0x7f816419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f816b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f81641129c0 con 0x7f81641083b0 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f816b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8164112840 con 0x7f8164102ac0 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f816b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8164112b40 con 0x7f8164108d00 2024-09-21T06:50:40.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f8163400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f816419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f8163400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f816419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35010/0 (socket says 172.21.15.114:35010) 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f8163400640 1 -- 172.21.15.114:0/402661441 learned_addr learned my addr 172.21.15.114:0/402661441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f8163e00640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 0x7f816419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.110+0000 7f8169200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164108d00 0x7f816419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163e00640 1 -- 172.21.15.114:0/402661441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164108d00 msgr2=0x7f816419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163e00640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164108d00 0x7f816419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163e00640 1 -- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 msgr2=0x7f816419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163e00640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f816419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163e00640 1 -- 172.21.15.114:0/402661441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81641a04b0 con 0x7f8164102ac0 2024-09-21T06:50:40.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163400640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f816419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:40.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8163e00640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 0x7f816419a540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f815400e9f0 tx=0x7f815400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815400ccc0 con 0x7f8164102ac0 2024-09-21T06:50:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81540108e0 con 0x7f8164102ac0 2024-09-21T06:50:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641a07a0 con 0x7f8164102ac0 2024-09-21T06:50:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.111+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154018880 con 0x7f8164102ac0 2024-09-21T06:50:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.112+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a6f00 con 0x7f8164102ac0 2024-09-21T06:50:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.113+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8164102ed0 con 0x7f8164102ac0 2024-09-21T06:50:40.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.115+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f8154010430 con 0x7f8164102ac0 2024-09-21T06:50:40.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.116+0000 7f8160c00640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81480b65e0 0x7f81480b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:40.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.117+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f81540e74f0 con 0x7f8164102ac0 2024-09-21T06:50:40.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.117+0000 7f8163400640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81480b65e0 0x7f81480b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:40.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.117+0000 7f8163400640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81480b65e0 0x7f81480b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8164079fa0 tx=0x7f8150032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:40.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.118+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81540b2490 con 0x7f8164102ac0 2024-09-21T06:50:40.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:40.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:40 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.322+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f816402e640 con 0x7f8164102ac0 2024-09-21T06:50:40.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.323+0000 7f8160c00640 1 -- 172.21.15.114:0/402661441 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f81540b73d0 con 0x7f8164102ac0 2024-09-21T06:50:40.325 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:50:40.326 INFO:teuthology.orchestra.run.smithi114.stdout:[ 2024-09-21T06:50:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T06:50:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "poolnum": 1, 2024-09-21T06:50:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: "poolname": ".mgr" 2024-09-21T06:50:40.326 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:50:40.326 INFO:teuthology.orchestra.run.smithi114.stdout:] 2024-09-21T06:50:40.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81480b65e0 msgr2=0x7f81480b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:40.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81480b65e0 0x7f81480b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8164079fa0 tx=0x7f8150032ed0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 msgr2=0x7f816419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 0x7f816419a540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f815400e9f0 tx=0x7f815400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 shutdown_connections 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8164108d00 0x7f816419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f81641083b0 0x7f816419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81480b65e0 0x7f81480b8af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 --2- 172.21.15.114:0/402661441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164102ac0 0x7f816419a540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 >> 172.21.15.114:0/402661441 conn(0x7f8164079720 msgr2=0x7f8164101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 shutdown_connections 2024-09-21T06:50:40.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:40.326+0000 7f816b200640 1 -- 172.21.15.114:0/402661441 wait complete. 2024-09-21T06:50:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:40.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:40.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:40.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:40.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:40 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:40.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:40 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.360 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T06:50:41.360 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T06:50:41.409 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/402661441' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:50:41.409 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.409 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.409 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.409 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:41 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/402661441' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:50:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:41 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/402661441' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T06:50:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:41 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:41.797 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:42.290 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:42 smithi114 ceph-mon[30654]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 -- 172.21.15.114:0/1729605468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c1049d0 msgr2=0x7f300c104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 --2- 172.21.15.114:0/1729605468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c1049d0 0x7f300c104de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f300002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:50:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 -- 172.21.15.114:0/1729605468 shutdown_connections 2024-09-21T06:50:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 --2- 172.21.15.114:0/1729605468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c10de30 0x7f300c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 --2- 172.21.15.114:0/1729605468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c105320 0x7f300c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:42.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 --2- 172.21.15.114:0/1729605468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c1049d0 0x7f300c104de0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:42.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 -- 172.21.15.114:0/1729605468 >> 172.21.15.114:0/1729605468 conn(0x7f300c079740 msgr2=0x7f300c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:42.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 -- 172.21.15.114:0/1729605468 shutdown_connections 2024-09-21T06:50:42.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.469+0000 7f3013800640 1 -- 172.21.15.114:0/1729605468 wait complete. 2024-09-21T06:50:42.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.471+0000 7f3013800640 1 Processor -- start 2024-09-21T06:50:42.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.471+0000 7f3013800640 1 -- start start 2024-09-21T06:50:42.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3013800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c1049d0 0x7f300c102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:42.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3013800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 0x7f300c102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:42.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3013800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 0x7f300c103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:42.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3013800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f300c113ab0 con 0x7f300c105320 2024-09-21T06:50:42.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3013800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f300c113930 con 0x7f300c10de30 2024-09-21T06:50:42.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3013800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f300c113c30 con 0x7f300c1049d0 2024-09-21T06:50:42.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f300be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 0x7f300c102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:42.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3010e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c1049d0 0x7f300c102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:42.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3011800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 0x7f300c103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:42.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f300be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 0x7f300c102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35032/0 (socket says 172.21.15.114:35032) 2024-09-21T06:50:42.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3011800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 0x7f300c103190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58536/0 (socket says 172.21.15.114:58536) 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3010e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c1049d0 0x7f300c102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51860/0 (socket says 172.21.15.114:51860) 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f300be00640 1 -- 172.21.15.114:0/3498667429 learned_addr learned my addr 172.21.15.114:0/3498667429 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3011800640 1 -- 172.21.15.114:0/3498667429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c1049d0 msgr2=0x7f300c102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3011800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c1049d0 0x7f300c102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3011800640 1 -- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 msgr2=0x7f300c102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.472+0000 7f3011800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 0x7f300c102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3011800640 1 -- 172.21.15.114:0/3498667429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1a87c0 con 0x7f300c10de30 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f300be00640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 0x7f300c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3011800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 0x7f300c103190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3004009fd0 tx=0x7f300400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300400ce20 con 0x7f300c10de30 2024-09-21T06:50:42.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3004005020 con 0x7f300c10de30 2024-09-21T06:50:42.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1a8ab0 con 0x7f300c10de30 2024-09-21T06:50:42.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004010690 con 0x7f300c10de30 2024-09-21T06:50:42.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.473+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a8ff0 con 0x7f300c10de30 2024-09-21T06:50:42.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.474+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c06c240 con 0x7f300c10de30 2024-09-21T06:50:42.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.477+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f30040054d0 con 0x7f300c10de30 2024-09-21T06:50:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.478+0000 7f3009600640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2ff40b65e0 0x7f2ff40b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:42.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.478+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f30040e76d0 con 0x7f300c10de30 2024-09-21T06:50:42.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.479+0000 7f3010e00640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2ff40b65e0 0x7f2ff40b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:42.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.479+0000 7f3010e00640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2ff40b65e0 0x7f2ff40b8af0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f3000031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:42.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.480+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30040ae290 con 0x7f300c10de30 2024-09-21T06:50:42.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:42 smithi183 ceph-mon[34219]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:42 smithi193 ceph-mon[34601]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:42.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:42.680+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f300c103a10 con 0x7f300c10de30 2024-09-21T06:50:43.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.182+0000 7f3009600640 1 -- 172.21.15.114:0/3498667429 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f30040b2670 con 0x7f300c10de30 2024-09-21T06:50:43.185 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' created 2024-09-21T06:50:43.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2ff40b65e0 msgr2=0x7f2ff40b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:43.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2ff40b65e0 0x7f2ff40b8af0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f3000031400 comp rx=0 tx=0).stop 2024-09-21T06:50:43.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 msgr2=0x7f300c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 0x7f300c103190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3004009fd0 tx=0x7f300400ef10 comp rx=0 tx=0).stop 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 shutdown_connections 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2ff40b65e0 0x7f2ff40b8af0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f300c10de30 0x7f300c103190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f300c105320 0x7f300c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 --2- 172.21.15.114:0/3498667429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c1049d0 0x7f300c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 >> 172.21.15.114:0/3498667429 conn(0x7f300c079740 msgr2=0x7f300c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 shutdown_connections 2024-09-21T06:50:43.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:43.185+0000 7f3013800640 1 -- 172.21.15.114:0/3498667429 wait complete. 2024-09-21T06:50:43.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3498667429' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:50:43.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:50:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3498667429' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:50:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:50:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:43.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3498667429' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:50:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T06:50:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:50:43.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:50:43.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:50:43.984 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T06:50:44.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:44 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:50:44.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:44 smithi114 ceph-mon[30654]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:50:44.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:44 smithi114 ceph-mon[30654]: pgmap v277: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:44 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:50:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:44 smithi183 ceph-mon[34219]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:50:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:44 smithi183 ceph-mon[34219]: pgmap v277: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:44 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T06:50:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:44 smithi193 ceph-mon[34601]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T06:50:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:44 smithi193 ceph-mon[34601]: pgmap v277: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:45.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 -- 172.21.15.114:0/2538961093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe80010de30 msgr2=0x7fe800110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:45.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 --2- 172.21.15.114:0/2538961093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe80010de30 0x7fe800110320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009ca0 tx=0x7fe7f802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:50:45.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 -- 172.21.15.114:0/2538961093 shutdown_connections 2024-09-21T06:50:45.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 --2- 172.21.15.114:0/2538961093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe80010de30 0x7fe800110320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 --2- 172.21.15.114:0/2538961093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe800105320 0x7fe80010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 --2- 172.21.15.114:0/2538961093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8001049d0 0x7fe800104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 -- 172.21.15.114:0/2538961093 >> 172.21.15.114:0/2538961093 conn(0x7fe800079740 msgr2=0x7fe800079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 -- 172.21.15.114:0/2538961093 shutdown_connections 2024-09-21T06:50:45.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.055+0000 7fe806400640 1 -- 172.21.15.114:0/2538961093 wait complete. 2024-09-21T06:50:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.056+0000 7fe806400640 1 Processor -- start 2024-09-21T06:50:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.056+0000 7fe806400640 1 -- start start 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe806400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 0x7fe800102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe806400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe800105320 0x7fe800102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe806400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80010de30 0x7fe800103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe806400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113b00 con 0x7fe8001049d0 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe806400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113980 con 0x7fe80010de30 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe806400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe800113c80 con 0x7fe800105320 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 0x7fe800102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe7ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 0x7fe800102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42792/0 (socket says 172.21.15.114:42792) 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe7ff400640 1 -- 172.21.15.114:0/3114071968 learned_addr learned my addr 172.21.15.114:0/3114071968 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe7f7e00640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe800105320 0x7fe800102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.057+0000 7fe7ffe00640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80010de30 0x7fe800103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7f7e00640 1 -- 172.21.15.114:0/3114071968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80010de30 msgr2=0x7fe800103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7f7e00640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80010de30 0x7fe800103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7f7e00640 1 -- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 msgr2=0x7fe800102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7f7e00640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 0x7fe800102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7f7e00640 1 -- 172.21.15.114:0/3114071968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8001a87b0 con 0x7fe800105320 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7ff400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 0x7fe800102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7f7e00640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe800105320 0x7fe800102c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005fd0 tx=0x7fe7e800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8009c00 con 0x7fe800105320 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8001a8aa0 con 0x7fe800105320 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7e8009da0 con 0x7fe800105320 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.058+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003db0 con 0x7fe800105320 2024-09-21T06:50:45.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.059+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001a8fe0 con 0x7fe800105320 2024-09-21T06:50:45.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.059+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80006c240 con 0x7fe800105320 2024-09-21T06:50:45.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.062+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fe7e8004000 con 0x7fe800105320 2024-09-21T06:50:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.064+0000 7fe7fcc00640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe7e40b6560 0x7fe7e40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.064+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7712+0+0 (secure 0 0 0) 0x7fe7e80e6bc0 con 0x7fe800105320 2024-09-21T06:50:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.065+0000 7fe7ff400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe7e40b6560 0x7fe7e40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:45.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.065+0000 7fe7ff400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe7e40b6560 0x7fe7e40b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe800104160 tx=0x7fe7f0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:45.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.065+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7e8017020 con 0x7fe800105320 2024-09-21T06:50:45.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:45.255+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fe8001a9320 con 0x7fe800105320 2024-09-21T06:50:45.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:45 smithi114 ceph-mon[30654]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:50:45.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:45 smithi183 ceph-mon[34219]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:50:45.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:45 smithi193 ceph-mon[34601]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T06:50:46.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.202+0000 7fe7fcc00640 1 -- 172.21.15.114:0/3114071968 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7fe7e80ad630 con 0x7fe800105320 2024-09-21T06:50:46.207 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' created 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.208+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe7e40b6560 msgr2=0x7fe7e40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.208+0000 7fe806400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe7e40b6560 0x7fe7e40b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe800104160 tx=0x7fe7f0005f90 comp rx=0 tx=0).stop 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.208+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe800105320 msgr2=0x7fe800102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.208+0000 7fe806400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe800105320 0x7fe800102c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8005fd0 tx=0x7fe7e800dde0 comp rx=0 tx=0).stop 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 shutdown_connections 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe7e40b6560 0x7fe7e40b8a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe80010de30 0x7fe800103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe800105320 0x7fe800102c50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:46.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 --2- 172.21.15.114:0/3114071968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe8001049d0 0x7fe800102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 >> 172.21.15.114:0/3114071968 conn(0x7fe800079740 msgr2=0x7fe8000ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 shutdown_connections 2024-09-21T06:50:46.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:46.209+0000 7fe806400640 1 -- 172.21.15.114:0/3114071968 wait complete. 2024-09-21T06:50:46.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:46 smithi114 ceph-mon[30654]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:50:46.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:46 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3114071968' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:50:46.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:46 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:50:46.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:46 smithi114 ceph-mon[30654]: pgmap v280: 33 pgs: 7 active+clean, 2 creating+peering, 24 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:46.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:46 smithi183 ceph-mon[34219]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:50:46.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:46 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3114071968' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:50:46.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:46 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:50:46.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:46 smithi183 ceph-mon[34219]: pgmap v280: 33 pgs: 7 active+clean, 2 creating+peering, 24 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:46 smithi193 ceph-mon[34601]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T06:50:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:46 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3114071968' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:50:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:46 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-21T06:50:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:46 smithi193 ceph-mon[34601]: pgmap v280: 33 pgs: 7 active+clean, 2 creating+peering, 24 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:47.129 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T06:50:47.517 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:47.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:47 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T06:50:47.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:47 smithi114 ceph-mon[30654]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:50:47.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:47 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T06:50:47.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:47 smithi183 ceph-mon[34219]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:50:47.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:47 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-21T06:50:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:47 smithi193 ceph-mon[34601]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T06:50:48.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.163+0000 7f24bec00640 1 -- 172.21.15.114:0/651889199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 msgr2=0x7f24b8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:48.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.163+0000 7f24bec00640 1 --2- 172.21.15.114:0/651889199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b8110470 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009ca0 tx=0x7f24b002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 -- 172.21.15.114:0/651889199 shutdown_connections 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 --2- 172.21.15.114:0/651889199 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b8110470 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 --2- 172.21.15.114:0/651889199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81092f0 0x7f24b8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 --2- 172.21.15.114:0/651889199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b81039b0 0x7f24b8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 -- 172.21.15.114:0/651889199 >> 172.21.15.114:0/651889199 conn(0x7f24b80ff400 msgr2=0x7f24b8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 -- 172.21.15.114:0/651889199 shutdown_connections 2024-09-21T06:50:48.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.164+0000 7f24bec00640 1 -- 172.21.15.114:0/651889199 wait complete. 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 Processor -- start 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 -- start start 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 0x7f24b819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b81092f0 0x7f24b819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b81a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8113900 con 0x7f24b8109c40 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8113780 con 0x7f24b81092f0 2024-09-21T06:50:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.165+0000 7f24bec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8113a80 con 0x7f24b81039b0 2024-09-21T06:50:48.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24be200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b81a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:48.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 0x7f24b819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:48.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24be200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b81a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42814/0 (socket says 172.21.15.114:42814) 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 0x7f24b819c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43072/0 (socket says 172.21.15.114:43072) 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24be200640 1 -- 172.21.15.114:0/4201809197 learned_addr learned my addr 172.21.15.114:0/4201809197 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bce00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b81092f0 0x7f24b819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 -- 172.21.15.114:0/4201809197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b81092f0 msgr2=0x7f24b819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b81092f0 0x7f24b819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 -- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 msgr2=0x7f24b81a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b81a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 -- 172.21.15.114:0/4201809197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b81a2420 con 0x7f24b81039b0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.166+0000 7f24bd800640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 0x7f24b819c4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00e9f0 tx=0x7f24ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.167+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00ccc0 con 0x7f24b81039b0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.167+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b81a2710 con 0x7f24b81039b0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.167+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24ac0108e0 con 0x7f24b81039b0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.167+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac018880 con 0x7f24b81039b0 2024-09-21T06:50:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.167+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81a8e70 con 0x7f24b81039b0 2024-09-21T06:50:48.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.168+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b8103dc0 con 0x7f24b81039b0 2024-09-21T06:50:48.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.170+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f24ac010430 con 0x7f24b81039b0 2024-09-21T06:50:48.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.172+0000 7f24b6000640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24a00b6560 0x7f24a00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:48.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.172+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8094+0+0 (secure 0 0 0) 0x7f24ac0e7a50 con 0x7f24b81039b0 2024-09-21T06:50:48.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.172+0000 7f24bce00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24a00b6560 0x7f24a00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:48.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.173+0000 7f24bce00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24a00b6560 0x7f24a00b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24b8105a60 tx=0x7f24a4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:48.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.174+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24ac0b27a0 con 0x7f24b81039b0 2024-09-21T06:50:48.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:48 smithi114 ceph-mon[30654]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:50:48.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:48 smithi114 ceph-mon[30654]: pgmap v283: 97 pgs: 22 active+clean, 26 creating+peering, 49 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:48.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:48.404+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f24b81a9160 con 0x7f24b81039b0 2024-09-21T06:50:48.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:48 smithi183 ceph-mon[34219]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:50:48.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:48 smithi183 ceph-mon[34219]: pgmap v283: 97 pgs: 22 active+clean, 26 creating+peering, 49 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:48.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:48 smithi193 ceph-mon[34601]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T06:50:48.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:48 smithi193 ceph-mon[34601]: pgmap v283: 97 pgs: 22 active+clean, 26 creating+peering, 49 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:49.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.220+0000 7f24b6000640 1 -- 172.21.15.114:0/4201809197 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f24ac0b76e0 con 0x7f24b81039b0 2024-09-21T06:50:49.223 INFO:teuthology.orchestra.run.smithi114.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T06:50:49.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.225+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24a00b6560 msgr2=0x7f24a00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:49.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.225+0000 7f24bec00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24a00b6560 0x7f24a00b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f24b8105a60 tx=0x7f24a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:50:49.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.225+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 msgr2=0x7f24b819c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:49.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.225+0000 7f24bec00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 0x7f24b819c4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00e9f0 tx=0x7f24ac00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:50:49.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.226+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 shutdown_connections 2024-09-21T06:50:49.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.226+0000 7f24bec00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24b8109c40 0x7f24b81a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:49.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.226+0000 7f24bec00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24a00b6560 0x7f24a00b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:49.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.226+0000 7f24bec00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24b81092f0 0x7f24b819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:49.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.226+0000 7f24bec00640 1 --2- 172.21.15.114:0/4201809197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24b81039b0 0x7f24b819c4b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:49.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.226+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 >> 172.21.15.114:0/4201809197 conn(0x7f24b80ff400 msgr2=0x7f24b8107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:49.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.227+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 shutdown_connections 2024-09-21T06:50:49.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:49.227+0000 7f24bec00640 1 -- 172.21.15.114:0/4201809197 wait complete. 2024-09-21T06:50:49.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:49 smithi114 ceph-mon[30654]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:50:49.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4201809197' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:50:49.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:49 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:50:49.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:49 smithi114 ceph-mon[30654]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:50:49.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:49 smithi183 ceph-mon[34219]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:50:49.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4201809197' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:50:49.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:49 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:50:49.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:49 smithi183 ceph-mon[34219]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:50:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:49 smithi193 ceph-mon[34601]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:50:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4201809197' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:50:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:49 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T06:50:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:49 smithi193 ceph-mon[34601]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T06:50:50.145 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T06:50:50.145 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T06:50:50.516 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:50.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:50:50.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:50:50.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:50:50.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:50:50.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: pgmap v286: 97 pgs: 61 active+clean, 24 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:50.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] up:active 2024-09-21T06:50:50.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:50 smithi114 ceph-mon[30654]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:50:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:50:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:50:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:50:50.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:50:50.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: pgmap v286: 97 pgs: 61 active+clean, 24 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:50.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] up:active 2024-09-21T06:50:50.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:50 smithi183 ceph-mon[34219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: pgmap v286: 97 pgs: 61 active+clean, 24 creating+peering, 12 unknown; 577 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] up:active 2024-09-21T06:50:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:50 smithi193 ceph-mon[34601]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.142+0000 7f3ce7200640 1 -- 172.21.15.114:0/540190487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce0105d80 msgr2=0x7f3ce0106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.142+0000 7f3ce7200640 1 --2- 172.21.15.114:0/540190487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce0105d80 0x7f3ce0106190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009900 tx=0x7f3cd002a410 comp rx=0 tx=0).stop 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 -- 172.21.15.114:0/540190487 shutdown_connections 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 --2- 172.21.15.114:0/540190487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ce010c010 0x7f3ce0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 --2- 172.21.15.114:0/540190487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ce010b6c0 0x7f3ce010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 --2- 172.21.15.114:0/540190487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce0105d80 0x7f3ce0106190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 -- 172.21.15.114:0/540190487 >> 172.21.15.114:0/540190487 conn(0x7f3ce0101810 msgr2=0x7f3ce0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 -- 172.21.15.114:0/540190487 shutdown_connections 2024-09-21T06:50:51.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.143+0000 7f3ce7200640 1 -- 172.21.15.114:0/540190487 wait complete. 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 Processor -- start 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 -- start start 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ce0105d80 0x7f3ce019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 0x7f3ce019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ce010c010 0x7f3ce01a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0115d20 con 0x7f3ce010b6c0 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0115ba0 con 0x7f3ce010c010 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.144+0000 7f3ce7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0115ea0 con 0x7f3ce0105d80 2024-09-21T06:50:51.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 0x7f3ce019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cdfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ce0105d80 0x7f3ce019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 0x7f3ce019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42838/0 (socket says 172.21.15.114:42838) 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 -- 172.21.15.114:0/3146313228 learned_addr learned my addr 172.21.15.114:0/3146313228 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3ce5200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ce010c010 0x7f3ce01a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 -- 172.21.15.114:0/3146313228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ce0105d80 msgr2=0x7f3ce019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ce0105d80 0x7f3ce019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 -- 172.21.15.114:0/3146313228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ce010c010 msgr2=0x7f3ce01a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ce010c010 0x7f3ce01a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 -- 172.21.15.114:0/3146313228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce01a2640 con 0x7f3ce010b6c0 2024-09-21T06:50:51.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.145+0000 7f3cd7400640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 0x7f3ce019cc70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800e9f0 tx=0x7f3cc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:51.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.146+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800ccc0 con 0x7f3ce010b6c0 2024-09-21T06:50:51.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.146+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cc80108e0 con 0x7f3ce010b6c0 2024-09-21T06:50:51.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.146+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce01a2930 con 0x7f3ce010b6c0 2024-09-21T06:50:51.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.146+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8018860 con 0x7f3ce010b6c0 2024-09-21T06:50:51.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.146+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce01a9090 con 0x7f3ce010b6c0 2024-09-21T06:50:51.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.147+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0106190 con 0x7f3ce010b6c0 2024-09-21T06:50:51.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.149+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f3cc8010430 con 0x7f3ce010b6c0 2024-09-21T06:50:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.151+0000 7f3cdd600640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3cc40b6560 0x7f3cc40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.152+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8194+0+0 (secure 0 0 0) 0x7f3cc80e7f00 con 0x7f3ce010b6c0 2024-09-21T06:50:51.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.152+0000 7f3cdfe00640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3cc40b6560 0x7f3cc40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:51.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.152+0000 7f3cdfe00640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3cc40b6560 0x7f3cc40b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0005ab0 tx=0x7f3cd0005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:51.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.153+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3cc80ae7b0 con 0x7f3ce010b6c0 2024-09-21T06:50:51.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.350+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f3ce019d800 con 0x7f3ce010b6c0 2024-09-21T06:50:51.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.403+0000 7f3cdd600640 1 -- 172.21.15.114:0/3146313228 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f3ce019d800 con 0x7f3ce010b6c0 2024-09-21T06:50:51.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3cc40b6560 msgr2=0x7f3cc40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3cc40b6560 0x7f3cc40b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0005ab0 tx=0x7f3cd0005a20 comp rx=0 tx=0).stop 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 msgr2=0x7f3ce019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 0x7f3ce019cc70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800e9f0 tx=0x7f3cc800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 shutdown_connections 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3cc40b6560 0x7f3cc40b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ce010c010 0x7f3ce01a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ce010b6c0 0x7f3ce019cc70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 --2- 172.21.15.114:0/3146313228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ce0105d80 0x7f3ce019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 >> 172.21.15.114:0/3146313228 conn(0x7f3ce0101810 msgr2=0x7f3ce010ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 shutdown_connections 2024-09-21T06:50:51.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:51.406+0000 7f3ce7200640 1 -- 172.21.15.114:0/3146313228 wait complete. 2024-09-21T06:50:51.766 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3146313228' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T06:50:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3146313228' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T06:50:51.865 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:50:51 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: 2024-09-21T06:50:51.549+0000 7fba7d9e8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T06:50:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3146313228' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-21T06:50:52.403 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T06:50:52.786 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:52 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3146313228' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T06:50:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:52 smithi114 ceph-mon[30654]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:50:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:52 smithi114 ceph-mon[30654]: pgmap v288: 97 pgs: 75 active+clean, 22 creating+peering; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s wr, 2 op/s 2024-09-21T06:50:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:52 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3146313228' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T06:50:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:52 smithi183 ceph-mon[34219]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:50:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:52 smithi183 ceph-mon[34219]: pgmap v288: 97 pgs: 75 active+clean, 22 creating+peering; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s wr, 2 op/s 2024-09-21T06:50:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:52 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3146313228' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-21T06:50:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:52 smithi193 ceph-mon[34601]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:50:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:52 smithi193 ceph-mon[34601]: pgmap v288: 97 pgs: 75 active+clean, 22 creating+peering; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 716 B/s wr, 2 op/s 2024-09-21T06:50:53.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.402+0000 7fbf08e00640 1 -- 172.21.15.114:0/1380334895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 msgr2=0x7fbf04077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.402+0000 7fbf08e00640 1 --2- 172.21.15.114:0/1380334895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf04077d50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402a410 comp rx=0 tx=0).stop 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 -- 172.21.15.114:0/1380334895 shutdown_connections 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 --2- 172.21.15.114:0/1380334895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04114780 0x7fbf04116ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 --2- 172.21.15.114:0/1380334895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04078290 0x7fbf04114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 --2- 172.21.15.114:0/1380334895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf04077d50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 -- 172.21.15.114:0/1380334895 >> 172.21.15.114:0/1380334895 conn(0x7fbf041017d0 msgr2=0x7fbf04103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 -- 172.21.15.114:0/1380334895 shutdown_connections 2024-09-21T06:50:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.403+0000 7fbf08e00640 1 -- 172.21.15.114:0/1380334895 wait complete. 2024-09-21T06:50:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 Processor -- start 2024-09-21T06:50:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 -- start start 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf041a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04078290 0x7fbf041a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04114780 0x7fbf041a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0411a110 con 0x7fbf04077940 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04119f90 con 0x7fbf04078290 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf08e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0411a290 con 0x7fbf04114780 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.404+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf041a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf041a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42852/0 (socket says 172.21.15.114:42852) 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01e00640 1 -- 172.21.15.114:0/538273599 learned_addr learned my addr 172.21.15.114:0/538273599 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf02800640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04114780 0x7fbf041a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04078290 0x7fbf041a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 -- 172.21.15.114:0/538273599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04114780 msgr2=0x7fbf041a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:53.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04114780 0x7fbf041a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:53.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 -- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 msgr2=0x7fbf041a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:53.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf041a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:53.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 -- 172.21.15.114:0/538273599 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041a6a30 con 0x7fbf04078290 2024-09-21T06:50:53.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf041a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:53.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.405+0000 7fbf01400640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04078290 0x7fbf041a1060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf0406c6b0 tx=0x7fbef000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:53.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.406+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef000cc40 con 0x7fbf04078290 2024-09-21T06:50:53.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.406+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf041a6d20 con 0x7fbf04078290 2024-09-21T06:50:53.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.406+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef000cde0 con 0x7fbf04078290 2024-09-21T06:50:53.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.406+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0004a00 con 0x7fbf04078290 2024-09-21T06:50:53.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.406+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041ad480 con 0x7fbf04078290 2024-09-21T06:50:53.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.407+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf04077d50 con 0x7fbf04078290 2024-09-21T06:50:53.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.409+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fbef0004ba0 con 0x7fbf04078290 2024-09-21T06:50:53.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.411+0000 7fbefaa00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee80b6560 0x7fbee80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:53.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.411+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fbef00e74c0 con 0x7fbf04078290 2024-09-21T06:50:53.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.411+0000 7fbf01e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee80b6560 0x7fbee80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:53.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.412+0000 7fbf01e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee80b6560 0x7fbee80b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef4031440 tx=0x7fbef40058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:53.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.413+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbef00adc80 con 0x7fbf04078290 2024-09-21T06:50:53.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:53.604+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbf041a1bf0 con 0x7fbf04078290 2024-09-21T06:50:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:54 smithi114 ceph-mon[30654]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-21T06:50:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:54 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/538273599' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:50:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:54 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:50:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:54 smithi183 ceph-mon[34219]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-21T06:50:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:54 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/538273599' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:50:54.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:54 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:50:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:54 smithi193 ceph-mon[34601]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-21T06:50:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:54 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/538273599' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:50:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:54 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T06:50:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.550+0000 7fbefaa00640 1 -- 172.21.15.114:0/538273599 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7fbf041a1bf0 con 0x7fbf04078290 2024-09-21T06:50:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data_ec' created 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.556+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee80b6560 msgr2=0x7fbee80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.556+0000 7fbf08e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee80b6560 0x7fbee80b8a70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef4031440 tx=0x7fbef40058d0 comp rx=0 tx=0).stop 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.556+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04078290 msgr2=0x7fbf041a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.556+0000 7fbf08e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04078290 0x7fbf041a1060 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbf0406c6b0 tx=0x7fbef000ee50 comp rx=0 tx=0).stop 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 shutdown_connections 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf04114780 0x7fbf041a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee80b6560 0x7fbee80b8a70 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbef4031440 tx=0x7fbef40058d0 comp rx=0 tx=0).stop 2024-09-21T06:50:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf04078290 0x7fbf041a1060 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 --2- 172.21.15.114:0/538273599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf04077940 0x7fbf041a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 >> 172.21.15.114:0/538273599 conn(0x7fbf041017d0 msgr2=0x7fbf04103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 shutdown_connections 2024-09-21T06:50:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:55.557+0000 7fbf08e00640 1 -- 172.21.15.114:0/538273599 wait complete. 2024-09-21T06:50:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:55 smithi114 ceph-mon[30654]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:50:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:55 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:50:55.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:55 smithi114 ceph-mon[30654]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:50:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:55 smithi183 ceph-mon[34219]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:50:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:55 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:50:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:55 smithi183 ceph-mon[34219]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:50:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:55 smithi193 ceph-mon[34601]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:50:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:55 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T06:50:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:55 smithi193 ceph-mon[34601]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:50:56.586 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T06:50:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:56 smithi114 ceph-mon[30654]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-21T06:50:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:56 smithi114 ceph-mon[30654]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:50:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:56 smithi183 ceph-mon[34219]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-21T06:50:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:56 smithi183 ceph-mon[34219]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:50:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:56 smithi193 ceph-mon[34601]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-21T06:50:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:56 smithi193 ceph-mon[34601]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:50:56.960 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:57.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 -- 172.21.15.114:0/2037679481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10c020 msgr2=0x7f5f9c112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:57.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 --2- 172.21.15.114:0/2037679481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10c020 0x7f5f9c112850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009ca0 tx=0x7f5f9402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:50:57.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 -- 172.21.15.114:0/2037679481 shutdown_connections 2024-09-21T06:50:57.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 --2- 172.21.15.114:0/2037679481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10c020 0x7f5f9c112850 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:57.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 --2- 172.21.15.114:0/2037679481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c10b6d0 0x7f5f9c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:57.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 --2- 172.21.15.114:0/2037679481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:57.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 -- 172.21.15.114:0/2037679481 >> 172.21.15.114:0/2037679481 conn(0x7f5f9c101860 msgr2=0x7f5f9c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:57.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 -- 172.21.15.114:0/2037679481 shutdown_connections 2024-09-21T06:50:57.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.617+0000 7f5fa3800640 1 -- 172.21.15.114:0/2037679481 wait complete. 2024-09-21T06:50:57.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.618+0000 7f5fa3800640 1 Processor -- start 2024-09-21T06:50:57.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.618+0000 7f5fa3800640 1 -- start start 2024-09-21T06:50:57.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa3800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:57.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa3800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10b6d0 0x7f5f9c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa3800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c10c020 0x7f5f9c1a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa3800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c115ce0 con 0x7f5f9c10b6d0 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa3800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c115b60 con 0x7f5f9c10c020 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa3800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c115e60 con 0x7f5f9c105d90 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60868/0 (socket says 172.21.15.114:60868) 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5f9be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10b6d0 0x7f5f9c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa0e00640 1 -- 172.21.15.114:0/1463149582 learned_addr learned my addr 172.21.15.114:0/1463149582 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.619+0000 7f5fa1800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c10c020 0x7f5f9c1a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa0e00640 1 -- 172.21.15.114:0/1463149582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c10c020 msgr2=0x7f5f9c1a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa0e00640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c10c020 0x7f5f9c1a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa0e00640 1 -- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10b6d0 msgr2=0x7f5f9c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:57.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa0e00640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10b6d0 0x7f5f9c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa0e00640 1 -- 172.21.15.114:0/1463149582 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c1a26e0 con 0x7f5f9c105d90 2024-09-21T06:50:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5f9be00640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10b6d0 0x7f5f9c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:50:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa0e00640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c19c770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e9f0 tx=0x7f5f9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000cd80 con 0x7f5f9c105d90 2024-09-21T06:50:57.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c1a29d0 con 0x7f5f9c105d90 2024-09-21T06:50:57.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f900108e0 con 0x7f5f9c105d90 2024-09-21T06:50:57.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.620+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90018980 con 0x7f5f9c105d90 2024-09-21T06:50:57.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.621+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c1a9130 con 0x7f5f9c105d90 2024-09-21T06:50:57.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.622+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9c1061a0 con 0x7f5f9c105d90 2024-09-21T06:50:57.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.624+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f5f90010430 con 0x7f5f9c105d90 2024-09-21T06:50:57.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.625+0000 7f5f99600640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5f840b6560 0x7f5f840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:50:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.625+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8985+0+0 (secure 0 0 0) 0x7f5f900e7e60 con 0x7f5f9c105d90 2024-09-21T06:50:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.625+0000 7f5f9be00640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5f840b6560 0x7f5f840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:50:57.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.626+0000 7f5f9be00640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5f840b6560 0x7f5f840b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c104f60 tx=0x7f5f88005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:50:57.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.627+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f900b2780 con 0x7f5f9c105d90 2024-09-21T06:50:57.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:57.822+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5f9c19d8a0 con 0x7f5f9c105d90 2024-09-21T06:50:58.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.581+0000 7f5f99600640 1 -- 172.21.15.114:0/1463149582 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f5f900b76c0 con 0x7f5f9c105d90 2024-09-21T06:50:58.583 INFO:teuthology.orchestra.run.smithi114.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T06:50:58.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.583+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5f840b6560 msgr2=0x7f5f840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:58.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.583+0000 7f5fa3800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5f840b6560 0x7f5f840b8a70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c104f60 tx=0x7f5f88005f90 comp rx=0 tx=0).stop 2024-09-21T06:50:58.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.583+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 msgr2=0x7f5f9c19c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:50:58.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.583+0000 7f5fa3800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c19c770 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e9f0 tx=0x7f5f9000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:50:58.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 shutdown_connections 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5f840b6560 0x7f5f840b8a70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f9c10c020 0x7f5f9c1a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5f9c10b6d0 0x7f5f9c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 --2- 172.21.15.114:0/1463149582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f9c105d90 0x7f5f9c19c770 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 >> 172.21.15.114:0/1463149582 conn(0x7f5f9c101860 msgr2=0x7f5f9c108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 shutdown_connections 2024-09-21T06:50:58.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:50:58.584+0000 7f5fa3800640 1 -- 172.21.15.114:0/1463149582 wait complete. 2024-09-21T06:50:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:58 smithi114 ceph-mon[30654]: pgmap v294: 129 pgs: 17 creating+peering, 14 unknown, 98 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T06:50:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:58 smithi114 ceph-mon[30654]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:50:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:58 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1463149582' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:50:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:58 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:50:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:58 smithi183 ceph-mon[34219]: pgmap v294: 129 pgs: 17 creating+peering, 14 unknown, 98 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T06:50:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:58 smithi183 ceph-mon[34219]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:50:58.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:58 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1463149582' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:50:58.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:58 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:50:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:58 smithi193 ceph-mon[34601]: pgmap v294: 129 pgs: 17 creating+peering, 14 unknown, 98 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T06:50:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:58 smithi193 ceph-mon[34601]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:50:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:58 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1463149582' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:50:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:58 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-21T06:50:59.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T06:50:59.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:59 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T06:50:59.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:50:59 smithi114 ceph-mon[30654]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:50:59.816 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:50:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:59 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T06:50:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:50:59 smithi183 ceph-mon[34219]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:50:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:59 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-21T06:50:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:50:59 smithi193 ceph-mon[34601]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 -- 172.21.15.114:0/666508794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 msgr2=0x7f447c110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 --2- 172.21.15.114:0/666508794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c110450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4474009ca0 tx=0x7f447402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 -- 172.21.15.114:0/666508794 shutdown_connections 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 --2- 172.21.15.114:0/666508794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c110450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 --2- 172.21.15.114:0/666508794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c1092d0 0x7f447c1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 --2- 172.21.15.114:0/666508794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f447c103990 0x7f447c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:00.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 -- 172.21.15.114:0/666508794 >> 172.21.15.114:0/666508794 conn(0x7f447c0ff3e0 msgr2=0x7f447c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:00.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 -- 172.21.15.114:0/666508794 shutdown_connections 2024-09-21T06:51:00.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.421+0000 7f4482600640 1 -- 172.21.15.114:0/666508794 wait complete. 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.422+0000 7f4482600640 1 Processor -- start 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.422+0000 7f4482600640 1 -- start start 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4482600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 0x7f447c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4482600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f447c1092d0 0x7f447c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4482600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4482600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1138e0 con 0x7f447c109c20 2024-09-21T06:51:00.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4482600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f447c113760 con 0x7f447c1092d0 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4482600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f447c113a60 con 0x7f447c103990 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 0x7f447c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 0x7f447c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60892/0 (socket says 172.21.15.114:60892) 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 -- 172.21.15.114:0/3655260270 learned_addr learned my addr 172.21.15.114:0/3655260270 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c19fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50766/0 (socket says 172.21.15.114:50766) 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4473e00640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f447c1092d0 0x7f447c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 -- 172.21.15.114:0/3655260270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f447c1092d0 msgr2=0x7f447c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f447c1092d0 0x7f447c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 -- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 msgr2=0x7f447c19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.423+0000 7f4481200640 1 -- 172.21.15.114:0/3655260270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1a0240 con 0x7f447c103990 2024-09-21T06:51:00.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.424+0000 7f4481200640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 0x7f447c19a2d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f446c00d950 tx=0x7f446c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:00.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.424+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c017030 con 0x7f447c103990 2024-09-21T06:51:00.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.424+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1a0530 con 0x7f447c103990 2024-09-21T06:51:00.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.424+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f446c004020 con 0x7f447c103990 2024-09-21T06:51:00.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.424+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0047a0 con 0x7f447c103990 2024-09-21T06:51:00.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.424+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a6c90 con 0x7f447c103990 2024-09-21T06:51:00.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.425+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f447c103da0 con 0x7f447c103990 2024-09-21T06:51:00.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.428+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f446c009a30 con 0x7f447c103990 2024-09-21T06:51:00.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.429+0000 7f447a000640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44600b6560 0x7f44600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:00.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.429+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8985+0+0 (secure 0 0 0) 0x7f446c0e77f0 con 0x7f447c103990 2024-09-21T06:51:00.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.430+0000 7f4473e00640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44600b6560 0x7f44600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:00.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.430+0000 7f4473e00640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44600b6560 0x7f44600b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f447c108dd0 tx=0x7f4464005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:00.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.431+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f446c0add30 con 0x7f447c103990 2024-09-21T06:51:00.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:00.664+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f447c19b400 con 0x7f447c103990 2024-09-21T06:51:00.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:00 smithi183 ceph-mon[34219]: pgmap v297: 129 pgs: 17 creating+peering, 112 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:00 smithi193 ceph-mon[34601]: pgmap v297: 129 pgs: 17 creating+peering, 112 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:01.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:00 smithi114 ceph-mon[30654]: pgmap v297: 129 pgs: 17 creating+peering, 112 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:01.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.600+0000 7f447a000640 1 -- 172.21.15.114:0/3655260270 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f446c0b2110 con 0x7f447c103990 2024-09-21T06:51:01.602 INFO:teuthology.orchestra.run.smithi114.stderr:added data pool 4 to fsmap 2024-09-21T06:51:01.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.603+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44600b6560 msgr2=0x7f44600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:01.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44600b6560 0x7f44600b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f447c108dd0 tx=0x7f4464005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:01.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 msgr2=0x7f447c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:01.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 0x7f447c19a2d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f446c00d950 tx=0x7f446c00de10 comp rx=0 tx=0).stop 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 shutdown_connections 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f447c109c20 0x7f447c19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44600b6560 0x7f44600b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f447c1092d0 0x7f447c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 --2- 172.21.15.114:0/3655260270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f447c103990 0x7f447c19a2d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 >> 172.21.15.114:0/3655260270 conn(0x7f447c0ff3e0 msgr2=0x7f447c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 shutdown_connections 2024-09-21T06:51:01.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:01.604+0000 7f4482600640 1 -- 172.21.15.114:0/3655260270 wait complete. 2024-09-21T06:51:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:01 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3655260270' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:51:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:01 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:51:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:01 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3655260270' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:51:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:01 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:51:01.968 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:01 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3655260270' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:51:01.968 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:01 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-21T06:51:02.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:51:02.764 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:02 smithi183 ceph-mon[34219]: pgmap v298: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:02 smithi183 ceph-mon[34219]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:51:02.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:02 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T06:51:02.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:02 smithi183 ceph-mon[34219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:02.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:02 smithi114 ceph-mon[30654]: pgmap v298: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:02.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:02 smithi114 ceph-mon[30654]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:51:02.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:02 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T06:51:02.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:02 smithi114 ceph-mon[30654]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:02 smithi193 ceph-mon[34601]: pgmap v298: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:02 smithi193 ceph-mon[34601]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:51:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:02 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-21T06:51:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:02 smithi193 ceph-mon[34601]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:03.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 -- 172.21.15.114:0/2715380180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b381039b0 msgr2=0x7f6b38103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:03.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 --2- 172.21.15.114:0/2715380180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b381039b0 0x7f6b38103dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009900 tx=0x7f6b2802a410 comp rx=0 tx=0).stop 2024-09-21T06:51:03.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 -- 172.21.15.114:0/2715380180 shutdown_connections 2024-09-21T06:51:03.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 --2- 172.21.15.114:0/2715380180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b38109c40 0x7f6b38110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 --2- 172.21.15.114:0/2715380180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381092f0 0x7f6b38109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 --2- 172.21.15.114:0/2715380180 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b381039b0 0x7f6b38103dc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 -- 172.21.15.114:0/2715380180 >> 172.21.15.114:0/2715380180 conn(0x7f6b380ff440 msgr2=0x7f6b38101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:03.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 -- 172.21.15.114:0/2715380180 shutdown_connections 2024-09-21T06:51:03.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.416+0000 7f6b3d400640 1 -- 172.21.15.114:0/2715380180 wait complete. 2024-09-21T06:51:03.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.417+0000 7f6b3d400640 1 Processor -- start 2024-09-21T06:51:03.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.417+0000 7f6b3d400640 1 -- start start 2024-09-21T06:51:03.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381039b0 0x7f6b3819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:03.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381092f0 0x7f6b3819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:03.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b38109c40 0x7f6b3819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:03.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38113900 con 0x7f6b38109c40 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38113780 con 0x7f6b381039b0 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6b38113a80 con 0x7f6b381092f0 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381092f0 0x7f6b3819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381039b0 0x7f6b3819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381092f0 0x7f6b3819a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60912/0 (socket says 172.21.15.114:60912) 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b2fe00640 1 -- 172.21.15.114:0/1687702308 learned_addr learned my addr 172.21.15.114:0/1687702308 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.418+0000 7f6b3ca00640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b38109c40 0x7f6b3819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b37e00640 1 -- 172.21.15.114:0/1687702308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381092f0 msgr2=0x7f6b3819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b37e00640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381092f0 0x7f6b3819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b37e00640 1 -- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b38109c40 msgr2=0x7f6b3819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b37e00640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b38109c40 0x7f6b3819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b37e00640 1 -- 172.21.15.114:0/1687702308 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b381a02a0 con 0x7f6b381039b0 2024-09-21T06:51:03.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b37e00640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381039b0 0x7f6b3819a330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b2802a930 tx=0x7f6b28031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:03.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2803f030 con 0x7f6b381039b0 2024-09-21T06:51:03.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b381a0530 con 0x7f6b381039b0 2024-09-21T06:51:03.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b28004490 con 0x7f6b381039b0 2024-09-21T06:51:03.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.419+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b28035070 con 0x7f6b381039b0 2024-09-21T06:51:03.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.420+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b381a6cf0 con 0x7f6b381039b0 2024-09-21T06:51:03.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.421+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b38103dc0 con 0x7f6b381039b0 2024-09-21T06:51:03.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.425+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f6b28042030 con 0x7f6b381039b0 2024-09-21T06:51:03.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.426+0000 7f6b35600640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6b1c0b65e0 0x7f6b1c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:03.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.427+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f6b28107ad0 con 0x7f6b381039b0 2024-09-21T06:51:03.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.427+0000 7f6b2fe00640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6b1c0b65e0 0x7f6b1c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:03.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.427+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b280d23d0 con 0x7f6b381039b0 2024-09-21T06:51:03.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.427+0000 7f6b2fe00640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6b1c0b65e0 0x7f6b1c0b8af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b38108ea0 tx=0x7f6b20005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:03.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.680+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b3819b460 con 0x7f6b381039b0 2024-09-21T06:51:03.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.681+0000 7f6b35600640 1 -- 172.21.15.114:0/1687702308 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11854 (secure 0 0 0) 0x7f6b280d7310 con 0x7f6b381039b0 2024-09-21T06:51:03.683 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:03.683 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":16,"btime":"2024-09-21T06:51:01:585514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14742,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24725,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24742,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24760,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:00.665998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24761,"qdb_cluster":[24761]},"id":1}]} 2024-09-21T06:51:03.683 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 16 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6b1c0b65e0 msgr2=0x7f6b1c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6b1c0b65e0 0x7f6b1c0b8af0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6b38108ea0 tx=0x7f6b20005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381039b0 msgr2=0x7f6b3819a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381039b0 0x7f6b3819a330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b2802a930 tx=0x7f6b28031a20 comp rx=0 tx=0).stop 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 shutdown_connections 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6b38109c40 0x7f6b3819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b381092f0 0x7f6b3819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6b1c0b65e0 0x7f6b1c0b8af0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 --2- 172.21.15.114:0/1687702308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b381039b0 0x7f6b3819a330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:03.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 >> 172.21.15.114:0/1687702308 conn(0x7f6b380ff440 msgr2=0x7f6b38101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:03.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 shutdown_connections 2024-09-21T06:51:03.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:03.684+0000 7f6b3d400640 1 -- 172.21.15.114:0/1687702308 wait complete. 2024-09-21T06:51:04.633 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:51:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:04 smithi114 ceph-mon[30654]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:04 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1687702308' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:04 smithi183 ceph-mon[34219]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:04 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1687702308' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:04.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:04 smithi193 ceph-mon[34601]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:04 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1687702308' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:05.004 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:05.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 -- 172.21.15.114:0/457255084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc1039b0 msgr2=0x7f94dc103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 --2- 172.21.15.114:0/457255084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc1039b0 0x7f94dc103dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009900 tx=0x7f94cc02a410 comp rx=0 tx=0).stop 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 -- 172.21.15.114:0/457255084 shutdown_connections 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 --2- 172.21.15.114:0/457255084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc109c40 0x7f94dc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 --2- 172.21.15.114:0/457255084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc1092f0 0x7f94dc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 --2- 172.21.15.114:0/457255084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc1039b0 0x7f94dc103dc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 -- 172.21.15.114:0/457255084 >> 172.21.15.114:0/457255084 conn(0x7f94dc0ff400 msgr2=0x7f94dc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.622+0000 7f94db200640 1 -- 172.21.15.114:0/457255084 shutdown_connections 2024-09-21T06:51:05.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.623+0000 7f94db200640 1 -- 172.21.15.114:0/457255084 wait complete. 2024-09-21T06:51:05.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 Processor -- start 2024-09-21T06:51:05.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 -- start start 2024-09-21T06:51:05.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc1039b0 0x7f94dc19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc1092f0 0x7f94dc19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 0x7f94dc1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc113940 con 0x7f94dc109c40 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc1137c0 con 0x7f94dc1039b0 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.624+0000 7f94db200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc113ac0 con 0x7f94dc1092f0 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 0x7f94dc1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94d3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc1092f0 0x7f94dc19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 0x7f94dc1a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46440/0 (socket says 172.21.15.114:46440) 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 -- 172.21.15.114:0/3730301826 learned_addr learned my addr 172.21.15.114:0/3730301826 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94d9e00640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc1039b0 0x7f94dc19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 -- 172.21.15.114:0/3730301826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc1092f0 msgr2=0x7f94dc19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:05.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc1092f0 0x7f94dc19cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 -- 172.21.15.114:0/3730301826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc1039b0 msgr2=0x7f94dc19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc1039b0 0x7f94dc19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 -- 172.21.15.114:0/3730301826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc1a24e0 con 0x7f94dc109c40 2024-09-21T06:51:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94d9e00640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc1039b0 0x7f94dc19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:51:05.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.625+0000 7f94da800640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 0x7f94dc1a1db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94d400eb10 tx=0x7f94d400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.626+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d400ce20 con 0x7f94dc109c40 2024-09-21T06:51:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.626+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc1a27d0 con 0x7f94dc109c40 2024-09-21T06:51:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.626+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94d4005020 con 0x7f94dc109c40 2024-09-21T06:51:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.626+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4010690 con 0x7f94dc109c40 2024-09-21T06:51:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.626+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1a8f30 con 0x7f94dc109c40 2024-09-21T06:51:05.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.627+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc103dc0 con 0x7f94dc109c40 2024-09-21T06:51:05.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.630+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f94d4005510 con 0x7f94dc109c40 2024-09-21T06:51:05.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.631+0000 7f94d2a00640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94c00b65e0 0x7f94c00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.631+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f94d40e7e20 con 0x7f94dc109c40 2024-09-21T06:51:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.631+0000 7f94d9e00640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94c00b65e0 0x7f94c00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:05.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.632+0000 7f94d9e00640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94c00b65e0 0x7f94c00b8af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94cc031440 tx=0x7f94cc032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:05.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.634+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94d4056710 con 0x7f94dc109c40 2024-09-21T06:51:05.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.824+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94dc1a9270 con 0x7f94dc109c40 2024-09-21T06:51:05.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.825+0000 7f94d2a00640 1 -- 172.21.15.114:0/3730301826 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24592 (secure 0 0 0) 0x7f94d40ae340 con 0x7f94dc109c40 2024-09-21T06:51:05.827 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:05.828 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":85,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:00.665846+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94c00b65e0 msgr2=0x7f94c00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94c00b65e0 0x7f94c00b8af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f94cc031440 tx=0x7f94cc032ed0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 msgr2=0x7f94dc1a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 0x7f94dc1a1db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94d400eb10 tx=0x7f94d400ef10 comp rx=0 tx=0).stop 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 shutdown_connections 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f94dc109c40 0x7f94dc1a1db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94dc1092f0 0x7f94dc19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94c00b65e0 0x7f94c00b8af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 --2- 172.21.15.114:0/3730301826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94dc1039b0 0x7f94dc19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:05.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 >> 172.21.15.114:0/3730301826 conn(0x7f94dc0ff400 msgr2=0x7f94dc1089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:05.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 shutdown_connections 2024-09-21T06:51:05.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:05.828+0000 7f94db200640 1 -- 172.21.15.114:0/3730301826 wait complete. 2024-09-21T06:51:06.810 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:51:06.858 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:06 smithi114 ceph-mon[30654]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:06.858 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:06 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3730301826' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:06 smithi183 ceph-mon[34219]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:06 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3730301826' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:06 smithi193 ceph-mon[34601]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:06 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3730301826' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:07.174 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 -- 172.21.15.114:0/4125428284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 msgr2=0x7fe248112370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 --2- 172.21.15.114:0/4125428284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 0x7fe248112370 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe240009ca0 tx=0x7fe24002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 -- 172.21.15.114:0/4125428284 shutdown_connections 2024-09-21T06:51:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 --2- 172.21.15.114:0/4125428284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 0x7fe248112370 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:07.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 --2- 172.21.15.114:0/4125428284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2480760a0 0x7fe24810fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:07.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 --2- 172.21.15.114:0/4125428284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe248075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:07.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 -- 172.21.15.114:0/4125428284 >> 172.21.15.114:0/4125428284 conn(0x7fe2480ff3e0 msgr2=0x7fe248101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:07.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 -- 172.21.15.114:0/4125428284 shutdown_connections 2024-09-21T06:51:07.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.806+0000 7fe247e00640 1 -- 172.21.15.114:0/4125428284 wait complete. 2024-09-21T06:51:07.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.807+0000 7fe247e00640 1 Processor -- start 2024-09-21T06:51:07.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.807+0000 7fe247e00640 1 -- start start 2024-09-21T06:51:07.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe24819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:07.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2480760a0 0x7fe24819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 0x7fe2481a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe248115b10 con 0x7fe24810ff50 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe248115990 con 0x7fe2480760a0 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe248115c90 con 0x7fe248075750 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe24819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe24819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48454/0 (socket says 172.21.15.114:48454) 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 -- 172.21.15.114:0/3081823260 learned_addr learned my addr 172.21.15.114:0/3081823260 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe247400640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 0x7fe2481a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246000640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2480760a0 0x7fe24819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 -- 172.21.15.114:0/3081823260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2480760a0 msgr2=0x7fe24819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2480760a0 0x7fe24819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 -- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 msgr2=0x7fe2481a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 0x7fe2481a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.808+0000 7fe246a00640 1 -- 172.21.15.114:0/3081823260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2481a23d0 con 0x7fe248075750 2024-09-21T06:51:07.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.809+0000 7fe246a00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe24819c460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e9f0 tx=0x7fe23800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:07.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.809+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23800cd80 con 0x7fe248075750 2024-09-21T06:51:07.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.809+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2380054d0 con 0x7fe248075750 2024-09-21T06:51:07.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.809+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2481a26c0 con 0x7fe248075750 2024-09-21T06:51:07.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.809+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238010690 con 0x7fe248075750 2024-09-21T06:51:07.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.809+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2481a8e20 con 0x7fe248075750 2024-09-21T06:51:07.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.810+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe248075b60 con 0x7fe248075750 2024-09-21T06:51:07.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.813+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7fe238004e30 con 0x7fe248075750 2024-09-21T06:51:07.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.814+0000 7fe23f400640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe22c0b6560 0x7fe22c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:07.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.814+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7fe2380e7c90 con 0x7fe248075750 2024-09-21T06:51:07.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.815+0000 7fe246000640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe22c0b6560 0x7fe22c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:07.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.815+0000 7fe246000640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe22c0b6560 0x7fe22c0b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe248112340 tx=0x7fe234005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:07.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:07.816+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2380ae1b0 con 0x7fe248075750 2024-09-21T06:51:08.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.008+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe24819d590 con 0x7fe248075750 2024-09-21T06:51:08.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.010+0000 7fe23f400640 1 -- 172.21.15.114:0/3081823260 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24592 (secure 0 0 0) 0x7fe2380a9030 con 0x7fe248075750 2024-09-21T06:51:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:08.012 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":85,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:00.665846+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe22c0b6560 msgr2=0x7fe22c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe22c0b6560 0x7fe22c0b8a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe248112340 tx=0x7fe234005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 msgr2=0x7fe24819c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe24819c460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe23800e9f0 tx=0x7fe23800edf0 comp rx=0 tx=0).stop 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 shutdown_connections 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe24810ff50 0x7fe2481a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:08.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe22c0b6560 0x7fe22c0b8a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:08.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2480760a0 0x7fe24819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:08.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 --2- 172.21.15.114:0/3081823260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe248075750 0x7fe24819c460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:08.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 >> 172.21.15.114:0/3081823260 conn(0x7fe2480ff3e0 msgr2=0x7fe248107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:08.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 shutdown_connections 2024-09-21T06:51:08.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:08.014+0000 7fe247e00640 1 -- 172.21.15.114:0/3081823260 wait complete. 2024-09-21T06:51:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:08 smithi114 ceph-mon[30654]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:08 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3081823260' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:08 smithi183 ceph-mon[34219]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:08 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3081823260' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:08 smithi193 ceph-mon[34601]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:08 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3081823260' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:08.952 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:51:09.318 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 -- 172.21.15.114:0/2946106084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc10dc20 msgr2=0x7febbc110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 --2- 172.21.15.114:0/2946106084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc10dc20 0x7febbc110110 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febb4009ca0 tx=0x7febb402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 -- 172.21.15.114:0/2946106084 shutdown_connections 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 --2- 172.21.15.114:0/2946106084 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc10dc20 0x7febbc110110 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 --2- 172.21.15.114:0/2946106084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc105110 0x7febbc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 --2- 172.21.15.114:0/2946106084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc1047c0 0x7febbc104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 -- 172.21.15.114:0/2946106084 >> 172.21.15.114:0/2946106084 conn(0x7febbc0fe2c0 msgr2=0x7febbc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 -- 172.21.15.114:0/2946106084 shutdown_connections 2024-09-21T06:51:09.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.943+0000 7febc1200640 1 -- 172.21.15.114:0/2946106084 wait complete. 2024-09-21T06:51:09.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.944+0000 7febc1200640 1 Processor -- start 2024-09-21T06:51:09.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.944+0000 7febc1200640 1 -- start start 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febc1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 0x7febbc19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febc1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc105110 0x7febbc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febc1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc10dc20 0x7febbc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febc1200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febbc113880 con 0x7febbc105110 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febc1200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febbc113700 con 0x7febbc1047c0 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febc1200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febbc113a00 con 0x7febbc10dc20 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 0x7febbc19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbaa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc105110 0x7febbc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 0x7febbc19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52494/0 (socket says 172.21.15.114:52494) 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 -- 172.21.15.114:0/2508672765 learned_addr learned my addr 172.21.15.114:0/2508672765 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:09.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbbe00640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc10dc20 0x7febbc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:09.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 -- 172.21.15.114:0/2508672765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc10dc20 msgr2=0x7febbc1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:09.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc10dc20 0x7febbc1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:09.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 -- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc105110 msgr2=0x7febbc19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc105110 0x7febbc19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.945+0000 7febbb400640 1 -- 172.21.15.114:0/2508672765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc1a23c0 con 0x7febbc1047c0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.946+0000 7febbb400640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 0x7febbc19c450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febac00e9f0 tx=0x7febac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.946+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac00cd80 con 0x7febbc1047c0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.946+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc1a26b0 con 0x7febbc1047c0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.946+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac0108e0 con 0x7febbc1047c0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.946+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac018920 con 0x7febbc1047c0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.946+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a8e10 con 0x7febbc1047c0 2024-09-21T06:51:09.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.947+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc104bd0 con 0x7febbc1047c0 2024-09-21T06:51:09.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.949+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7febac010430 con 0x7febbc1047c0 2024-09-21T06:51:09.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.951+0000 7febb3e00640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba00b6560 0x7feba00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:09.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.951+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7febac0e7990 con 0x7febbc1047c0 2024-09-21T06:51:09.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.951+0000 7febbaa00640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba00b6560 0x7feba00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:09.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.952+0000 7febbaa00640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba00b6560 0x7feba00b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7febbc101aa0 tx=0x7feba8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:09.955+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7febac0b2290 con 0x7febbc1047c0 2024-09-21T06:51:10.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.184+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febbc1a9180 con 0x7febbc1047c0 2024-09-21T06:51:10.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.185+0000 7febb3e00640 1 -- 172.21.15.114:0/2508672765 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11854 (secure 0 0 0) 0x7febac0b71d0 con 0x7febbc1047c0 2024-09-21T06:51:10.187 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:10.187 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":16,"btime":"2024-09-21T06:51:01:585514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14742,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24667,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24668,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24704,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24715,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24725,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24727,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24742,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24760,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:00.665998+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24761,"qdb_cluster":[24761]},"id":1}]} 2024-09-21T06:51:10.187 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 16 2024-09-21T06:51:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba00b6560 msgr2=0x7feba00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:10.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba00b6560 0x7feba00b8a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7febbc101aa0 tx=0x7feba8005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 msgr2=0x7febbc19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 0x7febbc19c450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7febac00e9f0 tx=0x7febac00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 shutdown_connections 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febbc10dc20 0x7febbc1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febbc105110 0x7febbc19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba00b6560 0x7feba00b8a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 --2- 172.21.15.114:0/2508672765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc1047c0 0x7febbc19c450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 >> 172.21.15.114:0/2508672765 conn(0x7febbc0fe2c0 msgr2=0x7febbc1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 shutdown_connections 2024-09-21T06:51:10.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:10.188+0000 7febc1200640 1 -- 172.21.15.114:0/2508672765 wait complete. 2024-09-21T06:51:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:10 smithi183 ceph-mon[34219]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:10 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2508672765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:10.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:10 smithi193 ceph-mon[34601]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:10 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2508672765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:10.949 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:10 smithi114 ceph-mon[30654]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:10.950 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:10 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2508672765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:10.989 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:51:11.363 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:12.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 -- 172.21.15.114:0/3650948331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410be50 msgr2=0x7f25b4112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:12.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3650948331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410be50 0x7f25b4112680 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009ca0 tx=0x7f25ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 -- 172.21.15.114:0/3650948331 shutdown_connections 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3650948331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410be50 0x7f25b4112680 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3650948331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25b410b500 0x7f25b410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3650948331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4105bc0 0x7f25b4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.075+0000 7f25b9c00640 1 -- 172.21.15.114:0/3650948331 >> 172.21.15.114:0/3650948331 conn(0x7f25b4101670 msgr2=0x7f25b4103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.076+0000 7f25b9c00640 1 -- 172.21.15.114:0/3650948331 shutdown_connections 2024-09-21T06:51:12.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.076+0000 7f25b9c00640 1 -- 172.21.15.114:0/3650948331 wait complete. 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 Processor -- start 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 -- start start 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4105bc0 0x7f25b419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 0x7f25b419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25b410be50 0x7f25b41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4115b10 con 0x7f25b410b500 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4115990 con 0x7f25b410be50 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b9c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f25b4115c90 con 0x7f25b4105bc0 2024-09-21T06:51:12.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.077+0000 7f25b3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 0x7f25b419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 0x7f25b419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46488/0 (socket says 172.21.15.114:46488) 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 -- 172.21.15.114:0/3116993439 learned_addr learned my addr 172.21.15.114:0/3116993439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3e00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4105bc0 0x7f25b419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b9200640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25b410be50 0x7f25b41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 -- 172.21.15.114:0/3116993439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4105bc0 msgr2=0x7f25b419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4105bc0 0x7f25b419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 -- 172.21.15.114:0/3116993439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25b410be50 msgr2=0x7f25b41a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25b410be50 0x7f25b41a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 -- 172.21.15.114:0/3116993439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b41a2410 con 0x7f25b410b500 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.078+0000 7f25b3400640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 0x7f25b419ca40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25a000e970 tx=0x7f25a000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.079+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a000cd80 con 0x7f25b410b500 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.079+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25a00035f0 con 0x7f25b410b500 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.079+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b41a2700 con 0x7f25b410b500 2024-09-21T06:51:12.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.079+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0010690 con 0x7f25b410b500 2024-09-21T06:51:12.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.079+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b41a8e60 con 0x7f25b410b500 2024-09-21T06:51:12.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.080+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b4105fd0 con 0x7f25b410b500 2024-09-21T06:51:12.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.082+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f25a0003aa0 con 0x7f25b410b500 2024-09-21T06:51:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.084+0000 7f25b0c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f25980b65e0 0x7f25980b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.084+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f25a00e7980 con 0x7f25b410b500 2024-09-21T06:51:12.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.084+0000 7f25b3e00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f25980b65e0 0x7f25980b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:12.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.084+0000 7f25b3e00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f25980b65e0 0x7f25980b8af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009bd0 tx=0x7f25a4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:12.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.088+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25a00adea0 con 0x7f25b410b500 2024-09-21T06:51:12.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.278+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25b410d6b0 con 0x7f25b410b500 2024-09-21T06:51:12.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.280+0000 7f25b0c00640 1 -- 172.21.15.114:0/3116993439 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24592 (secure 0 0 0) 0x7f25a00b2280 con 0x7f25b410b500 2024-09-21T06:51:12.282 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:12.282 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":85,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:00.665846+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f25980b65e0 msgr2=0x7f25980b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f25980b65e0 0x7f25980b8af0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009bd0 tx=0x7f25a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 msgr2=0x7f25b419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 0x7f25b419ca40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25a000e970 tx=0x7f25a000ee30 comp rx=0 tx=0).stop 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 shutdown_connections 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f25980b65e0 0x7f25980b8af0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25b410be50 0x7f25b41a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f25b410b500 0x7f25b419ca40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 --2- 172.21.15.114:0/3116993439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b4105bc0 0x7f25b419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:12.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.284+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 >> 172.21.15.114:0/3116993439 conn(0x7f25b4101670 msgr2=0x7f25b4103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:12.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.285+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 shutdown_connections 2024-09-21T06:51:12.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:12.285+0000 7f25b9c00640 1 -- 172.21.15.114:0/3116993439 wait complete. 2024-09-21T06:51:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:12 smithi183 ceph-mon[34219]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:12 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3116993439' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:12 smithi193 ceph-mon[34601]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:12 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3116993439' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:13.008 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:12 smithi114 ceph-mon[30654]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:13.009 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:12 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3116993439' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:13.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T06:50:48.407210+0000', 'modified': '2024-09-21T06:51:00.665998+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24761}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.183:6839/42004963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 42004963}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 42004963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24761, 'qdb_cluster': [24761]} 2024-09-21T06:51:13.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T06:51:13.048 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:51:13.420 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:14.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 -- 172.21.15.114:0/1946549248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 msgr2=0x7f6040112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 --2- 172.21.15.114:0/1946549248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 0x7f6040112850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6038009ca0 tx=0x7f603802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 -- 172.21.15.114:0/1946549248 shutdown_connections 2024-09-21T06:51:14.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 --2- 172.21.15.114:0/1946549248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 0x7f6040112850 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 --2- 172.21.15.114:0/1946549248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604010b6d0 0x7f604010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 --2- 172.21.15.114:0/1946549248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6040105d90 0x7f60401061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 -- 172.21.15.114:0/1946549248 >> 172.21.15.114:0/1946549248 conn(0x7f60401017c0 msgr2=0x7f6040103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:14.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 -- 172.21.15.114:0/1946549248 shutdown_connections 2024-09-21T06:51:14.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.095+0000 7f6047c00640 1 -- 172.21.15.114:0/1946549248 wait complete. 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.096+0000 7f6047c00640 1 Processor -- start 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.096+0000 7f6047c00640 1 -- start start 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6047c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 0x7f604019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6047c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f604010b6d0 0x7f604019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6047c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 0x7f60401a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6047c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6040115ce0 con 0x7f604010c020 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6047c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6040115b60 con 0x7f6040105d90 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6047c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6040115e60 con 0x7f604010b6d0 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 0x7f604019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 0x7f604019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52526/0 (socket says 172.21.15.114:52526) 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 -- 172.21.15.114:0/793408902 learned_addr learned my addr 172.21.15.114:0/793408902 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:14.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 0x7f60401a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f603fe00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f604010b6d0 0x7f604019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 -- 172.21.15.114:0/793408902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f604010b6d0 msgr2=0x7f604019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f604010b6d0 0x7f604019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 -- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 msgr2=0x7f60401a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.097+0000 7f6045200640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 0x7f60401a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f6045200640 1 -- 172.21.15.114:0/793408902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60401a2640 con 0x7f6040105d90 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f6045200640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 0x7f604019c6d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f603000e9f0 tx=0x7f603000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f603d600640 1 -- 172.21.15.114:0/793408902 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603000cd80 con 0x7f6040105d90 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f603d600640 1 -- 172.21.15.114:0/793408902 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60300054d0 con 0x7f6040105d90 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f603d600640 1 -- 172.21.15.114:0/793408902 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030010690 con 0x7f6040105d90 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60401a2930 con 0x7f6040105d90 2024-09-21T06:51:14.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.098+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60401a9090 con 0x7f6040105d90 2024-09-21T06:51:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.102+0000 7f604e86b740 1 Processor -- start 2024-09-21T06:51:14.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.102+0000 7f604e86b740 1 -- start start 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f604e86b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060f3f60 0x56350649a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f604e86b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x56350649a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f604e86b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 0x56350649af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f604e86b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563506411d50 con 0x5635060cb0a0 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f604e86b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563506401180 con 0x5635060d3280 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f604e86b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563506404f90 con 0x5635060f3f60 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f603fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 0x56350649af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f603d600640 1 -- 172.21.15.114:0/793408902 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f6030004e90 con 0x7f6040105d90 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f603fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 0x56350649af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52538/0 (socket says 172.21.15.114:52538) 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f603fe00640 1 -- 172.21.15.114:0/4148436307 learned_addr learned my addr 172.21.15.114:0/4148436307 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:14.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.103+0000 7f6045c00640 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060f3f60 0x56350649a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f6045200640 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x56350649a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f603fe00640 1 -- 172.21.15.114:0/4148436307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060f3f60 msgr2=0x56350649a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f603fe00640 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060f3f60 0x56350649a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f603fe00640 1 -- 172.21.15.114:0/4148436307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 msgr2=0x56350649a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f603fe00640 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x56350649a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f603fe00640 1 -- 172.21.15.114:0/4148436307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635063cba70 con 0x5635060d3280 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f6045200640 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x56350649a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f603fe00640 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 0x56350649af10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f602c009ba0 tx=0x7f602c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56fc8cf6e1248439 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f6036000640 1 -- 172.21.15.114:0/4148436307 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c004230 con 0x5635060d3280 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f6036000640 1 -- 172.21.15.114:0/4148436307 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c009e10 con 0x5635060d3280 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.104+0000 7f6036000640 1 -- 172.21.15.114:0/4148436307 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c004cd0 con 0x5635060d3280 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f603d600640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60240b6560 0x7f60240b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 -- 172.21.15.114:0/4148436307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 msgr2=0x56350649af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 0x56350649af10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f602c009ba0 tx=0x7f602c011810 comp rx=0 tx=0).stop 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f603d600640 1 -- 172.21.15.114:0/793408902 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f60300e7a50 con 0x7f6040105d90 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 -- 172.21.15.114:0/4148436307 shutdown_connections 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060d3280 0x56350649af10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x56350649a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 --2- 172.21.15.114:0/4148436307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060f3f60 0x56350649a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 -- 172.21.15.114:0/4148436307 >> 172.21.15.114:0/4148436307 conn(0x5635060e8350 msgr2=0x563506498380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f6045c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60240b6560 0x7f60240b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.105+0000 7f604e86b740 1 -- 172.21.15.114:0/4148436307 shutdown_connections 2024-09-21T06:51:14.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.106+0000 7f604e86b740 1 -- 172.21.15.114:0/4148436307 wait complete. 2024-09-21T06:51:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.106+0000 7f6045c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60240b6560 0x7f60240b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6038005cc0 tx=0x7f6038005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.106+0000 7f604e86b740 1 -- start start 2024-09-21T06:51:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.106+0000 7f604e86b740 1 Processor -- start 2024-09-21T06:51:14.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5635062625f0 con 0x5635060cb0a0 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56350639c460 con 0x5635060f3f60 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5635064a8c70 con 0x5635060d3280 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f604e86b740 10 client.0 _ll_register_callbacks cb 0x5635063ac210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f6045200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f6045200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064a6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46530/0 (socket says 172.21.15.114:46530) 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f6045200640 1 -- 172.21.15.114:0/620719224 learned_addr learned my addr 172.21.15.114:0/620719224 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f603fe00640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.107+0000 7f6045c00640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f6045200640 1 -- 172.21.15.114:0/620719224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 msgr2=0x5635064a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f6045200640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f6045200640 1 -- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 msgr2=0x5635064b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f6045200640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f6045200640 1 -- 172.21.15.114:0/620719224 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350615f540 con 0x5635060cb0a0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f6045200640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064a6080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f603000e9c0 tx=0x7f60300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e562a294c7b8b9da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f602aa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f602aa00640 1 -- 172.21.15.114:0/620719224 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030005180 con 0x5635060cb0a0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f602aa00640 1 -- 172.21.15.114:0/620719224 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f60300a9220 con 0x5635060cb0a0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56350639ea70 con 0x5635060cb0a0 2024-09-21T06:51:14.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.108+0000 7f602aa00640 1 -- 172.21.15.114:0/620719224 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f60300b21b0 con 0x5635060cb0a0 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 10 client.14853 fetch_fsmap learned FSMap version 16 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5635063b96f0 con 0x5635060cb0a0 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f602aa00640 1 -- 172.21.15.114:0/620719224 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56350639ea70 con 0x5635060cb0a0 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f602aa00640 1 -- 172.21.15.114:0/620719224 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f60300d3020 con 0x5635060cb0a0 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 10 client.14853 fetch_fsmap finished waiting for FSMap version 16 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 10 client.14853 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 20 client.14853 populate_metadata read hostname 'smithi114' 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5635060b28e0 0x5635064b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 4 client.14853 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.109+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563506413ca0 con 0x5635060b28e0 2024-09-21T06:51:14.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.110+0000 7f603fe00640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5635060b28e0 0x5635064b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.110+0000 7f603fe00640 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5635060b28e0 0x5635064b8500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223c13e2238b0bbe server_cookie=9714ef54e2881416 in_seq=0 out_seq=0 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.110+0000 7f602aa00640 10 client.14853 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.111+0000 7f602aa00640 1 -- 172.21.15.114:0/620719224 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563506413ca0 con 0x5635060b28e0 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.111+0000 7f602aa00640 10 client.14853 handle_command_reply: tid=0 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.111+0000 7f604e86b740 1 client.14853 shutdown 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 msgr2=0x5635064a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064a6080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f603000e9c0 tx=0x7f60300048f0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 shutdown_connections 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5635060b28e0 0x5635064b8500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 --2- 172.21.15.114:0/620719224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064a6080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 >> 172.21.15.114:0/620719224 conn(0x5635060e8350 msgr2=0x56350649d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:14.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f602aa00640 0 client.14853 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:51:14.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 shutdown_connections 2024-09-21T06:51:14.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 1 -- 172.21.15.114:0/620719224 wait complete. 2024-09-21T06:51:14.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.112+0000 7f604e86b740 20 client.14853 trim_cache size 0 max 16384 2024-09-21T06:51:14.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 Processor -- start 2024-09-21T06:51:14.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 -- start start 2024-09-21T06:51:14.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064a1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5635064b6840 con 0x5635060cb0a0 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635064b6f10 con 0x5635060f3f60 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f604e86b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5635064b75e0 con 0x5635060d3280 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f6045200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f6045200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46540/0 (socket says 172.21.15.114:46540) 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f6045200640 1 -- 172.21.15.114:0/3623057850 learned_addr learned my addr 172.21.15.114:0/3623057850 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f6045c00640 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.131+0000 7f603fe00640 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6045200640 1 -- 172.21.15.114:0/3623057850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 msgr2=0x5635064a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6045200640 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6045200640 1 -- 172.21.15.114:0/3623057850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 msgr2=0x5635064a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6045200640 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6045200640 1 -- 172.21.15.114:0/3623057850 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350649cbe0 con 0x5635060cb0a0 2024-09-21T06:51:14.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6045200640 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064b6160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f603000c6a0 tx=0x7f6030005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0f7868bd17ed5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6035600640 1 -- 172.21.15.114:0/3623057850 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030005180 con 0x5635060cb0a0 2024-09-21T06:51:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6035600640 1 -- 172.21.15.114:0/3623057850 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60300ad3e0 con 0x5635060cb0a0 2024-09-21T06:51:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.132+0000 7f6035600640 1 -- 172.21.15.114:0/3623057850 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300ad220 con 0x5635060cb0a0 2024-09-21T06:51:14.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 -- 172.21.15.114:0/3623057850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 msgr2=0x5635064b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064b6160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f603000c6a0 tx=0x7f6030005680 comp rx=0 tx=0).stop 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 -- 172.21.15.114:0/3623057850 shutdown_connections 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060f3f60 0x5635064a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 --2- 172.21.15.114:0/3623057850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060cb0a0 0x5635064b6160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 -- 172.21.15.114:0/3623057850 >> 172.21.15.114:0/3623057850 conn(0x5635060e8350 msgr2=0x5635064b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 -- 172.21.15.114:0/3623057850 shutdown_connections 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.133+0000 7f604e86b740 1 -- 172.21.15.114:0/3623057850 wait complete. 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 -- start start 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 Processor -- start 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060cb0a0 0x5635064c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 0x5635064ba890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5635064b8380 con 0x5635060f3f60 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635064b7cb0 con 0x5635060cb0a0 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5635064b8a50 con 0x5635060d3280 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.134+0000 7f604e86b740 10 client.0 _ll_register_callbacks cb 0x7f60300a9f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 0x5635064ba890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060cb0a0 0x5635064c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f603fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 0x5635064ba890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46552/0 (socket says 172.21.15.114:46552) 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 -- 172.21.15.114:0/2224045291 learned_addr learned my addr 172.21.15.114:0/2224045291 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 -- 172.21.15.114:0/2224045291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 msgr2=0x5635064ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 -- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060cb0a0 msgr2=0x5635064c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:51:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060cb0a0 0x5635064c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 -- 172.21.15.114:0/2224045291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635064baf30 con 0x5635060f3f60 2024-09-21T06:51:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f6045c00640 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 0x5635064ba890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6038009ca0 tx=0x7f60380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd125d8a8d2ffa50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.135+0000 7f602aa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:51:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f602aa00640 1 -- 172.21.15.114:0/2224045291 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038058060 con 0x5635060f3f60 2024-09-21T06:51:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f602aa00640 1 -- 172.21.15.114:0/2224045291 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f6038004040 con 0x5635060f3f60 2024-09-21T06:51:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60300ad3e0 con 0x5635060f3f60 2024-09-21T06:51:14.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f602aa00640 1 -- 172.21.15.114:0/2224045291 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f603805f050 con 0x5635060f3f60 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f604e86b740 10 client.14859 fetch_fsmap learned FSMap version 16 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5635065239d0 con 0x5635060f3f60 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f602aa00640 1 -- 172.21.15.114:0/2224045291 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60300ad3e0 con 0x5635060f3f60 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f602aa00640 1 -- 172.21.15.114:0/2224045291 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f603802fa70 con 0x5635060f3f60 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.136+0000 7f604e86b740 10 client.14859 fetch_fsmap finished waiting for FSMap version 16 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f604e86b740 10 client.14859 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f604e86b740 20 client.14859 populate_metadata read hostname 'smithi114' 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f604e86b740 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563506087c90 0x563506527180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f604e86b740 4 client.14859 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "status"}) -- 0x56350640f750 con 0x563506087c90 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f6045200640 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563506087c90 0x563506527180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:14.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f6045200640 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563506087c90 0x563506527180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d92db22b2d262 server_cookie=dbfa57733120d405 in_seq=0 out_seq=0 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.137+0000 7f602aa00640 10 client.14859 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f602aa00640 1 -- 172.21.15.114:0/2224045291 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56350640f750 con 0x563506087c90 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f602aa00640 10 client.14859 handle_command_reply: tid=0 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 client.14859 shutdown 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 msgr2=0x5635064ba890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 0x5635064ba890 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6038009ca0 tx=0x7f60380095c0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 shutdown_connections 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5635060f3f60 0x5635064ba890 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563506087c90 0x563506527180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5635060d3280 0x5635064ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 --2- 172.21.15.114:0/2224045291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635060cb0a0 0x5635064c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 >> 172.21.15.114:0/2224045291 conn(0x5635060e8350 msgr2=0x5635064a6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f602aa00640 0 client.14859 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 shutdown_connections 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 1 -- 172.21.15.114:0/2224045291 wait complete. 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.138+0000 7f604e86b740 20 client.14859 trim_cache size 0 max 16384 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24761, 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:51:14.144 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 24.920663463, 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 16, 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 85, 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 85, 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 38.537384801999998 2024-09-21T06:51:14.145 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60240b6560 msgr2=0x7f60240b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60240b6560 0x7f60240b8a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6038005cc0 tx=0x7f6038005c30 comp rx=0 tx=0).stop 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 msgr2=0x7f604019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 0x7f604019c6d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f603000e9f0 tx=0x7f603000edf0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 shutdown_connections 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f604010c020 0x7f60401a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f604010b6d0 0x7f604019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60240b6560 0x7f60240b8a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 --2- 172.21.15.114:0/793408902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6040105d90 0x7f604019c6d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:14.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 >> 172.21.15.114:0/793408902 conn(0x7f60401017c0 msgr2=0x7f6040108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:14.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 shutdown_connections 2024-09-21T06:51:14.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:14.139+0000 7f6047c00640 1 -- 172.21.15.114:0/793408902 wait complete. 2024-09-21T06:51:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:14 smithi193 ceph-mon[34601]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:15.054 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:14 smithi114 ceph-mon[30654]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:15.093 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:51:15.161 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:15.162 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T06:51:15.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:14 smithi183 ceph-mon[34219]: pgmap v305: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:15.530 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:16.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 -- 172.21.15.114:0/351325519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 msgr2=0x7f1718104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 --2- 172.21.15.114:0/351325519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718104de0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1708009860 tx=0x7f170802a340 comp rx=0 tx=0).stop 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 -- 172.21.15.114:0/351325519 shutdown_connections 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 --2- 172.21.15.114:0/351325519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171810de30 0x7f1718110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 --2- 172.21.15.114:0/351325519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 0x7f171810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 --2- 172.21.15.114:0/351325519 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718104de0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 -- 172.21.15.114:0/351325519 >> 172.21.15.114:0/351325519 conn(0x7f1718079740 msgr2=0x7f1718079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 -- 172.21.15.114:0/351325519 shutdown_connections 2024-09-21T06:51:16.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.205+0000 7f171ec00640 1 -- 172.21.15.114:0/351325519 wait complete. 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 Processor -- start 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 -- start start 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 0x7f1718102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171810de30 0x7f1718103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1718113ab0 con 0x7f17181049d0 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1718113930 con 0x7f1718105320 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.206+0000 7f171ec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1718113c30 con 0x7f171810de30 2024-09-21T06:51:16.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:16.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51104/0 (socket says 172.21.15.114:51104) 2024-09-21T06:51:16.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 -- 172.21.15.114:0/957134189 learned_addr learned my addr 172.21.15.114:0/957134189 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:16.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717400640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 0x7f1718102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:16.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f171cc00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171810de30 0x7f1718103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 -- 172.21.15.114:0/957134189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171810de30 msgr2=0x7f1718103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171810de30 0x7f1718103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 -- 172.21.15.114:0/957134189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 msgr2=0x7f1718102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 0x7f1718102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717e00640 1 -- 172.21.15.114:0/957134189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17181a87c0 con 0x7f17181049d0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.207+0000 7f1717400640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 0x7f1718102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.208+0000 7f1717e00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718102710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1708000c00 tx=0x7f1708032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.208+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708040020 con 0x7f17181049d0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.208+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1708030d50 con 0x7f17181049d0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.208+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17181a8a50 con 0x7f17181049d0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.208+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708037040 con 0x7f17181049d0 2024-09-21T06:51:16.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.208+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17181a8f30 con 0x7f17181049d0 2024-09-21T06:51:16.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.209+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171806c240 con 0x7f17181049d0 2024-09-21T06:51:16.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.211+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f1708043020 con 0x7f17181049d0 2024-09-21T06:51:16.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.213+0000 7f1714c00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16fc0b6560 0x7f16fc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:16.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.213+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f1708107a80 con 0x7f17181049d0 2024-09-21T06:51:16.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.213+0000 7f1717400640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16fc0b6560 0x7f16fc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:16.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.214+0000 7f1717400640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16fc0b6560 0x7f16fc0b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f171810fc30 tx=0x7f1704009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:16.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.215+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f170804a030 con 0x7f17181049d0 2024-09-21T06:51:16.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.442+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1718103a10 con 0x7f17181049d0 2024-09-21T06:51:16.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.442+0000 7f1714c00640 1 -- 172.21.15.114:0/957134189 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f17080cdfa0 con 0x7f17181049d0 2024-09-21T06:51:16.444 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16fc0b6560 msgr2=0x7f16fc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16fc0b6560 0x7f16fc0b8a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f171810fc30 tx=0x7f1704009260 comp rx=0 tx=0).stop 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 msgr2=0x7f1718102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718102710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1708000c00 tx=0x7f1708032d40 comp rx=0 tx=0).stop 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 shutdown_connections 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171810de30 0x7f1718103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16fc0b6560 0x7f16fc0b8a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1718105320 0x7f1718102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 --2- 172.21.15.114:0/957134189 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f17181049d0 0x7f1718102710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:16.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.444+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 >> 172.21.15.114:0/957134189 conn(0x7f1718079740 msgr2=0x7f17180ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:16.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.445+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 shutdown_connections 2024-09-21T06:51:16.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:16.445+0000 7f171ec00640 1 -- 172.21.15.114:0/957134189 wait complete. 2024-09-21T06:51:16.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:16 smithi114 ceph-mon[30654]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:16.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:16 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/957134189' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:51:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:16 smithi193 ceph-mon[34601]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:16 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/957134189' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:51:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:16 smithi183 ceph-mon[34219]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:16 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/957134189' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:51:17.324 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:51:17.325 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T06:51:17.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:17.325 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-09-21T06:51:17.360 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:51:17.361 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:51:17.362 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:51:17.362 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:51:17.362 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T06:51:17.362 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-09-21T06:51:17.362 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link noprefixroute 2024-09-21T06:51:17.362 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:51:17.362 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:51:17.363 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:51:17.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:51:17.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:17.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:17.772 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:51:17.828 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-21T06:51:17.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:17.831 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-09-21T06:51:17.877 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-09-21T06:51:17.878 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:51:17.878 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T06:51:17.878 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T06:51:17.878 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:51:17.879 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:51:17.879 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:51:17.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T06:51:17.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T06:51:17.879 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T06:51:17.879 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:51:17.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:51:18.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:18.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:18.458 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-21T06:51:18.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:18.494 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-09-21T06:51:18.569 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:51:18.569 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:51:18.569 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:18.570 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T06:51:18.570 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:51:18.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:51:18.704 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:18 smithi114 ceph-mon[30654]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:18.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:18.749 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T06:51:18.750 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:51:18.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:51:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:18 smithi193 ceph-mon[34601]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:18.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:18.997 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:51:18.997 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:51:18.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-09-21T06:51:18.997 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:51:18.997 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:51:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:51:19.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:18 smithi183 ceph-mon[34219]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:19.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:19.164 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T06:51:19.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:51:19.165 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:19.200 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T06:51:19.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:51:19.200 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:19.275 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-21T06:51:19.372 DEBUG:teuthology.orchestra.run.smithi114:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:51:19.408 INFO:teuthology.orchestra.run.smithi114.stdout:/proc 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/sys 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/dev 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/security 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/shm 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/pts 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/run 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/cgroup 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/pstore 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/bpf 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/ 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/selinux 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/hugepages 2024-09-21T06:51:19.409 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/mqueue 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/debug 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/tracing 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/fuse/connections 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/config 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/1000 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/ceph 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/6e81616780909e082ae7a42121c1583b7b2f0749d81dbdbe786b85c68d24791d/merged 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/660928c4550efa31188e1da2cddd567106a1b52832067728deb79b09aae916ef/merged 2024-09-21T06:51:19.410 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/0 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/bfda4946eedcc84c00eebd2fffab4880b883be2162607fefca37775431b98d6f/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/ebb79ed962701832c026c68baeb980d962b26d011b24c1a9cf876e92c850f2af/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/688af64e6b68941877e9459d04bcc084c052d72ef9d19ca8d1f14a53487dc7d6/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/9c9476b8fef914bb57a532138582cc3babc4ec60ee854bb431114c26dc2e29e8/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/bde84a79cdcb216816eb2298ba897d8046c79c3612bd280c02fee9f2e1dcf610/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/3d432b793574d4a69761fd439518d65550f89fe9b2ed905b446ed8f27a946569/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/1e59605f47d8725a1d03dc6a9e505801eacdc47631d0714836b5900c50e106ed/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/72337509574ab9a42a7547bc3b841917204524277596c4c986022c3ffd7725c9/merged 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:19.411 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:19.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:19.412 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-21T06:51:19.487 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T06:51:19.487 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T06:51:19.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-21T06:51:19.588 DEBUG:teuthology.orchestra.run.smithi114:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:51:19.626 INFO:teuthology.orchestra.run.smithi114.stdout:/proc 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/sys 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/dev 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/security 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/shm 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/pts 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/run 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/cgroup 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/pstore 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/bpf 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/ 2024-09-21T06:51:19.627 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/selinux 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/hugepages 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/mqueue 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/debug 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/tracing 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/fuse/connections 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/config 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/1000 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/ceph 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:51:19.628 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/6e81616780909e082ae7a42121c1583b7b2f0749d81dbdbe786b85c68d24791d/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/660928c4550efa31188e1da2cddd567106a1b52832067728deb79b09aae916ef/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/0 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/bfda4946eedcc84c00eebd2fffab4880b883be2162607fefca37775431b98d6f/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/ebb79ed962701832c026c68baeb980d962b26d011b24c1a9cf876e92c850f2af/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/688af64e6b68941877e9459d04bcc084c052d72ef9d19ca8d1f14a53487dc7d6/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/9c9476b8fef914bb57a532138582cc3babc4ec60ee854bb431114c26dc2e29e8/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/bde84a79cdcb216816eb2298ba897d8046c79c3612bd280c02fee9f2e1dcf610/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/3d432b793574d4a69761fd439518d65550f89fe9b2ed905b446ed8f27a946569/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/1e59605f47d8725a1d03dc6a9e505801eacdc47631d0714836b5900c50e106ed/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/72337509574ab9a42a7547bc3b841917204524277596c4c986022c3ffd7725c9/merged 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:19.629 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:19.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:19.630 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-21T06:51:19.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:2024-09-21T06:51:19.799+0000 7f3541d56480 -1 init, newargv = 0x7f352c004ba0 newargc=15 2024-09-21T06:51:19.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:ceph-fuse[75721]: starting ceph client 2024-09-21T06:51:19.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:ceph-fuse[75721]: starting fuse 2024-09-21T06:51:20.806 DEBUG:teuthology.orchestra.run.smithi114:> sudo modprobe fuse 2024-09-21T06:51:20.870 DEBUG:teuthology.orchestra.run.smithi114:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/proc 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/sys 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/dev 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/security 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/shm 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/pts 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/run 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/cgroup 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/pstore 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/bpf 2024-09-21T06:51:20.908 INFO:teuthology.orchestra.run.smithi114.stdout:/ 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/selinux 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/hugepages 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/dev/mqueue 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/debug 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/tracing 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/fs/fuse/connections 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/sys/kernel/config 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/1000 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/ceph 2024-09-21T06:51:20.909 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/6e81616780909e082ae7a42121c1583b7b2f0749d81dbdbe786b85c68d24791d/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/660928c4550efa31188e1da2cddd567106a1b52832067728deb79b09aae916ef/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/run/user/0 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/bfda4946eedcc84c00eebd2fffab4880b883be2162607fefca37775431b98d6f/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/ebb79ed962701832c026c68baeb980d962b26d011b24c1a9cf876e92c850f2af/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/688af64e6b68941877e9459d04bcc084c052d72ef9d19ca8d1f14a53487dc7d6/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/9c9476b8fef914bb57a532138582cc3babc4ec60ee854bb431114c26dc2e29e8/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/bde84a79cdcb216816eb2298ba897d8046c79c3612bd280c02fee9f2e1dcf610/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/3d432b793574d4a69761fd439518d65550f89fe9b2ed905b446ed8f27a946569/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/1e59605f47d8725a1d03dc6a9e505801eacdc47631d0714836b5900c50e106ed/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/var/lib/containers/storage/overlay/72337509574ab9a42a7547bc3b841917204524277596c4c986022c3ffd7725c9/merged 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns 2024-09-21T06:51:20.910 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:20.911 INFO:teuthology.orchestra.run.smithi114.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:51:20.911 INFO:teuthology.orchestra.run.smithi114.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:20.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:20.912 DEBUG:teuthology.orchestra.run.smithi114:> ls /sys/fs/fuse/connections 2024-09-21T06:51:20.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:20 smithi193 ceph-mon[34601]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:20.988 INFO:teuthology.orchestra.run.smithi114.stdout:120 2024-09-21T06:51:20.988 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> sudo stdin-killer -- python3 -c ' 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> import glob 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> import re 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> import os 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> import subprocess 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> def _find_admin_socket(client_name): 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> files = glob.glob(asok_path) 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> # Given a non-glob path, it better be there 2024-09-21T06:51:20.989 DEBUG:teuthology.orchestra.run.smithi114:> if "*" not in asok_path: 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> assert(len(files) == 1) 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> return files[0] 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> for f in files: 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> contents = proc_f.read() 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> if mountpoint in contents: 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> return f 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> print(_find_admin_socket("client.admin")) 2024-09-21T06:51:20.990 DEBUG:teuthology.orchestra.run.smithi114:> ' 2024-09-21T06:51:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:20 smithi114 ceph-mon[30654]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:20 smithi183 ceph-mon[34219]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:21.151 INFO:teuthology.orchestra.run.smithi114.stdout:/var/run/ceph/ceph-client.admin.75721.asok 2024-09-21T06:51:21.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:21.165 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75721.asok 2024-09-21T06:51:21.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:21.165 DEBUG:teuthology.orchestra.run.smithi114:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75721.asok status 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "metadata": { 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "entity_id": "admin", 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": "smithi114", 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "pid": "75721", 2024-09-21T06:51:21.409 INFO:teuthology.orchestra.run.smithi114.stdout: "root": "/" 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_count": 0, 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "dentry_pinned_count": 0, 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 14871, 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "inst": { 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "name": { 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "client", 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "num": 14871 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 2269807061 2024-09-21T06:51:21.410 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": { 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "192.168.0.1:0", 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 2269807061 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "inst_str": "client.14871 192.168.0.1:0/2269807061", 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "addr_str": "192.168.0.1:0/2269807061", 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "inode_count": 1, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_epoch": 16, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch": 85, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_epoch_barrier": 0, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "blocklisted": false, 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs" 2024-09-21T06:51:21.411 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:51:21.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:21.428 DEBUG:teuthology.orchestra.run.smithi114:> stat --file-system '--printf=%T 2024-09-21T06:51:21.428 DEBUG:teuthology.orchestra.run.smithi114:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:21.464 INFO:teuthology.orchestra.run.smithi114.stdout:fuseblk 2024-09-21T06:51:21.464 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:21.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:21.464 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:21.562 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T06:51:21.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:21 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:51:21.669 INFO:teuthology.orchestra.run.smithi114.stdout:# file: . 2024-09-21T06:51:21.669 INFO:teuthology.orchestra.run.smithi114.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T06:51:21.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:21.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:51:21.677 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:51:21.678 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:51:21.714 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:51:22.121 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 -- 172.21.15.114:0/2728558390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0105d90 msgr2=0x7f12b01061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 --2- 172.21.15.114:0/2728558390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0105d90 0x7f12b01061a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009900 tx=0x7f12a002a410 comp rx=0 tx=0).stop 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 -- 172.21.15.114:0/2728558390 shutdown_connections 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 --2- 172.21.15.114:0/2728558390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b010c020 0x7f12b0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 --2- 172.21.15.114:0/2728558390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b010b6d0 0x7f12b010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 --2- 172.21.15.114:0/2728558390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b0105d90 0x7f12b01061a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 -- 172.21.15.114:0/2728558390 >> 172.21.15.114:0/2728558390 conn(0x7f12b0101860 msgr2=0x7f12b0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 -- 172.21.15.114:0/2728558390 shutdown_connections 2024-09-21T06:51:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.794+0000 7f12b6c00640 1 -- 172.21.15.114:0/2728558390 wait complete. 2024-09-21T06:51:22.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 Processor -- start 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 -- start start 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 0x7f12b019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b010b6d0 0x7f12b019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b010c020 0x7f12b01a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0115d30 con 0x7f12b010b6d0 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0115bb0 con 0x7f12b0105d90 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12b6c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0115eb0 con 0x7f12b010c020 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.796+0000 7f12afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 0x7f12b019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12a7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b010b6d0 0x7f12b019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 0x7f12b019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56634/0 (socket says 172.21.15.114:56634) 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 -- 172.21.15.114:0/294409624 learned_addr learned my addr 172.21.15.114:0/294409624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12b4c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b010c020 0x7f12b01a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 -- 172.21.15.114:0/294409624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b010c020 msgr2=0x7f12b01a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b010c020 0x7f12b01a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 -- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b010b6d0 msgr2=0x7f12b019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b010b6d0 0x7f12b019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 -- 172.21.15.114:0/294409624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01a2650 con 0x7f12b0105d90 2024-09-21T06:51:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.797+0000 7f12afe00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 0x7f12b019c6e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12a0005ab0 tx=0x7f12a0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.798+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a003f070 con 0x7f12b0105d90 2024-09-21T06:51:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.798+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01a28e0 con 0x7f12b0105d90 2024-09-21T06:51:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.798+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a0031b60 con 0x7f12b0105d90 2024-09-21T06:51:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.798+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0035030 con 0x7f12b0105d90 2024-09-21T06:51:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.798+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a90a0 con 0x7f12b0105d90 2024-09-21T06:51:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.799+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b01061a0 con 0x7f12b0105d90 2024-09-21T06:51:22.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.801+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f12a0042070 con 0x7f12b0105d90 2024-09-21T06:51:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.802+0000 7f12ad600640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12940b65e0 0x7f12940b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.803+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7f12a0107ff0 con 0x7f12b0105d90 2024-09-21T06:51:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.803+0000 7f12a7e00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12940b65e0 0x7f12940b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:22.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.804+0000 7f12a7e00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12940b65e0 0x7f12940b8af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12b0105110 tx=0x7f1298005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:22.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:22.805+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12a00d28f0 con 0x7f12b0105d90 2024-09-21T06:51:22.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:22 smithi193 ceph-mon[34601]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:23.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.003+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12b019d810 con 0x7f12b0105d90 2024-09-21T06:51:23.005 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:22 smithi114 ceph-mon[30654]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:23.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.003+0000 7f12ad600640 1 -- 172.21.15.114:0/294409624 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v85) ==== 81+0+890 (secure 0 0 0) 0x7f12b019d810 con 0x7f12b0105d90 2024-09-21T06:51:23.005 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2966269937 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:23.005 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3575030312 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:23.005 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1583737438 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3364907862 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/150287858 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/150287858 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830030899 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830030899 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/673737563 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/791492062 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2622463004 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2480842130 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/200663390 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/8263225 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:23.006 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/8263225 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:23.007 INFO:teuthology.orchestra.run.smithi114.stderr:listed 15 entries 2024-09-21T06:51:23.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12940b65e0 msgr2=0x7f12940b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12940b65e0 0x7f12940b8af0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12b0105110 tx=0x7f1298005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 msgr2=0x7f12b019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 0x7f12b019c6e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f12a0005ab0 tx=0x7f12a0003c10 comp rx=0 tx=0).stop 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 shutdown_connections 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12b010c020 0x7f12b01a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12b010b6d0 0x7f12b019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12940b65e0 0x7f12940b8af0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 --2- 172.21.15.114:0/294409624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12b0105d90 0x7f12b019c6e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 >> 172.21.15.114:0/294409624 conn(0x7f12b0101860 msgr2=0x7f12b010ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 shutdown_connections 2024-09-21T06:51:23.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:23.006+0000 7f12b6c00640 1 -- 172.21.15.114:0/294409624 wait complete. 2024-09-21T06:51:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:22 smithi183 ceph-mon[34219]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:23 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/294409624' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:51:23.856 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:51:23.856 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:51:23.893 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:51:23.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:23 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/294409624' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:51:24.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:23 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/294409624' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:51:24.300 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:24.774 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:24 smithi114 ceph-mon[30654]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:24.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:24 smithi193 ceph-mon[34601]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:24.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.944+0000 7ff2cfe00640 1 -- 172.21.15.114:0/612971797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010b4f0 msgr2=0x7ff2d010b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:24.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.944+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/612971797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010b4f0 0x7ff2d010b900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8009900 tx=0x7ff2b802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:51:24.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 -- 172.21.15.114:0/612971797 shutdown_connections 2024-09-21T06:51:24.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/612971797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010be40 0x7ff2d0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:24.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/612971797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010b4f0 0x7ff2d010b900 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:24.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/612971797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d0105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:24.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 -- 172.21.15.114:0/612971797 >> 172.21.15.114:0/612971797 conn(0x7ff2d0101680 msgr2=0x7ff2d0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:24.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 -- 172.21.15.114:0/612971797 shutdown_connections 2024-09-21T06:51:24.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.945+0000 7ff2cfe00640 1 -- 172.21.15.114:0/612971797 wait complete. 2024-09-21T06:51:24.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 Processor -- start 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 -- start start 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010b4f0 0x7ff2d019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010be40 0x7ff2d01a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0115b50 con 0x7ff2d010be40 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d01159d0 con 0x7ff2d010b4f0 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.946+0000 7ff2cfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0115cd0 con 0x7ff2d0105bb0 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:24.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47526/0 (socket says 172.21.15.114:47526) 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 -- 172.21.15.114:0/4096871457 learned_addr learned my addr 172.21.15.114:0/4096871457 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010be40 0x7ff2d01a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2ce000640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010b4f0 0x7ff2d019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 -- 172.21.15.114:0/4096871457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010b4f0 msgr2=0x7ff2d019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010b4f0 0x7ff2d019cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 -- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010be40 msgr2=0x7ff2d01a1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010be40 0x7ff2d01a1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.947+0000 7ff2cea00640 1 -- 172.21.15.114:0/4096871457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d01a2510 con 0x7ff2d0105bb0 2024-09-21T06:51:24.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.948+0000 7ff2cea00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d019c5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000e9f0 tx=0x7ff2c000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:24.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.948+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c000cd80 con 0x7ff2d0105bb0 2024-09-21T06:51:24.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.948+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c00054d0 con 0x7ff2d0105bb0 2024-09-21T06:51:24.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.948+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d01a2800 con 0x7ff2d0105bb0 2024-09-21T06:51:24.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.948+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0010690 con 0x7ff2d0105bb0 2024-09-21T06:51:24.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.948+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d01a8f60 con 0x7ff2d0105bb0 2024-09-21T06:51:24.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.949+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d0105fc0 con 0x7ff2d0105bb0 2024-09-21T06:51:24.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.952+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7ff2c0004e30 con 0x7ff2d0105bb0 2024-09-21T06:51:24.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.953+0000 7ff2c7400640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2b40b6560 0x7ff2b40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:24.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.953+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9017+0+0 (secure 0 0 0) 0x7ff2c00e7c90 con 0x7ff2d0105bb0 2024-09-21T06:51:24.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.953+0000 7ff2ce000640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2b40b6560 0x7ff2b40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:24.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.954+0000 7ff2ce000640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2b40b6560 0x7ff2b40b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2b802a9e0 tx=0x7ff2b8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:24.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:24.955+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2c00ae1b0 con 0x7ff2d0105bb0 2024-09-21T06:51:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:24 smithi183 ceph-mon[34219]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:51:25.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.155+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2d010dc70 con 0x7ff2d0105bb0 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.155+0000 7ff2c7400640 1 -- 172.21.15.114:0/4096871457 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v85) ==== 81+0+890 (secure 0 0 0) 0x7ff2c00a9030 con 0x7ff2d0105bb0 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2966269937 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3575030312 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1583737438 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3364907862 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/150287858 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:25.158 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/150287858 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830030899 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830030899 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/673737563 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/791492062 2024-09-22T06:41:54.185935+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2622463004 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2480842130 2024-09-22T06:42:25.331257+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/200663390 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/8263225 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/8263225 2024-09-22T06:43:29.387989+0000 2024-09-21T06:51:25.159 INFO:teuthology.orchestra.run.smithi114.stderr:listed 15 entries 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2b40b6560 msgr2=0x7ff2b40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2b40b6560 0x7ff2b40b8a70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2b802a9e0 tx=0x7ff2b8005a20 comp rx=0 tx=0).stop 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 msgr2=0x7ff2d019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d019c5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000e9f0 tx=0x7ff2c000edf0 comp rx=0 tx=0).stop 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 shutdown_connections 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff2d010be40 0x7ff2d01a1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2b40b6560 0x7ff2b40b8a70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:25.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010b4f0 0x7ff2d019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:25.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 --2- 172.21.15.114:0/4096871457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2d0105bb0 0x7ff2d019c5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:25.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 >> 172.21.15.114:0/4096871457 conn(0x7ff2d0101680 msgr2=0x7ff2d0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:25.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 shutdown_connections 2024-09-21T06:51:25.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:25.158+0000 7ff2cfe00640 1 -- 172.21.15.114:0/4096871457 wait complete. 2024-09-21T06:51:25.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:25 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4096871457' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:51:25.874 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi114.front.sepia.ceph.com... 2024-09-21T06:51:25.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:25.874 DEBUG:teuthology.orchestra.run.smithi114:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T06:51:25.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:25 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4096871457' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:51:25.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi114.stderr:ceph-fuse[75721]: fuse finished with error 0 and tester_r 0 2024-09-21T06:51:25.936 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T06:51:26.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:25 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4096871457' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:51:26.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:26 smithi193 ceph-mon[34601]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:27.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:26 smithi114 ceph-mon[30654]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:26 smithi183 ceph-mon[34219]: pgmap v311: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:28 smithi193 ceph-mon[34601]: pgmap v312: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:28 smithi114 ceph-mon[30654]: pgmap v312: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:28 smithi183 ceph-mon[34219]: pgmap v312: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]} : dispatch 2024-09-21T06:51:29.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T06:51:30.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]} : dispatch 2024-09-21T06:51:30.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T06:51:30.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]} : dispatch 2024-09-21T06:51:30.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]} : dispatch 2024-09-21T06:51:31.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: pgmap v313: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T06:51:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:30 smithi114 ceph-mon[30654]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:51:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: pgmap v313: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T06:51:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:30 smithi183 ceph-mon[34219]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: pgmap v313: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 8]}]': finished 2024-09-21T06:51:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-21T06:51:31.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-21T06:51:31.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-21T06:51:31.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:30 smithi193 ceph-mon[34601]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:51:32.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:51:32.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:51:32.038 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T06:51:32.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:31 smithi114 ceph-mon[30654]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:51:32.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:31 smithi114 ceph-mon[30654]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:51:32.097 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:51:32.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:31 smithi183 ceph-mon[34219]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:51:32.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:31 smithi183 ceph-mon[34219]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:51:32.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:31 smithi193 ceph-mon[34601]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:51:32.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:31 smithi193 ceph-mon[34601]: pgmap v316: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-21T06:51:32.517 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:33.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.214+0000 7f4096600640 1 -- 172.21.15.114:0/1801656502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4090109c40 msgr2=0x7f4090110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:33.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.214+0000 7f4096600640 1 --2- 172.21.15.114:0/1801656502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4090109c40 0x7f4090110470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4088009ca0 tx=0x7f408802a8b0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 -- 172.21.15.114:0/1801656502 shutdown_connections 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 --2- 172.21.15.114:0/1801656502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4090109c40 0x7f4090110470 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 --2- 172.21.15.114:0/1801656502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40901092f0 0x7f4090109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 --2- 172.21.15.114:0/1801656502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 0x7f4090103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 -- 172.21.15.114:0/1801656502 >> 172.21.15.114:0/1801656502 conn(0x7f40900ff400 msgr2=0x7f4090101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 -- 172.21.15.114:0/1801656502 shutdown_connections 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.215+0000 7f4096600640 1 -- 172.21.15.114:0/1801656502 wait complete. 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.216+0000 7f4096600640 1 Processor -- start 2024-09-21T06:51:33.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.216+0000 7f4096600640 1 -- start start 2024-09-21T06:51:33.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4096600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 0x7f409019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:33.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4096600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 0x7f409019a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:33.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4096600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4090109c40 0x7f409019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4096600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113900 con 0x7f40901092f0 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4096600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113780 con 0x7f4090109c40 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4096600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4090113a80 con 0x7f40901039b0 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f408fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 0x7f409019a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f408fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 0x7f409019a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54920/0 (socket says 172.21.15.114:54920) 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f408fe00640 1 -- 172.21.15.114:0/2410407167 learned_addr learned my addr 172.21.15.114:0/2410407167 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:33.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4095200640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 0x7f409019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.217+0000 7f4095c00640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4090109c40 0x7f409019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408fe00640 1 -- 172.21.15.114:0/2410407167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 msgr2=0x7f409019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408fe00640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 0x7f409019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408fe00640 1 -- 172.21.15.114:0/2410407167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4090109c40 msgr2=0x7f409019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408fe00640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4090109c40 0x7f409019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408fe00640 1 -- 172.21.15.114:0/2410407167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40901a0280 con 0x7f40901092f0 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f4095200640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 0x7f409019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408fe00640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 0x7f409019a8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e9f0 tx=0x7f407c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c00ccc0 con 0x7f40901092f0 2024-09-21T06:51:33.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407c0108e0 con 0x7f40901092f0 2024-09-21T06:51:33.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40901a0570 con 0x7f40901092f0 2024-09-21T06:51:33.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407c018860 con 0x7f40901092f0 2024-09-21T06:51:33.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.218+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a6cd0 con 0x7f40901092f0 2024-09-21T06:51:33.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.219+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4090103dc0 con 0x7f40901092f0 2024-09-21T06:51:33.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.223+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f407c010430 con 0x7f40901092f0 2024-09-21T06:51:33.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.224+0000 7f408d600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f40780b6560 0x7f40780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:33.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.224+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f407c0e7ec0 con 0x7f40901092f0 2024-09-21T06:51:33.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.224+0000 7f4095200640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f40780b6560 0x7f40780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:33.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.225+0000 7f4095200640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f40780b6560 0x7f40780b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4084009fc0 tx=0x7f4084005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:33.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.225+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f407c0b26b0 con 0x7f40901092f0 2024-09-21T06:51:33.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.414+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f409019b440 con 0x7f40901092f0 2024-09-21T06:51:33.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.415+0000 7f408d600640 1 -- 172.21.15.114:0/2410407167 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+25090 (secure 0 0 0) 0x7f409019b440 con 0x7f40901092f0 2024-09-21T06:51:33.418 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:33.418 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":87,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:30.657710+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:51:33.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f40780b6560 msgr2=0x7f40780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:33.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f40780b6560 0x7f40780b8a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4084009fc0 tx=0x7f4084005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:33.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 msgr2=0x7f409019a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 0x7f409019a8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f407c00e9f0 tx=0x7f407c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 shutdown_connections 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f40780b6560 0x7f40780b8a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4090109c40 0x7f409019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f40901092f0 0x7f409019a8b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 --2- 172.21.15.114:0/2410407167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40901039b0 0x7f409019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 >> 172.21.15.114:0/2410407167 conn(0x7f40900ff400 msgr2=0x7f4090101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 shutdown_connections 2024-09-21T06:51:33.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:33.418+0000 7f4096600640 1 -- 172.21.15.114:0/2410407167 wait complete. 2024-09-21T06:51:33.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:33 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2410407167' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:33.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:33 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2410407167' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:33 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2410407167' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:34.227 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:51:34.599 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:34.736 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:34 smithi114 ceph-mon[30654]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:34 smithi183 ceph-mon[34219]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:34.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:34 smithi193 ceph-mon[34601]: pgmap v317: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.231+0000 7f1c3f000640 1 -- 172.21.15.114:0/104860415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c38109c40 msgr2=0x7f1c38110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.231+0000 7f1c3f000640 1 --2- 172.21.15.114:0/104860415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c38109c40 0x7f1c38110470 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009ca0 tx=0x7f1c3002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 -- 172.21.15.114:0/104860415 shutdown_connections 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 --2- 172.21.15.114:0/104860415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c38109c40 0x7f1c38110470 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 --2- 172.21.15.114:0/104860415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c381092f0 0x7f1c38109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 --2- 172.21.15.114:0/104860415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c381039b0 0x7f1c38103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 -- 172.21.15.114:0/104860415 >> 172.21.15.114:0/104860415 conn(0x7f1c380ff440 msgr2=0x7f1c38101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:35.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 -- 172.21.15.114:0/104860415 shutdown_connections 2024-09-21T06:51:35.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.232+0000 7f1c3f000640 1 -- 172.21.15.114:0/104860415 wait complete. 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 Processor -- start 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 -- start start 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c381039b0 0x7f1c38075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c381092f0 0x7f1c38075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 0x7f1c38076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113900 con 0x7f1c381039b0 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113780 con 0x7f1c381092f0 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.233+0000 7f1c3f000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c38113a80 con 0x7f1c38109c40 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 0x7f1c38076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:35.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 0x7f1c38076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36688/0 (socket says 172.21.15.114:36688) 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 -- 172.21.15.114:0/1945565587 learned_addr learned my addr 172.21.15.114:0/1945565587 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c381039b0 0x7f1c38075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3d200640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c381092f0 0x7f1c38075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 -- 172.21.15.114:0/1945565587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c381092f0 msgr2=0x7f1c38075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c381092f0 0x7f1c38075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 -- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c381039b0 msgr2=0x7f1c38075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c381039b0 0x7f1c38075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.234+0000 7f1c3e600640 1 -- 172.21.15.114:0/1945565587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c381a8530 con 0x7f1c38109c40 2024-09-21T06:51:35.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.235+0000 7f1c3e600640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 0x7f1c38076190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005cc0 tx=0x7f1c30031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.235+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30003b70 con 0x7f1c38109c40 2024-09-21T06:51:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.235+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c381a87c0 con 0x7f1c38109c40 2024-09-21T06:51:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.235+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c300318f0 con 0x7f1c38109c40 2024-09-21T06:51:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.235+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3002f950 con 0x7f1c38109c40 2024-09-21T06:51:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.235+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c381a8d00 con 0x7f1c38109c40 2024-09-21T06:51:35.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.236+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c38103dc0 con 0x7f1c38109c40 2024-09-21T06:51:35.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.239+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f1c3002fb10 con 0x7f1c38109c40 2024-09-21T06:51:35.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.240+0000 7f1c36000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c200b6560 0x7f1c200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:35.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.240+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f1c30108a30 con 0x7f1c38109c40 2024-09-21T06:51:35.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.240+0000 7f1c3dc00640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c200b6560 0x7f1c200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:35.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.241+0000 7f1c3dc00640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c200b6560 0x7f1c200b8a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c38077160 tx=0x7f1c2c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:35.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.242+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c300ceed0 con 0x7f1c38109c40 2024-09-21T06:51:35.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.430+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c3802e640 con 0x7f1c38109c40 2024-09-21T06:51:35.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.432+0000 7f1c36000640 1 -- 172.21.15.114:0/1945565587 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+25090 (secure 0 0 0) 0x7f1c300d32b0 con 0x7f1c38109c40 2024-09-21T06:51:35.435 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:35.436 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":87,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:30.657710+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:51:35.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c200b6560 msgr2=0x7f1c200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c200b6560 0x7f1c200b8a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c38077160 tx=0x7f1c2c005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 msgr2=0x7f1c38076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 0x7f1c38076190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c30005cc0 tx=0x7f1c30031700 comp rx=0 tx=0).stop 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 shutdown_connections 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c38109c40 0x7f1c38076190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c200b6560 0x7f1c200b8a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c381092f0 0x7f1c38075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 --2- 172.21.15.114:0/1945565587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c381039b0 0x7f1c38075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 >> 172.21.15.114:0/1945565587 conn(0x7f1c380ff440 msgr2=0x7f1c38101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 shutdown_connections 2024-09-21T06:51:35.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:35.435+0000 7f1c3f000640 1 -- 172.21.15.114:0/1945565587 wait complete. 2024-09-21T06:51:35.814 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:35 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1945565587' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:35 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1945565587' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:35.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:35 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1945565587' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:51:36.194 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T06:51:36.565 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:36.741 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:36 smithi114 ceph-mon[30654]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:36 smithi183 ceph-mon[34219]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:36 smithi193 ceph-mon[34601]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:37.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 -- 172.21.15.114:0/2370535112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed4102640 msgr2=0x7efed4102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:37.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 --2- 172.21.15.114:0/2370535112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed4102640 0x7efed4102a50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efec4009ca0 tx=0x7efec402a950 comp rx=0 tx=0).stop 2024-09-21T06:51:37.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 -- 172.21.15.114:0/2370535112 shutdown_connections 2024-09-21T06:51:37.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 --2- 172.21.15.114:0/2370535112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efed41101f0 0x7efed4112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 --2- 172.21.15.114:0/2370535112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102f90 0x7efed410fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 --2- 172.21.15.114:0/2370535112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed4102640 0x7efed4102a50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 -- 172.21.15.114:0/2370535112 >> 172.21.15.114:0/2370535112 conn(0x7efed40796e0 msgr2=0x7efed4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 -- 172.21.15.114:0/2370535112 shutdown_connections 2024-09-21T06:51:37.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.201+0000 7efeda400640 1 -- 172.21.15.114:0/2370535112 wait complete. 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.202+0000 7efeda400640 1 Processor -- start 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.202+0000 7efeda400640 1 -- start start 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efeda400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 0x7efed419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efeda400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efed4102f90 0x7efed419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efeda400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed41101f0 0x7efed41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efeda400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efed4115c90 con 0x7efed41101f0 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efeda400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efed4115b10 con 0x7efed4102640 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efeda400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efed4115e10 con 0x7efed4102f90 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efed3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 0x7efed419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efed2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efed4102f90 0x7efed419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efed3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 0x7efed419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45814/0 (socket says 172.21.15.114:45814) 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efed3400640 1 -- 172.21.15.114:0/3791899137 learned_addr learned my addr 172.21.15.114:0/3791899137 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:37.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.203+0000 7efed3e00640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed41101f0 0x7efed41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efed3400640 1 -- 172.21.15.114:0/3791899137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efed4102f90 msgr2=0x7efed419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efed3400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efed4102f90 0x7efed419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efed3400640 1 -- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed41101f0 msgr2=0x7efed41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efed3400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed41101f0 0x7efed41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efed3400640 1 -- 172.21.15.114:0/3791899137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed41a25b0 con 0x7efed4102640 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efed3400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 0x7efed419c640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efec4005cc0 tx=0x7efec402f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4034040 con 0x7efed4102640 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed41a2840 con 0x7efed4102640 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec40308f0 con 0x7efed4102640 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4004640 con 0x7efed4102640 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.204+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41a9000 con 0x7efed4102640 2024-09-21T06:51:37.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.205+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed406c1a0 con 0x7efed4102640 2024-09-21T06:51:37.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.208+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7efec4037020 con 0x7efed4102640 2024-09-21T06:51:37.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.209+0000 7efecbe00640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efeb80b65e0 0x7efeb80b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:37.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.210+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7efec4108490 con 0x7efed4102640 2024-09-21T06:51:37.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.210+0000 7efed2a00640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efeb80b65e0 0x7efeb80b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:37.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.210+0000 7efed2a00640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efeb80b65e0 0x7efeb80b8af0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efed4110910 tx=0x7efec0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:37.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.211+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efec40ce900 con 0x7efed4102640 2024-09-21T06:51:37.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.437+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efed4102a50 con 0x7efed4102640 2024-09-21T06:51:37.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.492+0000 7efecbe00640 1 -- 172.21.15.114:0/3791899137 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7efec40d2ce0 con 0x7efed4102640 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efeb80b65e0 msgr2=0x7efeb80b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efeb80b65e0 0x7efeb80b8af0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efed4110910 tx=0x7efec0005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 msgr2=0x7efed419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 0x7efed419c640 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efec4005cc0 tx=0x7efec402f3a0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 shutdown_connections 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efed41101f0 0x7efed41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efed4102f90 0x7efed419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efeb80b65e0 0x7efeb80b8af0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 --2- 172.21.15.114:0/3791899137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed4102640 0x7efed419c640 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 >> 172.21.15.114:0/3791899137 conn(0x7efed40796e0 msgr2=0x7efed41115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 shutdown_connections 2024-09-21T06:51:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:37.494+0000 7efeda400640 1 -- 172.21.15.114:0/3791899137 wait complete. 2024-09-21T06:51:37.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:37 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3791899137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:51:37.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:37 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:51:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:37 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3791899137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:51:37.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:37 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:51:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:37 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3791899137' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:51:37.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:37 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T06:51:38.246 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T06:51:38.246 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T06:51:38.615 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:38.754 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:38 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:51:38.754 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:38 smithi114 ceph-mon[30654]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:38.754 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:38 smithi114 ceph-mon[30654]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:38 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:51:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:38 smithi183 ceph-mon[34219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:38 smithi183 ceph-mon[34219]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:38 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T06:51:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:38 smithi193 ceph-mon[34601]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:38 smithi193 ceph-mon[34601]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:39.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 -- 172.21.15.114:0/2494790936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688109df0 msgr2=0x7f1688110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 --2- 172.21.15.114:0/2494790936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688109df0 0x7f1688110620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1680009ca0 tx=0x7f168002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 -- 172.21.15.114:0/2494790936 shutdown_connections 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 --2- 172.21.15.114:0/2494790936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688109df0 0x7f1688110620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 --2- 172.21.15.114:0/2494790936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16881094a0 0x7f16881098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 --2- 172.21.15.114:0/2494790936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1688103b60 0x7f1688103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 -- 172.21.15.114:0/2494790936 >> 172.21.15.114:0/2494790936 conn(0x7f168807b850 msgr2=0x7f168807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.270+0000 7f1690000640 1 -- 172.21.15.114:0/2494790936 shutdown_connections 2024-09-21T06:51:39.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.271+0000 7f1690000640 1 -- 172.21.15.114:0/2494790936 wait complete. 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 Processor -- start 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 -- start start 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 0x7f168819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16881094a0 0x7f168819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1688109df0 0x7f16881a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1688113ab0 con 0x7f1688103b60 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1688113930 con 0x7f16881094a0 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.272+0000 7f1690000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1688113c30 con 0x7f1688109df0 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 0x7f168819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 0x7f168819c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59850/0 (socket says 172.21.15.114:59850) 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 -- 172.21.15.114:0/1490074436 learned_addr learned my addr 172.21.15.114:0/1490074436 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:39.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168cc00640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16881094a0 0x7f168819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168e000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1688109df0 0x7f16881a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 -- 172.21.15.114:0/1490074436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1688109df0 msgr2=0x7f16881a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1688109df0 0x7f16881a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 -- 172.21.15.114:0/1490074436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16881094a0 msgr2=0x7f168819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16881094a0 0x7f168819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 -- 172.21.15.114:0/1490074436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16881a25b0 con 0x7f1688103b60 2024-09-21T06:51:39.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.273+0000 7f168d600640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 0x7f168819c640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9f0 tx=0x7f167c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.274+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c00cd80 con 0x7f1688103b60 2024-09-21T06:51:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.274+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c0108e0 con 0x7f1688103b60 2024-09-21T06:51:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.274+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c018980 con 0x7f1688103b60 2024-09-21T06:51:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.274+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16881a28a0 con 0x7f1688103b60 2024-09-21T06:51:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.274+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881a9000 con 0x7f1688103b60 2024-09-21T06:51:39.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.275+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1688103f70 con 0x7f1688103b60 2024-09-21T06:51:39.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.277+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f167c010430 con 0x7f1688103b60 2024-09-21T06:51:39.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.279+0000 7f1686000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16700b65e0 0x7f16700b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:39.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.279+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f167c0e7b50 con 0x7f1688103b60 2024-09-21T06:51:39.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.279+0000 7f168cc00640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16700b65e0 0x7f16700b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:39.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.280+0000 7f168cc00640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16700b65e0 0x7f16700b8af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1688109190 tx=0x7f1674005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.283+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f167c0adf60 con 0x7f1688103b60 2024-09-21T06:51:39.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.507+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f16881a9330 con 0x7f1688103b60 2024-09-21T06:51:39.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.560+0000 7f1686000640 1 -- 172.21.15.114:0/1490074436 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f167c0b2340 con 0x7f1688103b60 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.562+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16700b65e0 msgr2=0x7f16700b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.562+0000 7f1690000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16700b65e0 0x7f16700b8af0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1688109190 tx=0x7f1674005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.562+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 msgr2=0x7f168819c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.562+0000 7f1690000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 0x7f168819c640 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f167c00e9f0 tx=0x7f167c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 shutdown_connections 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1688109df0 0x7f16881a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16700b65e0 0x7f16700b8af0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16881094a0 0x7f168819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 --2- 172.21.15.114:0/1490074436 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1688103b60 0x7f168819c640 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:39.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 >> 172.21.15.114:0/1490074436 conn(0x7f168807b850 msgr2=0x7f16881072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:39.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 shutdown_connections 2024-09-21T06:51:39.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:39.563+0000 7f1690000640 1 -- 172.21.15.114:0/1490074436 wait complete. 2024-09-21T06:51:39.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:39 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1490074436' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T06:51:39.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:39 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1490074436' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T06:51:39.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:39 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1490074436' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T06:51:40.380 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T06:51:40.756 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T06:51:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1490074436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T06:51:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T06:51:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-21T06:51:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:40 smithi114 ceph-mon[30654]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-21T06:51:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T06:51:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1490074436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T06:51:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T06:51:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-21T06:51:40.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:40 smithi183 ceph-mon[34219]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1490074436' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-09-21T06:51:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:40 smithi193 ceph-mon[34601]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-09-21T06:51:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 -- 172.21.15.114:0/2563530874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 msgr2=0x7f821810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 --2- 172.21.15.114:0/2563530874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f821810d870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8200009900 tx=0x7f820002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:51:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 -- 172.21.15.114:0/2563530874 shutdown_connections 2024-09-21T06:51:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 --2- 172.21.15.114:0/2563530874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 0x7f8218110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 --2- 172.21.15.114:0/2563530874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f821810d870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 --2- 172.21.15.114:0/2563530874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82181049d0 0x7f8218104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 -- 172.21.15.114:0/2563530874 >> 172.21.15.114:0/2563530874 conn(0x7f8218079740 msgr2=0x7f8218079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 -- 172.21.15.114:0/2563530874 shutdown_connections 2024-09-21T06:51:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.441+0000 7f821d000640 1 -- 172.21.15.114:0/2563530874 wait complete. 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.442+0000 7f821d000640 1 Processor -- start 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 -- start start 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82181049d0 0x7f8218102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f8218102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 0x7f8218103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8218113ae0 con 0x7f8218105320 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8218113960 con 0x7f821810de30 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f821d000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8218113c60 con 0x7f82181049d0 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f820d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f8218102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f820d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f8218102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59864/0 (socket says 172.21.15.114:59864) 2024-09-21T06:51:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f820d400640 1 -- 172.21.15.114:0/917427827 learned_addr learned my addr 172.21.15.114:0/917427827 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.443+0000 7f8215e00640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82181049d0 0x7f8218102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f8216800640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 0x7f8218103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820d400640 1 -- 172.21.15.114:0/917427827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82181049d0 msgr2=0x7f8218102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820d400640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82181049d0 0x7f8218102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820d400640 1 -- 172.21.15.114:0/917427827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 msgr2=0x7f8218103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820d400640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 0x7f8218103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820d400640 1 -- 172.21.15.114:0/917427827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82181a87d0 con 0x7f8218105320 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f8216800640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 0x7f8218103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820d400640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f8218102c50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8200005ab0 tx=0x7f8200003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.444+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820003f070 con 0x7f8218105320 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.445+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8200031b50 con 0x7f8218105320 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.445+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82181a8a60 con 0x7f8218105320 2024-09-21T06:51:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.445+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200035030 con 0x7f8218105320 2024-09-21T06:51:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.445+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82181a8f90 con 0x7f8218105320 2024-09-21T06:51:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.446+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821806c240 con 0x7f8218105320 2024-09-21T06:51:41.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.448+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f8200042070 con 0x7f8218105320 2024-09-21T06:51:41.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.449+0000 7f820f400640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81fc0b65e0 0x7f81fc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:41.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.450+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f8200108650 con 0x7f8218105320 2024-09-21T06:51:41.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.450+0000 7f8215e00640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81fc0b65e0 0x7f81fc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:41.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.451+0000 7f8215e00640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81fc0b65e0 0x7f81fc0b8af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8218104160 tx=0x7f8208005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:41.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.452+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82000cea60 con 0x7f8218105320 2024-09-21T06:51:41.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:41.686+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f8218103a10 con 0x7f8218105320 2024-09-21T06:51:41.864 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:51:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: 2024-09-21T06:51:41.553+0000 7fba7d9e8640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T06:51:41.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:41 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] up:active 2024-09-21T06:51:41.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:41 smithi183 ceph-mon[34219]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T06:51:41.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:41 smithi183 ceph-mon[34219]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-21T06:51:41.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:41 smithi183 ceph-mon[34219]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-21T06:51:41.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:41 smithi183 ceph-mon[34219]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T06:51:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:41 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] up:active 2024-09-21T06:51:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:41 smithi193 ceph-mon[34601]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T06:51:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:41 smithi193 ceph-mon[34601]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-21T06:51:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:41 smithi193 ceph-mon[34601]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-21T06:51:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:41 smithi193 ceph-mon[34601]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T06:51:42.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:41 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] up:active 2024-09-21T06:51:42.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:41 smithi114 ceph-mon[30654]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T06:51:42.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:41 smithi114 ceph-mon[30654]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-09-21T06:51:42.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:41 smithi114 ceph-mon[30654]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-09-21T06:51:42.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:41 smithi114 ceph-mon[30654]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-21T06:51:42.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.592+0000 7f820f400640 1 -- 172.21.15.114:0/917427827 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f82000d2e40 con 0x7f8218105320 2024-09-21T06:51:42.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81fc0b65e0 msgr2=0x7f81fc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:42.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81fc0b65e0 0x7f81fc0b8af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8218104160 tx=0x7f8208005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:42.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 msgr2=0x7f8218102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:42.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f8218102c50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8200005ab0 tx=0x7f8200003b90 comp rx=0 tx=0).stop 2024-09-21T06:51:42.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 shutdown_connections 2024-09-21T06:51:42.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81fc0b65e0 0x7f81fc0b8af0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:42.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f821810de30 0x7f8218103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8218105320 0x7f8218102c50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 --2- 172.21.15.114:0/917427827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82181049d0 0x7f8218102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 >> 172.21.15.114:0/917427827 conn(0x7f8218079740 msgr2=0x7f82180ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 shutdown_connections 2024-09-21T06:51:42.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:42.596+0000 7f821d000640 1 -- 172.21.15.114:0/917427827 wait complete. 2024-09-21T06:51:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: pgmap v321: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 755 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] up:active 2024-09-21T06:51:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T06:51:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T06:51:42.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T06:51:42.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T06:51:42.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:42 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/917427827' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: pgmap v321: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 755 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] up:active 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T06:51:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:42 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/917427827' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: pgmap v321: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 755 B/s wr, 0 op/s; 2 B/s, 0 objects/s recovering 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] up:active 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-21T06:51:42.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:42 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/917427827' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T06:51:43.414 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/917427827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] up:active 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: Cluster is now healthy 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-21T06:51:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:43 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:51:43.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/917427827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] up:active 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: Cluster is now healthy 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-21T06:51:43.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:43 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/917427827' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] up:active 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: Cluster is now healthy 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-21T06:51:43.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:43 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:51:44.435 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T06:51:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] up:active 2024-09-21T06:51:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T06:51:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:51:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:44 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.891 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] up:active 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.892 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:44 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: pgmap v322: 129 pgs: 129 active+clean; 584 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] up:active 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:44 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 -- 172.21.15.114:0/2106977215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 msgr2=0x7f92e4112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 --2- 172.21.15.114:0/2106977215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 0x7f92e4112350 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009ca0 tx=0x7f92dc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 -- 172.21.15.114:0/2106977215 shutdown_connections 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 --2- 172.21.15.114:0/2106977215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 0x7f92e4112350 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 --2- 172.21.15.114:0/2106977215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e410f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 --2- 172.21.15.114:0/2106977215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92e40757b0 0x7f92e4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 -- 172.21.15.114:0/2106977215 >> 172.21.15.114:0/2106977215 conn(0x7f92e40ff440 msgr2=0x7f92e4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 -- 172.21.15.114:0/2106977215 shutdown_connections 2024-09-21T06:51:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.072+0000 7f92ea400640 1 -- 172.21.15.114:0/2106977215 wait complete. 2024-09-21T06:51:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.073+0000 7f92ea400640 1 Processor -- start 2024-09-21T06:51:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.073+0000 7f92ea400640 1 -- start start 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92ea400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92e40757b0 0x7f92e419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92ea400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92ea400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 0x7f92e41a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92ea400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4115ad0 con 0x7f92e410ff10 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92ea400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4115950 con 0x7f92e40757b0 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92ea400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4115c50 con 0x7f92e4076100 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e9000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92e40757b0 0x7f92e419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39904/0 (socket says 172.21.15.114:39904) 2024-09-21T06:51:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 -- 172.21.15.114:0/570149394 learned_addr learned my addr 172.21.15.114:0/570149394 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e9a00640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 0x7f92e41a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 -- 172.21.15.114:0/570149394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92e40757b0 msgr2=0x7f92e419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92e40757b0 0x7f92e419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 -- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 msgr2=0x7f92e41a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 0x7f92e41a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.074+0000 7f92e3e00640 1 -- 172.21.15.114:0/570149394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e41a2470 con 0x7f92e4076100 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.075+0000 7f92e3e00640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e419caa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92d000e9c0 tx=0x7f92d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.075+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d000cde0 con 0x7f92e4076100 2024-09-21T06:51:45.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.075+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e41a2760 con 0x7f92e4076100 2024-09-21T06:51:45.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.075+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d00108e0 con 0x7f92e4076100 2024-09-21T06:51:45.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.075+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d00188d0 con 0x7f92e4076100 2024-09-21T06:51:45.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.075+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e41a8ec0 con 0x7f92e4076100 2024-09-21T06:51:45.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.076+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e4075bc0 con 0x7f92e4076100 2024-09-21T06:51:45.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.079+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f92d001f020 con 0x7f92e4076100 2024-09-21T06:51:45.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.080+0000 7f92e1600640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92cc0b6560 0x7f92cc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:45.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.081+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f92d00e7f60 con 0x7f92e4076100 2024-09-21T06:51:45.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.081+0000 7f92e9000640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92cc0b6560 0x7f92cc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:45.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.082+0000 7f92e9000640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92cc0b6560 0x7f92cc0b8a70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f92d80095f0 tx=0x7f92d8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:45.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.082+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f92d00ae370 con 0x7f92e4076100 2024-09-21T06:51:45.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.334+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f92e419d630 con 0x7f92e4076100 2024-09-21T06:51:45.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.619+0000 7f92e1600640 1 -- 172.21.15.114:0/570149394 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f92d00b2750 con 0x7f92e4076100 2024-09-21T06:51:45.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.621+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92cc0b6560 msgr2=0x7f92cc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:45.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.621+0000 7f92ea400640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92cc0b6560 0x7f92cc0b8a70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f92d80095f0 tx=0x7f92d8005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:45.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 msgr2=0x7f92e419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e419caa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f92d000e9c0 tx=0x7f92d000ee80 comp rx=0 tx=0).stop 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 shutdown_connections 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92e410ff10 0x7f92e41a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e4076100 0x7f92e419caa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92cc0b6560 0x7f92cc0b8a70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 --2- 172.21.15.114:0/570149394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92e40757b0 0x7f92e419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 >> 172.21.15.114:0/570149394 conn(0x7f92e40ff440 msgr2=0x7f92e4109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 shutdown_connections 2024-09-21T06:51:45.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:45.622+0000 7f92ea400640 1 -- 172.21.15.114:0/570149394 wait complete. 2024-09-21T06:51:45.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:45 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/570149394' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:51:45.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:45 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:51:45.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:45.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:45.997 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:45 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/570149394' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:51:45.997 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:45 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:51:45.997 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:45.997 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:45 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/570149394' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:51:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:45 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T06:51:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:46.393 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-21T06:51:46.394 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T06:51:46.764 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:46.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:46 smithi114 ceph-mon[30654]: pgmap v323: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-21T06:51:46.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:46 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:51:46.904 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:46 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:51:46.904 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:51:46.904 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:51:46.904 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:46.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:46 smithi193 ceph-mon[34601]: pgmap v323: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-21T06:51:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:46 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:51:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:46 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:51:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:51:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:51:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:46 smithi183 ceph-mon[34219]: pgmap v323: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-21T06:51:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:46 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T06:51:47.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:46 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:51:47.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:51:47.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:51:47.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:51:47.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 -- 172.21.15.114:0/3843784396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 msgr2=0x7f951010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 --2- 172.21.15.114:0/3843784396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951010bad0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009900 tx=0x7f94fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 -- 172.21.15.114:0/3843784396 shutdown_connections 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 --2- 172.21.15.114:0/3843784396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951010c010 0x7f9510112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 --2- 172.21.15.114:0/3843784396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951010bad0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 --2- 172.21.15.114:0/3843784396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 0x7f9510106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 -- 172.21.15.114:0/3843784396 >> 172.21.15.114:0/3843784396 conn(0x7f9510101810 msgr2=0x7f9510103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 -- 172.21.15.114:0/3843784396 shutdown_connections 2024-09-21T06:51:47.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.408+0000 7f9516e00640 1 -- 172.21.15.114:0/3843784396 wait complete. 2024-09-21T06:51:47.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.409+0000 7f9516e00640 1 Processor -- start 2024-09-21T06:51:47.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.409+0000 7f9516e00640 1 -- start start 2024-09-21T06:51:47.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.409+0000 7f9516e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 0x7f951019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.409+0000 7f9516e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.409+0000 7f9516e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951010c010 0x7f95101a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.409+0000 7f9516e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9510115d20 con 0x7f951010b6c0 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f9516e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9510115ba0 con 0x7f9510105d80 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f9516e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9510115ea0 con 0x7f951010c010 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57688/0 (socket says 172.21.15.114:57688) 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 -- 172.21.15.114:0/3441955815 learned_addr learned my addr 172.21.15.114:0/3441955815 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950fe00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 0x7f951019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:47.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f9514e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951010c010 0x7f95101a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 -- 172.21.15.114:0/3441955815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951010c010 msgr2=0x7f95101a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951010c010 0x7f95101a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 -- 172.21.15.114:0/3441955815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 msgr2=0x7f951019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 0x7f951019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950f400640 1 -- 172.21.15.114:0/3441955815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95101a2640 con 0x7f951010b6c0 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.410+0000 7f950fe00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 0x7f951019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.411+0000 7f950f400640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951019cc70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94fc005ab0 tx=0x7f94fc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.411+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc03f070 con 0x7f951010b6c0 2024-09-21T06:51:47.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.411+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95101a28d0 con 0x7f951010b6c0 2024-09-21T06:51:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.411+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc031b50 con 0x7f951010b6c0 2024-09-21T06:51:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.411+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a9090 con 0x7f951010b6c0 2024-09-21T06:51:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.411+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc049060 con 0x7f951010b6c0 2024-09-21T06:51:47.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.412+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9510106190 con 0x7f951010b6c0 2024-09-21T06:51:47.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.415+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f94fc042070 con 0x7f951010b6c0 2024-09-21T06:51:47.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.416+0000 7f950cc00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f40b6560 0x7f94f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:47.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.416+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f94fc107d10 con 0x7f951010b6c0 2024-09-21T06:51:47.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.416+0000 7f950fe00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f40b6560 0x7f94f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:47.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.417+0000 7f950fe00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f40b6560 0x7f94f40b8a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9500005fd0 tx=0x7f9500009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:47.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.418+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f94fc0d2500 con 0x7f951010b6c0 2024-09-21T06:51:47.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.636+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9510063800 con 0x7f94f40b6560 2024-09-21T06:51:47.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.675+0000 7f950cc00640 1 -- 172.21.15.114:0/3441955815 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9510063800 con 0x7f94f40b6560 2024-09-21T06:51:47.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.677+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f40b6560 msgr2=0x7f94f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:47.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.677+0000 7f9516e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f40b6560 0x7f94f40b8a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9500005fd0 tx=0x7f9500009500 comp rx=0 tx=0).stop 2024-09-21T06:51:47.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.677+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 msgr2=0x7f951019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:47.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.677+0000 7f9516e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951019cc70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f94fc005ab0 tx=0x7f94fc003b90 comp rx=0 tx=0).stop 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.677+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 shutdown_connections 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.677+0000 7f9516e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951010c010 0x7f95101a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.678+0000 7f9516e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f951010b6c0 0x7f951019cc70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.678+0000 7f9516e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f40b6560 0x7f94f40b8a70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.678+0000 7f9516e00640 1 --2- 172.21.15.114:0/3441955815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9510105d80 0x7f951019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.678+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 >> 172.21.15.114:0/3441955815 conn(0x7f9510101810 msgr2=0x7f951010ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.678+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 shutdown_connections 2024-09-21T06:51:47.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:47.678+0000 7f9516e00640 1 -- 172.21.15.114:0/3441955815 wait complete. 2024-09-21T06:51:48.065 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:51:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:51:47.644+0000 7f54ca400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:51:48.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:51:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:51:47.644+0000 7f54ca400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:51:48.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:51:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:51:47.644+0000 7f54ca400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:51:48.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:51:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:51:47.644+0000 7f54ca400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:51:48.066 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:51:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:51:47.644+0000 7f54ca400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:51:48.545 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T06:51:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:48 smithi114 ceph-mon[30654]: pgmap v324: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-09-21T06:51:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:48 smithi114 ceph-mon[30654]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:48 smithi193 ceph-mon[34601]: pgmap v324: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-09-21T06:51:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:48 smithi193 ceph-mon[34601]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:48.920 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:48 smithi183 ceph-mon[34219]: pgmap v324: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.0 KiB/s wr, 16 op/s 2024-09-21T06:51:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:48 smithi183 ceph-mon[34219]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.562+0000 7f0711e00640 1 -- 172.21.15.114:0/3200820472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c1094c0 msgr2=0x7f070c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.562+0000 7f0711e00640 1 --2- 172.21.15.114:0/3200820472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c1094c0 0x7f070c1098d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009900 tx=0x7f06f802a410 comp rx=0 tx=0).stop 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 -- 172.21.15.114:0/3200820472 shutdown_connections 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 --2- 172.21.15.114:0/3200820472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f070c109e10 0x7f070c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 --2- 172.21.15.114:0/3200820472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c1094c0 0x7f070c1098d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 --2- 172.21.15.114:0/3200820472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f070c103b80 0x7f070c103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 -- 172.21.15.114:0/3200820472 >> 172.21.15.114:0/3200820472 conn(0x7f070c0ff610 msgr2=0x7f070c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 -- 172.21.15.114:0/3200820472 shutdown_connections 2024-09-21T06:51:49.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.563+0000 7f0711e00640 1 -- 172.21.15.114:0/3200820472 wait complete. 2024-09-21T06:51:49.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 Processor -- start 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 -- start start 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 0x7f070c19c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f070c1094c0 0x7f070c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f070c109e10 0x7f070c1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f070c113ad0 con 0x7f070c103b80 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f070c113950 con 0x7f070c1094c0 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.564+0000 7f0711e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f070c113c50 con 0x7f070c109e10 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 0x7f070c19c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 0x7f070c19c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57748/0 (socket says 172.21.15.114:57748) 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 -- 172.21.15.114:0/1074571607 learned_addr learned my addr 172.21.15.114:0/1074571607 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070b400640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f070c109e10 0x7f070c1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070a000640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f070c1094c0 0x7f070c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:49.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 -- 172.21.15.114:0/1074571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f070c109e10 msgr2=0x7f070c1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f070c109e10 0x7f070c1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 -- 172.21.15.114:0/1074571607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f070c1094c0 msgr2=0x7f070c19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f070c1094c0 0x7f070c19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 -- 172.21.15.114:0/1074571607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c1a26b0 con 0x7f070c103b80 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.565+0000 7f070aa00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 0x7f070c19c740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d950 tx=0x7f06fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.566+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc017030 con 0x7f070c103b80 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.566+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06fc004020 con 0x7f070c103b80 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.566+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c1a29a0 con 0x7f070c103b80 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.566+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0047a0 con 0x7f070c103b80 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.566+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1a9100 con 0x7f070c103b80 2024-09-21T06:51:49.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.567+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c104e00 con 0x7f070c103b80 2024-09-21T06:51:49.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.569+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 21) ==== 154530+0+0 (secure 0 0 0) 0x7f06fc009a30 con 0x7f070c103b80 2024-09-21T06:51:49.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.571+0000 7f0710a00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06f00b65e0 0x7f06f00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:49.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.571+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f06fc0e7750 con 0x7f070c103b80 2024-09-21T06:51:49.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.571+0000 7f070a000640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06f00b65e0 0x7f06f00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:49.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.572+0000 7f070a000640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06f00b65e0 0x7f06f00b8af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009230 tx=0x7f06f8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:49.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.573+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f06fc0adbe0 con 0x7f070c103b80 2024-09-21T06:51:49.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.642+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f06fc0a9750 con 0x7f070c103b80 2024-09-21T06:51:49.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.796+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f070c10f8f0 con 0x7f06f00b65e0 2024-09-21T06:51:49.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.825+0000 7f0710a00640 1 -- 172.21.15.114:0/1074571607 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f070c10f8f0 con 0x7f06f00b65e0 2024-09-21T06:51:49.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.827+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06f00b65e0 msgr2=0x7f06f00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.827+0000 7f0711e00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06f00b65e0 0x7f06f00b8af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009230 tx=0x7f06f8031400 comp rx=0 tx=0).stop 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.827+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 msgr2=0x7f070c19c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.827+0000 7f0711e00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 0x7f070c19c740 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00d950 tx=0x7f06fc00de10 comp rx=0 tx=0).stop 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 shutdown_connections 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f070c109e10 0x7f070c1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06f00b65e0 0x7f06f00b8af0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f070c1094c0 0x7f070c19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 --2- 172.21.15.114:0/1074571607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f070c103b80 0x7f070c19c740 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 >> 172.21.15.114:0/1074571607 conn(0x7f070c0ff610 msgr2=0x7f070c106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 shutdown_connections 2024-09-21T06:51:49.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:49.828+0000 7f0711e00640 1 -- 172.21.15.114:0/1074571607 wait complete. 2024-09-21T06:51:50.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T06:51:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:50 smithi193 ceph-mon[34601]: pgmap v325: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-21T06:51:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:50 smithi193 ceph-mon[34601]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T06:51:51.047 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:50 smithi114 ceph-mon[30654]: pgmap v325: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-21T06:51:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:50 smithi114 ceph-mon[30654]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T06:51:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:50 smithi183 ceph-mon[34219]: pgmap v325: 129 pgs: 129 active+clean; 587 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 4.1 KiB/s wr, 18 op/s 2024-09-21T06:51:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:50 smithi183 ceph-mon[34219]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T06:51:51.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 -- 172.21.15.114:0/2402509196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 msgr2=0x7f0160110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:51.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 --2- 172.21.15.114:0/2402509196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 0x7f0160110320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0158009ca0 tx=0x7f015802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 -- 172.21.15.114:0/2402509196 shutdown_connections 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 --2- 172.21.15.114:0/2402509196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 0x7f0160110320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 --2- 172.21.15.114:0/2402509196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0160105320 0x7f016010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 --2- 172.21.15.114:0/2402509196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 -- 172.21.15.114:0/2402509196 >> 172.21.15.114:0/2402509196 conn(0x7f0160079740 msgr2=0x7f0160079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 -- 172.21.15.114:0/2402509196 shutdown_connections 2024-09-21T06:51:51.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.708+0000 7f0166000640 1 -- 172.21.15.114:0/2402509196 wait complete. 2024-09-21T06:51:51.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 Processor -- start 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 -- start start 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0160105320 0x7f0160102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 0x7f0160103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0160113ab0 con 0x7f016010de30 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0160113930 con 0x7f0160105320 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.710+0000 7f0166000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0160113c30 con 0x7f01601049d0 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 0x7f0160103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39984/0 (socket says 172.21.15.114:39984) 2024-09-21T06:51:51.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 -- 172.21.15.114:0/1259452574 learned_addr learned my addr 172.21.15.114:0/1259452574 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:51.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f0157e00640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0160105320 0x7f0160102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:51.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 -- 172.21.15.114:0/1259452574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0160105320 msgr2=0x7f0160102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:51.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0160105320 0x7f0160102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 -- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 msgr2=0x7f0160103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:51.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 0x7f0160103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.711+0000 7f015f400640 1 -- 172.21.15.114:0/1259452574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01601a87c0 con 0x7f01601049d0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.712+0000 7f015f400640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160102710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9c0 tx=0x7f015000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.712+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015000cd80 con 0x7f01601049d0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.712+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01500035f0 con 0x7f01601049d0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.712+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01601a8ab0 con 0x7f01601049d0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.712+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150010690 con 0x7f01601049d0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.712+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01601a8ff0 con 0x7f01601049d0 2024-09-21T06:51:51.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.713+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016006c240 con 0x7f01601049d0 2024-09-21T06:51:51.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.715+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f0150004000 con 0x7f01601049d0 2024-09-21T06:51:51.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.717+0000 7f015cc00640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f01440b6670 0x7f01440b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:51.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.717+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f01500e80c0 con 0x7f01601049d0 2024-09-21T06:51:51.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.717+0000 7f0157e00640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f01440b6670 0x7f01440b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:51.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.718+0000 7f0157e00640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f01440b6670 0x7f01440b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f016010fa80 tx=0x7f0148005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:51.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.719+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01500ae370 con 0x7f01601049d0 2024-09-21T06:51:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:51 smithi114 ceph-mon[30654]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:51.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:51 smithi193 ceph-mon[34601]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:51.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.934+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0160100c90 con 0x7f01440b6670 2024-09-21T06:51:51.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.941+0000 7f015cc00640 1 -- 172.21.15.114:0/1259452574 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0160100c90 con 0x7f01440b6670 2024-09-21T06:51:51.942 INFO:teuthology.orchestra.run.smithi114.stdout:/volumes/qa/sv_0/871341aa-7349-4b26-b067-0957ad52a7ec 2024-09-21T06:51:51.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f01440b6670 msgr2=0x7f01440b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:51.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f01440b6670 0x7f01440b8b80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f016010fa80 tx=0x7f0148005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:51.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 msgr2=0x7f0160102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:51.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160102710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9c0 tx=0x7f015000ee80 comp rx=0 tx=0).stop 2024-09-21T06:51:51.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 shutdown_connections 2024-09-21T06:51:51.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f016010de30 0x7f0160103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f01440b6670 0x7f01440b8b80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0160105320 0x7f0160102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 --2- 172.21.15.114:0/1259452574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01601049d0 0x7f0160102710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:51.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 >> 172.21.15.114:0/1259452574 conn(0x7f0160079740 msgr2=0x7f01600ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:51.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 shutdown_connections 2024-09-21T06:51:51.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:51.943+0000 7f0166000640 1 -- 172.21.15.114:0/1259452574 wait complete. 2024-09-21T06:51:52.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:51 smithi183 ceph-mon[34219]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:52.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T06:51:52.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:52 smithi193 ceph-mon[34601]: pgmap v326: 129 pgs: 129 active+clean; 593 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T06:51:53.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:52 smithi114 ceph-mon[30654]: pgmap v326: 129 pgs: 129 active+clean; 593 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T06:51:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:52 smithi183 ceph-mon[34219]: pgmap v326: 129 pgs: 129 active+clean; 593 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.8 KiB/s wr, 21 op/s 2024-09-21T06:51:53.144 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 -- 172.21.15.114:0/3790217504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 msgr2=0x7efc34075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3790217504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc34075b60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efc24009ca0 tx=0x7efc2402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 -- 172.21.15.114:0/3790217504 shutdown_connections 2024-09-21T06:51:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3790217504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc34110120 0x7efc34112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3790217504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc340760a0 0x7efc3410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3790217504 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc34075b60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 -- 172.21.15.114:0/3790217504 >> 172.21.15.114:0/3790217504 conn(0x7efc3407b850 msgr2=0x7efc3407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 -- 172.21.15.114:0/3790217504 shutdown_connections 2024-09-21T06:51:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.786+0000 7efc3ba00640 1 -- 172.21.15.114:0/3790217504 wait complete. 2024-09-21T06:51:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.787+0000 7efc3ba00640 1 Processor -- start 2024-09-21T06:51:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 -- start start 2024-09-21T06:51:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc3419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc340760a0 0x7efc3419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc34110120 0x7efc341a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efc34115c90 con 0x7efc34075750 2024-09-21T06:51:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc34115b10 con 0x7efc34110120 2024-09-21T06:51:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc3ba00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efc34115e10 con 0x7efc340760a0 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc39000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc3419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc39000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc3419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:57800/0 (socket says 172.21.15.114:57800) 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.788+0000 7efc39000640 1 -- 172.21.15.114:0/3791474098 learned_addr learned my addr 172.21.15.114:0/3791474098 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc34110120 0x7efc341a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc33e00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc340760a0 0x7efc3419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 -- 172.21.15.114:0/3791474098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc340760a0 msgr2=0x7efc3419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc340760a0 0x7efc3419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 -- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 msgr2=0x7efc3419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:53.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc3419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 -- 172.21.15.114:0/3791474098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc341a25f0 con 0x7efc34110120 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc39a00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc34110120 0x7efc341a1ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00e970 tx=0x7efc2c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.789+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00ccc0 con 0x7efc34110120 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.790+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc2c005020 con 0x7efc34110120 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.790+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc341a28e0 con 0x7efc34110120 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.790+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004a00 con 0x7efc34110120 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.790+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a9040 con 0x7efc34110120 2024-09-21T06:51:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.791+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc34075b60 con 0x7efc34110120 2024-09-21T06:51:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.793+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7efc2c004ba0 con 0x7efc34110120 2024-09-21T06:51:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.795+0000 7efc31600640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efc180b65f0 0x7efc180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.795+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7efc2c0e7c40 con 0x7efc34110120 2024-09-21T06:51:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.795+0000 7efc39000640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efc180b65f0 0x7efc180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.795+0000 7efc39000640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efc180b65f0 0x7efc180b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc24004af0 tx=0x7efc24009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:53.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:53.797+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efc2c0adef0 con 0x7efc34110120 2024-09-21T06:51:53.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:53 smithi193 ceph-mon[34601]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:54.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.008+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efc34102ea0 con 0x7efc180b65f0 2024-09-21T06:51:54.010 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:53 smithi114 ceph-mon[30654]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:54.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.042+0000 7efc31600640 1 -- 172.21.15.114:0/3791474098 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc34102ea0 con 0x7efc180b65f0 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efc180b65f0 msgr2=0x7efc180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efc180b65f0 0x7efc180b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc24004af0 tx=0x7efc24009360 comp rx=0 tx=0).stop 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc34110120 msgr2=0x7efc341a1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc34110120 0x7efc341a1ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00e970 tx=0x7efc2c00ee30 comp rx=0 tx=0).stop 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 shutdown_connections 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efc180b65f0 0x7efc180b8b00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc34110120 0x7efc341a1ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc340760a0 0x7efc3419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 --2- 172.21.15.114:0/3791474098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efc34075750 0x7efc3419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:54.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 >> 172.21.15.114:0/3791474098 conn(0x7efc3407b850 msgr2=0x7efc34101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:54.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 shutdown_connections 2024-09-21T06:51:54.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:54.044+0000 7efc3ba00640 1 -- 172.21.15.114:0/3791474098 wait complete. 2024-09-21T06:51:54.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:53 smithi183 ceph-mon[34219]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:54.778 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:54 smithi114 ceph-mon[30654]: pgmap v327: 129 pgs: 129 active+clean; 594 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.6 KiB/s wr, 22 op/s 2024-09-21T06:51:54.819 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T06:51:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:54 smithi193 ceph-mon[34601]: pgmap v327: 129 pgs: 129 active+clean; 594 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.6 KiB/s wr, 22 op/s 2024-09-21T06:51:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:54 smithi183 ceph-mon[34219]: pgmap v327: 129 pgs: 129 active+clean; 594 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.6 KiB/s wr, 22 op/s 2024-09-21T06:51:55.184 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 -- 172.21.15.114:0/2460331024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac105d70 msgr2=0x7f72ac106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 --2- 172.21.15.114:0/2460331024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac105d70 0x7f72ac106180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f729c009900 tx=0x7f729c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 -- 172.21.15.114:0/2460331024 shutdown_connections 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 --2- 172.21.15.114:0/2460331024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac10c000 0x7f72ac112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 --2- 172.21.15.114:0/2460331024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72ac10b6b0 0x7f72ac10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 --2- 172.21.15.114:0/2460331024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac105d70 0x7f72ac106180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:55.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 -- 172.21.15.114:0/2460331024 >> 172.21.15.114:0/2460331024 conn(0x7f72ac101800 msgr2=0x7f72ac103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:55.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 -- 172.21.15.114:0/2460331024 shutdown_connections 2024-09-21T06:51:55.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.869+0000 7f72b0e00640 1 -- 172.21.15.114:0/2460331024 wait complete. 2024-09-21T06:51:55.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.870+0000 7f72b0e00640 1 Processor -- start 2024-09-21T06:51:55.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.870+0000 7f72b0e00640 1 -- start start 2024-09-21T06:51:55.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72b0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 0x7f72ac19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:55.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72b0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 0x7f72ac19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:55.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72b0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72ac10c000 0x7f72ac1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:55.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72b0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac115cc0 con 0x7f72ac10b6b0 2024-09-21T06:51:55.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72b0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac115b40 con 0x7f72ac10c000 2024-09-21T06:51:55.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72b0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac115e40 con 0x7f72ac105d70 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 0x7f72ac19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72a1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 0x7f72ac19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72a1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 0x7f72ac19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60356/0 (socket says 172.21.15.114:60356) 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72aa800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72ac10c000 0x7f72ac1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72a1400640 1 -- 172.21.15.114:0/1684266031 learned_addr learned my addr 172.21.15.114:0/1684266031 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.871+0000 7f72a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 0x7f72ac19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59468/0 (socket says 172.21.15.114:59468) 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a9e00640 1 -- 172.21.15.114:0/1684266031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72ac10c000 msgr2=0x7f72ac1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:55.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a9e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72ac10c000 0x7f72ac1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a9e00640 1 -- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 msgr2=0x7f72ac19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a9e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 0x7f72ac19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a9e00640 1 -- 172.21.15.114:0/1684266031 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac1a2640 con 0x7f72ac105d70 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a1400640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 0x7f72ac19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a9e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 0x7f72ac19c6d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f729c02a910 tx=0x7f729c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c03f030 con 0x7f72ac105d70 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f729c004490 con 0x7f72ac105d70 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac1a28d0 con 0x7f72ac105d70 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.872+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c035070 con 0x7f72ac105d70 2024-09-21T06:51:55.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.873+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a9090 con 0x7f72ac105d70 2024-09-21T06:51:55.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.874+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac106180 con 0x7f72ac105d70 2024-09-21T06:51:55.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.876+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f729c042030 con 0x7f72ac105d70 2024-09-21T06:51:55.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.877+0000 7f72a3400640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72900b6670 0x7f72900b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:55.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.878+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f729c107db0 con 0x7f72ac105d70 2024-09-21T06:51:55.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.878+0000 7f72a1400640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72900b6670 0x7f72900b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:55.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.878+0000 7f72a1400640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72900b6670 0x7f72900b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f72ac10b070 tx=0x7f7294005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:55.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:55.880+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f729c0d2440 con 0x7f72ac105d70 2024-09-21T06:51:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:55 smithi193 ceph-mon[34601]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:56.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:55 smithi114 ceph-mon[30654]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:56.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.093+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f72ac063800 con 0x7f72900b6670 2024-09-21T06:51:56.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.098+0000 7f72a3400640 1 -- 172.21.15.114:0/1684266031 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f72ac063800 con 0x7f72900b6670 2024-09-21T06:51:56.099 INFO:teuthology.orchestra.run.smithi114.stdout:/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644 2024-09-21T06:51:56.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72900b6670 msgr2=0x7f72900b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:56.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72900b6670 0x7f72900b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f72ac10b070 tx=0x7f7294005f90 comp rx=0 tx=0).stop 2024-09-21T06:51:56.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 msgr2=0x7f72ac19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:56.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 0x7f72ac19c6d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f729c02a910 tx=0x7f729c031a20 comp rx=0 tx=0).stop 2024-09-21T06:51:56.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 shutdown_connections 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72900b6670 0x7f72900b8b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72ac10c000 0x7f72ac1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72ac10b6b0 0x7f72ac19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 --2- 172.21.15.114:0/1684266031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72ac105d70 0x7f72ac19c6d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.100+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 >> 172.21.15.114:0/1684266031 conn(0x7f72ac101800 msgr2=0x7f72ac103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.101+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 shutdown_connections 2024-09-21T06:51:56.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:56.101+0000 7f72b0e00640 1 -- 172.21.15.114:0/1684266031 wait complete. 2024-09-21T06:51:56.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:55 smithi183 ceph-mon[34219]: from='client.24953 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:56.774 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:56 smithi114 ceph-mon[30654]: pgmap v328: 129 pgs: 129 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 20 op/s 2024-09-21T06:51:56.872 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:51:56.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:56 smithi193 ceph-mon[34601]: pgmap v328: 129 pgs: 129 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 20 op/s 2024-09-21T06:51:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:56 smithi183 ceph-mon[34219]: pgmap v328: 129 pgs: 129 active+clean; 630 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 20 op/s 2024-09-21T06:51:57.255 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:51:57.910 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:57 smithi114 ceph-mon[30654]: from='client.24964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:57.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 -- 172.21.15.114:0/2381574020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c109c40 msgr2=0x7fcb4c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:57.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2381574020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c109c40 0x7fcb4c110470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009ca0 tx=0x7fcb4402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:51:57.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 -- 172.21.15.114:0/2381574020 shutdown_connections 2024-09-21T06:51:57.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2381574020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c109c40 0x7fcb4c110470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:57.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2381574020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c1092f0 0x7fcb4c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:57.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2381574020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:57.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 -- 172.21.15.114:0/2381574020 >> 172.21.15.114:0/2381574020 conn(0x7fcb4c0ff440 msgr2=0x7fcb4c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:57.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 -- 172.21.15.114:0/2381574020 shutdown_connections 2024-09-21T06:51:57.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.978+0000 7fcb4be00640 1 -- 172.21.15.114:0/2381574020 wait complete. 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.979+0000 7fcb4be00640 1 Processor -- start 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.979+0000 7fcb4be00640 1 -- start start 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4c1092f0 0x7fcb4c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c109c40 0x7fcb4c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c113900 con 0x7fcb4c1039b0 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c113780 con 0x7fcb4c109c40 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4c113a80 con 0x7fcb4c1092f0 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60376/0 (socket says 172.21.15.114:60376) 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4aa00640 1 -- 172.21.15.114:0/2533576684 learned_addr learned my addr 172.21.15.114:0/2533576684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:51:57.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb43e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4c1092f0 0x7fcb4c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.980+0000 7fcb4b400640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c109c40 0x7fcb4c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4aa00640 1 -- 172.21.15.114:0/2533576684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4c1092f0 msgr2=0x7fcb4c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4aa00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4c1092f0 0x7fcb4c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4aa00640 1 -- 172.21.15.114:0/2533576684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c109c40 msgr2=0x7fcb4c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4aa00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c109c40 0x7fcb4c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4aa00640 1 -- 172.21.15.114:0/2533576684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4c1a8530 con 0x7fcb4c1039b0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4b400640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c109c40 0x7fcb4c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4aa00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c075710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00d950 tx=0x7fcb3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c017070 con 0x7fcb4c1039b0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c009c30 con 0x7fcb4c1039b0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4c1a8820 con 0x7fcb4c1039b0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c00f620 con 0x7fcb4c1039b0 2024-09-21T06:51:57.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.981+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4c1a8d60 con 0x7fcb4c1039b0 2024-09-21T06:51:57.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.983+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4c103dc0 con 0x7fcb4c1039b0 2024-09-21T06:51:57.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.985+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fcb3c010460 con 0x7fcb4c1039b0 2024-09-21T06:51:57.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.986+0000 7fcb43400640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcb300b65f0 0x7fcb300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:51:57.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.987+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fcb3c0e7590 con 0x7fcb4c1039b0 2024-09-21T06:51:57.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.987+0000 7fcb43e00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcb300b65f0 0x7fcb300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:51:57.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.987+0000 7fcb43e00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcb300b65f0 0x7fcb300b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c102d90 tx=0x7fcb3400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:51:57.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:57.991+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb3c0ad8a0 con 0x7fcb4c1039b0 2024-09-21T06:51:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:57 smithi183 ceph-mon[34219]: from='client.24964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:57 smithi193 ceph-mon[34601]: from='client.24964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:51:58.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.214+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb4c02e640 con 0x7fcb4c1039b0 2024-09-21T06:51:58.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.215+0000 7fcb43400640 1 -- 172.21.15.114:0/2533576684 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7fcb3c009de0 con 0x7fcb4c1039b0 2024-09-21T06:51:58.217 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:51:58.217 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"btime":"2024-09-21T06:51:45:610909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:45.336678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:51:58.218 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcb300b65f0 msgr2=0x7fcb300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcb300b65f0 0x7fcb300b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c102d90 tx=0x7fcb3400b390 comp rx=0 tx=0).stop 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 msgr2=0x7fcb4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c075710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00d950 tx=0x7fcb3c00de10 comp rx=0 tx=0).stop 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 shutdown_connections 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcb300b65f0 0x7fcb300b8b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb4c109c40 0x7fcb4c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb4c1092f0 0x7fcb4c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 --2- 172.21.15.114:0/2533576684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcb4c1039b0 0x7fcb4c075710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:51:58.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 >> 172.21.15.114:0/2533576684 conn(0x7fcb4c0ff440 msgr2=0x7fcb4c106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:51:58.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 shutdown_connections 2024-09-21T06:51:58.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:51:58.218+0000 7fcb4be00640 1 -- 172.21.15.114:0/2533576684 wait complete. 2024-09-21T06:51:59.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:58 smithi114 ceph-mon[30654]: pgmap v329: 129 pgs: 129 active+clean; 631 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-21T06:51:59.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:58 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2533576684' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:58 smithi183 ceph-mon[34219]: pgmap v329: 129 pgs: 129 active+clean; 631 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-21T06:51:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:58 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2533576684' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:58 smithi193 ceph-mon[34601]: pgmap v329: 129 pgs: 129 active+clean; 631 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-21T06:51:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:58 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2533576684' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:51:59.231 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:51:59.599 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:00.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:51:59 smithi114 ceph-mon[30654]: pgmap v330: 129 pgs: 129 active+clean; 631 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-21T06:52:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:51:59 smithi183 ceph-mon[34219]: pgmap v330: 129 pgs: 129 active+clean; 631 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-21T06:52:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:51:59 smithi193 ceph-mon[34601]: pgmap v330: 129 pgs: 129 active+clean; 631 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-21T06:52:00.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 -- 172.21.15.114:0/1951888912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 msgr2=0x7f80cc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:00.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 --2- 172.21.15.114:0/1951888912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc103f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009900 tx=0x7f80c002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 -- 172.21.15.114:0/1951888912 shutdown_connections 2024-09-21T06:52:00.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 --2- 172.21.15.114:0/1951888912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 0x7f80cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 --2- 172.21.15.114:0/1951888912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80cc1094a0 0x7f80cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 --2- 172.21.15.114:0/1951888912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc103f70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 -- 172.21.15.114:0/1951888912 >> 172.21.15.114:0/1951888912 conn(0x7f80cc07b850 msgr2=0x7f80cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:00.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 -- 172.21.15.114:0/1951888912 shutdown_connections 2024-09-21T06:52:00.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.275+0000 7f80d4600640 1 -- 172.21.15.114:0/1951888912 wait complete. 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.276+0000 7f80d4600640 1 Processor -- start 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 -- start start 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80cc1094a0 0x7f80cc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 0x7f80cc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc113b00 con 0x7f80cc103b60 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc113980 con 0x7f80cc1094a0 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d4600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f80cc113c80 con 0x7f80cc109df0 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60394/0 (socket says 172.21.15.114:60394) 2024-09-21T06:52:00.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d1c00640 1 -- 172.21.15.114:0/3713991836 learned_addr learned my addr 172.21.15.114:0/3713991836 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:00.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d2600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 0x7f80cc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:00.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.277+0000 7f80d1200640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80cc1094a0 0x7f80cc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:00.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d1c00640 1 -- 172.21.15.114:0/3713991836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 msgr2=0x7f80cc076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:00.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d1c00640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 0x7f80cc076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d1c00640 1 -- 172.21.15.114:0/3713991836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80cc1094a0 msgr2=0x7f80cc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d1c00640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80cc1094a0 0x7f80cc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d1c00640 1 -- 172.21.15.114:0/3713991836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80cc1a8810 con 0x7f80cc103b60 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d2600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 0x7f80cc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.278+0000 7f80d1c00640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005ab0 tx=0x7f80c0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.279+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c003f070 con 0x7f80cc103b60 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.279+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80cc1a8aa0 con 0x7f80cc103b60 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.279+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0031d60 con 0x7f80cc103b60 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.279+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0035030 con 0x7f80cc103b60 2024-09-21T06:52:00.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.279+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80cc1a8f80 con 0x7f80cc103b60 2024-09-21T06:52:00.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.280+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80cc103f70 con 0x7f80cc103b60 2024-09-21T06:52:00.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.282+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f80c0042070 con 0x7f80cc103b60 2024-09-21T06:52:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.283+0000 7f80ca000640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80b40b65f0 0x7f80b40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.284+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f80c0108750 con 0x7f80cc103b60 2024-09-21T06:52:00.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.284+0000 7f80d1200640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80b40b65f0 0x7f80b40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.284+0000 7f80d1200640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80b40b65f0 0x7f80b40b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80cc1091d0 tx=0x7f80b800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:00.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.286+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80c00d2de0 con 0x7f80cc103b60 2024-09-21T06:52:00.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.475+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80cc02e640 con 0x7f80cc103b60 2024-09-21T06:52:00.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.476+0000 7f80ca000640 1 -- 172.21.15.114:0/3713991836 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+25090 (secure 0 0 0) 0x7f80c00d7d20 con 0x7f80cc103b60 2024-09-21T06:52:00.478 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:00.479 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":87,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:30.657710+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80b40b65f0 msgr2=0x7f80b40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80b40b65f0 0x7f80b40b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80cc1091d0 tx=0x7f80b800b390 comp rx=0 tx=0).stop 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 msgr2=0x7f80cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f80c0005ab0 tx=0x7f80c0003c80 comp rx=0 tx=0).stop 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 shutdown_connections 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80cc109df0 0x7f80cc076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80b40b65f0 0x7f80b40b8b00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80cc1094a0 0x7f80cc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 --2- 172.21.15.114:0/3713991836 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80cc103b60 0x7f80cc075710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 >> 172.21.15.114:0/3713991836 conn(0x7f80cc07b850 msgr2=0x7f80cc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:00.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 shutdown_connections 2024-09-21T06:52:00.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:00.480+0000 7f80d4600640 1 -- 172.21.15.114:0/3713991836 wait complete. 2024-09-21T06:52:00.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:00 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3713991836' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:00 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3713991836' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:00 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3713991836' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:01.372 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:52:01.372 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:52:01.744 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:02.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:01 smithi114 ceph-mon[30654]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-09-21T06:52:02.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:01 smithi183 ceph-mon[34219]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-09-21T06:52:02.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:01 smithi193 ceph-mon[34601]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-09-21T06:52:02.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 -- 172.21.15.114:0/3881975072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 msgr2=0x7f45d4112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:02.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 --2- 172.21.15.114:0/3881975072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d4112870 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009ca0 tx=0x7f45cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 -- 172.21.15.114:0/3881975072 shutdown_connections 2024-09-21T06:52:02.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 --2- 172.21.15.114:0/3881975072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d4112870 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 --2- 172.21.15.114:0/3881975072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d410b6f0 0x7f45d410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 --2- 172.21.15.114:0/3881975072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d4105db0 0x7f45d41061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 -- 172.21.15.114:0/3881975072 >> 172.21.15.114:0/3881975072 conn(0x7f45d41017e0 msgr2=0x7f45d4103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:02.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 -- 172.21.15.114:0/3881975072 shutdown_connections 2024-09-21T06:52:02.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.402+0000 7f45d9e00640 1 -- 172.21.15.114:0/3881975072 wait complete. 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.403+0000 7f45d9e00640 1 Processor -- start 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 -- start start 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d4105db0 0x7f45d419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d410b6f0 0x7f45d419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d41a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4115d00 con 0x7f45d410c040 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4115b80 con 0x7f45d4105db0 2024-09-21T06:52:02.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d9e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4115e80 con 0x7f45d410b6f0 2024-09-21T06:52:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d41a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d41a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60418/0 (socket says 172.21.15.114:60418) 2024-09-21T06:52:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.404+0000 7f45d3400640 1 -- 172.21.15.114:0/4170331975 learned_addr learned my addr 172.21.15.114:0/4170331975 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d2a00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d4105db0 0x7f45d419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:02.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45ca000640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d410b6f0 0x7f45d419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d3400640 1 -- 172.21.15.114:0/4170331975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d410b6f0 msgr2=0x7f45d419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d3400640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d410b6f0 0x7f45d419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d3400640 1 -- 172.21.15.114:0/4170331975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d4105db0 msgr2=0x7f45d419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d3400640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d4105db0 0x7f45d419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d3400640 1 -- 172.21.15.114:0/4170331975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d41a2620 con 0x7f45d410c040 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.405+0000 7f45d3400640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d41a1ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005cc0 tx=0x7f45cc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.406+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc003bf0 con 0x7f45d410c040 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.406+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45cc003d90 con 0x7f45d410c040 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.406+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d41a28b0 con 0x7f45d410c040 2024-09-21T06:52:02.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.406+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d41a9070 con 0x7f45d410c040 2024-09-21T06:52:02.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.406+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc02fbc0 con 0x7f45d410c040 2024-09-21T06:52:02.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.407+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d41061c0 con 0x7f45d410c040 2024-09-21T06:52:02.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.409+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f45cc02fd60 con 0x7f45d410c040 2024-09-21T06:52:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.411+0000 7f45d8a00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45b80b65f0 0x7f45b80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.411+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f45cc108ff0 con 0x7f45d410c040 2024-09-21T06:52:02.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.411+0000 7f45d2a00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45b80b65f0 0x7f45b80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:02.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.412+0000 7f45d2a00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45b80b65f0 0x7f45b80b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45c400bbe0 tx=0x7f45c400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:02.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.413+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45cc083710 con 0x7f45d410c040 2024-09-21T06:52:02.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.597+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45d4063800 con 0x7f45b80b65f0 2024-09-21T06:52:02.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.604+0000 7f45d8a00640 1 -- 172.21.15.114:0/4170331975 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340382 (secure 0 0 0) 0x7f45d4063800 con 0x7f45b80b65f0 2024-09-21T06:52:02.608 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:02.612 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.609+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45b80b65f0 msgr2=0x7f45b80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.609+0000 7f45d9e00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45b80b65f0 0x7f45b80b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45c400bbe0 tx=0x7f45c400b390 comp rx=0 tx=0).stop 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.609+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 msgr2=0x7f45d41a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.609+0000 7f45d9e00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d41a1ef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f45cc005cc0 tx=0x7f45cc02f380 comp rx=0 tx=0).stop 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 shutdown_connections 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45d410c040 0x7f45d41a1ef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d410b6f0 0x7f45d419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45b80b65f0 0x7f45b80b8b00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 --2- 172.21.15.114:0/4170331975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d4105db0 0x7f45d419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:02.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 >> 172.21.15.114:0/4170331975 conn(0x7f45d41017e0 msgr2=0x7f45d4108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:02.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 shutdown_connections 2024-09-21T06:52:02.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:02.610+0000 7f45d9e00640 1 -- 172.21.15.114:0/4170331975 wait complete. 2024-09-21T06:52:03.042 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:02 smithi114 ceph-mon[30654]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:02 smithi183 ceph-mon[34219]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:02 smithi193 ceph-mon[34601]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:03.523 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-21T06:52:01.507262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654746,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":273,"num_read_kb":236,"num_write":365,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371448,"kb_used_data":13880,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570376,"statfs":{"total":1151940427776,"available":1151560065024,"internally_reserved":0,"allocated":14213120,"data_stored":8075660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53723,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":118,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004020"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:55.643475+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:51:55.643475+0000","last_peered":"2024-09-21T06:51:55.643475+0000","last_clean":"2024-09-21T06:51:55.643475+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:51:55.643475+0000","last_undegraded":"2024-09-21T06:51:55.643475+0000","last_fullsized":"2024-09-21T06:51:55.643475+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662712+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:51:30.662712+0000","last_peered":"2024-09-21T06:51:30.662712+0000","last_clean":"2024-09-21T06:51:30.662712+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:51:30.662712+0000","last_undegraded":"2024-09-21T06:51:30.662712+0000","last_fullsized":"2024-09-21T06:51:30.662712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669858+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:51:30.669858+0000","last_peered":"2024-09-21T06:51:30.669858+0000","last_clean":"2024-09-21T06:51:30.669858+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:51:30.669858+0000","last_undegraded":"2024-09-21T06:51:30.669858+0000","last_fullsized":"2024-09-21T06:51:30.669858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663062+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:51:30.663062+0000","last_peered":"2024-09-21T06:51:30.663062+0000","last_clean":"2024-09-21T06:51:30.663062+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:51:30.663062+0000","last_undegraded":"2024-09-21T06:51:30.663062+0000","last_fullsized":"2024-09-21T06:51:30.663062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666417+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:51:30.666417+0000","last_peered":"2024-09-21T06:51:30.666417+0000","last_clean":"2024-09-21T06:51:30.666417+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:51:30.666417+0000","last_undegraded":"2024-09-21T06:51:30.666417+0000","last_fullsized":"2024-09-21T06:51:30.666417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.755038+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:51:30.755038+0000","last_peered":"2024-09-21T06:51:30.755038+0000","last_clean":"2024-09-21T06:51:30.755038+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:51:30.755038+0000","last_undegraded":"2024-09-21T06:51:30.755038+0000","last_fullsized":"2024-09-21T06:51:30.755038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:43.621858+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:51:43.621858+0000","last_peered":"2024-09-21T06:51:43.621858+0000","last_clean":"2024-09-21T06:51:43.621858+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:51:43.621858+0000","last_undegraded":"2024-09-21T06:51:43.621858+0000","last_fullsized":"2024-09-21T06:51:43.621858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667290+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:51:30.667290+0000","last_peered":"2024-09-21T06:51:30.667290+0000","last_clean":"2024-09-21T06:51:30.667290+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:51:30.667290+0000","last_undegraded":"2024-09-21T06:51:30.667290+0000","last_fullsized":"2024-09-21T06:51:30.667290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662767+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:51:30.662767+0000","last_peered":"2024-09-21T06:51:30.662767+0000","last_clean":"2024-09-21T06:51:30.662767+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:51:30.662767+0000","last_undegraded":"2024-09-21T06:51:30.662767+0000","last_fullsized":"2024-09-21T06:51:30.662767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.610291+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:51:42.610291+0000","last_peered":"2024-09-21T06:51:42.610291+0000","last_clean":"2024-09-21T06:51:42.610291+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:51:42.610291+0000","last_undegraded":"2024-09-21T06:51:42.610291+0000","last_fullsized":"2024-09-21T06:51:42.610291+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.664316+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:51:30.664316+0000","last_peered":"2024-09-21T06:51:30.664316+0000","last_clean":"2024-09-21T06:51:30.664316+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:51:30.664316+0000","last_undegraded":"2024-09-21T06:51:30.664316+0000","last_fullsized":"2024-09-21T06:51:30.664316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662372+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:51:30.662372+0000","last_peered":"2024-09-21T06:51:30.662372+0000","last_clean":"2024-09-21T06:51:30.662372+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:51:30.662372+0000","last_undegraded":"2024-09-21T06:51:30.662372+0000","last_fullsized":"2024-09-21T06:51:30.662372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669079+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:51:30.669079+0000","last_peered":"2024-09-21T06:51:30.669079+0000","last_clean":"2024-09-21T06:51:30.669079+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:51:30.669079+0000","last_undegraded":"2024-09-21T06:51:30.669079+0000","last_fullsized":"2024-09-21T06:51:30.669079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:39.580312+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:51:39.580312+0000","last_peered":"2024-09-21T06:51:39.580312+0000","last_clean":"2024-09-21T06:51:39.580312+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:51:39.580312+0000","last_undegraded":"2024-09-21T06:51:39.580312+0000","last_fullsized":"2024-09-21T06:51:39.580312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662651+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:51:30.662651+0000","last_peered":"2024-09-21T06:51:30.662651+0000","last_clean":"2024-09-21T06:51:30.662651+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:51:30.662651+0000","last_undegraded":"2024-09-21T06:51:30.662651+0000","last_fullsized":"2024-09-21T06:51:30.662651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662201+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:51:30.662201+0000","last_peered":"2024-09-21T06:51:30.662201+0000","last_clean":"2024-09-21T06:51:30.662201+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:51:30.662201+0000","last_undegraded":"2024-09-21T06:51:30.662201+0000","last_fullsized":"2024-09-21T06:51:30.662201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:41.598608+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:51:41.598608+0000","last_peered":"2024-09-21T06:51:41.598608+0000","last_clean":"2024-09-21T06:51:41.598608+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:51:41.598608+0000","last_undegraded":"2024-09-21T06:51:41.598608+0000","last_fullsized":"2024-09-21T06:51:41.598608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662784+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:51:30.662784+0000","last_peered":"2024-09-21T06:51:30.662784+0000","last_clean":"2024-09-21T06:51:30.662784+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:51:30.662784+0000","last_undegraded":"2024-09-21T06:51:30.662784+0000","last_fullsized":"2024-09-21T06:51:30.662784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666991+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:51:30.666991+0000","last_peered":"2024-09-21T06:51:30.666991+0000","last_clean":"2024-09-21T06:51:30.666991+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:51:30.666991+0000","last_undegraded":"2024-09-21T06:51:30.666991+0000","last_fullsized":"2024-09-21T06:51:30.666991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.614614+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:51:42.614614+0000","last_peered":"2024-09-21T06:51:42.614614+0000","last_clean":"2024-09-21T06:51:42.614614+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:51:42.614614+0000","last_undegraded":"2024-09-21T06:51:42.614614+0000","last_fullsized":"2024-09-21T06:51:42.614614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921785+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:51:30.921785+0000","last_peered":"2024-09-21T06:51:30.921785+0000","last_clean":"2024-09-21T06:51:30.921785+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:51:30.921785+0000","last_undegraded":"2024-09-21T06:51:30.921785+0000","last_fullsized":"2024-09-21T06:51:30.921785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665136+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:51:30.665136+0000","last_peered":"2024-09-21T06:51:30.665136+0000","last_clean":"2024-09-21T06:51:30.665136+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:51:30.665136+0000","last_undegraded":"2024-09-21T06:51:30.665136+0000","last_fullsized":"2024-09-21T06:51:30.665136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:39.579256+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:51:39.579256+0000","last_peered":"2024-09-21T06:51:39.579256+0000","last_clean":"2024-09-21T06:51:39.579256+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:51:39.579256+0000","last_undegraded":"2024-09-21T06:51:39.579256+0000","last_fullsized":"2024-09-21T06:51:39.579256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:51:30.663150+0000","last_peered":"2024-09-21T06:51:30.663150+0000","last_clean":"2024-09-21T06:51:30.663150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:51:30.663150+0000","last_undegraded":"2024-09-21T06:51:30.663150+0000","last_fullsized":"2024-09-21T06:51:30.663150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669400+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:51:30.669400+0000","last_peered":"2024-09-21T06:51:30.669400+0000","last_clean":"2024-09-21T06:51:30.669400+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:51:30.669400+0000","last_undegraded":"2024-09-21T06:51:30.669400+0000","last_fullsized":"2024-09-21T06:51:30.669400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.612142+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:51:42.612142+0000","last_peered":"2024-09-21T06:51:42.612142+0000","last_clean":"2024-09-21T06:51:42.612142+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:51:42.612142+0000","last_undegraded":"2024-09-21T06:51:42.612142+0000","last_fullsized":"2024-09-21T06:51:42.612142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670021+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:51:30.670021+0000","last_peered":"2024-09-21T06:51:30.670021+0000","last_clean":"2024-09-21T06:51:30.670021+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:51:30.670021+0000","last_undegraded":"2024-09-21T06:51:30.670021+0000","last_fullsized":"2024-09-21T06:51:30.670021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663022+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:51:30.663022+0000","last_peered":"2024-09-21T06:51:30.663022+0000","last_clean":"2024-09-21T06:51:30.663022+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:51:30.663022+0000","last_undegraded":"2024-09-21T06:51:30.663022+0000","last_fullsized":"2024-09-21T06:51:30.663022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:54.233503+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:51:54.233503+0000","last_peered":"2024-09-21T06:51:54.233503+0000","last_clean":"2024-09-21T06:51:54.233503+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:51:54.233503+0000","last_undegraded":"2024-09-21T06:51:54.233503+0000","last_fullsized":"2024-09-21T06:51:54.233503+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.661943+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:51:30.661943+0000","last_peered":"2024-09-21T06:51:30.661943+0000","last_clean":"2024-09-21T06:51:30.661943+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:51:30.661943+0000","last_undegraded":"2024-09-21T06:51:30.661943+0000","last_fullsized":"2024-09-21T06:51:30.661943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752802+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:51:30.752802+0000","last_peered":"2024-09-21T06:51:30.752802+0000","last_clean":"2024-09-21T06:51:30.752802+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:51:30.752802+0000","last_undegraded":"2024-09-21T06:51:30.752802+0000","last_fullsized":"2024-09-21T06:51:30.752802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.614596+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:51:42.614596+0000","last_peered":"2024-09-21T06:51:42.614596+0000","last_clean":"2024-09-21T06:51:42.614596+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:51:42.614596+0000","last_undegraded":"2024-09-21T06:51:42.614596+0000","last_fullsized":"2024-09-21T06:51:42.614596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666136+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:51:30.666136+0000","last_peered":"2024-09-21T06:51:30.666136+0000","last_clean":"2024-09-21T06:51:30.666136+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:51:30.666136+0000","last_undegraded":"2024-09-21T06:51:30.666136+0000","last_fullsized":"2024-09-21T06:51:30.666136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665200+0000","last_change":"2024-09-21T06:50:47.222453+0000","last_active":"2024-09-21T06:51:30.665200+0000","last_peered":"2024-09-21T06:51:30.665200+0000","last_clean":"2024-09-21T06:51:30.665200+0000","last_became_active":"2024-09-21T06:50:47.222057+0000","last_became_peered":"2024-09-21T06:50:47.222057+0000","last_unstale":"2024-09-21T06:51:30.665200+0000","last_undegraded":"2024-09-21T06:51:30.665200+0000","last_fullsized":"2024-09-21T06:51:30.665200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:01.741607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921450+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:51:30.921450+0000","last_peered":"2024-09-21T06:51:30.921450+0000","last_clean":"2024-09-21T06:51:30.921450+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:51:30.921450+0000","last_undegraded":"2024-09-21T06:51:30.921450+0000","last_fullsized":"2024-09-21T06:51:30.921450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665260+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:51:30.665260+0000","last_peered":"2024-09-21T06:51:30.665260+0000","last_clean":"2024-09-21T06:51:30.665260+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:51:30.665260+0000","last_undegraded":"2024-09-21T06:51:30.665260+0000","last_fullsized":"2024-09-21T06:51:30.665260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666702+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:51:30.666702+0000","last_peered":"2024-09-21T06:51:30.666702+0000","last_clean":"2024-09-21T06:51:30.666702+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:51:30.666702+0000","last_undegraded":"2024-09-21T06:51:30.666702+0000","last_fullsized":"2024-09-21T06:51:30.666702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:57.646762+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:51:57.646762+0000","last_peered":"2024-09-21T06:51:57.646762+0000","last_clean":"2024-09-21T06:51:57.646762+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:51:57.646762+0000","last_undegraded":"2024-09-21T06:51:57.646762+0000","last_fullsized":"2024-09-21T06:51:57.646762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752442+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:51:30.752442+0000","last_peered":"2024-09-21T06:51:30.752442+0000","last_clean":"2024-09-21T06:51:30.752442+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:51:30.752442+0000","last_undegraded":"2024-09-21T06:51:30.752442+0000","last_fullsized":"2024-09-21T06:51:30.752442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663244+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:51:30.663244+0000","last_peered":"2024-09-21T06:51:30.663244+0000","last_clean":"2024-09-21T06:51:30.663244+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:51:30.663244+0000","last_undegraded":"2024-09-21T06:51:30.663244+0000","last_fullsized":"2024-09-21T06:51:30.663244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:55.642717+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:51:55.642717+0000","last_peered":"2024-09-21T06:51:55.642717+0000","last_clean":"2024-09-21T06:51:55.642717+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:51:55.642717+0000","last_undegraded":"2024-09-21T06:51:55.642717+0000","last_fullsized":"2024-09-21T06:51:55.642717+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752802+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:51:30.752802+0000","last_peered":"2024-09-21T06:51:30.752802+0000","last_clean":"2024-09-21T06:51:30.752802+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:51:30.752802+0000","last_undegraded":"2024-09-21T06:51:30.752802+0000","last_fullsized":"2024-09-21T06:51:30.752802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.751961+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:51:30.751961+0000","last_peered":"2024-09-21T06:51:30.751961+0000","last_clean":"2024-09-21T06:51:30.751961+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:51:30.751961+0000","last_undegraded":"2024-09-21T06:51:30.751961+0000","last_fullsized":"2024-09-21T06:51:30.751961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:40.589215+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:51:40.589215+0000","last_peered":"2024-09-21T06:51:40.589215+0000","last_clean":"2024-09-21T06:51:40.589215+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:51:40.589215+0000","last_undegraded":"2024-09-21T06:51:40.589215+0000","last_fullsized":"2024-09-21T06:51:40.589215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663611+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:51:30.663611+0000","last_peered":"2024-09-21T06:51:30.663611+0000","last_clean":"2024-09-21T06:51:30.663611+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:51:30.663611+0000","last_undegraded":"2024-09-21T06:51:30.663611+0000","last_fullsized":"2024-09-21T06:51:30.663611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662460+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:51:30.662460+0000","last_peered":"2024-09-21T06:51:30.662460+0000","last_clean":"2024-09-21T06:51:30.662460+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:51:30.662460+0000","last_undegraded":"2024-09-21T06:51:30.662460+0000","last_fullsized":"2024-09-21T06:51:30.662460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:43.621170+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:51:43.621170+0000","last_peered":"2024-09-21T06:51:43.621170+0000","last_clean":"2024-09-21T06:51:43.621170+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:51:43.621170+0000","last_undegraded":"2024-09-21T06:51:43.621170+0000","last_fullsized":"2024-09-21T06:51:43.621170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669035+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:51:30.669035+0000","last_peered":"2024-09-21T06:51:30.669035+0000","last_clean":"2024-09-21T06:51:30.669035+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:51:30.669035+0000","last_undegraded":"2024-09-21T06:51:30.669035+0000","last_fullsized":"2024-09-21T06:51:30.669035+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669753+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:51:30.669753+0000","last_peered":"2024-09-21T06:51:30.669753+0000","last_clean":"2024-09-21T06:51:30.669753+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:51:30.669753+0000","last_undegraded":"2024-09-21T06:51:30.669753+0000","last_fullsized":"2024-09-21T06:51:30.669753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669814+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:51:30.669814+0000","last_peered":"2024-09-21T06:51:30.669814+0000","last_clean":"2024-09-21T06:51:30.669814+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:51:30.669814+0000","last_undegraded":"2024-09-21T06:51:30.669814+0000","last_fullsized":"2024-09-21T06:51:30.669814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":61,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:55.643742+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:51:55.643742+0000","last_peered":"2024-09-21T06:51:55.643742+0000","last_clean":"2024-09-21T06:51:55.643742+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:51:55.643742+0000","last_undegraded":"2024-09-21T06:51:55.643742+0000","last_fullsized":"2024-09-21T06:51:55.643742+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.754336+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:51:30.754336+0000","last_peered":"2024-09-21T06:51:30.754336+0000","last_clean":"2024-09-21T06:51:30.754336+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:51:30.754336+0000","last_undegraded":"2024-09-21T06:51:30.754336+0000","last_fullsized":"2024-09-21T06:51:30.754336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669274+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:51:30.669274+0000","last_peered":"2024-09-21T06:51:30.669274+0000","last_clean":"2024-09-21T06:51:30.669274+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:51:30.669274+0000","last_undegraded":"2024-09-21T06:51:30.669274+0000","last_fullsized":"2024-09-21T06:51:30.669274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.613692+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:51:42.613692+0000","last_peered":"2024-09-21T06:51:42.613692+0000","last_clean":"2024-09-21T06:51:42.613692+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:51:42.613692+0000","last_undegraded":"2024-09-21T06:51:42.613692+0000","last_fullsized":"2024-09-21T06:51:42.613692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665284+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:51:30.665284+0000","last_peered":"2024-09-21T06:51:30.665284+0000","last_clean":"2024-09-21T06:51:30.665284+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:51:30.665284+0000","last_undegraded":"2024-09-21T06:51:30.665284+0000","last_fullsized":"2024-09-21T06:51:30.665284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669617+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:51:30.669617+0000","last_peered":"2024-09-21T06:51:30.669617+0000","last_clean":"2024-09-21T06:51:30.669617+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:51:30.669617+0000","last_undegraded":"2024-09-21T06:51:30.669617+0000","last_fullsized":"2024-09-21T06:51:30.669617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.612598+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:51:42.612598+0000","last_peered":"2024-09-21T06:51:42.612598+0000","last_clean":"2024-09-21T06:51:42.612598+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:51:42.612598+0000","last_undegraded":"2024-09-21T06:51:42.612598+0000","last_fullsized":"2024-09-21T06:51:42.612598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662666+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:51:30.662666+0000","last_peered":"2024-09-21T06:51:30.662666+0000","last_clean":"2024-09-21T06:51:30.662666+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:51:30.662666+0000","last_undegraded":"2024-09-21T06:51:30.662666+0000","last_fullsized":"2024-09-21T06:51:30.662666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666145+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:51:30.666145+0000","last_peered":"2024-09-21T06:51:30.666145+0000","last_clean":"2024-09-21T06:51:30.666145+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:51:30.666145+0000","last_undegraded":"2024-09-21T06:51:30.666145+0000","last_fullsized":"2024-09-21T06:51:30.666145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.584471+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:51:42.584471+0000","last_peered":"2024-09-21T06:51:42.584471+0000","last_clean":"2024-09-21T06:51:42.584471+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:51:42.584471+0000","last_undegraded":"2024-09-21T06:51:42.584471+0000","last_fullsized":"2024-09-21T06:51:42.584471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662770+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:51:30.662770+0000","last_peered":"2024-09-21T06:51:30.662770+0000","last_clean":"2024-09-21T06:51:30.662770+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:51:30.662770+0000","last_undegraded":"2024-09-21T06:51:30.662770+0000","last_fullsized":"2024-09-21T06:51:30.662770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662861+0000","last_change":"2024-09-21T06:50:47.224057+0000","last_active":"2024-09-21T06:51:30.662861+0000","last_peered":"2024-09-21T06:51:30.662861+0000","last_clean":"2024-09-21T06:51:30.662861+0000","last_became_active":"2024-09-21T06:50:47.223689+0000","last_became_peered":"2024-09-21T06:50:47.223689+0000","last_unstale":"2024-09-21T06:51:30.662861+0000","last_undegraded":"2024-09-21T06:51:30.662861+0000","last_fullsized":"2024-09-21T06:51:30.662861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:53.752342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.610626+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:51:42.610626+0000","last_peered":"2024-09-21T06:51:42.610626+0000","last_clean":"2024-09-21T06:51:42.610626+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:51:42.610626+0000","last_undegraded":"2024-09-21T06:51:42.610626+0000","last_fullsized":"2024-09-21T06:51:42.610626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752867+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:51:30.752867+0000","last_peered":"2024-09-21T06:51:30.752867+0000","last_clean":"2024-09-21T06:51:30.752867+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:51:30.752867+0000","last_undegraded":"2024-09-21T06:51:30.752867+0000","last_fullsized":"2024-09-21T06:51:30.752867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668680+0000","last_change":"2024-09-21T06:50:47.223163+0000","last_active":"2024-09-21T06:51:30.668680+0000","last_peered":"2024-09-21T06:51:30.668680+0000","last_clean":"2024-09-21T06:51:30.668680+0000","last_became_active":"2024-09-21T06:50:47.222742+0000","last_became_peered":"2024-09-21T06:50:47.222742+0000","last_unstale":"2024-09-21T06:51:30.668680+0000","last_undegraded":"2024-09-21T06:51:30.668680+0000","last_fullsized":"2024-09-21T06:51:30.668680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:17.570268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:59.649363+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:51:59.649363+0000","last_peered":"2024-09-21T06:51:59.649363+0000","last_clean":"2024-09-21T06:51:59.649363+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:51:59.649363+0000","last_undegraded":"2024-09-21T06:51:59.649363+0000","last_fullsized":"2024-09-21T06:51:59.649363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.672042+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:51:30.672042+0000","last_peered":"2024-09-21T06:51:30.672042+0000","last_clean":"2024-09-21T06:51:30.672042+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:51:30.672042+0000","last_undegraded":"2024-09-21T06:51:30.672042+0000","last_fullsized":"2024-09-21T06:51:30.672042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669840+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:51:30.669840+0000","last_peered":"2024-09-21T06:51:30.669840+0000","last_clean":"2024-09-21T06:51:30.669840+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:51:30.669840+0000","last_undegraded":"2024-09-21T06:51:30.669840+0000","last_fullsized":"2024-09-21T06:51:30.669840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.613738+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:51:42.613738+0000","last_peered":"2024-09-21T06:51:42.613738+0000","last_clean":"2024-09-21T06:51:42.613738+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:51:42.613738+0000","last_undegraded":"2024-09-21T06:51:42.613738+0000","last_fullsized":"2024-09-21T06:51:42.613738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666191+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:51:30.666191+0000","last_peered":"2024-09-21T06:51:30.666191+0000","last_clean":"2024-09-21T06:51:30.666191+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:51:30.666191+0000","last_undegraded":"2024-09-21T06:51:30.666191+0000","last_fullsized":"2024-09-21T06:51:30.666191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752529+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:51:30.752529+0000","last_peered":"2024-09-21T06:51:30.752529+0000","last_clean":"2024-09-21T06:51:30.752529+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:51:30.752529+0000","last_undegraded":"2024-09-21T06:51:30.752529+0000","last_fullsized":"2024-09-21T06:51:30.752529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:43.622643+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:51:43.622643+0000","last_peered":"2024-09-21T06:51:43.622643+0000","last_clean":"2024-09-21T06:51:43.622643+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:51:43.622643+0000","last_undegraded":"2024-09-21T06:51:43.622643+0000","last_fullsized":"2024-09-21T06:51:43.622643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.664269+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:51:30.664269+0000","last_peered":"2024-09-21T06:51:30.664269+0000","last_clean":"2024-09-21T06:51:30.664269+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:51:30.664269+0000","last_undegraded":"2024-09-21T06:51:30.664269+0000","last_fullsized":"2024-09-21T06:51:30.664269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667091+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:51:30.667091+0000","last_peered":"2024-09-21T06:51:30.667091+0000","last_clean":"2024-09-21T06:51:30.667091+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:51:30.667091+0000","last_undegraded":"2024-09-21T06:51:30.667091+0000","last_fullsized":"2024-09-21T06:51:30.667091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:58.649923+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:51:58.649923+0000","last_peered":"2024-09-21T06:51:58.649923+0000","last_clean":"2024-09-21T06:51:58.649923+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:51:58.649923+0000","last_undegraded":"2024-09-21T06:51:58.649923+0000","last_fullsized":"2024-09-21T06:51:58.649923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56549,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":74,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.673773+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:51:30.673773+0000","last_peered":"2024-09-21T06:51:30.673773+0000","last_clean":"2024-09-21T06:51:30.673773+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:51:30.673773+0000","last_undegraded":"2024-09-21T06:51:30.673773+0000","last_fullsized":"2024-09-21T06:51:30.673773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667452+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:51:30.667452+0000","last_peered":"2024-09-21T06:51:30.667452+0000","last_clean":"2024-09-21T06:51:30.667452+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:51:30.667452+0000","last_undegraded":"2024-09-21T06:51:30.667452+0000","last_fullsized":"2024-09-21T06:51:30.667452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:41.600537+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:51:41.600537+0000","last_peered":"2024-09-21T06:51:41.600537+0000","last_clean":"2024-09-21T06:51:41.600537+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:51:41.600537+0000","last_undegraded":"2024-09-21T06:51:41.600537+0000","last_fullsized":"2024-09-21T06:51:41.600537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662483+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:51:30.662483+0000","last_peered":"2024-09-21T06:51:30.662483+0000","last_clean":"2024-09-21T06:51:30.662483+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:51:30.662483+0000","last_undegraded":"2024-09-21T06:51:30.662483+0000","last_fullsized":"2024-09-21T06:51:30.662483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666598+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:51:30.666598+0000","last_peered":"2024-09-21T06:51:30.666598+0000","last_clean":"2024-09-21T06:51:30.666598+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:51:30.666598+0000","last_undegraded":"2024-09-21T06:51:30.666598+0000","last_fullsized":"2024-09-21T06:51:30.666598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.612148+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:51:42.612148+0000","last_peered":"2024-09-21T06:51:42.612148+0000","last_clean":"2024-09-21T06:51:42.612148+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:51:42.612148+0000","last_undegraded":"2024-09-21T06:51:42.612148+0000","last_fullsized":"2024-09-21T06:51:42.612148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.920845+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:51:30.920845+0000","last_peered":"2024-09-21T06:51:30.920845+0000","last_clean":"2024-09-21T06:51:30.920845+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:51:30.920845+0000","last_undegraded":"2024-09-21T06:51:30.920845+0000","last_fullsized":"2024-09-21T06:51:30.920845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668247+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:51:30.668247+0000","last_peered":"2024-09-21T06:51:30.668247+0000","last_clean":"2024-09-21T06:51:30.668247+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:51:30.668247+0000","last_undegraded":"2024-09-21T06:51:30.668247+0000","last_fullsized":"2024-09-21T06:51:30.668247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:58.647279+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:51:58.647279+0000","last_peered":"2024-09-21T06:51:58.647279+0000","last_clean":"2024-09-21T06:51:58.647279+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:51:58.647279+0000","last_undegraded":"2024-09-21T06:51:58.647279+0000","last_fullsized":"2024-09-21T06:51:58.647279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662156+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:51:30.662156+0000","last_peered":"2024-09-21T06:51:30.662156+0000","last_clean":"2024-09-21T06:51:30.662156+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:51:30.662156+0000","last_undegraded":"2024-09-21T06:51:30.662156+0000","last_fullsized":"2024-09-21T06:51:30.662156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665822+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:51:30.665822+0000","last_peered":"2024-09-21T06:51:30.665822+0000","last_clean":"2024-09-21T06:51:30.665822+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:51:30.665822+0000","last_undegraded":"2024-09-21T06:51:30.665822+0000","last_fullsized":"2024-09-21T06:51:30.665822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921124+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:51:30.921124+0000","last_peered":"2024-09-21T06:51:30.921124+0000","last_clean":"2024-09-21T06:51:30.921124+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:51:30.921124+0000","last_undegraded":"2024-09-21T06:51:30.921124+0000","last_fullsized":"2024-09-21T06:51:30.921124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.611622+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:51:42.611622+0000","last_peered":"2024-09-21T06:51:42.611622+0000","last_clean":"2024-09-21T06:51:42.611622+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:51:42.611622+0000","last_undegraded":"2024-09-21T06:51:42.611622+0000","last_fullsized":"2024-09-21T06:51:42.611622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667313+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:51:30.667313+0000","last_peered":"2024-09-21T06:51:30.667313+0000","last_clean":"2024-09-21T06:51:30.667313+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:51:30.667313+0000","last_undegraded":"2024-09-21T06:51:30.667313+0000","last_fullsized":"2024-09-21T06:51:30.667313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662532+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:51:30.662532+0000","last_peered":"2024-09-21T06:51:30.662532+0000","last_clean":"2024-09-21T06:51:30.662532+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:51:30.662532+0000","last_undegraded":"2024-09-21T06:51:30.662532+0000","last_fullsized":"2024-09-21T06:51:30.662532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:40.591629+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:51:40.591629+0000","last_peered":"2024-09-21T06:51:40.591629+0000","last_clean":"2024-09-21T06:51:40.591629+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:51:40.591629+0000","last_undegraded":"2024-09-21T06:51:40.591629+0000","last_fullsized":"2024-09-21T06:51:40.591629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:41.599791+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:51:41.599791+0000","last_peered":"2024-09-21T06:51:41.599791+0000","last_clean":"2024-09-21T06:51:41.599791+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:51:41.599791+0000","last_undegraded":"2024-09-21T06:51:41.599791+0000","last_fullsized":"2024-09-21T06:51:41.599791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663203+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:51:30.663203+0000","last_peered":"2024-09-21T06:51:30.663203+0000","last_clean":"2024-09-21T06:51:30.663203+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:51:30.663203+0000","last_undegraded":"2024-09-21T06:51:30.663203+0000","last_fullsized":"2024-09-21T06:51:30.663203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670164+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:51:30.670164+0000","last_peered":"2024-09-21T06:51:30.670164+0000","last_clean":"2024-09-21T06:51:30.670164+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:51:30.670164+0000","last_undegraded":"2024-09-21T06:51:30.670164+0000","last_fullsized":"2024-09-21T06:51:30.670164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:54.642275+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:51:54.642275+0000","last_peered":"2024-09-21T06:51:54.642275+0000","last_clean":"2024-09-21T06:51:54.642275+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:51:54.642275+0000","last_undegraded":"2024-09-21T06:51:54.642275+0000","last_fullsized":"2024-09-21T06:51:54.642275+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665423+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:51:30.665423+0000","last_peered":"2024-09-21T06:51:30.665423+0000","last_clean":"2024-09-21T06:51:30.665423+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:51:30.665423+0000","last_undegraded":"2024-09-21T06:51:30.665423+0000","last_fullsized":"2024-09-21T06:51:30.665423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670454+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:51:30.670454+0000","last_peered":"2024-09-21T06:51:30.670454+0000","last_clean":"2024-09-21T06:51:30.670454+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:51:30.670454+0000","last_undegraded":"2024-09-21T06:51:30.670454+0000","last_fullsized":"2024-09-21T06:51:30.670454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665134+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:51:30.665134+0000","last_peered":"2024-09-21T06:51:30.665134+0000","last_clean":"2024-09-21T06:51:30.665134+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:51:30.665134+0000","last_undegraded":"2024-09-21T06:51:30.665134+0000","last_fullsized":"2024-09-21T06:51:30.665134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670351+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:51:30.670351+0000","last_peered":"2024-09-21T06:51:30.670351+0000","last_clean":"2024-09-21T06:51:30.670351+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:51:30.670351+0000","last_undegraded":"2024-09-21T06:51:30.670351+0000","last_fullsized":"2024-09-21T06:51:30.670351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669924+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:51:30.669924+0000","last_peered":"2024-09-21T06:51:30.669924+0000","last_clean":"2024-09-21T06:51:30.669924+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:51:30.669924+0000","last_undegraded":"2024-09-21T06:51:30.669924+0000","last_fullsized":"2024-09-21T06:51:30.669924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665427+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:51:30.665427+0000","last_peered":"2024-09-21T06:51:30.665427+0000","last_clean":"2024-09-21T06:51:30.665427+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:51:30.665427+0000","last_undegraded":"2024-09-21T06:51:30.665427+0000","last_fullsized":"2024-09-21T06:51:30.665427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752677+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:51:30.752677+0000","last_peered":"2024-09-21T06:51:30.752677+0000","last_clean":"2024-09-21T06:51:30.752677+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:51:30.752677+0000","last_undegraded":"2024-09-21T06:51:30.752677+0000","last_fullsized":"2024-09-21T06:51:30.752677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666838+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:51:30.666838+0000","last_peered":"2024-09-21T06:51:30.666838+0000","last_clean":"2024-09-21T06:51:30.666838+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:51:30.666838+0000","last_undegraded":"2024-09-21T06:51:30.666838+0000","last_fullsized":"2024-09-21T06:51:30.666838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670711+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:51:30.670711+0000","last_peered":"2024-09-21T06:51:30.670711+0000","last_clean":"2024-09-21T06:51:30.670711+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:51:30.670711+0000","last_undegraded":"2024-09-21T06:51:30.670711+0000","last_fullsized":"2024-09-21T06:51:30.670711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663206+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:51:30.663206+0000","last_peered":"2024-09-21T06:51:30.663206+0000","last_clean":"2024-09-21T06:51:30.663206+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:51:30.663206+0000","last_undegraded":"2024-09-21T06:51:30.663206+0000","last_fullsized":"2024-09-21T06:51:30.663206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662767+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:51:30.662767+0000","last_peered":"2024-09-21T06:51:30.662767+0000","last_clean":"2024-09-21T06:51:30.662767+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:51:30.662767+0000","last_undegraded":"2024-09-21T06:51:30.662767+0000","last_fullsized":"2024-09-21T06:51:30.662767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662324+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:51:30.662324+0000","last_peered":"2024-09-21T06:51:30.662324+0000","last_clean":"2024-09-21T06:51:30.662324+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:51:30.662324+0000","last_undegraded":"2024-09-21T06:51:30.662324+0000","last_fullsized":"2024-09-21T06:51:30.662324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666912+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:51:30.666912+0000","last_peered":"2024-09-21T06:51:30.666912+0000","last_clean":"2024-09-21T06:51:30.666912+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:51:30.666912+0000","last_undegraded":"2024-09-21T06:51:30.666912+0000","last_fullsized":"2024-09-21T06:51:30.666912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752414+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:51:30.752414+0000","last_peered":"2024-09-21T06:51:30.752414+0000","last_clean":"2024-09-21T06:51:30.752414+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:51:30.752414+0000","last_undegraded":"2024-09-21T06:51:30.752414+0000","last_fullsized":"2024-09-21T06:51:30.752414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666327+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:51:30.666327+0000","last_peered":"2024-09-21T06:51:30.666327+0000","last_clean":"2024-09-21T06:51:30.666327+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:51:30.666327+0000","last_undegraded":"2024-09-21T06:51:30.666327+0000","last_fullsized":"2024-09-21T06:51:30.666327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670589+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:51:30.670589+0000","last_peered":"2024-09-21T06:51:30.670589+0000","last_clean":"2024-09-21T06:51:30.670589+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:51:30.670589+0000","last_undegraded":"2024-09-21T06:51:30.670589+0000","last_fullsized":"2024-09-21T06:51:30.670589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921709+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:51:30.921709+0000","last_peered":"2024-09-21T06:51:30.921709+0000","last_clean":"2024-09-21T06:51:30.921709+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:51:30.921709+0000","last_undegraded":"2024-09-21T06:51:30.921709+0000","last_fullsized":"2024-09-21T06:51:30.921709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668242+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:51:30.668242+0000","last_peered":"2024-09-21T06:51:30.668242+0000","last_clean":"2024-09-21T06:51:30.668242+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:51:30.668242+0000","last_undegraded":"2024-09-21T06:51:30.668242+0000","last_fullsized":"2024-09-21T06:51:30.668242+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665895+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:51:30.665895+0000","last_peered":"2024-09-21T06:51:30.665895+0000","last_clean":"2024-09-21T06:51:30.665895+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:51:30.665895+0000","last_undegraded":"2024-09-21T06:51:30.665895+0000","last_fullsized":"2024-09-21T06:51:30.665895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.673289+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:51:30.673289+0000","last_peered":"2024-09-21T06:51:30.673289+0000","last_clean":"2024-09-21T06:51:30.673289+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:51:30.673289+0000","last_undegraded":"2024-09-21T06:51:30.673289+0000","last_fullsized":"2024-09-21T06:51:30.673289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665282+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:51:30.665282+0000","last_peered":"2024-09-21T06:51:30.665282+0000","last_clean":"2024-09-21T06:51:30.665282+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:51:30.665282+0000","last_undegraded":"2024-09-21T06:51:30.665282+0000","last_fullsized":"2024-09-21T06:51:30.665282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752334+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:51:30.752334+0000","last_peered":"2024-09-21T06:51:30.752334+0000","last_clean":"2024-09-21T06:51:30.752334+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:51:30.752334+0000","last_undegraded":"2024-09-21T06:51:30.752334+0000","last_fullsized":"2024-09-21T06:51:30.752334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663076+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:51:30.663076+0000","last_peered":"2024-09-21T06:51:30.663076+0000","last_clean":"2024-09-21T06:51:30.663076+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:51:30.663076+0000","last_undegraded":"2024-09-21T06:51:30.663076+0000","last_fullsized":"2024-09-21T06:51:30.663076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669589+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:51:30.669589+0000","last_peered":"2024-09-21T06:51:30.669589+0000","last_clean":"2024-09-21T06:51:30.669589+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:51:30.669589+0000","last_undegraded":"2024-09-21T06:51:30.669589+0000","last_fullsized":"2024-09-21T06:51:30.669589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665818+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:51:30.665818+0000","last_peered":"2024-09-21T06:51:30.665818+0000","last_clean":"2024-09-21T06:51:30.665818+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:51:30.665818+0000","last_undegraded":"2024-09-21T06:51:30.665818+0000","last_fullsized":"2024-09-21T06:51:30.665818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668444+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:51:30.668444+0000","last_peered":"2024-09-21T06:51:30.668444+0000","last_clean":"2024-09-21T06:51:30.668444+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:51:30.668444+0000","last_undegraded":"2024-09-21T06:51:30.668444+0000","last_fullsized":"2024-09-21T06:51:30.668444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662205+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:51:30.662205+0000","last_peered":"2024-09-21T06:51:30.662205+0000","last_clean":"2024-09-21T06:51:30.662205+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:51:30.662205+0000","last_undegraded":"2024-09-21T06:51:30.662205+0000","last_fullsized":"2024-09-21T06:51:30.662205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668841+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:51:30.668841+0000","last_peered":"2024-09-21T06:51:30.668841+0000","last_clean":"2024-09-21T06:51:30.668841+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:51:30.668841+0000","last_undegraded":"2024-09-21T06:51:30.668841+0000","last_fullsized":"2024-09-21T06:51:30.668841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665667+0000","last_change":"2024-09-21T06:50:47.214866+0000","last_active":"2024-09-21T06:51:30.665667+0000","last_peered":"2024-09-21T06:51:30.665667+0000","last_clean":"2024-09-21T06:51:30.665667+0000","last_became_active":"2024-09-21T06:50:47.214451+0000","last_became_peered":"2024-09-21T06:50:47.214451+0000","last_unstale":"2024-09-21T06:51:30.665667+0000","last_undegraded":"2024-09-21T06:51:30.665667+0000","last_fullsized":"2024-09-21T06:51:30.665667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:37.914291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665432+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:51:30.665432+0000","last_peered":"2024-09-21T06:51:30.665432+0000","last_clean":"2024-09-21T06:51:30.665432+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:51:30.665432+0000","last_undegraded":"2024-09-21T06:51:30.665432+0000","last_fullsized":"2024-09-21T06:51:30.665432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662970+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:51:30.662970+0000","last_peered":"2024-09-21T06:51:30.662970+0000","last_clean":"2024-09-21T06:51:30.662970+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:51:30.662970+0000","last_undegraded":"2024-09-21T06:51:30.662970+0000","last_fullsized":"2024-09-21T06:51:30.662970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665099+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:51:30.665099+0000","last_peered":"2024-09-21T06:51:30.665099+0000","last_clean":"2024-09-21T06:51:30.665099+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:51:30.665099+0000","last_undegraded":"2024-09-21T06:51:30.665099+0000","last_fullsized":"2024-09-21T06:51:30.665099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665965+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:51:30.665965+0000","last_peered":"2024-09-21T06:51:30.665965+0000","last_clean":"2024-09-21T06:51:30.665965+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:51:30.665965+0000","last_undegraded":"2024-09-21T06:51:30.665965+0000","last_fullsized":"2024-09-21T06:51:30.665965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667562+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:51:30.667562+0000","last_peered":"2024-09-21T06:51:30.667562+0000","last_clean":"2024-09-21T06:51:30.667562+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:51:30.667562+0000","last_undegraded":"2024-09-21T06:51:30.667562+0000","last_fullsized":"2024-09-21T06:51:30.667562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64144,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":192,"num_read_kb":167,"num_write":240,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":192432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":512314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906976,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1691648,"data_stored":1155266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070501,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":515927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266732,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1109097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822717,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1104288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":511409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":507043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1077248,"data_stored":564903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":566318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":509182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:52:03.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:52:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:03 smithi114 ceph-mon[30654]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-21T06:52:03.897 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:04.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:03 smithi183 ceph-mon[34219]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-21T06:52:04.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:03 smithi193 ceph-mon[34601]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 -- 172.21.15.114:0/3117223909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10b4f0 msgr2=0x7feccc10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 --2- 172.21.15.114:0/3117223909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10b4f0 0x7feccc10b900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009900 tx=0x7fecb802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 -- 172.21.15.114:0/3117223909 shutdown_connections 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 --2- 172.21.15.114:0/3117223909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc10be40 0x7feccc112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 --2- 172.21.15.114:0/3117223909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10b4f0 0x7feccc10b900 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 --2- 172.21.15.114:0/3117223909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc105bb0 0x7feccc105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 -- 172.21.15.114:0/3117223909 >> 172.21.15.114:0/3117223909 conn(0x7feccc101680 msgr2=0x7feccc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 -- 172.21.15.114:0/3117223909 shutdown_connections 2024-09-21T06:52:04.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.544+0000 7feccbe00640 1 -- 172.21.15.114:0/3117223909 wait complete. 2024-09-21T06:52:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.545+0000 7feccbe00640 1 Processor -- start 2024-09-21T06:52:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 -- start start 2024-09-21T06:52:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 0x7feccc19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:04.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc10b4f0 0x7feccc19ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10be40 0x7feccc1a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7feccc115b00 con 0x7feccc10b4f0 2024-09-21T06:52:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feccc115980 con 0x7feccc105bb0 2024-09-21T06:52:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feccc115c80 con 0x7feccc10be40 2024-09-21T06:52:04.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 0x7feccc19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7fecca000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc10b4f0 0x7feccc19ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 0x7feccc19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53178/0 (socket says 172.21.15.114:53178) 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.546+0000 7feccaa00640 1 -- 172.21.15.114:0/2344160185 learned_addr learned my addr 172.21.15.114:0/2344160185 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccb400640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10be40 0x7feccc1a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccaa00640 1 -- 172.21.15.114:0/2344160185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10be40 msgr2=0x7feccc1a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccaa00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10be40 0x7feccc1a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccaa00640 1 -- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc10b4f0 msgr2=0x7feccc19ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccaa00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc10b4f0 0x7feccc19ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccaa00640 1 -- 172.21.15.114:0/2344160185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc1a2440 con 0x7feccc105bb0 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7fecca000640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc10b4f0 0x7feccc19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:04.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.547+0000 7feccaa00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 0x7feccc19c4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e9f0 tx=0x7fecbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:04.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.548+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00cd80 con 0x7feccc105bb0 2024-09-21T06:52:04.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.548+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc1a2730 con 0x7feccc105bb0 2024-09-21T06:52:04.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.548+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc0108e0 con 0x7feccc105bb0 2024-09-21T06:52:04.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.548+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc018980 con 0x7feccc105bb0 2024-09-21T06:52:04.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.548+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a8e90 con 0x7feccc105bb0 2024-09-21T06:52:04.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.549+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc105fc0 con 0x7feccc105bb0 2024-09-21T06:52:04.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.552+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fecbc010430 con 0x7feccc105bb0 2024-09-21T06:52:04.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.554+0000 7fecc3400640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb00b65f0 0x7fecb00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:04.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.554+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fecbc0e8130 con 0x7feccc105bb0 2024-09-21T06:52:04.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.554+0000 7fecca000640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb00b65f0 0x7fecb00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:04.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.554+0000 7fecca000640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb00b65f0 0x7fecb00b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009230 tx=0x7fecb8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:04.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.556+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fecbc0b27c0 con 0x7feccc105bb0 2024-09-21T06:52:04.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.741+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feccc063800 con 0x7fecb00b65f0 2024-09-21T06:52:04.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.748+0000 7fecc3400640 1 -- 172.21.15.114:0/2344160185 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340382 (secure 0 0 0) 0x7feccc063800 con 0x7fecb00b65f0 2024-09-21T06:52:04.753 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:04.755 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:52:04.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb00b65f0 msgr2=0x7fecb00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb00b65f0 0x7fecb00b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009230 tx=0x7fecb8031400 comp rx=0 tx=0).stop 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 msgr2=0x7feccc19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 0x7feccc19c4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00e9f0 tx=0x7fecbc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 shutdown_connections 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feccc10be40 0x7feccc1a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7feccc10b4f0 0x7feccc19ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fecb00b65f0 0x7fecb00b8b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 --2- 172.21.15.114:0/2344160185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feccc105bb0 0x7feccc19c4d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.753+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 >> 172.21.15.114:0/2344160185 conn(0x7feccc101680 msgr2=0x7feccc10aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.754+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 shutdown_connections 2024-09-21T06:52:04.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:04.754+0000 7feccbe00640 1 -- 172.21.15.114:0/2344160185 wait complete. 2024-09-21T06:52:05.525 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-21T06:52:03.507778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654746,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":293,"num_read_kb":246,"num_write":371,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371456,"kb_used_data":13888,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570368,"statfs":{"total":1151940427776,"available":1151560056832,"internally_reserved":0,"allocated":14221312,"data_stored":8084190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47123,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":106,"num_write":34,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003858"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":72,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:52:00.650221+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:52:00.650221+0000","last_peered":"2024-09-21T06:52:00.650221+0000","last_clean":"2024-09-21T06:52:00.650221+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:52:00.650221+0000","last_undegraded":"2024-09-21T06:52:00.650221+0000","last_fullsized":"2024-09-21T06:52:00.650221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662712+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:51:30.662712+0000","last_peered":"2024-09-21T06:51:30.662712+0000","last_clean":"2024-09-21T06:51:30.662712+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:51:30.662712+0000","last_undegraded":"2024-09-21T06:51:30.662712+0000","last_fullsized":"2024-09-21T06:51:30.662712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669858+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:51:30.669858+0000","last_peered":"2024-09-21T06:51:30.669858+0000","last_clean":"2024-09-21T06:51:30.669858+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:51:30.669858+0000","last_undegraded":"2024-09-21T06:51:30.669858+0000","last_fullsized":"2024-09-21T06:51:30.669858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663062+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:51:30.663062+0000","last_peered":"2024-09-21T06:51:30.663062+0000","last_clean":"2024-09-21T06:51:30.663062+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:51:30.663062+0000","last_undegraded":"2024-09-21T06:51:30.663062+0000","last_fullsized":"2024-09-21T06:51:30.663062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666417+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:51:30.666417+0000","last_peered":"2024-09-21T06:51:30.666417+0000","last_clean":"2024-09-21T06:51:30.666417+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:51:30.666417+0000","last_undegraded":"2024-09-21T06:51:30.666417+0000","last_fullsized":"2024-09-21T06:51:30.666417+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.755038+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:51:30.755038+0000","last_peered":"2024-09-21T06:51:30.755038+0000","last_clean":"2024-09-21T06:51:30.755038+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:51:30.755038+0000","last_undegraded":"2024-09-21T06:51:30.755038+0000","last_fullsized":"2024-09-21T06:51:30.755038+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":52,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:43.621858+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:51:43.621858+0000","last_peered":"2024-09-21T06:51:43.621858+0000","last_clean":"2024-09-21T06:51:43.621858+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:51:43.621858+0000","last_undegraded":"2024-09-21T06:51:43.621858+0000","last_fullsized":"2024-09-21T06:51:43.621858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667290+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:51:30.667290+0000","last_peered":"2024-09-21T06:51:30.667290+0000","last_clean":"2024-09-21T06:51:30.667290+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:51:30.667290+0000","last_undegraded":"2024-09-21T06:51:30.667290+0000","last_fullsized":"2024-09-21T06:51:30.667290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662767+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:51:30.662767+0000","last_peered":"2024-09-21T06:51:30.662767+0000","last_clean":"2024-09-21T06:51:30.662767+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:51:30.662767+0000","last_undegraded":"2024-09-21T06:51:30.662767+0000","last_fullsized":"2024-09-21T06:51:30.662767+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.610291+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:51:42.610291+0000","last_peered":"2024-09-21T06:51:42.610291+0000","last_clean":"2024-09-21T06:51:42.610291+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:51:42.610291+0000","last_undegraded":"2024-09-21T06:51:42.610291+0000","last_fullsized":"2024-09-21T06:51:42.610291+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.664316+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:51:30.664316+0000","last_peered":"2024-09-21T06:51:30.664316+0000","last_clean":"2024-09-21T06:51:30.664316+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:51:30.664316+0000","last_undegraded":"2024-09-21T06:51:30.664316+0000","last_fullsized":"2024-09-21T06:51:30.664316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662372+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:51:30.662372+0000","last_peered":"2024-09-21T06:51:30.662372+0000","last_clean":"2024-09-21T06:51:30.662372+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:51:30.662372+0000","last_undegraded":"2024-09-21T06:51:30.662372+0000","last_fullsized":"2024-09-21T06:51:30.662372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669079+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:51:30.669079+0000","last_peered":"2024-09-21T06:51:30.669079+0000","last_clean":"2024-09-21T06:51:30.669079+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:51:30.669079+0000","last_undegraded":"2024-09-21T06:51:30.669079+0000","last_fullsized":"2024-09-21T06:51:30.669079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:39.580312+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:51:39.580312+0000","last_peered":"2024-09-21T06:51:39.580312+0000","last_clean":"2024-09-21T06:51:39.580312+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:51:39.580312+0000","last_undegraded":"2024-09-21T06:51:39.580312+0000","last_fullsized":"2024-09-21T06:51:39.580312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662651+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:51:30.662651+0000","last_peered":"2024-09-21T06:51:30.662651+0000","last_clean":"2024-09-21T06:51:30.662651+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:51:30.662651+0000","last_undegraded":"2024-09-21T06:51:30.662651+0000","last_fullsized":"2024-09-21T06:51:30.662651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662201+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:51:30.662201+0000","last_peered":"2024-09-21T06:51:30.662201+0000","last_clean":"2024-09-21T06:51:30.662201+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:51:30.662201+0000","last_undegraded":"2024-09-21T06:51:30.662201+0000","last_fullsized":"2024-09-21T06:51:30.662201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:41.598608+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:51:41.598608+0000","last_peered":"2024-09-21T06:51:41.598608+0000","last_clean":"2024-09-21T06:51:41.598608+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:51:41.598608+0000","last_undegraded":"2024-09-21T06:51:41.598608+0000","last_fullsized":"2024-09-21T06:51:41.598608+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662784+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:51:30.662784+0000","last_peered":"2024-09-21T06:51:30.662784+0000","last_clean":"2024-09-21T06:51:30.662784+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:51:30.662784+0000","last_undegraded":"2024-09-21T06:51:30.662784+0000","last_fullsized":"2024-09-21T06:51:30.662784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666991+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:51:30.666991+0000","last_peered":"2024-09-21T06:51:30.666991+0000","last_clean":"2024-09-21T06:51:30.666991+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:51:30.666991+0000","last_undegraded":"2024-09-21T06:51:30.666991+0000","last_fullsized":"2024-09-21T06:51:30.666991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.614614+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:51:42.614614+0000","last_peered":"2024-09-21T06:51:42.614614+0000","last_clean":"2024-09-21T06:51:42.614614+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:51:42.614614+0000","last_undegraded":"2024-09-21T06:51:42.614614+0000","last_fullsized":"2024-09-21T06:51:42.614614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921785+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:51:30.921785+0000","last_peered":"2024-09-21T06:51:30.921785+0000","last_clean":"2024-09-21T06:51:30.921785+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:51:30.921785+0000","last_undegraded":"2024-09-21T06:51:30.921785+0000","last_fullsized":"2024-09-21T06:51:30.921785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665136+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:51:30.665136+0000","last_peered":"2024-09-21T06:51:30.665136+0000","last_clean":"2024-09-21T06:51:30.665136+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:51:30.665136+0000","last_undegraded":"2024-09-21T06:51:30.665136+0000","last_fullsized":"2024-09-21T06:51:30.665136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:39.579256+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:51:39.579256+0000","last_peered":"2024-09-21T06:51:39.579256+0000","last_clean":"2024-09-21T06:51:39.579256+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:51:39.579256+0000","last_undegraded":"2024-09-21T06:51:39.579256+0000","last_fullsized":"2024-09-21T06:51:39.579256+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:51:30.663150+0000","last_peered":"2024-09-21T06:51:30.663150+0000","last_clean":"2024-09-21T06:51:30.663150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:51:30.663150+0000","last_undegraded":"2024-09-21T06:51:30.663150+0000","last_fullsized":"2024-09-21T06:51:30.663150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669400+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:51:30.669400+0000","last_peered":"2024-09-21T06:51:30.669400+0000","last_clean":"2024-09-21T06:51:30.669400+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:51:30.669400+0000","last_undegraded":"2024-09-21T06:51:30.669400+0000","last_fullsized":"2024-09-21T06:51:30.669400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.612142+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:51:42.612142+0000","last_peered":"2024-09-21T06:51:42.612142+0000","last_clean":"2024-09-21T06:51:42.612142+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:51:42.612142+0000","last_undegraded":"2024-09-21T06:51:42.612142+0000","last_fullsized":"2024-09-21T06:51:42.612142+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670021+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:51:30.670021+0000","last_peered":"2024-09-21T06:51:30.670021+0000","last_clean":"2024-09-21T06:51:30.670021+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:51:30.670021+0000","last_undegraded":"2024-09-21T06:51:30.670021+0000","last_fullsized":"2024-09-21T06:51:30.670021+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663022+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:51:30.663022+0000","last_peered":"2024-09-21T06:51:30.663022+0000","last_clean":"2024-09-21T06:51:30.663022+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:51:30.663022+0000","last_undegraded":"2024-09-21T06:51:30.663022+0000","last_fullsized":"2024-09-21T06:51:30.663022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":54,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:52:00.235081+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:52:00.235081+0000","last_peered":"2024-09-21T06:52:00.235081+0000","last_clean":"2024-09-21T06:52:00.235081+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:52:00.235081+0000","last_undegraded":"2024-09-21T06:52:00.235081+0000","last_fullsized":"2024-09-21T06:52:00.235081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.661943+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:51:30.661943+0000","last_peered":"2024-09-21T06:51:30.661943+0000","last_clean":"2024-09-21T06:51:30.661943+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:51:30.661943+0000","last_undegraded":"2024-09-21T06:51:30.661943+0000","last_fullsized":"2024-09-21T06:51:30.661943+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752802+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:51:30.752802+0000","last_peered":"2024-09-21T06:51:30.752802+0000","last_clean":"2024-09-21T06:51:30.752802+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:51:30.752802+0000","last_undegraded":"2024-09-21T06:51:30.752802+0000","last_fullsized":"2024-09-21T06:51:30.752802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.614596+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:51:42.614596+0000","last_peered":"2024-09-21T06:51:42.614596+0000","last_clean":"2024-09-21T06:51:42.614596+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:51:42.614596+0000","last_undegraded":"2024-09-21T06:51:42.614596+0000","last_fullsized":"2024-09-21T06:51:42.614596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666136+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:51:30.666136+0000","last_peered":"2024-09-21T06:51:30.666136+0000","last_clean":"2024-09-21T06:51:30.666136+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:51:30.666136+0000","last_undegraded":"2024-09-21T06:51:30.666136+0000","last_fullsized":"2024-09-21T06:51:30.666136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665200+0000","last_change":"2024-09-21T06:50:47.222453+0000","last_active":"2024-09-21T06:51:30.665200+0000","last_peered":"2024-09-21T06:51:30.665200+0000","last_clean":"2024-09-21T06:51:30.665200+0000","last_became_active":"2024-09-21T06:50:47.222057+0000","last_became_peered":"2024-09-21T06:50:47.222057+0000","last_unstale":"2024-09-21T06:51:30.665200+0000","last_undegraded":"2024-09-21T06:51:30.665200+0000","last_fullsized":"2024-09-21T06:51:30.665200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:01.741607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921450+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:51:30.921450+0000","last_peered":"2024-09-21T06:51:30.921450+0000","last_clean":"2024-09-21T06:51:30.921450+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:51:30.921450+0000","last_undegraded":"2024-09-21T06:51:30.921450+0000","last_fullsized":"2024-09-21T06:51:30.921450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665260+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:51:30.665260+0000","last_peered":"2024-09-21T06:51:30.665260+0000","last_clean":"2024-09-21T06:51:30.665260+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:51:30.665260+0000","last_undegraded":"2024-09-21T06:51:30.665260+0000","last_fullsized":"2024-09-21T06:51:30.665260+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666702+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:51:30.666702+0000","last_peered":"2024-09-21T06:51:30.666702+0000","last_clean":"2024-09-21T06:51:30.666702+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:51:30.666702+0000","last_undegraded":"2024-09-21T06:51:30.666702+0000","last_fullsized":"2024-09-21T06:51:30.666702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:57.646762+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:51:57.646762+0000","last_peered":"2024-09-21T06:51:57.646762+0000","last_clean":"2024-09-21T06:51:57.646762+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:51:57.646762+0000","last_undegraded":"2024-09-21T06:51:57.646762+0000","last_fullsized":"2024-09-21T06:51:57.646762+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752442+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:51:30.752442+0000","last_peered":"2024-09-21T06:51:30.752442+0000","last_clean":"2024-09-21T06:51:30.752442+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:51:30.752442+0000","last_undegraded":"2024-09-21T06:51:30.752442+0000","last_fullsized":"2024-09-21T06:51:30.752442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663244+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:51:30.663244+0000","last_peered":"2024-09-21T06:51:30.663244+0000","last_clean":"2024-09-21T06:51:30.663244+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:51:30.663244+0000","last_undegraded":"2024-09-21T06:51:30.663244+0000","last_fullsized":"2024-09-21T06:51:30.663244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:52:00.650260+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:52:00.650260+0000","last_peered":"2024-09-21T06:52:00.650260+0000","last_clean":"2024-09-21T06:52:00.650260+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:52:00.650260+0000","last_undegraded":"2024-09-21T06:52:00.650260+0000","last_fullsized":"2024-09-21T06:52:00.650260+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752802+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:51:30.752802+0000","last_peered":"2024-09-21T06:51:30.752802+0000","last_clean":"2024-09-21T06:51:30.752802+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:51:30.752802+0000","last_undegraded":"2024-09-21T06:51:30.752802+0000","last_fullsized":"2024-09-21T06:51:30.752802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.751961+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:51:30.751961+0000","last_peered":"2024-09-21T06:51:30.751961+0000","last_clean":"2024-09-21T06:51:30.751961+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:51:30.751961+0000","last_undegraded":"2024-09-21T06:51:30.751961+0000","last_fullsized":"2024-09-21T06:51:30.751961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:40.589215+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:51:40.589215+0000","last_peered":"2024-09-21T06:51:40.589215+0000","last_clean":"2024-09-21T06:51:40.589215+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:51:40.589215+0000","last_undegraded":"2024-09-21T06:51:40.589215+0000","last_fullsized":"2024-09-21T06:51:40.589215+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663611+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:51:30.663611+0000","last_peered":"2024-09-21T06:51:30.663611+0000","last_clean":"2024-09-21T06:51:30.663611+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:51:30.663611+0000","last_undegraded":"2024-09-21T06:51:30.663611+0000","last_fullsized":"2024-09-21T06:51:30.663611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662460+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:51:30.662460+0000","last_peered":"2024-09-21T06:51:30.662460+0000","last_clean":"2024-09-21T06:51:30.662460+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:51:30.662460+0000","last_undegraded":"2024-09-21T06:51:30.662460+0000","last_fullsized":"2024-09-21T06:51:30.662460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:43.621170+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:51:43.621170+0000","last_peered":"2024-09-21T06:51:43.621170+0000","last_clean":"2024-09-21T06:51:43.621170+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:51:43.621170+0000","last_undegraded":"2024-09-21T06:51:43.621170+0000","last_fullsized":"2024-09-21T06:51:43.621170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669035+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:51:30.669035+0000","last_peered":"2024-09-21T06:51:30.669035+0000","last_clean":"2024-09-21T06:51:30.669035+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:51:30.669035+0000","last_undegraded":"2024-09-21T06:51:30.669035+0000","last_fullsized":"2024-09-21T06:51:30.669035+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669753+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:51:30.669753+0000","last_peered":"2024-09-21T06:51:30.669753+0000","last_clean":"2024-09-21T06:51:30.669753+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:51:30.669753+0000","last_undegraded":"2024-09-21T06:51:30.669753+0000","last_fullsized":"2024-09-21T06:51:30.669753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669814+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:51:30.669814+0000","last_peered":"2024-09-21T06:51:30.669814+0000","last_clean":"2024-09-21T06:51:30.669814+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:51:30.669814+0000","last_undegraded":"2024-09-21T06:51:30.669814+0000","last_fullsized":"2024-09-21T06:51:30.669814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:52:00.651209+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:52:00.651209+0000","last_peered":"2024-09-21T06:52:00.651209+0000","last_clean":"2024-09-21T06:52:00.651209+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:52:00.651209+0000","last_undegraded":"2024-09-21T06:52:00.651209+0000","last_fullsized":"2024-09-21T06:52:00.651209+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.754336+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:51:30.754336+0000","last_peered":"2024-09-21T06:51:30.754336+0000","last_clean":"2024-09-21T06:51:30.754336+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:51:30.754336+0000","last_undegraded":"2024-09-21T06:51:30.754336+0000","last_fullsized":"2024-09-21T06:51:30.754336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669274+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:51:30.669274+0000","last_peered":"2024-09-21T06:51:30.669274+0000","last_clean":"2024-09-21T06:51:30.669274+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:51:30.669274+0000","last_undegraded":"2024-09-21T06:51:30.669274+0000","last_fullsized":"2024-09-21T06:51:30.669274+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.613692+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:51:42.613692+0000","last_peered":"2024-09-21T06:51:42.613692+0000","last_clean":"2024-09-21T06:51:42.613692+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:51:42.613692+0000","last_undegraded":"2024-09-21T06:51:42.613692+0000","last_fullsized":"2024-09-21T06:51:42.613692+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665284+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:51:30.665284+0000","last_peered":"2024-09-21T06:51:30.665284+0000","last_clean":"2024-09-21T06:51:30.665284+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:51:30.665284+0000","last_undegraded":"2024-09-21T06:51:30.665284+0000","last_fullsized":"2024-09-21T06:51:30.665284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669617+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:51:30.669617+0000","last_peered":"2024-09-21T06:51:30.669617+0000","last_clean":"2024-09-21T06:51:30.669617+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:51:30.669617+0000","last_undegraded":"2024-09-21T06:51:30.669617+0000","last_fullsized":"2024-09-21T06:51:30.669617+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.612598+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:51:42.612598+0000","last_peered":"2024-09-21T06:51:42.612598+0000","last_clean":"2024-09-21T06:51:42.612598+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:51:42.612598+0000","last_undegraded":"2024-09-21T06:51:42.612598+0000","last_fullsized":"2024-09-21T06:51:42.612598+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662666+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:51:30.662666+0000","last_peered":"2024-09-21T06:51:30.662666+0000","last_clean":"2024-09-21T06:51:30.662666+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:51:30.662666+0000","last_undegraded":"2024-09-21T06:51:30.662666+0000","last_fullsized":"2024-09-21T06:51:30.662666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666145+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:51:30.666145+0000","last_peered":"2024-09-21T06:51:30.666145+0000","last_clean":"2024-09-21T06:51:30.666145+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:51:30.666145+0000","last_undegraded":"2024-09-21T06:51:30.666145+0000","last_fullsized":"2024-09-21T06:51:30.666145+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.584471+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:51:42.584471+0000","last_peered":"2024-09-21T06:51:42.584471+0000","last_clean":"2024-09-21T06:51:42.584471+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:51:42.584471+0000","last_undegraded":"2024-09-21T06:51:42.584471+0000","last_fullsized":"2024-09-21T06:51:42.584471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662770+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:51:30.662770+0000","last_peered":"2024-09-21T06:51:30.662770+0000","last_clean":"2024-09-21T06:51:30.662770+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:51:30.662770+0000","last_undegraded":"2024-09-21T06:51:30.662770+0000","last_fullsized":"2024-09-21T06:51:30.662770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662861+0000","last_change":"2024-09-21T06:50:47.224057+0000","last_active":"2024-09-21T06:51:30.662861+0000","last_peered":"2024-09-21T06:51:30.662861+0000","last_clean":"2024-09-21T06:51:30.662861+0000","last_became_active":"2024-09-21T06:50:47.223689+0000","last_became_peered":"2024-09-21T06:50:47.223689+0000","last_unstale":"2024-09-21T06:51:30.662861+0000","last_undegraded":"2024-09-21T06:51:30.662861+0000","last_fullsized":"2024-09-21T06:51:30.662861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:53.752342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.610626+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:51:42.610626+0000","last_peered":"2024-09-21T06:51:42.610626+0000","last_clean":"2024-09-21T06:51:42.610626+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:51:42.610626+0000","last_undegraded":"2024-09-21T06:51:42.610626+0000","last_fullsized":"2024-09-21T06:51:42.610626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752867+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:51:30.752867+0000","last_peered":"2024-09-21T06:51:30.752867+0000","last_clean":"2024-09-21T06:51:30.752867+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:51:30.752867+0000","last_undegraded":"2024-09-21T06:51:30.752867+0000","last_fullsized":"2024-09-21T06:51:30.752867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668680+0000","last_change":"2024-09-21T06:50:47.223163+0000","last_active":"2024-09-21T06:51:30.668680+0000","last_peered":"2024-09-21T06:51:30.668680+0000","last_clean":"2024-09-21T06:51:30.668680+0000","last_became_active":"2024-09-21T06:50:47.222742+0000","last_became_peered":"2024-09-21T06:50:47.222742+0000","last_unstale":"2024-09-21T06:51:30.668680+0000","last_undegraded":"2024-09-21T06:51:30.668680+0000","last_fullsized":"2024-09-21T06:51:30.668680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:17.570268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:59.649363+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:51:59.649363+0000","last_peered":"2024-09-21T06:51:59.649363+0000","last_clean":"2024-09-21T06:51:59.649363+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:51:59.649363+0000","last_undegraded":"2024-09-21T06:51:59.649363+0000","last_fullsized":"2024-09-21T06:51:59.649363+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.672042+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:51:30.672042+0000","last_peered":"2024-09-21T06:51:30.672042+0000","last_clean":"2024-09-21T06:51:30.672042+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:51:30.672042+0000","last_undegraded":"2024-09-21T06:51:30.672042+0000","last_fullsized":"2024-09-21T06:51:30.672042+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669840+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:51:30.669840+0000","last_peered":"2024-09-21T06:51:30.669840+0000","last_clean":"2024-09-21T06:51:30.669840+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:51:30.669840+0000","last_undegraded":"2024-09-21T06:51:30.669840+0000","last_fullsized":"2024-09-21T06:51:30.669840+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.613738+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:51:42.613738+0000","last_peered":"2024-09-21T06:51:42.613738+0000","last_clean":"2024-09-21T06:51:42.613738+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:51:42.613738+0000","last_undegraded":"2024-09-21T06:51:42.613738+0000","last_fullsized":"2024-09-21T06:51:42.613738+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666191+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:51:30.666191+0000","last_peered":"2024-09-21T06:51:30.666191+0000","last_clean":"2024-09-21T06:51:30.666191+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:51:30.666191+0000","last_undegraded":"2024-09-21T06:51:30.666191+0000","last_fullsized":"2024-09-21T06:51:30.666191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752529+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:51:30.752529+0000","last_peered":"2024-09-21T06:51:30.752529+0000","last_clean":"2024-09-21T06:51:30.752529+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:51:30.752529+0000","last_undegraded":"2024-09-21T06:51:30.752529+0000","last_fullsized":"2024-09-21T06:51:30.752529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:43.622643+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:51:43.622643+0000","last_peered":"2024-09-21T06:51:43.622643+0000","last_clean":"2024-09-21T06:51:43.622643+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:51:43.622643+0000","last_undegraded":"2024-09-21T06:51:43.622643+0000","last_fullsized":"2024-09-21T06:51:43.622643+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.664269+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:51:30.664269+0000","last_peered":"2024-09-21T06:51:30.664269+0000","last_clean":"2024-09-21T06:51:30.664269+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:51:30.664269+0000","last_undegraded":"2024-09-21T06:51:30.664269+0000","last_fullsized":"2024-09-21T06:51:30.664269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667091+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:51:30.667091+0000","last_peered":"2024-09-21T06:51:30.667091+0000","last_clean":"2024-09-21T06:51:30.667091+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:51:30.667091+0000","last_undegraded":"2024-09-21T06:51:30.667091+0000","last_fullsized":"2024-09-21T06:51:30.667091+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'31","reported_seq":107,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:58.649923+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:51:58.649923+0000","last_peered":"2024-09-21T06:51:58.649923+0000","last_clean":"2024-09-21T06:51:58.649923+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:51:58.649923+0000","last_undegraded":"2024-09-21T06:51:58.649923+0000","last_fullsized":"2024-09-21T06:51:58.649923+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56549,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":74,"num_write":33,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.673773+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:51:30.673773+0000","last_peered":"2024-09-21T06:51:30.673773+0000","last_clean":"2024-09-21T06:51:30.673773+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:51:30.673773+0000","last_undegraded":"2024-09-21T06:51:30.673773+0000","last_fullsized":"2024-09-21T06:51:30.673773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667452+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:51:30.667452+0000","last_peered":"2024-09-21T06:51:30.667452+0000","last_clean":"2024-09-21T06:51:30.667452+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:51:30.667452+0000","last_undegraded":"2024-09-21T06:51:30.667452+0000","last_fullsized":"2024-09-21T06:51:30.667452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:41.600537+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:51:41.600537+0000","last_peered":"2024-09-21T06:51:41.600537+0000","last_clean":"2024-09-21T06:51:41.600537+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:51:41.600537+0000","last_undegraded":"2024-09-21T06:51:41.600537+0000","last_fullsized":"2024-09-21T06:51:41.600537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662483+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:51:30.662483+0000","last_peered":"2024-09-21T06:51:30.662483+0000","last_clean":"2024-09-21T06:51:30.662483+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:51:30.662483+0000","last_undegraded":"2024-09-21T06:51:30.662483+0000","last_fullsized":"2024-09-21T06:51:30.662483+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666598+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:51:30.666598+0000","last_peered":"2024-09-21T06:51:30.666598+0000","last_clean":"2024-09-21T06:51:30.666598+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:51:30.666598+0000","last_undegraded":"2024-09-21T06:51:30.666598+0000","last_fullsized":"2024-09-21T06:51:30.666598+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.612148+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:51:42.612148+0000","last_peered":"2024-09-21T06:51:42.612148+0000","last_clean":"2024-09-21T06:51:42.612148+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:51:42.612148+0000","last_undegraded":"2024-09-21T06:51:42.612148+0000","last_fullsized":"2024-09-21T06:51:42.612148+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.920845+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:51:30.920845+0000","last_peered":"2024-09-21T06:51:30.920845+0000","last_clean":"2024-09-21T06:51:30.920845+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:51:30.920845+0000","last_undegraded":"2024-09-21T06:51:30.920845+0000","last_fullsized":"2024-09-21T06:51:30.920845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668247+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:51:30.668247+0000","last_peered":"2024-09-21T06:51:30.668247+0000","last_clean":"2024-09-21T06:51:30.668247+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:51:30.668247+0000","last_undegraded":"2024-09-21T06:51:30.668247+0000","last_fullsized":"2024-09-21T06:51:30.668247+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:58.647279+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:51:58.647279+0000","last_peered":"2024-09-21T06:51:58.647279+0000","last_clean":"2024-09-21T06:51:58.647279+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:51:58.647279+0000","last_undegraded":"2024-09-21T06:51:58.647279+0000","last_fullsized":"2024-09-21T06:51:58.647279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662156+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:51:30.662156+0000","last_peered":"2024-09-21T06:51:30.662156+0000","last_clean":"2024-09-21T06:51:30.662156+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:51:30.662156+0000","last_undegraded":"2024-09-21T06:51:30.662156+0000","last_fullsized":"2024-09-21T06:51:30.662156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665822+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:51:30.665822+0000","last_peered":"2024-09-21T06:51:30.665822+0000","last_clean":"2024-09-21T06:51:30.665822+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:51:30.665822+0000","last_undegraded":"2024-09-21T06:51:30.665822+0000","last_fullsized":"2024-09-21T06:51:30.665822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921124+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:51:30.921124+0000","last_peered":"2024-09-21T06:51:30.921124+0000","last_clean":"2024-09-21T06:51:30.921124+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:51:30.921124+0000","last_undegraded":"2024-09-21T06:51:30.921124+0000","last_fullsized":"2024-09-21T06:51:30.921124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:42.611622+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:51:42.611622+0000","last_peered":"2024-09-21T06:51:42.611622+0000","last_clean":"2024-09-21T06:51:42.611622+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:51:42.611622+0000","last_undegraded":"2024-09-21T06:51:42.611622+0000","last_fullsized":"2024-09-21T06:51:42.611622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667313+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:51:30.667313+0000","last_peered":"2024-09-21T06:51:30.667313+0000","last_clean":"2024-09-21T06:51:30.667313+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:51:30.667313+0000","last_undegraded":"2024-09-21T06:51:30.667313+0000","last_fullsized":"2024-09-21T06:51:30.667313+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662532+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:51:30.662532+0000","last_peered":"2024-09-21T06:51:30.662532+0000","last_clean":"2024-09-21T06:51:30.662532+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:51:30.662532+0000","last_undegraded":"2024-09-21T06:51:30.662532+0000","last_fullsized":"2024-09-21T06:51:30.662532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:40.591629+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:51:40.591629+0000","last_peered":"2024-09-21T06:51:40.591629+0000","last_clean":"2024-09-21T06:51:40.591629+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:51:40.591629+0000","last_undegraded":"2024-09-21T06:51:40.591629+0000","last_fullsized":"2024-09-21T06:51:40.591629+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:41.599791+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:51:41.599791+0000","last_peered":"2024-09-21T06:51:41.599791+0000","last_clean":"2024-09-21T06:51:41.599791+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:51:41.599791+0000","last_undegraded":"2024-09-21T06:51:41.599791+0000","last_fullsized":"2024-09-21T06:51:41.599791+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663203+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:51:30.663203+0000","last_peered":"2024-09-21T06:51:30.663203+0000","last_clean":"2024-09-21T06:51:30.663203+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:51:30.663203+0000","last_undegraded":"2024-09-21T06:51:30.663203+0000","last_fullsized":"2024-09-21T06:51:30.663203+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670164+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:51:30.670164+0000","last_peered":"2024-09-21T06:51:30.670164+0000","last_clean":"2024-09-21T06:51:30.670164+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:51:30.670164+0000","last_undegraded":"2024-09-21T06:51:30.670164+0000","last_fullsized":"2024-09-21T06:51:30.670164+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:59.649524+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:51:59.649524+0000","last_peered":"2024-09-21T06:51:59.649524+0000","last_clean":"2024-09-21T06:51:59.649524+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:51:59.649524+0000","last_undegraded":"2024-09-21T06:51:59.649524+0000","last_fullsized":"2024-09-21T06:51:59.649524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665423+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:51:30.665423+0000","last_peered":"2024-09-21T06:51:30.665423+0000","last_clean":"2024-09-21T06:51:30.665423+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:51:30.665423+0000","last_undegraded":"2024-09-21T06:51:30.665423+0000","last_fullsized":"2024-09-21T06:51:30.665423+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670454+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:51:30.670454+0000","last_peered":"2024-09-21T06:51:30.670454+0000","last_clean":"2024-09-21T06:51:30.670454+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:51:30.670454+0000","last_undegraded":"2024-09-21T06:51:30.670454+0000","last_fullsized":"2024-09-21T06:51:30.670454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665134+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:51:30.665134+0000","last_peered":"2024-09-21T06:51:30.665134+0000","last_clean":"2024-09-21T06:51:30.665134+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:51:30.665134+0000","last_undegraded":"2024-09-21T06:51:30.665134+0000","last_fullsized":"2024-09-21T06:51:30.665134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670351+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:51:30.670351+0000","last_peered":"2024-09-21T06:51:30.670351+0000","last_clean":"2024-09-21T06:51:30.670351+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:51:30.670351+0000","last_undegraded":"2024-09-21T06:51:30.670351+0000","last_fullsized":"2024-09-21T06:51:30.670351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669924+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:51:30.669924+0000","last_peered":"2024-09-21T06:51:30.669924+0000","last_clean":"2024-09-21T06:51:30.669924+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:51:30.669924+0000","last_undegraded":"2024-09-21T06:51:30.669924+0000","last_fullsized":"2024-09-21T06:51:30.669924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665427+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:51:30.665427+0000","last_peered":"2024-09-21T06:51:30.665427+0000","last_clean":"2024-09-21T06:51:30.665427+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:51:30.665427+0000","last_undegraded":"2024-09-21T06:51:30.665427+0000","last_fullsized":"2024-09-21T06:51:30.665427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752677+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:51:30.752677+0000","last_peered":"2024-09-21T06:51:30.752677+0000","last_clean":"2024-09-21T06:51:30.752677+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:51:30.752677+0000","last_undegraded":"2024-09-21T06:51:30.752677+0000","last_fullsized":"2024-09-21T06:51:30.752677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666838+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:51:30.666838+0000","last_peered":"2024-09-21T06:51:30.666838+0000","last_clean":"2024-09-21T06:51:30.666838+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:51:30.666838+0000","last_undegraded":"2024-09-21T06:51:30.666838+0000","last_fullsized":"2024-09-21T06:51:30.666838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670711+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:51:30.670711+0000","last_peered":"2024-09-21T06:51:30.670711+0000","last_clean":"2024-09-21T06:51:30.670711+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:51:30.670711+0000","last_undegraded":"2024-09-21T06:51:30.670711+0000","last_fullsized":"2024-09-21T06:51:30.670711+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663206+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:51:30.663206+0000","last_peered":"2024-09-21T06:51:30.663206+0000","last_clean":"2024-09-21T06:51:30.663206+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:51:30.663206+0000","last_undegraded":"2024-09-21T06:51:30.663206+0000","last_fullsized":"2024-09-21T06:51:30.663206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662767+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:51:30.662767+0000","last_peered":"2024-09-21T06:51:30.662767+0000","last_clean":"2024-09-21T06:51:30.662767+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:51:30.662767+0000","last_undegraded":"2024-09-21T06:51:30.662767+0000","last_fullsized":"2024-09-21T06:51:30.662767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662324+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:51:30.662324+0000","last_peered":"2024-09-21T06:51:30.662324+0000","last_clean":"2024-09-21T06:51:30.662324+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:51:30.662324+0000","last_undegraded":"2024-09-21T06:51:30.662324+0000","last_fullsized":"2024-09-21T06:51:30.662324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666912+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:51:30.666912+0000","last_peered":"2024-09-21T06:51:30.666912+0000","last_clean":"2024-09-21T06:51:30.666912+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:51:30.666912+0000","last_undegraded":"2024-09-21T06:51:30.666912+0000","last_fullsized":"2024-09-21T06:51:30.666912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752414+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:51:30.752414+0000","last_peered":"2024-09-21T06:51:30.752414+0000","last_clean":"2024-09-21T06:51:30.752414+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:51:30.752414+0000","last_undegraded":"2024-09-21T06:51:30.752414+0000","last_fullsized":"2024-09-21T06:51:30.752414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.666327+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:51:30.666327+0000","last_peered":"2024-09-21T06:51:30.666327+0000","last_clean":"2024-09-21T06:51:30.666327+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:51:30.666327+0000","last_undegraded":"2024-09-21T06:51:30.666327+0000","last_fullsized":"2024-09-21T06:51:30.666327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.670589+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:51:30.670589+0000","last_peered":"2024-09-21T06:51:30.670589+0000","last_clean":"2024-09-21T06:51:30.670589+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:51:30.670589+0000","last_undegraded":"2024-09-21T06:51:30.670589+0000","last_fullsized":"2024-09-21T06:51:30.670589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.921709+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:51:30.921709+0000","last_peered":"2024-09-21T06:51:30.921709+0000","last_clean":"2024-09-21T06:51:30.921709+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:51:30.921709+0000","last_undegraded":"2024-09-21T06:51:30.921709+0000","last_fullsized":"2024-09-21T06:51:30.921709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668242+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:51:30.668242+0000","last_peered":"2024-09-21T06:51:30.668242+0000","last_clean":"2024-09-21T06:51:30.668242+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:51:30.668242+0000","last_undegraded":"2024-09-21T06:51:30.668242+0000","last_fullsized":"2024-09-21T06:51:30.668242+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665895+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:51:30.665895+0000","last_peered":"2024-09-21T06:51:30.665895+0000","last_clean":"2024-09-21T06:51:30.665895+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:51:30.665895+0000","last_undegraded":"2024-09-21T06:51:30.665895+0000","last_fullsized":"2024-09-21T06:51:30.665895+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.673289+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:51:30.673289+0000","last_peered":"2024-09-21T06:51:30.673289+0000","last_clean":"2024-09-21T06:51:30.673289+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:51:30.673289+0000","last_undegraded":"2024-09-21T06:51:30.673289+0000","last_fullsized":"2024-09-21T06:51:30.673289+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665282+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:51:30.665282+0000","last_peered":"2024-09-21T06:51:30.665282+0000","last_clean":"2024-09-21T06:51:30.665282+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:51:30.665282+0000","last_undegraded":"2024-09-21T06:51:30.665282+0000","last_fullsized":"2024-09-21T06:51:30.665282+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.752334+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:51:30.752334+0000","last_peered":"2024-09-21T06:51:30.752334+0000","last_clean":"2024-09-21T06:51:30.752334+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:51:30.752334+0000","last_undegraded":"2024-09-21T06:51:30.752334+0000","last_fullsized":"2024-09-21T06:51:30.752334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.663076+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:51:30.663076+0000","last_peered":"2024-09-21T06:51:30.663076+0000","last_clean":"2024-09-21T06:51:30.663076+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:51:30.663076+0000","last_undegraded":"2024-09-21T06:51:30.663076+0000","last_fullsized":"2024-09-21T06:51:30.663076+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.669589+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:51:30.669589+0000","last_peered":"2024-09-21T06:51:30.669589+0000","last_clean":"2024-09-21T06:51:30.669589+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:51:30.669589+0000","last_undegraded":"2024-09-21T06:51:30.669589+0000","last_fullsized":"2024-09-21T06:51:30.669589+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665818+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:51:30.665818+0000","last_peered":"2024-09-21T06:51:30.665818+0000","last_clean":"2024-09-21T06:51:30.665818+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:51:30.665818+0000","last_undegraded":"2024-09-21T06:51:30.665818+0000","last_fullsized":"2024-09-21T06:51:30.665818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668444+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:51:30.668444+0000","last_peered":"2024-09-21T06:51:30.668444+0000","last_clean":"2024-09-21T06:51:30.668444+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:51:30.668444+0000","last_undegraded":"2024-09-21T06:51:30.668444+0000","last_fullsized":"2024-09-21T06:51:30.668444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662205+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:51:30.662205+0000","last_peered":"2024-09-21T06:51:30.662205+0000","last_clean":"2024-09-21T06:51:30.662205+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:51:30.662205+0000","last_undegraded":"2024-09-21T06:51:30.662205+0000","last_fullsized":"2024-09-21T06:51:30.662205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.668841+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:51:30.668841+0000","last_peered":"2024-09-21T06:51:30.668841+0000","last_clean":"2024-09-21T06:51:30.668841+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:51:30.668841+0000","last_undegraded":"2024-09-21T06:51:30.668841+0000","last_fullsized":"2024-09-21T06:51:30.668841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665667+0000","last_change":"2024-09-21T06:50:47.214866+0000","last_active":"2024-09-21T06:51:30.665667+0000","last_peered":"2024-09-21T06:51:30.665667+0000","last_clean":"2024-09-21T06:51:30.665667+0000","last_became_active":"2024-09-21T06:50:47.214451+0000","last_became_peered":"2024-09-21T06:50:47.214451+0000","last_unstale":"2024-09-21T06:51:30.665667+0000","last_undegraded":"2024-09-21T06:51:30.665667+0000","last_fullsized":"2024-09-21T06:51:30.665667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:37.914291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665432+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:51:30.665432+0000","last_peered":"2024-09-21T06:51:30.665432+0000","last_clean":"2024-09-21T06:51:30.665432+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:51:30.665432+0000","last_undegraded":"2024-09-21T06:51:30.665432+0000","last_fullsized":"2024-09-21T06:51:30.665432+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.662970+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:51:30.662970+0000","last_peered":"2024-09-21T06:51:30.662970+0000","last_clean":"2024-09-21T06:51:30.662970+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:51:30.662970+0000","last_undegraded":"2024-09-21T06:51:30.662970+0000","last_fullsized":"2024-09-21T06:51:30.662970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665099+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:51:30.665099+0000","last_peered":"2024-09-21T06:51:30.665099+0000","last_clean":"2024-09-21T06:51:30.665099+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:51:30.665099+0000","last_undegraded":"2024-09-21T06:51:30.665099+0000","last_fullsized":"2024-09-21T06:51:30.665099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.665965+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:51:30.665965+0000","last_peered":"2024-09-21T06:51:30.665965+0000","last_clean":"2024-09-21T06:51:30.665965+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:51:30.665965+0000","last_undegraded":"2024-09-21T06:51:30.665965+0000","last_fullsized":"2024-09-21T06:51:30.665965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-21T06:51:30.667562+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:51:30.667562+0000","last_peered":"2024-09-21T06:51:30.667562+0000","last_clean":"2024-09-21T06:51:30.667562+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:51:30.667562+0000","last_undegraded":"2024-09-21T06:51:30.667562+0000","last_fullsized":"2024-09-21T06:51:30.667562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64144,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":212,"num_read_kb":177,"num_write":246,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":200962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":512314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906976,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1691648,"data_stored":1155266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070502,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":515927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":512187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1622016,"data_stored":1109097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626488,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":507726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1104288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":511409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":507043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378704,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1081344,"data_stored":569168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":570583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705754,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":509182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:52:05.530 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:52:05.531 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T06:52:05.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T06:52:05.937 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 -- 172.21.15.114:0/2915556587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10701039c0 msgr2=0x7f1070103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 --2- 172.21.15.114:0/2915556587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10701039c0 0x7f1070103dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1064009900 tx=0x7f106402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 -- 172.21.15.114:0/2915556587 shutdown_connections 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 --2- 172.21.15.114:0/2915556587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1070109c50 0x7f1070110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 --2- 172.21.15.114:0/2915556587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1070109300 0x7f1070109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 --2- 172.21.15.114:0/2915556587 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10701039c0 0x7f1070103dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 -- 172.21.15.114:0/2915556587 >> 172.21.15.114:0/2915556587 conn(0x7f10700ff430 msgr2=0x7f1070101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 -- 172.21.15.114:0/2915556587 shutdown_connections 2024-09-21T06:52:06.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.580+0000 7f1076e00640 1 -- 172.21.15.114:0/2915556587 wait complete. 2024-09-21T06:52:06.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.581+0000 7f1076e00640 1 Processor -- start 2024-09-21T06:52:06.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.581+0000 7f1076e00640 1 -- start start 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.581+0000 7f1076e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701039c0 0x7f107019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.581+0000 7f1076e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1070109300 0x7f107019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 0x7f107019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1070113910 con 0x7f1070109c50 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1070113790 con 0x7f10701039c0 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1070113a90 con 0x7f1070109300 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 0x7f107019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1075000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1070109300 0x7f107019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 0x7f107019fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35990/0 (socket says 172.21.15.114:35990) 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 -- 172.21.15.114:0/1593289434 learned_addr learned my addr 172.21.15.114:0/1593289434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:06.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1075a00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701039c0 0x7f107019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 -- 172.21.15.114:0/1593289434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1070109300 msgr2=0x7f107019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1070109300 0x7f107019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 -- 172.21.15.114:0/1593289434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701039c0 msgr2=0x7f107019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701039c0 0x7f107019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.582+0000 7f1076400640 1 -- 172.21.15.114:0/1593289434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10701a02d0 con 0x7f1070109c50 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.583+0000 7f1076400640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 0x7f107019fba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f106800efd0 tx=0x7f106800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.583+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068003b70 con 0x7f1070109c50 2024-09-21T06:52:06.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.583+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1068003d10 con 0x7f1070109c50 2024-09-21T06:52:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.583+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10680108a0 con 0x7f1070109c50 2024-09-21T06:52:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.583+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10701a05c0 con 0x7f1070109c50 2024-09-21T06:52:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.583+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701a6d20 con 0x7f1070109c50 2024-09-21T06:52:06.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.584+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1070103dd0 con 0x7f1070109c50 2024-09-21T06:52:06.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.587+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f1068004b40 con 0x7f1070109c50 2024-09-21T06:52:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.588+0000 7f106e000640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10580b6660 0x7f10580b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.588+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f10680e80d0 con 0x7f1070109c50 2024-09-21T06:52:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.589+0000 7f1075a00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10580b6660 0x7f10580b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:06.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.589+0000 7f1075a00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10580b6660 0x7f10580b8b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1064031440 tx=0x7f1064009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:06.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.590+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10680ae380 con 0x7f1070109c50 2024-09-21T06:52:06.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:06 smithi114 ceph-mon[30654]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:06.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:06 smithi114 ceph-mon[30654]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-09-21T06:52:06.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.848+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f107019b490 con 0x7f1070109c50 2024-09-21T06:52:06.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.852+0000 7f106e000640 1 -- 172.21.15.114:0/1593289434 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f107019b490 con 0x7f1070109c50 2024-09-21T06:52:06.853 INFO:teuthology.orchestra.run.smithi114.stdout:[client.0] 2024-09-21T06:52:06.853 INFO:teuthology.orchestra.run.smithi114.stdout: key = AQAWbe5mKDirMhAAM7Gok5heYzPCalVpxtXB3Q== 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10580b6660 msgr2=0x7f10580b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10580b6660 0x7f10580b8b70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1064031440 tx=0x7f1064009560 comp rx=0 tx=0).stop 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 msgr2=0x7f107019fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 0x7f107019fba0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f106800efd0 tx=0x7f106800c6a0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 shutdown_connections 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1070109c50 0x7f107019fba0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1070109300 0x7f107019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10580b6660 0x7f10580b8b70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 --2- 172.21.15.114:0/1593289434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10701039c0 0x7f107019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 >> 172.21.15.114:0/1593289434 conn(0x7f10700ff430 msgr2=0x7f1070101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:06.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.854+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 shutdown_connections 2024-09-21T06:52:06.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:06.855+0000 7f1076e00640 1 -- 172.21.15.114:0/1593289434 wait complete. 2024-09-21T06:52:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:06 smithi183 ceph-mon[34219]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:06.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:06 smithi183 ceph-mon[34219]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-09-21T06:52:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:06 smithi193 ceph-mon[34601]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:06 smithi193 ceph-mon[34601]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-09-21T06:52:07.669 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:52:07.669 DEBUG:teuthology.orchestra.run.smithi114:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T06:52:07.669 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T06:52:07.750 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T06:52:07.838 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:07 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1593289434' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:52:07.838 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:07 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1593289434' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:52:07.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:07 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1593289434' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:52:07.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:07 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1593289434' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:52:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:07 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1593289434' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T06:52:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:07 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1593289434' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T06:52:08.157 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 -- 172.21.15.114:0/4267134438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd58103b80 msgr2=0x7fbd58103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/4267134438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd58103b80 0x7fbd58103f90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009be0 tx=0x7fbd4802a6c0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 -- 172.21.15.114:0/4267134438 shutdown_connections 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/4267134438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd58109e10 0x7fbd58110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/4267134438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd581094c0 0x7fbd581098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/4267134438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd58103b80 0x7fbd58103f90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 -- 172.21.15.114:0/4267134438 >> 172.21.15.114:0/4267134438 conn(0x7fbd580ff5d0 msgr2=0x7fbd58101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 -- 172.21.15.114:0/4267134438 shutdown_connections 2024-09-21T06:52:08.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.782+0000 7fbd5dc00640 1 -- 172.21.15.114:0/4267134438 wait complete. 2024-09-21T06:52:08.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.783+0000 7fbd5dc00640 1 Processor -- start 2024-09-21T06:52:08.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 -- start start 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd58103b80 0x7fbd5819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd581094c0 0x7fbd5819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 0x7fbd5819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58113ad0 con 0x7fbd58109e10 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58113950 con 0x7fbd58103b80 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd5dc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbd58113c50 con 0x7fbd581094c0 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd57400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 0x7fbd5819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd57400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 0x7fbd5819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36006/0 (socket says 172.21.15.114:36006) 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd57400640 1 -- 172.21.15.114:0/1685402903 learned_addr learned my addr 172.21.15.114:0/1685402903 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:08.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd56a00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd58103b80 0x7fbd5819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.784+0000 7fbd56000640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd581094c0 0x7fbd5819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd57400640 1 -- 172.21.15.114:0/1685402903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd581094c0 msgr2=0x7fbd5819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd57400640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd581094c0 0x7fbd5819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd57400640 1 -- 172.21.15.114:0/1685402903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd58103b80 msgr2=0x7fbd5819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd57400640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd58103b80 0x7fbd5819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd57400640 1 -- 172.21.15.114:0/1685402903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd581a04b0 con 0x7fbd58109e10 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd57400640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 0x7fbd5819fd80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000e970 tx=0x7fbd5000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.785+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000ccc0 con 0x7fbd58109e10 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.786+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd581a07a0 con 0x7fbd58109e10 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.786+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd5000ce60 con 0x7fbd58109e10 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.786+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd50010430 con 0x7fbd58109e10 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.786+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd581a6f00 con 0x7fbd58109e10 2024-09-21T06:52:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.787+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd58103f90 con 0x7fbd58109e10 2024-09-21T06:52:08.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.789+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fbd50004f90 con 0x7fbd58109e10 2024-09-21T06:52:08.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.790+0000 7fbd4f400640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbd3c0b6670 0x7fbd3c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:08.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.791+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fbd500e7c80 con 0x7fbd58109e10 2024-09-21T06:52:08.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.791+0000 7fbd56a00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbd3c0b6670 0x7fbd3c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:08.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.791+0000 7fbd56a00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbd3c0b6670 0x7fbd3c0b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006010 tx=0x7fbd48004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:08.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.794+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd500adf30 con 0x7fbd58109e10 2024-09-21T06:52:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:08 smithi183 ceph-mon[34219]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T06:52:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:08 smithi193 ceph-mon[34601]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T06:52:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.982+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fbd5810b650 con 0x7fbd58109e10 2024-09-21T06:52:08.985 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:08 smithi114 ceph-mon[30654]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-21T06:52:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.983+0000 7fbd4f400640 1 -- 172.21.15.114:0/1685402903 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v55) ==== 86+0+133 (secure 0 0 0) 0x7fbd500b2310 con 0x7fbd58109e10 2024-09-21T06:52:08.985 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbd3c0b6670 msgr2=0x7fbd3c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbd3c0b6670 0x7fbd3c0b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd48006010 tx=0x7fbd48004a40 comp rx=0 tx=0).stop 2024-09-21T06:52:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 msgr2=0x7fbd5819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 0x7fbd5819fd80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000e970 tx=0x7fbd5000ed70 comp rx=0 tx=0).stop 2024-09-21T06:52:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 shutdown_connections 2024-09-21T06:52:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbd58109e10 0x7fbd5819fd80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd581094c0 0x7fbd5819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbd3c0b6670 0x7fbd3c0b8b80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 --2- 172.21.15.114:0/1685402903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd58103b80 0x7fbd5819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.985+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 >> 172.21.15.114:0/1685402903 conn(0x7fbd580ff5d0 msgr2=0x7fbd581019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.986+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 shutdown_connections 2024-09-21T06:52:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:08.986+0000 7fbd5dc00640 1 -- 172.21.15.114:0/1685402903 wait complete. 2024-09-21T06:52:09.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:09 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1685402903' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:52:09.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:09 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1685402903' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:52:09.887 INFO:teuthology.orchestra.run.smithi114.stdout:[{"version":55,"timestamp":"2024-09-21T06:50:35.738760+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T06:52:09.888 INFO:tasks.ceph_manager:config epoch is 55 2024-09-21T06:52:09.888 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T06:52:09.888 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T06:52:09.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T06:52:09.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:09 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1685402903' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T06:52:10.253 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:10.736 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:10 smithi114 ceph-mon[30654]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-21T06:52:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:10 smithi183 ceph-mon[34219]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-21T06:52:10.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 -- 172.21.15.114:0/120236201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 msgr2=0x7f8b2810fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:10.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 --2- 172.21.15.114:0/120236201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2810fce0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009900 tx=0x7f8b1402a480 comp rx=0 tx=0).stop 2024-09-21T06:52:10.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 -- 172.21.15.114:0/120236201 shutdown_connections 2024-09-21T06:52:10.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 --2- 172.21.15.114:0/120236201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b28110220 0x7f8b28112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:10.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 --2- 172.21.15.114:0/120236201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2810fce0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:10.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 --2- 172.21.15.114:0/120236201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 0x7f8b28102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:10.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 -- 172.21.15.114:0/120236201 >> 172.21.15.114:0/120236201 conn(0x7f8b28079740 msgr2=0x7f8b28079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:10.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 -- 172.21.15.114:0/120236201 shutdown_connections 2024-09-21T06:52:10.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.908+0000 7f8b30000640 1 -- 172.21.15.114:0/120236201 wait complete. 2024-09-21T06:52:10.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:10 smithi193 ceph-mon[34601]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-21T06:52:10.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.909+0000 7f8b30000640 1 Processor -- start 2024-09-21T06:52:10.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.909+0000 7f8b30000640 1 -- start start 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b30000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 0x7f8b2819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b30000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b30000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b28110220 0x7f8b281a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b30000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28115c90 con 0x7f8b28102fc0 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b30000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28115b10 con 0x7f8b28110220 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b30000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b28115e10 con 0x7f8b28102670 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b2cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b2d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 0x7f8b2819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b2d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 0x7f8b2819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35052/0 (socket says 172.21.15.114:35052) 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b2cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36034/0 (socket says 172.21.15.114:36034) 2024-09-21T06:52:10.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b2d600640 1 -- 172.21.15.114:0/1804012011 learned_addr learned my addr 172.21.15.114:0/1804012011 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.910+0000 7f8b2e000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b28110220 0x7f8b281a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b2cc00640 1 -- 172.21.15.114:0/1804012011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 msgr2=0x7f8b2819c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b2cc00640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 0x7f8b2819c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b2cc00640 1 -- 172.21.15.114:0/1804012011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b28110220 msgr2=0x7f8b281a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b2cc00640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b28110220 0x7f8b281a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b2cc00640 1 -- 172.21.15.114:0/1804012011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b281a25d0 con 0x7f8b28102fc0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b2cc00640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2819cc00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b1402a9a0 tx=0x7f8b14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.911+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1403f030 con 0x7f8b28102fc0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.912+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b14003aa0 con 0x7f8b28102fc0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.912+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b281a2860 con 0x7f8b28102fc0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.912+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14035070 con 0x7f8b28102fc0 2024-09-21T06:52:10.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.912+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b281a9020 con 0x7f8b28102fc0 2024-09-21T06:52:10.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.913+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b2806c1c0 con 0x7f8b28102fc0 2024-09-21T06:52:10.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.915+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f8b14042030 con 0x7f8b28102fc0 2024-09-21T06:52:10.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.917+0000 7f8b26000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b100b65f0 0x7f8b100b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:10.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.917+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f8b14107f40 con 0x7f8b28102fc0 2024-09-21T06:52:10.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.917+0000 7f8b2d600640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b100b65f0 0x7f8b100b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:10.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.918+0000 7f8b2d600640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b100b65f0 0x7f8b100b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c005fd0 tx=0x7f8b1c00b480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:10.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:10.921+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b140d25d0 con 0x7f8b28102fc0 2024-09-21T06:52:11.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.157+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8b281a91f0 con 0x7f8b28102fc0 2024-09-21T06:52:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.162+0000 7f8b26000640 1 -- 172.21.15.114:0/1804012011 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v22) ==== 74+0+296462 (secure 0 0 0) 0x7f8b140d7510 con 0x7f8b28102fc0 2024-09-21T06:52:11.169 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b100b65f0 msgr2=0x7f8b100b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b100b65f0 0x7f8b100b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c005fd0 tx=0x7f8b1c00b480 comp rx=0 tx=0).stop 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 msgr2=0x7f8b2819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2819cc00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b1402a9a0 tx=0x7f8b14004270 comp rx=0 tx=0).stop 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 shutdown_connections 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b100b65f0 0x7f8b100b8b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b28110220 0x7f8b281a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b28102fc0 0x7f8b2819cc00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:11.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 --2- 172.21.15.114:0/1804012011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b28102670 0x7f8b2819c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:11.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.167+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 >> 172.21.15.114:0/1804012011 conn(0x7f8b28079740 msgr2=0x7f8b28108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:11.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.168+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 shutdown_connections 2024-09-21T06:52:11.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:11.168+0000 7f8b30000640 1 -- 172.21.15.114:0/1804012011 wait complete. 2024-09-21T06:52:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:11 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1804012011' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:52:11.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:11 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1804012011' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:52:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:11 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1804012011' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T06:52:12.038 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":22,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6800","nonce":1914683713},{"type":"v1","addr":"172.21.15.114:6801","nonce":1914683713}]},"active_addr":"172.21.15.114:6801/1914683713","active_change":"2024-09-21T06:43:29.388120+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24157,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24158,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.114:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":117743516}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":1996042013}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":3447237743}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":3970362811}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.114:0","nonce":1289873613}]}]} 2024-09-21T06:52:12.050 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T06:52:12.050 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T06:52:12.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:52:12.457 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:12.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:12 smithi183 ceph-mon[34219]: pgmap v336: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-21T06:52:12.904 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:12 smithi114 ceph-mon[30654]: pgmap v336: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-21T06:52:12.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:12 smithi193 ceph-mon[34601]: pgmap v336: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 -- 172.21.15.114:0/1449479695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b41094e0 msgr2=0x7f37b41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 --2- 172.21.15.114:0/1449479695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b41094e0 0x7f37b41098f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009900 tx=0x7f37a002a410 comp rx=0 tx=0).stop 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 -- 172.21.15.114:0/1449479695 shutdown_connections 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 --2- 172.21.15.114:0/1449479695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b4109e30 0x7f37b4110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 --2- 172.21.15.114:0/1449479695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b41094e0 0x7f37b41098f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 --2- 172.21.15.114:0/1449479695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b4103ba0 0x7f37b4103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 -- 172.21.15.114:0/1449479695 >> 172.21.15.114:0/1449479695 conn(0x7f37b40ff5f0 msgr2=0x7f37b4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 -- 172.21.15.114:0/1449479695 shutdown_connections 2024-09-21T06:52:13.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.072+0000 7f37bc200640 1 -- 172.21.15.114:0/1449479695 wait complete. 2024-09-21T06:52:13.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.073+0000 7f37bc200640 1 Processor -- start 2024-09-21T06:52:13.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.073+0000 7f37bc200640 1 -- start start 2024-09-21T06:52:13.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37bc200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 0x7f37b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:13.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37bc200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b41094e0 0x7f37b4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:13.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37bc200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b4109e30 0x7f37b4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:13.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37bc200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4113af0 con 0x7f37b4103ba0 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37bc200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4113970 con 0x7f37b41094e0 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37bc200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4113c70 con 0x7f37b4109e30 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 0x7f37b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37b8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b41094e0 0x7f37b4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 0x7f37b4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36060/0 (socket says 172.21.15.114:36060) 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37b8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b41094e0 0x7f37b4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60250/0 (socket says 172.21.15.114:60250) 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37b9800640 1 -- 172.21.15.114:0/2418812727 learned_addr learned my addr 172.21.15.114:0/2418812727 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.074+0000 7f37ba200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b4109e30 0x7f37b4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b9800640 1 -- 172.21.15.114:0/2418812727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b4109e30 msgr2=0x7f37b4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b9800640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b4109e30 0x7f37b4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b9800640 1 -- 172.21.15.114:0/2418812727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b41094e0 msgr2=0x7f37b4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b9800640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b41094e0 0x7f37b4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b9800640 1 -- 172.21.15.114:0/2418812727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b41a8780 con 0x7f37b4103ba0 2024-09-21T06:52:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b9800640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 0x7f37b4075710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37a800e9f0 tx=0x7f37a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a800cd80 con 0x7f37b4103ba0 2024-09-21T06:52:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b41a8a70 con 0x7f37b4103ba0 2024-09-21T06:52:13.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37a80054d0 con 0x7f37b4103ba0 2024-09-21T06:52:13.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8010690 con 0x7f37b4103ba0 2024-09-21T06:52:13.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.075+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b41a8f80 con 0x7f37b4103ba0 2024-09-21T06:52:13.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.076+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b4103fb0 con 0x7f37b4103ba0 2024-09-21T06:52:13.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.079+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f37a8004e90 con 0x7f37b4103ba0 2024-09-21T06:52:13.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.080+0000 7f37b2000640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f379c0b6670 0x7f379c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:13.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.080+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f37a80e7cf0 con 0x7f37b4103ba0 2024-09-21T06:52:13.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.080+0000 7f37b8e00640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f379c0b6670 0x7f379c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:13.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.081+0000 7f37b8e00640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f379c0b6670 0x7f379c0b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009230 tx=0x7f37a0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:13.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.084+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37a80adfa0 con 0x7f37b4103ba0 2024-09-21T06:52:13.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.271+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37b402e640 con 0x7f37b4103ba0 2024-09-21T06:52:13.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.273+0000 7f37b2000640 1 -- 172.21.15.114:0/2418812727 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+25090 (secure 0 0 0) 0x7f37a80b2380 con 0x7f37b4103ba0 2024-09-21T06:52:13.276 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:13.276 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":87,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:30.657710+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:52:13.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f379c0b6670 msgr2=0x7f379c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:13.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f379c0b6670 0x7f379c0b8b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009230 tx=0x7f37a0031400 comp rx=0 tx=0).stop 2024-09-21T06:52:13.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 msgr2=0x7f37b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:13.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 0x7f37b4075710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f37a800e9f0 tx=0x7f37a800edf0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 shutdown_connections 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b4109e30 0x7f37b4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f379c0b6670 0x7f379c0b8b80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b41094e0 0x7f37b4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.275+0000 7f37bc200640 1 --2- 172.21.15.114:0/2418812727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37b4103ba0 0x7f37b4075710 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.276+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 >> 172.21.15.114:0/2418812727 conn(0x7f37b40ff5f0 msgr2=0x7f37b4101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.276+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 shutdown_connections 2024-09-21T06:52:13.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:13.276+0000 7f37bc200640 1 -- 172.21.15.114:0/2418812727 wait complete. 2024-09-21T06:52:13.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:13 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2418812727' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:13.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:13 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2418812727' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:14.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:13 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2418812727' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:14.197 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T06:52:14.197 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:52:14.564 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:14 smithi183 ceph-mon[34219]: pgmap v337: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T06:52:14.882 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:14 smithi114 ceph-mon[30654]: pgmap v337: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T06:52:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:14 smithi193 ceph-mon[34601]: pgmap v337: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T06:52:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 -- 172.21.15.114:0/49030722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c106c20 msgr2=0x7f649c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 --2- 172.21.15.114:0/49030722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c106c20 0x7f649c110670 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6494009900 tx=0x7f649402a410 comp rx=0 tx=0).stop 2024-09-21T06:52:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 -- 172.21.15.114:0/49030722 shutdown_connections 2024-09-21T06:52:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 --2- 172.21.15.114:0/49030722 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c106c20 0x7f649c110670 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 --2- 172.21.15.114:0/49030722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c1062d0 0x7f649c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 --2- 172.21.15.114:0/49030722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c06bd50 0x7f649c06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 -- 172.21.15.114:0/49030722 >> 172.21.15.114:0/49030722 conn(0x7f649c0799a0 msgr2=0x7f649c079db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:15.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 -- 172.21.15.114:0/49030722 shutdown_connections 2024-09-21T06:52:15.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.207+0000 7f64a2000640 1 -- 172.21.15.114:0/49030722 wait complete. 2024-09-21T06:52:15.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 Processor -- start 2024-09-21T06:52:15.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 -- start start 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c06bd50 0x7f649c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c1062d0 0x7f649c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c106c20 0x7f649c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f649c113ac0 con 0x7f649c1062d0 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f649c113940 con 0x7f649c106c20 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.209+0000 7f64a2000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f649c113c40 con 0x7f649c06bd50 2024-09-21T06:52:15.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c06bd50 0x7f649c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:15.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c1062d0 0x7f649c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:15.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c06bd50 0x7f649c19a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42280/0 (socket says 172.21.15.114:42280) 2024-09-21T06:52:15.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649b400640 1 -- 172.21.15.114:0/1586259805 learned_addr learned my addr 172.21.15.114:0/1586259805 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:15.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649be00640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c106c20 0x7f649c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649aa00640 1 -- 172.21.15.114:0/1586259805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c06bd50 msgr2=0x7f649c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649aa00640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c06bd50 0x7f649c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649aa00640 1 -- 172.21.15.114:0/1586259805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c106c20 msgr2=0x7f649c19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649aa00640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c106c20 0x7f649c19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649aa00640 1 -- 172.21.15.114:0/1586259805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649c1a0360 con 0x7f649c1062d0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.210+0000 7f649be00640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c106c20 0x7f649c19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.211+0000 7f649aa00640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c1062d0 0x7f649c19a990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f648800e9c0 tx=0x7f648800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.211+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648800ccc0 con 0x7f649c1062d0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.211+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6488005020 con 0x7f649c1062d0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.211+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488004a00 con 0x7f649c1062d0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.211+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649c1a0650 con 0x7f649c1062d0 2024-09-21T06:52:15.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.211+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649c1a6db0 con 0x7f649c1062d0 2024-09-21T06:52:15.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.212+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649c06c160 con 0x7f649c1062d0 2024-09-21T06:52:15.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.216+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f64880051d0 con 0x7f649c1062d0 2024-09-21T06:52:15.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.217+0000 7f6493e00640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64800b65f0 0x7f64800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:15.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.218+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f64880e7990 con 0x7f649c1062d0 2024-09-21T06:52:15.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.218+0000 7f649b400640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64800b65f0 0x7f64800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:15.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.218+0000 7f649b400640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64800b65f0 0x7f64800b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f648c009fd0 tx=0x7f648c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:15.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.220+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64880add30 con 0x7f649c1062d0 2024-09-21T06:52:15.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.414+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f649c19b520 con 0x7f649c1062d0 2024-09-21T06:52:15.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.415+0000 7f6493e00640 1 -- 172.21.15.114:0/1586259805 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+25090 (secure 0 0 0) 0x7f64880b2110 con 0x7f649c1062d0 2024-09-21T06:52:15.418 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:15.418 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":87,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:51:30.657710+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64800b65f0 msgr2=0x7f64800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64800b65f0 0x7f64800b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f648c009fd0 tx=0x7f648c009500 comp rx=0 tx=0).stop 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c1062d0 msgr2=0x7f649c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c1062d0 0x7f649c19a990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f648800e9c0 tx=0x7f648800ee80 comp rx=0 tx=0).stop 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 shutdown_connections 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64800b65f0 0x7f64800b8b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f649c106c20 0x7f649c19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f649c1062d0 0x7f649c19a990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 --2- 172.21.15.114:0/1586259805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f649c06bd50 0x7f649c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:15.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 >> 172.21.15.114:0/1586259805 conn(0x7f649c0799a0 msgr2=0x7f649c1074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:15.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.418+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 shutdown_connections 2024-09-21T06:52:15.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:15.419+0000 7f64a2000640 1 -- 172.21.15.114:0/1586259805 wait complete. 2024-09-21T06:52:15.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:15 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1586259805' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:15.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:15 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1586259805' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:15.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:15 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1586259805' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:52:16.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T06:52:16.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T06:52:16.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T06:52:16.318 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T06:52:16.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T06:52:16.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T06:52:16.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T06:52:16.320 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T06:52:16.320 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T06:52:16.321 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T06:52:16.321 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T06:52:16.321 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T06:52:16.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:16 smithi183 ceph-mon[34219]: pgmap v338: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T06:52:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:16 smithi193 ceph-mon[34601]: pgmap v338: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T06:52:17.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:16 smithi114 ceph-mon[30654]: pgmap v338: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-21T06:52:17.308 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.312 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.342 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.347 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.440 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.516 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.517 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.522 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:17.523 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:18.560 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:18.673 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:18 smithi114 ceph-mon[30654]: pgmap v339: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T06:52:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:18 smithi183 ceph-mon[34219]: pgmap v339: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T06:52:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:18 smithi193 ceph-mon[34601]: pgmap v339: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-21T06:52:19.816 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 -- 172.21.15.114:0/2202399367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7144105320 msgr2=0x7f714410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 --2- 172.21.15.114:0/2202399367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7144105320 0x7f714410d870 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7130009900 tx=0x7f713002a410 comp rx=0 tx=0).stop 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 -- 172.21.15.114:0/2202399367 shutdown_connections 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 --2- 172.21.15.114:0/2202399367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714410de30 0x7f7144110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 --2- 172.21.15.114:0/2202399367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7144105320 0x7f714410d870 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 --2- 172.21.15.114:0/2202399367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71441049d0 0x7f7144104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 -- 172.21.15.114:0/2202399367 >> 172.21.15.114:0/2202399367 conn(0x7f7144079740 msgr2=0x7f7144079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 -- 172.21.15.114:0/2202399367 shutdown_connections 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.267+0000 7f714c600640 1 -- 172.21.15.114:0/2202399367 wait complete. 2024-09-21T06:52:20.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.268+0000 7f714c600640 1 Processor -- start 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 -- start start 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71441049d0 0x7f714419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7144105320 0x7f714419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f714410de30 0x7f714419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7144113ae0 con 0x7f71441049d0 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7144113960 con 0x7f714410de30 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714c600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7144113c60 con 0x7f7144105320 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f7149c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71441049d0 0x7f714419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f7149c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71441049d0 0x7f714419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58454/0 (socket says 172.21.15.114:58454) 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f7149c00640 1 -- 172.21.15.114:0/4278890572 learned_addr learned my addr 172.21.15.114:0/4278890572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f7149200640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7144105320 0x7f714419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.269+0000 7f714a600640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f714410de30 0x7f714419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:20.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7149c00640 1 -- 172.21.15.114:0/4278890572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7144105320 msgr2=0x7f714419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7149c00640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7144105320 0x7f714419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7149c00640 1 -- 172.21.15.114:0/4278890572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f714410de30 msgr2=0x7f714419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7149c00640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f714410de30 0x7f714419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7149c00640 1 -- 172.21.15.114:0/4278890572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71441a0440 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7149c00640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71441049d0 0x7f714419a4d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f713800e9c0 tx=0x7f713800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713800cd80 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71441a0730 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.270+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71380035f0 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.271+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138018630 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.271+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71441a6e90 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.272+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7144100fe0 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.274+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f71380037a0 con 0x7f71441049d0 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.275+0000 7f7142000640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f712c0b65f0 0x7f712c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:20.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.275+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f71380e7ea0 con 0x7f71441049d0 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.276+0000 7f7149200640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f712c0b65f0 0x7f712c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.276+0000 7f7142000640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] conn(0x7f712c0c1900 0x7f712c0c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.276+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 --> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f712c0c44c0 con 0x7f712c0c1900 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.276+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f71380e8250 con 0x7f71441049d0 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.276+0000 7f714a600640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] conn(0x7f712c0c1900 0x7f712c0c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.276+0000 7f7149200640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f712c0b65f0 0x7f712c0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f713002aa00 tx=0x7f7130005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.277+0000 7f714a600640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] conn(0x7f712c0c1900 0x7f712c0c3df0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.278+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== osd.3 v2:172.21.15.114:6810/2440023671 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f712c0c44c0 con 0x7f712c0c1900 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.292+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 --> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f714406b390 con 0x7f712c0c1900 2024-09-21T06:52:20.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.292+0000 7f7142000640 1 -- 172.21.15.114:0/4278890572 <== osd.3 v2:172.21.15.114:6810/2440023671 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f714406b390 con 0x7f712c0c1900 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] conn(0x7f712c0c1900 msgr2=0x7f712c0c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] conn(0x7f712c0c1900 0x7f712c0c3df0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f712c0b65f0 msgr2=0x7f712c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f712c0b65f0 0x7f712c0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f713002aa00 tx=0x7f7130005a20 comp rx=0 tx=0).stop 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71441049d0 msgr2=0x7f714419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 --2- 172.21.15.114:0/4278890572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71441049d0 0x7f714419a4d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f713800e9c0 tx=0x7f713800ee80 comp rx=0 tx=0).stop 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714a600640 1 -- 172.21.15.114:0/4278890572 reap_dead start 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 shutdown_connections 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 >> 172.21.15.114:0/4278890572 conn(0x7f7144079740 msgr2=0x7f71440ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 shutdown_connections 2024-09-21T06:52:20.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.293+0000 7f714c600640 1 -- 172.21.15.114:0/4278890572 wait complete. 2024-09-21T06:52:20.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:20 smithi183 ceph-mon[34219]: pgmap v340: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:52:20.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:20 smithi193 ceph-mon[34601]: pgmap v340: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:52:20.929 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:20 smithi114 ceph-mon[30654]: pgmap v340: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 -- 172.21.15.114:0/3972621872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32201050a0 msgr2=0x7f322010d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 --2- 172.21.15.114:0/3972621872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32201050a0 0x7f322010d6e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3208009900 tx=0x7f320802a410 comp rx=0 tx=0).stop 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 -- 172.21.15.114:0/3972621872 shutdown_connections 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 --2- 172.21.15.114:0/3972621872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f322010dc20 0x7f3220110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 --2- 172.21.15.114:0/3972621872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f32201050a0 0x7f322010d6e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 --2- 172.21.15.114:0/3972621872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220104670 0x7f3220104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 -- 172.21.15.114:0/3972621872 >> 172.21.15.114:0/3972621872 conn(0x7f32200fe2c0 msgr2=0x7f3220100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.968+0000 7f321f200640 1 -- 172.21.15.114:0/3972621872 shutdown_connections 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.969+0000 7f321f200640 1 -- 172.21.15.114:0/3972621872 wait complete. 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.969+0000 7f321f200640 1 Processor -- start 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 -- start start 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220104670 0x7f322019a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32201050a0 0x7f322019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f322010dc20 0x7f322019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3220113880 con 0x7f322010dc20 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3220113700 con 0x7f3220104670 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3220113a00 con 0x7f32201050a0 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f322010dc20 0x7f322019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32201050a0 0x7f322019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f322010dc20 0x7f322019faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58518/0 (socket says 172.21.15.114:58518) 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321e800640 1 -- 172.21.15.114:0/3283877795 learned_addr learned my addr 172.21.15.114:0/3283877795 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.970+0000 7f321de00640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220104670 0x7f322019a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321d400640 1 -- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220104670 msgr2=0x7f322019a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321d400640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3220104670 0x7f322019a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321d400640 1 -- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f322010dc20 msgr2=0x7f322019faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321d400640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f322010dc20 0x7f322019faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321d400640 1 -- 172.21.15.114:0/3283877795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32201a0220 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321e800640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f322010dc20 0x7f322019faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f321d400640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32201050a0 0x7f322019a850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f320802a930 tx=0x7f3208031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.971+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320803f070 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.972+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32201a04b0 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.972+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3208004490 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.972+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208035070 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.972+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32201a6c70 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.973+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3220108200 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.976+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f320803b460 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.977+0000 7f3216a00640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f32040b65f0 0x7f32040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.977+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f3208108460 con 0x7f32201050a0 2024-09-21T06:52:21.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.977+0000 7f321de00640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f32040b65f0 0x7f32040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.977+0000 7f3216a00640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] conn(0x7f32040c1900 0x7f32040c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.977+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 --> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3208032970 con 0x7f32040c1900 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.977+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f3208108810 con 0x7f32201050a0 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.978+0000 7f321e800640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] conn(0x7f32040c1900 0x7f32040c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.978+0000 7f321de00640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f32040b65f0 0x7f32040b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3210009bd0 tx=0x7f3210005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.978+0000 7f321e800640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] conn(0x7f32040c1900 0x7f32040c3df0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.979+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== osd.7 v2:172.21.15.183:6816/3203682724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3208032970 con 0x7f32040c1900 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.994+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 --> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f322006b4e0 con 0x7f32040c1900 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.994+0000 7f3216a00640 1 -- 172.21.15.114:0/3283877795 <== osd.7 v2:172.21.15.183:6816/3203682724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f322006b4e0 con 0x7f32040c1900 2024-09-21T06:52:21.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] conn(0x7f32040c1900 msgr2=0x7f32040c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] conn(0x7f32040c1900 0x7f32040c3df0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f32040b65f0 msgr2=0x7f32040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f32040b65f0 0x7f32040b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3210009bd0 tx=0x7f3210005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32201050a0 msgr2=0x7f322019a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 --2- 172.21.15.114:0/3283877795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32201050a0 0x7f322019a850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f320802a930 tx=0x7f3208031a20 comp rx=0 tx=0).stop 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321e800640 1 -- 172.21.15.114:0/3283877795 reap_dead start 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 shutdown_connections 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 >> 172.21.15.114:0/3283877795 conn(0x7f32200fe2c0 msgr2=0x7f32201077a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 shutdown_connections 2024-09-21T06:52:21.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.996+0000 7f321f200640 1 -- 172.21.15.114:0/3283877795 wait complete. 2024-09-21T06:52:21.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 -- 172.21.15.114:0/858593843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 msgr2=0x7f2368110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 --2- 172.21.15.114:0/858593843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 0x7f2368110470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2360009ca0 tx=0x7f236002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 -- 172.21.15.114:0/858593843 shutdown_connections 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 --2- 172.21.15.114:0/858593843 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 0x7f2368110470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 --2- 172.21.15.114:0/858593843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23681092f0 0x7f2368109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 --2- 172.21.15.114:0/858593843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23681039b0 0x7f2368103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 -- 172.21.15.114:0/858593843 >> 172.21.15.114:0/858593843 conn(0x7f23680ff440 msgr2=0x7f2368101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 -- 172.21.15.114:0/858593843 shutdown_connections 2024-09-21T06:52:21.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.578+0000 7f236d400640 1 -- 172.21.15.114:0/858593843 wait complete. 2024-09-21T06:52:21.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 Processor -- start 2024-09-21T06:52:21.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 -- start start 2024-09-21T06:52:21.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23681039b0 0x7f236819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23681092f0 0x7f236819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 0x7f23681a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113900 con 0x7f2368109c40 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113780 con 0x7f23681039b0 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.579+0000 7f236d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2368113a80 con 0x7f23681092f0 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f236ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 0x7f23681a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f236ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 0x7f23681a1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58524/0 (socket says 172.21.15.114:58524) 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f2367e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23681039b0 0x7f236819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f236ca00640 1 -- 172.21.15.114:0/420705017 learned_addr learned my addr 172.21.15.114:0/420705017 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:21.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f235f400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23681092f0 0x7f236819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f2367e00640 1 -- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23681092f0 msgr2=0x7f236819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f2367e00640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23681092f0 0x7f236819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f2367e00640 1 -- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 msgr2=0x7f23681a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f2367e00640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2368109c40 0x7f23681a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f2367e00640 1 -- 172.21.15.114:0/420705017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23681a2400 con 0x7f23681039b0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.580+0000 7f235f400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23681092f0 0x7f236819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.581+0000 7f2367e00640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23681039b0 0x7f236819c490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f235800e9f0 tx=0x7f235800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.581+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235800cd80 con 0x7f23681039b0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.581+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23580035f0 con 0x7f23681039b0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.581+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358010670 con 0x7f23681039b0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.581+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23681a26f0 con 0x7f23681039b0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.581+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681a8e50 con 0x7f23681039b0 2024-09-21T06:52:21.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.582+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f236810fa10 con 0x7f23681039b0 2024-09-21T06:52:21.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.585+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f23580037a0 con 0x7f23681039b0 2024-09-21T06:52:21.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.586+0000 7f2365600640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f234c0b6670 0x7f234c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.587+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f23580e7f50 con 0x7f23681039b0 2024-09-21T06:52:21.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.587+0000 7f235f400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f234c0b6670 0x7f234c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.587+0000 7f2365600640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] conn(0x7f234c0c1980 0x7f234c0c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.587+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 --> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2358020e50 con 0x7f234c0c1980 2024-09-21T06:52:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.587+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f23580e8300 con 0x7f23681039b0 2024-09-21T06:52:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.587+0000 7f236ca00640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] conn(0x7f234c0c1980 0x7f234c0c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.588+0000 7f235f400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f234c0b6670 0x7f234c0b8b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2368105a80 tx=0x7f2350005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.588+0000 7f236ca00640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] conn(0x7f234c0c1980 0x7f234c0c3e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.590+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== osd.2 v2:172.21.15.193:6800/1721723108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2358020e50 con 0x7f234c0c1980 2024-09-21T06:52:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.604+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 --> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2368104c30 con 0x7f234c0c1980 2024-09-21T06:52:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.605+0000 7f2365600640 1 -- 172.21.15.114:0/420705017 <== osd.2 v2:172.21.15.193:6800/1721723108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2368104c30 con 0x7f234c0c1980 2024-09-21T06:52:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.605+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] conn(0x7f234c0c1980 msgr2=0x7f234c0c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.605+0000 7f236d400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] conn(0x7f234c0c1980 0x7f234c0c3e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f234c0b6670 msgr2=0x7f234c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f234c0b6670 0x7f234c0b8b80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2368105a80 tx=0x7f2350005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23681039b0 msgr2=0x7f236819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 --2- 172.21.15.114:0/420705017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23681039b0 0x7f236819c490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f235800e9f0 tx=0x7f235800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236ca00640 1 -- 172.21.15.114:0/420705017 reap_dead start 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 shutdown_connections 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 >> 172.21.15.114:0/420705017 conn(0x7f23680ff440 msgr2=0x7f23681088e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 shutdown_connections 2024-09-21T06:52:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:21.606+0000 7f236d400640 1 -- 172.21.15.114:0/420705017 wait complete. 2024-09-21T06:52:21.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.582+0000 7f5dc7800640 1 -- 172.21.15.114:0/3978832560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 msgr2=0x7f5dc0110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.582+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3978832560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0110630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009be0 tx=0x7f5db802a6c0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 -- 172.21.15.114:0/3978832560 shutdown_connections 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3978832560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0110630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3978832560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5dc01094b0 0x7f5dc01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3978832560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dc0103b70 0x7f5dc0103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 -- 172.21.15.114:0/3978832560 >> 172.21.15.114:0/3978832560 conn(0x7f5dc00ff5e0 msgr2=0x7f5dc0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 -- 172.21.15.114:0/3978832560 shutdown_connections 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.583+0000 7f5dc7800640 1 -- 172.21.15.114:0/3978832560 wait complete. 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.584+0000 7f5dc7800640 1 Processor -- start 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.584+0000 7f5dc7800640 1 -- start start 2024-09-21T06:52:21.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc7800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dc0103b70 0x7f5dc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc7800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5dc01094b0 0x7f5dc0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc7800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc7800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0113ac0 con 0x7f5dc01094b0 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc7800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0113940 con 0x7f5dc0103b70 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc7800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0113c40 con 0x7f5dc0109e00 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc5800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dbfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5dc01094b0 0x7f5dc0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc5800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42322/0 (socket says 172.21.15.114:42322) 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc5800640 1 -- 172.21.15.114:0/3036641305 learned_addr learned my addr 172.21.15.114:0/3036641305 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:21.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc4e00640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dc0103b70 0x7f5dc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.585+0000 7f5dc5800640 1 -- 172.21.15.114:0/3036641305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dc0103b70 msgr2=0x7f5dc0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.586+0000 7f5dc5800640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dc0103b70 0x7f5dc0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.586+0000 7f5dc5800640 1 -- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5dc01094b0 msgr2=0x7f5dc0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dc5800640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5dc01094b0 0x7f5dc0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dc5800640 1 -- 172.21.15.114:0/3036641305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01a8770 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dbfe00640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5dc01094b0 0x7f5dc0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dc5800640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0076190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5db8004af0 tx=0x7f5db8003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8041020 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5db8003e10 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8036040 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.587+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01a8a00 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.588+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a8e90 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.588+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5dc0063b50 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.593+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f5db8044020 con 0x7f5dc0109e00 2024-09-21T06:52:21.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.595+0000 7f5dbd600640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5da80b6670 0x7f5da80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.595+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f5db8108880 con 0x7f5dc0109e00 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.595+0000 7f5dc4e00640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5da80b6670 0x7f5da80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.595+0000 7f5dbd600640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] conn(0x7f5da80c1980 0x7f5da80c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.595+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 --> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5da80c4540 con 0x7f5da80c1980 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.595+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f5db8108c30 con 0x7f5dc0109e00 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.596+0000 7f5dbfe00640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] conn(0x7f5da80c1980 0x7f5da80c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.596+0000 7f5dc4e00640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5da80b6670 0x7f5da80b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5db40096c0 tx=0x7f5db4009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.597+0000 7f5dbfe00640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] conn(0x7f5da80c1980 0x7f5da80c3e70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.598+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== osd.0 v2:172.21.15.114:6802/4089006446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5da80c4540 con 0x7f5da80c1980 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.613+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 --> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5dc0064020 con 0x7f5da80c1980 2024-09-21T06:52:21.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.614+0000 7f5dbd600640 1 -- 172.21.15.114:0/3036641305 <== osd.0 v2:172.21.15.114:6802/4089006446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5dc0064020 con 0x7f5da80c1980 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.614+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] conn(0x7f5da80c1980 msgr2=0x7f5da80c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.614+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] conn(0x7f5da80c1980 0x7f5da80c3e70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5da80b6670 msgr2=0x7f5da80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5da80b6670 0x7f5da80b8b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5db40096c0 tx=0x7f5db4009260 comp rx=0 tx=0).stop 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 msgr2=0x7f5dc0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 --2- 172.21.15.114:0/3036641305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5dc0109e00 0x7f5dc0076190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5db8004af0 tx=0x7f5db8003c20 comp rx=0 tx=0).stop 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc5800640 1 -- 172.21.15.114:0/3036641305 reap_dead start 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 shutdown_connections 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 >> 172.21.15.114:0/3036641305 conn(0x7f5dc00ff5e0 msgr2=0x7f5dc0106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 shutdown_connections 2024-09-21T06:52:21.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:18.615+0000 7f5dc7800640 1 -- 172.21.15.114:0/3036641305 wait complete. 2024-09-21T06:52:22.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 -- 172.21.15.114:0/4064886411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48075dd0 msgr2=0x7f7a480761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 --2- 172.21.15.114:0/4064886411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48075dd0 0x7f7a480761e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a34009900 tx=0x7f7a3402a410 comp rx=0 tx=0).stop 2024-09-21T06:52:22.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 -- 172.21.15.114:0/4064886411 shutdown_connections 2024-09-21T06:52:22.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 --2- 172.21.15.114:0/4064886411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a48114280 0x7f7a48116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 --2- 172.21.15.114:0/4064886411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48075dd0 0x7f7a480761e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 --2- 172.21.15.114:0/4064886411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a48077380 0x7f7a480757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 -- 172.21.15.114:0/4064886411 >> 172.21.15.114:0/4064886411 conn(0x7f7a480ff440 msgr2=0x7f7a48101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 -- 172.21.15.114:0/4064886411 shutdown_connections 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.356+0000 7f7a4e400640 1 -- 172.21.15.114:0/4064886411 wait complete. 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 Processor -- start 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 -- start start 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a48075dd0 0x7f7a481a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a48077380 0x7f7a481a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48114280 0x7f7a481a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48119ec0 con 0x7f7a48114280 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a48119d40 con 0x7f7a48077380 2024-09-21T06:52:22.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.358+0000 7f7a4e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811a040 con 0x7f7a48075dd0 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48114280 0x7f7a481a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48114280 0x7f7a481a6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58560/0 (socket says 172.21.15.114:58560) 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 -- 172.21.15.114:0/386938126 learned_addr learned my addr 172.21.15.114:0/386938126 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a47e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a48077380 0x7f7a481a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4d000640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a48075dd0 0x7f7a481a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 -- 172.21.15.114:0/386938126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a48075dd0 msgr2=0x7f7a481a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a48075dd0 0x7f7a481a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 -- 172.21.15.114:0/386938126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a48077380 msgr2=0x7f7a481a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a48077380 0x7f7a481a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.359+0000 7f7a4da00640 1 -- 172.21.15.114:0/386938126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481a6860 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.360+0000 7f7a4da00640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48114280 0x7f7a481a6130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000eb10 tx=0x7f7a4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.360+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40004b20 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.360+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a481a6b50 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.360+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a40004cc0 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.360+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40010690 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.360+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481ad2b0 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.361+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a48063b50 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.364+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f7a40005000 con 0x7f7a48114280 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.365+0000 7f7a45600640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7a300b65f0 0x7f7a300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.365+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f7a400e8270 con 0x7f7a48114280 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.365+0000 7f7a4d000640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7a300b65f0 0x7f7a300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.365+0000 7f7a45600640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] conn(0x7f7a300c1900 0x7f7a300c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.365+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 --> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a40021e50 con 0x7f7a300c1900 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.365+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f7a400e86c0 con 0x7f7a48114280 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.366+0000 7f7a47e00640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] conn(0x7f7a300c1900 0x7f7a300c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.366+0000 7f7a4d000640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7a300b65f0 0x7f7a300b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009bd0 tx=0x7f7a3c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.366+0000 7f7a47e00640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] conn(0x7f7a300c1900 0x7f7a300c3df0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.368+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== osd.9 v2:172.21.15.114:6826/903490275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a40021e50 con 0x7f7a300c1900 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.382+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 --> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a48063fd0 con 0x7f7a300c1900 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a45600640 1 -- 172.21.15.114:0/386938126 <== osd.9 v2:172.21.15.114:6826/903490275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7a48063fd0 con 0x7f7a300c1900 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] conn(0x7f7a300c1900 msgr2=0x7f7a300c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] conn(0x7f7a300c1900 0x7f7a300c3df0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7a300b65f0 msgr2=0x7f7a300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7a300b65f0 0x7f7a300b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009bd0 tx=0x7f7a3c005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48114280 msgr2=0x7f7a481a6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 --2- 172.21.15.114:0/386938126 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7a48114280 0x7f7a481a6130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000eb10 tx=0x7f7a4000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4da00640 1 -- 172.21.15.114:0/386938126 reap_dead start 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 shutdown_connections 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 >> 172.21.15.114:0/386938126 conn(0x7f7a480ff440 msgr2=0x7f7a48110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 shutdown_connections 2024-09-21T06:52:22.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.383+0000 7f7a4e400640 1 -- 172.21.15.114:0/386938126 wait complete. 2024-09-21T06:52:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 -- 172.21.15.114:0/2993025666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc0109c40 msgr2=0x7f6fc0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2993025666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc0109c40 0x7f6fc0110470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009ca0 tx=0x7f6fb802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 -- 172.21.15.114:0/2993025666 shutdown_connections 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2993025666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc0109c40 0x7f6fc0110470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2993025666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc01092f0 0x7f6fc0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2993025666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc01039b0 0x7f6fc0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.148+0000 7f6fc6800640 1 -- 172.21.15.114:0/2993025666 >> 172.21.15.114:0/2993025666 conn(0x7f6fc00ff440 msgr2=0x7f6fc0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.149+0000 7f6fc6800640 1 -- 172.21.15.114:0/2993025666 shutdown_connections 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.149+0000 7f6fc6800640 1 -- 172.21.15.114:0/2993025666 wait complete. 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 Processor -- start 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 -- start start 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc01039b0 0x7f6fc019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc01092f0 0x7f6fc019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc0109c40 0x7f6fc01a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113950 con 0x7f6fc01092f0 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc01137d0 con 0x7f6fc0109c40 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc6800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113ad0 con 0x7f6fc01039b0 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.150+0000 7f6fc4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc01092f0 0x7f6fc019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc01092f0 0x7f6fc019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58538/0 (socket says 172.21.15.114:58538) 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 -- 172.21.15.114:0/2924559360 learned_addr learned my addr 172.21.15.114:0/2924559360 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc5400640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc01039b0 0x7f6fc019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc5e00640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc0109c40 0x7f6fc01a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 -- 172.21.15.114:0/2924559360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc01039b0 msgr2=0x7f6fc019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc01039b0 0x7f6fc019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 -- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc0109c40 msgr2=0x7f6fc01a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc0109c40 0x7f6fc01a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 -- 172.21.15.114:0/2924559360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc01a2430 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.151+0000 7f6fc4a00640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc01092f0 0x7f6fc019ca60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00d950 tx=0x7f6fac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.152+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac017070 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.152+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fac009c30 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.152+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc01a2720 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.152+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0047a0 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.152+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc01a8e80 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.153+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fc010fa90 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.155+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f6fac004020 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.157+0000 7f6fbe000640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6fa80b65f0 0x7f6fa80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.157+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f6fac0e7a40 con 0x7f6fc01092f0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.157+0000 7f6fc5400640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6fa80b65f0 0x7f6fa80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.157+0000 7f6fbe000640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] conn(0x7f6fa80c1900 0x7f6fa80c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.157+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 --> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fa80c44c0 con 0x7f6fa80c1900 2024-09-21T06:52:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.157+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6fac0e7df0 con 0x7f6fc01092f0 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.158+0000 7f6fc5e00640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] conn(0x7f6fa80c1900 0x7f6fa80c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.158+0000 7f6fc5400640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6fa80b65f0 0x7f6fa80b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005d00 tx=0x7f6fb400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.158+0000 7f6fc5e00640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] conn(0x7f6fa80c1900 0x7f6fa80c3df0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.159+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== osd.4 v2:172.21.15.183:6808/204187293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6fa80c44c0 con 0x7f6fa80c1900 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.173+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 --> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6fc0104c30 con 0x7f6fa80c1900 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.174+0000 7f6fbe000640 1 -- 172.21.15.114:0/2924559360 <== osd.4 v2:172.21.15.183:6808/204187293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6fc0104c30 con 0x7f6fa80c1900 2024-09-21T06:52:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.174+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] conn(0x7f6fa80c1900 msgr2=0x7f6fa80c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.174+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] conn(0x7f6fa80c1900 0x7f6fa80c3df0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6fa80b65f0 msgr2=0x7f6fa80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6fa80b65f0 0x7f6fa80b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005d00 tx=0x7f6fb400b410 comp rx=0 tx=0).stop 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc01092f0 msgr2=0x7f6fc019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 --2- 172.21.15.114:0/2924559360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6fc01092f0 0x7f6fc019ca60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00d950 tx=0x7f6fac00de10 comp rx=0 tx=0).stop 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc5e00640 1 -- 172.21.15.114:0/2924559360 reap_dead start 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 shutdown_connections 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 >> 172.21.15.114:0/2924559360 conn(0x7f6fc00ff440 msgr2=0x7f6fc0106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 shutdown_connections 2024-09-21T06:52:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.175+0000 7f6fc6800640 1 -- 172.21.15.114:0/2924559360 wait complete. 2024-09-21T06:52:22.897 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:22 smithi114 ceph-mon[30654]: pgmap v341: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:52:22.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:22 smithi193 ceph-mon[34601]: pgmap v341: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:52:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:22 smithi183 ceph-mon[34219]: pgmap v341: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:52:23.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 -- 172.21.15.114:0/3715409173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9620076100 msgr2=0x7f962010f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 --2- 172.21.15.114:0/3715409173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9620076100 0x7f962010f9d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f960c009900 tx=0x7f960c02a410 comp rx=0 tx=0).stop 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 -- 172.21.15.114:0/3715409173 shutdown_connections 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 --2- 172.21.15.114:0/3715409173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f962010ff10 0x7f9620112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 --2- 172.21.15.114:0/3715409173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9620076100 0x7f962010f9d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 --2- 172.21.15.114:0/3715409173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96200757b0 0x7f9620075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 -- 172.21.15.114:0/3715409173 >> 172.21.15.114:0/3715409173 conn(0x7f96200ff440 msgr2=0x7f9620101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.599+0000 7f9626c00640 1 -- 172.21.15.114:0/3715409173 shutdown_connections 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.600+0000 7f9626c00640 1 -- 172.21.15.114:0/3715409173 wait complete. 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.600+0000 7f9626c00640 1 Processor -- start 2024-09-21T06:52:23.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 -- start start 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96200757b0 0x7f962019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9620076100 0x7f962019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f962010ff10 0x7f96201a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9620115ad0 con 0x7f96200757b0 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9620115950 con 0x7f962010ff10 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9626c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9620115c50 con 0x7f9620076100 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9625800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96200757b0 0x7f962019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9625800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96200757b0 0x7f962019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58590/0 (socket says 172.21.15.114:58590) 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9625800640 1 -- 172.21.15.114:0/4261103594 learned_addr learned my addr 172.21.15.114:0/4261103594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.601+0000 7f9624e00640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9620076100 0x7f962019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9626200640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f962010ff10 0x7f96201a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9625800640 1 -- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9620076100 msgr2=0x7f962019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:23.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9625800640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9620076100 0x7f962019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9625800640 1 -- 172.21.15.114:0/4261103594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f962010ff10 msgr2=0x7f96201a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9625800640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f962010ff10 0x7f96201a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9625800640 1 -- 172.21.15.114:0/4261103594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96201a2470 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9625800640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96200757b0 0x7f962019c500 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400cd80 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96140108e0 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614018980 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.602+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96201a2760 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.603+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96201a8ec0 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.603+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f962010a4f0 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.606+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f9614010430 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.607+0000 7f961e000640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96080b6670 0x7f96080b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.607+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f96140e8350 con 0x7f96200757b0 2024-09-21T06:52:23.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.607+0000 7f9624e00640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96080b6670 0x7f96080b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.608+0000 7f961e000640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] conn(0x7f96080c1980 0x7f96080c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.608+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 --> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9614027ea0 con 0x7f96080c1980 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.608+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f96140e8700 con 0x7f96200757b0 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.608+0000 7f9626200640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] conn(0x7f96080c1980 0x7f96080c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.608+0000 7f9624e00640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96080b6670 0x7f96080b8b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f960c031440 tx=0x7f960c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.609+0000 7f9626200640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] conn(0x7f96080c1980 0x7f96080c3e70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.609+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== osd.5 v2:172.21.15.193:6808/3650564948 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9614027ea0 con 0x7f96080c1980 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.626+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 --> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9620076cc0 con 0x7f96080c1980 2024-09-21T06:52:23.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.627+0000 7f961e000640 1 -- 172.21.15.114:0/4261103594 <== osd.5 v2:172.21.15.193:6808/3650564948 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9620076cc0 con 0x7f96080c1980 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.627+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] conn(0x7f96080c1980 msgr2=0x7f96080c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.627+0000 7f9626c00640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] conn(0x7f96080c1980 0x7f96080c3e70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96080b6670 msgr2=0x7f96080b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96080b6670 0x7f96080b8b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f960c031440 tx=0x7f960c005a20 comp rx=0 tx=0).stop 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96200757b0 msgr2=0x7f962019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 --2- 172.21.15.114:0/4261103594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96200757b0 0x7f962019c500 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9f0 tx=0x7f961400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626200640 1 -- 172.21.15.114:0/4261103594 reap_dead start 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 shutdown_connections 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 >> 172.21.15.114:0/4261103594 conn(0x7f96200ff440 msgr2=0x7f9620109a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 shutdown_connections 2024-09-21T06:52:23.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.628+0000 7f9626c00640 1 -- 172.21.15.114:0/4261103594 wait complete. 2024-09-21T06:52:24.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.847+0000 7ffa25400640 1 -- 172.21.15.114:0/3336158619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20105bc0 msgr2=0x7ffa20105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.847+0000 7ffa25400640 1 --2- 172.21.15.114:0/3336158619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20105bc0 0x7ffa20105fd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009900 tx=0x7ffa1002a410 comp rx=0 tx=0).stop 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 -- 172.21.15.114:0/3336158619 shutdown_connections 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 --2- 172.21.15.114:0/3336158619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa2010be50 0x7ffa20112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 --2- 172.21.15.114:0/3336158619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa2010b500 0x7ffa2010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 --2- 172.21.15.114:0/3336158619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa20105bc0 0x7ffa20105fd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 -- 172.21.15.114:0/3336158619 >> 172.21.15.114:0/3336158619 conn(0x7ffa20101670 msgr2=0x7ffa20103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 -- 172.21.15.114:0/3336158619 shutdown_connections 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.848+0000 7ffa25400640 1 -- 172.21.15.114:0/3336158619 wait complete. 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.849+0000 7ffa25400640 1 Processor -- start 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.849+0000 7ffa25400640 1 -- start start 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa25400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa20105bc0 0x7ffa2019c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:24.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa25400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa2010b500 0x7ffa2019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa25400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa2010be50 0x7ffa201a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa25400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20115b60 con 0x7ffa2010be50 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa25400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201159e0 con 0x7ffa20105bc0 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa25400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20115ce0 con 0x7ffa2010b500 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa1fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa20105bc0 0x7ffa2019c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa1fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa20105bc0 0x7ffa2019c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:44822/0 (socket says 172.21.15.114:44822) 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa1fe00640 1 -- 172.21.15.114:0/1962788996 learned_addr learned my addr 172.21.15.114:0/1962788996 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa24a00640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa2010be50 0x7ffa201a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.850+0000 7ffa17400640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa2010b500 0x7ffa2019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1fe00640 1 -- 172.21.15.114:0/1962788996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa2010b500 msgr2=0x7ffa2019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1fe00640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa2010b500 0x7ffa2019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1fe00640 1 -- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa2010be50 msgr2=0x7ffa201a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1fe00640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffa2010be50 0x7ffa201a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:24.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1fe00640 1 -- 172.21.15.114:0/1962788996 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201a24a0 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1fe00640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa20105bc0 0x7ffa2019c530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005ab0 tx=0x7ffa10003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1003f070 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201a2730 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10031d60 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10035030 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.851+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201a8ef0 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.852+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa20063b50 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.855+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7ffa10042070 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.856+0000 7ffa1d600640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffa040b65f0 0x7ffa040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7ffa101087a0 con 0x7ffa20105bc0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa17400640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffa040b65f0 0x7ffa040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa1d600640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] conn(0x7ffa040c1900 0x7ffa040c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:24.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 --> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa040c44c0 con 0x7ffa040c1900 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7ffa10108b50 con 0x7ffa20105bc0 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa24a00640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] conn(0x7ffa040c1900 0x7ffa040c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.857+0000 7ffa17400640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffa040b65f0 0x7ffa040b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa20104ec0 tx=0x7ffa0800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.858+0000 7ffa24a00640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] conn(0x7ffa040c1900 0x7ffa040c3df0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.859+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== osd.6 v2:172.21.15.114:6818/1058136039 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa040c44c0 con 0x7ffa040c1900 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.873+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 --> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa20063f50 con 0x7ffa040c1900 2024-09-21T06:52:24.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.873+0000 7ffa1d600640 1 -- 172.21.15.114:0/1962788996 <== osd.6 v2:172.21.15.114:6818/1058136039 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa20063f50 con 0x7ffa040c1900 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] conn(0x7ffa040c1900 msgr2=0x7ffa040c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] conn(0x7ffa040c1900 0x7ffa040c3df0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffa040b65f0 msgr2=0x7ffa040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffa040b65f0 0x7ffa040b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa20104ec0 tx=0x7ffa0800b410 comp rx=0 tx=0).stop 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa20105bc0 msgr2=0x7ffa2019c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 --2- 172.21.15.114:0/1962788996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa20105bc0 0x7ffa2019c530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005ab0 tx=0x7ffa10003c80 comp rx=0 tx=0).stop 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa24a00640 1 -- 172.21.15.114:0/1962788996 reap_dead start 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 shutdown_connections 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 >> 172.21.15.114:0/1962788996 conn(0x7ffa20101670 msgr2=0x7ffa2010ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 shutdown_connections 2024-09-21T06:52:24.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:23.874+0000 7ffa25400640 1 -- 172.21.15.114:0/1962788996 wait complete. 2024-09-21T06:52:24.775 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:24 smithi114 ceph-mon[30654]: pgmap v342: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-21T06:52:24.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:24 smithi193 ceph-mon[34601]: pgmap v342: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-21T06:52:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:24 smithi183 ceph-mon[34219]: pgmap v342: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-21T06:52:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 -- 172.21.15.114:0/4124137040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01039b0 msgr2=0x7f1ad0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:25.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 --2- 172.21.15.114:0/4124137040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01039b0 0x7f1ad0103dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009900 tx=0x7f1ac002a410 comp rx=0 tx=0).stop 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 -- 172.21.15.114:0/4124137040 shutdown_connections 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 --2- 172.21.15.114:0/4124137040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ad0109c40 0x7f1ad0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 --2- 172.21.15.114:0/4124137040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ad01092f0 0x7f1ad0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 --2- 172.21.15.114:0/4124137040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01039b0 0x7f1ad0103dc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.479+0000 7f1acf200640 1 -- 172.21.15.114:0/4124137040 >> 172.21.15.114:0/4124137040 conn(0x7f1ad00ff440 msgr2=0x7f1ad0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.480+0000 7f1acf200640 1 -- 172.21.15.114:0/4124137040 shutdown_connections 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.480+0000 7f1acf200640 1 -- 172.21.15.114:0/4124137040 wait complete. 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.481+0000 7f1acf200640 1 Processor -- start 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.481+0000 7f1acf200640 1 -- start start 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acf200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ad01039b0 0x7f1ad019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acf200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01092f0 0x7f1ad019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:25.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acf200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ad0109c40 0x7f1ad01a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acf200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0076390 con 0x7f1ad01039b0 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acf200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0076210 con 0x7f1ad0109c40 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acf200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad0076510 con 0x7f1ad01092f0 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ad01039b0 0x7f1ad019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ad01039b0 0x7f1ad019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58570/0 (socket says 172.21.15.114:58570) 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acde00640 1 -- 172.21.15.114:0/1184532240 learned_addr learned my addr 172.21.15.114:0/1184532240 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acd400640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01092f0 0x7f1ad019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1ace800640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ad0109c40 0x7f1ad01a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acd400640 1 -- 172.21.15.114:0/1184532240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ad0109c40 msgr2=0x7f1ad01a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acd400640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ad0109c40 0x7f1ad01a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acd400640 1 -- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ad01039b0 msgr2=0x7f1ad019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acd400640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1ad01039b0 0x7f1ad019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.482+0000 7f1acd400640 1 -- 172.21.15.114:0/1184532240 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad01a2500 con 0x7f1ad01092f0 2024-09-21T06:52:25.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.483+0000 7f1acd400640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01092f0 0x7f1ad019cb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ad006b990 tx=0x7f1abc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.483+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00cc40 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.483+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc00cde0 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.483+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01a27f0 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.483+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc018780 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.483+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad01a8f50 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.484+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ad0102b40 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.487+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f1abc010a90 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.489+0000 7f1ac6a00640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1ab40b6670 0x7f1ab40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.489+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f1abc0e83a0 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.489+0000 7f1acde00640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1ab40b6670 0x7f1ab40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.490+0000 7f1ac6a00640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] conn(0x7f1ab40c1980 0x7f1ab40c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.490+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 --> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1abc01bec0 con 0x7f1ab40c1980 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.490+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f1abc0e8750 con 0x7f1ad01092f0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.490+0000 7f1ace800640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] conn(0x7f1ab40c1980 0x7f1ab40c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:25.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.490+0000 7f1acde00640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1ab40b6670 0x7f1ab40b8b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0031440 tx=0x7f1ac00058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.491+0000 7f1ace800640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] conn(0x7f1ab40c1980 0x7f1ab40c3e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.492+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== osd.8 v2:172.21.15.193:6816/2425758688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1abc01bec0 con 0x7f1ab40c1980 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.507+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 --> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1ad0104c30 con 0x7f1ab40c1980 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.507+0000 7f1ac6a00640 1 -- 172.21.15.114:0/1184532240 <== osd.8 v2:172.21.15.193:6816/2425758688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1ad0104c30 con 0x7f1ab40c1980 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] conn(0x7f1ab40c1980 msgr2=0x7f1ab40c3e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] conn(0x7f1ab40c1980 0x7f1ab40c3e70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1ab40b6670 msgr2=0x7f1ab40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1ab40b6670 0x7f1ab40b8b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0031440 tx=0x7f1ac00058d0 comp rx=0 tx=0).stop 2024-09-21T06:52:25.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01092f0 msgr2=0x7f1ad019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 --2- 172.21.15.114:0/1184532240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad01092f0 0x7f1ad019cb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1ad006b990 tx=0x7f1abc00ee50 comp rx=0 tx=0).stop 2024-09-21T06:52:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1ace800640 1 -- 172.21.15.114:0/1184532240 reap_dead start 2024-09-21T06:52:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 shutdown_connections 2024-09-21T06:52:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 >> 172.21.15.114:0/1184532240 conn(0x7f1ad00ff440 msgr2=0x7f1ad0107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 shutdown_connections 2024-09-21T06:52:25.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:22.508+0000 7f1acf200640 1 -- 172.21.15.114:0/1184532240 wait complete. 2024-09-21T06:52:25.916 INFO:teuthology.orchestra.run.smithi114.stdout:81604378708 2024-09-21T06:52:25.916 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T06:52:26.681 INFO:teuthology.orchestra.run.smithi114.stdout:111669149773 2024-09-21T06:52:26.681 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T06:52:26.730 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:26 smithi114 ceph-mon[30654]: pgmap v343: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T06:52:26.818 INFO:teuthology.orchestra.run.smithi114.stdout:38654705758 2024-09-21T06:52:26.902 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T06:52:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:26 smithi193 ceph-mon[34601]: pgmap v343: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T06:52:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:26 smithi183 ceph-mon[34219]: pgmap v343: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T06:52:27.163 INFO:teuthology.orchestra.run.smithi114.stdout:227633266738 2024-09-21T06:52:27.163 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T06:52:27.277 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:27.873 INFO:teuthology.orchestra.run.smithi114.stdout:201863462966 2024-09-21T06:52:27.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 -- 172.21.15.114:0/1486612413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8324103990 msgr2=0x7f8324103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 --2- 172.21.15.114:0/1486612413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8324103990 0x7f8324103da0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8314009900 tx=0x7f831402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 -- 172.21.15.114:0/1486612413 shutdown_connections 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 --2- 172.21.15.114:0/1486612413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8324109c20 0x7f8324110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 --2- 172.21.15.114:0/1486612413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83241092d0 0x7f83241096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 --2- 172.21.15.114:0/1486612413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8324103990 0x7f8324103da0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 -- 172.21.15.114:0/1486612413 >> 172.21.15.114:0/1486612413 conn(0x7f83240ff3e0 msgr2=0x7f8324101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 -- 172.21.15.114:0/1486612413 shutdown_connections 2024-09-21T06:52:28.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.301+0000 7f8323200640 1 -- 172.21.15.114:0/1486612413 wait complete. 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.302+0000 7f8323200640 1 Processor -- start 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.302+0000 7f8323200640 1 -- start start 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8323200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8324103990 0x7f8324075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8323200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83241092d0 0x7f8324075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8323200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8324109c20 0x7f8324076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8323200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f83241138e0 con 0x7f8324109c20 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8323200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8324113760 con 0x7f8324103990 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8323200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8324113a60 con 0x7f83241092d0 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83241092d0 0x7f8324075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83241092d0 0x7f8324075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42342/0 (socket says 172.21.15.114:42342) 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 -- 172.21.15.114:0/3753688853 learned_addr learned my addr 172.21.15.114:0/3753688853 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321e00640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8324103990 0x7f8324075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:28.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8322800640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8324109c20 0x7f8324076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 -- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8324103990 msgr2=0x7f8324075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8324103990 0x7f8324075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 -- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8324109c20 msgr2=0x7f8324076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8324109c20 0x7f8324076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.303+0000 7f8321400640 1 -- 172.21.15.114:0/3753688853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a85b0 con 0x7f83241092d0 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.304+0000 7f8321400640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83241092d0 0x7f8324075c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f831000e9f0 tx=0x7f831000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.304+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000ccc0 con 0x7f83241092d0 2024-09-21T06:52:28.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.304+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83100108e0 con 0x7f83241092d0 2024-09-21T06:52:28.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.304+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a88a0 con 0x7f83241092d0 2024-09-21T06:52:28.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.304+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310018880 con 0x7f83241092d0 2024-09-21T06:52:28.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.304+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a8de0 con 0x7f83241092d0 2024-09-21T06:52:28.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.305+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8324063b50 con 0x7f83241092d0 2024-09-21T06:52:28.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.308+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f8310010430 con 0x7f83241092d0 2024-09-21T06:52:28.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.309+0000 7f831aa00640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f83080b65f0 0x7f83080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.309+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7f83100e84c0 con 0x7f83241092d0 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.309+0000 7f8321e00640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f83080b65f0 0x7f83080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.310+0000 7f831aa00640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] conn(0x7f83080c1900 0x7f83080c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.310+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 --> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8310015e80 con 0x7f83080c1900 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.310+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f83100e8910 con 0x7f83241092d0 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.310+0000 7f8321e00640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f83080b65f0 0x7f83080b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f8314031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.310+0000 7f8322800640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] conn(0x7f83080c1900 0x7f83080c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.311+0000 7f8322800640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] conn(0x7f83080c1900 0x7f83080c3df0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.311+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== osd.1 v2:172.21.15.183:6800/427115039 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8310015e80 con 0x7f83080c1900 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.325+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 --> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8324063fd0 con 0x7f83080c1900 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.326+0000 7f831aa00640 1 -- 172.21.15.114:0/3753688853 <== osd.1 v2:172.21.15.183:6800/427115039 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8324063fd0 con 0x7f83080c1900 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] conn(0x7f83080c1900 msgr2=0x7f83080c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:28.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] conn(0x7f83080c1900 0x7f83080c3df0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f83080b65f0 msgr2=0x7f83080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f83080b65f0 0x7f83080b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8314000c00 tx=0x7f8314031400 comp rx=0 tx=0).stop 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83241092d0 msgr2=0x7f8324075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 --2- 172.21.15.114:0/3753688853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83241092d0 0x7f8324075c50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f831000e9f0 tx=0x7f831000edf0 comp rx=0 tx=0).stop 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8322800640 1 -- 172.21.15.114:0/3753688853 reap_dead start 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 shutdown_connections 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 >> 172.21.15.114:0/3753688853 conn(0x7f83240ff3e0 msgr2=0x7f8324101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 shutdown_connections 2024-09-21T06:52:28.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:20.327+0000 7f8323200640 1 -- 172.21.15.114:0/3753688853 wait complete. 2024-09-21T06:52:28.903 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:28 smithi114 ceph-mon[30654]: pgmap v344: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T06:52:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:28 smithi193 ceph-mon[34601]: pgmap v344: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T06:52:28.941 INFO:teuthology.orchestra.run.smithi114.stdout:137438953545 2024-09-21T06:52:28.941 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T06:52:29.030 INFO:teuthology.orchestra.run.smithi114.stdout:180388626493 2024-09-21T06:52:29.031 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T06:52:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:28 smithi183 ceph-mon[34219]: pgmap v344: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-21T06:52:29.771 INFO:teuthology.orchestra.run.smithi114.stdout:60129542232 2024-09-21T06:52:29.771 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T06:52:30.427 INFO:teuthology.orchestra.run.smithi114.stdout:154618822723 2024-09-21T06:52:30.427 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T06:52:30.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 -- 172.21.15.114:0/599847051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6001101f0 msgr2=0x7fb600112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 --2- 172.21.15.114:0/599847051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6001101f0 0x7fb600112610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009ca0 tx=0x7fb5f802a950 comp rx=0 tx=0).stop 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 -- 172.21.15.114:0/599847051 shutdown_connections 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 --2- 172.21.15.114:0/599847051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb6001101f0 0x7fb600112610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 --2- 172.21.15.114:0/599847051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb600102f90 0x7fb60010fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 --2- 172.21.15.114:0/599847051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb600102640 0x7fb600102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 -- 172.21.15.114:0/599847051 >> 172.21.15.114:0/599847051 conn(0x7fb6000796e0 msgr2=0x7fb600079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 -- 172.21.15.114:0/599847051 shutdown_connections 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.443+0000 7fb605000640 1 -- 172.21.15.114:0/599847051 wait complete. 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 Processor -- start 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 -- start start 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb600102640 0x7fb60019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:30.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb600102f90 0x7fb60019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6001101f0 0x7fb6001a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb600115c90 con 0x7fb600102f90 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb600115b10 con 0x7fb6001101f0 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb605000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb600115e10 con 0x7fb600102640 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.445+0000 7fb5f5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb600102f90 0x7fb60019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb600102640 0x7fb60019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5f5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb600102f90 0x7fb60019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45470/0 (socket says 172.21.15.114:45470) 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5f5400640 1 -- 172.21.15.114:0/4089916315 learned_addr learned my addr 172.21.15.114:0/4089916315 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fe800640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6001101f0 0x7fb6001a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 -- 172.21.15.114:0/4089916315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6001101f0 msgr2=0x7fb6001a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6001101f0 0x7fb6001a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 -- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb600102f90 msgr2=0x7fb60019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb600102f90 0x7fb60019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 -- 172.21.15.114:0/4089916315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6001a2530 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5f5400640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb600102f90 0x7fb60019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.446+0000 7fb5fde00640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb600102640 0x7fb60019c5c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9f0 tx=0x7fb5f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.447+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000ccc0 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.447+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6001a2820 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.447+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f00044d0 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.447+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f001c070 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.447+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001a8f80 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.448+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6001096c0 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.451+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fb5f0005020 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.452+0000 7fb5f7400640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5e40b65f0 0x7fb5e40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.452+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9286+0+0 (secure 0 0 0) 0x7fb5f00e7c10 con 0x7fb600102640 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.452+0000 7fb5f5400640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5e40b65f0 0x7fb5e40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.452+0000 7fb5f7400640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] conn(0x7fb5e40c1900 0x7fb5e40c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:30.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.453+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 --> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5f0012ec0 con 0x7fb5e40c1900 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.453+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb5f00e7fc0 con 0x7fb600102640 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.453+0000 7fb5fe800640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] conn(0x7fb5e40c1900 0x7fb5e40c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.453+0000 7fb5f5400640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5e40b65f0 0x7fb5e40b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb600110c70 tx=0x7fb5e800b4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.453+0000 7fb5fe800640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] conn(0x7fb5e40c1900 0x7fb5e40c3df0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.454+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== osd.11 v2:172.21.15.193:6824/3118065190 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5f0012ec0 con 0x7fb5e40c1900 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.468+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 --> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb60006b3b0 con 0x7fb5e40c1900 2024-09-21T06:52:30.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.469+0000 7fb5f7400640 1 -- 172.21.15.114:0/4089916315 <== osd.11 v2:172.21.15.193:6824/3118065190 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb60006b3b0 con 0x7fb5e40c1900 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] conn(0x7fb5e40c1900 msgr2=0x7fb5e40c3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] conn(0x7fb5e40c1900 0x7fb5e40c3df0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5e40b65f0 msgr2=0x7fb5e40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5e40b65f0 0x7fb5e40b8b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb600110c70 tx=0x7fb5e800b4a0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb600102640 msgr2=0x7fb60019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 --2- 172.21.15.114:0/4089916315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb600102640 0x7fb60019c5c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9f0 tx=0x7fb5f000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb5fe800640 1 -- 172.21.15.114:0/4089916315 reap_dead start 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 shutdown_connections 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 >> 172.21.15.114:0/4089916315 conn(0x7fb6000796e0 msgr2=0x7fb600108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 shutdown_connections 2024-09-21T06:52:30.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:29.470+0000 7fb605000640 1 -- 172.21.15.114:0/4089916315 wait complete. 2024-09-21T06:52:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:30 smithi114 ceph-mon[30654]: pgmap v345: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]} : dispatch 2024-09-21T06:52:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T06:52:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]} : dispatch 2024-09-21T06:52:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]} : dispatch 2024-09-21T06:52:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:30 smithi193 ceph-mon[34601]: pgmap v345: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]} : dispatch 2024-09-21T06:52:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T06:52:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]} : dispatch 2024-09-21T06:52:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]} : dispatch 2024-09-21T06:52:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:30 smithi183 ceph-mon[34219]: pgmap v345: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]} : dispatch 2024-09-21T06:52:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]} : dispatch 2024-09-21T06:52:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]} : dispatch 2024-09-21T06:52:31.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]} : dispatch 2024-09-21T06:52:31.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 -- 172.21.15.114:0/1414899850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc1049d0 msgr2=0x7f16cc104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:31.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 --2- 172.21.15.114:0/1414899850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc1049d0 0x7f16cc104de0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009900 tx=0x7f16c002a410 comp rx=0 tx=0).stop 2024-09-21T06:52:31.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 -- 172.21.15.114:0/1414899850 shutdown_connections 2024-09-21T06:52:31.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 --2- 172.21.15.114:0/1414899850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16cc10de30 0x7f16cc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:31.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 --2- 172.21.15.114:0/1414899850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16cc105320 0x7f16cc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:31.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 --2- 172.21.15.114:0/1414899850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc1049d0 0x7f16cc104de0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:31.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 -- 172.21.15.114:0/1414899850 >> 172.21.15.114:0/1414899850 conn(0x7f16cc079740 msgr2=0x7f16cc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:31.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 -- 172.21.15.114:0/1414899850 shutdown_connections 2024-09-21T06:52:31.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.134+0000 7f16d4800640 1 -- 172.21.15.114:0/1414899850 wait complete. 2024-09-21T06:52:31.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.135+0000 7f16d4800640 1 Processor -- start 2024-09-21T06:52:31.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 -- start start 2024-09-21T06:52:31.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16cc1049d0 0x7f16cc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:31.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc105320 0x7f16cc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:31.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16cc10de30 0x7f16cc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113a90 con 0x7f16cc105320 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113910 con 0x7f16cc1049d0 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d4800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc113c10 con 0x7f16cc10de30 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16cc1049d0 0x7f16cc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc105320 0x7f16cc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16cc1049d0 0x7f16cc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37672/0 (socket says 172.21.15.114:37672) 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.136+0000 7f16d1e00640 1 -- 172.21.15.114:0/3744603596 learned_addr learned my addr 172.21.15.114:0/3744603596 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:31.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d2800640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16cc10de30 0x7f16cc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d1400640 1 -- 172.21.15.114:0/3744603596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16cc10de30 msgr2=0x7f16cc19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d1400640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16cc10de30 0x7f16cc19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d1400640 1 -- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16cc1049d0 msgr2=0x7f16cc19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d1400640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16cc1049d0 0x7f16cc19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d1400640 1 -- 172.21.15.114:0/3744603596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc1a0510 con 0x7f16cc105320 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16d1400640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc105320 0x7f16cc19ab40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f16cc06b950 tx=0x7f16bc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:31.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.137+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc00cc40 con 0x7f16cc105320 2024-09-21T06:52:31.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.138+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc1a0800 con 0x7f16cc105320 2024-09-21T06:52:31.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.138+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16bc00cde0 con 0x7f16cc105320 2024-09-21T06:52:31.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.138+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc018760 con 0x7f16cc105320 2024-09-21T06:52:31.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.138+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc1a6f60 con 0x7f16cc105320 2024-09-21T06:52:31.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.139+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16cc063ad0 con 0x7f16cc105320 2024-09-21T06:52:31.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.141+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f16bc010a90 con 0x7f16cc105320 2024-09-21T06:52:31.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.143+0000 7f16c6a00640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16b40b6670 0x7f16b40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.143+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9402+0+0 (secure 0 0 0) 0x7f16bc0e8420 con 0x7f16cc105320 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.143+0000 7f16d1e00640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16b40b6670 0x7f16b40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.143+0000 7f16c6a00640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] conn(0x7f16b40c1b80 0x7f16b40c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.143+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 --> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16b40c4740 con 0x7f16b40c1b80 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.143+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f16bc0e87d0 con 0x7f16cc105320 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.144+0000 7f16d2800640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] conn(0x7f16b40c1b80 0x7f16b40c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:31.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.144+0000 7f16d1e00640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16b40b6670 0x7f16b40b8b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005f70 tx=0x7f16c0009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:31.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.145+0000 7f16d2800640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] conn(0x7f16b40c1b80 0x7f16b40c4070 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:31.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.146+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== osd.10 v2:172.21.15.183:6824/4150013196 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16b40c4740 con 0x7f16b40c1b80 2024-09-21T06:52:31.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.160+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 --> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16cc063ed0 con 0x7f16b40c1b80 2024-09-21T06:52:31.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16c6a00640 1 -- 172.21.15.114:0/3744603596 <== osd.10 v2:172.21.15.183:6824/4150013196 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16cc063ed0 con 0x7f16b40c1b80 2024-09-21T06:52:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] conn(0x7f16b40c1b80 msgr2=0x7f16b40c4070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d4800640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] conn(0x7f16b40c1b80 0x7f16b40c4070 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:31.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16b40b6670 msgr2=0x7f16b40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d4800640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16b40b6670 0x7f16b40b8b80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f16c0005f70 tx=0x7f16c0009560 comp rx=0 tx=0).stop 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc105320 msgr2=0x7f16cc19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d4800640 1 --2- 172.21.15.114:0/3744603596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16cc105320 0x7f16cc19ab40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f16cc06b950 tx=0x7f16bc00ee50 comp rx=0 tx=0).stop 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.161+0000 7f16d2800640 1 -- 172.21.15.114:0/3744603596 reap_dead start 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.162+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 shutdown_connections 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.162+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 >> 172.21.15.114:0/3744603596 conn(0x7f16cc079740 msgr2=0x7f16cc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.162+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 shutdown_connections 2024-09-21T06:52:31.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:31.162+0000 7f16d4800640 1 -- 172.21.15.114:0/3744603596 wait complete. 2024-09-21T06:52:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-21T06:52:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T06:52:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-21T06:52:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:31 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-21T06:52:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:31 smithi193 ceph-mon[34601]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:52:32.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-21T06:52:32.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T06:52:32.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-21T06:52:32.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:31 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-21T06:52:32.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:31 smithi114 ceph-mon[30654]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:52:32.111 INFO:teuthology.orchestra.run.smithi114.stdout:274877906983 2024-09-21T06:52:32.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T06:52:32.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-09-21T06:52:32.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-21T06:52:32.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-09-21T06:52:32.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:31 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-09-21T06:52:32.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:31 smithi183 ceph-mon[34219]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:52:32.349 INFO:teuthology.orchestra.run.smithi114.stdout:300647710755 2024-09-21T06:52:32.349 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T06:52:32.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:32 smithi193 ceph-mon[34601]: pgmap v347: 129 pgs: 2 peering, 127 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T06:52:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:32 smithi193 ceph-mon[34601]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-21T06:52:32.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:32 smithi193 ceph-mon[34601]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:52:32.996 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:32 smithi114 ceph-mon[30654]: pgmap v347: 129 pgs: 2 peering, 127 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T06:52:32.996 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:32 smithi114 ceph-mon[30654]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-21T06:52:32.996 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:32 smithi114 ceph-mon[30654]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:52:33.035 INFO:teuthology.orchestra.run.smithi114.stdout:253403070506 2024-09-21T06:52:33.035 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T06:52:33.114 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:32 smithi183 ceph-mon[34219]: pgmap v347: 129 pgs: 2 peering, 127 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-21T06:52:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:32 smithi183 ceph-mon[34219]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-21T06:52:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:32 smithi183 ceph-mon[34219]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:52:33.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 -- 172.21.15.114:0/1663286175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e4102f40 msgr2=0x7fd6e410fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:33.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1663286175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e4102f40 0x7fd6e410fc60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009900 tx=0x7fd6cc02a480 comp rx=0 tx=0).stop 2024-09-21T06:52:33.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 -- 172.21.15.114:0/1663286175 shutdown_connections 2024-09-21T06:52:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1663286175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e41101a0 0x7fd6e41125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1663286175 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e4102f40 0x7fd6e410fc60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1663286175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e41025f0 0x7fd6e4102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.886+0000 7fd6e9200640 1 -- 172.21.15.114:0/1663286175 >> 172.21.15.114:0/1663286175 conn(0x7fd6e4079740 msgr2=0x7fd6e4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.887+0000 7fd6e9200640 1 -- 172.21.15.114:0/1663286175 shutdown_connections 2024-09-21T06:52:33.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.887+0000 7fd6e9200640 1 -- 172.21.15.114:0/1663286175 wait complete. 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 Processor -- start 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 -- start start 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e41025f0 0x7fd6e419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e4102f40 0x7fd6e419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 0x7fd6e41a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4115c60 con 0x7fd6e41101a0 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4115ae0 con 0x7fd6e4102f40 2024-09-21T06:52:33.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.888+0000 7fd6e9200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4115de0 con 0x7fd6e41025f0 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 0x7fd6e41a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 0x7fd6e41a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45494/0 (socket says 172.21.15.114:45494) 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 -- 172.21.15.114:0/1304308981 learned_addr learned my addr 172.21.15.114:0/1304308981 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e1e00640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e41025f0 0x7fd6e419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6dbe00640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e4102f40 0x7fd6e419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 -- 172.21.15.114:0/1304308981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e41025f0 msgr2=0x7fd6e419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e41025f0 0x7fd6e419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 -- 172.21.15.114:0/1304308981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e4102f40 msgr2=0x7fd6e419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e4102f40 0x7fd6e419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:33.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 -- 172.21.15.114:0/1304308981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e41a25e0 con 0x7fd6e41101a0 2024-09-21T06:52:33.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.889+0000 7fd6e2800640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 0x7fd6e41a1eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00eb10 tx=0x7fd6dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:33.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.890+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc004b20 con 0x7fd6e41101a0 2024-09-21T06:52:33.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.890+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6dc004cc0 con 0x7fd6e41101a0 2024-09-21T06:52:33.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.890+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc010670 con 0x7fd6e41101a0 2024-09-21T06:52:33.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.890+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e41a28d0 con 0x7fd6e41101a0 2024-09-21T06:52:33.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.890+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e41a9030 con 0x7fd6e41101a0 2024-09-21T06:52:33.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.891+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e406c240 con 0x7fd6e41101a0 2024-09-21T06:52:33.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.893+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fd6dc005000 con 0x7fd6e41101a0 2024-09-21T06:52:33.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.895+0000 7fd6daa00640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6c80b6670 0x7fd6c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:33.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.895+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd6dc0e80f0 con 0x7fd6e41101a0 2024-09-21T06:52:33.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.895+0000 7fd6e1e00640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6c80b6670 0x7fd6c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:33.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.896+0000 7fd6e1e00640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6c80b6670 0x7fd6c80b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009bd0 tx=0x7fd6d4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:33.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:33.897+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd6dc056710 con 0x7fd6e41101a0 2024-09-21T06:52:33.941 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.945 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.953 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.956 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.961 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.972 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.979 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:33.992 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:34.002 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:34.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.184+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd6e410ada0 con 0x7fd6e41101a0 2024-09-21T06:52:34.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.185+0000 7fd6daa00640 1 -- 172.21.15.114:0/1304308981 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd6dc0ae330 con 0x7fd6e41101a0 2024-09-21T06:52:34.187 INFO:teuthology.orchestra.run.smithi114.stdout:81604378711 2024-09-21T06:52:34.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6c80b6670 msgr2=0x7fd6c80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:34.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6c80b6670 0x7fd6c80b8b80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009bd0 tx=0x7fd6d4005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:34.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 msgr2=0x7fd6e41a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:34.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 0x7fd6e41a1eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00eb10 tx=0x7fd6dc00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 shutdown_connections 2024-09-21T06:52:34.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6e41101a0 0x7fd6e41a1eb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6c80b6670 0x7fd6c80b8b80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6e4102f40 0x7fd6e419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 --2- 172.21.15.114:0/1304308981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6e41025f0 0x7fd6e419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.187+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 >> 172.21.15.114:0/1304308981 conn(0x7fd6e4079740 msgr2=0x7fd6e4111620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:34.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.188+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 shutdown_connections 2024-09-21T06:52:34.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.188+0000 7fd6e9200640 1 -- 172.21.15.114:0/1304308981 wait complete. 2024-09-21T06:52:34.839 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.896+0000 7ff4e3e00640 1 -- 172.21.15.114:0/396849389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 msgr2=0x7ff4e4103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/396849389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e4103dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009900 tx=0x7ff4d402a410 comp rx=0 tx=0).stop 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 -- 172.21.15.114:0/396849389 shutdown_connections 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/396849389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e4109c40 0x7ff4e4110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/396849389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4e41092f0 0x7ff4e4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/396849389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e4103dc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 -- 172.21.15.114:0/396849389 >> 172.21.15.114:0/396849389 conn(0x7ff4e40ff440 msgr2=0x7ff4e4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 -- 172.21.15.114:0/396849389 shutdown_connections 2024-09-21T06:52:34.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.897+0000 7ff4e3e00640 1 -- 172.21.15.114:0/396849389 wait complete. 2024-09-21T06:52:34.900 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:34 smithi114 ceph-mon[30654]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T06:52:34.900 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:34 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1304308981' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:52:34.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.898+0000 7ff4e3e00640 1 Processor -- start 2024-09-21T06:52:34.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.898+0000 7ff4e3e00640 1 -- start start 2024-09-21T06:52:34.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.898+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 0x7ff4e419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4e4109c40 0x7ff4e41a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113900 con 0x7ff4e41039b0 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113780 con 0x7ff4e4109c40 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4113a80 con 0x7ff4e41092f0 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 0x7ff4e419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 0x7ff4e419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42846/0 (socket says 172.21.15.114:42846) 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e2000640 1 -- 172.21.15.114:0/1942646234 learned_addr learned my addr 172.21.15.114:0/1942646234 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.899+0000 7ff4e3400640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4e4109c40 0x7ff4e41a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2000640 1 -- 172.21.15.114:0/1942646234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4e4109c40 msgr2=0x7ff4e41a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:34.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2000640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4e4109c40 0x7ff4e41a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2000640 1 -- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 msgr2=0x7ff4e419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2000640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2000640 1 -- 172.21.15.114:0/1942646234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e41a2480 con 0x7ff4e41092f0 2024-09-21T06:52:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2a00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e2000640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 0x7ff4e419cab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e9f0 tx=0x7ff4d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:34.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d000ccc0 con 0x7ff4e41092f0 2024-09-21T06:52:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41a2770 con 0x7ff4e41092f0 2024-09-21T06:52:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d00108e0 con 0x7ff4e41092f0 2024-09-21T06:52:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.900+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0018860 con 0x7ff4e41092f0 2024-09-21T06:52:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.901+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41a8ed0 con 0x7ff4e41092f0 2024-09-21T06:52:34.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.901+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4103dc0 con 0x7ff4e41092f0 2024-09-21T06:52:34.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.904+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7ff4d0010430 con 0x7ff4e41092f0 2024-09-21T06:52:34.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.905+0000 7ff4db400640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4c80b6670 0x7ff4c80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:34.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.906+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff4d00e80c0 con 0x7ff4e41092f0 2024-09-21T06:52:34.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.906+0000 7ff4e2a00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4c80b6670 0x7ff4c80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:34.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.906+0000 7ff4e2a00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4c80b6670 0x7ff4c80b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4031440 tx=0x7ff4d4032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:34.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:34.907+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4d00b26e0 con 0x7ff4e41092f0 2024-09-21T06:52:34.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:34 smithi193 ceph-mon[34601]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T06:52:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:34 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1304308981' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:52:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:34 smithi183 ceph-mon[34219]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T06:52:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:34 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1304308981' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T06:52:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.117+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff4e41a9200 con 0x7ff4e41092f0 2024-09-21T06:52:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.117+0000 7ff4db400640 1 -- 172.21.15.114:0/1942646234 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff4d00b7620 con 0x7ff4e41092f0 2024-09-21T06:52:35.119 INFO:teuthology.orchestra.run.smithi114.stdout:201863462968 2024-09-21T06:52:35.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4c80b6670 msgr2=0x7ff4c80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:35.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4c80b6670 0x7ff4c80b8b80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4031440 tx=0x7ff4d4032ed0 comp rx=0 tx=0).stop 2024-09-21T06:52:35.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 msgr2=0x7ff4e419cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:35.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 0x7ff4e419cab0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000e9f0 tx=0x7ff4d000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 shutdown_connections 2024-09-21T06:52:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4c80b6670 0x7ff4c80b8b80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4e4109c40 0x7ff4e41a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4e41092f0 0x7ff4e419cab0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 --2- 172.21.15.114:0/1942646234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4e41039b0 0x7ff4e419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:35.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 >> 172.21.15.114:0/1942646234 conn(0x7ff4e40ff440 msgr2=0x7ff4e410e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:35.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 shutdown_connections 2024-09-21T06:52:35.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:35.120+0000 7ff4e3e00640 1 -- 172.21.15.114:0/1942646234 wait complete. 2024-09-21T06:52:35.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:35 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1942646234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:52:35.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:35 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1942646234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:52:36.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:35 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1942646234' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 -- 172.21.15.114:0/3219963451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad80109c40 msgr2=0x7fad80110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 --2- 172.21.15.114:0/3219963451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad80109c40 0x7fad80110470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fad78009ca0 tx=0x7fad7802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 -- 172.21.15.114:0/3219963451 shutdown_connections 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 --2- 172.21.15.114:0/3219963451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad80109c40 0x7fad80110470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 --2- 172.21.15.114:0/3219963451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 0x7fad80109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 --2- 172.21.15.114:0/3219963451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad801039b0 0x7fad80103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 -- 172.21.15.114:0/3219963451 >> 172.21.15.114:0/3219963451 conn(0x7fad800ff440 msgr2=0x7fad80101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 -- 172.21.15.114:0/3219963451 shutdown_connections 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.117+0000 7fad86600640 1 -- 172.21.15.114:0/3219963451 wait complete. 2024-09-21T06:52:36.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.118+0000 7fad86600640 1 Processor -- start 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.118+0000 7fad86600640 1 -- start start 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad86600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 0x7fad8019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad86600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 0x7fad8019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad86600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad80109c40 0x7fad801a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad86600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fad80113950 con 0x7fad801039b0 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad86600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad801137d0 con 0x7fad801092f0 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad86600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad80113ad0 con 0x7fad80109c40 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad85200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 0x7fad8019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad85200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 0x7fad8019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38860/0 (socket says 172.21.15.114:38860) 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 0x7fad8019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad85200640 1 -- 172.21.15.114:0/3687860397 learned_addr learned my addr 172.21.15.114:0/3687860397 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.119+0000 7fad85c00640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad80109c40 0x7fad801a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad85200640 1 -- 172.21.15.114:0/3687860397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad80109c40 msgr2=0x7fad801a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:36.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad85200640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad80109c40 0x7fad801a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad85200640 1 -- 172.21.15.114:0/3687860397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 msgr2=0x7fad8019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad85200640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 0x7fad8019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad85200640 1 -- 172.21.15.114:0/3687860397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801a2430 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad7fe00640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 0x7fad8019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad85200640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 0x7fad8019c4c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9f0 tx=0x7fad7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7400cd80 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad740108e0 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad801a2720 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74018980 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.120+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad801a8e80 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.122+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80103dc0 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.125+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fad74010430 con 0x7fad801039b0 2024-09-21T06:52:36.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.126+0000 7fad7d600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad680b65f0 0x7fad680b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:36.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.126+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fad740e7d20 con 0x7fad801039b0 2024-09-21T06:52:36.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.127+0000 7fad7fe00640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad680b65f0 0x7fad680b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:36.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.127+0000 7fad7fe00640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad680b65f0 0x7fad680b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad80108a20 tx=0x7fad6c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:36.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.130+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fad740b2340 con 0x7fad801039b0 2024-09-21T06:52:36.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.333+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fad801a9240 con 0x7fad801039b0 2024-09-21T06:52:36.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.333+0000 7fad7d600640 1 -- 172.21.15.114:0/3687860397 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fad740b7280 con 0x7fad801039b0 2024-09-21T06:52:36.335 INFO:teuthology.orchestra.run.smithi114.stdout:274877906984 2024-09-21T06:52:36.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad680b65f0 msgr2=0x7fad680b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:36.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad680b65f0 0x7fad680b8b00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fad80108a20 tx=0x7fad6c005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 msgr2=0x7fad8019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 0x7fad8019c4c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9f0 tx=0x7fad7400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 shutdown_connections 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad80109c40 0x7fad801a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad680b65f0 0x7fad680b8b00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad801092f0 0x7fad8019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 --2- 172.21.15.114:0/3687860397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fad801039b0 0x7fad8019c4c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 >> 172.21.15.114:0/3687860397 conn(0x7fad800ff440 msgr2=0x7fad80106860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 shutdown_connections 2024-09-21T06:52:36.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.336+0000 7fad86600640 1 -- 172.21.15.114:0/3687860397 wait complete. 2024-09-21T06:52:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:36 smithi193 ceph-mon[34601]: pgmap v350: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T06:52:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:36 smithi193 ceph-mon[34601]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-21T06:52:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:36 smithi193 ceph-mon[34601]: Cluster is now healthy 2024-09-21T06:52:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:36 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3687860397' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:52:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:36 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/129203790' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:52:36.965 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:36 smithi114 ceph-mon[30654]: pgmap v350: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T06:52:36.965 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:36 smithi114 ceph-mon[30654]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-21T06:52:36.965 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:36 smithi114 ceph-mon[30654]: Cluster is now healthy 2024-09-21T06:52:36.965 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3687860397' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:52:36.965 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:36 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/129203790' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 -- 172.21.15.114:0/3394376166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73501101b0 msgr2=0x7f7350112610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 --2- 172.21.15.114:0/3394376166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73501101b0 0x7f7350112610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7348009ca0 tx=0x7f734802a950 comp rx=0 tx=0).stop 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 -- 172.21.15.114:0/3394376166 shutdown_connections 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 --2- 172.21.15.114:0/3394376166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f73501101b0 0x7f7350112610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 --2- 172.21.15.114:0/3394376166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7350102f50 0x7f735010fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 --2- 172.21.15.114:0/3394376166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 0x7f7350102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 -- 172.21.15.114:0/3394376166 >> 172.21.15.114:0/3394376166 conn(0x7f7350079650 msgr2=0x7f7350079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 -- 172.21.15.114:0/3394376166 shutdown_connections 2024-09-21T06:52:37.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.029+0000 7f7355000640 1 -- 172.21.15.114:0/3394376166 wait complete. 2024-09-21T06:52:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 Processor -- start 2024-09-21T06:52:37.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 -- start start 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 0x7f735019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7350102f50 0x7f735019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73501101b0 0x7f73501a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7350115e40 con 0x7f7350102f50 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7350115cc0 con 0x7f73501101b0 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.031+0000 7f7355000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7350115fc0 con 0x7f7350102600 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 0x7f735019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7350102f50 0x7f735019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 0x7f735019c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42918/0 (socket says 172.21.15.114:42918) 2024-09-21T06:52:37.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734de00640 1 -- 172.21.15.114:0/59999965 learned_addr learned my addr 172.21.15.114:0/59999965 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734e800640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73501101b0 0x7f73501a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 -- 172.21.15.114:0/59999965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 msgr2=0x7f735019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 0x7f735019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 -- 172.21.15.114:0/59999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73501101b0 msgr2=0x7f73501a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73501101b0 0x7f73501a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 -- 172.21.15.114:0/59999965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73501a27e0 con 0x7f7350102f50 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734e800640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73501101b0 0x7f73501a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.032+0000 7f734d400640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7350102f50 0x7f735019ce10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b3f0 tx=0x7f733c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.033+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c015020 con 0x7f7350102f50 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.033+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501a2ad0 con 0x7f7350102f50 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.033+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733c011ce0 con 0x7f7350102f50 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.033+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501a92b0 con 0x7f7350102f50 2024-09-21T06:52:37.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.033+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c010d60 con 0x7f7350102f50 2024-09-21T06:52:37.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.034+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f735006c1b0 con 0x7f7350102f50 2024-09-21T06:52:37.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.037+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f733c018020 con 0x7f7350102f50 2024-09-21T06:52:37.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.039+0000 7f7346a00640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f73340b65f0 0x7f73340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.039+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f733c0e7610 con 0x7f7350102f50 2024-09-21T06:52:37.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.039+0000 7f734de00640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f73340b65f0 0x7f73340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.040+0000 7f734de00640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f73340b65f0 0x7f73340b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7340009a40 tx=0x7f73400091c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:37.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.040+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f733c01f020 con 0x7f7350102f50 2024-09-21T06:52:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:36 smithi183 ceph-mon[34219]: pgmap v350: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T06:52:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:36 smithi183 ceph-mon[34219]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-21T06:52:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:36 smithi183 ceph-mon[34219]: Cluster is now healthy 2024-09-21T06:52:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:36 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3687860397' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T06:52:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:36 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/129203790' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T06:52:37.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.236+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7350102a90 con 0x7f7350102f50 2024-09-21T06:52:37.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.237+0000 7f7346a00640 1 -- 172.21.15.114:0/59999965 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f733c0ad8b0 con 0x7f7350102f50 2024-09-21T06:52:37.239 INFO:teuthology.orchestra.run.smithi114.stdout:38654705761 2024-09-21T06:52:37.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.239+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f73340b65f0 msgr2=0x7f73340b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.239+0000 7f7355000640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f73340b65f0 0x7f73340b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7340009a40 tx=0x7f73400091c0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7350102f50 msgr2=0x7f735019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7350102f50 0x7f735019ce10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b3f0 tx=0x7f733c00b8b0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 shutdown_connections 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f73340b65f0 0x7f73340b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73501101b0 0x7f73501a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7350102f50 0x7f735019ce10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 --2- 172.21.15.114:0/59999965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7350102600 0x7f735019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 >> 172.21.15.114:0/59999965 conn(0x7f7350079650 msgr2=0x7f7350109d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 shutdown_connections 2024-09-21T06:52:37.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:37.240+0000 7f7355000640 1 -- 172.21.15.114:0/59999965 wait complete. 2024-09-21T06:52:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 -- 172.21.15.114:0/2821114162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c1094a0 msgr2=0x7f981c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 --2- 172.21.15.114:0/2821114162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c1094a0 0x7f981c1098b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9808009900 tx=0x7f980802a410 comp rx=0 tx=0).stop 2024-09-21T06:52:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 -- 172.21.15.114:0/2821114162 shutdown_connections 2024-09-21T06:52:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 --2- 172.21.15.114:0/2821114162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f981c109df0 0x7f981c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 --2- 172.21.15.114:0/2821114162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c1094a0 0x7f981c1098b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 --2- 172.21.15.114:0/2821114162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c103b60 0x7f981c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 -- 172.21.15.114:0/2821114162 >> 172.21.15.114:0/2821114162 conn(0x7f981c07b850 msgr2=0x7f981c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 -- 172.21.15.114:0/2821114162 shutdown_connections 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.259+0000 7f9822a00640 1 -- 172.21.15.114:0/2821114162 wait complete. 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 Processor -- start 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 -- start start 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c103b60 0x7f981c19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c1094a0 0x7f981c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f981c109df0 0x7f981c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f981c113ab0 con 0x7f981c103b60 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f981c113930 con 0x7f981c1094a0 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.260+0000 7f9822a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f981c113c30 con 0x7f981c109df0 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c103b60 0x7f981c19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c103b60 0x7f981c19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38884/0 (socket says 172.21.15.114:38884) 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981be00640 1 -- 172.21.15.114:0/129203790 learned_addr learned my addr 172.21.15.114:0/129203790 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981b400640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c1094a0 0x7f981c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f9820a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f981c109df0 0x7f981c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981b400640 1 -- 172.21.15.114:0/129203790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f981c109df0 msgr2=0x7f981c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981b400640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f981c109df0 0x7f981c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981b400640 1 -- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c103b60 msgr2=0x7f981c19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981b400640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c103b60 0x7f981c19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.261+0000 7f981b400640 1 -- 172.21.15.114:0/129203790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c1a04d0 con 0x7f981c1094a0 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.262+0000 7f981b400640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c1094a0 0x7f981c19ab00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f980802a930 tx=0x7f9808031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.262+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980803f070 con 0x7f981c1094a0 2024-09-21T06:52:37.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.262+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c1a0760 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.262+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808004490 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.262+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808035070 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.262+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1a6f20 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.264+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f981c104de0 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.266+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f980803b460 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.267+0000 7f9818c00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98000b65f0 0x7f98000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.267+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9808108010 con 0x7f981c1094a0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.268+0000 7f981be00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98000b65f0 0x7f98000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.268+0000 7f981be00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98000b65f0 0x7f98000b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f980c0095f0 tx=0x7f980c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:37.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.269+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98080d2690 con 0x7f981c1094a0 2024-09-21T06:52:37.330 INFO:teuthology.orchestra.run.smithi114.stdout:253403070509 2024-09-21T06:52:37.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.477+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f981c10bbe0 con 0x7f981c1094a0 2024-09-21T06:52:37.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.477+0000 7f9818c00640 1 -- 172.21.15.114:0/129203790 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f98080d75d0 con 0x7f981c1094a0 2024-09-21T06:52:37.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98000b65f0 msgr2=0x7f98000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98000b65f0 0x7f98000b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f980c0095f0 tx=0x7f980c005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c1094a0 msgr2=0x7f981c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c1094a0 0x7f981c19ab00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f980802a930 tx=0x7f9808031a20 comp rx=0 tx=0).stop 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 shutdown_connections 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f981c109df0 0x7f981c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f98000b65f0 0x7f98000b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f981c1094a0 0x7f981c19ab00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 --2- 172.21.15.114:0/129203790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f981c103b60 0x7f981c19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 >> 172.21.15.114:0/129203790 conn(0x7f981c07b850 msgr2=0x7f981c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 shutdown_connections 2024-09-21T06:52:37.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:36.480+0000 7f9822a00640 1 -- 172.21.15.114:0/129203790 wait complete. 2024-09-21T06:52:37.964 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:37 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/59999965' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:52:38.000 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070506 got 253403070509 for osd.9 2024-09-21T06:52:38.000 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:38.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:37 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/59999965' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:52:38.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:37 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/59999965' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T06:52:38.999 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:38 smithi114 ceph-mon[30654]: pgmap v351: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:39.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 -- 172.21.15.114:0/209671428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b010ff90 msgr2=0x7f99b01123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 --2- 172.21.15.114:0/209671428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b010ff90 0x7f99b01123b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009ca0 tx=0x7f99ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 -- 172.21.15.114:0/209671428 shutdown_connections 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 --2- 172.21.15.114:0/209671428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b010ff90 0x7f99b01123b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 --2- 172.21.15.114:0/209671428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99b0076080 0x7f99b010fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 --2- 172.21.15.114:0/209671428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 0x7f99b0075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 -- 172.21.15.114:0/209671428 >> 172.21.15.114:0/209671428 conn(0x7f99b00ff400 msgr2=0x7f99b0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 -- 172.21.15.114:0/209671428 shutdown_connections 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.062+0000 7f99b7200640 1 -- 172.21.15.114:0/209671428 wait complete. 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.063+0000 7f99b7200640 1 Processor -- start 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 -- start start 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 0x7f99b019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 0x7f99b019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99b010ff90 0x7f99b01a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0115b40 con 0x7f99b0076080 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99b01159c0 con 0x7f99b0075730 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0115cc0 con 0x7f99b010ff90 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 0x7f99b019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 0x7f99b019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38914/0 (socket says 172.21.15.114:38914) 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.064+0000 7f99b5400640 1 -- 172.21.15.114:0/709868339 learned_addr learned my addr 172.21.15.114:0/709868339 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5e00640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 0x7f99b019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b6800640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99b010ff90 0x7f99b01a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5400640 1 -- 172.21.15.114:0/709868339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99b010ff90 msgr2=0x7f99b01a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5400640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99b010ff90 0x7f99b01a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5400640 1 -- 172.21.15.114:0/709868339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 msgr2=0x7f99b019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5400640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 0x7f99b019c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5400640 1 -- 172.21.15.114:0/709868339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b01a23d0 con 0x7f99b0076080 2024-09-21T06:52:39.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5e00640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 0x7f99b019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.065+0000 7f99b5400640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 0x7f99b019ca00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.066+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a000ccc0 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.066+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b01a26c0 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.066+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99a00054d0 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.066+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0010430 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.066+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b01a8e20 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.067+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b0075b40 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.070+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f99a0005000 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.072+0000 7f99aaa00640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99980b65f0 0x7f99980b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.072+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f99a00e7ec0 con 0x7f99b0076080 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.072+0000 7f99b5e00640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99980b65f0 0x7f99980b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.073+0000 7f99b5e00640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99980b65f0 0x7f99980b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009fc0 tx=0x7f99a4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:39.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.074+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f99a0056710 con 0x7f99b0076080 2024-09-21T06:52:39.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:38 smithi183 ceph-mon[34219]: pgmap v351: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:39.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:38 smithi193 ceph-mon[34601]: pgmap v351: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:39.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 -- 172.21.15.114:0/1153515223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c1039b0 msgr2=0x7f0a1c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 --2- 172.21.15.114:0/1153515223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c103dc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009900 tx=0x7f0a0c02a400 comp rx=0 tx=0).stop 2024-09-21T06:52:39.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 -- 172.21.15.114:0/1153515223 shutdown_connections 2024-09-21T06:52:39.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 --2- 172.21.15.114:0/1153515223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c109c40 0x7f0a1c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 --2- 172.21.15.114:0/1153515223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a1c1092f0 0x7f0a1c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 --2- 172.21.15.114:0/1153515223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c103dc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 -- 172.21.15.114:0/1153515223 >> 172.21.15.114:0/1153515223 conn(0x7f0a1c0ff440 msgr2=0x7f0a1c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.177+0000 7f0a1be00640 1 -- 172.21.15.114:0/1153515223 shutdown_connections 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.178+0000 7f0a1be00640 1 -- 172.21.15.114:0/1153515223 wait complete. 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.178+0000 7f0a1be00640 1 Processor -- start 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 -- start start 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a1c1092f0 0x7f0a1c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 0x7f0a1c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c113900 con 0x7f0a1c109c40 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c113780 con 0x7f0a1c1039b0 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c113a80 con 0x7f0a1c1092f0 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 0x7f0a1c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57286/0 (socket says 172.21.15.114:57286) 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1aa00640 1 -- 172.21.15.114:0/329243806 learned_addr learned my addr 172.21.15.114:0/329243806 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1a000640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a1c1092f0 0x7f0a1c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.179+0000 7f0a1b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 0x7f0a1c19fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38946/0 (socket says 172.21.15.114:38946) 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1aa00640 1 -- 172.21.15.114:0/329243806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a1c1092f0 msgr2=0x7f0a1c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1aa00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a1c1092f0 0x7f0a1c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1aa00640 1 -- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 msgr2=0x7f0a1c19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1aa00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 0x7f0a1c19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1aa00640 1 -- 172.21.15.114:0/329243806 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c1a01c0 con 0x7f0a1c1039b0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1b400640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 0x7f0a1c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.180+0000 7f0a1aa00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c19a310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005940 tx=0x7f0a0c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.181+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c03f030 con 0x7f0a1c1039b0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.181+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c1a0450 con 0x7f0a1c1039b0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.181+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a0c003aa0 con 0x7f0a1c1039b0 2024-09-21T06:52:39.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.181+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c035070 con 0x7f0a1c1039b0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.181+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1a0980 con 0x7f0a1c1039b0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.182+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a1c103dc0 con 0x7f0a1c1039b0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.184+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f0a0c042030 con 0x7f0a1c1039b0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.186+0000 7f0a13400640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0a000b66a0 0x7f0a000b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.186+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0a0c108200 con 0x7f0a1c1039b0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.186+0000 7f0a1a000640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0a000b66a0 0x7f0a000b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.187+0000 7f0a1a000640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0a000b66a0 0x7f0a000b8bb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c108f00 tx=0x7f0a0800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:39.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.188+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a1c1a0980 con 0x7f0a1c1039b0 2024-09-21T06:52:39.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.278+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f99b019d590 con 0x7f99b0076080 2024-09-21T06:52:39.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.278+0000 7f99aaa00640 1 -- 172.21.15.114:0/709868339 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99b019d590 con 0x7f99b0076080 2024-09-21T06:52:39.280 INFO:teuthology.orchestra.run.smithi114.stdout:180388626495 2024-09-21T06:52:39.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99980b65f0 msgr2=0x7f99980b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99980b65f0 0x7f99980b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009fc0 tx=0x7f99a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:39.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 msgr2=0x7f99b019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 0x7f99b019ca00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e9c0 tx=0x7f99a000ee80 comp rx=0 tx=0).stop 2024-09-21T06:52:39.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 shutdown_connections 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99b010ff90 0x7f99b01a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99b0076080 0x7f99b019ca00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99980b65f0 0x7f99980b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 --2- 172.21.15.114:0/709868339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99b0075730 0x7f99b019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.281+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 >> 172.21.15.114:0/709868339 conn(0x7f99b00ff400 msgr2=0x7f99b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.282+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 shutdown_connections 2024-09-21T06:52:39.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.282+0000 7f99b7200640 1 -- 172.21.15.114:0/709868339 wait complete. 2024-09-21T06:52:39.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.398+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0a1c19b440 con 0x7f0a1c1039b0 2024-09-21T06:52:39.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.399+0000 7f0a13400640 1 -- 172.21.15.114:0/329243806 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0a1c19b440 con 0x7f0a1c1039b0 2024-09-21T06:52:39.401 INFO:teuthology.orchestra.run.smithi114.stdout:227633266741 2024-09-21T06:52:39.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.401+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0a000b66a0 msgr2=0x7f0a000b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.401+0000 7f0a1be00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0a000b66a0 0x7f0a000b8bb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c108f00 tx=0x7f0a0800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.401+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 msgr2=0x7f0a1c19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:39.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.401+0000 7f0a1be00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c19a310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005940 tx=0x7f0a0c004270 comp rx=0 tx=0).stop 2024-09-21T06:52:39.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 shutdown_connections 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0a1c109c40 0x7f0a1c19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a1c1092f0 0x7f0a1c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0a000b66a0 0x7f0a000b8bb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 --2- 172.21.15.114:0/329243806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a1c1039b0 0x7f0a1c19a310 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 >> 172.21.15.114:0/329243806 conn(0x7f0a1c0ff440 msgr2=0x7f0a1c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 shutdown_connections 2024-09-21T06:52:39.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:39.402+0000 7f0a1be00640 1 -- 172.21.15.114:0/329243806 wait complete. 2024-09-21T06:52:39.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:39 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/709868339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:52:39.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:39 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/329243806' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:52:40.016 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:39 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/709868339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:52:40.016 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:39 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/329243806' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:52:40.063 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:39 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/709868339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T06:52:40.063 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:39 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/329243806' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T06:52:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:40 smithi114 ceph-mon[30654]: pgmap v352: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-21T06:52:40.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:40 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1993026213' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:52:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:40 smithi183 ceph-mon[34219]: pgmap v352: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-21T06:52:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:40 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1993026213' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:52:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:40 smithi193 ceph-mon[34601]: pgmap v352: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-09-21T06:52:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:40 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1993026213' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T06:52:41.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462966 got 201863462968 for osd.7 2024-09-21T06:52:41.399 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 -- 172.21.15.114:0/2555060045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 msgr2=0x7efd64102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 --2- 172.21.15.114:0/2555060045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd64102f40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efd54009900 tx=0x7efd5402a410 comp rx=0 tx=0).stop 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 -- 172.21.15.114:0/2555060045 shutdown_connections 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 --2- 172.21.15.114:0/2555060045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 0x7efd6410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 --2- 172.21.15.114:0/2555060045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd641051b0 0x7efd641055c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 --2- 172.21.15.114:0/2555060045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd64102f40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 -- 172.21.15.114:0/2555060045 >> 172.21.15.114:0/2555060045 conn(0x7efd640799a0 msgr2=0x7efd64079db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 -- 172.21.15.114:0/2555060045 shutdown_connections 2024-09-21T06:52:41.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.636+0000 7efd6a200640 1 -- 172.21.15.114:0/2555060045 wait complete. 2024-09-21T06:52:41.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.637+0000 7efd6a200640 1 Processor -- start 2024-09-21T06:52:41.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.637+0000 7efd6a200640 1 -- start start 2024-09-21T06:52:41.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd6a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd6419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd6a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd641051b0 0x7efd6419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd6a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 0x7efd641a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd6a200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efd64113b50 con 0x7efd64102b30 2024-09-21T06:52:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd6a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efd641139d0 con 0x7efd64105b00 2024-09-21T06:52:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd6a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efd64113cd0 con 0x7efd641051b0 2024-09-21T06:52:41.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd63400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd6419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd63400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd6419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38992/0 (socket says 172.21.15.114:38992) 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd5be00640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd641051b0 0x7efd6419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd63400640 1 -- 172.21.15.114:0/1209823632 learned_addr learned my addr 172.21.15.114:0/1209823632 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.638+0000 7efd63e00640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 0x7efd641a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd63e00640 1 -- 172.21.15.114:0/1209823632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd641051b0 msgr2=0x7efd6419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd63e00640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd641051b0 0x7efd6419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd63e00640 1 -- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 msgr2=0x7efd6419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd63e00640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd6419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd63e00640 1 -- 172.21.15.114:0/1209823632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd641a2670 con 0x7efd64105b00 2024-09-21T06:52:41.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd63e00640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 0x7efd641a1f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009fd0 tx=0x7efd5c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:41.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00ce20 con 0x7efd64105b00 2024-09-21T06:52:41.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c005020 con 0x7efd64105b00 2024-09-21T06:52:41.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641a2960 con 0x7efd64105b00 2024-09-21T06:52:41.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.639+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c010690 con 0x7efd64105b00 2024-09-21T06:52:41.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.640+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641a90c0 con 0x7efd64105b00 2024-09-21T06:52:41.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.641+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd64102f40 con 0x7efd64105b00 2024-09-21T06:52:41.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.643+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7efd5c0054d0 con 0x7efd64105b00 2024-09-21T06:52:41.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.645+0000 7efd60c00640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efd480b65f0 0x7efd480b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.645+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7efd5c0e7fe0 con 0x7efd64105b00 2024-09-21T06:52:41.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.645+0000 7efd63400640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efd480b65f0 0x7efd480b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.646+0000 7efd63400640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efd480b65f0 0x7efd480b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efd540096c0 tx=0x7efd54005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:41.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.647+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7efd5c0ae220 con 0x7efd64105b00 2024-09-21T06:52:41.804 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/359630148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:52:41.804 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:41 smithi114 ceph-mon[30654]: pgmap v353: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T06:52:41.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.847+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efd6419d830 con 0x7efd64105b00 2024-09-21T06:52:41.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.848+0000 7efd60c00640 1 -- 172.21.15.114:0/1209823632 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efd6419d830 con 0x7efd64105b00 2024-09-21T06:52:41.850 INFO:teuthology.orchestra.run.smithi114.stdout:137438953549 2024-09-21T06:52:41.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.850+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efd480b65f0 msgr2=0x7efd480b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.850+0000 7efd6a200640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efd480b65f0 0x7efd480b8b00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efd540096c0 tx=0x7efd54005a20 comp rx=0 tx=0).stop 2024-09-21T06:52:41.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.850+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 msgr2=0x7efd641a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.850+0000 7efd6a200640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 0x7efd641a1f40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009fd0 tx=0x7efd5c00ef10 comp rx=0 tx=0).stop 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 shutdown_connections 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efd480b65f0 0x7efd480b8b00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd64105b00 0x7efd641a1f40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efd641051b0 0x7efd6419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 --2- 172.21.15.114:0/1209823632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efd64102b30 0x7efd6419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 >> 172.21.15.114:0/1209823632 conn(0x7efd640799a0 msgr2=0x7efd64111020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 shutdown_connections 2024-09-21T06:52:41.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.851+0000 7efd6a200640 1 -- 172.21.15.114:0/1209823632 wait complete. 2024-09-21T06:52:41.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 -- 172.21.15.114:0/114007972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 msgr2=0x7f3398103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 --2- 172.21.15.114:0/114007972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f3398103f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3388009900 tx=0x7f338802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 -- 172.21.15.114:0/114007972 shutdown_connections 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 --2- 172.21.15.114:0/114007972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3398109e00 0x7f3398110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 --2- 172.21.15.114:0/114007972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33981094b0 0x7f33981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 --2- 172.21.15.114:0/114007972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f3398103f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 -- 172.21.15.114:0/114007972 >> 172.21.15.114:0/114007972 conn(0x7f33980ff5e0 msgr2=0x7f3398101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 -- 172.21.15.114:0/114007972 shutdown_connections 2024-09-21T06:52:41.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.858+0000 7f339da00640 1 -- 172.21.15.114:0/114007972 wait complete. 2024-09-21T06:52:41.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.859+0000 7f339da00640 1 Processor -- start 2024-09-21T06:52:41.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.859+0000 7f339da00640 1 -- start start 2024-09-21T06:52:41.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.859+0000 7f339da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f339819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f339da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33981094b0 0x7f339819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f339da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3398109e00 0x7f33981a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f339da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3398113ac0 con 0x7f3398109e00 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f339da00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3398113940 con 0x7f3398103b70 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f339da00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3398113c40 con 0x7f33981094b0 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f339819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f339819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57350/0 (socket says 172.21.15.114:57350) 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 -- 172.21.15.114:0/3517225430 learned_addr learned my addr 172.21.15.114:0/3517225430 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396000640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33981094b0 0x7f339819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3397400640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3398109e00 0x7f33981a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 -- 172.21.15.114:0/3517225430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33981094b0 msgr2=0x7f339819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33981094b0 0x7f339819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 -- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3398109e00 msgr2=0x7f33981a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.860+0000 7f3396a00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3398109e00 0x7f33981a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f3396a00640 1 -- 172.21.15.114:0/3517225430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33981a2620 con 0x7f3398103b70 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f3396a00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f339819c6b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3388005940 tx=0x7f3388004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:41.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338803f030 con 0x7f3398103b70 2024-09-21T06:52:41.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33981a28b0 con 0x7f3398103b70 2024-09-21T06:52:41.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3388003aa0 con 0x7f3398103b70 2024-09-21T06:52:41.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388035070 con 0x7f3398103b70 2024-09-21T06:52:41.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.861+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33981a9070 con 0x7f3398103b70 2024-09-21T06:52:41.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.862+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3398103f80 con 0x7f3398103b70 2024-09-21T06:52:41.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.865+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f3388042030 con 0x7f3398103b70 2024-09-21T06:52:41.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.866+0000 7f338f400640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f337c0b65f0 0x7f337c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.867+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3388107f70 con 0x7f3398103b70 2024-09-21T06:52:41.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.867+0000 7f3396000640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f337c0b65f0 0x7f337c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.867+0000 7f3396000640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f337c0b65f0 0x7f337c0b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3398105bb0 tx=0x7f3384005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:41.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.869+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f33880d2680 con 0x7f3398103b70 2024-09-21T06:52:41.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 -- 172.21.15.114:0/4048856405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 msgr2=0x7f6134105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 --2- 172.21.15.114:0/4048856405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f6134105fc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6124009900 tx=0x7f612402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 -- 172.21.15.114:0/4048856405 shutdown_connections 2024-09-21T06:52:41.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 --2- 172.21.15.114:0/4048856405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 0x7f6134112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 --2- 172.21.15.114:0/4048856405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 0x7f613410b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 --2- 172.21.15.114:0/4048856405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f6134105fc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 -- 172.21.15.114:0/4048856405 >> 172.21.15.114:0/4048856405 conn(0x7f6134101680 msgr2=0x7f6134103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 -- 172.21.15.114:0/4048856405 shutdown_connections 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.149+0000 7f6133e00640 1 -- 172.21.15.114:0/4048856405 wait complete. 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 Processor -- start 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 -- start start 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f613419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 0x7f613419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 0x7f61341a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6134115b00 con 0x7f613410be40 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6134115980 con 0x7f613410b4f0 2024-09-21T06:52:41.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.150+0000 7f6133e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6134115c80 con 0x7f6134105bb0 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6133400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 0x7f61341a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f613419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6133400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 0x7f61341a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38964/0 (socket says 172.21.15.114:38964) 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f613419c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42966/0 (socket says 172.21.15.114:42966) 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6133400640 1 -- 172.21.15.114:0/1993026213 learned_addr learned my addr 172.21.15.114:0/1993026213 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f612a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 0x7f613419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 -- 172.21.15.114:0/1993026213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 msgr2=0x7f613419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 0x7f613419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 -- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 msgr2=0x7f61341a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 0x7f61341a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f6132a00640 1 -- 172.21.15.114:0/1993026213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61341a2500 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.151+0000 7f612a000640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 0x7f613419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.152+0000 7f6132a00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f613419c590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f612402a910 tx=0x7f6124031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.152+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612403f030 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.152+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6124004490 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.152+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61341a2790 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.152+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124035070 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.152+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61341a8f50 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.153+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6134106e30 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.155+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f6124042030 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.157+0000 7f612be00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f61180b65f0 0x7f61180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.157+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6124107ea0 con 0x7f6134105bb0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.157+0000 7f612a000640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f61180b65f0 0x7f61180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:41.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.158+0000 7f612a000640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f61180b65f0 0x7f61180b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f613410aec0 tx=0x7f611c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:41.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.159+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61240d24c0 con 0x7f6134105bb0 2024-09-21T06:52:41.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.354+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f613410dc10 con 0x7f6134105bb0 2024-09-21T06:52:41.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.355+0000 7f612be00640 1 -- 172.21.15.114:0/1993026213 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61240d7400 con 0x7f6134105bb0 2024-09-21T06:52:41.917 INFO:teuthology.orchestra.run.smithi114.stdout:60129542236 2024-09-21T06:52:41.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.357+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f61180b65f0 msgr2=0x7f61180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.357+0000 7f6133e00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f61180b65f0 0x7f61180b8b00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f613410aec0 tx=0x7f611c005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 msgr2=0x7f613419c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f613419c590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f612402a910 tx=0x7f6124031a20 comp rx=0 tx=0).stop 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 shutdown_connections 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f613410be40 0x7f61341a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f61180b65f0 0x7f61180b8b00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f613410b4f0 0x7f613419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 --2- 172.21.15.114:0/1993026213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6134105bb0 0x7f613419c590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 >> 172.21.15.114:0/1993026213 conn(0x7f6134101680 msgr2=0x7f6134103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 shutdown_connections 2024-09-21T06:52:41.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:40.358+0000 7f6133e00640 1 -- 172.21.15.114:0/1993026213 wait complete. 2024-09-21T06:52:42.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.064+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f33981a92d0 con 0x7f3398103b70 2024-09-21T06:52:42.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.065+0000 7f338f400640 1 -- 172.21.15.114:0/3517225430 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f33880d7530 con 0x7f3398103b70 2024-09-21T06:52:42.067 INFO:teuthology.orchestra.run.smithi114.stdout:154618822726 2024-09-21T06:52:42.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f337c0b65f0 msgr2=0x7f337c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f337c0b65f0 0x7f337c0b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3398105bb0 tx=0x7f3384005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 msgr2=0x7f339819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f339819c6b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3388005940 tx=0x7f3388004270 comp rx=0 tx=0).stop 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 shutdown_connections 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3398109e00 0x7f33981a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33981094b0 0x7f339819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f337c0b65f0 0x7f337c0b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 --2- 172.21.15.114:0/3517225430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3398103b70 0x7f339819c6b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 >> 172.21.15.114:0/3517225430 conn(0x7f33980ff5e0 msgr2=0x7f3398108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:42.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 shutdown_connections 2024-09-21T06:52:42.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:42.068+0000 7f339da00640 1 -- 172.21.15.114:0/3517225430 wait complete. 2024-09-21T06:52:42.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/359630148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:52:42.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:41 smithi183 ceph-mon[34219]: pgmap v353: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T06:52:42.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/359630148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T06:52:42.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:41 smithi193 ceph-mon[34601]: pgmap v353: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T06:52:42.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 -- 172.21.15.114:0/2703562376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc105bc0 msgr2=0x7f84bc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 --2- 172.21.15.114:0/2703562376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc105bc0 0x7f84bc105fd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009860 tx=0x7f84ac02a340 comp rx=0 tx=0).stop 2024-09-21T06:52:42.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 -- 172.21.15.114:0/2703562376 shutdown_connections 2024-09-21T06:52:42.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 --2- 172.21.15.114:0/2703562376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84bc10be50 0x7f84bc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 --2- 172.21.15.114:0/2703562376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10b500 0x7f84bc10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 --2- 172.21.15.114:0/2703562376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc105bc0 0x7f84bc105fd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 -- 172.21.15.114:0/2703562376 >> 172.21.15.114:0/2703562376 conn(0x7f84bc101670 msgr2=0x7f84bc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 -- 172.21.15.114:0/2703562376 shutdown_connections 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.172+0000 7f84c0e00640 1 -- 172.21.15.114:0/2703562376 wait complete. 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.173+0000 7f84c0e00640 1 Processor -- start 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.173+0000 7f84c0e00640 1 -- start start 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84c0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84bc105bc0 0x7f84bc19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84c0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 0x7f84bc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84c0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10be50 0x7f84bc1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84c0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc115b10 con 0x7f84bc10b500 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84c0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc115990 con 0x7f84bc10be50 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84c0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc115c90 con 0x7f84bc105bc0 2024-09-21T06:52:42.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 0x7f84bc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 0x7f84bc19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38984/0 (socket says 172.21.15.114:38984) 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 -- 172.21.15.114:0/359630148 learned_addr learned my addr 172.21.15.114:0/359630148 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84bbe00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10be50 0x7f84bc1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84bb400640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84bc105bc0 0x7f84bc19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 -- 172.21.15.114:0/359630148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84bc105bc0 msgr2=0x7f84bc19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84bc105bc0 0x7f84bc19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 -- 172.21.15.114:0/359630148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10be50 msgr2=0x7f84bc1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10be50 0x7f84bc1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84baa00640 1 -- 172.21.15.114:0/359630148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc1a2490 con 0x7f84bc10b500 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.174+0000 7f84bbe00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10be50 0x7f84bc1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.175+0000 7f84baa00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 0x7f84bc19cac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f84a800e9c0 tx=0x7f84a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.175+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a800ccc0 con 0x7f84bc10b500 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.175+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84a80108e0 con 0x7f84bc10b500 2024-09-21T06:52:42.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.175+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc1a2780 con 0x7f84bc10b500 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.175+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80187e0 con 0x7f84bc10b500 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.175+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a8ee0 con 0x7f84bc10b500 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.176+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84bc105fd0 con 0x7f84bc10b500 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.178+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f84a8010430 con 0x7f84bc10b500 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.180+0000 7f84b3e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f84a00b65f0 0x7f84a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.180+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f84a80e80c0 con 0x7f84bc10b500 2024-09-21T06:52:42.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.180+0000 7f84bb400640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f84a00b65f0 0x7f84a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:42.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.181+0000 7f84bb400640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f84a00b65f0 0x7f84a00b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009830 tx=0x7f84ac030340 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:42.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.182+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84a80b26e0 con 0x7f84bc10b500 2024-09-21T06:52:42.236 INFO:teuthology.orchestra.run.smithi114.stdout:300647710757 2024-09-21T06:52:42.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.378+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f84bc19d650 con 0x7f84bc10b500 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.379+0000 7f84b3e00640 1 -- 172.21.15.114:0/359630148 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f84bc19d650 con 0x7f84bc10b500 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.381+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f84a00b65f0 msgr2=0x7f84a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.381+0000 7f84c0e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f84a00b65f0 0x7f84a00b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009830 tx=0x7f84ac030340 comp rx=0 tx=0).stop 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.381+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 msgr2=0x7f84bc19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.381+0000 7f84c0e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 0x7f84bc19cac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f84a800e9c0 tx=0x7f84a800ee80 comp rx=0 tx=0).stop 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 shutdown_connections 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f84a00b65f0 0x7f84a00b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84bc10be50 0x7f84bc1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f84bc10b500 0x7f84bc19cac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 --2- 172.21.15.114:0/359630148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84bc105bc0 0x7f84bc19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 >> 172.21.15.114:0/359630148 conn(0x7f84bc101670 msgr2=0x7f84bc103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 shutdown_connections 2024-09-21T06:52:42.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:41.382+0000 7f84c0e00640 1 -- 172.21.15.114:0/359630148 wait complete. 2024-09-21T06:52:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:42 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1209823632' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:52:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:42 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3517225430' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:52:42.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378708 got 81604378711 for osd.2 2024-09-21T06:52:42.917 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:42 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1209823632' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:52:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:42 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3517225430' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:52:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:42 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1209823632' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T06:52:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:42 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3517225430' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T06:52:43.717 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:43 smithi114 ceph-mon[30654]: pgmap v354: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 -- 172.21.15.114:0/1415255163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 msgr2=0x7fadb8112350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 --2- 172.21.15.114:0/1415255163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb8112350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009ca0 tx=0x7fadb002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 -- 172.21.15.114:0/1415255163 shutdown_connections 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 --2- 172.21.15.114:0/1415255163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb8112350 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 --2- 172.21.15.114:0/1415255163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb8076100 0x7fadb810f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 --2- 172.21.15.114:0/1415255163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb80757b0 0x7fadb8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:43.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 -- 172.21.15.114:0/1415255163 >> 172.21.15.114:0/1415255163 conn(0x7fadb80ff440 msgr2=0x7fadb8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:43.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 -- 172.21.15.114:0/1415255163 shutdown_connections 2024-09-21T06:52:43.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.881+0000 7fadbca00640 1 -- 172.21.15.114:0/1415255163 wait complete. 2024-09-21T06:52:43.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.882+0000 7fadbca00640 1 Processor -- start 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.882+0000 7fadbca00640 1 -- start start 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadbca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb80757b0 0x7fadb819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadbca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 0x7fadb819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadbca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb81a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadbca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8115ac0 con 0x7fadb810ff10 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadbca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8115940 con 0x7fadb80757b0 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadbca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8115c40 con 0x7fadb8076100 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadafe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 0x7fadb819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadb7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb81a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadafe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 0x7fadb819ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43068/0 (socket says 172.21.15.114:43068) 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadafe00640 1 -- 172.21.15.114:0/4113695254 learned_addr learned my addr 172.21.15.114:0/4113695254 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:43.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadb6800640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb80757b0 0x7fadb819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:43.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadafe00640 1 -- 172.21.15.114:0/4113695254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb80757b0 msgr2=0x7fadb819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:43.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.883+0000 7fadafe00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb80757b0 0x7fadb819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:43.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadafe00640 1 -- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 msgr2=0x7fadb81a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:43.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadafe00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb81a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadafe00640 1 -- 172.21.15.114:0/4113695254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb81a2460 con 0x7fadb8076100 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadb7200640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb81a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadafe00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 0x7fadb819ca90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fada000e9c0 tx=0x7fada000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada000cd80 con 0x7fadb8076100 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb81a2750 con 0x7fadb8076100 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada00035f0 con 0x7fadb8076100 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0010690 con 0x7fadb8076100 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.884+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81a8eb0 con 0x7fadb8076100 2024-09-21T06:52:43.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.885+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb8075bc0 con 0x7fadb8076100 2024-09-21T06:52:43.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.888+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fada00037a0 con 0x7fadb8076100 2024-09-21T06:52:43.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.891+0000 7fadaf400640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad9c0b65f0 0x7fad9c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:43.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.891+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fada00e7d80 con 0x7fadb8076100 2024-09-21T06:52:43.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.891+0000 7fadb6800640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad9c0b65f0 0x7fad9c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:43.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.891+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fada00adfc0 con 0x7fadb8076100 2024-09-21T06:52:43.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:43.892+0000 7fadb6800640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad9c0b65f0 0x7fad9c0b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fada8009bb0 tx=0x7fada8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:44.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.104+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fadb819d620 con 0x7fadb8076100 2024-09-21T06:52:44.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.105+0000 7fadaf400640 1 -- 172.21.15.114:0/4113695254 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fada00b23a0 con 0x7fadb8076100 2024-09-21T06:52:44.107 INFO:teuthology.orchestra.run.smithi114.stdout:111669149778 2024-09-21T06:52:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad9c0b65f0 msgr2=0x7fad9c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad9c0b65f0 0x7fad9c0b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fada8009bb0 tx=0x7fada8005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 msgr2=0x7fadb819ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:44.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 0x7fadb819ca90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fada000e9c0 tx=0x7fada000ee80 comp rx=0 tx=0).stop 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 shutdown_connections 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fadb810ff10 0x7fadb81a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb8076100 0x7fadb819ca90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fad9c0b65f0 0x7fad9c0b8b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 --2- 172.21.15.114:0/4113695254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb80757b0 0x7fadb819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.107+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 >> 172.21.15.114:0/4113695254 conn(0x7fadb80ff440 msgr2=0x7fadb8109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.108+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 shutdown_connections 2024-09-21T06:52:44.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:44.108+0000 7fadbca00640 1 -- 172.21.15.114:0/4113695254 wait complete. 2024-09-21T06:52:44.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:43 smithi183 ceph-mon[34219]: pgmap v354: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T06:52:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:43 smithi193 ceph-mon[34601]: pgmap v354: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T06:52:44.985 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:44 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4113695254' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:52:45.022 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705758 got 38654705761 for osd.0 2024-09-21T06:52:45.023 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:44 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4113695254' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:52:45.161 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906983 got 274877906984 for osd.10 2024-09-21T06:52:45.161 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:45.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:44 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4113695254' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T06:52:45.636 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710755 got 300647710757 for osd.11 2024-09-21T06:52:45.636 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:45.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:45 smithi114 ceph-mon[30654]: pgmap v355: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T06:52:45.955 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953545 got 137438953549 for osd.4 2024-09-21T06:52:45.955 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:46.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:45 smithi183 ceph-mon[34219]: pgmap v355: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T06:52:46.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:45 smithi193 ceph-mon[34601]: pgmap v355: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-09-21T06:52:46.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266738 got 227633266741 for osd.8 2024-09-21T06:52:46.388 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:46.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626493 got 180388626495 for osd.6 2024-09-21T06:52:46.760 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:47.045 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:46 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:52:47.049 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:46 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:52:47.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:46 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:52:47.228 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542232 got 60129542236 for osd.1 2024-09-21T06:52:47.228 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:47.534 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149773 got 111669149778 for osd.3 2024-09-21T06:52:47.535 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:47.892 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822723 got 154618822726 for osd.5 2024-09-21T06:52:47.893 DEBUG:teuthology.parallel:result is None 2024-09-21T06:52:47.893 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:52:47.893 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:52:48.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:52:48.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:47 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:47 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:47 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:48.927 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 -- 172.21.15.114:0/929506230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe881092f0 msgr2=0x7fbe88109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 --2- 172.21.15.114:0/929506230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe881092f0 0x7fbe88109700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009900 tx=0x7fbe7402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 -- 172.21.15.114:0/929506230 shutdown_connections 2024-09-21T06:52:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 --2- 172.21.15.114:0/929506230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe88109c40 0x7fbe88110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 --2- 172.21.15.114:0/929506230 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe881092f0 0x7fbe88109700 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 --2- 172.21.15.114:0/929506230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe88103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 -- 172.21.15.114:0/929506230 >> 172.21.15.114:0/929506230 conn(0x7fbe880ff440 msgr2=0x7fbe88101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 -- 172.21.15.114:0/929506230 shutdown_connections 2024-09-21T06:52:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.550+0000 7fbe87200640 1 -- 172.21.15.114:0/929506230 wait complete. 2024-09-21T06:52:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.551+0000 7fbe87200640 1 Processor -- start 2024-09-21T06:52:49.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.551+0000 7fbe87200640 1 -- start start 2024-09-21T06:52:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe87200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe8819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe87200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe881092f0 0x7fbe8819caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe87200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe88109c40 0x7fbe881a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe87200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88113900 con 0x7fbe88109c40 2024-09-21T06:52:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe87200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88113780 con 0x7fbe881039b0 2024-09-21T06:52:49.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe87200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88113a80 con 0x7fbe881092f0 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe85e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe8819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe85400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe881092f0 0x7fbe8819caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe85e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe8819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59182/0 (socket says 172.21.15.114:59182) 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe85e00640 1 -- 172.21.15.114:0/1386920828 learned_addr learned my addr 172.21.15.114:0/1386920828 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe85400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe881092f0 0x7fbe8819caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34932/0 (socket says 172.21.15.114:34932) 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.552+0000 7fbe86800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe88109c40 0x7fbe881a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe85e00640 1 -- 172.21.15.114:0/1386920828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe881092f0 msgr2=0x7fbe8819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:49.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe85e00640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe881092f0 0x7fbe8819caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe85e00640 1 -- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe88109c40 msgr2=0x7fbe881a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe85e00640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe88109c40 0x7fbe881a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe85e00640 1 -- 172.21.15.114:0/1386920828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe881a24c0 con 0x7fbe881039b0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe85e00640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe8819c550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800e9f0 tx=0x7fbe7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800cd80 con 0x7fbe881039b0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe780108e0 con 0x7fbe881039b0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.553+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78018980 con 0x7fbe881039b0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.554+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe881a27b0 con 0x7fbe881039b0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.554+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe881a8f10 con 0x7fbe881039b0 2024-09-21T06:52:49.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.555+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe88104c30 con 0x7fbe881039b0 2024-09-21T06:52:49.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.557+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7fbe78010430 con 0x7fbe881039b0 2024-09-21T06:52:49.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.559+0000 7fbe7ea00640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbe6c0b6670 0x7fbe6c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:49.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.559+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbe780e8120 con 0x7fbe881039b0 2024-09-21T06:52:49.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.559+0000 7fbe85400640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbe6c0b6670 0x7fbe6c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:49.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.560+0000 7fbe85400640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbe6c0b6670 0x7fbe6c0b8b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009230 tx=0x7fbe74031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:49.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.561+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe780b2740 con 0x7fbe881039b0 2024-09-21T06:52:49.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.773+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe8810f740 con 0x7fbe6c0b6670 2024-09-21T06:52:49.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.779+0000 7fbe7ea00640 1 -- 172.21.15.114:0/1386920828 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7fbe8810f740 con 0x7fbe6c0b6670 2024-09-21T06:52:49.785 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:49.786 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbe6c0b6670 msgr2=0x7fbe6c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbe6c0b6670 0x7fbe6c0b8b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009230 tx=0x7fbe74031400 comp rx=0 tx=0).stop 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 msgr2=0x7fbe8819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe8819c550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800e9f0 tx=0x7fbe7800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 shutdown_connections 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbe88109c40 0x7fbe881a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe881092f0 0x7fbe8819caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbe6c0b6670 0x7fbe6c0b8b80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 --2- 172.21.15.114:0/1386920828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe881039b0 0x7fbe8819c550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:49.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.784+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 >> 172.21.15.114:0/1386920828 conn(0x7fbe880ff440 msgr2=0x7fbe881089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:49.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.785+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 shutdown_connections 2024-09-21T06:52:49.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:49.785+0000 7fbe87200640 1 -- 172.21.15.114:0/1386920828 wait complete. 2024-09-21T06:52:50.572 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-21T06:52:49.522394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665264,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":750,"num_read_kb":483,"num_write":375,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371960,"kb_used_data":14300,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569864,"statfs":{"total":1151940427776,"available":1151559540736,"internally_reserved":0,"allocated":14643200,"data_stored":8339342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003804"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":121,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:45.720814+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:52:45.720814+0000","last_peered":"2024-09-21T06:52:45.720814+0000","last_clean":"2024-09-21T06:52:45.720814+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:52:45.720814+0000","last_undegraded":"2024-09-21T06:52:45.720814+0000","last_fullsized":"2024-09-21T06:52:45.720814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892422+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:31.892422+0000","last_peered":"2024-09-21T06:52:31.892422+0000","last_clean":"2024-09-21T06:52:31.892422+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:31.892422+0000","last_undegraded":"2024-09-21T06:52:31.892422+0000","last_fullsized":"2024-09-21T06:52:31.892422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703592+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703592+0000","last_peered":"2024-09-21T06:52:31.703592+0000","last_clean":"2024-09-21T06:52:31.703592+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703592+0000","last_undegraded":"2024-09-21T06:52:31.703592+0000","last_fullsized":"2024-09-21T06:52:31.703592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892890+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:31.892890+0000","last_peered":"2024-09-21T06:52:31.892890+0000","last_clean":"2024-09-21T06:52:31.892890+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:31.892890+0000","last_undegraded":"2024-09-21T06:52:31.892890+0000","last_fullsized":"2024-09-21T06:52:31.892890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699187+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699187+0000","last_peered":"2024-09-21T06:52:31.699187+0000","last_clean":"2024-09-21T06:52:31.699187+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699187+0000","last_undegraded":"2024-09-21T06:52:31.699187+0000","last_fullsized":"2024-09-21T06:52:31.699187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891998+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:52:31.891998+0000","last_peered":"2024-09-21T06:52:31.891998+0000","last_clean":"2024-09-21T06:52:31.891998+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:52:31.891998+0000","last_undegraded":"2024-09-21T06:52:31.891998+0000","last_fullsized":"2024-09-21T06:52:31.891998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837017+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:31.837017+0000","last_peered":"2024-09-21T06:52:31.837017+0000","last_clean":"2024-09-21T06:52:31.837017+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:31.837017+0000","last_undegraded":"2024-09-21T06:52:31.837017+0000","last_fullsized":"2024-09-21T06:52:31.837017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699576+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699576+0000","last_peered":"2024-09-21T06:52:31.699576+0000","last_clean":"2024-09-21T06:52:31.699576+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699576+0000","last_undegraded":"2024-09-21T06:52:31.699576+0000","last_fullsized":"2024-09-21T06:52:31.699576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891874+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891874+0000","last_peered":"2024-09-21T06:52:31.891874+0000","last_clean":"2024-09-21T06:52:31.891874+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891874+0000","last_undegraded":"2024-09-21T06:52:31.891874+0000","last_fullsized":"2024-09-21T06:52:31.891874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702135+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:31.702135+0000","last_peered":"2024-09-21T06:52:31.702135+0000","last_clean":"2024-09-21T06:52:31.702135+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:31.702135+0000","last_undegraded":"2024-09-21T06:52:31.702135+0000","last_fullsized":"2024-09-21T06:52:31.702135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699290+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699290+0000","last_peered":"2024-09-21T06:52:31.699290+0000","last_clean":"2024-09-21T06:52:31.699290+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699290+0000","last_undegraded":"2024-09-21T06:52:31.699290+0000","last_fullsized":"2024-09-21T06:52:31.699290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.647910+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:31.647910+0000","last_peered":"2024-09-21T06:52:31.647910+0000","last_clean":"2024-09-21T06:52:31.647910+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:31.647910+0000","last_undegraded":"2024-09-21T06:52:31.647910+0000","last_fullsized":"2024-09-21T06:52:31.647910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645383+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645383+0000","last_peered":"2024-09-21T06:52:31.645383+0000","last_clean":"2024-09-21T06:52:31.645383+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645383+0000","last_undegraded":"2024-09-21T06:52:31.645383+0000","last_fullsized":"2024-09-21T06:52:31.645383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700086+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700086+0000","last_peered":"2024-09-21T06:52:31.700086+0000","last_clean":"2024-09-21T06:52:31.700086+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700086+0000","last_undegraded":"2024-09-21T06:52:31.700086+0000","last_fullsized":"2024-09-21T06:52:31.700086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892183+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:31.892183+0000","last_peered":"2024-09-21T06:52:31.892183+0000","last_clean":"2024-09-21T06:52:31.892183+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:31.892183+0000","last_undegraded":"2024-09-21T06:52:31.892183+0000","last_fullsized":"2024-09-21T06:52:31.892183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703785+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.703785+0000","last_peered":"2024-09-21T06:52:31.703785+0000","last_clean":"2024-09-21T06:52:31.703785+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.703785+0000","last_undegraded":"2024-09-21T06:52:31.703785+0000","last_fullsized":"2024-09-21T06:52:31.703785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699906+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699906+0000","last_peered":"2024-09-21T06:52:31.699906+0000","last_clean":"2024-09-21T06:52:31.699906+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699906+0000","last_undegraded":"2024-09-21T06:52:31.699906+0000","last_fullsized":"2024-09-21T06:52:31.699906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645385+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:31.645385+0000","last_peered":"2024-09-21T06:52:31.645385+0000","last_clean":"2024-09-21T06:52:31.645385+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:31.645385+0000","last_undegraded":"2024-09-21T06:52:31.645385+0000","last_fullsized":"2024-09-21T06:52:31.645385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892881+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892881+0000","last_peered":"2024-09-21T06:52:31.892881+0000","last_clean":"2024-09-21T06:52:31.892881+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892881+0000","last_undegraded":"2024-09-21T06:52:31.892881+0000","last_fullsized":"2024-09-21T06:52:31.892881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646433+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646433+0000","last_peered":"2024-09-21T06:52:31.646433+0000","last_clean":"2024-09-21T06:52:31.646433+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646433+0000","last_undegraded":"2024-09-21T06:52:31.646433+0000","last_fullsized":"2024-09-21T06:52:31.646433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891409+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:31.891409+0000","last_peered":"2024-09-21T06:52:31.891409+0000","last_clean":"2024-09-21T06:52:31.891409+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:31.891409+0000","last_undegraded":"2024-09-21T06:52:31.891409+0000","last_fullsized":"2024-09-21T06:52:31.891409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836806+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836806+0000","last_peered":"2024-09-21T06:52:31.836806+0000","last_clean":"2024-09-21T06:52:31.836806+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836806+0000","last_undegraded":"2024-09-21T06:52:31.836806+0000","last_fullsized":"2024-09-21T06:52:31.836806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700100+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700100+0000","last_peered":"2024-09-21T06:52:31.700100+0000","last_clean":"2024-09-21T06:52:31.700100+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700100+0000","last_undegraded":"2024-09-21T06:52:31.700100+0000","last_fullsized":"2024-09-21T06:52:31.700100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892339+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:31.892339+0000","last_peered":"2024-09-21T06:52:31.892339+0000","last_clean":"2024-09-21T06:52:31.892339+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:31.892339+0000","last_undegraded":"2024-09-21T06:52:31.892339+0000","last_fullsized":"2024-09-21T06:52:31.892339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700333+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700333+0000","last_peered":"2024-09-21T06:52:31.700333+0000","last_clean":"2024-09-21T06:52:31.700333+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700333+0000","last_undegraded":"2024-09-21T06:52:31.700333+0000","last_fullsized":"2024-09-21T06:52:31.700333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703798+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703798+0000","last_peered":"2024-09-21T06:52:31.703798+0000","last_clean":"2024-09-21T06:52:31.703798+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703798+0000","last_undegraded":"2024-09-21T06:52:31.703798+0000","last_fullsized":"2024-09-21T06:52:31.703798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892672+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:31.892672+0000","last_peered":"2024-09-21T06:52:31.892672+0000","last_clean":"2024-09-21T06:52:31.892672+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:31.892672+0000","last_undegraded":"2024-09-21T06:52:31.892672+0000","last_fullsized":"2024-09-21T06:52:31.892672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646318+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:52:31.646318+0000","last_peered":"2024-09-21T06:52:31.646318+0000","last_clean":"2024-09-21T06:52:31.646318+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:52:31.646318+0000","last_undegraded":"2024-09-21T06:52:31.646318+0000","last_fullsized":"2024-09-21T06:52:31.646318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891855+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891855+0000","last_peered":"2024-09-21T06:52:31.891855+0000","last_clean":"2024-09-21T06:52:31.891855+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891855+0000","last_undegraded":"2024-09-21T06:52:31.891855+0000","last_fullsized":"2024-09-21T06:52:31.891855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701399+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:31.701399+0000","last_peered":"2024-09-21T06:52:31.701399+0000","last_clean":"2024-09-21T06:52:31.701399+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:31.701399+0000","last_undegraded":"2024-09-21T06:52:31.701399+0000","last_fullsized":"2024-09-21T06:52:31.701399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892003+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892003+0000","last_peered":"2024-09-21T06:52:31.892003+0000","last_clean":"2024-09-21T06:52:31.892003+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892003+0000","last_undegraded":"2024-09-21T06:52:31.892003+0000","last_fullsized":"2024-09-21T06:52:31.892003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645472+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645472+0000","last_peered":"2024-09-21T06:52:31.645472+0000","last_clean":"2024-09-21T06:52:31.645472+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645472+0000","last_undegraded":"2024-09-21T06:52:31.645472+0000","last_fullsized":"2024-09-21T06:52:31.645472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701974+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:31.701974+0000","last_peered":"2024-09-21T06:52:31.701974+0000","last_clean":"2024-09-21T06:52:31.701974+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:31.701974+0000","last_undegraded":"2024-09-21T06:52:31.701974+0000","last_fullsized":"2024-09-21T06:52:31.701974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645921+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645921+0000","last_peered":"2024-09-21T06:52:31.645921+0000","last_clean":"2024-09-21T06:52:31.645921+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645921+0000","last_undegraded":"2024-09-21T06:52:31.645921+0000","last_fullsized":"2024-09-21T06:52:31.645921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891445+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891445+0000","last_peered":"2024-09-21T06:52:31.891445+0000","last_clean":"2024-09-21T06:52:31.891445+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891445+0000","last_undegraded":"2024-09-21T06:52:31.891445+0000","last_fullsized":"2024-09-21T06:52:31.891445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645944+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:31.645944+0000","last_peered":"2024-09-21T06:52:31.645944+0000","last_clean":"2024-09-21T06:52:31.645944+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:31.645944+0000","last_undegraded":"2024-09-21T06:52:31.645944+0000","last_fullsized":"2024-09-21T06:52:31.645944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:48.725170+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:52:48.725170+0000","last_peered":"2024-09-21T06:52:48.725170+0000","last_clean":"2024-09-21T06:52:48.725170+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:52:48.725170+0000","last_undegraded":"2024-09-21T06:52:48.725170+0000","last_fullsized":"2024-09-21T06:52:48.725170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701217+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701217+0000","last_peered":"2024-09-21T06:52:31.701217+0000","last_clean":"2024-09-21T06:52:31.701217+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701217+0000","last_undegraded":"2024-09-21T06:52:31.701217+0000","last_fullsized":"2024-09-21T06:52:31.701217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701471+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:31.701471+0000","last_peered":"2024-09-21T06:52:31.701471+0000","last_clean":"2024-09-21T06:52:31.701471+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:31.701471+0000","last_undegraded":"2024-09-21T06:52:31.701471+0000","last_fullsized":"2024-09-21T06:52:31.701471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:45.721396+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:52:45.721396+0000","last_peered":"2024-09-21T06:52:45.721396+0000","last_clean":"2024-09-21T06:52:45.721396+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:52:45.721396+0000","last_undegraded":"2024-09-21T06:52:45.721396+0000","last_fullsized":"2024-09-21T06:52:45.721396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701395+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701395+0000","last_peered":"2024-09-21T06:52:31.701395+0000","last_clean":"2024-09-21T06:52:31.701395+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701395+0000","last_undegraded":"2024-09-21T06:52:31.701395+0000","last_fullsized":"2024-09-21T06:52:31.701395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700554+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:31.700554+0000","last_peered":"2024-09-21T06:52:31.700554+0000","last_clean":"2024-09-21T06:52:31.700554+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:31.700554+0000","last_undegraded":"2024-09-21T06:52:31.700554+0000","last_fullsized":"2024-09-21T06:52:31.700554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701801+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701801+0000","last_peered":"2024-09-21T06:52:31.701801+0000","last_clean":"2024-09-21T06:52:31.701801+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701801+0000","last_undegraded":"2024-09-21T06:52:31.701801+0000","last_fullsized":"2024-09-21T06:52:31.701801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892142+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:31.892142+0000","last_peered":"2024-09-21T06:52:31.892142+0000","last_clean":"2024-09-21T06:52:31.892142+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:31.892142+0000","last_undegraded":"2024-09-21T06:52:31.892142+0000","last_fullsized":"2024-09-21T06:52:31.892142+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699788+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699788+0000","last_peered":"2024-09-21T06:52:31.699788+0000","last_clean":"2024-09-21T06:52:31.699788+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699788+0000","last_undegraded":"2024-09-21T06:52:31.699788+0000","last_fullsized":"2024-09-21T06:52:31.699788+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648426+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648426+0000","last_peered":"2024-09-21T06:52:31.648426+0000","last_clean":"2024-09-21T06:52:31.648426+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648426+0000","last_undegraded":"2024-09-21T06:52:31.648426+0000","last_fullsized":"2024-09-21T06:52:31.648426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648654+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648654+0000","last_peered":"2024-09-21T06:52:31.648654+0000","last_clean":"2024-09-21T06:52:31.648654+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648654+0000","last_undegraded":"2024-09-21T06:52:31.648654+0000","last_fullsized":"2024-09-21T06:52:31.648654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648955+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:31.648955+0000","last_peered":"2024-09-21T06:52:31.648955+0000","last_clean":"2024-09-21T06:52:31.648955+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:31.648955+0000","last_undegraded":"2024-09-21T06:52:31.648955+0000","last_fullsized":"2024-09-21T06:52:31.648955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:45.722834+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:52:45.722834+0000","last_peered":"2024-09-21T06:52:45.722834+0000","last_clean":"2024-09-21T06:52:45.722834+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:52:45.722834+0000","last_undegraded":"2024-09-21T06:52:45.722834+0000","last_fullsized":"2024-09-21T06:52:45.722834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836703+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836703+0000","last_peered":"2024-09-21T06:52:31.836703+0000","last_clean":"2024-09-21T06:52:31.836703+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836703+0000","last_undegraded":"2024-09-21T06:52:31.836703+0000","last_fullsized":"2024-09-21T06:52:31.836703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648636+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:31.648636+0000","last_peered":"2024-09-21T06:52:31.648636+0000","last_clean":"2024-09-21T06:52:31.648636+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:31.648636+0000","last_undegraded":"2024-09-21T06:52:31.648636+0000","last_fullsized":"2024-09-21T06:52:31.648636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702276+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:52:31.702276+0000","last_peered":"2024-09-21T06:52:31.702276+0000","last_clean":"2024-09-21T06:52:31.702276+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:52:31.702276+0000","last_undegraded":"2024-09-21T06:52:31.702276+0000","last_fullsized":"2024-09-21T06:52:31.702276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891809+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891809+0000","last_peered":"2024-09-21T06:52:31.891809+0000","last_clean":"2024-09-21T06:52:31.891809+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891809+0000","last_undegraded":"2024-09-21T06:52:31.891809+0000","last_fullsized":"2024-09-21T06:52:31.891809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648799+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:31.648799+0000","last_peered":"2024-09-21T06:52:31.648799+0000","last_clean":"2024-09-21T06:52:31.648799+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:31.648799+0000","last_undegraded":"2024-09-21T06:52:31.648799+0000","last_fullsized":"2024-09-21T06:52:31.648799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701582+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701582+0000","last_peered":"2024-09-21T06:52:31.701582+0000","last_clean":"2024-09-21T06:52:31.701582+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701582+0000","last_undegraded":"2024-09-21T06:52:31.701582+0000","last_fullsized":"2024-09-21T06:52:31.701582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701272+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701272+0000","last_peered":"2024-09-21T06:52:31.701272+0000","last_clean":"2024-09-21T06:52:31.701272+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701272+0000","last_undegraded":"2024-09-21T06:52:31.701272+0000","last_fullsized":"2024-09-21T06:52:31.701272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645790+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:31.645790+0000","last_peered":"2024-09-21T06:52:31.645790+0000","last_clean":"2024-09-21T06:52:31.645790+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:31.645790+0000","last_undegraded":"2024-09-21T06:52:31.645790+0000","last_fullsized":"2024-09-21T06:52:31.645790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645903+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645903+0000","last_peered":"2024-09-21T06:52:31.645903+0000","last_clean":"2024-09-21T06:52:31.645903+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645903+0000","last_undegraded":"2024-09-21T06:52:31.645903+0000","last_fullsized":"2024-09-21T06:52:31.645903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701111+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701111+0000","last_peered":"2024-09-21T06:52:31.701111+0000","last_clean":"2024-09-21T06:52:31.701111+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701111+0000","last_undegraded":"2024-09-21T06:52:31.701111+0000","last_fullsized":"2024-09-21T06:52:31.701111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.838673+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:31.838673+0000","last_peered":"2024-09-21T06:52:31.838673+0000","last_clean":"2024-09-21T06:52:31.838673+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:31.838673+0000","last_undegraded":"2024-09-21T06:52:31.838673+0000","last_fullsized":"2024-09-21T06:52:31.838673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891438+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891438+0000","last_peered":"2024-09-21T06:52:31.891438+0000","last_clean":"2024-09-21T06:52:31.891438+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891438+0000","last_undegraded":"2024-09-21T06:52:31.891438+0000","last_fullsized":"2024-09-21T06:52:31.891438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701495+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701495+0000","last_peered":"2024-09-21T06:52:31.701495+0000","last_clean":"2024-09-21T06:52:31.701495+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701495+0000","last_undegraded":"2024-09-21T06:52:31.701495+0000","last_fullsized":"2024-09-21T06:52:31.701495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.649831+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:31.649831+0000","last_peered":"2024-09-21T06:52:31.649831+0000","last_clean":"2024-09-21T06:52:31.649831+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:31.649831+0000","last_undegraded":"2024-09-21T06:52:31.649831+0000","last_fullsized":"2024-09-21T06:52:31.649831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:44.720564+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:52:44.720564+0000","last_peered":"2024-09-21T06:52:44.720564+0000","last_clean":"2024-09-21T06:52:44.720564+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:52:44.720564+0000","last_undegraded":"2024-09-21T06:52:44.720564+0000","last_fullsized":"2024-09-21T06:52:44.720564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891921+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891921+0000","last_peered":"2024-09-21T06:52:31.891921+0000","last_clean":"2024-09-21T06:52:31.891921+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891921+0000","last_undegraded":"2024-09-21T06:52:31.891921+0000","last_fullsized":"2024-09-21T06:52:31.891921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648303+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:31.648303+0000","last_peered":"2024-09-21T06:52:31.648303+0000","last_clean":"2024-09-21T06:52:31.648303+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:31.648303+0000","last_undegraded":"2024-09-21T06:52:31.648303+0000","last_fullsized":"2024-09-21T06:52:31.648303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701337+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701337+0000","last_peered":"2024-09-21T06:52:31.701337+0000","last_clean":"2024-09-21T06:52:31.701337+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701337+0000","last_undegraded":"2024-09-21T06:52:31.701337+0000","last_fullsized":"2024-09-21T06:52:31.701337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645436+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645436+0000","last_peered":"2024-09-21T06:52:31.645436+0000","last_clean":"2024-09-21T06:52:31.645436+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645436+0000","last_undegraded":"2024-09-21T06:52:31.645436+0000","last_fullsized":"2024-09-21T06:52:31.645436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700887+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:31.700887+0000","last_peered":"2024-09-21T06:52:31.700887+0000","last_clean":"2024-09-21T06:52:31.700887+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:31.700887+0000","last_undegraded":"2024-09-21T06:52:31.700887+0000","last_fullsized":"2024-09-21T06:52:31.700887+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703771+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703771+0000","last_peered":"2024-09-21T06:52:31.703771+0000","last_clean":"2024-09-21T06:52:31.703771+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703771+0000","last_undegraded":"2024-09-21T06:52:31.703771+0000","last_fullsized":"2024-09-21T06:52:31.703771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701998+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.701998+0000","last_peered":"2024-09-21T06:52:31.701998+0000","last_clean":"2024-09-21T06:52:31.701998+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.701998+0000","last_undegraded":"2024-09-21T06:52:31.701998+0000","last_fullsized":"2024-09-21T06:52:31.701998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891820+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:31.891820+0000","last_peered":"2024-09-21T06:52:31.891820+0000","last_clean":"2024-09-21T06:52:31.891820+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:31.891820+0000","last_undegraded":"2024-09-21T06:52:31.891820+0000","last_fullsized":"2024-09-21T06:52:31.891820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":207,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:43.719070+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:52:43.719070+0000","last_peered":"2024-09-21T06:52:43.719070+0000","last_clean":"2024-09-21T06:52:43.719070+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:52:43.719070+0000","last_undegraded":"2024-09-21T06:52:43.719070+0000","last_fullsized":"2024-09-21T06:52:43.719070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67067,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":131,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645656+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645656+0000","last_peered":"2024-09-21T06:52:31.645656+0000","last_clean":"2024-09-21T06:52:31.645656+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645656+0000","last_undegraded":"2024-09-21T06:52:31.645656+0000","last_fullsized":"2024-09-21T06:52:31.645656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837113+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:31.837113+0000","last_peered":"2024-09-21T06:52:31.837113+0000","last_clean":"2024-09-21T06:52:31.837113+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:31.837113+0000","last_undegraded":"2024-09-21T06:52:31.837113+0000","last_fullsized":"2024-09-21T06:52:31.837113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702253+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702253+0000","last_peered":"2024-09-21T06:52:31.702253+0000","last_clean":"2024-09-21T06:52:31.702253+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702253+0000","last_undegraded":"2024-09-21T06:52:31.702253+0000","last_fullsized":"2024-09-21T06:52:31.702253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699437+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699437+0000","last_peered":"2024-09-21T06:52:31.699437+0000","last_clean":"2024-09-21T06:52:31.699437+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699437+0000","last_undegraded":"2024-09-21T06:52:31.699437+0000","last_fullsized":"2024-09-21T06:52:31.699437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892920+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:31.892920+0000","last_peered":"2024-09-21T06:52:31.892920+0000","last_clean":"2024-09-21T06:52:31.892920+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:31.892920+0000","last_undegraded":"2024-09-21T06:52:31.892920+0000","last_fullsized":"2024-09-21T06:52:31.892920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648834+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648834+0000","last_peered":"2024-09-21T06:52:31.648834+0000","last_clean":"2024-09-21T06:52:31.648834+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648834+0000","last_undegraded":"2024-09-21T06:52:31.648834+0000","last_fullsized":"2024-09-21T06:52:31.648834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645497+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645497+0000","last_peered":"2024-09-21T06:52:31.645497+0000","last_clean":"2024-09-21T06:52:31.645497+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645497+0000","last_undegraded":"2024-09-21T06:52:31.645497+0000","last_fullsized":"2024-09-21T06:52:31.645497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837389+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:31.837389+0000","last_peered":"2024-09-21T06:52:31.837389+0000","last_clean":"2024-09-21T06:52:31.837389+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:31.837389+0000","last_undegraded":"2024-09-21T06:52:31.837389+0000","last_fullsized":"2024-09-21T06:52:31.837389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":168,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:43.718826+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:52:43.718826+0000","last_peered":"2024-09-21T06:52:43.718826+0000","last_clean":"2024-09-21T06:52:43.718826+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:52:43.718826+0000","last_undegraded":"2024-09-21T06:52:43.718826+0000","last_fullsized":"2024-09-21T06:52:43.718826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892649+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892649+0000","last_peered":"2024-09-21T06:52:31.892649+0000","last_clean":"2024-09-21T06:52:31.892649+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892649+0000","last_undegraded":"2024-09-21T06:52:31.892649+0000","last_fullsized":"2024-09-21T06:52:31.892649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892135+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892135+0000","last_peered":"2024-09-21T06:52:31.892135+0000","last_clean":"2024-09-21T06:52:31.892135+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892135+0000","last_undegraded":"2024-09-21T06:52:31.892135+0000","last_fullsized":"2024-09-21T06:52:31.892135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646289+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:31.646289+0000","last_peered":"2024-09-21T06:52:31.646289+0000","last_clean":"2024-09-21T06:52:31.646289+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:31.646289+0000","last_undegraded":"2024-09-21T06:52:31.646289+0000","last_fullsized":"2024-09-21T06:52:31.646289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701545+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701545+0000","last_peered":"2024-09-21T06:52:31.701545+0000","last_clean":"2024-09-21T06:52:31.701545+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701545+0000","last_undegraded":"2024-09-21T06:52:31.701545+0000","last_fullsized":"2024-09-21T06:52:31.701545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645757+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645757+0000","last_peered":"2024-09-21T06:52:31.645757+0000","last_clean":"2024-09-21T06:52:31.645757+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645757+0000","last_undegraded":"2024-09-21T06:52:31.645757+0000","last_fullsized":"2024-09-21T06:52:31.645757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892107+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:31.892107+0000","last_peered":"2024-09-21T06:52:31.892107+0000","last_clean":"2024-09-21T06:52:31.892107+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:31.892107+0000","last_undegraded":"2024-09-21T06:52:31.892107+0000","last_fullsized":"2024-09-21T06:52:31.892107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700649+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700649+0000","last_peered":"2024-09-21T06:52:31.700649+0000","last_clean":"2024-09-21T06:52:31.700649+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700649+0000","last_undegraded":"2024-09-21T06:52:31.700649+0000","last_fullsized":"2024-09-21T06:52:31.700649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645893+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645893+0000","last_peered":"2024-09-21T06:52:31.645893+0000","last_clean":"2024-09-21T06:52:31.645893+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645893+0000","last_undegraded":"2024-09-21T06:52:31.645893+0000","last_fullsized":"2024-09-21T06:52:31.645893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701357+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:31.701357+0000","last_peered":"2024-09-21T06:52:31.701357+0000","last_clean":"2024-09-21T06:52:31.701357+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:31.701357+0000","last_undegraded":"2024-09-21T06:52:31.701357+0000","last_fullsized":"2024-09-21T06:52:31.701357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703769+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703769+0000","last_peered":"2024-09-21T06:52:31.703769+0000","last_clean":"2024-09-21T06:52:31.703769+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703769+0000","last_undegraded":"2024-09-21T06:52:31.703769+0000","last_fullsized":"2024-09-21T06:52:31.703769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:44.721239+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:52:44.721239+0000","last_peered":"2024-09-21T06:52:44.721239+0000","last_clean":"2024-09-21T06:52:44.721239+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:52:44.721239+0000","last_undegraded":"2024-09-21T06:52:44.721239+0000","last_fullsized":"2024-09-21T06:52:44.721239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891607+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:31.891607+0000","last_peered":"2024-09-21T06:52:31.891607+0000","last_clean":"2024-09-21T06:52:31.891607+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:31.891607+0000","last_undegraded":"2024-09-21T06:52:31.891607+0000","last_fullsized":"2024-09-21T06:52:31.891607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703996+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.703996+0000","last_peered":"2024-09-21T06:52:31.703996+0000","last_clean":"2024-09-21T06:52:31.703996+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.703996+0000","last_undegraded":"2024-09-21T06:52:31.703996+0000","last_fullsized":"2024-09-21T06:52:31.703996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645287+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:31.645287+0000","last_peered":"2024-09-21T06:52:31.645287+0000","last_clean":"2024-09-21T06:52:31.645287+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:31.645287+0000","last_undegraded":"2024-09-21T06:52:31.645287+0000","last_fullsized":"2024-09-21T06:52:31.645287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703972+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:31.703972+0000","last_peered":"2024-09-21T06:52:31.703972+0000","last_clean":"2024-09-21T06:52:31.703972+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:31.703972+0000","last_undegraded":"2024-09-21T06:52:31.703972+0000","last_fullsized":"2024-09-21T06:52:31.703972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703691+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:31.703691+0000","last_peered":"2024-09-21T06:52:31.703691+0000","last_clean":"2024-09-21T06:52:31.703691+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:31.703691+0000","last_undegraded":"2024-09-21T06:52:31.703691+0000","last_fullsized":"2024-09-21T06:52:31.703691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891918+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:31.891918+0000","last_peered":"2024-09-21T06:52:31.891918+0000","last_clean":"2024-09-21T06:52:31.891918+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:31.891918+0000","last_undegraded":"2024-09-21T06:52:31.891918+0000","last_fullsized":"2024-09-21T06:52:31.891918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701596+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:31.701596+0000","last_peered":"2024-09-21T06:52:31.701596+0000","last_clean":"2024-09-21T06:52:31.701596+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:31.701596+0000","last_undegraded":"2024-09-21T06:52:31.701596+0000","last_fullsized":"2024-09-21T06:52:31.701596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645796+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:31.645796+0000","last_peered":"2024-09-21T06:52:31.645796+0000","last_clean":"2024-09-21T06:52:31.645796+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:31.645796+0000","last_undegraded":"2024-09-21T06:52:31.645796+0000","last_fullsized":"2024-09-21T06:52:31.645796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704275+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:31.704275+0000","last_peered":"2024-09-21T06:52:31.704275+0000","last_clean":"2024-09-21T06:52:31.704275+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:31.704275+0000","last_undegraded":"2024-09-21T06:52:31.704275+0000","last_fullsized":"2024-09-21T06:52:31.704275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892641+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:31.892641+0000","last_peered":"2024-09-21T06:52:31.892641+0000","last_clean":"2024-09-21T06:52:31.892641+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:31.892641+0000","last_undegraded":"2024-09-21T06:52:31.892641+0000","last_fullsized":"2024-09-21T06:52:31.892641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701411+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:31.701411+0000","last_peered":"2024-09-21T06:52:31.701411+0000","last_clean":"2024-09-21T06:52:31.701411+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:31.701411+0000","last_undegraded":"2024-09-21T06:52:31.701411+0000","last_fullsized":"2024-09-21T06:52:31.701411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892776+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:31.892776+0000","last_peered":"2024-09-21T06:52:31.892776+0000","last_clean":"2024-09-21T06:52:31.892776+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:31.892776+0000","last_undegraded":"2024-09-21T06:52:31.892776+0000","last_fullsized":"2024-09-21T06:52:31.892776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836962+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:31.836962+0000","last_peered":"2024-09-21T06:52:31.836962+0000","last_clean":"2024-09-21T06:52:31.836962+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:31.836962+0000","last_undegraded":"2024-09-21T06:52:31.836962+0000","last_fullsized":"2024-09-21T06:52:31.836962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701208+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:31.701208+0000","last_peered":"2024-09-21T06:52:31.701208+0000","last_clean":"2024-09-21T06:52:31.701208+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:31.701208+0000","last_undegraded":"2024-09-21T06:52:31.701208+0000","last_fullsized":"2024-09-21T06:52:31.701208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645509+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:31.645509+0000","last_peered":"2024-09-21T06:52:31.645509+0000","last_clean":"2024-09-21T06:52:31.645509+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:31.645509+0000","last_undegraded":"2024-09-21T06:52:31.645509+0000","last_fullsized":"2024-09-21T06:52:31.645509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704175+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:31.704175+0000","last_peered":"2024-09-21T06:52:31.704175+0000","last_clean":"2024-09-21T06:52:31.704175+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:31.704175+0000","last_undegraded":"2024-09-21T06:52:31.704175+0000","last_fullsized":"2024-09-21T06:52:31.704175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646280+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:31.646280+0000","last_peered":"2024-09-21T06:52:31.646280+0000","last_clean":"2024-09-21T06:52:31.646280+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:31.646280+0000","last_undegraded":"2024-09-21T06:52:31.646280+0000","last_fullsized":"2024-09-21T06:52:31.646280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646011+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:31.646011+0000","last_peered":"2024-09-21T06:52:31.646011+0000","last_clean":"2024-09-21T06:52:31.646011+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:31.646011+0000","last_undegraded":"2024-09-21T06:52:31.646011+0000","last_fullsized":"2024-09-21T06:52:31.646011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836631+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:31.836631+0000","last_peered":"2024-09-21T06:52:31.836631+0000","last_clean":"2024-09-21T06:52:31.836631+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:31.836631+0000","last_undegraded":"2024-09-21T06:52:31.836631+0000","last_fullsized":"2024-09-21T06:52:31.836631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891475+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:31.891475+0000","last_peered":"2024-09-21T06:52:31.891475+0000","last_clean":"2024-09-21T06:52:31.891475+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:31.891475+0000","last_undegraded":"2024-09-21T06:52:31.891475+0000","last_fullsized":"2024-09-21T06:52:31.891475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891521+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:31.891521+0000","last_peered":"2024-09-21T06:52:31.891521+0000","last_clean":"2024-09-21T06:52:31.891521+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:31.891521+0000","last_undegraded":"2024-09-21T06:52:31.891521+0000","last_fullsized":"2024-09-21T06:52:31.891521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701074+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:31.701074+0000","last_peered":"2024-09-21T06:52:31.701074+0000","last_clean":"2024-09-21T06:52:31.701074+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:31.701074+0000","last_undegraded":"2024-09-21T06:52:31.701074+0000","last_fullsized":"2024-09-21T06:52:31.701074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700052+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:31.700052+0000","last_peered":"2024-09-21T06:52:31.700052+0000","last_clean":"2024-09-21T06:52:31.700052+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:31.700052+0000","last_undegraded":"2024-09-21T06:52:31.700052+0000","last_fullsized":"2024-09-21T06:52:31.700052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648794+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:31.648794+0000","last_peered":"2024-09-21T06:52:31.648794+0000","last_clean":"2024-09-21T06:52:31.648794+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:31.648794+0000","last_undegraded":"2024-09-21T06:52:31.648794+0000","last_fullsized":"2024-09-21T06:52:31.648794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646996+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:31.646996+0000","last_peered":"2024-09-21T06:52:31.646996+0000","last_clean":"2024-09-21T06:52:31.646996+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:31.646996+0000","last_undegraded":"2024-09-21T06:52:31.646996+0000","last_fullsized":"2024-09-21T06:52:31.646996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837517+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:31.837517+0000","last_peered":"2024-09-21T06:52:31.837517+0000","last_clean":"2024-09-21T06:52:31.837517+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:31.837517+0000","last_undegraded":"2024-09-21T06:52:31.837517+0000","last_fullsized":"2024-09-21T06:52:31.837517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891996+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:31.891996+0000","last_peered":"2024-09-21T06:52:31.891996+0000","last_clean":"2024-09-21T06:52:31.891996+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:31.891996+0000","last_undegraded":"2024-09-21T06:52:31.891996+0000","last_fullsized":"2024-09-21T06:52:31.891996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.647741+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:31.647741+0000","last_peered":"2024-09-21T06:52:31.647741+0000","last_clean":"2024-09-21T06:52:31.647741+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:31.647741+0000","last_undegraded":"2024-09-21T06:52:31.647741+0000","last_fullsized":"2024-09-21T06:52:31.647741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.705828+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:31.705828+0000","last_peered":"2024-09-21T06:52:31.705828+0000","last_clean":"2024-09-21T06:52:31.705828+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:31.705828+0000","last_undegraded":"2024-09-21T06:52:31.705828+0000","last_fullsized":"2024-09-21T06:52:31.705828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891528+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:31.891528+0000","last_peered":"2024-09-21T06:52:31.891528+0000","last_clean":"2024-09-21T06:52:31.891528+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:31.891528+0000","last_undegraded":"2024-09-21T06:52:31.891528+0000","last_fullsized":"2024-09-21T06:52:31.891528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700010+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:31.700010+0000","last_peered":"2024-09-21T06:52:31.700010+0000","last_clean":"2024-09-21T06:52:31.700010+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:31.700010+0000","last_undegraded":"2024-09-21T06:52:31.700010+0000","last_fullsized":"2024-09-21T06:52:31.700010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892360+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:31.892360+0000","last_peered":"2024-09-21T06:52:31.892360+0000","last_clean":"2024-09-21T06:52:31.892360+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:31.892360+0000","last_undegraded":"2024-09-21T06:52:31.892360+0000","last_fullsized":"2024-09-21T06:52:31.892360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892241+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:31.892241+0000","last_peered":"2024-09-21T06:52:31.892241+0000","last_clean":"2024-09-21T06:52:31.892241+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:31.892241+0000","last_undegraded":"2024-09-21T06:52:31.892241+0000","last_fullsized":"2024-09-21T06:52:31.892241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645985+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:31.645985+0000","last_peered":"2024-09-21T06:52:31.645985+0000","last_clean":"2024-09-21T06:52:31.645985+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:31.645985+0000","last_undegraded":"2024-09-21T06:52:31.645985+0000","last_fullsized":"2024-09-21T06:52:31.645985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74662,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":669,"num_read_kb":414,"num_write":250,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1040384,"data_stored":531658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1736704,"data_stored":1185128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":535271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":531531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1654784,"data_stored":1128441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":527070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822728,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1662976,"data_stored":1123632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":526387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378714,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1122304,"data_stored":594765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542238,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1118208,"data_stored":596180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705764,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":528526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:52:50.577 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:52:50.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:50 smithi114 ceph-mon[30654]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:50 smithi183 ceph-mon[34219]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:50 smithi193 ceph-mon[34601]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:50.964 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.588+0000 7f3812a00640 1 -- 172.21.15.114:0/941880686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 msgr2=0x7f380c0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.588+0000 7f3812a00640 1 --2- 172.21.15.114:0/941880686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c0761e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009900 tx=0x7f37f802a410 comp rx=0 tx=0).stop 2024-09-21T06:52:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 -- 172.21.15.114:0/941880686 shutdown_connections 2024-09-21T06:52:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 --2- 172.21.15.114:0/941880686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 0x7f380c116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 --2- 172.21.15.114:0/941880686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c0761e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 --2- 172.21.15.114:0/941880686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f380c077380 0x7f380c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 -- 172.21.15.114:0/941880686 >> 172.21.15.114:0/941880686 conn(0x7f380c0ff440 msgr2=0x7f380c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 -- 172.21.15.114:0/941880686 shutdown_connections 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.589+0000 7f3812a00640 1 -- 172.21.15.114:0/941880686 wait complete. 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 Processor -- start 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 -- start start 2024-09-21T06:52:51.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f380c077380 0x7f380c1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 0x7f380c1a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f380c119ec0 con 0x7f380c075dd0 2024-09-21T06:52:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f380c119d40 con 0x7f380c114280 2024-09-21T06:52:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.590+0000 7f3812a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f380c11a040 con 0x7f380c077380 2024-09-21T06:52:51.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:51.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c1a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50460/0 (socket says 172.21.15.114:50460) 2024-09-21T06:52:51.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 -- 172.21.15.114:0/3561077002 learned_addr learned my addr 172.21.15.114:0/3561077002 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:51.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3810c00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f380c077380 0x7f380c1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:51.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3812000640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 0x7f380c1a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:51.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 -- 172.21.15.114:0/3561077002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f380c077380 msgr2=0x7f380c1a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:51.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f380c077380 0x7f380c1a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 -- 172.21.15.114:0/3561077002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 msgr2=0x7f380c1a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 0x7f380c1a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 -- 172.21.15.114:0/3561077002 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c1a6920 con 0x7f380c075dd0 2024-09-21T06:52:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3812000640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 0x7f380c1a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:52:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.591+0000 7f3811600640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c1a09b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f380000e9f0 tx=0x7f380000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:51.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.592+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380000cd80 con 0x7f380c075dd0 2024-09-21T06:52:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.592+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38000054d0 con 0x7f380c075dd0 2024-09-21T06:52:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.592+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800010690 con 0x7f380c075dd0 2024-09-21T06:52:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.592+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c1a6c10 con 0x7f380c075dd0 2024-09-21T06:52:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.592+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1ad370 con 0x7f380c075dd0 2024-09-21T06:52:51.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.593+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c0761e0 con 0x7f380c075dd0 2024-09-21T06:52:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.595+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f3800004e90 con 0x7f380c075dd0 2024-09-21T06:52:51.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.597+0000 7f380a000640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37f40b65f0 0x7f37f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.597+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f38000e7e70 con 0x7f380c075dd0 2024-09-21T06:52:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.597+0000 7f3810c00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37f40b65f0 0x7f37f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:51.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.598+0000 7f3810c00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37f40b65f0 0x7f37f40b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f37f8000c00 tx=0x7f37f8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:51.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.601+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38000ae0b0 con 0x7f380c075dd0 2024-09-21T06:52:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.789+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f380c063800 con 0x7f37f40b65f0 2024-09-21T06:52:51.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:51 smithi114 ceph-mon[30654]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:51.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:52:51.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:52:51.792 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.795+0000 7f380a000640 1 -- 172.21.15.114:0/3561077002 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340392 (secure 0 0 0) 0x7f380c063800 con 0x7f37f40b65f0 2024-09-21T06:52:51.801 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:51.803 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.800+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37f40b65f0 msgr2=0x7f37f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.800+0000 7f3812a00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37f40b65f0 0x7f37f40b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f37f8000c00 tx=0x7f37f8031400 comp rx=0 tx=0).stop 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.800+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 msgr2=0x7f380c1a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.800+0000 7f3812a00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c1a09b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f380000e9f0 tx=0x7f380000edf0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 shutdown_connections 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37f40b65f0 0x7f37f40b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380c114280 0x7f380c1a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f380c077380 0x7f380c1a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 --2- 172.21.15.114:0/3561077002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f380c075dd0 0x7f380c1a09b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:51.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 >> 172.21.15.114:0/3561077002 conn(0x7f380c0ff440 msgr2=0x7f380c110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:51.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 shutdown_connections 2024-09-21T06:52:51.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:51.801+0000 7f3812a00640 1 -- 172.21.15.114:0/3561077002 wait complete. 2024-09-21T06:52:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:51 smithi183 ceph-mon[34219]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:52:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:52:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:51 smithi193 ceph-mon[34601]: from='client.25157 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:52:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:52:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:52:52.646 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-21T06:52:51.522977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665264,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":775,"num_read_kb":498,"num_write":375,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371960,"kb_used_data":14300,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569864,"statfs":{"total":1151940427776,"available":1151559540736,"internally_reserved":0,"allocated":14643200,"data_stored":8339342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003566"},"pg_stats":[{"pgid":"2.1f","version":"87'13","reported_seq":121,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:45.720814+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:52:45.720814+0000","last_peered":"2024-09-21T06:52:45.720814+0000","last_clean":"2024-09-21T06:52:45.720814+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:52:45.720814+0000","last_undegraded":"2024-09-21T06:52:45.720814+0000","last_fullsized":"2024-09-21T06:52:45.720814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892422+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:31.892422+0000","last_peered":"2024-09-21T06:52:31.892422+0000","last_clean":"2024-09-21T06:52:31.892422+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:31.892422+0000","last_undegraded":"2024-09-21T06:52:31.892422+0000","last_fullsized":"2024-09-21T06:52:31.892422+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703592+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703592+0000","last_peered":"2024-09-21T06:52:31.703592+0000","last_clean":"2024-09-21T06:52:31.703592+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703592+0000","last_undegraded":"2024-09-21T06:52:31.703592+0000","last_fullsized":"2024-09-21T06:52:31.703592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892890+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:31.892890+0000","last_peered":"2024-09-21T06:52:31.892890+0000","last_clean":"2024-09-21T06:52:31.892890+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:31.892890+0000","last_undegraded":"2024-09-21T06:52:31.892890+0000","last_fullsized":"2024-09-21T06:52:31.892890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699187+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699187+0000","last_peered":"2024-09-21T06:52:31.699187+0000","last_clean":"2024-09-21T06:52:31.699187+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699187+0000","last_undegraded":"2024-09-21T06:52:31.699187+0000","last_fullsized":"2024-09-21T06:52:31.699187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891998+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:52:31.891998+0000","last_peered":"2024-09-21T06:52:31.891998+0000","last_clean":"2024-09-21T06:52:31.891998+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:52:31.891998+0000","last_undegraded":"2024-09-21T06:52:31.891998+0000","last_fullsized":"2024-09-21T06:52:31.891998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837017+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:31.837017+0000","last_peered":"2024-09-21T06:52:31.837017+0000","last_clean":"2024-09-21T06:52:31.837017+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:31.837017+0000","last_undegraded":"2024-09-21T06:52:31.837017+0000","last_fullsized":"2024-09-21T06:52:31.837017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699576+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699576+0000","last_peered":"2024-09-21T06:52:31.699576+0000","last_clean":"2024-09-21T06:52:31.699576+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699576+0000","last_undegraded":"2024-09-21T06:52:31.699576+0000","last_fullsized":"2024-09-21T06:52:31.699576+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891874+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891874+0000","last_peered":"2024-09-21T06:52:31.891874+0000","last_clean":"2024-09-21T06:52:31.891874+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891874+0000","last_undegraded":"2024-09-21T06:52:31.891874+0000","last_fullsized":"2024-09-21T06:52:31.891874+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702135+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:31.702135+0000","last_peered":"2024-09-21T06:52:31.702135+0000","last_clean":"2024-09-21T06:52:31.702135+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:31.702135+0000","last_undegraded":"2024-09-21T06:52:31.702135+0000","last_fullsized":"2024-09-21T06:52:31.702135+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699290+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699290+0000","last_peered":"2024-09-21T06:52:31.699290+0000","last_clean":"2024-09-21T06:52:31.699290+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699290+0000","last_undegraded":"2024-09-21T06:52:31.699290+0000","last_fullsized":"2024-09-21T06:52:31.699290+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.647910+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:31.647910+0000","last_peered":"2024-09-21T06:52:31.647910+0000","last_clean":"2024-09-21T06:52:31.647910+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:31.647910+0000","last_undegraded":"2024-09-21T06:52:31.647910+0000","last_fullsized":"2024-09-21T06:52:31.647910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645383+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645383+0000","last_peered":"2024-09-21T06:52:31.645383+0000","last_clean":"2024-09-21T06:52:31.645383+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645383+0000","last_undegraded":"2024-09-21T06:52:31.645383+0000","last_fullsized":"2024-09-21T06:52:31.645383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700086+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700086+0000","last_peered":"2024-09-21T06:52:31.700086+0000","last_clean":"2024-09-21T06:52:31.700086+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700086+0000","last_undegraded":"2024-09-21T06:52:31.700086+0000","last_fullsized":"2024-09-21T06:52:31.700086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892183+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:31.892183+0000","last_peered":"2024-09-21T06:52:31.892183+0000","last_clean":"2024-09-21T06:52:31.892183+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:31.892183+0000","last_undegraded":"2024-09-21T06:52:31.892183+0000","last_fullsized":"2024-09-21T06:52:31.892183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703785+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.703785+0000","last_peered":"2024-09-21T06:52:31.703785+0000","last_clean":"2024-09-21T06:52:31.703785+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.703785+0000","last_undegraded":"2024-09-21T06:52:31.703785+0000","last_fullsized":"2024-09-21T06:52:31.703785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699906+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699906+0000","last_peered":"2024-09-21T06:52:31.699906+0000","last_clean":"2024-09-21T06:52:31.699906+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699906+0000","last_undegraded":"2024-09-21T06:52:31.699906+0000","last_fullsized":"2024-09-21T06:52:31.699906+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645385+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:31.645385+0000","last_peered":"2024-09-21T06:52:31.645385+0000","last_clean":"2024-09-21T06:52:31.645385+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:31.645385+0000","last_undegraded":"2024-09-21T06:52:31.645385+0000","last_fullsized":"2024-09-21T06:52:31.645385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892881+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892881+0000","last_peered":"2024-09-21T06:52:31.892881+0000","last_clean":"2024-09-21T06:52:31.892881+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892881+0000","last_undegraded":"2024-09-21T06:52:31.892881+0000","last_fullsized":"2024-09-21T06:52:31.892881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646433+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646433+0000","last_peered":"2024-09-21T06:52:31.646433+0000","last_clean":"2024-09-21T06:52:31.646433+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646433+0000","last_undegraded":"2024-09-21T06:52:31.646433+0000","last_fullsized":"2024-09-21T06:52:31.646433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891409+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:31.891409+0000","last_peered":"2024-09-21T06:52:31.891409+0000","last_clean":"2024-09-21T06:52:31.891409+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:31.891409+0000","last_undegraded":"2024-09-21T06:52:31.891409+0000","last_fullsized":"2024-09-21T06:52:31.891409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836806+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836806+0000","last_peered":"2024-09-21T06:52:31.836806+0000","last_clean":"2024-09-21T06:52:31.836806+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836806+0000","last_undegraded":"2024-09-21T06:52:31.836806+0000","last_fullsized":"2024-09-21T06:52:31.836806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700100+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700100+0000","last_peered":"2024-09-21T06:52:31.700100+0000","last_clean":"2024-09-21T06:52:31.700100+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700100+0000","last_undegraded":"2024-09-21T06:52:31.700100+0000","last_fullsized":"2024-09-21T06:52:31.700100+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892339+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:31.892339+0000","last_peered":"2024-09-21T06:52:31.892339+0000","last_clean":"2024-09-21T06:52:31.892339+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:31.892339+0000","last_undegraded":"2024-09-21T06:52:31.892339+0000","last_fullsized":"2024-09-21T06:52:31.892339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700333+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700333+0000","last_peered":"2024-09-21T06:52:31.700333+0000","last_clean":"2024-09-21T06:52:31.700333+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700333+0000","last_undegraded":"2024-09-21T06:52:31.700333+0000","last_fullsized":"2024-09-21T06:52:31.700333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703798+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703798+0000","last_peered":"2024-09-21T06:52:31.703798+0000","last_clean":"2024-09-21T06:52:31.703798+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703798+0000","last_undegraded":"2024-09-21T06:52:31.703798+0000","last_fullsized":"2024-09-21T06:52:31.703798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892672+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:31.892672+0000","last_peered":"2024-09-21T06:52:31.892672+0000","last_clean":"2024-09-21T06:52:31.892672+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:31.892672+0000","last_undegraded":"2024-09-21T06:52:31.892672+0000","last_fullsized":"2024-09-21T06:52:31.892672+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'15","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646318+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:52:31.646318+0000","last_peered":"2024-09-21T06:52:31.646318+0000","last_clean":"2024-09-21T06:52:31.646318+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:52:31.646318+0000","last_undegraded":"2024-09-21T06:52:31.646318+0000","last_fullsized":"2024-09-21T06:52:31.646318+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891855+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891855+0000","last_peered":"2024-09-21T06:52:31.891855+0000","last_clean":"2024-09-21T06:52:31.891855+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891855+0000","last_undegraded":"2024-09-21T06:52:31.891855+0000","last_fullsized":"2024-09-21T06:52:31.891855+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701399+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:31.701399+0000","last_peered":"2024-09-21T06:52:31.701399+0000","last_clean":"2024-09-21T06:52:31.701399+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:31.701399+0000","last_undegraded":"2024-09-21T06:52:31.701399+0000","last_fullsized":"2024-09-21T06:52:31.701399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892003+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892003+0000","last_peered":"2024-09-21T06:52:31.892003+0000","last_clean":"2024-09-21T06:52:31.892003+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892003+0000","last_undegraded":"2024-09-21T06:52:31.892003+0000","last_fullsized":"2024-09-21T06:52:31.892003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645472+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645472+0000","last_peered":"2024-09-21T06:52:31.645472+0000","last_clean":"2024-09-21T06:52:31.645472+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645472+0000","last_undegraded":"2024-09-21T06:52:31.645472+0000","last_fullsized":"2024-09-21T06:52:31.645472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701974+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:31.701974+0000","last_peered":"2024-09-21T06:52:31.701974+0000","last_clean":"2024-09-21T06:52:31.701974+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:31.701974+0000","last_undegraded":"2024-09-21T06:52:31.701974+0000","last_fullsized":"2024-09-21T06:52:31.701974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645921+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645921+0000","last_peered":"2024-09-21T06:52:31.645921+0000","last_clean":"2024-09-21T06:52:31.645921+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645921+0000","last_undegraded":"2024-09-21T06:52:31.645921+0000","last_fullsized":"2024-09-21T06:52:31.645921+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891445+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891445+0000","last_peered":"2024-09-21T06:52:31.891445+0000","last_clean":"2024-09-21T06:52:31.891445+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891445+0000","last_undegraded":"2024-09-21T06:52:31.891445+0000","last_fullsized":"2024-09-21T06:52:31.891445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645944+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:31.645944+0000","last_peered":"2024-09-21T06:52:31.645944+0000","last_clean":"2024-09-21T06:52:31.645944+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:31.645944+0000","last_undegraded":"2024-09-21T06:52:31.645944+0000","last_fullsized":"2024-09-21T06:52:31.645944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:48.725170+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:52:48.725170+0000","last_peered":"2024-09-21T06:52:48.725170+0000","last_clean":"2024-09-21T06:52:48.725170+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:52:48.725170+0000","last_undegraded":"2024-09-21T06:52:48.725170+0000","last_fullsized":"2024-09-21T06:52:48.725170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701217+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701217+0000","last_peered":"2024-09-21T06:52:31.701217+0000","last_clean":"2024-09-21T06:52:31.701217+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701217+0000","last_undegraded":"2024-09-21T06:52:31.701217+0000","last_fullsized":"2024-09-21T06:52:31.701217+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701471+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:31.701471+0000","last_peered":"2024-09-21T06:52:31.701471+0000","last_clean":"2024-09-21T06:52:31.701471+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:31.701471+0000","last_undegraded":"2024-09-21T06:52:31.701471+0000","last_fullsized":"2024-09-21T06:52:31.701471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:45.721396+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:52:45.721396+0000","last_peered":"2024-09-21T06:52:45.721396+0000","last_clean":"2024-09-21T06:52:45.721396+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:52:45.721396+0000","last_undegraded":"2024-09-21T06:52:45.721396+0000","last_fullsized":"2024-09-21T06:52:45.721396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701395+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701395+0000","last_peered":"2024-09-21T06:52:31.701395+0000","last_clean":"2024-09-21T06:52:31.701395+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701395+0000","last_undegraded":"2024-09-21T06:52:31.701395+0000","last_fullsized":"2024-09-21T06:52:31.701395+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700554+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:31.700554+0000","last_peered":"2024-09-21T06:52:31.700554+0000","last_clean":"2024-09-21T06:52:31.700554+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:31.700554+0000","last_undegraded":"2024-09-21T06:52:31.700554+0000","last_fullsized":"2024-09-21T06:52:31.700554+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701801+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701801+0000","last_peered":"2024-09-21T06:52:31.701801+0000","last_clean":"2024-09-21T06:52:31.701801+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701801+0000","last_undegraded":"2024-09-21T06:52:31.701801+0000","last_fullsized":"2024-09-21T06:52:31.701801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892142+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:31.892142+0000","last_peered":"2024-09-21T06:52:31.892142+0000","last_clean":"2024-09-21T06:52:31.892142+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:31.892142+0000","last_undegraded":"2024-09-21T06:52:31.892142+0000","last_fullsized":"2024-09-21T06:52:31.892142+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648426+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648426+0000","last_peered":"2024-09-21T06:52:31.648426+0000","last_clean":"2024-09-21T06:52:31.648426+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648426+0000","last_undegraded":"2024-09-21T06:52:31.648426+0000","last_fullsized":"2024-09-21T06:52:31.648426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648654+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648654+0000","last_peered":"2024-09-21T06:52:31.648654+0000","last_clean":"2024-09-21T06:52:31.648654+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648654+0000","last_undegraded":"2024-09-21T06:52:31.648654+0000","last_fullsized":"2024-09-21T06:52:31.648654+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648955+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:31.648955+0000","last_peered":"2024-09-21T06:52:31.648955+0000","last_clean":"2024-09-21T06:52:31.648955+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:31.648955+0000","last_undegraded":"2024-09-21T06:52:31.648955+0000","last_fullsized":"2024-09-21T06:52:31.648955+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'6","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:45.722834+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:52:45.722834+0000","last_peered":"2024-09-21T06:52:45.722834+0000","last_clean":"2024-09-21T06:52:45.722834+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:52:45.722834+0000","last_undegraded":"2024-09-21T06:52:45.722834+0000","last_fullsized":"2024-09-21T06:52:45.722834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836703+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836703+0000","last_peered":"2024-09-21T06:52:31.836703+0000","last_clean":"2024-09-21T06:52:31.836703+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836703+0000","last_undegraded":"2024-09-21T06:52:31.836703+0000","last_fullsized":"2024-09-21T06:52:31.836703+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648636+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:31.648636+0000","last_peered":"2024-09-21T06:52:31.648636+0000","last_clean":"2024-09-21T06:52:31.648636+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:31.648636+0000","last_undegraded":"2024-09-21T06:52:31.648636+0000","last_fullsized":"2024-09-21T06:52:31.648636+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702276+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:52:31.702276+0000","last_peered":"2024-09-21T06:52:31.702276+0000","last_clean":"2024-09-21T06:52:31.702276+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:52:31.702276+0000","last_undegraded":"2024-09-21T06:52:31.702276+0000","last_fullsized":"2024-09-21T06:52:31.702276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891809+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891809+0000","last_peered":"2024-09-21T06:52:31.891809+0000","last_clean":"2024-09-21T06:52:31.891809+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891809+0000","last_undegraded":"2024-09-21T06:52:31.891809+0000","last_fullsized":"2024-09-21T06:52:31.891809+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648799+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:31.648799+0000","last_peered":"2024-09-21T06:52:31.648799+0000","last_clean":"2024-09-21T06:52:31.648799+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:31.648799+0000","last_undegraded":"2024-09-21T06:52:31.648799+0000","last_fullsized":"2024-09-21T06:52:31.648799+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701582+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701582+0000","last_peered":"2024-09-21T06:52:31.701582+0000","last_clean":"2024-09-21T06:52:31.701582+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701582+0000","last_undegraded":"2024-09-21T06:52:31.701582+0000","last_fullsized":"2024-09-21T06:52:31.701582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701272+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701272+0000","last_peered":"2024-09-21T06:52:31.701272+0000","last_clean":"2024-09-21T06:52:31.701272+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701272+0000","last_undegraded":"2024-09-21T06:52:31.701272+0000","last_fullsized":"2024-09-21T06:52:31.701272+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645790+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:31.645790+0000","last_peered":"2024-09-21T06:52:31.645790+0000","last_clean":"2024-09-21T06:52:31.645790+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:31.645790+0000","last_undegraded":"2024-09-21T06:52:31.645790+0000","last_fullsized":"2024-09-21T06:52:31.645790+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645903+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645903+0000","last_peered":"2024-09-21T06:52:31.645903+0000","last_clean":"2024-09-21T06:52:31.645903+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645903+0000","last_undegraded":"2024-09-21T06:52:31.645903+0000","last_fullsized":"2024-09-21T06:52:31.645903+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701111+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701111+0000","last_peered":"2024-09-21T06:52:31.701111+0000","last_clean":"2024-09-21T06:52:31.701111+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701111+0000","last_undegraded":"2024-09-21T06:52:31.701111+0000","last_fullsized":"2024-09-21T06:52:31.701111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.838673+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:31.838673+0000","last_peered":"2024-09-21T06:52:31.838673+0000","last_clean":"2024-09-21T06:52:31.838673+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:31.838673+0000","last_undegraded":"2024-09-21T06:52:31.838673+0000","last_fullsized":"2024-09-21T06:52:31.838673+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891438+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891438+0000","last_peered":"2024-09-21T06:52:31.891438+0000","last_clean":"2024-09-21T06:52:31.891438+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891438+0000","last_undegraded":"2024-09-21T06:52:31.891438+0000","last_fullsized":"2024-09-21T06:52:31.891438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701495+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701495+0000","last_peered":"2024-09-21T06:52:31.701495+0000","last_clean":"2024-09-21T06:52:31.701495+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701495+0000","last_undegraded":"2024-09-21T06:52:31.701495+0000","last_fullsized":"2024-09-21T06:52:31.701495+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.649831+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:31.649831+0000","last_peered":"2024-09-21T06:52:31.649831+0000","last_clean":"2024-09-21T06:52:31.649831+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:31.649831+0000","last_undegraded":"2024-09-21T06:52:31.649831+0000","last_fullsized":"2024-09-21T06:52:31.649831+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:49.727399+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:52:49.727399+0000","last_peered":"2024-09-21T06:52:49.727399+0000","last_clean":"2024-09-21T06:52:49.727399+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:52:49.727399+0000","last_undegraded":"2024-09-21T06:52:49.727399+0000","last_fullsized":"2024-09-21T06:52:49.727399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891921+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891921+0000","last_peered":"2024-09-21T06:52:31.891921+0000","last_clean":"2024-09-21T06:52:31.891921+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891921+0000","last_undegraded":"2024-09-21T06:52:31.891921+0000","last_fullsized":"2024-09-21T06:52:31.891921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648303+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:31.648303+0000","last_peered":"2024-09-21T06:52:31.648303+0000","last_clean":"2024-09-21T06:52:31.648303+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:31.648303+0000","last_undegraded":"2024-09-21T06:52:31.648303+0000","last_fullsized":"2024-09-21T06:52:31.648303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701337+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701337+0000","last_peered":"2024-09-21T06:52:31.701337+0000","last_clean":"2024-09-21T06:52:31.701337+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701337+0000","last_undegraded":"2024-09-21T06:52:31.701337+0000","last_fullsized":"2024-09-21T06:52:31.701337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645436+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645436+0000","last_peered":"2024-09-21T06:52:31.645436+0000","last_clean":"2024-09-21T06:52:31.645436+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645436+0000","last_undegraded":"2024-09-21T06:52:31.645436+0000","last_fullsized":"2024-09-21T06:52:31.645436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700887+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:31.700887+0000","last_peered":"2024-09-21T06:52:31.700887+0000","last_clean":"2024-09-21T06:52:31.700887+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:31.700887+0000","last_undegraded":"2024-09-21T06:52:31.700887+0000","last_fullsized":"2024-09-21T06:52:31.700887+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703771+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703771+0000","last_peered":"2024-09-21T06:52:31.703771+0000","last_clean":"2024-09-21T06:52:31.703771+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703771+0000","last_undegraded":"2024-09-21T06:52:31.703771+0000","last_fullsized":"2024-09-21T06:52:31.703771+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701998+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.701998+0000","last_peered":"2024-09-21T06:52:31.701998+0000","last_clean":"2024-09-21T06:52:31.701998+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.701998+0000","last_undegraded":"2024-09-21T06:52:31.701998+0000","last_fullsized":"2024-09-21T06:52:31.701998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891820+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:31.891820+0000","last_peered":"2024-09-21T06:52:31.891820+0000","last_clean":"2024-09-21T06:52:31.891820+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:31.891820+0000","last_undegraded":"2024-09-21T06:52:31.891820+0000","last_fullsized":"2024-09-21T06:52:31.891820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":217,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:48.726056+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:52:48.726056+0000","last_peered":"2024-09-21T06:52:48.726056+0000","last_clean":"2024-09-21T06:52:48.726056+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:52:48.726056+0000","last_undegraded":"2024-09-21T06:52:48.726056+0000","last_fullsized":"2024-09-21T06:52:48.726056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67067,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":136,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645656+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645656+0000","last_peered":"2024-09-21T06:52:31.645656+0000","last_clean":"2024-09-21T06:52:31.645656+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645656+0000","last_undegraded":"2024-09-21T06:52:31.645656+0000","last_fullsized":"2024-09-21T06:52:31.645656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837113+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:31.837113+0000","last_peered":"2024-09-21T06:52:31.837113+0000","last_clean":"2024-09-21T06:52:31.837113+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:31.837113+0000","last_undegraded":"2024-09-21T06:52:31.837113+0000","last_fullsized":"2024-09-21T06:52:31.837113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702253+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702253+0000","last_peered":"2024-09-21T06:52:31.702253+0000","last_clean":"2024-09-21T06:52:31.702253+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702253+0000","last_undegraded":"2024-09-21T06:52:31.702253+0000","last_fullsized":"2024-09-21T06:52:31.702253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699437+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699437+0000","last_peered":"2024-09-21T06:52:31.699437+0000","last_clean":"2024-09-21T06:52:31.699437+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699437+0000","last_undegraded":"2024-09-21T06:52:31.699437+0000","last_fullsized":"2024-09-21T06:52:31.699437+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892920+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:31.892920+0000","last_peered":"2024-09-21T06:52:31.892920+0000","last_clean":"2024-09-21T06:52:31.892920+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:31.892920+0000","last_undegraded":"2024-09-21T06:52:31.892920+0000","last_fullsized":"2024-09-21T06:52:31.892920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648834+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648834+0000","last_peered":"2024-09-21T06:52:31.648834+0000","last_clean":"2024-09-21T06:52:31.648834+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648834+0000","last_undegraded":"2024-09-21T06:52:31.648834+0000","last_fullsized":"2024-09-21T06:52:31.648834+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645497+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645497+0000","last_peered":"2024-09-21T06:52:31.645497+0000","last_clean":"2024-09-21T06:52:31.645497+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645497+0000","last_undegraded":"2024-09-21T06:52:31.645497+0000","last_fullsized":"2024-09-21T06:52:31.645497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837389+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:31.837389+0000","last_peered":"2024-09-21T06:52:31.837389+0000","last_clean":"2024-09-21T06:52:31.837389+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:31.837389+0000","last_undegraded":"2024-09-21T06:52:31.837389+0000","last_fullsized":"2024-09-21T06:52:31.837389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:48.725765+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:52:48.725765+0000","last_peered":"2024-09-21T06:52:48.725765+0000","last_clean":"2024-09-21T06:52:48.725765+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:52:48.725765+0000","last_undegraded":"2024-09-21T06:52:48.725765+0000","last_fullsized":"2024-09-21T06:52:48.725765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892649+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892649+0000","last_peered":"2024-09-21T06:52:31.892649+0000","last_clean":"2024-09-21T06:52:31.892649+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892649+0000","last_undegraded":"2024-09-21T06:52:31.892649+0000","last_fullsized":"2024-09-21T06:52:31.892649+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892135+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892135+0000","last_peered":"2024-09-21T06:52:31.892135+0000","last_clean":"2024-09-21T06:52:31.892135+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892135+0000","last_undegraded":"2024-09-21T06:52:31.892135+0000","last_fullsized":"2024-09-21T06:52:31.892135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646289+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:31.646289+0000","last_peered":"2024-09-21T06:52:31.646289+0000","last_clean":"2024-09-21T06:52:31.646289+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:31.646289+0000","last_undegraded":"2024-09-21T06:52:31.646289+0000","last_fullsized":"2024-09-21T06:52:31.646289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701545+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701545+0000","last_peered":"2024-09-21T06:52:31.701545+0000","last_clean":"2024-09-21T06:52:31.701545+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701545+0000","last_undegraded":"2024-09-21T06:52:31.701545+0000","last_fullsized":"2024-09-21T06:52:31.701545+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645757+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645757+0000","last_peered":"2024-09-21T06:52:31.645757+0000","last_clean":"2024-09-21T06:52:31.645757+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645757+0000","last_undegraded":"2024-09-21T06:52:31.645757+0000","last_fullsized":"2024-09-21T06:52:31.645757+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892107+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:31.892107+0000","last_peered":"2024-09-21T06:52:31.892107+0000","last_clean":"2024-09-21T06:52:31.892107+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:31.892107+0000","last_undegraded":"2024-09-21T06:52:31.892107+0000","last_fullsized":"2024-09-21T06:52:31.892107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700649+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700649+0000","last_peered":"2024-09-21T06:52:31.700649+0000","last_clean":"2024-09-21T06:52:31.700649+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700649+0000","last_undegraded":"2024-09-21T06:52:31.700649+0000","last_fullsized":"2024-09-21T06:52:31.700649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645893+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645893+0000","last_peered":"2024-09-21T06:52:31.645893+0000","last_clean":"2024-09-21T06:52:31.645893+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645893+0000","last_undegraded":"2024-09-21T06:52:31.645893+0000","last_fullsized":"2024-09-21T06:52:31.645893+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701357+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:31.701357+0000","last_peered":"2024-09-21T06:52:31.701357+0000","last_clean":"2024-09-21T06:52:31.701357+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:31.701357+0000","last_undegraded":"2024-09-21T06:52:31.701357+0000","last_fullsized":"2024-09-21T06:52:31.701357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703769+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703769+0000","last_peered":"2024-09-21T06:52:31.703769+0000","last_clean":"2024-09-21T06:52:31.703769+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703769+0000","last_undegraded":"2024-09-21T06:52:31.703769+0000","last_fullsized":"2024-09-21T06:52:31.703769+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:44.721239+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:52:44.721239+0000","last_peered":"2024-09-21T06:52:44.721239+0000","last_clean":"2024-09-21T06:52:44.721239+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:52:44.721239+0000","last_undegraded":"2024-09-21T06:52:44.721239+0000","last_fullsized":"2024-09-21T06:52:44.721239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891607+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:31.891607+0000","last_peered":"2024-09-21T06:52:31.891607+0000","last_clean":"2024-09-21T06:52:31.891607+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:31.891607+0000","last_undegraded":"2024-09-21T06:52:31.891607+0000","last_fullsized":"2024-09-21T06:52:31.891607+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703996+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.703996+0000","last_peered":"2024-09-21T06:52:31.703996+0000","last_clean":"2024-09-21T06:52:31.703996+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.703996+0000","last_undegraded":"2024-09-21T06:52:31.703996+0000","last_fullsized":"2024-09-21T06:52:31.703996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645287+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:31.645287+0000","last_peered":"2024-09-21T06:52:31.645287+0000","last_clean":"2024-09-21T06:52:31.645287+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:31.645287+0000","last_undegraded":"2024-09-21T06:52:31.645287+0000","last_fullsized":"2024-09-21T06:52:31.645287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703972+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:31.703972+0000","last_peered":"2024-09-21T06:52:31.703972+0000","last_clean":"2024-09-21T06:52:31.703972+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:31.703972+0000","last_undegraded":"2024-09-21T06:52:31.703972+0000","last_fullsized":"2024-09-21T06:52:31.703972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703691+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:31.703691+0000","last_peered":"2024-09-21T06:52:31.703691+0000","last_clean":"2024-09-21T06:52:31.703691+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:31.703691+0000","last_undegraded":"2024-09-21T06:52:31.703691+0000","last_fullsized":"2024-09-21T06:52:31.703691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891918+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:31.891918+0000","last_peered":"2024-09-21T06:52:31.891918+0000","last_clean":"2024-09-21T06:52:31.891918+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:31.891918+0000","last_undegraded":"2024-09-21T06:52:31.891918+0000","last_fullsized":"2024-09-21T06:52:31.891918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701596+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:31.701596+0000","last_peered":"2024-09-21T06:52:31.701596+0000","last_clean":"2024-09-21T06:52:31.701596+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:31.701596+0000","last_undegraded":"2024-09-21T06:52:31.701596+0000","last_fullsized":"2024-09-21T06:52:31.701596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645796+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:31.645796+0000","last_peered":"2024-09-21T06:52:31.645796+0000","last_clean":"2024-09-21T06:52:31.645796+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:31.645796+0000","last_undegraded":"2024-09-21T06:52:31.645796+0000","last_fullsized":"2024-09-21T06:52:31.645796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704275+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:31.704275+0000","last_peered":"2024-09-21T06:52:31.704275+0000","last_clean":"2024-09-21T06:52:31.704275+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:31.704275+0000","last_undegraded":"2024-09-21T06:52:31.704275+0000","last_fullsized":"2024-09-21T06:52:31.704275+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892641+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:31.892641+0000","last_peered":"2024-09-21T06:52:31.892641+0000","last_clean":"2024-09-21T06:52:31.892641+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:31.892641+0000","last_undegraded":"2024-09-21T06:52:31.892641+0000","last_fullsized":"2024-09-21T06:52:31.892641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701411+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:31.701411+0000","last_peered":"2024-09-21T06:52:31.701411+0000","last_clean":"2024-09-21T06:52:31.701411+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:31.701411+0000","last_undegraded":"2024-09-21T06:52:31.701411+0000","last_fullsized":"2024-09-21T06:52:31.701411+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892776+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:31.892776+0000","last_peered":"2024-09-21T06:52:31.892776+0000","last_clean":"2024-09-21T06:52:31.892776+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:31.892776+0000","last_undegraded":"2024-09-21T06:52:31.892776+0000","last_fullsized":"2024-09-21T06:52:31.892776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836962+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:31.836962+0000","last_peered":"2024-09-21T06:52:31.836962+0000","last_clean":"2024-09-21T06:52:31.836962+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:31.836962+0000","last_undegraded":"2024-09-21T06:52:31.836962+0000","last_fullsized":"2024-09-21T06:52:31.836962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701208+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:31.701208+0000","last_peered":"2024-09-21T06:52:31.701208+0000","last_clean":"2024-09-21T06:52:31.701208+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:31.701208+0000","last_undegraded":"2024-09-21T06:52:31.701208+0000","last_fullsized":"2024-09-21T06:52:31.701208+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645509+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:31.645509+0000","last_peered":"2024-09-21T06:52:31.645509+0000","last_clean":"2024-09-21T06:52:31.645509+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:31.645509+0000","last_undegraded":"2024-09-21T06:52:31.645509+0000","last_fullsized":"2024-09-21T06:52:31.645509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704175+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:31.704175+0000","last_peered":"2024-09-21T06:52:31.704175+0000","last_clean":"2024-09-21T06:52:31.704175+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:31.704175+0000","last_undegraded":"2024-09-21T06:52:31.704175+0000","last_fullsized":"2024-09-21T06:52:31.704175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646280+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:31.646280+0000","last_peered":"2024-09-21T06:52:31.646280+0000","last_clean":"2024-09-21T06:52:31.646280+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:31.646280+0000","last_undegraded":"2024-09-21T06:52:31.646280+0000","last_fullsized":"2024-09-21T06:52:31.646280+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646011+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:31.646011+0000","last_peered":"2024-09-21T06:52:31.646011+0000","last_clean":"2024-09-21T06:52:31.646011+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:31.646011+0000","last_undegraded":"2024-09-21T06:52:31.646011+0000","last_fullsized":"2024-09-21T06:52:31.646011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836631+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:31.836631+0000","last_peered":"2024-09-21T06:52:31.836631+0000","last_clean":"2024-09-21T06:52:31.836631+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:31.836631+0000","last_undegraded":"2024-09-21T06:52:31.836631+0000","last_fullsized":"2024-09-21T06:52:31.836631+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891475+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:31.891475+0000","last_peered":"2024-09-21T06:52:31.891475+0000","last_clean":"2024-09-21T06:52:31.891475+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:31.891475+0000","last_undegraded":"2024-09-21T06:52:31.891475+0000","last_fullsized":"2024-09-21T06:52:31.891475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891521+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:31.891521+0000","last_peered":"2024-09-21T06:52:31.891521+0000","last_clean":"2024-09-21T06:52:31.891521+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:31.891521+0000","last_undegraded":"2024-09-21T06:52:31.891521+0000","last_fullsized":"2024-09-21T06:52:31.891521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701074+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:31.701074+0000","last_peered":"2024-09-21T06:52:31.701074+0000","last_clean":"2024-09-21T06:52:31.701074+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:31.701074+0000","last_undegraded":"2024-09-21T06:52:31.701074+0000","last_fullsized":"2024-09-21T06:52:31.701074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700052+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:31.700052+0000","last_peered":"2024-09-21T06:52:31.700052+0000","last_clean":"2024-09-21T06:52:31.700052+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:31.700052+0000","last_undegraded":"2024-09-21T06:52:31.700052+0000","last_fullsized":"2024-09-21T06:52:31.700052+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648794+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:31.648794+0000","last_peered":"2024-09-21T06:52:31.648794+0000","last_clean":"2024-09-21T06:52:31.648794+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:31.648794+0000","last_undegraded":"2024-09-21T06:52:31.648794+0000","last_fullsized":"2024-09-21T06:52:31.648794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646996+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:31.646996+0000","last_peered":"2024-09-21T06:52:31.646996+0000","last_clean":"2024-09-21T06:52:31.646996+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:31.646996+0000","last_undegraded":"2024-09-21T06:52:31.646996+0000","last_fullsized":"2024-09-21T06:52:31.646996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.837517+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:31.837517+0000","last_peered":"2024-09-21T06:52:31.837517+0000","last_clean":"2024-09-21T06:52:31.837517+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:31.837517+0000","last_undegraded":"2024-09-21T06:52:31.837517+0000","last_fullsized":"2024-09-21T06:52:31.837517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891996+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:31.891996+0000","last_peered":"2024-09-21T06:52:31.891996+0000","last_clean":"2024-09-21T06:52:31.891996+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:31.891996+0000","last_undegraded":"2024-09-21T06:52:31.891996+0000","last_fullsized":"2024-09-21T06:52:31.891996+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.647741+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:31.647741+0000","last_peered":"2024-09-21T06:52:31.647741+0000","last_clean":"2024-09-21T06:52:31.647741+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:31.647741+0000","last_undegraded":"2024-09-21T06:52:31.647741+0000","last_fullsized":"2024-09-21T06:52:31.647741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.705828+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:31.705828+0000","last_peered":"2024-09-21T06:52:31.705828+0000","last_clean":"2024-09-21T06:52:31.705828+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:31.705828+0000","last_undegraded":"2024-09-21T06:52:31.705828+0000","last_fullsized":"2024-09-21T06:52:31.705828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891528+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:31.891528+0000","last_peered":"2024-09-21T06:52:31.891528+0000","last_clean":"2024-09-21T06:52:31.891528+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:31.891528+0000","last_undegraded":"2024-09-21T06:52:31.891528+0000","last_fullsized":"2024-09-21T06:52:31.891528+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700010+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:31.700010+0000","last_peered":"2024-09-21T06:52:31.700010+0000","last_clean":"2024-09-21T06:52:31.700010+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:31.700010+0000","last_undegraded":"2024-09-21T06:52:31.700010+0000","last_fullsized":"2024-09-21T06:52:31.700010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892360+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:31.892360+0000","last_peered":"2024-09-21T06:52:31.892360+0000","last_clean":"2024-09-21T06:52:31.892360+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:31.892360+0000","last_undegraded":"2024-09-21T06:52:31.892360+0000","last_fullsized":"2024-09-21T06:52:31.892360+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892241+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:31.892241+0000","last_peered":"2024-09-21T06:52:31.892241+0000","last_clean":"2024-09-21T06:52:31.892241+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:31.892241+0000","last_undegraded":"2024-09-21T06:52:31.892241+0000","last_fullsized":"2024-09-21T06:52:31.892241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645985+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:31.645985+0000","last_peered":"2024-09-21T06:52:31.645985+0000","last_clean":"2024-09-21T06:52:31.645985+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:31.645985+0000","last_undegraded":"2024-09-21T06:52:31.645985+0000","last_fullsized":"2024-09-21T06:52:31.645985+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74662,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":694,"num_read_kb":429,"num_write":250,"num_write_kb":147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":223986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1040384,"data_stored":531658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906987,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28728,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716424,"statfs":{"total":95995035648,"available":95965618176,"internally_reserved":0,"allocated":1736704,"data_stored":1185128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070512,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1064960,"data_stored":535271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266743,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":531531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1654784,"data_stored":1128441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":527070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822728,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1662976,"data_stored":1123632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149780,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1048576,"data_stored":526387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378714,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1122304,"data_stored":594765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542238,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1118208,"data_stored":596180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705765,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":528526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:52:52.651 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:52:52.651 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T06:52:52.651 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T06:52:52.652 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T06:52:52.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:52 smithi183 ceph-mon[34219]: pgmap v358: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:52:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:52 smithi193 ceph-mon[34601]: pgmap v358: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:52:53.056 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:53.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:52 smithi114 ceph-mon[30654]: pgmap v358: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 -- 172.21.15.114:0/3449744197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190078290 msgr2=0x7f8190114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 --2- 172.21.15.114:0/3449744197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190078290 0x7f8190114030 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c02a410 comp rx=0 tx=0).stop 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 -- 172.21.15.114:0/3449744197 shutdown_connections 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 --2- 172.21.15.114:0/3449744197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 0x7f8190116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 --2- 172.21.15.114:0/3449744197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190078290 0x7f8190114030 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 --2- 172.21.15.114:0/3449744197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190077940 0x7f8190077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 -- 172.21.15.114:0/3449744197 >> 172.21.15.114:0/3449744197 conn(0x7f8190101680 msgr2=0x7f8190103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 -- 172.21.15.114:0/3449744197 shutdown_connections 2024-09-21T06:52:53.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.680+0000 7f818f200640 1 -- 172.21.15.114:0/3449744197 wait complete. 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 Processor -- start 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 -- start start 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 0x7f81901a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190078290 0x7f81901a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 0x7f81901a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8190119ed0 con 0x7f8190077940 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8190119d50 con 0x7f8190078290 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.681+0000 7f818f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f819011a050 con 0x7f8190114570 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 0x7f81901a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 0x7f81901a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:50468/0 (socket says 172.21.15.114:50468) 2024-09-21T06:52:53.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 -- 172.21.15.114:0/3000823464 learned_addr learned my addr 172.21.15.114:0/3000823464 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818d400640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190078290 0x7f81901a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818e800640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 0x7f81901a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 -- 172.21.15.114:0/3000823464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 msgr2=0x7f81901a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 0x7f81901a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 -- 172.21.15.114:0/3000823464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190078290 msgr2=0x7f81901a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190078290 0x7f81901a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 -- 172.21.15.114:0/3000823464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81901a6930 con 0x7f8190077940 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818e800640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 0x7f81901a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:52:53.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.682+0000 7f818de00640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 0x7f81901a09c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:53.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.683+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000cd80 con 0x7f8190077940 2024-09-21T06:52:53.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.683+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81800108e0 con 0x7f8190077940 2024-09-21T06:52:53.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.683+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81901a6c20 con 0x7f8190077940 2024-09-21T06:52:53.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.683+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180018980 con 0x7f8190077940 2024-09-21T06:52:53.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.683+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901ad380 con 0x7f8190077940 2024-09-21T06:52:53.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.684+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8190077d50 con 0x7f8190077940 2024-09-21T06:52:53.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.686+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f8180010430 con 0x7f8190077940 2024-09-21T06:52:53.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.689+0000 7f8186a00640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81740b65f0 0x7f81740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.689+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f81800e8250 con 0x7f8190077940 2024-09-21T06:52:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.689+0000 7f818d400640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81740b65f0 0x7f81740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:53.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.690+0000 7f818d400640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81740b65f0 0x7f81740b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f817c009230 tx=0x7f817c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:53.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.691+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81800ae490 con 0x7f8190077940 2024-09-21T06:52:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:53 smithi114 ceph-mon[30654]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:53.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:53 smithi183 ceph-mon[34219]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:53.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:53 smithi193 ceph-mon[34601]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:52:53.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.920+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f81901a1af0 con 0x7f8190077940 2024-09-21T06:52:53.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.921+0000 7f8186a00640 1 -- 172.21.15.114:0/3000823464 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f81901a1af0 con 0x7f8190077940 2024-09-21T06:52:53.923 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:53.923 INFO:teuthology.orchestra.run.smithi114.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T06:52:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.923+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81740b65f0 msgr2=0x7f81740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:53.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.923+0000 7f818f200640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81740b65f0 0x7f81740b8b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f817c009230 tx=0x7f817c031400 comp rx=0 tx=0).stop 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.923+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 msgr2=0x7f81901a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.923+0000 7f818f200640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 0x7f81901a09c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 shutdown_connections 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8190114570 0x7f81901a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81740b65f0 0x7f81740b8b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190078290 0x7f81901a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 --2- 172.21.15.114:0/3000823464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8190077940 0x7f81901a09c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 >> 172.21.15.114:0/3000823464 conn(0x7f8190101680 msgr2=0x7f81901166a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 shutdown_connections 2024-09-21T06:52:53.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:53.924+0000 7f818f200640 1 -- 172.21.15.114:0/3000823464 wait complete. 2024-09-21T06:52:54.705 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T06:52:54.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:52:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:54 smithi193 ceph-mon[34601]: pgmap v359: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:54.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:54 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3000823464' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:52:55.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:54 smithi114 ceph-mon[30654]: pgmap v359: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:55.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:54 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3000823464' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:52:55.073 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:54 smithi183 ceph-mon[34219]: pgmap v359: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:52:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:54 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3000823464' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 -- 172.21.15.114:0/2925970813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f81092f0 msgr2=0x7ff4f8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 --2- 172.21.15.114:0/2925970813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f81092f0 0x7ff4f8109700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009900 tx=0x7ff4e402a410 comp rx=0 tx=0).stop 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 -- 172.21.15.114:0/2925970813 shutdown_connections 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 --2- 172.21.15.114:0/2925970813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f8109c40 0x7ff4f8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 --2- 172.21.15.114:0/2925970813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f81092f0 0x7ff4f8109700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 --2- 172.21.15.114:0/2925970813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81039b0 0x7ff4f8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 -- 172.21.15.114:0/2925970813 >> 172.21.15.114:0/2925970813 conn(0x7ff4f80ff440 msgr2=0x7ff4f8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 -- 172.21.15.114:0/2925970813 shutdown_connections 2024-09-21T06:52:55.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.729+0000 7ff4fca00640 1 -- 172.21.15.114:0/2925970813 wait complete. 2024-09-21T06:52:55.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 Processor -- start 2024-09-21T06:52:55.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 -- start start 2024-09-21T06:52:55.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f81039b0 0x7ff4f819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:55.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 0x7ff4f819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:55.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f8109c40 0x7ff4f81a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:55.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8113900 con 0x7ff4f8109c40 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8113780 con 0x7ff4f81039b0 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.731+0000 7ff4fca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8113a80 con 0x7ff4f81092f0 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 0x7ff4f819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f8109c40 0x7ff4f81a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 0x7ff4f819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45790/0 (socket says 172.21.15.114:45790) 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f6a00640 1 -- 172.21.15.114:0/3914571601 learned_addr learned my addr 172.21.15.114:0/3914571601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:55.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f81039b0 0x7ff4f819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 -- 172.21.15.114:0/3914571601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 msgr2=0x7ff4f819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 0x7ff4f819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 -- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f8109c40 msgr2=0x7ff4f81a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f8109c40 0x7ff4f81a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 -- 172.21.15.114:0/3914571601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f81a24a0 con 0x7ff4f81039b0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f6a00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 0x7ff4f819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.732+0000 7ff4f7400640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f81039b0 0x7ff4f819c530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005fd0 tx=0x7ff4e800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.733+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8009c30 con 0x7ff4f81039b0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.733+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a2790 con 0x7ff4f81039b0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.733+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e8009dd0 con 0x7ff4f81039b0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.733+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8003e20 con 0x7ff4f81039b0 2024-09-21T06:52:55.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.733+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a8ef0 con 0x7ff4f81039b0 2024-09-21T06:52:55.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.734+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f8103dc0 con 0x7ff4f81039b0 2024-09-21T06:52:55.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.737+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7ff4e8003fc0 con 0x7ff4f81039b0 2024-09-21T06:52:55.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.738+0000 7ff4efe00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4dc0b6670 0x7ff4dc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:55.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.738+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff4e80e74e0 con 0x7ff4f81039b0 2024-09-21T06:52:55.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.739+0000 7ff4f6a00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4dc0b6670 0x7ff4dc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:55.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.739+0000 7ff4f6a00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4dc0b6670 0x7ff4dc0b8b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8105ab0 tx=0x7ff4e4032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:55.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.742+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4e8017020 con 0x7ff4f81039b0 2024-09-21T06:52:55.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.976+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4f819d660 con 0x7ff4f81039b0 2024-09-21T06:52:55.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.977+0000 7ff4efe00640 1 -- 172.21.15.114:0/3914571601 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7ff4e80b1b00 con 0x7ff4f81039b0 2024-09-21T06:52:55.980 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:55.980 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"btime":"2024-09-21T06:51:45:610909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:45.336678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:52:55.981 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:52:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4dc0b6670 msgr2=0x7ff4dc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4dc0b6670 0x7ff4dc0b8b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8105ab0 tx=0x7ff4e4032ed0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f81039b0 msgr2=0x7ff4f819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f81039b0 0x7ff4f819c530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8005fd0 tx=0x7ff4e800dde0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 shutdown_connections 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4f8109c40 0x7ff4f81a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f81092f0 0x7ff4f819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4dc0b6670 0x7ff4dc0b8b80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 --2- 172.21.15.114:0/3914571601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4f81039b0 0x7ff4f819c530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 >> 172.21.15.114:0/3914571601 conn(0x7ff4f80ff440 msgr2=0x7ff4f81089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:55.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 shutdown_connections 2024-09-21T06:52:55.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:55.981+0000 7ff4fca00640 1 -- 172.21.15.114:0/3914571601 wait complete. 2024-09-21T06:52:56.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:52:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:56 smithi193 ceph-mon[34601]: pgmap v360: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:52:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:56 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3914571601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:52:56.953 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:56 smithi114 ceph-mon[30654]: pgmap v360: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:52:56.953 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:56 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3914571601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:52:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:56 smithi183 ceph-mon[34219]: pgmap v360: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:52:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:56 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3914571601' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:52:57.153 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:57.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 -- 172.21.15.114:0/2509478168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410b500 msgr2=0x7f4b2410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:57.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 --2- 172.21.15.114:0/2509478168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410b500 0x7f4b2410b910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009900 tx=0x7f4b0c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 -- 172.21.15.114:0/2509478168 shutdown_connections 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 --2- 172.21.15.114:0/2509478168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b2410be50 0x7f4b24112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 --2- 172.21.15.114:0/2509478168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410b500 0x7f4b2410b910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 --2- 172.21.15.114:0/2509478168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b24105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 -- 172.21.15.114:0/2509478168 >> 172.21.15.114:0/2509478168 conn(0x7f4b24101670 msgr2=0x7f4b24103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 -- 172.21.15.114:0/2509478168 shutdown_connections 2024-09-21T06:52:57.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.719+0000 7f4b23e00640 1 -- 172.21.15.114:0/2509478168 wait complete. 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.720+0000 7f4b23e00640 1 Processor -- start 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 -- start start 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b2419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b2410b500 0x7f4b2419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410be50 0x7f4b241a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24115b10 con 0x7f4b2410be50 2024-09-21T06:52:57.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24115990 con 0x7f4b2410b500 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24115c90 con 0x7f4b24105bc0 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b22a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b2419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b22a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b2419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45794/0 (socket says 172.21.15.114:45794) 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b22a00640 1 -- 172.21.15.114:0/4123174875 learned_addr learned my addr 172.21.15.114:0/4123174875 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b23400640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410be50 0x7f4b241a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.721+0000 7f4b1a000640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b2410b500 0x7f4b2419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b22a00640 1 -- 172.21.15.114:0/4123174875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b2410b500 msgr2=0x7f4b2419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b22a00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b2410b500 0x7f4b2419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b22a00640 1 -- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410be50 msgr2=0x7f4b241a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b22a00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410be50 0x7f4b241a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b22a00640 1 -- 172.21.15.114:0/4123174875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b241a2470 con 0x7f4b24105bc0 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b22a00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b2419c500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400e9f0 tx=0x7f4b1400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:57.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400cd80 con 0x7f4b24105bc0 2024-09-21T06:52:57.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b140054d0 con 0x7f4b24105bc0 2024-09-21T06:52:57.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b241a2760 con 0x7f4b24105bc0 2024-09-21T06:52:57.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.722+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14010690 con 0x7f4b24105bc0 2024-09-21T06:52:57.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.723+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241a8ec0 con 0x7f4b24105bc0 2024-09-21T06:52:57.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.723+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b24106e40 con 0x7f4b24105bc0 2024-09-21T06:52:57.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.726+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f4b14004e90 con 0x7f4b24105bc0 2024-09-21T06:52:57.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.727+0000 7f4b1be00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4b080b65f0 0x7f4b080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:57.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.727+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4b140e80c0 con 0x7f4b24105bc0 2024-09-21T06:52:57.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.727+0000 7f4b1a000640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4b080b65f0 0x7f4b080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:57.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.728+0000 7f4b1a000640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4b080b65f0 0x7f4b080b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009230 tx=0x7f4b0c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:57.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.729+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b140ae300 con 0x7f4b24105bc0 2024-09-21T06:52:57.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.954+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b24105fd0 con 0x7f4b24105bc0 2024-09-21T06:52:57.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.956+0000 7f4b1be00640 1 -- 172.21.15.114:0/4123174875 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f4b24105fd0 con 0x7f4b24105bc0 2024-09-21T06:52:57.958 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:57.959 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"btime":"2024-09-21T06:51:45:610909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:45.336678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:52:57.959 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4b080b65f0 msgr2=0x7f4b080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4b080b65f0 0x7f4b080b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c009230 tx=0x7f4b0c031400 comp rx=0 tx=0).stop 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 msgr2=0x7f4b2419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b2419c500 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400e9f0 tx=0x7f4b1400edf0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 shutdown_connections 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4b2410be50 0x7f4b241a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4b080b65f0 0x7f4b080b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b2410b500 0x7f4b2419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 --2- 172.21.15.114:0/4123174875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b24105bc0 0x7f4b2419c500 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 >> 172.21.15.114:0/4123174875 conn(0x7f4b24101670 msgr2=0x7f4b2410abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:57.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 shutdown_connections 2024-09-21T06:52:57.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:57.958+0000 7f4b23e00640 1 -- 172.21.15.114:0/4123174875 wait complete. 2024-09-21T06:52:58.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:52:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:58 smithi193 ceph-mon[34601]: pgmap v361: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:52:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:52:58 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4123174875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:52:59.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:58 smithi114 ceph-mon[30654]: pgmap v361: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:52:59.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:52:58 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4123174875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:52:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:58 smithi183 ceph-mon[34219]: pgmap v361: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:52:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:52:58 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4123174875' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:52:59.148 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:52:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 -- 172.21.15.114:0/4045025381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b20105320 msgr2=0x7f7b2010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 --2- 172.21.15.114:0/4045025381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b20105320 0x7f7b2010d870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009900 tx=0x7f7b0802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 -- 172.21.15.114:0/4045025381 shutdown_connections 2024-09-21T06:52:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 --2- 172.21.15.114:0/4045025381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b2010de30 0x7f7b20110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 --2- 172.21.15.114:0/4045025381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b20105320 0x7f7b2010d870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 --2- 172.21.15.114:0/4045025381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b201049d0 0x7f7b20104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 -- 172.21.15.114:0/4045025381 >> 172.21.15.114:0/4045025381 conn(0x7f7b20079740 msgr2=0x7f7b20079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 -- 172.21.15.114:0/4045025381 shutdown_connections 2024-09-21T06:52:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.791+0000 7f7b26200640 1 -- 172.21.15.114:0/4045025381 wait complete. 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.792+0000 7f7b26200640 1 Processor -- start 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.792+0000 7f7b26200640 1 -- start start 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b26200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b201049d0 0x7f7b2019c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b26200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 0x7f7b2019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b26200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b2010de30 0x7f7b201a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b26200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20113a90 con 0x7f7b201049d0 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b26200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20113910 con 0x7f7b20105320 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b26200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20113c10 con 0x7f7b2010de30 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 0x7f7b2019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b16a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 0x7f7b2019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48890/0 (socket says 172.21.15.114:48890) 2024-09-21T06:52:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b16a00640 1 -- 172.21.15.114:0/2738408342 learned_addr learned my addr 172.21.15.114:0/2738408342 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:52:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b1fe00640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b2010de30 0x7f7b201a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.793+0000 7f7b1f400640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b201049d0 0x7f7b2019c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b16a00640 1 -- 172.21.15.114:0/2738408342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b2010de30 msgr2=0x7f7b201a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b16a00640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b2010de30 0x7f7b201a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b16a00640 1 -- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b201049d0 msgr2=0x7f7b2019c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b16a00640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b201049d0 0x7f7b2019c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b16a00640 1 -- 172.21.15.114:0/2738408342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b201a2690 con 0x7f7b20105320 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b16a00640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 0x7f7b2019ccc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b0802a910 tx=0x7f7b08031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.794+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0803f030 con 0x7f7b20105320 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.795+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b201a2920 con 0x7f7b20105320 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.795+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b08004490 con 0x7f7b20105320 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.795+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08035070 con 0x7f7b20105320 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.795+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b201a90e0 con 0x7f7b20105320 2024-09-21T06:52:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.796+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2006c240 con 0x7f7b20105320 2024-09-21T06:52:59.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.798+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 154581+0+0 (secure 0 0 0) 0x7f7b08042030 con 0x7f7b20105320 2024-09-21T06:52:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.799+0000 7f7b1cc00640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7b040b65f0 0x7f7b040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:52:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.800+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7b081080c0 con 0x7f7b20105320 2024-09-21T06:52:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.800+0000 7f7b1f400640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7b040b65f0 0x7f7b040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:52:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.800+0000 7f7b1f400640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7b040b65f0 0x7f7b040b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7b2006c940 tx=0x7f7b10005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:52:59.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.801+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b080d26e0 con 0x7f7b20105320 2024-09-21T06:52:59.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.987+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b20104de0 con 0x7f7b20105320 2024-09-21T06:52:59.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.988+0000 7f7b1cc00640 1 -- 172.21.15.114:0/2738408342 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25277 (secure 0 0 0) 0x7f7b080d7620 con 0x7f7b20105320 2024-09-21T06:52:59.992 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:52:59.992 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:52:31.637070+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:52:59.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7b040b65f0 msgr2=0x7f7b040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7b040b65f0 0x7f7b040b8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7b2006c940 tx=0x7f7b10005f90 comp rx=0 tx=0).stop 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 msgr2=0x7f7b2019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 0x7f7b2019ccc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b0802a910 tx=0x7f7b08031a20 comp rx=0 tx=0).stop 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 shutdown_connections 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b2010de30 0x7f7b201a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7b040b65f0 0x7f7b040b8b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b20105320 0x7f7b2019ccc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 --2- 172.21.15.114:0/2738408342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7b201049d0 0x7f7b2019c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 >> 172.21.15.114:0/2738408342 conn(0x7f7b20079740 msgr2=0x7f7b20100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 shutdown_connections 2024-09-21T06:52:59.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:52:59.991+0000 7f7b26200640 1 -- 172.21.15.114:0/2738408342 wait complete. 2024-09-21T06:53:00.774 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:53:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:00 smithi193 ceph-mon[34601]: pgmap v362: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:00.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:00 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2738408342' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:01.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:00 smithi114 ceph-mon[30654]: pgmap v362: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:01.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:00 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2738408342' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:00 smithi183 ceph-mon[34219]: pgmap v362: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:00 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2738408342' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:01.141 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 -- 172.21.15.114:0/2420255087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 msgr2=0x7f152410fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 --2- 172.21.15.114:0/2420255087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 0x7f152410fcb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f150c009900 tx=0x7f150c02a480 comp rx=0 tx=0).stop 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 -- 172.21.15.114:0/2420255087 shutdown_connections 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 --2- 172.21.15.114:0/2420255087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15241101f0 0x7f1524112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 --2- 172.21.15.114:0/2420255087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 0x7f152410fcb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 --2- 172.21.15.114:0/2420255087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1524102640 0x7f1524102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 -- 172.21.15.114:0/2420255087 >> 172.21.15.114:0/2420255087 conn(0x7f15240796e0 msgr2=0x7f1524079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 -- 172.21.15.114:0/2420255087 shutdown_connections 2024-09-21T06:53:01.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.770+0000 7f152b800640 1 -- 172.21.15.114:0/2420255087 wait complete. 2024-09-21T06:53:01.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.771+0000 7f152b800640 1 Processor -- start 2024-09-21T06:53:01.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.771+0000 7f152b800640 1 -- start start 2024-09-21T06:53:01.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f152b800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 0x7f152419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:01.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f152b800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 0x7f152419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:01.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f152b800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15241101f0 0x7f15241a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:01.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f152b800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1524115c90 con 0x7f1524102f90 2024-09-21T06:53:01.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f152b800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1524115b10 con 0x7f1524102640 2024-09-21T06:53:01.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f152b800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1524115e10 con 0x7f15241101f0 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 0x7f152419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f151be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 0x7f152419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 0x7f152419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48908/0 (socket says 172.21.15.114:48908) 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 -- 172.21.15.114:0/2734114430 learned_addr learned my addr 172.21.15.114:0/2734114430 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1529800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15241101f0 0x7f15241a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 -- 172.21.15.114:0/2734114430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15241101f0 msgr2=0x7f15241a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15241101f0 0x7f15241a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 -- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 msgr2=0x7f152419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 0x7f152419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.772+0000 7f1528e00640 1 -- 172.21.15.114:0/2734114430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15241a25d0 con 0x7f1524102640 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.773+0000 7f1528e00640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 0x7f152419c660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f151400e9f0 tx=0x7f151400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:01.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.773+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151400cd80 con 0x7f1524102640 2024-09-21T06:53:01.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.773+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15140054d0 con 0x7f1524102640 2024-09-21T06:53:01.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.773+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15241a28c0 con 0x7f1524102640 2024-09-21T06:53:01.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.773+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514010690 con 0x7f1524102640 2024-09-21T06:53:01.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.773+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241a9020 con 0x7f1524102640 2024-09-21T06:53:01.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.774+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152406c1a0 con 0x7f1524102640 2024-09-21T06:53:01.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.777+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f1514004e90 con 0x7f1524102640 2024-09-21T06:53:01.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.778+0000 7f1522000640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15080b65e0 0x7f15080b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:01.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.778+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f15140e7bd0 con 0x7f1524102640 2024-09-21T06:53:01.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.778+0000 7f151be00640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15080b65e0 0x7f15080b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:01.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.779+0000 7f151be00640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15080b65e0 0x7f15080b8af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f150c031440 tx=0x7f150c009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:01.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:01.782+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15140adf70 con 0x7f1524102640 2024-09-21T06:53:02.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.008+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f152402e640 con 0x7f1524102640 2024-09-21T06:53:02.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.009+0000 7f1522000640 1 -- 172.21.15.114:0/2734114430 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7f15140b2350 con 0x7f1524102640 2024-09-21T06:53:02.012 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:02.012 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"btime":"2024-09-21T06:51:45:610909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:45.336678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:53:02.012 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:53:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15080b65e0 msgr2=0x7f15080b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15080b65e0 0x7f15080b8af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f150c031440 tx=0x7f150c009560 comp rx=0 tx=0).stop 2024-09-21T06:53:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 msgr2=0x7f152419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 0x7f152419c660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f151400e9f0 tx=0x7f151400edf0 comp rx=0 tx=0).stop 2024-09-21T06:53:02.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 shutdown_connections 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15241101f0 0x7f15241a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1524102f90 0x7f152419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15080b65e0 0x7f15080b8af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 --2- 172.21.15.114:0/2734114430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1524102640 0x7f152419c660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.013+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 >> 172.21.15.114:0/2734114430 conn(0x7f15240796e0 msgr2=0x7f1524111640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.014+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 shutdown_connections 2024-09-21T06:53:02.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:02.014+0000 7f152b800640 1 -- 172.21.15.114:0/2734114430 wait complete. 2024-09-21T06:53:02.797 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:02 smithi114 ceph-mon[30654]: pgmap v363: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:02.797 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:02 smithi114 ceph-mon[30654]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-09-21T06:53:02.797 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:02 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2734114430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:02.832 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:50:48.407210+0000', 'modified': '2024-09-21T06:51:45.336678+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24761, 'mds_1': 24760, 'mds_2': 24742, 'mds_3': 24727, 'mds_4': 24725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14742': {'gid': 14742, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6839/244481807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 244481807}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 244481807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/2086656361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 2086656361}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 2086656361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/919969847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 919969847}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 919969847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2729823804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2729823804}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2729823804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24715': {'gid': 24715, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/2057077471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2057077471}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2057077471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24725': {'gid': 24725, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.183:6837/841299119', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 841299119}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 841299119}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24727': {'gid': 24727, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.193:6837/1013898612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1013898612}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1013898612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24742': {'gid': 24742, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.193:6839/3226153327', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3226153327}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3226153327}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24760': {'gid': 24760, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.114:6841/2055898271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6840', 'nonce': 2055898271}, {'type': 'v1', 'addr': '172.21.15.114:6841', 'nonce': 2055898271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.183:6839/42004963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 42004963}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 42004963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24761, 'qdb_cluster': [24725, 24727, 24742, 24760, 24761]} 2024-09-21T06:53:02.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T06:53:02.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T06:53:02.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:02 smithi193 ceph-mon[34601]: pgmap v363: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:02.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:02 smithi193 ceph-mon[34601]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-09-21T06:53:02.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:02 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2734114430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:02 smithi183 ceph-mon[34219]: pgmap v363: 129 pgs: 129 active+clean; 650 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:03.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:02 smithi183 ceph-mon[34219]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-09-21T06:53:03.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:02 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2734114430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:03.205 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:03.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 -- 172.21.15.114:0/3130146911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5334105d80 msgr2=0x7f5334106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 --2- 172.21.15.114:0/3130146911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5334105d80 0x7f5334106190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5328009900 tx=0x7f532802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:03.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 -- 172.21.15.114:0/3130146911 shutdown_connections 2024-09-21T06:53:03.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 --2- 172.21.15.114:0/3130146911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f533410c010 0x7f5334112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 --2- 172.21.15.114:0/3130146911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533410b6c0 0x7f533410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 --2- 172.21.15.114:0/3130146911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5334105d80 0x7f5334106190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 -- 172.21.15.114:0/3130146911 >> 172.21.15.114:0/3130146911 conn(0x7f5334101810 msgr2=0x7f5334103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 -- 172.21.15.114:0/3130146911 shutdown_connections 2024-09-21T06:53:03.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.863+0000 7f533c000640 1 -- 172.21.15.114:0/3130146911 wait complete. 2024-09-21T06:53:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.864+0000 7f533c000640 1 Processor -- start 2024-09-21T06:53:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.864+0000 7f533c000640 1 -- start start 2024-09-21T06:53:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533c000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5334105d80 0x7f533419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533c000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 0x7f533419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533c000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533410c010 0x7f53341a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533c000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5334115d20 con 0x7f533410b6c0 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533c000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5334115ba0 con 0x7f533410c010 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533c000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5334115ea0 con 0x7f5334105d80 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f5338c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 0x7f533419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f5338c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 0x7f533419cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40778/0 (socket says 172.21.15.114:40778) 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f5338c00640 1 -- 172.21.15.114:0/3490667001 learned_addr learned my addr 172.21.15.114:0/3490667001 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f5339600640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5334105d80 0x7f533419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f533a000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533410c010 0x7f53341a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f5338c00640 1 -- 172.21.15.114:0/3490667001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5334105d80 msgr2=0x7f533419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.865+0000 7f5338c00640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5334105d80 0x7f533419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5338c00640 1 -- 172.21.15.114:0/3490667001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533410c010 msgr2=0x7f53341a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5338c00640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533410c010 0x7f53341a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5338c00640 1 -- 172.21.15.114:0/3490667001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53341a2660 con 0x7f533410b6c0 2024-09-21T06:53:03.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5339600640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5334105d80 0x7f533419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:03.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5338c00640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 0x7f533419cc90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f532000e9f0 tx=0x7f532000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:03.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5332000640 1 -- 172.21.15.114:0/3490667001 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532000ccc0 con 0x7f533410b6c0 2024-09-21T06:53:03.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5332000640 1 -- 172.21.15.114:0/3490667001 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53200108e0 con 0x7f533410b6c0 2024-09-21T06:53:03.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53341a2950 con 0x7f533410b6c0 2024-09-21T06:53:03.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f5332000640 1 -- 172.21.15.114:0/3490667001 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320018860 con 0x7f533410b6c0 2024-09-21T06:53:03.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.866+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53341a90b0 con 0x7f533410b6c0 2024-09-21T06:53:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 Processor -- start 2024-09-21T06:53:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 -- start start 2024-09-21T06:53:03.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5332000640 1 -- 172.21.15.114:0/3490667001 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f5320010430 con 0x7f533410b6c0 2024-09-21T06:53:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c8b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 0x5649f7c8b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78c4280 0x5649f7c8bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f7c02d50 con 0x5649f78bc0a0 2024-09-21T06:53:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649f7bf2180 con 0x5649f78e4f60 2024-09-21T06:53:03.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.870+0000 7f5342c31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649f7bf5f90 con 0x5649f78c4280 2024-09-21T06:53:03.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 0x5649f7c8b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f533a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c8b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 0x5649f7c8b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40794/0 (socket says 172.21.15.114:40794) 2024-09-21T06:53:03.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 -- 172.21.15.114:0/639874518 learned_addr learned my addr 172.21.15.114:0/639874518 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:03.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5338c00640 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78c4280 0x5649f7c8bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 -- 172.21.15.114:0/639874518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78c4280 msgr2=0x5649f7c8bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78c4280 0x5649f7c8bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 -- 172.21.15.114:0/639874518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 msgr2=0x5649f7c8b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c8b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 -- 172.21.15.114:0/639874518 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f7bbca70 con 0x5649f78bc0a0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f533a000640 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c8b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.871+0000 7f5339600640 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 0x5649f7c8b9d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5328009230 tx=0x7f5328004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e92964c7980f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.872+0000 7f5326600640 1 -- 172.21.15.114:0/639874518 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328035040 con 0x5649f78bc0a0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.872+0000 7f5326600640 1 -- 172.21.15.114:0/639874518 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5328004770 con 0x5649f78bc0a0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.872+0000 7f5326600640 1 -- 172.21.15.114:0/639874518 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280039e0 con 0x5649f78bc0a0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.872+0000 7f5332000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f531c0b6560 0x7f531c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.872+0000 7f5332000640 1 -- 172.21.15.114:0/3490667001 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f53200e7ae0 con 0x7f533410b6c0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.872+0000 7f533a000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f531c0b6560 0x7f531c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 -- 172.21.15.114:0/639874518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 msgr2=0x5649f7c8b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 0x5649f7c8b9d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5328009230 tx=0x7f5328004470 comp rx=0 tx=0).stop 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f533a000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f531c0b6560 0x7f531c0b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f532c009fd0 tx=0x7f532c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 -- 172.21.15.114:0/639874518 shutdown_connections 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78c4280 0x5649f7c8bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78bc0a0 0x5649f7c8b9d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 --2- 172.21.15.114:0/639874518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c8b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 -- 172.21.15.114:0/639874518 >> 172.21.15.114:0/639874518 conn(0x5649f78d9350 msgr2=0x5649f7c89380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 -- 172.21.15.114:0/639874518 shutdown_connections 2024-09-21T06:53:03.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.873+0000 7f5342c31740 1 -- 172.21.15.114:0/639874518 wait complete. 2024-09-21T06:53:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.874+0000 7f5342c31740 1 -- start start 2024-09-21T06:53:03.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.874+0000 7f5342c31740 1 Processor -- start 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.874+0000 7f5342c31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5342c31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5342c31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7ca2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5342c31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f7a535f0 con 0x5649f78c4280 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5342c31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649f7b8d460 con 0x5649f78bc0a0 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5342c31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649f7c99c10 con 0x5649f78e4f60 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5342c31740 10 client.0 _ll_register_callbacks cb 0x5649f7b9d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5338c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48946/0 (socket says 172.21.15.114:48946) 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 -- 172.21.15.114:0/1630599543 learned_addr learned my addr 172.21.15.114:0/1630599543 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:03.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f533a000640 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7ca2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 -- 172.21.15.114:0/1630599543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 msgr2=0x5649f7ca2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7ca2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 -- 172.21.15.114:0/1630599543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 msgr2=0x5649f7c975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.875+0000 7f5339600640 1 -- 172.21.15.114:0/1630599543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f7950540 con 0x5649f78bc0a0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f5339600640 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c970b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5328005ab0 tx=0x7f532802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0486d33e4efd25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f531aa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f531aa00640 1 -- 172.21.15.114:0/1630599543 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328052060 con 0x5649f78bc0a0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f531aa00640 1 -- 172.21.15.114:0/1630599543 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5328004970 con 0x5649f78bc0a0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649f7b8fa70 con 0x5649f78bc0a0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f531aa00640 1 -- 172.21.15.114:0/1630599543 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5328059050 con 0x5649f78bc0a0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f5342c31740 10 client.25205 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.876+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649f7baa6f0 con 0x5649f78bc0a0 2024-09-21T06:53:03.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f531aa00640 1 -- 172.21.15.114:0/1630599543 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5649f7b8fa70 con 0x5649f78bc0a0 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f531aa00640 1 -- 172.21.15.114:0/1630599543 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f532804c070 con 0x5649f78bc0a0 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f5342c31740 10 client.25205 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f5342c31740 10 client.25205 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f5342c31740 20 client.25205 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f5342c31740 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f78a38e0 0x5649f7ca9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f5342c31740 4 client.25205 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:03.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.877+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f7c04ca0 con 0x5649f78a38e0 2024-09-21T06:53:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.878+0000 7f5338c00640 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f78a38e0 0x5649f7ca9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.878+0000 7f5338c00640 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f78a38e0 0x5649f7ca9000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ce109dff91aace37 server_cookie=6dcea805fdb95c9d in_seq=0 out_seq=0 2024-09-21T06:53:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.878+0000 7f531aa00640 10 client.25205 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f531aa00640 1 -- 172.21.15.114:0/1630599543 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649f7c04ca0 con 0x5649f78a38e0 2024-09-21T06:53:03.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f531aa00640 10 client.25205 handle_command_reply: tid=0 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 client.25205 shutdown 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 msgr2=0x5649f7c970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c970b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5328005ab0 tx=0x7f532802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 shutdown_connections 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7ca2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f78a38e0 0x5649f7ca9000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 --2- 172.21.15.114:0/1630599543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c970b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f531aa00640 0 client.25205 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 >> 172.21.15.114:0/1630599543 conn(0x5649f78d9350 msgr2=0x5649f7c8e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 shutdown_connections 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.879+0000 7f5342c31740 1 -- 172.21.15.114:0/1630599543 wait complete. 2024-09-21T06:53:03.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.880+0000 7f5342c31740 20 client.25205 trim_cache size 0 max 16384 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 Processor -- start 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 -- start start 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c90c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7c95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f7ca7340 con 0x5649f78c4280 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649f7ca7a10 con 0x5649f78bc0a0 2024-09-21T06:53:03.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.898+0000 7f5342c31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649f7ca80e0 con 0x5649f78e4f60 2024-09-21T06:53:03.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c90c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c90c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40810/0 (socket says 172.21.15.114:40810) 2024-09-21T06:53:03.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 -- 172.21.15.114:0/2539315006 learned_addr learned my addr 172.21.15.114:0/2539315006 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f533a000640 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7c95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5339600640 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 -- 172.21.15.114:0/2539315006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 msgr2=0x5649f7c95730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7c95730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 -- 172.21.15.114:0/2539315006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 msgr2=0x5649f7c906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c906f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 -- 172.21.15.114:0/2539315006 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f7c98170 con 0x5649f78c4280 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.899+0000 7f5338c00640 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c90c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53200aaaa0 tx=0x7f53200eb4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f554f045635d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.900+0000 7f5325c00640 1 -- 172.21.15.114:0/2539315006 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200ad600 con 0x5649f78c4280 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.900+0000 7f5325c00640 1 -- 172.21.15.114:0/2539315006 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5320010720 con 0x5649f78c4280 2024-09-21T06:53:03.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.900+0000 7f5325c00640 1 -- 172.21.15.114:0/2539315006 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53200acfa0 con 0x5649f78c4280 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.900+0000 7f5342c31740 1 -- 172.21.15.114:0/2539315006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 msgr2=0x5649f7c90c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.900+0000 7f5342c31740 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c90c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f53200aaaa0 tx=0x7f53200eb4a0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 -- 172.21.15.114:0/2539315006 shutdown_connections 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78e4f60 0x5649f7c95730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7c90c30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 --2- 172.21.15.114:0/2539315006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78bc0a0 0x5649f7c906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 -- 172.21.15.114:0/2539315006 >> 172.21.15.114:0/2539315006 conn(0x5649f78d9350 msgr2=0x5649f7c91300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 -- 172.21.15.114:0/2539315006 shutdown_connections 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 -- 172.21.15.114:0/2539315006 wait complete. 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 -- start start 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.901+0000 7f5342c31740 1 Processor -- start 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 0x5649f7cb15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7cb1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649f7ca8e80 con 0x5649f78c4280 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649f7ca87b0 con 0x5649f78e4f60 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649f7ca9550 con 0x5649f78bc0a0 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5342c31740 10 client.0 _ll_register_callbacks cb 0x7f5328009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 0x5649f7cb15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 0x5649f7cb15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45884/0 (socket says 172.21.15.114:45884) 2024-09-21T06:53:03.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5338c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7cb1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 -- 172.21.15.114:0/1470678892 learned_addr learned my addr 172.21.15.114:0/1470678892 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f533a000640 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 -- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 msgr2=0x5649f7c883e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c883e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 -- 172.21.15.114:0/1470678892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 msgr2=0x5649f7cb1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7cb1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.902+0000 7f5339600640 1 -- 172.21.15.114:0/1470678892 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f7c88b70 con 0x5649f78bc0a0 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.903+0000 7f5339600640 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 0x5649f7cb15b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5649f7c99700 tx=0x7f5328031b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad09b979ff049f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:03.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.903+0000 7f531aa00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:53:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.903+0000 7f531aa00640 1 -- 172.21.15.114:0/1470678892 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328061020 con 0x5649f78bc0a0 2024-09-21T06:53:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.903+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649f7c00750 con 0x5649f78bc0a0 2024-09-21T06:53:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.903+0000 7f531aa00640 1 -- 172.21.15.114:0/1470678892 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f53280037f0 con 0x5649f78bc0a0 2024-09-21T06:53:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f531aa00640 1 -- 172.21.15.114:0/1470678892 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f532806c050 con 0x5649f78bc0a0 2024-09-21T06:53:03.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 10 client.25216 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649f7cabe50 con 0x5649f78bc0a0 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f531aa00640 1 -- 172.21.15.114:0/1470678892 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5649f7c00750 con 0x5649f78bc0a0 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f531aa00640 1 -- 172.21.15.114:0/1470678892 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f532804c040 con 0x5649f78bc0a0 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 10 client.25216 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 10 client.25216 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 20 client.25216 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f7878c90 0x5649f7d179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 4 client.25216 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.904+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "status"}) -- 0x5649f7bfdb80 con 0x5649f7878c90 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f5338c00640 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f7878c90 0x5649f7d179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f5338c00640 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f7878c90 0x5649f7d179a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=24df1e69d50f701d server_cookie=a330579a68cfd518 in_seq=0 out_seq=0 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f531aa00640 10 client.25216 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:03.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f531aa00640 1 -- 172.21.15.114:0/1470678892 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5649f7bfdb80 con 0x5649f7878c90 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f531aa00640 10 client.25216 handle_command_reply: tid=0 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f5342c31740 1 client.25216 shutdown 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 msgr2=0x5649f7cb15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.905+0000 7f5342c31740 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 0x5649f7cb15b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5649f7c99700 tx=0x7f5328031b90 comp rx=0 tx=0).stop 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 shutdown_connections 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5649f7878c90 0x5649f7d179a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649f78e4f60 0x5649f7c883e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649f78c4280 0x5649f7cb1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 --2- 172.21.15.114:0/1470678892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649f78bc0a0 0x5649f7cb15b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f531aa00640 0 client.25216 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 >> 172.21.15.114:0/1470678892 conn(0x5649f78d9350 msgr2=0x5649f7c97090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 shutdown_connections 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 1 -- 172.21.15.114:0/1470678892 wait complete. 2024-09-21T06:53:03.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f5342c31740 20 client.25216 trim_cache size 0 max 16384 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 4, 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24725, 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 81.305930313999994, 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:03.912 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:03.913 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:03.913 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 160.57626668399999 2024-09-21T06:53:03.913 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:03.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f531c0b6560 msgr2=0x7f531c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f533c000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f531c0b6560 0x7f531c0b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f532c009fd0 tx=0x7f532c009500 comp rx=0 tx=0).stop 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.906+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 msgr2=0x7f533419cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 0x7f533419cc90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f532000e9f0 tx=0x7f532000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 shutdown_connections 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f531c0b6560 0x7f531c0b8a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533410c010 0x7f53341a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f533410b6c0 0x7f533419cc90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 --2- 172.21.15.114:0/3490667001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5334105d80 0x7f533419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 >> 172.21.15.114:0/3490667001 conn(0x7f5334101810 msgr2=0x7f533410ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 shutdown_connections 2024-09-21T06:53:03.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:03.907+0000 7f533c000640 1 -- 172.21.15.114:0/3490667001 wait complete. 2024-09-21T06:53:04.760 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T06:53:04.914 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:04 smithi193 ceph-mon[34601]: pgmap v364: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:53:05.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:04 smithi114 ceph-mon[30654]: pgmap v364: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:53:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:04 smithi183 ceph-mon[34219]: pgmap v364: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:53:05.128 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:05.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 -- 172.21.15.114:0/1615768455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 msgr2=0x7fa2901098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 --2- 172.21.15.114:0/1615768455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa2901098d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa278009900 tx=0x7fa27802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 -- 172.21.15.114:0/1615768455 shutdown_connections 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 --2- 172.21.15.114:0/1615768455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290109e10 0x7fa290110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 --2- 172.21.15.114:0/1615768455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa2901098d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 --2- 172.21.15.114:0/1615768455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa290103b80 0x7fa290103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 -- 172.21.15.114:0/1615768455 >> 172.21.15.114:0/1615768455 conn(0x7fa2900ff610 msgr2=0x7fa290101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 -- 172.21.15.114:0/1615768455 shutdown_connections 2024-09-21T06:53:05.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.767+0000 7fa294e00640 1 -- 172.21.15.114:0/1615768455 wait complete. 2024-09-21T06:53:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.768+0000 7fa294e00640 1 Processor -- start 2024-09-21T06:53:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 -- start start 2024-09-21T06:53:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290103b80 0x7fa29019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa29019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa290109e10 0x7fa29019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa290113b20 con 0x7fa290103b80 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901139a0 con 0x7fa2901094c0 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa294e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa290113ca0 con 0x7fa290109e10 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa285400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa29019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa28de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290103b80 0x7fa29019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa285400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa29019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53816/0 (socket says 172.21.15.114:53816) 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa285400640 1 -- 172.21.15.114:0/1596887721 learned_addr learned my addr 172.21.15.114:0/1596887721 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.769+0000 7fa28e800640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa290109e10 0x7fa29019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa285400640 1 -- 172.21.15.114:0/1596887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa290109e10 msgr2=0x7fa29019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa285400640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa290109e10 0x7fa29019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa285400640 1 -- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290103b80 msgr2=0x7fa29019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa285400640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290103b80 0x7fa29019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa285400640 1 -- 172.21.15.114:0/1596887721 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2901a0420 con 0x7fa2901094c0 2024-09-21T06:53:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa28de00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290103b80 0x7fa29019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.770+0000 7fa285400640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa29019aa50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa278005ab0 tx=0x7fa278003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.771+0000 7fa287400640 1 -- 172.21.15.114:0/1596887721 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27803f070 con 0x7fa2901094c0 2024-09-21T06:53:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.771+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2901a06b0 con 0x7fa2901094c0 2024-09-21T06:53:05.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.771+0000 7fa287400640 1 -- 172.21.15.114:0/1596887721 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa278031b50 con 0x7fa2901094c0 2024-09-21T06:53:05.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.771+0000 7fa287400640 1 -- 172.21.15.114:0/1596887721 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278035030 con 0x7fa2901094c0 2024-09-21T06:53:05.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.771+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901a6e70 con 0x7fa2901094c0 2024-09-21T06:53:05.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 Processor -- start 2024-09-21T06:53:05.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 -- start start 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06da3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06da39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06da3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55da06d1ad50 con 0x55da069fcf60 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da06d0a180 con 0x55da069dc280 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da06d0df90 con 0x55da069d40a0 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.774+0000 7fa287400640 1 -- 172.21.15.114:0/1596887721 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fa278042070 con 0x7fa2901094c0 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06da3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06da3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49206/0 (socket says 172.21.15.114:49206) 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 -- 172.21.15.114:0/2550091629 learned_addr learned my addr 172.21.15.114:0/2550091629 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa285400640 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06da3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28de00640 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06da39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 -- 172.21.15.114:0/2550091629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 msgr2=0x55da06da39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06da39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 -- 172.21.15.114:0/2550091629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 msgr2=0x55da06da3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06da3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.775+0000 7fa28e800640 1 -- 172.21.15.114:0/2550091629 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da06cd4a70 con 0x55da069fcf60 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.776+0000 7fa28e800640 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06da3490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa288011440 tx=0x7fa288011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e1c915c92bbf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.776+0000 7fa27f400640 1 -- 172.21.15.114:0/2550091629 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288003c30 con 0x55da069fcf60 2024-09-21T06:53:05.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.776+0000 7fa27f400640 1 -- 172.21.15.114:0/2550091629 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa288003dd0 con 0x55da069fcf60 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.776+0000 7fa27f400640 1 -- 172.21.15.114:0/2550091629 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800fa20 con 0x55da069fcf60 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2550091629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 msgr2=0x55da06da3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06da3490 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa288011440 tx=0x7fa288011900 comp rx=0 tx=0).stop 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2550091629 shutdown_connections 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06da3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06da39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2550091629 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06da3490 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2550091629 >> 172.21.15.114:0/2550091629 conn(0x55da069f1350 msgr2=0x55da06da1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2550091629 shutdown_connections 2024-09-21T06:53:05.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2550091629 wait complete. 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.777+0000 7fa287400640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa2740b6470 0x7fa2740b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.778+0000 7fa287400640 1 -- 172.21.15.114:0/1596887721 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa278107df0 con 0x7fa2901094c0 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.778+0000 7fa28de00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa2740b6470 0x7fa2740b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.778+0000 7fa29bbd9740 1 -- start start 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.778+0000 7fa29bbd9740 1 Processor -- start 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.778+0000 7fa28de00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa2740b6470 0x7fa2740b8980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55da06da2110 tx=0x7fa280005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069d40a0 0x55da06db3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06db3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dbf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55da06b6b5f0 con 0x55da069fcf60 2024-09-21T06:53:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da06ca5460 con 0x55da069d40a0 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da06db5fd0 con 0x55da069dc280 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa29bbd9740 10 client.0 _ll_register_callbacks cb 0x55da06cb5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa28de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069d40a0 0x55da06db3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa285400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06db3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa28e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dbf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa285400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06db3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51480/0 (socket says 172.21.15.114:51480) 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa285400640 1 -- 172.21.15.114:0/1434873172 learned_addr learned my addr 172.21.15.114:0/1434873172 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.779+0000 7fa28de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069d40a0 0x55da06db3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53840/0 (socket says 172.21.15.114:53840) 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa285400640 1 -- 172.21.15.114:0/1434873172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069d40a0 msgr2=0x55da06db3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa285400640 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069d40a0 0x55da06db3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa285400640 1 -- 172.21.15.114:0/1434873172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 msgr2=0x55da06dbf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa285400640 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dbf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa285400640 1 -- 172.21.15.114:0/1434873172 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da06a68540 con 0x55da069dc280 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa285400640 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06db3940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa278108090 tx=0x7fa27803b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6af7af7fac1ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa273e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa273e00640 1 -- 172.21.15.114:0/1434873172 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278049070 con 0x55da069dc280 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da06ca7a70 con 0x55da069dc280 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa273e00640 1 -- 172.21.15.114:0/1434873172 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa2780316a0 con 0x55da069dc280 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.780+0000 7fa273e00640 1 -- 172.21.15.114:0/1434873172 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2780c9930 con 0x55da069dc280 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 10 client.25228 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da06cc26f0 con 0x55da069dc280 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa273e00640 1 -- 172.21.15.114:0/1434873172 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55da06ca7a70 con 0x55da069dc280 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa273e00640 1 -- 172.21.15.114:0/1434873172 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa2780d21a0 con 0x55da069dc280 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 10 client.25228 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 10 client.25228 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 20 client.25228 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da069bb8e0 0x55da06dc5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 4 client.25228 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.781+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da06d1cca0 con 0x55da069bb8e0 2024-09-21T06:53:05.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.782+0000 7fa28de00640 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da069bb8e0 0x55da06dc5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.782+0000 7fa28de00640 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da069bb8e0 0x55da06dc5900 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c33a1f942cf93586 server_cookie=3698a62a65288316 in_seq=0 out_seq=0 2024-09-21T06:53:05.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.782+0000 7fa273e00640 10 client.25228 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:05.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.783+0000 7fa273e00640 1 -- 172.21.15.114:0/1434873172 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da06d1cca0 con 0x55da069bb8e0 2024-09-21T06:53:05.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.783+0000 7fa273e00640 10 client.25228 handle_command_reply: tid=0 2024-09-21T06:53:05.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.783+0000 7fa29bbd9740 1 client.25228 shutdown 2024-09-21T06:53:05.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 msgr2=0x55da06db3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06db3940 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa278108090 tx=0x7fa27803b640 comp rx=0 tx=0).stop 2024-09-21T06:53:05.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 shutdown_connections 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da069bb8e0 0x55da06dc5900 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dbf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06db3940 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa273e00640 0 client.25228 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/1434873172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069d40a0 0x55da06db3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 >> 172.21.15.114:0/1434873172 conn(0x55da069f1350 msgr2=0x55da06da4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 shutdown_connections 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 1 -- 172.21.15.114:0/1434873172 wait complete. 2024-09-21T06:53:05.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.784+0000 7fa29bbd9740 20 client.25228 trim_cache size 0 max 16384 2024-09-21T06:53:05.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.803+0000 7fa29bbd9740 1 Processor -- start 2024-09-21T06:53:05.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.803+0000 7fa29bbd9740 1 -- start start 2024-09-21T06:53:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.803+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06db33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.803+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06db3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.803+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55da06dc3c40 con 0x55da069fcf60 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da06dc4310 con 0x55da069dc280 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da06dc49e0 con 0x55da069d40a0 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06db33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06db33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51482/0 (socket says 172.21.15.114:51482) 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 -- 172.21.15.114:0/3468615900 learned_addr learned my addr 172.21.15.114:0/3468615900 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa285400640 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06db3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 -- 172.21.15.114:0/3468615900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 msgr2=0x55da06db3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06db3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 -- 172.21.15.114:0/3468615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 msgr2=0x55da06dae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.804+0000 7fa28de00640 1 -- 172.21.15.114:0/3468615900 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da06da42d0 con 0x55da069d40a0 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.805+0000 7fa28de00640 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06db33e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55da06db5a50 tx=0x7fa28001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a624da44e2e202 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.805+0000 7fa27ea00640 1 -- 172.21.15.114:0/3468615900 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280005020 con 0x55da069d40a0 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.805+0000 7fa27ea00640 1 -- 172.21.15.114:0/3468615900 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2800047a0 con 0x55da069d40a0 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.805+0000 7fa27ea00640 1 -- 172.21.15.114:0/3468615900 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28002b820 con 0x55da069d40a0 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 -- 172.21.15.114:0/3468615900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 msgr2=0x55da06db33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06db33e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55da06db5a50 tx=0x7fa28001b000 comp rx=0 tx=0).stop 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 -- 172.21.15.114:0/3468615900 shutdown_connections 2024-09-21T06:53:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069fcf60 0x55da06dae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069dc280 0x55da06db3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/3468615900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069d40a0 0x55da06db33e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 -- 172.21.15.114:0/3468615900 >> 172.21.15.114:0/3468615900 conn(0x55da069f1350 msgr2=0x55da06dc3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:05.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 -- 172.21.15.114:0/3468615900 shutdown_connections 2024-09-21T06:53:05.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 -- 172.21.15.114:0/3468615900 wait complete. 2024-09-21T06:53:05.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 -- start start 2024-09-21T06:53:05.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.806+0000 7fa29bbd9740 1 Processor -- start 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069d40a0 0x55da06dce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06dc7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069fcf60 0x55da06dc7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55da06dc5780 con 0x55da069d40a0 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da06dc50b0 con 0x55da069fcf60 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da06dc5e50 con 0x55da069dc280 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa29bbd9740 10 client.0 _ll_register_callbacks cb 0x7fa278005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa28e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069fcf60 0x55da06dc7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa285400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06dc7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa28de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069d40a0 0x55da06dce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa28e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069fcf60 0x55da06dc7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53854/0 (socket says 172.21.15.114:53854) 2024-09-21T06:53:05.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa28e800640 1 -- 172.21.15.114:0/2960642536 learned_addr learned my addr 172.21.15.114:0/2960642536 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.807+0000 7fa285400640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06dc7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51492/0 (socket says 172.21.15.114:51492) 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa285400640 1 -- 172.21.15.114:0/2960642536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069fcf60 msgr2=0x55da06dc7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa285400640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069fcf60 0x55da06dc7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa285400640 1 -- 172.21.15.114:0/2960642536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069d40a0 msgr2=0x55da06dce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa285400640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069d40a0 0x55da06dce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa285400640 1 -- 172.21.15.114:0/2960642536 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da06dc7ff0 con 0x55da069dc280 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa28de00640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069d40a0 0x55da06dce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa285400640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06dc7570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2780d7500 tx=0x7fa278003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7efccaae14c7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa273e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa273e00640 1 -- 172.21.15.114:0/2960642536 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278049070 con 0x55da069dc280 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2800047a0 con 0x55da069dc280 2024-09-21T06:53:05.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.808+0000 7fa273e00640 1 -- 172.21.15.114:0/2960642536 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa2780318b0 con 0x55da069dc280 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa273e00640 1 -- 172.21.15.114:0/2960642536 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2780c9200 con 0x55da069dc280 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 10 client.25234 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da06dcd970 con 0x55da069dc280 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa273e00640 1 -- 172.21.15.114:0/2960642536 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2800047a0 con 0x55da069dc280 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa273e00640 1 -- 172.21.15.114:0/2960642536 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa2780cccf0 con 0x55da069dc280 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 10 client.25234 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 10 client.25234 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 20 client.25234 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da06990c90 0x55da06e341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 4 client.25234 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "status"}) -- 0x55da06d18750 con 0x55da06990c90 2024-09-21T06:53:05.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.809+0000 7fa28de00640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da06990c90 0x55da06e341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa28de00640 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da06990c90 0x55da06e341a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51a3a174d4ece529 server_cookie=99636972b5b4bd20 in_seq=0 out_seq=0 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa273e00640 10 client.25234 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa273e00640 1 -- 172.21.15.114:0/2960642536 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55da06d18750 con 0x55da06990c90 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa273e00640 10 client.25234 handle_command_reply: tid=0 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 client.25234 shutdown 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 msgr2=0x55da06dc7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06dc7570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa2780d7500 tx=0x7fa278003f90 comp rx=0 tx=0).stop 2024-09-21T06:53:05.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 shutdown_connections 2024-09-21T06:53:05.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55da06990c90 0x55da06e341a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da069fcf60 0x55da06dc7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da069dc280 0x55da06dc7570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 --2- 172.21.15.114:0/2960642536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da069d40a0 0x55da06dce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 >> 172.21.15.114:0/2960642536 conn(0x55da069f1350 msgr2=0x55da06da85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.810+0000 7fa273e00640 0 client.25234 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 shutdown_connections 2024-09-21T06:53:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa29bbd9740 1 -- 172.21.15.114:0/2960642536 wait complete. 2024-09-21T06:53:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa29bbd9740 20 client.25234 trim_cache size 0 max 16384 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 3, 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24727, 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 84.222111076000004, 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:05.818 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:05.819 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:05.819 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 165.05628733699999 2024-09-21T06:53:05.819 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa2740b6470 msgr2=0x7fa2740b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa294e00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa2740b6470 0x7fa2740b8980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55da06da2110 tx=0x7fa280005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 msgr2=0x7fa29019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.811+0000 7fa294e00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa29019aa50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa278005ab0 tx=0x7fa278003b90 comp rx=0 tx=0).stop 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 shutdown_connections 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa290109e10 0x7fa29019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa2740b6470 0x7fa2740b8980 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2901094c0 0x7fa29019aa50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 --2- 172.21.15.114:0/1596887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa290103b80 0x7fa29019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:05.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 >> 172.21.15.114:0/1596887721 conn(0x7fa2900ff610 msgr2=0x7fa290101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:05.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 shutdown_connections 2024-09-21T06:53:05.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:05.812+0000 7fa294e00640 1 -- 172.21.15.114:0/1596887721 wait complete. 2024-09-21T06:53:06.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T06:53:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:06 smithi193 ceph-mon[34601]: pgmap v365: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:07.011 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:07.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:06 smithi114 ceph-mon[30654]: pgmap v365: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:06 smithi183 ceph-mon[34219]: pgmap v365: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 -- 172.21.15.114:0/3715496545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408103b60 msgr2=0x7fb408103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 --2- 172.21.15.114:0/3715496545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408103b60 0x7fb408103f70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009860 tx=0x7fb3f802a340 comp rx=0 tx=0).stop 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 -- 172.21.15.114:0/3715496545 shutdown_connections 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 --2- 172.21.15.114:0/3715496545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408109df0 0x7fb408110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 --2- 172.21.15.114:0/3715496545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081094a0 0x7fb4081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 --2- 172.21.15.114:0/3715496545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408103b60 0x7fb408103f70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 -- 172.21.15.114:0/3715496545 >> 172.21.15.114:0/3715496545 conn(0x7fb40807b850 msgr2=0x7fb40807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 -- 172.21.15.114:0/3715496545 shutdown_connections 2024-09-21T06:53:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.646+0000 7fb40f200640 1 -- 172.21.15.114:0/3715496545 wait complete. 2024-09-21T06:53:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.647+0000 7fb40f200640 1 Processor -- start 2024-09-21T06:53:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.647+0000 7fb40f200640 1 -- start start 2024-09-21T06:53:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 0x7fb40819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081094a0 0x7fb40819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408109df0 0x7fb4081a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb408113ab0 con 0x7fb408103b60 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb408113930 con 0x7fb4081094a0 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb408113c30 con 0x7fb408109df0 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 0x7fb40819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 0x7fb40819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49266/0 (socket says 172.21.15.114:49266) 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb3ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081094a0 0x7fb40819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 -- 172.21.15.114:0/2049591025 learned_addr learned my addr 172.21.15.114:0/2049591025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40d200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408109df0 0x7fb4081a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 -- 172.21.15.114:0/2049591025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408109df0 msgr2=0x7fb4081a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408109df0 0x7fb4081a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 -- 172.21.15.114:0/2049591025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081094a0 msgr2=0x7fb40819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081094a0 0x7fb40819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb407e00640 1 -- 172.21.15.114:0/2049591025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4081a2630 con 0x7fb408103b60 2024-09-21T06:53:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.648+0000 7fb40d200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408109df0 0x7fb4081a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:53:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.649+0000 7fb407e00640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 0x7fb40819c6c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8035040 tx=0x7fb3f8032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.649+0000 7fb405600640 1 -- 172.21.15.114:0/2049591025 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8037040 con 0x7fb408103b60 2024-09-21T06:53:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.649+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4081a28c0 con 0x7fb408103b60 2024-09-21T06:53:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.649+0000 7fb405600640 1 -- 172.21.15.114:0/2049591025 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f8041070 con 0x7fb408103b60 2024-09-21T06:53:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.649+0000 7fb405600640 1 -- 172.21.15.114:0/2049591025 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80035f0 con 0x7fb408103b60 2024-09-21T06:53:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.649+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4081a2d40 con 0x7fb408103b60 2024-09-21T06:53:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.652+0000 7fb416019740 1 Processor -- start 2024-09-21T06:53:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 -- start start 2024-09-21T06:53:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb405600640 1 -- 172.21.15.114:0/2049591025 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fb3f8004050 con 0x7fb408103b60 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fe40a0 0x55c5d23b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fec280 0x55c5d23b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c5d232ad50 con 0x55c5d200cf60 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c5d231a180 con 0x55c5d1fe40a0 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb416019740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c5d231df90 con 0x55c5d1fec280 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb40d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb40d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49282/0 (socket says 172.21.15.114:49282) 2024-09-21T06:53:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb407e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fe40a0 0x55c5d23b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb40d200640 1 -- 172.21.15.114:0/4017669779 learned_addr learned my addr 172.21.15.114:0/4017669779 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.653+0000 7fb3ff400640 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fec280 0x55c5d23b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb407e00640 1 -- 172.21.15.114:0/4017669779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fec280 msgr2=0x55c5d23b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb407e00640 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fec280 0x55c5d23b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb407e00640 1 -- 172.21.15.114:0/4017669779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 msgr2=0x55c5d23b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb407e00640 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb407e00640 1 -- 172.21.15.114:0/4017669779 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d22e4a70 con 0x55c5d1fe40a0 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb407e00640 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fe40a0 0x55c5d23b39d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8104ac0 tx=0x7fb3f810d510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfdf4567d711adc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb3fd600640 1 -- 172.21.15.114:0/4017669779 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8107b00 con 0x55c5d1fe40a0 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb3fd600640 1 -- 172.21.15.114:0/4017669779 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f8114030 con 0x55c5d1fe40a0 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.654+0000 7fb3fd600640 1 -- 172.21.15.114:0/4017669779 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f81196f0 con 0x55c5d1fe40a0 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.655+0000 7fb405600640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6560 0x7fb3ec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.655+0000 7fb405600640 1 -- 172.21.15.114:0/2049591025 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb3f8107510 con 0x7fb408103b60 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.655+0000 7fb40d200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6560 0x7fb3ec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.655+0000 7fb416019740 1 -- 172.21.15.114:0/4017669779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fe40a0 msgr2=0x55c5d23b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.655+0000 7fb416019740 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fe40a0 0x55c5d23b39d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8104ac0 tx=0x7fb3f810d510 comp rx=0 tx=0).stop 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 -- 172.21.15.114:0/4017669779 shutdown_connections 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fec280 0x55c5d23b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fe40a0 0x55c5d23b39d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 --2- 172.21.15.114:0/4017669779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 -- 172.21.15.114:0/4017669779 >> 172.21.15.114:0/4017669779 conn(0x55c5d2001350 msgr2=0x55c5d23b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb40d200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6560 0x7fb3ec0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb400005c40 tx=0x7fb400009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 -- 172.21.15.114:0/4017669779 shutdown_connections 2024-09-21T06:53:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 -- 172.21.15.114:0/4017669779 wait complete. 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.656+0000 7fb416019740 1 -- start start 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 Processor -- start 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fec280 0x55c5d23bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d200cf60 0x55c5d23cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c5d217b5f0 con 0x55c5d1fec280 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c5d22b5460 con 0x55c5d200cf60 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23c1a30 con 0x55c5d1fe40a0 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb416019740 10 client.0 _ll_register_callbacks cb 0x55c5d22c5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:07.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb407e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb407e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23bf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51534/0 (socket says 172.21.15.114:51534) 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.657+0000 7fb407e00640 1 -- 172.21.15.114:0/1171035049 learned_addr learned my addr 172.21.15.114:0/1171035049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb3ff400640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fec280 0x55c5d23bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb40d200640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d200cf60 0x55c5d23cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb407e00640 1 -- 172.21.15.114:0/1171035049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d200cf60 msgr2=0x55c5d23cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb407e00640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d200cf60 0x55c5d23cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb407e00640 1 -- 172.21.15.114:0/1171035049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fec280 msgr2=0x55c5d23bf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb407e00640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fec280 0x55c5d23bf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb407e00640 1 -- 172.21.15.114:0/1171035049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d2078540 con 0x55c5d1fe40a0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb3ff400640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fec280 0x55c5d23bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb407e00640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23bf060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80c9780 tx=0x7fb3f8030d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802c7b3fcfef6ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb3f6000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb3f6000640 1 -- 172.21.15.114:0/1171035049 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80c9800 con 0x55c5d1fe40a0 2024-09-21T06:53:07.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb3f6000640 1 -- 172.21.15.114:0/1171035049 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb3f80c9420 con 0x55c5d1fe40a0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.658+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5d22b7a70 con 0x55c5d1fe40a0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb3f6000640 1 -- 172.21.15.114:0/1171035049 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f80cc920 con 0x55c5d1fe40a0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 10 client.25246 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5d22d26f0 con 0x55c5d1fe40a0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb3f6000640 1 -- 172.21.15.114:0/1171035049 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5d22b7a70 con 0x55c5d1fe40a0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb3f6000640 1 -- 172.21.15.114:0/1171035049 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb3f8004340 con 0x55c5d1fe40a0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 10 client.25246 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 10 client.25246 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 20 client.25246 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fcb8e0 0x55c5d23d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 4 client.25246 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.659+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5d232cca0 con 0x55c5d1fcb8e0 2024-09-21T06:53:07.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.660+0000 7fb3ff400640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fcb8e0 0x55c5d23d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.660+0000 7fb3ff400640 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fcb8e0 0x55c5d23d0f40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21067bdd5d428a0e server_cookie=1c39fba7d48a152b in_seq=0 out_seq=0 2024-09-21T06:53:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.660+0000 7fb3f6000640 10 client.25246 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb3f6000640 1 -- 172.21.15.114:0/1171035049 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5d232cca0 con 0x55c5d1fcb8e0 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb3f6000640 10 client.25246 handle_command_reply: tid=0 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb416019740 1 client.25246 shutdown 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 msgr2=0x55c5d23bf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb416019740 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23bf060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80c9780 tx=0x7fb3f8030d50 comp rx=0 tx=0).stop 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 shutdown_connections 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.661+0000 7fb416019740 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fcb8e0 0x55c5d23d0f40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d200cf60 0x55c5d23cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fec280 0x55c5d23bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 1 --2- 172.21.15.114:0/1171035049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23bf060 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb3f6000640 0 client.25246 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 >> 172.21.15.114:0/1171035049 conn(0x55c5d2001350 msgr2=0x55c5d23b6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 shutdown_connections 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 1 -- 172.21.15.114:0/1171035049 wait complete. 2024-09-21T06:53:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.662+0000 7fb416019740 20 client.25246 trim_cache size 0 max 16384 2024-09-21T06:53:07.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.680+0000 7fb416019740 1 Processor -- start 2024-09-21T06:53:07.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 -- start start 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23cf280 con 0x55c5d200cf60 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23cf950 con 0x55c5d1fec280 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb416019740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23d0020 con 0x55c5d1fe40a0 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb40d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb40d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49290/0 (socket says 172.21.15.114:49290) 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb40d200640 1 -- 172.21.15.114:0/455447510 learned_addr learned my addr 172.21.15.114:0/455447510 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb3ff400640 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.681+0000 7fb407e00640 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb40d200640 1 -- 172.21.15.114:0/455447510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 msgr2=0x55c5d23cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb40d200640 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb40d200640 1 -- 172.21.15.114:0/455447510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 msgr2=0x55c5d23cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb40d200640 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb40d200640 1 -- 172.21.15.114:0/455447510 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d23b5340 con 0x55c5d200cf60 2024-09-21T06:53:07.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb40d200640 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b9fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb4000109b0 tx=0x7fb400016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa2503e21fe4c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:07.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb3fcc00640 1 -- 172.21.15.114:0/455447510 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400004b20 con 0x55c5d200cf60 2024-09-21T06:53:07.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb3fcc00640 1 -- 172.21.15.114:0/455447510 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40001fde0 con 0x55c5d200cf60 2024-09-21T06:53:07.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.682+0000 7fb3fcc00640 1 -- 172.21.15.114:0/455447510 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400005460 con 0x55c5d200cf60 2024-09-21T06:53:07.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 -- 172.21.15.114:0/455447510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 msgr2=0x55c5d23b9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b9fa0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb4000109b0 tx=0x7fb400016000 comp rx=0 tx=0).stop 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 -- 172.21.15.114:0/455447510 shutdown_connections 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d200cf60 0x55c5d23b9fa0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 --2- 172.21.15.114:0/455447510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d1fe40a0 0x55c5d23cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 -- 172.21.15.114:0/455447510 >> 172.21.15.114:0/455447510 conn(0x55c5d2001350 msgr2=0x55c5d23ac010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 -- 172.21.15.114:0/455447510 shutdown_connections 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.683+0000 7fb416019740 1 -- 172.21.15.114:0/455447510 wait complete. 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 -- start start 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 Processor -- start 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 0x55c5d23d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d200cf60 0x55c5d23d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23d0dc0 con 0x55c5d1fe40a0 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23d06f0 con 0x55c5d1fec280 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.684+0000 7fb416019740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c5d23d1490 con 0x55c5d200cf60 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb416019740 10 client.0 _ll_register_callbacks cb 0x7fb3f810d2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 0x55c5d23d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 0x55c5d23d96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49304/0 (socket says 172.21.15.114:49304) 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 -- 172.21.15.114:0/4204775366 learned_addr learned my addr 172.21.15.114:0/4204775366 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb3ff400640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb40d200640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d200cf60 0x55c5d23d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 -- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d200cf60 msgr2=0x55c5d23d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d200cf60 0x55c5d23d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 -- 172.21.15.114:0/4204775366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 msgr2=0x55c5d23d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23d2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 -- 172.21.15.114:0/4204775366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d23bfa00 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb407e00640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 0x55c5d23d96e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80305e0 tx=0x7fb3f8004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4feff61d664f959f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.685+0000 7fb3f6000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb3f6000640 1 -- 172.21.15.114:0/4204775366 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80ccb00 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb3f6000640 1 -- 172.21.15.114:0/4204775366 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb3f80d6740 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40001fde0 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb3f6000640 1 -- 172.21.15.114:0/4204775366 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3f80ccf80 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb416019740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5d23bfa00 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb3f6000640 1 -- 172.21.15.114:0/4204775366 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40001fde0 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.686+0000 7fb3f6000640 1 -- 172.21.15.114:0/4204775366 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb3f80c8b00 con 0x55c5d1fe40a0 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb416019740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:07.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb416019740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb416019740 20 client.15246 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb416019740 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fa0c90 0x55c5d243f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb416019740 4 client.15246 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "status"}) -- 0x55c5d2328750 con 0x55c5d1fa0c90 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.687+0000 7fb3ff400640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fa0c90 0x55c5d243f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb3f6000640 10 client.15246 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb3ff400640 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fa0c90 0x55c5d243f260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b58bb7aaf962e33 server_cookie=8d030591cebbae64 in_seq=0 out_seq=0 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb3f6000640 1 -- 172.21.15.114:0/4204775366 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c5d2328750 con 0x55c5d1fa0c90 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb3f6000640 10 client.15246 handle_command_reply: tid=0 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 client.15246 shutdown 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 msgr2=0x55c5d23d96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 0x55c5d23d96e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80305e0 tx=0x7fb3f8004570 comp rx=0 tx=0).stop 2024-09-21T06:53:07.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 shutdown_connections 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c5d1fa0c90 0x55c5d243f260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5d200cf60 0x55c5d23d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c5d1fec280 0x55c5d23d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 --2- 172.21.15.114:0/4204775366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c5d1fe40a0 0x55c5d23d96e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 >> 172.21.15.114:0/4204775366 conn(0x55c5d2001350 msgr2=0x55c5d23b9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.688+0000 7fb3f6000640 0 client.15246 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 shutdown_connections 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb416019740 1 -- 172.21.15.114:0/4204775366 wait complete. 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb416019740 20 client.15246 trim_cache size 0 max 16384 2024-09-21T06:53:07.693 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 2, 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24742, 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 87.109626227999996, 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 161.71173853100001 2024-09-21T06:53:07.694 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6560 msgr2=0x7fb3ec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6560 0x7fb3ec0b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb400005c40 tx=0x7fb400009500 comp rx=0 tx=0).stop 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 msgr2=0x7fb40819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 0x7fb40819c6c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8035040 tx=0x7fb3f8032d50 comp rx=0 tx=0).stop 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 shutdown_connections 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb408109df0 0x7fb4081a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.689+0000 7fb40f200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6560 0x7fb3ec0b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.690+0000 7fb40f200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081094a0 0x7fb40819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.690+0000 7fb40f200640 1 --2- 172.21.15.114:0/2049591025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb408103b60 0x7fb40819c6c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:07.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.690+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 >> 172.21.15.114:0/2049591025 conn(0x7fb40807b850 msgr2=0x7fb408107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:07.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.690+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 shutdown_connections 2024-09-21T06:53:07.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:07.690+0000 7fb40f200640 1 -- 172.21.15.114:0/2049591025 wait complete. 2024-09-21T06:53:08.581 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T06:53:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:08 smithi114 ceph-mon[30654]: pgmap v366: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:53:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:08 smithi193 ceph-mon[34601]: pgmap v366: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:53:08.953 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:09.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:08 smithi183 ceph-mon[34219]: pgmap v366: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.575+0000 7f7096400640 1 -- 172.21.15.114:0/3580315817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709010be40 msgr2=0x7f7090112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.575+0000 7f7096400640 1 --2- 172.21.15.114:0/3580315817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709010be40 0x7f7090112670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7088009ca0 tx=0x7f708802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 -- 172.21.15.114:0/3580315817 shutdown_connections 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 --2- 172.21.15.114:0/3580315817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f709010be40 0x7f7090112670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 --2- 172.21.15.114:0/3580315817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f709010b4f0 0x7f709010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 --2- 172.21.15.114:0/3580315817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7090105bb0 0x7f7090105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 -- 172.21.15.114:0/3580315817 >> 172.21.15.114:0/3580315817 conn(0x7f7090101680 msgr2=0x7f7090103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 -- 172.21.15.114:0/3580315817 shutdown_connections 2024-09-21T06:53:09.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.576+0000 7f7096400640 1 -- 172.21.15.114:0/3580315817 wait complete. 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 Processor -- start 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 -- start start 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 0x7f709019c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f709010b4f0 0x7f709019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709010be40 0x7f70901a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7090115b00 con 0x7f7090105bb0 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7090115980 con 0x7f709010be40 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.577+0000 7f7096400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7090115c80 con 0x7f709010b4f0 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 0x7f709019c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 0x7f709019c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49320/0 (socket says 172.21.15.114:49320) 2024-09-21T06:53:09.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 -- 172.21.15.114:0/327706178 learned_addr learned my addr 172.21.15.114:0/327706178 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f708fe00640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f709010b4f0 0x7f709019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095a00640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709010be40 0x7f70901a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 -- 172.21.15.114:0/327706178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f709010b4f0 msgr2=0x7f709019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f709010b4f0 0x7f709019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 -- 172.21.15.114:0/327706178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709010be40 msgr2=0x7f70901a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709010be40 0x7f70901a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 -- 172.21.15.114:0/327706178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70901a23e0 con 0x7f7090105bb0 2024-09-21T06:53:09.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.578+0000 7f7095000640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 0x7f709019c470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f708400e9f0 tx=0x7f708400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:09.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.579+0000 7f708d600640 1 -- 172.21.15.114:0/327706178 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708400cd80 con 0x7f7090105bb0 2024-09-21T06:53:09.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.579+0000 7f708d600640 1 -- 172.21.15.114:0/327706178 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70840108e0 con 0x7f7090105bb0 2024-09-21T06:53:09.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.579+0000 7f708d600640 1 -- 172.21.15.114:0/327706178 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084018980 con 0x7f7090105bb0 2024-09-21T06:53:09.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.579+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70901a26d0 con 0x7f7090105bb0 2024-09-21T06:53:09.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.579+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901a8e30 con 0x7f7090105bb0 2024-09-21T06:53:09.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.582+0000 7f709e6d2740 1 Processor -- start 2024-09-21T06:53:09.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.582+0000 7f709e6d2740 1 -- start start 2024-09-21T06:53:09.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.582+0000 7f708d600640 1 -- 172.21.15.114:0/327706178 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f7084010430 con 0x7f7090105bb0 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe90fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe90fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f709e6d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55efe9071d50 con 0x55efe8d33280 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f709e6d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efe9061180 con 0x55efe8d2b0a0 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f709e6d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efe9064f90 con 0x55efe8d53f60 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe90fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe90fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f708fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe90fa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51596/0 (socket says 172.21.15.114:51596) 2024-09-21T06:53:09.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 -- 172.21.15.114:0/3854614200 learned_addr learned my addr 172.21.15.114:0/3854614200 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:09.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 -- 172.21.15.114:0/3854614200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 msgr2=0x55efe90fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe90fabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 -- 172.21.15.114:0/3854614200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 msgr2=0x55efe90fb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90fb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.583+0000 7f7095a00640 1 -- 172.21.15.114:0/3854614200 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe902ba70 con 0x55efe8d53f60 2024-09-21T06:53:09.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.584+0000 7f7095a00640 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe90fa670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f708802f4c0 tx=0x7f7088009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c764c8e0b5f04f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:09.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.584+0000 7f7082000640 1 -- 172.21.15.114:0/3854614200 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088031cf0 con 0x55efe8d53f60 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.584+0000 7f7082000640 1 -- 172.21.15.114:0/3854614200 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7088036030 con 0x55efe8d53f60 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.584+0000 7f7082000640 1 -- 172.21.15.114:0/3854614200 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088004240 con 0x55efe8d53f60 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.584+0000 7f708d600640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70780b65e0 0x7f70780b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f708d600640 1 -- 172.21.15.114:0/327706178 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f70840e7b10 con 0x7f7090105bb0 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f7095000640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70780b65e0 0x7f70780b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 -- 172.21.15.114:0/3854614200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 msgr2=0x55efe90fa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe90fa670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f708802f4c0 tx=0x7f7088009800 comp rx=0 tx=0).stop 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f7095000640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70780b65e0 0x7f70780b8af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70840e5120 tx=0x7f70840eb4a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 -- 172.21.15.114:0/3854614200 shutdown_connections 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe90fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3854614200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe90fa670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 -- 172.21.15.114:0/3854614200 >> 172.21.15.114:0/3854614200 conn(0x55efe8d48350 msgr2=0x55efe90f8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 -- 172.21.15.114:0/3854614200 shutdown_connections 2024-09-21T06:53:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.585+0000 7f709e6d2740 1 -- 172.21.15.114:0/3854614200 wait complete. 2024-09-21T06:53:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.586+0000 7f709e6d2740 1 -- start start 2024-09-21T06:53:09.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.586+0000 7f709e6d2740 1 Processor -- start 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 0x55efe91061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d33280 0x55efe9106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe9111f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55efe8ec25f0 con 0x55efe8d2b0a0 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efe8ffc460 con 0x55efe8d53f60 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efe9108c70 con 0x55efe8d33280 2024-09-21T06:53:09.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f709e6d2740 10 client.0 _ll_register_callbacks cb 0x55efe900c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f7095a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 0x55efe91061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f708fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d33280 0x55efe9106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f7095a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 0x55efe91061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49334/0 (socket says 172.21.15.114:49334) 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f7095a00640 1 -- 172.21.15.114:0/465771395 learned_addr learned my addr 172.21.15.114:0/465771395 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f7095a00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe9111f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f7095a00640 1 -- 172.21.15.114:0/465771395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d33280 msgr2=0x55efe9106730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.587+0000 7f7095a00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d33280 0x55efe9106730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7095a00640 1 -- 172.21.15.114:0/465771395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 msgr2=0x55efe9111f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7095a00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe9111f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7095a00640 1 -- 172.21.15.114:0/465771395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe8dbf540 con 0x55efe8d2b0a0 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f708fe00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d33280 0x55efe9106730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7095a00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 0x55efe91061f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70880096c0 tx=0x7f7088003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94da1bc2a342f1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:09.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7076a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7076a00640 1 -- 172.21.15.114:0/465771395 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088057060 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7076a00640 1 -- 172.21.15.114:0/465771395 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7088031650 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efe8ffea70 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f7076a00640 1 -- 172.21.15.114:0/465771395 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f708805e050 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f709e6d2740 10 client.15258 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.588+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55efe90196f0 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f7076a00640 1 -- 172.21.15.114:0/465771395 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55efe8ffea70 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f7076a00640 1 -- 172.21.15.114:0/465771395 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7088043070 con 0x55efe8d2b0a0 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f709e6d2740 10 client.15258 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f709e6d2740 10 client.15258 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f709e6d2740 20 client.15258 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f709e6d2740 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8d128e0 0x55efe9118170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f709e6d2740 4 client.15258 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:09.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.589+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efe9073ca0 con 0x55efe8d128e0 2024-09-21T06:53:09.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.590+0000 7f708fe00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8d128e0 0x55efe9118170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.590+0000 7f708fe00640 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8d128e0 0x55efe9118170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=408de9b947a0834a server_cookie=38c388d4f78b4a18 in_seq=0 out_seq=0 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.590+0000 7f7076a00640 10 client.15258 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f7076a00640 1 -- 172.21.15.114:0/465771395 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efe9073ca0 con 0x55efe8d128e0 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f7076a00640 10 client.15258 handle_command_reply: tid=0 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 client.15258 shutdown 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 msgr2=0x55efe91061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 0x55efe91061f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f70880096c0 tx=0x7f7088003d70 comp rx=0 tx=0).stop 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 shutdown_connections 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8d128e0 0x55efe9118170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe9111f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d33280 0x55efe9106730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 --2- 172.21.15.114:0/465771395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d2b0a0 0x55efe91061f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f7076a00640 0 client.15258 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.591+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 >> 172.21.15.114:0/465771395 conn(0x55efe8d48350 msgr2=0x55efe90fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.592+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 shutdown_connections 2024-09-21T06:53:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.592+0000 7f709e6d2740 1 -- 172.21.15.114:0/465771395 wait complete. 2024-09-21T06:53:09.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.592+0000 7f709e6d2740 20 client.15258 trim_cache size 0 max 16384 2024-09-21T06:53:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.616+0000 7f709e6d2740 1 Processor -- start 2024-09-21T06:53:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.616+0000 7f709e6d2740 1 -- start start 2024-09-21T06:53:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe9100790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90f8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe9111f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f709e6d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55efe91164b0 con 0x55efe8d33280 2024-09-21T06:53:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f709e6d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efe9116b80 con 0x55efe8d2b0a0 2024-09-21T06:53:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f709e6d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efe9117250 con 0x55efe8d53f60 2024-09-21T06:53:09.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90f8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49342/0 (socket says 172.21.15.114:49342) 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 -- 172.21.15.114:0/3075522720 learned_addr learned my addr 172.21.15.114:0/3075522720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f7095a00640 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe9100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f7095a00640 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe9111f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 -- 172.21.15.114:0/3075522720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 msgr2=0x55efe9111f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe9111f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 -- 172.21.15.114:0/3075522720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 msgr2=0x55efe9100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe9100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.617+0000 7f708fe00640 1 -- 172.21.15.114:0/3075522720 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe91072b0 con 0x55efe8d33280 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.618+0000 7f708fe00640 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90f8500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f707c005fd0 tx=0x7f707c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c5836387567cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.618+0000 7f7081600640 1 -- 172.21.15.114:0/3075522720 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c004220 con 0x55efe8d33280 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.618+0000 7f7081600640 1 -- 172.21.15.114:0/3075522720 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c0043c0 con 0x55efe8d33280 2024-09-21T06:53:09.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.618+0000 7f7081600640 1 -- 172.21.15.114:0/3075522720 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0047a0 con 0x55efe8d33280 2024-09-21T06:53:09.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 -- 172.21.15.114:0/3075522720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 msgr2=0x55efe90f8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90f8500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f707c005fd0 tx=0x7f707c00b410 comp rx=0 tx=0).stop 2024-09-21T06:53:09.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 -- 172.21.15.114:0/3075522720 shutdown_connections 2024-09-21T06:53:09.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d53f60 0x55efe9111f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe90f8500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3075522720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d2b0a0 0x55efe9100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 -- 172.21.15.114:0/3075522720 >> 172.21.15.114:0/3075522720 conn(0x55efe8d48350 msgr2=0x55efe90f7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 -- 172.21.15.114:0/3075522720 shutdown_connections 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.619+0000 7f709e6d2740 1 -- 172.21.15.114:0/3075522720 wait complete. 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 -- start start 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 Processor -- start 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d2b0a0 0x55efe9120df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe911a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe911aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55efe9117ff0 con 0x55efe8d33280 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efe9117920 con 0x55efe8d53f60 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efe91186c0 con 0x55efe8d2b0a0 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.620+0000 7f709e6d2740 10 client.0 _ll_register_callbacks cb 0x7f708802f410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:09.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f708fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe911a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe911aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f708fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe911a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49352/0 (socket says 172.21.15.114:49352) 2024-09-21T06:53:09.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe911aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53934/0 (socket says 172.21.15.114:53934) 2024-09-21T06:53:09.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f708fe00640 1 -- 172.21.15.114:0/3166481188 learned_addr learned my addr 172.21.15.114:0/3166481188 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:09.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d2b0a0 0x55efe9120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 -- 172.21.15.114:0/3166481188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d2b0a0 msgr2=0x55efe9120df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d2b0a0 0x55efe9120df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 -- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 msgr2=0x55efe911a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:09.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe911a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f7095a00640 1 -- 172.21.15.114:0/3166481188 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe911fa30 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.621+0000 7f708fe00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe911a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f7095a00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe911aa70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f708802ade0 tx=0x7f7088003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1e78f0c2b30aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f7076a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f7076a00640 1 -- 172.21.15.114:0/3166481188 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7088004b40 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707c0043c0 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f7076a00640 1 -- 172.21.15.114:0/3166481188 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7088004ce0 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f7076a00640 1 -- 172.21.15.114:0/3166481188 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7088031450 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f709e6d2740 10 client.25265 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.622+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55efe91837c0 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f7076a00640 1 -- 172.21.15.114:0/3166481188 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707c0043c0 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f7076a00640 1 -- 172.21.15.114:0/3166481188 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7088043070 con 0x55efe8d53f60 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f709e6d2740 10 client.25265 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f709e6d2740 10 client.25265 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:53:09.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f709e6d2740 20 client.25265 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8ce7c90 0x55efe9186f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f709e6d2740 4 client.25265 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "status"}) -- 0x55efe906f750 con 0x55efe8ce7c90 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.623+0000 7f708fe00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8ce7c90 0x55efe9186f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f7076a00640 10 client.25265 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f708fe00640 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8ce7c90 0x55efe9186f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=400243812933ce8b server_cookie=398ce039b9773d19 in_seq=0 out_seq=0 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f7076a00640 1 -- 172.21.15.114:0/3166481188 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55efe906f750 con 0x55efe8ce7c90 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f7076a00640 10 client.25265 handle_command_reply: tid=0 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 client.25265 shutdown 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 msgr2=0x55efe911aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe911aa70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f708802ade0 tx=0x7f7088003cd0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 shutdown_connections 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55efe8ce7c90 0x55efe9186f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efe8d53f60 0x55efe911aa70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efe8d33280 0x55efe911a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 --2- 172.21.15.114:0/3166481188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efe8d2b0a0 0x55efe9120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 >> 172.21.15.114:0/3166481188 conn(0x55efe8d48350 msgr2=0x55efe9107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.624+0000 7f7076a00640 0 client.25265 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.625+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 shutdown_connections 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.625+0000 7f709e6d2740 1 -- 172.21.15.114:0/3166481188 wait complete. 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.625+0000 7f709e6d2740 20 client.25265 trim_cache size 0 max 16384 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:09.629 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 1, 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24760, 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 90.057169810000005, 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 156.741138525 2024-09-21T06:53:09.630 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70780b65e0 msgr2=0x7f70780b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70780b65e0 0x7f70780b8af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f70840e5120 tx=0x7f70840eb4a0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 msgr2=0x7f709019c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 0x7f709019c470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f708400e9f0 tx=0x7f708400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 shutdown_connections 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f70780b65e0 0x7f70780b8af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f709010be40 0x7f70901a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f709010b4f0 0x7f709019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 --2- 172.21.15.114:0/327706178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7090105bb0 0x7f709019c470 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:09.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 >> 172.21.15.114:0/327706178 conn(0x7f7090101680 msgr2=0x7f7090108aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:09.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 shutdown_connections 2024-09-21T06:53:09.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:09.626+0000 7f7096400640 1 -- 172.21.15.114:0/327706178 wait complete. 2024-09-21T06:53:10.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:53:10.794 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:10 smithi193 ceph-mon[34601]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:10.917 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:10 smithi183 ceph-mon[34219]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:10.931 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:10 smithi114 ceph-mon[30654]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 -- 172.21.15.114:0/591375987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf81092d0 msgr2=0x7f9bf81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 --2- 172.21.15.114:0/591375987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf81092d0 0x7f9bf81096e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009900 tx=0x7f9be402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 -- 172.21.15.114:0/591375987 shutdown_connections 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 --2- 172.21.15.114:0/591375987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8109c20 0x7f9bf8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 --2- 172.21.15.114:0/591375987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf81092d0 0x7f9bf81096e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 --2- 172.21.15.114:0/591375987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bf8103990 0x7f9bf8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 -- 172.21.15.114:0/591375987 >> 172.21.15.114:0/591375987 conn(0x7f9bf80ff3e0 msgr2=0x7f9bf8101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 -- 172.21.15.114:0/591375987 shutdown_connections 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.458+0000 7f9bfd200640 1 -- 172.21.15.114:0/591375987 wait complete. 2024-09-21T06:53:11.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 Processor -- start 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 -- start start 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 0x7f9bf819c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bf81092d0 0x7f9bf819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8109c20 0x7f9bf81a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf81138e0 con 0x7f9bf8103990 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8113760 con 0x7f9bf81092d0 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.460+0000 7f9bfd200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8113a60 con 0x7f9bf8109c20 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 0x7f9bf819c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 0x7f9bf819c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49364/0 (socket says 172.21.15.114:49364) 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 -- 172.21.15.114:0/800306161 learned_addr learned my addr 172.21.15.114:0/800306161 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf6a00640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bf81092d0 0x7f9bf819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8109c20 0x7f9bf81a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 -- 172.21.15.114:0/800306161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8109c20 msgr2=0x7f9bf81a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8109c20 0x7f9bf81a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 -- 172.21.15.114:0/800306161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bf81092d0 msgr2=0x7f9bf819cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bf81092d0 0x7f9bf819cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.461+0000 7f9bf7400640 1 -- 172.21.15.114:0/800306161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf81a24a0 con 0x7f9bf8103990 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.462+0000 7f9bf7400640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 0x7f9bf819c530 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e9f0 tx=0x7f9be800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.462+0000 7f9befe00640 1 -- 172.21.15.114:0/800306161 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be800cd80 con 0x7f9bf8103990 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.462+0000 7f9befe00640 1 -- 172.21.15.114:0/800306161 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be80108e0 con 0x7f9bf8103990 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.462+0000 7f9befe00640 1 -- 172.21.15.114:0/800306161 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8018980 con 0x7f9bf8103990 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.462+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf81a2790 con 0x7f9bf8103990 2024-09-21T06:53:11.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.462+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf81a8ef0 con 0x7f9bf8103990 2024-09-21T06:53:11.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 Processor -- start 2024-09-21T06:53:11.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 -- start start 2024-09-21T06:53:11.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9befe00640 1 -- 172.21.15.114:0/800306161 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f9be8010430 con 0x7f9bf8103990 2024-09-21T06:53:11.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9db8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9db8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9db90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5605a9d2fd50 con 0x5605a99e90a0 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605a9d1f180 con 0x5605a99f1280 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9c055bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605a9d22f90 con 0x5605a9a11f60 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.466+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9db8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9db8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49370/0 (socket says 172.21.15.114:49370) 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 -- 172.21.15.114:0/949926472 learned_addr learned my addr 172.21.15.114:0/949926472 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9db8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf6a00640 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9db90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 -- 172.21.15.114:0/949926472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 msgr2=0x5605a9db8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9db8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 -- 172.21.15.114:0/949926472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 msgr2=0x5605a9db90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9db90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.467+0000 7f9bf7400640 1 -- 172.21.15.114:0/949926472 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a9ce9a70 con 0x5605a99e90a0 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9bf7400640 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9db8bb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9be80ebcc0 tx=0x7f9be80f05e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6caf5433255b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:11.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9befe00640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bdc0b6560 0x7f9bdc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9becc00640 1 -- 172.21.15.114:0/949926472 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80ed050 con 0x5605a99e90a0 2024-09-21T06:53:11.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bdc0b6560 0x7f9bdc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9becc00640 1 -- 172.21.15.114:0/949926472 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be80f9070 con 0x5605a99e90a0 2024-09-21T06:53:11.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9becc00640 1 -- 172.21.15.114:0/949926472 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80ad230 con 0x5605a99e90a0 2024-09-21T06:53:11.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.468+0000 7f9befe00640 1 -- 172.21.15.114:0/800306161 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9be80e7b90 con 0x7f9bf8103990 2024-09-21T06:53:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 -- 172.21.15.114:0/949926472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 msgr2=0x5605a9db8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9db8bb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9be80ebcc0 tx=0x7f9be80f05e0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bdc0b6560 0x7f9bdc0b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005fd0 tx=0x7f9bf00096a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 -- 172.21.15.114:0/949926472 shutdown_connections 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9db90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9db8bb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 --2- 172.21.15.114:0/949926472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9db8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 -- 172.21.15.114:0/949926472 >> 172.21.15.114:0/949926472 conn(0x5605a9a06350 msgr2=0x5605a9db6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 -- 172.21.15.114:0/949926472 shutdown_connections 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.469+0000 7f9c055bd740 1 -- 172.21.15.114:0/949926472 wait complete. 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.470+0000 7f9c055bd740 1 -- start start 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.470+0000 7f9c055bd740 1 Processor -- start 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dc42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99f1280 0x5605a9dc4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 0x5605a9dd0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5605a9b805f0 con 0x5605a99e90a0 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605a9cba460 con 0x5605a9a11f60 2024-09-21T06:53:11.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dc6df0 con 0x5605a99f1280 2024-09-21T06:53:11.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9c055bd740 10 client.0 _ll_register_callbacks cb 0x5605a9cca210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:11.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dc42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9bf7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 0x5605a9dd0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dc42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49376/0 (socket says 172.21.15.114:49376) 2024-09-21T06:53:11.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9bf7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 0x5605a9dd0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53950/0 (socket says 172.21.15.114:53950) 2024-09-21T06:53:11.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99f1280 0x5605a9dc4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.471+0000 7f9bf7400640 1 -- 172.21.15.114:0/2917735995 learned_addr learned my addr 172.21.15.114:0/2917735995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7e00640 1 -- 172.21.15.114:0/2917735995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99f1280 msgr2=0x5605a9dc4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99f1280 0x5605a9dc4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7e00640 1 -- 172.21.15.114:0/2917735995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 msgr2=0x5605a9dc42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dc42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7e00640 1 -- 172.21.15.114:0/2917735995 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a9a7d540 con 0x5605a9a11f60 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7400640 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dc42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 0x5605a9dd0110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0004980 tx=0x7f9bf0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ff19f8300a11c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9be1600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9be1600640 1 -- 172.21.15.114:0/2917735995 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf002c060 con 0x5605a9a11f60 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.472+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5605a9cbca70 con 0x5605a9a11f60 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9be1600640 1 -- 172.21.15.114:0/2917735995 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bf0017030 con 0x5605a9a11f60 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9be1600640 1 -- 172.21.15.114:0/2917735995 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bf0033050 con 0x5605a9a11f60 2024-09-21T06:53:11.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9c055bd740 10 client.25277 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605a9cd76f0 con 0x5605a9a11f60 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9be1600640 1 -- 172.21.15.114:0/2917735995 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5605a9cbca70 con 0x5605a9a11f60 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9be1600640 1 -- 172.21.15.114:0/2917735995 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9bf0027770 con 0x5605a9a11f60 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9c055bd740 10 client.25277 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9c055bd740 10 client.25277 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.473+0000 7f9c055bd740 20 client.25277 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.474+0000 7f9c055bd740 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99d08e0 0x5605a9dd67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.474+0000 7f9c055bd740 4 client.25277 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.474+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605a9d31ca0 con 0x5605a99d08e0 2024-09-21T06:53:11.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.474+0000 7f9bf7400640 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99d08e0 0x5605a9dd67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.474+0000 7f9bf7400640 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99d08e0 0x5605a9dd67e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9323caa3d949a89 server_cookie=8824ec2abf885afa in_seq=0 out_seq=0 2024-09-21T06:53:11.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.474+0000 7f9be1600640 10 client.25277 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:11.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.475+0000 7f9be1600640 1 -- 172.21.15.114:0/2917735995 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605a9d31ca0 con 0x5605a99d08e0 2024-09-21T06:53:11.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.475+0000 7f9be1600640 10 client.25277 handle_command_reply: tid=0 2024-09-21T06:53:11.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.475+0000 7f9c055bd740 1 client.25277 shutdown 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 msgr2=0x5605a9dd0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 0x5605a9dd0110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0004980 tx=0x7f9bf0004ab0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 shutdown_connections 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a9a11f60 0x5605a9dd0110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99d08e0 0x5605a9dd67e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99f1280 0x5605a9dc4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 --2- 172.21.15.114:0/2917735995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dc42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 >> 172.21.15.114:0/2917735995 conn(0x5605a9a06350 msgr2=0x5605a9dbb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9be1600640 0 client.25277 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 shutdown_connections 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 1 -- 172.21.15.114:0/2917735995 wait complete. 2024-09-21T06:53:11.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.476+0000 7f9c055bd740 20 client.25277 trim_cache size 0 max 16384 2024-09-21T06:53:11.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 Processor -- start 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 -- start start 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dd67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dc4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9dc47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dd4b20 con 0x5605a99e90a0 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dd51f0 con 0x5605a99f1280 2024-09-21T06:53:11.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9c055bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dd58c0 con 0x5605a9a11f60 2024-09-21T06:53:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dd67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9bf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dd67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49390/0 (socket says 172.21.15.114:49390) 2024-09-21T06:53:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.495+0000 7f9bf7400640 1 -- 172.21.15.114:0/676897349 learned_addr learned my addr 172.21.15.114:0/676897349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9dc47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf6a00640 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dc4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7400640 1 -- 172.21.15.114:0/676897349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 msgr2=0x5605a9dc47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7400640 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9dc47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7400640 1 -- 172.21.15.114:0/676897349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 msgr2=0x5605a9dc4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7400640 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dc4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7400640 1 -- 172.21.15.114:0/676897349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a9dd2d70 con 0x5605a99e90a0 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9bf7400640 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dd67e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9be80ebbe0 tx=0x7f9be800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599d9f0e0ff58691 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.496+0000 7f9be3e00640 1 -- 172.21.15.114:0/676897349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be80ed030 con 0x5605a99e90a0 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9be3e00640 1 -- 172.21.15.114:0/676897349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be80f9070 con 0x5605a99e90a0 2024-09-21T06:53:11.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9be3e00640 1 -- 172.21.15.114:0/676897349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8089910 con 0x5605a99e90a0 2024-09-21T06:53:11.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 -- 172.21.15.114:0/676897349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 msgr2=0x5605a9dd67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dd67e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9be80ebbe0 tx=0x7f9be800cf20 comp rx=0 tx=0).stop 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 -- 172.21.15.114:0/676897349 shutdown_connections 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a9a11f60 0x5605a9dc47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dc4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 --2- 172.21.15.114:0/676897349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a99e90a0 0x5605a9dd67e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.497+0000 7f9c055bd740 1 -- 172.21.15.114:0/676897349 >> 172.21.15.114:0/676897349 conn(0x5605a9a06350 msgr2=0x5605a9dd41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.498+0000 7f9c055bd740 1 -- 172.21.15.114:0/676897349 shutdown_connections 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.498+0000 7f9c055bd740 1 -- 172.21.15.114:0/676897349 wait complete. 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.498+0000 7f9c055bd740 1 -- start start 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.498+0000 7f9c055bd740 1 Processor -- start 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99e90a0 0x5605a9dd81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dd8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a9a11f60 0x5605a9ddd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dd6660 con 0x5605a9a11f60 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dd5f90 con 0x5605a99f1280 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605a9dd6d30 con 0x5605a99e90a0 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9c055bd740 10 client.0 _ll_register_callbacks cb 0x7f9bf0004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dd8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dd8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53978/0 (socket says 172.21.15.114:53978) 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 -- 172.21.15.114:0/3711933080 learned_addr learned my addr 172.21.15.114:0/3711933080 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf7e00640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a9a11f60 0x5605a9ddd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf7400640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99e90a0 0x5605a9dd81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 -- 172.21.15.114:0/3711933080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99e90a0 msgr2=0x5605a9dd81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99e90a0 0x5605a9dd81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 -- 172.21.15.114:0/3711933080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a9a11f60 msgr2=0x5605a9ddd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a9a11f60 0x5605a9ddd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.499+0000 7f9bf6a00640 1 -- 172.21.15.114:0/3711933080 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a9ddde40 con 0x5605a99f1280 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9bf6a00640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dd8790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009900 tx=0x7f9be40317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8392eb40b6dc54a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9be1600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9be1600640 1 -- 172.21.15.114:0/3711933080 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4035040 con 0x5605a99f1280 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9be80f9070 con 0x5605a99f1280 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9be1600640 1 -- 172.21.15.114:0/3711933080 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9be4004040 con 0x5605a99f1280 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9be1600640 1 -- 172.21.15.114:0/3711933080 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9be4051050 con 0x5605a99f1280 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9c055bd740 10 client.25283 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:11.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.500+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605a9ddf200 con 0x5605a99f1280 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9be1600640 1 -- 172.21.15.114:0/3711933080 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9be80f9070 con 0x5605a99f1280 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9be1600640 1 -- 172.21.15.114:0/3711933080 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9be4004ac0 con 0x5605a99f1280 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9c055bd740 10 client.25283 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9c055bd740 10 client.25283 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9c055bd740 20 client.25283 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9c055bd740 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99a5c90 0x5605a9e44e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9c055bd740 4 client.25283 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.501+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "status"}) -- 0x5605a9d2d750 con 0x5605a99a5c90 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.502+0000 7f9bf7400640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99a5c90 0x5605a9e44e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.502+0000 7f9bf7400640 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99a5c90 0x5605a9e44e30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a972bd8eb240ad6 server_cookie=f1bc28bc293fc470 in_seq=0 out_seq=0 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.502+0000 7f9be1600640 10 client.25283 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.502+0000 7f9be1600640 1 -- 172.21.15.114:0/3711933080 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5605a9d2d750 con 0x5605a99a5c90 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.502+0000 7f9be1600640 10 client.25283 handle_command_reply: tid=0 2024-09-21T06:53:11.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.502+0000 7f9c055bd740 1 client.25283 shutdown 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 msgr2=0x5605a9dd8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dd8790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009900 tx=0x7f9be40317b0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 shutdown_connections 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605a9a11f60 0x5605a9ddd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605a99f1280 0x5605a9dd8790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605a99a5c90 0x5605a9e44e30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 --2- 172.21.15.114:0/3711933080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605a99e90a0 0x5605a9dd81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 >> 172.21.15.114:0/3711933080 conn(0x5605a9a06350 msgr2=0x5605a9dbf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9be1600640 0 client.25283 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 shutdown_connections 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 1 -- 172.21.15.114:0/3711933080 wait complete. 2024-09-21T06:53:11.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.503+0000 7f9c055bd740 20 client.25283 trim_cache size 0 max 16384 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24761, 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 142.28527761300001, 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 155.90200007000001 2024-09-21T06:53:11.508 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bdc0b6560 msgr2=0x7f9bdc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bdc0b6560 0x7f9bdc0b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0005fd0 tx=0x7f9bf00096a0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 msgr2=0x7f9bf819c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 0x7f9bf819c530 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9be800e9f0 tx=0x7f9be800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 shutdown_connections 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bf8109c20 0x7f9bf81a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bdc0b6560 0x7f9bdc0b8a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bf81092d0 0x7f9bf819cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 --2- 172.21.15.114:0/800306161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bf8103990 0x7f9bf819c530 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:11.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 >> 172.21.15.114:0/800306161 conn(0x7f9bf80ff3e0 msgr2=0x7f9bf81069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:11.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 shutdown_connections 2024-09-21T06:53:11.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:11.504+0000 7f9bfd200640 1 -- 172.21.15.114:0/800306161 wait complete. 2024-09-21T06:53:12.442 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T06:53:12.443 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:53:12.452 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:53:12.453 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T06:53:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:12 smithi114 ceph-mon[30654]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:12.825 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:12 smithi193 ceph-mon[34601]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:13.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:12 smithi183 ceph-mon[34219]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:13.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 -- 172.21.15.114:0/3424878294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff7410c020 msgr2=0x7eff74112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 --2- 172.21.15.114:0/3424878294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff7410c020 0x7eff74112850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009ca0 tx=0x7eff6c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 -- 172.21.15.114:0/3424878294 shutdown_connections 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 --2- 172.21.15.114:0/3424878294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff7410c020 0x7eff74112850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 --2- 172.21.15.114:0/3424878294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 0x7eff7410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 --2- 172.21.15.114:0/3424878294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff74105d90 0x7eff741061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 -- 172.21.15.114:0/3424878294 >> 172.21.15.114:0/3424878294 conn(0x7eff74101860 msgr2=0x7eff74103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 -- 172.21.15.114:0/3424878294 shutdown_connections 2024-09-21T06:53:13.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.512+0000 7eff79a00640 1 -- 172.21.15.114:0/3424878294 wait complete. 2024-09-21T06:53:13.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.513+0000 7eff79a00640 1 Processor -- start 2024-09-21T06:53:13.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 -- start start 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff74105d90 0x7eff7419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 0x7eff7419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff7410c020 0x7eff741a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7eff74115d30 con 0x7eff74105d90 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff74115bb0 con 0x7eff7410b6d0 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff79a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7eff74115eb0 con 0x7eff7410c020 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff72a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff74105d90 0x7eff7419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff72000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 0x7eff7419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.514+0000 7eff72a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff74105d90 0x7eff7419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49414/0 (socket says 172.21.15.114:49414) 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72a00640 1 -- 172.21.15.114:0/3837452585 learned_addr learned my addr 172.21.15.114:0/3837452585 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff73400640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff7410c020 0x7eff741a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:13.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72000640 1 -- 172.21.15.114:0/3837452585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff7410c020 msgr2=0x7eff741a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:13.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72000640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff7410c020 0x7eff741a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72000640 1 -- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff74105d90 msgr2=0x7eff7419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:13.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72000640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff74105d90 0x7eff7419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72000640 1 -- 172.21.15.114:0/3837452585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff741a2610 con 0x7eff7410b6d0 2024-09-21T06:53:13.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.515+0000 7eff72000640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 0x7eff7419cc40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff7406ca80 tx=0x7eff6000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:13.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.516+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6000cd00 con 0x7eff7410b6d0 2024-09-21T06:53:13.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.516+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff741a2900 con 0x7eff7410b6d0 2024-09-21T06:53:13.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.516+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff600035f0 con 0x7eff7410b6d0 2024-09-21T06:53:13.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.516+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741a9060 con 0x7eff7410b6d0 2024-09-21T06:53:13.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.516+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60018690 con 0x7eff7410b6d0 2024-09-21T06:53:13.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.517+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff741061a0 con 0x7eff7410b6d0 2024-09-21T06:53:13.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.519+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7eff60003a30 con 0x7eff7410b6d0 2024-09-21T06:53:13.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.521+0000 7eff6b400640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7eff580b6560 0x7eff580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:13.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.521+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7eff600e7b40 con 0x7eff7410b6d0 2024-09-21T06:53:13.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.521+0000 7eff72a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7eff580b6560 0x7eff580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:13.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.522+0000 7eff72a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7eff580b6560 0x7eff580b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff64009cd0 tx=0x7eff64005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:13.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.523+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7eff600adf70 con 0x7eff7410b6d0 2024-09-21T06:53:13.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.726+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7eff74063800 con 0x7eff580b6560 2024-09-21T06:53:13.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.727+0000 7eff6b400640 1 -- 172.21.15.114:0/3837452585 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7eff74063800 con 0x7eff580b6560 2024-09-21T06:53:13.729 INFO:teuthology.orchestra.run.smithi114.stdout:Backend: cephadm 2024-09-21T06:53:13.729 INFO:teuthology.orchestra.run.smithi114.stdout:Available: Yes 2024-09-21T06:53:13.729 INFO:teuthology.orchestra.run.smithi114.stdout:Paused: No 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7eff580b6560 msgr2=0x7eff580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7eff580b6560 0x7eff580b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7eff64009cd0 tx=0x7eff64005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 msgr2=0x7eff7419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 0x7eff7419cc40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff7406ca80 tx=0x7eff6000ee50 comp rx=0 tx=0).stop 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 shutdown_connections 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff7410c020 0x7eff741a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7eff580b6560 0x7eff580b8a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff7410b6d0 0x7eff7419cc40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 --2- 172.21.15.114:0/3837452585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7eff74105d90 0x7eff7419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:13.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 >> 172.21.15.114:0/3837452585 conn(0x7eff74101860 msgr2=0x7eff74108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:13.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.729+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 shutdown_connections 2024-09-21T06:53:13.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:13.730+0000 7eff79a00640 1 -- 172.21.15.114:0/3837452585 wait complete. 2024-09-21T06:53:14.482 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T06:53:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:14 smithi114 ceph-mon[30654]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:14.853 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:14 smithi193 ceph-mon[34601]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:15.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:14 smithi183 ceph-mon[34219]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 -- 172.21.15.114:0/1418438477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c10c020 msgr2=0x7f729c112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 --2- 172.21.15.114:0/1418438477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c10c020 0x7f729c112850 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7294009ca0 tx=0x7f729402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 -- 172.21.15.114:0/1418438477 shutdown_connections 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 --2- 172.21.15.114:0/1418438477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c10c020 0x7f729c112850 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 --2- 172.21.15.114:0/1418438477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f729c10b6d0 0x7f729c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 --2- 172.21.15.114:0/1418438477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f729c105d90 0x7f729c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 -- 172.21.15.114:0/1418438477 >> 172.21.15.114:0/1418438477 conn(0x7f729c1017c0 msgr2=0x7f729c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 -- 172.21.15.114:0/1418438477 shutdown_connections 2024-09-21T06:53:15.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.522+0000 7f72a3400640 1 -- 172.21.15.114:0/1418438477 wait complete. 2024-09-21T06:53:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.523+0000 7f72a3400640 1 Processor -- start 2024-09-21T06:53:15.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.523+0000 7f72a3400640 1 -- start start 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 0x7f729c19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f729c10b6d0 0x7f729c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f729c10c020 0x7f729c1a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a3400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f729c115d30 con 0x7f729c105d90 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a3400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f729c115bb0 con 0x7f729c10b6d0 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a3400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f729c115eb0 con 0x7f729c10c020 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 0x7f729c19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 0x7f729c19c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33626/0 (socket says 172.21.15.114:33626) 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a0a00640 1 -- 172.21.15.114:0/4018723616 learned_addr learned my addr 172.21.15.114:0/4018723616 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f7293e00640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f729c10b6d0 0x7f729c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:15.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a1400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f729c10c020 0x7f729c1a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.524+0000 7f72a0a00640 1 -- 172.21.15.114:0/4018723616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f729c10c020 msgr2=0x7f729c1a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a0a00640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f729c10c020 0x7f729c1a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a0a00640 1 -- 172.21.15.114:0/4018723616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f729c10b6d0 msgr2=0x7f729c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a0a00640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f729c10b6d0 0x7f729c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a0a00640 1 -- 172.21.15.114:0/4018723616 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c1a2690 con 0x7f729c105d90 2024-09-21T06:53:15.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a0a00640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 0x7f729c19c720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f728c00d950 tx=0x7f728c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c017030 con 0x7f729c105d90 2024-09-21T06:53:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f728c004020 con 0x7f729c105d90 2024-09-21T06:53:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c0047a0 con 0x7f729c105d90 2024-09-21T06:53:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c1a2980 con 0x7f729c105d90 2024-09-21T06:53:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.525+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c1a90e0 con 0x7f729c105d90 2024-09-21T06:53:15.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.526+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c1061a0 con 0x7f729c105d90 2024-09-21T06:53:15.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.529+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f728c009a30 con 0x7f729c105d90 2024-09-21T06:53:15.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.530+0000 7f729a000640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72800b65e0 0x7f72800b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:15.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.530+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f728c0e7180 con 0x7f729c105d90 2024-09-21T06:53:15.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.531+0000 7f7293e00640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72800b65e0 0x7f72800b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:15.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.531+0000 7f7293e00640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72800b65e0 0x7f72800b8af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f729c105010 tx=0x7f728400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:15.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.534+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f728c0ad520 con 0x7f729c105d90 2024-09-21T06:53:15.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.737+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f729c063800 con 0x7f72800b65e0 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.749+0000 7f729a000640 1 -- 172.21.15.114:0/4018723616 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f729c063800 con 0x7f72800b65e0 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.a smithi114 running (2m) 25s ago 2m 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 c674d3125868 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.b smithi183 running (2m) 28s ago 2m 94.3M - 19.3.0-5120-g18432218 ddce307e1f32 6ed66db88e22 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.c smithi193 running (2m) 27s ago 2m 86.6M - 19.3.0-5120-g18432218 ddce307e1f32 a12693ffa40c 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.d smithi114 running (3m) 25s ago 3m 80.8M - 19.3.0-5120-g18432218 ddce307e1f32 8dc8f81b7599 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.e smithi183 running (3m) 28s ago 3m 85.2M - 19.3.0-5120-g18432218 ddce307e1f32 86959222bd3a 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.f smithi193 running (2m) 27s ago 2m 87.6M - 19.3.0-5120-g18432218 ddce307e1f32 bc6d2fcb8cb3 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.g smithi114 running (2m) 25s ago 2m 84.1M - 19.3.0-5120-g18432218 ddce307e1f32 c6a6a9c4caf1 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.h smithi183 running (2m) 28s ago 2m 87.3M - 19.3.0-5120-g18432218 ddce307e1f32 2f8e30bc6280 2024-09-21T06:53:15.751 INFO:teuthology.orchestra.run.smithi114.stdout:mds.i smithi193 running (2m) 27s ago 2m 86.6M - 19.3.0-5120-g18432218 ddce307e1f32 44628a12ac79 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mds.j smithi114 running (3m) 25s ago 3m 80.9M - 19.3.0-5120-g18432218 ddce307e1f32 1704241ca44d 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mds.k smithi183 running (3m) 28s ago 3m 86.5M - 19.3.0-5120-g18432218 ddce307e1f32 9d4f9f8b7951 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mds.l smithi193 running (3m) 27s ago 3m 86.0M - 19.3.0-5120-g18432218 ddce307e1f32 e091ad4273b9 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.x smithi114 *:9283,8765 running (11m) 25s ago 11m 587M - 19.3.0-5120-g18432218 ddce307e1f32 65183b67ba24 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.y smithi183 *:8443,8765 running (9m) 28s ago 9m 527M - 19.3.0-5120-g18432218 ddce307e1f32 09d611622614 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mgr.z smithi193 *:8443,8765 running (9m) 27s ago 9m 525M - 19.3.0-5120-g18432218 ddce307e1f32 f152bc6190f3 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mon.a smithi114 running (11m) 25s ago 11m 119M 2048M 19.3.0-5120-g18432218 ddce307e1f32 1743bb6a5e1a 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mon.b smithi183 running (10m) 28s ago 10m 114M 2048M 19.3.0-5120-g18432218 ddce307e1f32 bd48b0f8fac8 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:mon.c smithi193 running (10m) 27s ago 10m 111M 2048M 19.3.0-5120-g18432218 ddce307e1f32 5930fe7c63e7 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 smithi114 running (8m) 25s ago 8m 120M 4272M 19.3.0-5120-g18432218 ddce307e1f32 c8c3ebd2f308 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 smithi183 running (8m) 28s ago 8m 117M 4272M 19.3.0-5120-g18432218 ddce307e1f32 5ba97e2bf3af 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 smithi193 running (7m) 27s ago 7m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 123bb3f3a6c5 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 smithi114 running (7m) 25s ago 7m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 076a25ec4d1b 2024-09-21T06:53:15.752 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 smithi183 running (6m) 28s ago 6m 119M 4272M 19.3.0-5120-g18432218 ddce307e1f32 13f76befa663 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 smithi193 running (6m) 27s ago 6m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 fccf002e6bfe 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.6 smithi114 running (5m) 25s ago 5m 118M 4272M 19.3.0-5120-g18432218 ddce307e1f32 11c68249b0bb 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.7 smithi183 running (5m) 28s ago 5m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 d4002361a870 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.8 smithi193 running (4m) 27s ago 4m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 e31260718315 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.9 smithi114 running (4m) 25s ago 4m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 964bbf005dda 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.10 smithi183 running (3m) 28s ago 3m 112M 4272M 19.3.0-5120-g18432218 ddce307e1f32 d1a381e9df5a 2024-09-21T06:53:15.753 INFO:teuthology.orchestra.run.smithi114.stdout:osd.11 smithi193 running (3m) 27s ago 3m 117M 4272M 19.3.0-5120-g18432218 ddce307e1f32 8708f1fef542 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.751+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72800b65e0 msgr2=0x7f72800b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.751+0000 7f72a3400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72800b65e0 0x7f72800b8af0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f729c105010 tx=0x7f728400b390 comp rx=0 tx=0).stop 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.751+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 msgr2=0x7f729c19c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.751+0000 7f72a3400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 0x7f729c19c720 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f728c00d950 tx=0x7f728c00de10 comp rx=0 tx=0).stop 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 shutdown_connections 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f729c10c020 0x7f729c1a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72800b65e0 0x7f72800b8af0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f729c10b6d0 0x7f729c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 --2- 172.21.15.114:0/4018723616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f729c105d90 0x7f729c19c720 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 >> 172.21.15.114:0/4018723616 conn(0x7f729c1017c0 msgr2=0x7f729c109c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:15.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 shutdown_connections 2024-09-21T06:53:15.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:15.752+0000 7f72a3400640 1 -- 172.21.15.114:0/4018723616 wait complete. 2024-09-21T06:53:15.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:15 smithi193 ceph-mon[34601]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:16.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:15 smithi114 ceph-mon[30654]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:16.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:15 smithi183 ceph-mon[34219]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:16.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T06:53:17.002 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:17.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:16 smithi114 ceph-mon[30654]: pgmap v370: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T06:53:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:16 smithi183 ceph-mon[34219]: pgmap v370: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T06:53:17.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:16 smithi193 ceph-mon[34601]: pgmap v370: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T06:53:17.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 -- 172.21.15.114:0/902636721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d01094a0 msgr2=0x7f60d01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:17.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 --2- 172.21.15.114:0/902636721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d01094a0 0x7f60d01098b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f60b8009900 tx=0x7f60b802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:17.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 -- 172.21.15.114:0/902636721 shutdown_connections 2024-09-21T06:53:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 --2- 172.21.15.114:0/902636721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0109df0 0x7f60d0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 --2- 172.21.15.114:0/902636721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d01094a0 0x7f60d01098b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 --2- 172.21.15.114:0/902636721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d0103b60 0x7f60d0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 -- 172.21.15.114:0/902636721 >> 172.21.15.114:0/902636721 conn(0x7f60d007b850 msgr2=0x7f60d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 -- 172.21.15.114:0/902636721 shutdown_connections 2024-09-21T06:53:17.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.652+0000 7f60d7200640 1 -- 172.21.15.114:0/902636721 wait complete. 2024-09-21T06:53:17.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.653+0000 7f60d7200640 1 Processor -- start 2024-09-21T06:53:17.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 -- start start 2024-09-21T06:53:17.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 0x7f60d019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:17.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d01094a0 0x7f60d019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0109df0 0x7f60d019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113b00 con 0x7f60d0103b60 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113980 con 0x7f60d0109df0 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60d0113c80 con 0x7f60d01094a0 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60cfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 0x7f60d019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60cfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 0x7f60d019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33634/0 (socket says 172.21.15.114:33634) 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60cfe00640 1 -- 172.21.15.114:0/1756209572 learned_addr learned my addr 172.21.15.114:0/1756209572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60c7400640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d01094a0 0x7f60d019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.654+0000 7f60d5200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0109df0 0x7f60d019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cfe00640 1 -- 172.21.15.114:0/1756209572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d01094a0 msgr2=0x7f60d019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cfe00640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d01094a0 0x7f60d019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cfe00640 1 -- 172.21.15.114:0/1756209572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0109df0 msgr2=0x7f60d019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cfe00640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0109df0 0x7f60d019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cfe00640 1 -- 172.21.15.114:0/1756209572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d01a0420 con 0x7f60d0103b60 2024-09-21T06:53:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cfe00640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 0x7f60d019a4b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60c000d950 tx=0x7f60c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:17.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0017070 con 0x7f60d0103b60 2024-09-21T06:53:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d01a0710 con 0x7f60d0103b60 2024-09-21T06:53:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.655+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60c0009c30 con 0x7f60d0103b60 2024-09-21T06:53:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.656+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c000f620 con 0x7f60d0103b60 2024-09-21T06:53:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.656+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d01a6e70 con 0x7f60d0103b60 2024-09-21T06:53:17.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.657+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d0103f70 con 0x7f60d0103b60 2024-09-21T06:53:17.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.659+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f60c0005030 con 0x7f60d0103b60 2024-09-21T06:53:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.660+0000 7f60cd600640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60b40b6560 0x7f60b40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.660+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f60c00e73b0 con 0x7f60d0103b60 2024-09-21T06:53:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.661+0000 7f60c7400640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60b40b6560 0x7f60b40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:17.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.661+0000 7f60c7400640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60b40b6560 0x7f60b40b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f60b802aa00 tx=0x7f60b8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:17.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.662+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f60c0009de0 con 0x7f60d0103b60 2024-09-21T06:53:17.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:17 smithi114 ceph-mon[30654]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:17.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.862+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f60d0063800 con 0x7f60b40b6560 2024-09-21T06:53:17.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.868+0000 7f60cd600640 1 -- 172.21.15.114:0/1756209572 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f60d0063800 con 0x7f60b40b6560 2024-09-21T06:53:17.870 INFO:teuthology.orchestra.run.smithi114.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T06:53:17.870 INFO:teuthology.orchestra.run.smithi114.stdout:mds.all 12/12 30s ago 3m smithi114=a;smithi114=d;smithi114=g;smithi114=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-09-21T06:53:17.870 INFO:teuthology.orchestra.run.smithi114.stdout:mgr 3/3 30s ago 9m smithi114=x;smithi183=y;smithi193=z;count:3 2024-09-21T06:53:17.870 INFO:teuthology.orchestra.run.smithi114.stdout:mon 3/3 30s ago 10m smithi114:172.21.15.114=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-09-21T06:53:17.870 INFO:teuthology.orchestra.run.smithi114.stdout:osd 12 30s ago - 2024-09-21T06:53:17.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60b40b6560 msgr2=0x7f60b40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:17.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60b40b6560 0x7f60b40b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f60b802aa00 tx=0x7f60b8005a20 comp rx=0 tx=0).stop 2024-09-21T06:53:17.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 msgr2=0x7f60d019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:17.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 0x7f60d019a4b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f60c000d950 tx=0x7f60c000de10 comp rx=0 tx=0).stop 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 shutdown_connections 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60b40b6560 0x7f60b40b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60d0109df0 0x7f60d019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60d01094a0 0x7f60d019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 --2- 172.21.15.114:0/1756209572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60d0103b60 0x7f60d019a4b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.870+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 >> 172.21.15.114:0/1756209572 conn(0x7f60d007b850 msgr2=0x7f60d0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.871+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 shutdown_connections 2024-09-21T06:53:17.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:17.871+0000 7f60d7200640 1 -- 172.21.15.114:0/1756209572 wait complete. 2024-09-21T06:53:18.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:17 smithi183 ceph-mon[34219]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:18.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:17 smithi193 ceph-mon[34601]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:18.693 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T06:53:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:19.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:18 smithi114 ceph-mon[30654]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:19.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:18 smithi183 ceph-mon[34219]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:19.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:18 smithi193 ceph-mon[34601]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 -- 172.21.15.114:0/3005408545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f010b6c0 msgr2=0x7fd1f010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 --2- 172.21.15.114:0/3005408545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f010b6c0 0x7fd1f010bad0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009900 tx=0x7fd1dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 -- 172.21.15.114:0/3005408545 shutdown_connections 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 --2- 172.21.15.114:0/3005408545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010c010 0x7fd1f0112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 --2- 172.21.15.114:0/3005408545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f010b6c0 0x7fd1f010bad0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 --2- 172.21.15.114:0/3005408545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f0105d80 0x7fd1f0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 -- 172.21.15.114:0/3005408545 >> 172.21.15.114:0/3005408545 conn(0x7fd1f0101810 msgr2=0x7fd1f0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 -- 172.21.15.114:0/3005408545 shutdown_connections 2024-09-21T06:53:19.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.710+0000 7fd1f8600640 1 -- 172.21.15.114:0/3005408545 wait complete. 2024-09-21T06:53:19.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.711+0000 7fd1f8600640 1 Processor -- start 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.711+0000 7fd1f8600640 1 -- start start 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f0105d80 0x7fd1f019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 0x7fd1f019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f010c010 0x7fd1f01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f8600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0115cd0 con 0x7fd1f0105d80 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f8600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0115b50 con 0x7fd1f010c010 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f8600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f0115e50 con 0x7fd1f010b6c0 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f0105d80 0x7fd1f019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 0x7fd1f019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 0x7fd1f019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41340/0 (socket says 172.21.15.114:41340) 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f5200640 1 -- 172.21.15.114:0/1467147438 learned_addr learned my addr 172.21.15.114:0/1467147438 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:19.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.712+0000 7fd1f6600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f010c010 0x7fd1f01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:19.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5200640 1 -- 172.21.15.114:0/1467147438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f010c010 msgr2=0x7fd1f01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:19.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5200640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f010c010 0x7fd1f01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5200640 1 -- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f0105d80 msgr2=0x7fd1f019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5200640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f0105d80 0x7fd1f019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5200640 1 -- 172.21.15.114:0/1467147438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f01a2630 con 0x7fd1f010b6c0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5c00640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f0105d80 0x7fd1f019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f5200640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 0x7fd1f019cc60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc006010 tx=0x7fd1dc004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc03f030 con 0x7fd1f010b6c0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1dc003aa0 con 0x7fd1f010b6c0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f01a28c0 con 0x7fd1f010b6c0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc035070 con 0x7fd1f010b6c0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.713+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f01a9080 con 0x7fd1f010b6c0 2024-09-21T06:53:19.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.714+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f0107000 con 0x7fd1f010b6c0 2024-09-21T06:53:19.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.717+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fd1dc042030 con 0x7fd1f010b6c0 2024-09-21T06:53:19.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.718+0000 7fd1ee000640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1d80b6560 0x7fd1d80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:19.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.718+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd1dc107bc0 con 0x7fd1f010b6c0 2024-09-21T06:53:19.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.718+0000 7fd1f5c00640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1d80b6560 0x7fd1d80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:19.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.719+0000 7fd1f5c00640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1d80b6560 0x7fd1d80b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009fc0 tx=0x7fd1e4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:19.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.720+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1dc0d23a0 con 0x7fd1f010b6c0 2024-09-21T06:53:19.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:19 smithi114 ceph-mon[30654]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:19.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.916+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd1f0063800 con 0x7fd1d80b6560 2024-09-21T06:53:19.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.917+0000 7fd1ee000640 1 -- 172.21.15.114:0/1467147438 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fd1f0063800 con 0x7fd1d80b6560 2024-09-21T06:53:19.919 INFO:teuthology.orchestra.run.smithi114.stdout:HOST ADDR LABELS STATUS 2024-09-21T06:53:19.919 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114 172.21.15.114 2024-09-21T06:53:19.919 INFO:teuthology.orchestra.run.smithi114.stdout:smithi183 172.21.15.183 2024-09-21T06:53:19.919 INFO:teuthology.orchestra.run.smithi114.stdout:smithi193 172.21.15.193 2024-09-21T06:53:19.919 INFO:teuthology.orchestra.run.smithi114.stdout:3 hosts in cluster 2024-09-21T06:53:19.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1d80b6560 msgr2=0x7fd1d80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1d80b6560 0x7fd1d80b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009fc0 tx=0x7fd1e4005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 msgr2=0x7fd1f019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 0x7fd1f019cc60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc006010 tx=0x7fd1dc004060 comp rx=0 tx=0).stop 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 shutdown_connections 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1d80b6560 0x7fd1d80b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f010c010 0x7fd1f01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1f010b6c0 0x7fd1f019cc60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 --2- 172.21.15.114:0/1467147438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1f0105d80 0x7fd1f019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 >> 172.21.15.114:0/1467147438 conn(0x7fd1f0101810 msgr2=0x7fd1f0103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:19.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 shutdown_connections 2024-09-21T06:53:19.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:19.920+0000 7fd1f8600640 1 -- 172.21.15.114:0/1467147438 wait complete. 2024-09-21T06:53:20.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:19 smithi183 ceph-mon[34219]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:20.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:19 smithi193 ceph-mon[34601]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:20.723 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T06:53:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:20 smithi114 ceph-mon[30654]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:21.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:20 smithi114 ceph-mon[30654]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:21.092 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:20 smithi183 ceph-mon[34219]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:20 smithi183 ceph-mon[34219]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:20 smithi193 ceph-mon[34601]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:20 smithi193 ceph-mon[34601]: from='client.25312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:21.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 -- 172.21.15.114:0/416358132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 msgr2=0x7fe5b4104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:21.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 --2- 172.21.15.114:0/416358132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b4104c90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009900 tx=0x7fe5a402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 -- 172.21.15.114:0/416358132 shutdown_connections 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 --2- 172.21.15.114:0/416358132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5b410de30 0x7fe5b4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 --2- 172.21.15.114:0/416358132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b41052b0 0x7fe5b410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 --2- 172.21.15.114:0/416358132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b4104c90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 -- 172.21.15.114:0/416358132 >> 172.21.15.114:0/416358132 conn(0x7fe5b4079740 msgr2=0x7fe5b4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 -- 172.21.15.114:0/416358132 shutdown_connections 2024-09-21T06:53:21.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.716+0000 7fe5ba600640 1 -- 172.21.15.114:0/416358132 wait complete. 2024-09-21T06:53:21.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.718+0000 7fe5ba600640 1 Processor -- start 2024-09-21T06:53:21.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.718+0000 7fe5ba600640 1 -- start start 2024-09-21T06:53:21.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5ba600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:21.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5ba600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b41052b0 0x7fe5b419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:21.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5ba600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5b410de30 0x7fe5b419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:21.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5ba600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4113ad0 con 0x7fe5b4104880 2024-09-21T06:53:21.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5ba600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4113950 con 0x7fe5b41052b0 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5ba600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4113c50 con 0x7fe5b410de30 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33670/0 (socket says 172.21.15.114:33670) 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 -- 172.21.15.114:0/1932536997 learned_addr learned my addr 172.21.15.114:0/1932536997 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3e00640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5b410de30 0x7fe5b419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5abe00640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b41052b0 0x7fe5b419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 -- 172.21.15.114:0/1932536997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5b410de30 msgr2=0x7fe5b419fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5b410de30 0x7fe5b419fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 -- 172.21.15.114:0/1932536997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b41052b0 msgr2=0x7fe5b419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b41052b0 0x7fe5b419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.719+0000 7fe5b3400640 1 -- 172.21.15.114:0/1932536997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b41a05b0 con 0x7fe5b4104880 2024-09-21T06:53:21.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.720+0000 7fe5b3400640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b419a640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4005ab0 tx=0x7fe5a4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:21.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.720+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a403f070 con 0x7fe5b4104880 2024-09-21T06:53:21.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.720+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a4031d60 con 0x7fe5b4104880 2024-09-21T06:53:21.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.720+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b41a0840 con 0x7fe5b4104880 2024-09-21T06:53:21.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.720+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4035030 con 0x7fe5b4104880 2024-09-21T06:53:21.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.720+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41a7000 con 0x7fe5b4104880 2024-09-21T06:53:21.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.721+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b406c1c0 con 0x7fe5b4104880 2024-09-21T06:53:21.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.724+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fe5a4042070 con 0x7fe5b4104880 2024-09-21T06:53:21.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.725+0000 7fe5b0c00640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5980b6560 0x7fe5980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:21.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.725+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe5a4108640 con 0x7fe5b4104880 2024-09-21T06:53:21.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.725+0000 7fe5abe00640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5980b6560 0x7fe5980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:21.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.726+0000 7fe5abe00640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5980b6560 0x7fe5980b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4102440 tx=0x7fe59c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:21.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.728+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe5a40d2dc0 con 0x7fe5b4104880 2024-09-21T06:53:21.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:21 smithi114 ceph-mon[30654]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:21.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.925+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe5b4063800 con 0x7fe5980b6560 2024-09-21T06:53:21.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.927+0000 7fe5b0c00640 1 -- 172.21.15.114:0/1932536997 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe5b4063800 con 0x7fe5980b6560 2024-09-21T06:53:21.929 INFO:teuthology.orchestra.run.smithi114.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T06:53:21.929 INFO:teuthology.orchestra.run.smithi114.stdout:smithi114 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WB400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-21T06:53:21.929 INFO:teuthology.orchestra.run.smithi114.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T06:53:21.929 INFO:teuthology.orchestra.run.smithi114.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T06:53:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5980b6560 msgr2=0x7fe5980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5980b6560 0x7fe5980b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4102440 tx=0x7fe59c00b410 comp rx=0 tx=0).stop 2024-09-21T06:53:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 msgr2=0x7fe5b419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b419a640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4005ab0 tx=0x7fe5a4003cd0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 shutdown_connections 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5b410de30 0x7fe5b419fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5980b6560 0x7fe5980b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b41052b0 0x7fe5b419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 --2- 172.21.15.114:0/1932536997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5b4104880 0x7fe5b419a640 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 >> 172.21.15.114:0/1932536997 conn(0x7fe5b4079740 msgr2=0x7fe5b40ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 shutdown_connections 2024-09-21T06:53:21.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:21.929+0000 7fe5ba600640 1 -- 172.21.15.114:0/1932536997 wait complete. 2024-09-21T06:53:22.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:21 smithi183 ceph-mon[34219]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:21 smithi193 ceph-mon[34601]: pgmap v373: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:22.655 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T06:53:22.664 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:53:22.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T06:53:23.032 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:23.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:22 smithi114 ceph-mon[30654]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:22 smithi183 ceph-mon[34219]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:22 smithi193 ceph-mon[34601]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:53:23.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.678+0000 7fc385400640 1 -- 172.21.15.114:0/1352487000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 msgr2=0x7fc380103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.678+0000 7fc385400640 1 --2- 172.21.15.114:0/1352487000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc380103f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc370009900 tx=0x7fc37002a410 comp rx=0 tx=0).stop 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 -- 172.21.15.114:0/1352487000 shutdown_connections 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 --2- 172.21.15.114:0/1352487000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc380109df0 0x7fc380110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 --2- 172.21.15.114:0/1352487000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3801094a0 0x7fc3801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 --2- 172.21.15.114:0/1352487000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc380103f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 -- 172.21.15.114:0/1352487000 >> 172.21.15.114:0/1352487000 conn(0x7fc38007b850 msgr2=0x7fc38007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 -- 172.21.15.114:0/1352487000 shutdown_connections 2024-09-21T06:53:23.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.679+0000 7fc385400640 1 -- 172.21.15.114:0/1352487000 wait complete. 2024-09-21T06:53:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 Processor -- start 2024-09-21T06:53:23.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 -- start start 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc38019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3801094a0 0x7fc38019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc380109df0 0x7fc38019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc380113ab0 con 0x7fc380103b60 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc380113930 con 0x7fc380109df0 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.680+0000 7fc385400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc380113c30 con 0x7fc3801094a0 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc38019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc38019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33678/0 (socket says 172.21.15.114:33678) 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 -- 172.21.15.114:0/1859639178 learned_addr learned my addr 172.21.15.114:0/1859639178 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37f200640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc380109df0 0x7fc38019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:23.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37de00640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3801094a0 0x7fc38019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:23.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 -- 172.21.15.114:0/1859639178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3801094a0 msgr2=0x7fc38019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:23.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3801094a0 0x7fc38019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 -- 172.21.15.114:0/1859639178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc380109df0 msgr2=0x7fc38019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:23.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc380109df0 0x7fc38019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.681+0000 7fc37e800640 1 -- 172.21.15.114:0/1859639178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3801a04f0 con 0x7fc380103b60 2024-09-21T06:53:23.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.682+0000 7fc37e800640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc38019a580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc370005940 tx=0x7fc370004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:23.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.682+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37003f030 con 0x7fc380103b60 2024-09-21T06:53:23.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.682+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc370003aa0 con 0x7fc380103b60 2024-09-21T06:53:23.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.682+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3801a0780 con 0x7fc380103b60 2024-09-21T06:53:23.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.682+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370035070 con 0x7fc380103b60 2024-09-21T06:53:23.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.682+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3801a6f40 con 0x7fc380103b60 2024-09-21T06:53:23.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.683+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc380104de0 con 0x7fc380103b60 2024-09-21T06:53:23.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.685+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fc370042030 con 0x7fc380103b60 2024-09-21T06:53:23.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.687+0000 7fc377400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc3640b65e0 0x7fc3640b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:23.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.687+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc370108140 con 0x7fc380103b60 2024-09-21T06:53:23.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.687+0000 7fc37de00640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc3640b65e0 0x7fc3640b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:23.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.688+0000 7fc37de00640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc3640b65e0 0x7fc3640b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0095f0 tx=0x7fc36c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:23.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.689+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3700d28c0 con 0x7fc380103b60 2024-09-21T06:53:23.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:23 smithi114 ceph-mon[30654]: pgmap v374: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:23.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.914+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc380103f70 con 0x7fc380103b60 2024-09-21T06:53:23.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.915+0000 7fc377400640 1 -- 172.21.15.114:0/1859639178 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3140 (secure 0 0 0) 0x7fc380103f70 con 0x7fc380103b60 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:e28 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:btime 2024-09-21T06:51:45:610909+0000 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:legacy client fscid: 1 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem 'cephfs' (1) 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:fs_name cephfs 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 28 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-09-21T06:50:48.407210+0000 2024-09-21T06:53:23.917 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-09-21T06:51:45.336678+0000 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:tableserver 0 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:root 0 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:session_timeout 300 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:session_autoclose 300 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:max_file_size 1099511627776 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:max_xattr_size 65536 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:required_client_features {} 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure 0 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:last_failure_osd_epoch 0 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:max_mds 5 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:in 0,1,2,3,4 2024-09-21T06:53:23.918 INFO:teuthology.orchestra.run.smithi114.stdout:up {0=24761,1=24760,2=24742,3=24727,4=24725} 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:failed 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:damaged 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:stopped 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:data_pools [3,4] 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:metadata_pool 2 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:inline_data disabled 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:balancer 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:bal_rank_mask -1 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:standby_count_wanted 0 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:qdb_cluster leader: 24761 members: 24725,24727,24742,24760,24761 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.b{0:24761} state up:active seq 5 addr [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.f{0:24715} state up:standby-replay seq 1 addr [v2:172.21.15.193:6834/2057077471,v1:172.21.15.193:6835/2057077471] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.a{1:24760} state up:active seq 18 addr [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.e{1:24704} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/2729823804,v1:172.21.15.183:6835/2729823804] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.919 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.i{2:24742} state up:active seq 20 addr [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.k{2:24668} state up:standby-replay seq 1 addr [v2:172.21.15.183:6832/919969847,v1:172.21.15.183:6833/919969847] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.c{3:24727} state up:active seq 22 addr [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.l{3:24667} state up:standby-replay seq 1 addr [v2:172.21.15.193:6832/2086656361,v1:172.21.15.193:6833/2086656361] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.h{4:24725} state up:active seq 21 addr [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.g{4:14742} state up:standby-replay seq 1 addr [v2:172.21.15.114:6838/244481807,v1:172.21.15.114:6839/244481807] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:Standby daemons: 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.j{-1:14685} state up:standby seq 1 addr [v2:172.21.15.114:6834/3420506424,v1:172.21.15.114:6835/3420506424] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.920 INFO:teuthology.orchestra.run.smithi114.stdout:[mds.d{-1:14697} state up:standby seq 1 addr [v2:172.21.15.114:6836/1280617419,v1:172.21.15.114:6837/1280617419] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc3640b65e0 msgr2=0x7fc3640b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc3640b65e0 0x7fc3640b8af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc36c0095f0 tx=0x7fc36c005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 msgr2=0x7fc38019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc38019a580 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc370005940 tx=0x7fc370004270 comp rx=0 tx=0).stop 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 shutdown_connections 2024-09-21T06:53:23.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc3640b65e0 0x7fc3640b8af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc380109df0 0x7fc38019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3801094a0 0x7fc38019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 --2- 172.21.15.114:0/1859639178 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc380103b60 0x7fc38019a580 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:23.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 >> 172.21.15.114:0/1859639178 conn(0x7fc38007b850 msgr2=0x7fc3801075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:23.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 shutdown_connections 2024-09-21T06:53:23.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:23.917+0000 7fc385400640 1 -- 172.21.15.114:0/1859639178 wait complete. 2024-09-21T06:53:24.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:23 smithi183 ceph-mon[34219]: pgmap v374: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:24.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:23 smithi193 ceph-mon[34601]: pgmap v374: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:24.745 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T06:53:25.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:24 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1859639178' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:53:25.112 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:24 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1859639178' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:53:25.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:24 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1859639178' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 -- 172.21.15.114:0/3291047773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 msgr2=0x7fa860110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 --2- 172.21.15.114:0/3291047773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 0x7fa860110470 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa858009ca0 tx=0x7fa85802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 -- 172.21.15.114:0/3291047773 shutdown_connections 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 --2- 172.21.15.114:0/3291047773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 0x7fa860110470 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 --2- 172.21.15.114:0/3291047773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8601092f0 0x7fa860109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 --2- 172.21.15.114:0/3291047773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa860103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.747+0000 7fa85f200640 1 -- 172.21.15.114:0/3291047773 >> 172.21.15.114:0/3291047773 conn(0x7fa8600ff440 msgr2=0x7fa860101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.748+0000 7fa85f200640 1 -- 172.21.15.114:0/3291047773 shutdown_connections 2024-09-21T06:53:25.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.748+0000 7fa85f200640 1 -- 172.21.15.114:0/3291047773 wait complete. 2024-09-21T06:53:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.749+0000 7fa85f200640 1 Processor -- start 2024-09-21T06:53:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.749+0000 7fa85f200640 1 -- start start 2024-09-21T06:53:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa86019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:25.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8601092f0 0x7fa86019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 0x7fa8601a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa860113950 con 0x7fa860109c40 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601137d0 con 0x7fa8601092f0 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa860113ad0 con 0x7fa8601039b0 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa86019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa86019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52714/0 (socket says 172.21.15.114:52714) 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85de00640 1 -- 172.21.15.114:0/1310170345 learned_addr learned my addr 172.21.15.114:0/1310170345 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:25.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa85e800640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 0x7fa8601a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.750+0000 7fa857e00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8601092f0 0x7fa86019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85de00640 1 -- 172.21.15.114:0/1310170345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8601092f0 msgr2=0x7fa86019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85de00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8601092f0 0x7fa86019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85de00640 1 -- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 msgr2=0x7fa8601a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85de00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 0x7fa8601a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85de00640 1 -- 172.21.15.114:0/1310170345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8601a2450 con 0x7fa8601039b0 2024-09-21T06:53:25.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85de00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa86019c4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e9f0 tx=0x7fa85000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:25.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000cd80 con 0x7fa8601039b0 2024-09-21T06:53:25.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601a2740 con 0x7fa8601039b0 2024-09-21T06:53:25.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8500108e0 con 0x7fa8601039b0 2024-09-21T06:53:25.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850018980 con 0x7fa8601039b0 2024-09-21T06:53:25.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.751+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8601a8ea0 con 0x7fa8601039b0 2024-09-21T06:53:25.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.752+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa860103dc0 con 0x7fa8601039b0 2024-09-21T06:53:25.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.755+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fa850010430 con 0x7fa8601039b0 2024-09-21T06:53:25.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.756+0000 7fa856a00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8400b6560 0x7fa8400b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:25.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.756+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa8500e7fe0 con 0x7fa8601039b0 2024-09-21T06:53:25.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.757+0000 7fa857e00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8400b6560 0x7fa8400b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:25.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.757+0000 7fa857e00640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8400b6560 0x7fa8400b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa860105a10 tx=0x7fa848005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:25.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.758+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa8500b2780 con 0x7fa8601039b0 2024-09-21T06:53:25.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.954+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa8601a9190 con 0x7fa8601039b0 2024-09-21T06:53:25.956 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:25 smithi114 ceph-mon[30654]: pgmap v375: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:25.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.956+0000 7fa856a00640 1 -- 172.21.15.114:0/1310170345 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+6089 (secure 0 0 0) 0x7fa8500b76c0 con 0x7fa8601039b0 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:epoch 89 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:fsid 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:created 2024-09-21T06:41:28.662923+0000 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:modified 2024-09-21T06:52:31.637070+0000 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:crush_version 27 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:full_ratio 0.95 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:backfillfull_ratio 0.9 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:nearfull_ratio 0.85 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:require_min_compat_client luminous 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:min_compat_client luminous 2024-09-21T06:53:25.958 INFO:teuthology.orchestra.run.smithi114.stdout:require_osd_release squid 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:stretch_mode_enabled false 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:max_osd 12 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.0 up in weight 1 up_from 9 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6802/4089006446,v1:172.21.15.114:6803/4089006446] [v2:172.21.15.114:6804/4089006446,v1:172.21.15.114:6805/4089006446] exists,up 89d9168b-a941-4432-a508-6ab83fc6148c 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.1 up in weight 1 up_from 14 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/427115039,v1:172.21.15.183:6801/427115039] [v2:172.21.15.183:6802/427115039,v1:172.21.15.183:6803/427115039] exists,up a9bba525-a4a1-4988-8288-4a9023f07724 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.2 up in weight 1 up_from 19 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/1721723108,v1:172.21.15.193:6801/1721723108] [v2:172.21.15.193:6802/1721723108,v1:172.21.15.193:6803/1721723108] exists,up 8bc9357c-f3a5-4370-954a-e1654bab5e75 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.3 up in weight 1 up_from 26 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6810/2440023671,v1:172.21.15.114:6811/2440023671] [v2:172.21.15.114:6812/2440023671,v1:172.21.15.114:6813/2440023671] exists,up 36ee2356-5e1f-491f-bdf5-d9bd22956581 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.4 up in weight 1 up_from 32 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/204187293,v1:172.21.15.183:6809/204187293] [v2:172.21.15.183:6810/204187293,v1:172.21.15.183:6811/204187293] exists,up b7db874e-11e1-44a2-9f0f-3a1f8738356a 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/3650564948,v1:172.21.15.193:6809/3650564948] [v2:172.21.15.193:6810/3650564948,v1:172.21.15.193:6811/3650564948] exists,up 0798fc1c-9adf-42ec-b897-385b1be1bd71 2024-09-21T06:53:25.959 INFO:teuthology.orchestra.run.smithi114.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6818/1058136039,v1:172.21.15.114:6819/1058136039] [v2:172.21.15.114:6820/1058136039,v1:172.21.15.114:6821/1058136039] exists,up 6d41daa7-6623-4431-9bbb-fe54c1329dc9 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:osd.7 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/3203682724,v1:172.21.15.183:6817/3203682724] [v2:172.21.15.183:6818/3203682724,v1:172.21.15.183:6819/3203682724] exists,up 88b7c23c-2407-4f8e-895e-5349e4fb3083 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/2425758688,v1:172.21.15.193:6817/2425758688] [v2:172.21.15.193:6818/2425758688,v1:172.21.15.193:6819/2425758688] exists,up ca4001c9-2aa5-4258-9263-2ed19f7dde20 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:osd.9 up in weight 1 up_from 59 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.114:6826/903490275,v1:172.21.15.114:6827/903490275] [v2:172.21.15.114:6828/903490275,v1:172.21.15.114:6829/903490275] exists,up 76a2c023-5c26-46da-9de8-7d42a9b048df 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:osd.10 up in weight 1 up_from 64 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/4150013196,v1:172.21.15.183:6825/4150013196] [v2:172.21.15.183:6826/4150013196,v1:172.21.15.183:6827/4150013196] exists,up 5cf450ee-20ef-4871-afbf-3f9dc1d0c45c 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/3118065190,v1:172.21.15.193:6825/3118065190] [v2:172.21.15.193:6826/3118065190,v1:172.21.15.193:6827/3118065190] exists,up 0bdaf146-f4e6-4675-834d-1d130fa491ae 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.f [8,11] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 2.1b [2,1] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.5 [3,9] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.6 [4,7] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.10 [5,9] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.11 [5,9] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.31 [4,8] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 3.3a [4,6] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.a [5,8] 2024-09-21T06:53:25.960 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.12 [9,10] 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.17 [9,8] 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:pg_upmap_items 4.18 [9,8] 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/2966269937 expires 2024-09-22T06:43:29.387989+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/3575030312 expires 2024-09-22T06:41:54.185935+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/1583737438 expires 2024-09-22T06:42:25.331257+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/3364907862 expires 2024-09-22T06:41:54.185935+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6800/150287858 expires 2024-09-22T06:41:54.185935+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6801/150287858 expires 2024-09-22T06:41:54.185935+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6800/830030899 expires 2024-09-22T06:42:25.331257+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6801/830030899 expires 2024-09-22T06:42:25.331257+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/673737563 expires 2024-09-22T06:42:25.331257+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/791492062 expires 2024-09-22T06:41:54.185935+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/2622463004 expires 2024-09-22T06:43:29.387989+0000 2024-09-21T06:53:25.961 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/2480842130 expires 2024-09-22T06:42:25.331257+0000 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:0/200663390 expires 2024-09-22T06:43:29.387989+0000 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6800/8263225 expires 2024-09-22T06:43:29.387989+0000 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stdout:blocklist 172.21.15.114:6801/8263225 expires 2024-09-22T06:43:29.387989+0000 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.958+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8400b6560 msgr2=0x7fa8400b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.958+0000 7fa85f200640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8400b6560 0x7fa8400b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa860105a10 tx=0x7fa848005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.958+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 msgr2=0x7fa86019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:25.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.958+0000 7fa85f200640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa86019c4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa85000e9f0 tx=0x7fa85000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 shutdown_connections 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa860109c40 0x7fa8601a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8400b6560 0x7fa8400b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa8601092f0 0x7fa86019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 --2- 172.21.15.114:0/1310170345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8601039b0 0x7fa86019c4e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 >> 172.21.15.114:0/1310170345 conn(0x7fa8600ff440 msgr2=0x7fa860108900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 shutdown_connections 2024-09-21T06:53:25.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:25.959+0000 7fa85f200640 1 -- 172.21.15.114:0/1310170345 wait complete. 2024-09-21T06:53:26.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:25 smithi183 ceph-mon[34219]: pgmap v375: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:26.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:25 smithi193 ceph-mon[34601]: pgmap v375: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:26.835 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T06:53:26.845 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:53:27.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:26 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1310170345' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:53:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:26 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1310170345' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:53:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:26 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1310170345' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T06:53:27.222 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:27.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 -- 172.21.15.114:0/1937722645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc075dd0 msgr2=0x7fd2bc0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:27.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/1937722645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc0761e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009900 tx=0x7fd2a402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:27.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 -- 172.21.15.114:0/1937722645 shutdown_connections 2024-09-21T06:53:27.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/1937722645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc114450 0x7fd2bc116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:27.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/1937722645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc0761e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:27.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/1937722645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2bc077380 0x7fd2bc0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:27.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 -- 172.21.15.114:0/1937722645 >> 172.21.15.114:0/1937722645 conn(0x7fd2bc0ff5d0 msgr2=0x7fd2bc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:27.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 -- 172.21.15.114:0/1937722645 shutdown_connections 2024-09-21T06:53:27.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.868+0000 7fd2c2a00640 1 -- 172.21.15.114:0/1937722645 wait complete. 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.869+0000 7fd2c2a00640 1 Processor -- start 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.869+0000 7fd2c2a00640 1 -- start start 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc077380 0x7fd2bc1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2bc114450 0x7fd2bc1a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc11a060 con 0x7fd2bc077380 2024-09-21T06:53:27.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc119ee0 con 0x7fd2bc114450 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2bc11a1e0 con 0x7fd2bc075dd0 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52724/0 (socket says 172.21.15.114:52724) 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 -- 172.21.15.114:0/2074909575 learned_addr learned my addr 172.21.15.114:0/2074909575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2b3e00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc077380 0x7fd2bc1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2c0a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2bc114450 0x7fd2bc1a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 -- 172.21.15.114:0/2074909575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2bc114450 msgr2=0x7fd2bc1a62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:27.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2bc114450 0x7fd2bc1a62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 -- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc077380 msgr2=0x7fd2bc1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc077380 0x7fd2bc1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.870+0000 7fd2bbe00640 1 -- 172.21.15.114:0/2074909575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2bc1a6a00 con 0x7fd2bc075dd0 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.871+0000 7fd2bbe00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc1a0a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00e9f0 tx=0x7fd2ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.871+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00cd80 con 0x7fd2bc075dd0 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.871+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2ac0108e0 con 0x7fd2bc075dd0 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.871+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2bc1a6cf0 con 0x7fd2bc075dd0 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.871+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac018980 con 0x7fd2bc075dd0 2024-09-21T06:53:27.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.871+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2bc1ad450 con 0x7fd2bc075dd0 2024-09-21T06:53:27.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.872+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2bc0761e0 con 0x7fd2bc075dd0 2024-09-21T06:53:27.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.875+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fd2ac010430 con 0x7fd2bc075dd0 2024-09-21T06:53:27.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.876+0000 7fd2b9600640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd2a00b6560 0x7fd2a00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:27.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.876+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd2ac0e7ff0 con 0x7fd2bc075dd0 2024-09-21T06:53:27.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.876+0000 7fd2b3e00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd2a00b6560 0x7fd2a00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:27.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.877+0000 7fd2b3e00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd2a00b6560 0x7fd2a00b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4000c00 tx=0x7fd2a4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:27.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:27.878+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd2ac0b2790 con 0x7fd2bc075dd0 2024-09-21T06:53:28.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:27 smithi114 ceph-mon[30654]: pgmap v376: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:28.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.102+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2bc02e640 con 0x7fd2bc075dd0 2024-09-21T06:53:28.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.104+0000 7fd2b9600640 1 -- 172.21.15.114:0/2074909575 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7fd2ac0b76d0 con 0x7fd2bc075dd0 2024-09-21T06:53:28.105 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:28.106 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"btime":"2024-09-21T06:51:45:610909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:45.336678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:53:28.106 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd2a00b6560 msgr2=0x7fd2a00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd2a00b6560 0x7fd2a00b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4000c00 tx=0x7fd2a4031400 comp rx=0 tx=0).stop 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 msgr2=0x7fd2bc1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc1a0a90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00e9f0 tx=0x7fd2ac00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 shutdown_connections 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd2a00b6560 0x7fd2a00b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2bc114450 0x7fd2bc1a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:28.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2bc077380 0x7fd2bc1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 --2- 172.21.15.114:0/2074909575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2bc075dd0 0x7fd2bc1a0a90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 >> 172.21.15.114:0/2074909575 conn(0x7fd2bc0ff5d0 msgr2=0x7fd2bc110350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 shutdown_connections 2024-09-21T06:53:28.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:28.106+0000 7fd2c2a00640 1 -- 172.21.15.114:0/2074909575 wait complete. 2024-09-21T06:53:28.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:27 smithi183 ceph-mon[34219]: pgmap v376: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:28.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:27 smithi193 ceph-mon[34601]: pgmap v376: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:28.970 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:28 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2074909575' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:29.002 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:53:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:28 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2074909575' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:29.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:28 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2074909575' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:29.371 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:30.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 -- 172.21.15.114:0/2531499630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc514109c40 msgr2=0x7fc514110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:30.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 --2- 172.21.15.114:0/2531499630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc514109c40 0x7fc514110470 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009ca0 tx=0x7fc50c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 -- 172.21.15.114:0/2531499630 shutdown_connections 2024-09-21T06:53:30.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 --2- 172.21.15.114:0/2531499630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc514109c40 0x7fc514110470 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 --2- 172.21.15.114:0/2531499630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141092f0 0x7fc514109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 --2- 172.21.15.114:0/2531499630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5141039b0 0x7fc514103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 -- 172.21.15.114:0/2531499630 >> 172.21.15.114:0/2531499630 conn(0x7fc5140ff440 msgr2=0x7fc514101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:30.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 -- 172.21.15.114:0/2531499630 shutdown_connections 2024-09-21T06:53:30.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.998+0000 7fc51a600640 1 -- 172.21.15.114:0/2531499630 wait complete. 2024-09-21T06:53:30.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:29.999+0000 7fc51a600640 1 Processor -- start 2024-09-21T06:53:30.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 -- start start 2024-09-21T06:53:30.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 0x7fc51419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:30.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 0x7fc51419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:30.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc514109c40 0x7fc5141a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:30.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc514113900 con 0x7fc5141092f0 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc514113780 con 0x7fc514109c40 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc51a600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc514113a80 con 0x7fc5141039b0 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc513e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 0x7fc51419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc519200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 0x7fc51419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc519200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 0x7fc51419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52750/0 (socket says 172.21.15.114:52750) 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc513e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 0x7fc51419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60552/0 (socket says 172.21.15.114:60552) 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.000+0000 7fc519200640 1 -- 172.21.15.114:0/3259697274 learned_addr learned my addr 172.21.15.114:0/3259697274 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:30.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519c00640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc514109c40 0x7fc5141a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:30.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519200640 1 -- 172.21.15.114:0/3259697274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc514109c40 msgr2=0x7fc5141a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:30.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519200640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc514109c40 0x7fc5141a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519200640 1 -- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 msgr2=0x7fc51419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:30.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519200640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 0x7fc51419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519200640 1 -- 172.21.15.114:0/3259697274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5141a2440 con 0x7fc5141039b0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc513e00640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 0x7fc51419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.001+0000 7fc519200640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 0x7fc51419c4d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc50800d950 tx=0x7fc50800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.002+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508017030 con 0x7fc5141039b0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.002+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc508004020 con 0x7fc5141039b0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.002+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5080047a0 con 0x7fc5141039b0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.002+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5141a2730 con 0x7fc5141039b0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.002+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141a8e90 con 0x7fc5141039b0 2024-09-21T06:53:30.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.003+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc514103dc0 con 0x7fc5141039b0 2024-09-21T06:53:30.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.005+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fc508009a30 con 0x7fc5141039b0 2024-09-21T06:53:30.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.007+0000 7fc511600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4fc0b65e0 0x7fc4fc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:30.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.007+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc5080e7860 con 0x7fc5141039b0 2024-09-21T06:53:30.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.007+0000 7fc513e00640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4fc0b65e0 0x7fc4fc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:30.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.008+0000 7fc513e00640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4fc0b65e0 0x7fc4fc0b8af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5141059f0 tx=0x7fc500005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:30.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.009+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc5080adc80 con 0x7fc5141039b0 2024-09-21T06:53:30.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.231+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc51419d600 con 0x7fc5141039b0 2024-09-21T06:53:30.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.232+0000 7fc511600640 1 -- 172.21.15.114:0/3259697274 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11989 (secure 0 0 0) 0x7fc5080b2060 con 0x7fc5141039b0 2024-09-21T06:53:30.234 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:30.234 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":28,"btime":"2024-09-21T06:51:45:610909+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:51:45.336678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:53:30.235 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 28 2024-09-21T06:53:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4fc0b65e0 msgr2=0x7fc4fc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4fc0b65e0 0x7fc4fc0b8af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5141059f0 tx=0x7fc500005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 msgr2=0x7fc51419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:30.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 0x7fc51419c4d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc50800d950 tx=0x7fc50800de10 comp rx=0 tx=0).stop 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 shutdown_connections 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4fc0b65e0 0x7fc4fc0b8af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc514109c40 0x7fc5141a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc5141092f0 0x7fc51419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 --2- 172.21.15.114:0/3259697274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5141039b0 0x7fc51419c4d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 >> 172.21.15.114:0/3259697274 conn(0x7fc5140ff440 msgr2=0x7fc514108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 shutdown_connections 2024-09-21T06:53:30.237 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:30.235+0000 7fc51a600640 1 -- 172.21.15.114:0/3259697274 wait complete. 2024-09-21T06:53:30.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:30 smithi114 ceph-mon[30654]: pgmap v377: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:30.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:30 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3259697274' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:30 smithi183 ceph-mon[34219]: pgmap v377: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:30.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:30 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3259697274' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:30 smithi193 ceph-mon[34601]: pgmap v377: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:30 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3259697274' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:31.062 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:53:31.429 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:32.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 -- 172.21.15.114:0/764314568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c110440 msgr2=0x7f718c116c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 --2- 172.21.15.114:0/764314568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c110440 0x7f718c116c70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7184009900 tx=0x7f718402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 -- 172.21.15.114:0/764314568 shutdown_connections 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 --2- 172.21.15.114:0/764314568 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c110440 0x7f718c116c70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 --2- 172.21.15.114:0/764314568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f718c10faf0 0x7f718c10ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 --2- 172.21.15.114:0/764314568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f718c10a1b0 0x7f718c10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 -- 172.21.15.114:0/764314568 >> 172.21.15.114:0/764314568 conn(0x7f718c0780e0 msgr2=0x7f718c1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 -- 172.21.15.114:0/764314568 shutdown_connections 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.057+0000 7f7193000640 1 -- 172.21.15.114:0/764314568 wait complete. 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.058+0000 7f7193000640 1 Processor -- start 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 -- start start 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 0x7f718c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:32.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f718c10faf0 0x7f718c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f718c110440 0x7f718c1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f718c11a0f0 con 0x7f718c10a1b0 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f718c119f70 con 0x7f718c110440 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7193000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f718c11a270 con 0x7f718c10faf0 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f718be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 0x7f718c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f718be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 0x7f718c1a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60578/0 (socket says 172.21.15.114:60578) 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f718be00640 1 -- 172.21.15.114:0/2314810859 learned_addr learned my addr 172.21.15.114:0/2314810859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.059+0000 7f7183400640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f718c10faf0 0x7f718c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f7191000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f718c110440 0x7f718c1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f718be00640 1 -- 172.21.15.114:0/2314810859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f718c10faf0 msgr2=0x7f718c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f718be00640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f718c10faf0 0x7f718c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f718be00640 1 -- 172.21.15.114:0/2314810859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f718c110440 msgr2=0x7f718c1a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:32.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f718be00640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f718c110440 0x7f718c1a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f718be00640 1 -- 172.21.15.114:0/2314810859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c1a6a50 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f718be00640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 0x7f718c1a0ae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f717c00ebc0 tx=0x7f717c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c010630 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f717c010c80 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c1a6d40 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.060+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c018b60 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.061+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1ad4a0 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.062+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c10a5c0 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.064+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f717c018d00 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.065+0000 7f7189600640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71700b6560 0x7f71700b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.065+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f717c0e7e40 con 0x7f718c10a1b0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.066+0000 7f7183400640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71700b6560 0x7f71700b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:32.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.066+0000 7f7183400640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71700b6560 0x7f71700b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f718c06b150 tx=0x7f717400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:32.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.070+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f717c0b25c0 con 0x7f718c10a1b0 2024-09-21T06:53:32.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.259+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f718c1a1c10 con 0x7f718c10a1b0 2024-09-21T06:53:32.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.262+0000 7f7189600640 1 -- 172.21.15.114:0/2314810859 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25277 (secure 0 0 0) 0x7f717c0b7500 con 0x7f718c10a1b0 2024-09-21T06:53:32.266 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:32.266 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:52:31.637070+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:53:32.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71700b6560 msgr2=0x7f71700b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71700b6560 0x7f71700b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f718c06b150 tx=0x7f717400b390 comp rx=0 tx=0).stop 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 msgr2=0x7f718c1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 0x7f718c1a0ae0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f717c00ebc0 tx=0x7f717c00c6a0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 shutdown_connections 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71700b6560 0x7f71700b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f718c110440 0x7f718c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f718c10faf0 0x7f718c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 --2- 172.21.15.114:0/2314810859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f718c10a1b0 0x7f718c1a0ae0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.264+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 >> 172.21.15.114:0/2314810859 conn(0x7f718c0780e0 msgr2=0x7f718c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.265+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 shutdown_connections 2024-09-21T06:53:32.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:32.265+0000 7f7193000640 1 -- 172.21.15.114:0/2314810859 wait complete. 2024-09-21T06:53:32.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:32 smithi114 ceph-mon[30654]: pgmap v378: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:32.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:32 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2314810859' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:32 smithi183 ceph-mon[34219]: pgmap v378: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:32.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:32 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2314810859' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:32 smithi193 ceph-mon[34601]: pgmap v378: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:32 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2314810859' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:33.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:50:48.407210+0000', 'modified': '2024-09-21T06:51:45.336678+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24761, 'mds_1': 24760, 'mds_2': 24742, 'mds_3': 24727, 'mds_4': 24725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14742': {'gid': 14742, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6839/244481807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 244481807}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 244481807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/2086656361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 2086656361}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 2086656361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/919969847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 919969847}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 919969847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2729823804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2729823804}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2729823804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24715': {'gid': 24715, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/2057077471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2057077471}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2057077471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24725': {'gid': 24725, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.183:6837/841299119', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 841299119}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 841299119}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24727': {'gid': 24727, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.193:6837/1013898612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1013898612}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1013898612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24742': {'gid': 24742, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.193:6839/3226153327', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3226153327}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3226153327}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24760': {'gid': 24760, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.114:6841/2055898271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6840', 'nonce': 2055898271}, {'type': 'v1', 'addr': '172.21.15.114:6841', 'nonce': 2055898271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.183:6839/42004963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 42004963}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 42004963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24761, 'qdb_cluster': [24725, 24727, 24742, 24760, 24761]} 2024-09-21T06:53:33.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T06:53:33.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T06:53:33.392 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:34.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 -- 172.21.15.114:0/3730421015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55981049d0 msgr2=0x7f5598104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 --2- 172.21.15.114:0/3730421015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55981049d0 0x7f5598104de0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5588009900 tx=0x7f558802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 -- 172.21.15.114:0/3730421015 shutdown_connections 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 --2- 172.21.15.114:0/3730421015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 0x7f5598110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 --2- 172.21.15.114:0/3730421015 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5598105320 0x7f559810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 --2- 172.21.15.114:0/3730421015 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f55981049d0 0x7f5598104de0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 -- 172.21.15.114:0/3730421015 >> 172.21.15.114:0/3730421015 conn(0x7f5598079740 msgr2=0x7f5598079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 -- 172.21.15.114:0/3730421015 shutdown_connections 2024-09-21T06:53:34.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.072+0000 7f559f000640 1 -- 172.21.15.114:0/3730421015 wait complete. 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 Processor -- start 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 -- start start 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55981049d0 0x7f559819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 0x7f559819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 0x7f559819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5598113ab0 con 0x7f5598105320 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5598113930 con 0x7f559810de30 2024-09-21T06:53:34.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f559f000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5598113c30 con 0x7f55981049d0 2024-09-21T06:53:34.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f5597400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 0x7f559819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55981049d0 0x7f559819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.074+0000 7f5597400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 0x7f559819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60602/0 (socket says 172.21.15.114:60602) 2024-09-21T06:53:34.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597400640 1 -- 172.21.15.114:0/1873706816 learned_addr learned my addr 172.21.15.114:0/1873706816 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:34.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f559d000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 0x7f559819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 -- 172.21.15.114:0/1873706816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 msgr2=0x7f559819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 0x7f559819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 -- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 msgr2=0x7f559819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 0x7f559819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 -- 172.21.15.114:0/1873706816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55981a0450 con 0x7f55981049d0 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f559d000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 0x7f559819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597400640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 0x7f559819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.075+0000 7f5597e00640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55981049d0 0x7f559819a4e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f558802a930 tx=0x7f5588031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.076+0000 7f5594c00640 1 -- 172.21.15.114:0/1873706816 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558803f070 con 0x7f55981049d0 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.076+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55981a06e0 con 0x7f55981049d0 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.076+0000 7f5594c00640 1 -- 172.21.15.114:0/1873706816 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5588004490 con 0x7f55981049d0 2024-09-21T06:53:34.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.076+0000 7f5594c00640 1 -- 172.21.15.114:0/1873706816 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588035070 con 0x7f55981049d0 2024-09-21T06:53:34.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.076+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a6ea0 con 0x7f55981049d0 2024-09-21T06:53:34.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f55a5be2740 1 Processor -- start 2024-09-21T06:53:34.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f55a5be2740 1 -- start start 2024-09-21T06:53:34.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdee10f60 0x561fdf1b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f5594c00640 1 -- 172.21.15.114:0/1873706816 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f558803b460 con 0x7f55981049d0 2024-09-21T06:53:34.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 0x561fdf1b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.079+0000 7f55a5be2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561fdf12ed50 con 0x561fdedf0280 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f55a5be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561fdf11e180 con 0x561fdee10f60 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f55a5be2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fdf121f90 con 0x561fdede80a0 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 0x561fdf1b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 0x561fdf1b7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60610/0 (socket says 172.21.15.114:60610) 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 -- 172.21.15.114:0/2843338591 learned_addr learned my addr 172.21.15.114:0/2843338591 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597e00640 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f559d000640 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdee10f60 0x561fdf1b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 -- 172.21.15.114:0/2843338591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 msgr2=0x561fdf1b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 -- 172.21.15.114:0/2843338591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdee10f60 msgr2=0x561fdf1b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdee10f60 0x561fdf1b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.080+0000 7f5597400640 1 -- 172.21.15.114:0/2843338591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdf0e8a70 con 0x561fdedf0280 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.081+0000 7f5597400640 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 0x561fdf1b7f10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5584011540 tx=0x7f5584011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78f9ac12bb6dbfa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.081+0000 7f558d600640 1 -- 172.21.15.114:0/2843338591 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584005020 con 0x561fdedf0280 2024-09-21T06:53:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.081+0000 7f558d600640 1 -- 172.21.15.114:0/2843338591 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55840047a0 con 0x561fdedf0280 2024-09-21T06:53:34.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.081+0000 7f558d600640 1 -- 172.21.15.114:0/2843338591 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400f9b0 con 0x561fdedf0280 2024-09-21T06:53:34.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f5594c00640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f557c0b6560 0x7f557c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 -- 172.21.15.114:0/2843338591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 msgr2=0x561fdf1b7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 0x561fdf1b7f10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5584011540 tx=0x7f5584011a00 comp rx=0 tx=0).stop 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 -- 172.21.15.114:0/2843338591 shutdown_connections 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdedf0280 0x561fdf1b7f10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2843338591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdee10f60 0x561fdf1b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f5594c00640 1 -- 172.21.15.114:0/1873706816 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5588107ca0 con 0x7f55981049d0 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 -- 172.21.15.114:0/2843338591 >> 172.21.15.114:0/2843338591 conn(0x561fdee05350 msgr2=0x561fdf1b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f559d000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f557c0b6560 0x7f557c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 -- 172.21.15.114:0/2843338591 shutdown_connections 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.082+0000 7f55a5be2740 1 -- 172.21.15.114:0/2843338591 wait complete. 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.083+0000 7f559d000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f557c0b6560 0x7f557c0b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5590009fd0 tx=0x7f5590009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.083+0000 7f55a5be2740 1 -- start start 2024-09-21T06:53:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.083+0000 7f55a5be2740 1 Processor -- start 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdede80a0 0x561fdf1c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 0x561fdf1c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee10f60 0x561fdf1cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561fdef7f5f0 con 0x561fdee10f60 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561fdf0b9460 con 0x561fdede80a0 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1c5c70 con 0x561fdedf0280 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f55a5be2740 10 client.0 _ll_register_callbacks cb 0x561fdf0c9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 0x561fdf1c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f559d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee10f60 0x561fdf1cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 0x561fdf1c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52810/0 (socket says 172.21.15.114:52810) 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdede80a0 0x561fdf1c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 -- 172.21.15.114:0/1253383756 learned_addr learned my addr 172.21.15.114:0/1253383756 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 -- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdede80a0 msgr2=0x561fdf1c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdede80a0 0x561fdf1c2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 -- 172.21.15.114:0/1253383756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee10f60 msgr2=0x561fdf1cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee10f60 0x561fdf1cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597400640 1 -- 172.21.15.114:0/1253383756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdee7c540 con 0x561fdedf0280 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.084+0000 7f5597e00640 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdede80a0 0x561fdf1c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f5597400640 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 0x561fdf1c34d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5584004ee0 tx=0x7f5584005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a653a05a5f3d1fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f5582000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f5582000640 1 -- 172.21.15.114:0/1253383756 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584027070 con 0x561fdedf0280 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fdf0bba70 con 0x561fdedf0280 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f5582000640 1 -- 172.21.15.114:0/1253383756 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f55840048d0 con 0x561fdedf0280 2024-09-21T06:53:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f5582000640 1 -- 172.21.15.114:0/1253383756 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5584033050 con 0x561fdedf0280 2024-09-21T06:53:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f55a5be2740 10 client.25360 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.085+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fdf0d66f0 con 0x561fdedf0280 2024-09-21T06:53:34.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f5582000640 1 -- 172.21.15.114:0/1253383756 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561fdf0bba70 con 0x561fdedf0280 2024-09-21T06:53:34.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f5582000640 1 -- 172.21.15.114:0/1253383756 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5584022460 con 0x561fdedf0280 2024-09-21T06:53:34.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f55a5be2740 10 client.25360 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:34.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f55a5be2740 10 client.25360 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f55a5be2740 20 client.25360 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f55a5be2740 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdedcf8e0 0x561fdf1d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f55a5be2740 4 client.25360 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.086+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fdf130ca0 con 0x561fdedcf8e0 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.087+0000 7f5597e00640 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdedcf8e0 0x561fdf1d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.087+0000 7f5597e00640 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdedcf8e0 0x561fdf1d5290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f191c4656bcae9e server_cookie=2c18da1ef6cd9a7e in_seq=0 out_seq=0 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.087+0000 7f5582000640 10 client.25360 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f5582000640 1 -- 172.21.15.114:0/1253383756 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561fdf130ca0 con 0x561fdedcf8e0 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f5582000640 10 client.25360 handle_command_reply: tid=0 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 client.25360 shutdown 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 msgr2=0x561fdf1c34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 0x561fdf1c34d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5584004ee0 tx=0x7f5584005410 comp rx=0 tx=0).stop 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 shutdown_connections 2024-09-21T06:53:34.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee10f60 0x561fdf1cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdedf0280 0x561fdf1c34d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.088+0000 7f55a5be2740 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdedcf8e0 0x561fdf1d5290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.089+0000 7f55a5be2740 1 --2- 172.21.15.114:0/1253383756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdede80a0 0x561fdf1c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.089+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 >> 172.21.15.114:0/1253383756 conn(0x561fdee05350 msgr2=0x561fdf1ba620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.089+0000 7f5582000640 0 client.25360 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.089+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 shutdown_connections 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.089+0000 7f55a5be2740 1 -- 172.21.15.114:0/1253383756 wait complete. 2024-09-21T06:53:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.089+0000 7f55a5be2740 20 client.25360 trim_cache size 0 max 16384 2024-09-21T06:53:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.107+0000 7f55a5be2740 1 Processor -- start 2024-09-21T06:53:34.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.107+0000 7f55a5be2740 1 -- start start 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdee05350 0x561fdf1d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 0x561fdf1bd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f55a5be2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1d35d0 con 0x561fdede80a0 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f55a5be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1d3ca0 con 0x561fdedf0280 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f55a5be2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1d4370 con 0x561fdee05350 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 0x561fdf1bd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 0x561fdf1bd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60638/0 (socket says 172.21.15.114:60638) 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 -- 172.21.15.114:0/2738450251 learned_addr learned my addr 172.21.15.114:0/2738450251 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f559d000640 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597e00640 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdee05350 0x561fdf1d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 -- 172.21.15.114:0/2738450251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdee05350 msgr2=0x561fdf1d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdee05350 0x561fdf1d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 -- 172.21.15.114:0/2738450251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 msgr2=0x561fdf1b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.108+0000 7f5597400640 1 -- 172.21.15.114:0/2738450251 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdf1b9780 con 0x561fdede80a0 2024-09-21T06:53:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.109+0000 7f5597400640 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 0x561fdf1bd570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5584011460 tx=0x7f5584011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168488b9d4470c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.109+0000 7f558cc00640 1 -- 172.21.15.114:0/2738450251 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558403c070 con 0x561fdede80a0 2024-09-21T06:53:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.109+0000 7f558cc00640 1 -- 172.21.15.114:0/2738450251 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584022660 con 0x561fdede80a0 2024-09-21T06:53:34.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.109+0000 7f558cc00640 1 -- 172.21.15.114:0/2738450251 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584027040 con 0x561fdede80a0 2024-09-21T06:53:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 -- 172.21.15.114:0/2738450251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 msgr2=0x561fdf1bd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 0x561fdf1bd570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5584011460 tx=0x7f5584011a00 comp rx=0 tx=0).stop 2024-09-21T06:53:34.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 -- 172.21.15.114:0/2738450251 shutdown_connections 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdede80a0 0x561fdf1bd570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 --2- 172.21.15.114:0/2738450251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdee05350 0x561fdf1d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 -- 172.21.15.114:0/2738450251 >> 172.21.15.114:0/2738450251 conn(0x561fdf1c3650 msgr2=0x561fdf1b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 -- 172.21.15.114:0/2738450251 shutdown_connections 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 -- 172.21.15.114:0/2738450251 wait complete. 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.110+0000 7f55a5be2740 1 -- start start 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 Processor -- start 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1d63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 0x561fdf243790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1d5110 con 0x561fdee05350 2024-09-21T06:53:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1d4a40 con 0x561fdedf0280 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561fdf1d57e0 con 0x561fdede80a0 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f55a5be2740 10 client.0 _ll_register_callbacks cb 0x7f558810b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f559d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 0x561fdf243790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.111+0000 7f5597400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1d68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54028/0 (socket says 172.21.15.114:54028) 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f559d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 0x561fdf243790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60646/0 (socket says 172.21.15.114:60646) 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 -- 172.21.15.114:0/4150414501 learned_addr learned my addr 172.21.15.114:0/4150414501 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597e00640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1d63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 -- 172.21.15.114:0/4150414501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 msgr2=0x561fdf1d63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1d63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 -- 172.21.15.114:0/4150414501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 msgr2=0x561fdf243790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 0x561fdf243790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 -- 172.21.15.114:0/4150414501 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdf1d71d0 con 0x561fdedf0280 2024-09-21T06:53:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f559d000640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 0x561fdf243790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:34.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5597400640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1d68e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5584022950 tx=0x7f5584022e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d346d54951a1acf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.112+0000 7f5582000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f5582000640 1 -- 172.21.15.114:0/4150414501 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558403c030 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5584022660 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f5582000640 1 -- 172.21.15.114:0/4150414501 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5584022460 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f5582000640 1 -- 172.21.15.114:0/4150414501 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5584027040 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f55a5be2740 10 client.25361 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fdf244d80 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.113+0000 7f5582000640 1 -- 172.21.15.114:0/4150414501 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5584022660 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f5582000640 1 -- 172.21.15.114:0/4150414501 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f558400fb80 con 0x561fdedf0280 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f55a5be2740 10 client.25361 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:34.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f55a5be2740 10 client.25361 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f55a5be2740 20 client.25361 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f55a5be2740 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdee10f60 0x561fdf248430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f55a5be2740 4 client.25361 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "status"}) -- 0x561fdf12c750 con 0x561fdee10f60 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.114+0000 7f5597e00640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdee10f60 0x561fdf248430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f5597e00640 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdee10f60 0x561fdf248430 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=621520c7a9e33d05 server_cookie=21d04fcb5f0c97a9 in_seq=0 out_seq=0 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f5582000640 10 client.25361 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f5582000640 1 -- 172.21.15.114:0/4150414501 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561fdf12c750 con 0x561fdee10f60 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f5582000640 10 client.25361 handle_command_reply: tid=0 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 client.25361 shutdown 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 msgr2=0x561fdf1d68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1d68e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5584022950 tx=0x7f5584022e40 comp rx=0 tx=0).stop 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 shutdown_connections 2024-09-21T06:53:34.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561fdee05350 0x561fdf243790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x561fdee10f60 0x561fdf248430 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561fdedf0280 0x561fdf1d68e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 --2- 172.21.15.114:0/4150414501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561fdede80a0 0x561fdf1d63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f5582000640 0 client.25361 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 >> 172.21.15.114:0/4150414501 conn(0x561fdf1c3650 msgr2=0x561fdf1b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 shutdown_connections 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.115+0000 7f55a5be2740 1 -- 172.21.15.114:0/4150414501 wait complete. 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f55a5be2740 20 client.25361 trim_cache size 0 max 16384 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:34.120 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 4, 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24725, 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 111.514603232, 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 190.78494025200001 2024-09-21T06:53:34.121 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f557c0b6560 msgr2=0x7f557c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f557c0b6560 0x7f557c0b8a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5590009fd0 tx=0x7f5590009500 comp rx=0 tx=0).stop 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55981049d0 msgr2=0x7f559819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55981049d0 0x7f559819a4e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f558802a930 tx=0x7f5588031a20 comp rx=0 tx=0).stop 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 shutdown_connections 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f557c0b6560 0x7f557c0b8a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f559810de30 0x7f559819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5598105320 0x7f559819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 --2- 172.21.15.114:0/1873706816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55981049d0 0x7f559819a4e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:34.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 >> 172.21.15.114:0/1873706816 conn(0x7f5598079740 msgr2=0x7f5598100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:34.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.116+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 shutdown_connections 2024-09-21T06:53:34.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:34.117+0000 7f559f000640 1 -- 172.21.15.114:0/1873706816 wait complete. 2024-09-21T06:53:34.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:34 smithi114 ceph-mon[30654]: pgmap v379: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:34.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:34 smithi183 ceph-mon[34219]: pgmap v379: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:34.899 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T06:53:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:34 smithi193 ceph-mon[34601]: pgmap v379: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:35.270 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 -- 172.21.15.114:0/4071808128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb25810b4f0 msgr2=0x7fb25810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 --2- 172.21.15.114:0/4071808128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb25810b4f0 0x7fb25810b900 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb240009ca0 tx=0x7fb24002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 -- 172.21.15.114:0/4071808128 shutdown_connections 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 --2- 172.21.15.114:0/4071808128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810be40 0x7fb258112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 --2- 172.21.15.114:0/4071808128 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb25810b4f0 0x7fb25810b900 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 --2- 172.21.15.114:0/4071808128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb258105bb0 0x7fb258105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 -- 172.21.15.114:0/4071808128 >> 172.21.15.114:0/4071808128 conn(0x7fb258101680 msgr2=0x7fb258103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 -- 172.21.15.114:0/4071808128 shutdown_connections 2024-09-21T06:53:35.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.921+0000 7fb25cc00640 1 -- 172.21.15.114:0/4071808128 wait complete. 2024-09-21T06:53:35.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.922+0000 7fb25cc00640 1 Processor -- start 2024-09-21T06:53:35.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.922+0000 7fb25cc00640 1 -- start start 2024-09-21T06:53:35.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb25cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 0x7fb25819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb25cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810b4f0 0x7fb25819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb25cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb25810be40 0x7fb2581a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb25cc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb258115b00 con 0x7fb258105bb0 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb25cc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb258115980 con 0x7fb25810be40 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb25cc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb258115c80 con 0x7fb25810b4f0 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 0x7fb25819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb24ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810b4f0 0x7fb25819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 0x7fb25819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44010/0 (socket says 172.21.15.114:44010) 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb24ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810b4f0 0x7fb25819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36578/0 (socket says 172.21.15.114:36578) 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 -- 172.21.15.114:0/1922686322 learned_addr learned my addr 172.21.15.114:0/1922686322 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:35.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257e00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb25810be40 0x7fb2581a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 -- 172.21.15.114:0/1922686322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810b4f0 msgr2=0x7fb25819ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810b4f0 0x7fb25819ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 -- 172.21.15.114:0/1922686322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb25810be40 msgr2=0x7fb2581a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb25810be40 0x7fb2581a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.923+0000 7fb257400640 1 -- 172.21.15.114:0/1922686322 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2581a2440 con 0x7fb258105bb0 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.924+0000 7fb257400640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 0x7fb25819c4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb24800e9f0 tx=0x7fb24800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.924+0000 7fb254c00640 1 -- 172.21.15.114:0/1922686322 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24800cd80 con 0x7fb258105bb0 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.924+0000 7fb254c00640 1 -- 172.21.15.114:0/1922686322 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2480108e0 con 0x7fb258105bb0 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.924+0000 7fb254c00640 1 -- 172.21.15.114:0/1922686322 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248018980 con 0x7fb258105bb0 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.924+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2581a2730 con 0x7fb258105bb0 2024-09-21T06:53:35.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.924+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2581a8e90 con 0x7fb258105bb0 2024-09-21T06:53:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.927+0000 7fb264dcc740 1 Processor -- start 2024-09-21T06:53:35.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.927+0000 7fb264dcc740 1 -- start start 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb254c00640 1 -- 172.21.15.114:0/1922686322 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fb248010430 con 0x7fb258105bb0 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926af9a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926af9abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926af9b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb264dcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55926af11d50 con 0x55926abf3f60 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb264dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55926af01180 con 0x55926abd3280 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb264dcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55926af04f90 con 0x55926abcb0a0 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb257e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926af9a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb257e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926af9a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44026/0 (socket says 172.21.15.114:44026) 2024-09-21T06:53:35.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb257e00640 1 -- 172.21.15.114:0/2550871751 learned_addr learned my addr 172.21.15.114:0/2550871751 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb257400640 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926af9abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.928+0000 7fb24ea00640 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926af9b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb257e00640 1 -- 172.21.15.114:0/2550871751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 msgr2=0x55926af9abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb257e00640 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926af9abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb257e00640 1 -- 172.21.15.114:0/2550871751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 msgr2=0x55926af9b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb257e00640 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926af9b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb257e00640 1 -- 172.21.15.114:0/2550871751 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926aecba70 con 0x55926abf3f60 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb257e00640 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926af9a670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb250009ca0 tx=0x7fb250011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bcd72bb2b2232f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.929+0000 7fb247e00640 1 -- 172.21.15.114:0/2550871751 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25001e070 con 0x55926abf3f60 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb247e00640 1 -- 172.21.15.114:0/2550871751 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb250009e50 con 0x55926abf3f60 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb247e00640 1 -- 172.21.15.114:0/2550871751 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25000e9f0 con 0x55926abf3f60 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb254c00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb23c0b6560 0x7fb23c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb254c00640 1 -- 172.21.15.114:0/1922686322 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb2480e7b90 con 0x7fb258105bb0 2024-09-21T06:53:35.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb257400640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb23c0b6560 0x7fb23c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb264dcc740 1 -- 172.21.15.114:0/2550871751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 msgr2=0x55926af9a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.930+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926af9a670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb250009ca0 tx=0x7fb250011810 comp rx=0 tx=0).stop 2024-09-21T06:53:35.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 -- 172.21.15.114:0/2550871751 shutdown_connections 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926af9b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926af9abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2550871751 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926af9a670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 -- 172.21.15.114:0/2550871751 >> 172.21.15.114:0/2550871751 conn(0x55926abe8350 msgr2=0x55926af98560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 -- 172.21.15.114:0/2550871751 shutdown_connections 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb257400640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb23c0b6560 0x7fb23c0b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2480e51a0 tx=0x7fb2480eb5d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.931+0000 7fb264dcc740 1 -- 172.21.15.114:0/2550871751 wait complete. 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 -- start start 2024-09-21T06:53:35.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 Processor -- start 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 0x55926afa6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926afa67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abf3f60 0x55926afb2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55926ad625f0 con 0x55926abcb0a0 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55926ae9c460 con 0x55926abd3280 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.932+0000 7fb264dcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55926afa8e50 con 0x55926abf3f60 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb264dcc740 10 client.0 _ll_register_callbacks cb 0x55926aeac210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 0x55926afa6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 0x55926afa6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44036/0 (socket says 172.21.15.114:44036) 2024-09-21T06:53:35.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 -- 172.21.15.114:0/435678083 learned_addr learned my addr 172.21.15.114:0/435678083 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb24ea00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926afa67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abf3f60 0x55926afb2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 -- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abf3f60 msgr2=0x55926afb2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abf3f60 0x55926afb2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 -- 172.21.15.114:0/435678083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 msgr2=0x55926afa67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926afa67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.933+0000 7fb257e00640 1 -- 172.21.15.114:0/435678083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926ac5f540 con 0x55926abcb0a0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb257e00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 0x55926afa6270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb2500057c0 tx=0x7fb250005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf28b05fb40a114 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb244c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb244c00640 1 -- 172.21.15.114:0/435678083 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25002d060 con 0x55926abcb0a0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb244c00640 1 -- 172.21.15.114:0/435678083 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb25000edd0 con 0x55926abcb0a0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55926ae9ea70 con 0x55926abcb0a0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb244c00640 1 -- 172.21.15.114:0/435678083 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb25001e070 con 0x55926abcb0a0 2024-09-21T06:53:35.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb264dcc740 10 client.15372 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:35.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.934+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55926aeb96f0 con 0x55926abcb0a0 2024-09-21T06:53:35.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb244c00640 1 -- 172.21.15.114:0/435678083 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55926ae9ea70 con 0x55926abcb0a0 2024-09-21T06:53:35.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb244c00640 1 -- 172.21.15.114:0/435678083 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb250019430 con 0x55926abcb0a0 2024-09-21T06:53:35.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb264dcc740 10 client.15372 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:35.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb264dcc740 10 client.15372 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:53:35.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb264dcc740 20 client.15372 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:35.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb264dcc740 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926abb28e0 0x55926afb8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb264dcc740 4 client.15372 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55926af13ca0 con 0x55926abb28e0 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.935+0000 7fb24ea00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926abb28e0 0x55926afb8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.936+0000 7fb24ea00640 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926abb28e0 0x55926afb8460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b14622d3a21b0eaa server_cookie=57bc2dcba7d60190 in_seq=0 out_seq=0 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.936+0000 7fb244c00640 10 client.15372 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb244c00640 1 -- 172.21.15.114:0/435678083 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55926af13ca0 con 0x55926abb28e0 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb244c00640 10 client.15372 handle_command_reply: tid=0 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 client.15372 shutdown 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 msgr2=0x55926afa6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 0x55926afa6270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb2500057c0 tx=0x7fb250005710 comp rx=0 tx=0).stop 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 shutdown_connections 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926abb28e0 0x55926afb8460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abf3f60 0x55926afb2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abd3280 0x55926afa67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 --2- 172.21.15.114:0/435678083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abcb0a0 0x55926afa6270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 >> 172.21.15.114:0/435678083 conn(0x55926abe8350 msgr2=0x55926af9d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:35.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb244c00640 0 client.15372 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:35.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.937+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 shutdown_connections 2024-09-21T06:53:35.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.938+0000 7fb264dcc740 1 -- 172.21.15.114:0/435678083 wait complete. 2024-09-21T06:53:35.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.938+0000 7fb264dcc740 20 client.15372 trim_cache size 0 max 16384 2024-09-21T06:53:35.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.956+0000 7fb264dcc740 1 Processor -- start 2024-09-21T06:53:35.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.956+0000 7fb264dcc740 1 -- start start 2024-09-21T06:53:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926afb8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 0x55926afa04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abf3f60 0x55926afa48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb264dcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55926afb67a0 con 0x55926abd3280 2024-09-21T06:53:35.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb264dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55926afb6e70 con 0x55926abf3f60 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb264dcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55926afb7540 con 0x55926abcb0a0 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 0x55926afa04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 0x55926afa04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44042/0 (socket says 172.21.15.114:44042) 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 -- 172.21.15.114:0/3270454624 learned_addr learned my addr 172.21.15.114:0/3270454624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb257e00640 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abf3f60 0x55926afa48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb257e00640 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926afb8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 -- 172.21.15.114:0/3270454624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 msgr2=0x55926afb8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926afb8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 -- 172.21.15.114:0/3270454624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abf3f60 msgr2=0x55926afa48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abf3f60 0x55926afa48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb24ea00640 1 -- 172.21.15.114:0/3270454624 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926af9d4d0 con 0x55926abd3280 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.957+0000 7fb257e00640 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abf3f60 0x55926afa48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:35.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.958+0000 7fb24ea00640 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 0x55926afa04a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb25806c8c0 tx=0x7fb240009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ea4cfecf242cad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.958+0000 7fb247400640 1 -- 172.21.15.114:0/3270454624 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240031650 con 0x55926abd3280 2024-09-21T06:53:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.958+0000 7fb247400640 1 -- 172.21.15.114:0/3270454624 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb240036040 con 0x55926abd3280 2024-09-21T06:53:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.958+0000 7fb247400640 1 -- 172.21.15.114:0/3270454624 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400043c0 con 0x55926abd3280 2024-09-21T06:53:35.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 -- 172.21.15.114:0/3270454624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 msgr2=0x55926afa04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 0x55926afa04a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb25806c8c0 tx=0x7fb240009360 comp rx=0 tx=0).stop 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 -- 172.21.15.114:0/3270454624 shutdown_connections 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abf3f60 0x55926afa48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abd3280 0x55926afa04a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 --2- 172.21.15.114:0/3270454624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abcb0a0 0x55926afb8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 -- 172.21.15.114:0/3270454624 >> 172.21.15.114:0/3270454624 conn(0x55926abe8350 msgr2=0x55926afa1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 -- 172.21.15.114:0/3270454624 shutdown_connections 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 -- 172.21.15.114:0/3270454624 wait complete. 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.959+0000 7fb264dcc740 1 -- start start 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 Processor -- start 2024-09-21T06:53:35.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abcb0a0 0x55926afc1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 0x55926afba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926afbadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55926afb82e0 con 0x55926abf3f60 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55926afb7c10 con 0x55926abcb0a0 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55926afb89b0 con 0x55926abd3280 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb264dcc740 10 client.0 _ll_register_callbacks cb 0x7fb2480eb4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb257e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926afbadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb24ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 0x55926afba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.960+0000 7fb24ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 0x55926afba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36604/0 (socket says 172.21.15.114:36604) 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 -- 172.21.15.114:0/2868400420 learned_addr learned my addr 172.21.15.114:0/2868400420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb257e00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abcb0a0 0x55926afc1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 -- 172.21.15.114:0/2868400420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abcb0a0 msgr2=0x55926afc1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abcb0a0 0x55926afc1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 -- 172.21.15.114:0/2868400420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 msgr2=0x55926afbadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926afbadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 -- 172.21.15.114:0/2868400420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926afa5890 con 0x55926abd3280 2024-09-21T06:53:35.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb257e00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926afbadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:53:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb24ea00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 0x55926afba6b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb240005e30 tx=0x7fb240031bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20063fc07498e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb244c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:53:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.961+0000 7fb244c00640 1 -- 172.21.15.114:0/2868400420 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24005b020 con 0x55926abd3280 2024-09-21T06:53:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.962+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb240036040 con 0x55926abd3280 2024-09-21T06:53:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.962+0000 7fb244c00640 1 -- 172.21.15.114:0/2868400420 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb240003970 con 0x55926abd3280 2024-09-21T06:53:35.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.962+0000 7fb244c00640 1 -- 172.21.15.114:0/2868400420 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb240004b40 con 0x55926abd3280 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.962+0000 7fb264dcc740 10 client.25384 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.962+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55926b023930 con 0x55926abd3280 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.962+0000 7fb244c00640 1 -- 172.21.15.114:0/2868400420 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb240036040 con 0x55926abd3280 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb244c00640 1 -- 172.21.15.114:0/2868400420 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb240004560 con 0x55926abd3280 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb264dcc740 10 client.25384 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb264dcc740 10 client.25384 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb264dcc740 20 client.25384 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926ab87c90 0x55926b026d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb264dcc740 4 client.25384 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "status"}) -- 0x55926af0f750 con 0x55926ab87c90 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.963+0000 7fb257e00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926ab87c90 0x55926b026d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb257e00640 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926ab87c90 0x55926b026d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cccb7dedd446935b server_cookie=cdd64375a4780417 in_seq=0 out_seq=0 2024-09-21T06:53:35.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb244c00640 10 client.25384 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb244c00640 1 -- 172.21.15.114:0/2868400420 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55926af0f750 con 0x55926ab87c90 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb244c00640 10 client.25384 handle_command_reply: tid=0 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 client.25384 shutdown 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 msgr2=0x55926afba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 0x55926afba6b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb240005e30 tx=0x7fb240031bf0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 shutdown_connections 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55926ab87c90 0x55926b026d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55926abf3f60 0x55926afbadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55926abd3280 0x55926afba6b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 --2- 172.21.15.114:0/2868400420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55926abcb0a0 0x55926afc1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 >> 172.21.15.114:0/2868400420 conn(0x55926abe8350 msgr2=0x55926af97df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb244c00640 0 client.25384 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 shutdown_connections 2024-09-21T06:53:35.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.964+0000 7fb264dcc740 1 -- 172.21.15.114:0/2868400420 wait complete. 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb264dcc740 20 client.25384 trim_cache size 0 max 16384 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 3, 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24727, 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 114.376819195, 2024-09-21T06:53:35.971 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:35.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:35.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:35.972 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 195.210995965 2024-09-21T06:53:35.972 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:35.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb23c0b6560 msgr2=0x7fb23c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb23c0b6560 0x7fb23c0b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb2480e51a0 tx=0x7fb2480eb5d0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 msgr2=0x7fb25819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 0x7fb25819c4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb24800e9f0 tx=0x7fb24800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 shutdown_connections 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb23c0b6560 0x7fb23c0b8a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb25810be40 0x7fb2581a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb25810b4f0 0x7fb25819ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 --2- 172.21.15.114:0/1922686322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb258105bb0 0x7fb25819c4d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.965+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 >> 172.21.15.114:0/1922686322 conn(0x7fb258101680 msgr2=0x7fb25810abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.966+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 shutdown_connections 2024-09-21T06:53:35.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:35.966+0000 7fb25cc00640 1 -- 172.21.15.114:0/1922686322 wait complete. 2024-09-21T06:53:36.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:36 smithi114 ceph-mon[30654]: pgmap v380: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:36.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:36 smithi183 ceph-mon[34219]: pgmap v380: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:36 smithi193 ceph-mon[34601]: pgmap v380: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:36.958 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T06:53:37.326 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 -- 172.21.15.114:0/267465375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc10b6d0 msgr2=0x7fbabc10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 --2- 172.21.15.114:0/267465375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc10b6d0 0x7fbabc10bae0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4009900 tx=0x7fbaa402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 -- 172.21.15.114:0/267465375 shutdown_connections 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 --2- 172.21.15.114:0/267465375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10c020 0x7fbabc112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 --2- 172.21.15.114:0/267465375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc10b6d0 0x7fbabc10bae0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 --2- 172.21.15.114:0/267465375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc105d90 0x7fbabc1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 -- 172.21.15.114:0/267465375 >> 172.21.15.114:0/267465375 conn(0x7fbabc1017c0 msgr2=0x7fbabc103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:37.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 -- 172.21.15.114:0/267465375 shutdown_connections 2024-09-21T06:53:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.939+0000 7fbac1200640 1 -- 172.21.15.114:0/267465375 wait complete. 2024-09-21T06:53:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.940+0000 7fbac1200640 1 Processor -- start 2024-09-21T06:53:37.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.940+0000 7fbac1200640 1 -- start start 2024-09-21T06:53:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbac1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc105d90 0x7fbabc19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbac1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10b6d0 0x7fbabc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbac1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 0x7fbabc1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbac1200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc115ce0 con 0x7fbabc105d90 2024-09-21T06:53:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbac1200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc115b60 con 0x7fbabc10c020 2024-09-21T06:53:37.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbac1200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc115e60 con 0x7fbabc10b6d0 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc105d90 0x7fbabc19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbab1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10b6d0 0x7fbabc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbaba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 0x7fbabc1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc105d90 0x7fbabc19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44070/0 (socket says 172.21.15.114:44070) 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbab1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10b6d0 0x7fbabc19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36622/0 (socket says 172.21.15.114:36622) 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbab9e00640 1 -- 172.21.15.114:0/2784936607 learned_addr learned my addr 172.21.15.114:0/2784936607 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.941+0000 7fbaba800640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 0x7fbabc1a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43330/0 (socket says 172.21.15.114:43330) 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbaba800640 1 -- 172.21.15.114:0/2784936607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10b6d0 msgr2=0x7fbabc19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbaba800640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10b6d0 0x7fbabc19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbaba800640 1 -- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc105d90 msgr2=0x7fbabc19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbaba800640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc105d90 0x7fbabc19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbaba800640 1 -- 172.21.15.114:0/2784936607 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc1a26c0 con 0x7fbabc10c020 2024-09-21T06:53:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbaba800640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 0x7fbabc1a1f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009fd0 tx=0x7fbab400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbab3400640 1 -- 172.21.15.114:0/2784936607 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab400ce20 con 0x7fbabc10c020 2024-09-21T06:53:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc1a29b0 con 0x7fbabc10c020 2024-09-21T06:53:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbab3400640 1 -- 172.21.15.114:0/2784936607 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab4005020 con 0x7fbabc10c020 2024-09-21T06:53:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.942+0000 7fbab3400640 1 -- 172.21.15.114:0/2784936607 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4010690 con 0x7fbabc10c020 2024-09-21T06:53:37.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.943+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc1a9110 con 0x7fbabc10c020 2024-09-21T06:53:37.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.945+0000 7fbac7ead740 1 Processor -- start 2024-09-21T06:53:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.946+0000 7fbab3400640 1 -- 172.21.15.114:0/2784936607 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fbab40054d0 con 0x7fbabc10c020 2024-09-21T06:53:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.946+0000 7fbac7ead740 1 -- start start 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 0x55a0602b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbac7ead740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a06022dd50 con 0x55a05ff0ff60 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbac7ead740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a06021d180 con 0x55a05fee70a0 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbac7ead740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a060220f90 con 0x55a05feef280 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 0x55a0602b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 0x55a0602b6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44078/0 (socket says 172.21.15.114:44078) 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab1400640 1 -- 172.21.15.114:0/1067002226 learned_addr learned my addr 172.21.15.114:0/1067002226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab1400640 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab9e00640 1 -- 172.21.15.114:0/1067002226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 msgr2=0x55a0602b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab9e00640 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602b6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab9e00640 1 -- 172.21.15.114:0/1067002226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 msgr2=0x55a0602b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab9e00640 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 0x55a0602b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.947+0000 7fbab9e00640 1 -- 172.21.15.114:0/1067002226 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0601e7a70 con 0x55a05fee70a0 2024-09-21T06:53:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbab1400640 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 0x55a0602b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbab9e00640 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602b69d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009c80 tx=0x7fbaac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8030d7738c60b0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbab3400640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbaa00b6560 0x7fbaa00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbaab400640 1 -- 172.21.15.114:0/1067002226 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac004230 con 0x55a05fee70a0 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbaab400640 1 -- 172.21.15.114:0/1067002226 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaac0043d0 con 0x55a05fee70a0 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbaab400640 1 -- 172.21.15.114:0/1067002226 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac004c70 con 0x55a05fee70a0 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.948+0000 7fbab3400640 1 -- 172.21.15.114:0/2784936607 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbab40e7eb0 con 0x7fbabc10c020 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbab1400640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbaa00b6560 0x7fbaa00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 -- 172.21.15.114:0/1067002226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 msgr2=0x55a0602b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbab1400640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbaa00b6560 0x7fbaa00b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40057c0 tx=0x7fbaa4000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602b69d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009c80 tx=0x7fbaac011810 comp rx=0 tx=0).stop 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 -- 172.21.15.114:0/1067002226 shutdown_connections 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602b69d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1067002226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff0ff60 0x55a0602b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.949+0000 7fbac7ead740 1 -- 172.21.15.114:0/1067002226 >> 172.21.15.114:0/1067002226 conn(0x55a05ff04350 msgr2=0x55a0602b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.950+0000 7fbac7ead740 1 -- 172.21.15.114:0/1067002226 shutdown_connections 2024-09-21T06:53:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.950+0000 7fbac7ead740 1 -- 172.21.15.114:0/1067002226 wait complete. 2024-09-21T06:53:37.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.950+0000 7fbac7ead740 1 -- start start 2024-09-21T06:53:37.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.950+0000 7fbac7ead740 1 Processor -- start 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 0x55a0602c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05feef280 0x55a0602c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05ff0ff60 0x55a0602cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a06007e5f0 con 0x55a05fee70a0 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0601b8460 con 0x55a05feef280 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0602c4c70 con 0x55a05ff0ff60 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbac7ead740 10 client.0 _ll_register_callbacks cb 0x55a0601c8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 0x55a0602c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 0x55a0602c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44086/0 (socket says 172.21.15.114:44086) 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbab9e00640 1 -- 172.21.15.114:0/533299681 learned_addr learned my addr 172.21.15.114:0/533299681 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbaba800640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05ff0ff60 0x55a0602cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.951+0000 7fbab1400640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05feef280 0x55a0602c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbab9e00640 1 -- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05ff0ff60 msgr2=0x55a0602cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbab9e00640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05ff0ff60 0x55a0602cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbab9e00640 1 -- 172.21.15.114:0/533299681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05feef280 msgr2=0x55a0602c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbab9e00640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05feef280 0x55a0602c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbab9e00640 1 -- 172.21.15.114:0/533299681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05ff7b540 con 0x55a05fee70a0 2024-09-21T06:53:37.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbab9e00640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 0x55a0602c2110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0046e0 tx=0x7fbaac004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7f377a622c74fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:37.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fba9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:53:37.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fba9fe00640 1 -- 172.21.15.114:0/533299681 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac027020 con 0x55a05fee70a0 2024-09-21T06:53:37.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fba9fe00640 1 -- 172.21.15.114:0/533299681 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbaac0038f0 con 0x55a05fee70a0 2024-09-21T06:53:37.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.952+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0601baa70 con 0x55a05fee70a0 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fba9fe00640 1 -- 172.21.15.114:0/533299681 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaac032050 con 0x55a05fee70a0 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a0601d56f0 con 0x55a05fee70a0 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fba9fe00640 1 -- 172.21.15.114:0/533299681 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a0601baa70 con 0x55a05fee70a0 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fba9fe00640 1 -- 172.21.15.114:0/533299681 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbaac018070 con 0x55a05fee70a0 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 10 client.15390 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 20 client.15390 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05fece8e0 0x55a0602d4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 4 client.15390 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.953+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a06022fca0 con 0x55a05fece8e0 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.954+0000 7fbab1400640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05fece8e0 0x55a0602d4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.954+0000 7fbab1400640 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05fece8e0 0x55a0602d4320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1c6fc69f6b29c4f server_cookie=7bb4aa7135752db in_seq=0 out_seq=0 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.954+0000 7fba9fe00640 10 client.15390 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fba9fe00640 1 -- 172.21.15.114:0/533299681 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a06022fca0 con 0x55a05fece8e0 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fba9fe00640 10 client.15390 handle_command_reply: tid=0 2024-09-21T06:53:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 client.15390 shutdown 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 msgr2=0x55a0602c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 0x55a0602c2110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0046e0 tx=0x7fbaac004710 comp rx=0 tx=0).stop 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 shutdown_connections 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05fece8e0 0x55a0602d4320 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05ff0ff60 0x55a0602cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05feef280 0x55a0602c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 --2- 172.21.15.114:0/533299681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05fee70a0 0x55a0602c2110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 >> 172.21.15.114:0/533299681 conn(0x55a05ff04350 msgr2=0x55a0602b9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.955+0000 7fba9fe00640 0 client.15390 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.956+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 shutdown_connections 2024-09-21T06:53:37.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.956+0000 7fbac7ead740 1 -- 172.21.15.114:0/533299681 wait complete. 2024-09-21T06:53:37.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.956+0000 7fbac7ead740 20 client.15390 trim_cache size 0 max 16384 2024-09-21T06:53:37.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.974+0000 7fbac7ead740 1 Processor -- start 2024-09-21T06:53:37.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.974+0000 7fbac7ead740 1 -- start start 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff04350 0x55a0602d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbac7ead740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a0602d2660 con 0x55a05ff04350 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbac7ead740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0602d2d30 con 0x55a05fee70a0 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbac7ead740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0602d3400 con 0x55a05feef280 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff04350 0x55a0602d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43352/0 (socket says 172.21.15.114:43352) 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 -- 172.21.15.114:0/1669245049 learned_addr learned my addr 172.21.15.114:0/1669245049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbaba800640 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 -- 172.21.15.114:0/1669245049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 msgr2=0x55a0602bc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602bc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 -- 172.21.15.114:0/1669245049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff04350 msgr2=0x55a0602d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff04350 0x55a0602d3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.975+0000 7fbab1400640 1 -- 172.21.15.114:0/1669245049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0602b8780 con 0x55a05fee70a0 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.976+0000 7fbab1400640 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602cdf00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4006010 tx=0x7fbaa403b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f47fd29815ff71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.976+0000 7fbaaaa00640 1 -- 172.21.15.114:0/1669245049 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4058070 con 0x55a05fee70a0 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.976+0000 7fbaaaa00640 1 -- 172.21.15.114:0/1669245049 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa40035f0 con 0x55a05fee70a0 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.976+0000 7fbaaaa00640 1 -- 172.21.15.114:0/1669245049 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4035040 con 0x55a05fee70a0 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 -- 172.21.15.114:0/1669245049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 msgr2=0x55a0602cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602cdf00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4006010 tx=0x7fbaa403b000 comp rx=0 tx=0).stop 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 -- 172.21.15.114:0/1669245049 shutdown_connections 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05feef280 0x55a0602bc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05fee70a0 0x55a0602cdf00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 --2- 172.21.15.114:0/1669245049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05ff04350 0x55a0602d3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 -- 172.21.15.114:0/1669245049 >> 172.21.15.114:0/1669245049 conn(0x55a0602d2250 msgr2=0x55a0602bd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:37.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 -- 172.21.15.114:0/1669245049 shutdown_connections 2024-09-21T06:53:37.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 -- 172.21.15.114:0/1669245049 wait complete. 2024-09-21T06:53:37.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 -- start start 2024-09-21T06:53:37.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.977+0000 7fbac7ead740 1 Processor -- start 2024-09-21T06:53:37.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05fee70a0 0x55a0602dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05feef280 0x55a0602dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 0x55a0602d5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a0602d41a0 con 0x55a05feef280 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0602d3ad0 con 0x55a05ff04350 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0602d4870 con 0x55a05fee70a0 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbac7ead740 10 client.0 _ll_register_callbacks cb 0x7fbaac004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbaba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 0x55a0602d5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbab1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05feef280 0x55a0602dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbaba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 0x55a0602d5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43366/0 (socket says 172.21.15.114:43366) 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbab9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05fee70a0 0x55a0602dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.978+0000 7fbaba800640 1 -- 172.21.15.114:0/3828687988 learned_addr learned my addr 172.21.15.114:0/3828687988 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbaba800640 1 -- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05fee70a0 msgr2=0x55a0602dc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbaba800640 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05fee70a0 0x55a0602dc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbaba800640 1 -- 172.21.15.114:0/3828687988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05feef280 msgr2=0x55a0602dcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbaba800640 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05feef280 0x55a0602dcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbaba800640 1 -- 172.21.15.114:0/3828687988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0602dd2d0 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbab9e00640 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05fee70a0 0x55a0602dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbaba800640 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 0x55a0602d5520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbab40e5460 tx=0x7fbab40a99f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=553b54457f474a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fba9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fba9fe00640 1 -- 172.21.15.114:0/3828687988 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab40b2350 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fba9fe00640 1 -- 172.21.15.114:0/3828687988 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbab40cfb60 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaa40035f0 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fba9fe00640 1 -- 172.21.15.114:0/3828687988 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbab40ae250 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.979+0000 7fbac7ead740 10 client.25397 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a06033fd20 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fba9fe00640 1 -- 172.21.15.114:0/3828687988 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaa40035f0 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fba9fe00640 1 -- 172.21.15.114:0/3828687988 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fbab40a94a0 con 0x55a05ff04350 2024-09-21T06:53:37.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 10 client.25397 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:37.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 10 client.25397 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:53:37.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 20 client.25397 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:37.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05ff0ff60 0x55a060343610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:37.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 4 client.25397 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:37.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "status"}) -- 0x55a06022b750 con 0x55a05ff0ff60 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.980+0000 7fbab9e00640 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05ff0ff60 0x55a060343610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fbab9e00640 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05ff0ff60 0x55a060343610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71b1145e7aa192d server_cookie=577ee22bb86b4511 in_seq=0 out_seq=0 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fba9fe00640 10 client.25397 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fba9fe00640 1 -- 172.21.15.114:0/3828687988 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a06022b750 con 0x55a05ff0ff60 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fba9fe00640 10 client.25397 handle_command_reply: tid=0 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fbac7ead740 1 client.25397 shutdown 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 msgr2=0x55a0602d5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.981+0000 7fbac7ead740 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 0x55a0602d5520 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbab40e5460 tx=0x7fbab40a99f0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 shutdown_connections 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55a05ff0ff60 0x55a060343610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a05ff04350 0x55a0602d5520 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a05feef280 0x55a0602dcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 --2- 172.21.15.114:0/3828687988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a05fee70a0 0x55a0602dc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fba9fe00640 0 client.25397 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 >> 172.21.15.114:0/3828687988 conn(0x55a0602d2250 msgr2=0x55a0602c06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 shutdown_connections 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 1 -- 172.21.15.114:0/3828687988 wait complete. 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac7ead740 20 client.25397 trim_cache size 0 max 16384 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:37.987 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 2, 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24742, 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 117.40393980499999, 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 192.00605139199999 2024-09-21T06:53:37.988 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbaa00b6560 msgr2=0x7fbaa00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac1200640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbaa00b6560 0x7fbaa00b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40057c0 tx=0x7fbaa4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 msgr2=0x7fbabc1a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.982+0000 7fbac1200640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 0x7fbabc1a1f90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009fd0 tx=0x7fbab400ef10 comp rx=0 tx=0).stop 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 shutdown_connections 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbaa00b6560 0x7fbaa00b8a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc10c020 0x7fbabc1a1f90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbabc10b6d0 0x7fbabc19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 --2- 172.21.15.114:0/2784936607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbabc105d90 0x7fbabc19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 >> 172.21.15.114:0/2784936607 conn(0x7fbabc1017c0 msgr2=0x7fbabc10ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 shutdown_connections 2024-09-21T06:53:37.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:37.983+0000 7fbac1200640 1 -- 172.21.15.114:0/2784936607 wait complete. 2024-09-21T06:53:38.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:38 smithi114 ceph-mon[30654]: pgmap v381: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:38.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:38 smithi183 ceph-mon[34219]: pgmap v381: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:38.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:38 smithi193 ceph-mon[34601]: pgmap v381: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:38.979 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T06:53:39.348 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 -- 172.21.15.114:0/1035969554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18105d80 msgr2=0x7f8a18106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1035969554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18105d80 0x7f8a18106190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009900 tx=0x7f8a0802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 -- 172.21.15.114:0/1035969554 shutdown_connections 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1035969554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a1810c010 0x7f8a18112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1035969554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1035969554 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18105d80 0x7f8a18106190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 -- 172.21.15.114:0/1035969554 >> 172.21.15.114:0/1035969554 conn(0x7f8a181017d0 msgr2=0x7f8a18103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 -- 172.21.15.114:0/1035969554 shutdown_connections 2024-09-21T06:53:39.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.982+0000 7f8a1d400640 1 -- 172.21.15.114:0/1035969554 wait complete. 2024-09-21T06:53:39.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.983+0000 7f8a1d400640 1 Processor -- start 2024-09-21T06:53:39.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.983+0000 7f8a1d400640 1 -- start start 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a1d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18105d80 0x7f8a1819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a1d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a1d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a1810c010 0x7f8a181a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a1d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18115cd0 con 0x7f8a1810c010 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a1d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18115b50 con 0x7f8a1810b6c0 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a1d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18115e50 con 0x7f8a18105d80 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a17200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a1810c010 0x7f8a181a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a16800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18105d80 0x7f8a1819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43380/0 (socket says 172.21.15.114:43380) 2024-09-21T06:53:39.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a16800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18105d80 0x7f8a1819c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36676/0 (socket says 172.21.15.114:36676) 2024-09-21T06:53:39.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.984+0000 7f8a0de00640 1 -- 172.21.15.114:0/1887229145 learned_addr learned my addr 172.21.15.114:0/1887229145 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:39.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0de00640 1 -- 172.21.15.114:0/1887229145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18105d80 msgr2=0x7f8a1819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:39.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18105d80 0x7f8a1819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0de00640 1 -- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a1810c010 msgr2=0x7f8a181a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a1810c010 0x7f8a181a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0de00640 1 -- 172.21.15.114:0/1887229145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a181a2650 con 0x7f8a1810b6c0 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1819cc80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9f0 tx=0x7f8a0000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0fe00640 1 -- 172.21.15.114:0/1887229145 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000ccc0 con 0x7f8a1810b6c0 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181a2940 con 0x7f8a1810b6c0 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0fe00640 1 -- 172.21.15.114:0/1887229145 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a000108e0 con 0x7f8a1810b6c0 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.985+0000 7f8a0fe00640 1 -- 172.21.15.114:0/1887229145 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00018860 con 0x7f8a1810b6c0 2024-09-21T06:53:39.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.986+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181a90a0 con 0x7f8a1810b6c0 2024-09-21T06:53:39.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.988+0000 7f8a240b7740 1 Processor -- start 2024-09-21T06:53:39.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 -- start start 2024-09-21T06:53:39.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b85490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b85f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ee09afcd50 con 0x55ee097def60 2024-09-21T06:53:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee09aec180 con 0x55ee097b60a0 2024-09-21T06:53:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a240b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee09aeff90 con 0x55ee097be280 2024-09-21T06:53:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a17200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b85490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a17200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b85490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44158/0 (socket says 172.21.15.114:44158) 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a17200640 1 -- 172.21.15.114:0/2238000041 learned_addr learned my addr 172.21.15.114:0/2238000041 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a16800640 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.989+0000 7f8a0fe00640 1 -- 172.21.15.114:0/1887229145 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f8a00010430 con 0x7f8a1810b6c0 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a17200640 1 -- 172.21.15.114:0/2238000041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 msgr2=0x55ee09b85f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a17200640 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b85f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a17200640 1 -- 172.21.15.114:0/2238000041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 msgr2=0x55ee09b859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a17200640 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b859d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a17200640 1 -- 172.21.15.114:0/2238000041 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee09ab6a70 con 0x55ee097def60 2024-09-21T06:53:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a16800640 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a17200640 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b85490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a10010370 tx=0x7f8a10010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6ecaa3c7bbf515 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a07400640 1 -- 172.21.15.114:0/2238000041 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10016070 con 0x55ee097def60 2024-09-21T06:53:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a07400640 1 -- 172.21.15.114:0/2238000041 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10004d20 con 0x55ee097def60 2024-09-21T06:53:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.990+0000 7f8a07400640 1 -- 172.21.15.114:0/2238000041 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100053c0 con 0x55ee097def60 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.991+0000 7f8a240b7740 1 -- 172.21.15.114:0/2238000041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 msgr2=0x55ee09b85490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.991+0000 7f8a240b7740 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b85490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a10010370 tx=0x7f8a10010830 comp rx=0 tx=0).stop 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.991+0000 7f8a0fe00640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6560 0x7f89fc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 -- 172.21.15.114:0/2238000041 shutdown_connections 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a0fe00640 1 -- 172.21.15.114:0/1887229145 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8a000e7ae0 con 0x7f8a1810b6c0 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b85f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b859d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 --2- 172.21.15.114:0/2238000041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b85490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 -- 172.21.15.114:0/2238000041 >> 172.21.15.114:0/2238000041 conn(0x55ee097d3350 msgr2=0x55ee09b83380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a17200640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6560 0x7f89fc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 -- 172.21.15.114:0/2238000041 shutdown_connections 2024-09-21T06:53:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 -- 172.21.15.114:0/2238000041 wait complete. 2024-09-21T06:53:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a240b7740 1 -- start start 2024-09-21T06:53:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.992+0000 7f8a17200640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6560 0x7f89fc0b8a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a10010fa0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 Processor -- start 2024-09-21T06:53:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 0x55ee09b910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097def60 0x55ee09b9cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ee0994d5f0 con 0x55ee097b60a0 2024-09-21T06:53:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee09a87460 con 0x55ee097def60 2024-09-21T06:53:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee09b93c70 con 0x55ee097be280 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a240b7740 10 client.0 _ll_register_callbacks cb 0x55ee09a97210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a16800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 0x55ee09b910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a16800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 0x55ee09b910b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44164/0 (socket says 172.21.15.114:44164) 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a16800640 1 -- 172.21.15.114:0/1542371591 learned_addr learned my addr 172.21.15.114:0/1542371591 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a17200640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097def60 0x55ee09b9cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.993+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f8a16800640 1 -- 172.21.15.114:0/1542371591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 msgr2=0x55ee09b915f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f8a16800640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b915f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f8a16800640 1 -- 172.21.15.114:0/1542371591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097def60 msgr2=0x55ee09b9cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f8a16800640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097def60 0x55ee09b9cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f8a16800640 1 -- 172.21.15.114:0/1542371591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0984a540 con 0x55ee097b60a0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f8a16800640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 0x55ee09b910b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802a910 tx=0x7f8a08032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f418a4f12bb1862 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f89fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:53:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f89fbe00640 1 -- 172.21.15.114:0/1542371591 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08035040 con 0x55ee097b60a0 2024-09-21T06:53:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.994+0000 7f89fbe00640 1 -- 172.21.15.114:0/1542371591 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8a080317a0 con 0x55ee097b60a0 2024-09-21T06:53:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee09a89a70 con 0x55ee097b60a0 2024-09-21T06:53:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f89fbe00640 1 -- 172.21.15.114:0/1542371591 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a08004040 con 0x55ee097b60a0 2024-09-21T06:53:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f8a240b7740 10 client.15408 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee09aa46f0 con 0x55ee097b60a0 2024-09-21T06:53:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f89fbe00640 1 -- 172.21.15.114:0/1542371591 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee09a89a70 con 0x55ee097b60a0 2024-09-21T06:53:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f89fbe00640 1 -- 172.21.15.114:0/1542371591 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8a08004870 con 0x55ee097b60a0 2024-09-21T06:53:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.995+0000 7f8a240b7740 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a240b7740 10 client.15408 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a240b7740 20 client.15408 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee0979d8e0 0x55ee09ba3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a240b7740 4 client.15408 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee09afeca0 con 0x55ee0979d8e0 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee0979d8e0 0x55ee09ba3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee0979d8e0 0x55ee09ba3560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5e8e4450224b6da server_cookie=4baf34e888bc6b2 in_seq=0 out_seq=0 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.996+0000 7f89fbe00640 10 client.15408 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.997+0000 7f89fbe00640 1 -- 172.21.15.114:0/1542371591 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee09afeca0 con 0x55ee0979d8e0 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.997+0000 7f89fbe00640 10 client.15408 handle_command_reply: tid=0 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.997+0000 7f8a240b7740 1 client.15408 shutdown 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.997+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 msgr2=0x55ee09b910b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.997+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 0x55ee09b910b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8a0802a910 tx=0x7f8a08032f20 comp rx=0 tx=0).stop 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.997+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 shutdown_connections 2024-09-21T06:53:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee0979d8e0 0x55ee09ba3560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097def60 0x55ee09b9cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b915f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1542371591 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097b60a0 0x55ee09b910b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f89fbe00640 0 client.15408 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 >> 172.21.15.114:0/1542371591 conn(0x55ee097d3350 msgr2=0x55ee09b88670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 shutdown_connections 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 1 -- 172.21.15.114:0/1542371591 wait complete. 2024-09-21T06:53:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:39.998+0000 7f8a240b7740 20 client.15408 trim_cache size 0 max 16384 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.016+0000 7f8a240b7740 1 Processor -- start 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.016+0000 7f8a240b7740 1 -- start start 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.016+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b9ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b8f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b8fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a240b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ee09ba18a0 con 0x55ee097def60 2024-09-21T06:53:40.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a240b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee09ba1f70 con 0x55ee097b60a0 2024-09-21T06:53:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a240b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee09ba2640 con 0x55ee097be280 2024-09-21T06:53:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b8f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b8f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36716/0 (socket says 172.21.15.114:36716) 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 -- 172.21.15.114:0/3640425118 learned_addr learned my addr 172.21.15.114:0/3640425118 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a16800640 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b9ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a17200640 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b8fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 -- 172.21.15.114:0/3640425118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 msgr2=0x55ee09b9ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b9ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 -- 172.21.15.114:0/3640425118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 msgr2=0x55ee09b8fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b8fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.017+0000 7f8a0de00640 1 -- 172.21.15.114:0/3640425118 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee09b92170 con 0x55ee097be280 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.018+0000 7f8a0de00640 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b8f730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a000ad530 tx=0x7f8a00000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d082582378cb0b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.018+0000 7f8a06a00640 1 -- 172.21.15.114:0/3640425118 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000b2260 con 0x55ee097be280 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.018+0000 7f8a06a00640 1 -- 172.21.15.114:0/3640425118 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a000b71a0 con 0x55ee097be280 2024-09-21T06:53:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.018+0000 7f8a06a00640 1 -- 172.21.15.114:0/3640425118 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000ac500 con 0x55ee097be280 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 -- 172.21.15.114:0/3640425118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 msgr2=0x55ee09b8f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b8f730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a000ad530 tx=0x7f8a00000bc0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 -- 172.21.15.114:0/3640425118 shutdown_connections 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097def60 0x55ee09b8fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097be280 0x55ee09b8f730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 --2- 172.21.15.114:0/3640425118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09b9ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 -- 172.21.15.114:0/3640425118 >> 172.21.15.114:0/3640425118 conn(0x55ee097d3350 msgr2=0x55ee09b91090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 -- 172.21.15.114:0/3640425118 shutdown_connections 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 -- 172.21.15.114:0/3640425118 wait complete. 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 -- start start 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.019+0000 7f8a240b7740 1 Processor -- start 2024-09-21T06:53:40.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09ba5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 0x55ee09ba5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097def60 0x55ee09ba63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ee09ba33e0 con 0x55ee097be280 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee09ba2d10 con 0x55ee097b60a0 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee09ba3ab0 con 0x55ee097def60 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a240b7740 10 client.0 _ll_register_callbacks cb 0x7f8a08032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 0x55ee09ba5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a0de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 0x55ee09ba5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44186/0 (socket says 172.21.15.114:44186) 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a0de00640 1 -- 172.21.15.114:0/1984370673 learned_addr learned my addr 172.21.15.114:0/1984370673 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a17200640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097def60 0x55ee09ba63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.020+0000 7f8a16800640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09ba5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a0de00640 1 -- 172.21.15.114:0/1984370673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097def60 msgr2=0x55ee09ba63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097def60 0x55ee09ba63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a0de00640 1 -- 172.21.15.114:0/1984370673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 msgr2=0x55ee09ba5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09ba5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a0de00640 1 -- 172.21.15.114:0/1984370673 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee09bab210 con 0x55ee097be280 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a0de00640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 0x55ee09ba5e60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a000aaaa0 tx=0x7f8a0000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91820a689aecdc81 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f89fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f89fbe00640 1 -- 172.21.15.114:0/1984370673 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000a9380 con 0x55ee097be280 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f89fbe00640 1 -- 172.21.15.114:0/1984370673 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8a000aeab0 con 0x55ee097be280 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a000b71a0 con 0x55ee097be280 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f89fbe00640 1 -- 172.21.15.114:0/1984370673 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8a00010a90 con 0x55ee097be280 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a240b7740 10 client.15414 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:40.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.021+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee09c0ede0 con 0x55ee097be280 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f89fbe00640 1 -- 172.21.15.114:0/1984370673 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a000b71a0 con 0x55ee097be280 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f89fbe00640 1 -- 172.21.15.114:0/1984370673 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8a000b33f0 con 0x55ee097be280 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a240b7740 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a240b7740 10 client.15414 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a240b7740 20 client.15414 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee09772c90 0x55ee09c121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a240b7740 4 client.15414 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "status"}) -- 0x55ee09afa750 con 0x55ee09772c90 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.022+0000 7f8a16800640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee09772c90 0x55ee09c121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f89fbe00640 10 client.15414 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a16800640 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee09772c90 0x55ee09c121c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91840adf21bda244 server_cookie=aefd5218363ec39f in_seq=0 out_seq=0 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f89fbe00640 1 -- 172.21.15.114:0/1984370673 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ee09afa750 con 0x55ee09772c90 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f89fbe00640 10 client.15414 handle_command_reply: tid=0 2024-09-21T06:53:40.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a240b7740 1 client.15414 shutdown 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 msgr2=0x55ee09ba5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 0x55ee09ba5e60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8a000aaaa0 tx=0x7f8a0000ce60 comp rx=0 tx=0).stop 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 shutdown_connections 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55ee09772c90 0x55ee09c121c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.023+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee097def60 0x55ee09ba63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee097be280 0x55ee09ba5e60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f8a240b7740 1 --2- 172.21.15.114:0/1984370673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee097b60a0 0x55ee09ba5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 >> 172.21.15.114:0/1984370673 conn(0x55ee097d3350 msgr2=0x55ee09b8f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f89fbe00640 0 client.15414 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 shutdown_connections 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f8a240b7740 1 -- 172.21.15.114:0/1984370673 wait complete. 2024-09-21T06:53:40.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.024+0000 7f8a240b7740 20 client.15414 trim_cache size 0 max 16384 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 1, 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24760, 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 120.45719275, 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 187.14116164000001 2024-09-21T06:53:40.030 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:40.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6560 msgr2=0x7f89fc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:40.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6560 0x7f89fc0b8a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a10005fd0 tx=0x7f8a10010fa0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 msgr2=0x7f8a1819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:40.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1819cc80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e9f0 tx=0x7f8a0000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 shutdown_connections 2024-09-21T06:53:40.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a1810c010 0x7f8a181a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6560 0x7f89fc0b8a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1810b6c0 0x7f8a1819cc80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 --2- 172.21.15.114:0/1887229145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18105d80 0x7f8a1819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:40.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 >> 172.21.15.114:0/1887229145 conn(0x7f8a181017d0 msgr2=0x7f8a18103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:40.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 shutdown_connections 2024-09-21T06:53:40.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:40.025+0000 7f8a1d400640 1 -- 172.21.15.114:0/1887229145 wait complete. 2024-09-21T06:53:40.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:40 smithi114 ceph-mon[30654]: pgmap v382: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:40.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:40 smithi183 ceph-mon[34219]: pgmap v382: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:40.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:40 smithi193 ceph-mon[34601]: pgmap v382: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:41.058 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:53:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 -- 172.21.15.114:0/1883482196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab6010be50 msgr2=0x7fab60112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 --2- 172.21.15.114:0/1883482196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab6010be50 0x7fab60112680 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fab58009900 tx=0x7fab5802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 -- 172.21.15.114:0/1883482196 shutdown_connections 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 --2- 172.21.15.114:0/1883482196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab6010be50 0x7fab60112680 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 --2- 172.21.15.114:0/1883482196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 0x7fab6010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 --2- 172.21.15.114:0/1883482196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab60105bc0 0x7fab60105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 -- 172.21.15.114:0/1883482196 >> 172.21.15.114:0/1883482196 conn(0x7fab60101670 msgr2=0x7fab60103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:42.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 -- 172.21.15.114:0/1883482196 shutdown_connections 2024-09-21T06:53:42.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.069+0000 7fab65400640 1 -- 172.21.15.114:0/1883482196 wait complete. 2024-09-21T06:53:42.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.070+0000 7fab65400640 1 Processor -- start 2024-09-21T06:53:42.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.070+0000 7fab65400640 1 -- start start 2024-09-21T06:53:42.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab65400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab60105bc0 0x7fab6019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab65400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 0x7fab6019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab65400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab6010be50 0x7fab601a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab65400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fab60115b10 con 0x7fab60105bc0 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab65400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab60115990 con 0x7fab6010b500 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab65400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fab60115c90 con 0x7fab6010be50 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab60105bc0 0x7fab6019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab60105bc0 0x7fab6019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44202/0 (socket says 172.21.15.114:44202) 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab57400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 0x7fab6019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab5fe00640 1 -- 172.21.15.114:0/2626606347 learned_addr learned my addr 172.21.15.114:0/2626606347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.071+0000 7fab64a00640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab6010be50 0x7fab601a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab57400640 1 -- 172.21.15.114:0/2626606347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab6010be50 msgr2=0x7fab601a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab57400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab6010be50 0x7fab601a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab57400640 1 -- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab60105bc0 msgr2=0x7fab6019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab57400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab60105bc0 0x7fab6019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab57400640 1 -- 172.21.15.114:0/2626606347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab601a2430 con 0x7fab6010b500 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab57400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 0x7fab6019ca60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab6006c8a0 tx=0x7fab4800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab5d600640 1 -- 172.21.15.114:0/2626606347 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4800cc40 con 0x7fab6010b500 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab601a2720 con 0x7fab6010b500 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab5d600640 1 -- 172.21.15.114:0/2626606347 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab4800cde0 con 0x7fab6010b500 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab5d600640 1 -- 172.21.15.114:0/2626606347 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48018760 con 0x7fab6010b500 2024-09-21T06:53:42.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.072+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab601a8e80 con 0x7fab6010b500 2024-09-21T06:53:42.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.076+0000 7fab6d652740 1 Processor -- start 2024-09-21T06:53:42.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.076+0000 7fab6d652740 1 -- start start 2024-09-21T06:53:42.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab6d652740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab6d652740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab6d652740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab6d652740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a99a049d50 con 0x55a999d2bf60 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab6d652740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a99a039180 con 0x55a999d0b280 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab6d652740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a99a03cf90 con 0x55a999d030a0 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab64a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab64a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0d2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44208/0 (socket says 172.21.15.114:44208) 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab64a00640 1 -- 172.21.15.114:0/2072654666 learned_addr learned my addr 172.21.15.114:0/2072654666 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:42.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab5fe00640 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.077+0000 7fab5d600640 1 -- 172.21.15.114:0/2626606347 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fab48010a90 con 0x7fab6010b500 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab64a00640 1 -- 172.21.15.114:0/2072654666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 msgr2=0x55a99a0d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab64a00640 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab64a00640 1 -- 172.21.15.114:0/2072654666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 msgr2=0x55a99a0d30f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab64a00640 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0d30f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab64a00640 1 -- 172.21.15.114:0/2072654666 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99a003a70 con 0x55a999d2bf60 2024-09-21T06:53:42.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab5fe00640 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:42.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab64a00640 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0d2670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fab58009900 tx=0x7fab58032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334a7315ec6ac1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:42.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab55200640 1 -- 172.21.15.114:0/2072654666 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58040070 con 0x55a999d2bf60 2024-09-21T06:53:42.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab55200640 1 -- 172.21.15.114:0/2072654666 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab58004030 con 0x55a999d2bf60 2024-09-21T06:53:42.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.078+0000 7fab55200640 1 -- 172.21.15.114:0/2072654666 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58035040 con 0x55a999d2bf60 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.079+0000 7fab5d600640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fab440b6560 0x7fab440b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.079+0000 7fab6d652740 1 -- 172.21.15.114:0/2072654666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 msgr2=0x55a99a0d2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.079+0000 7fab6d652740 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0d2670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fab58009900 tx=0x7fab58032b20 comp rx=0 tx=0).stop 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.079+0000 7fab5d600640 1 -- 172.21.15.114:0/2626606347 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fab480e7a80 con 0x7fab6010b500 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 -- 172.21.15.114:0/2072654666 shutdown_connections 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 --2- 172.21.15.114:0/2072654666 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0d2670 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 -- 172.21.15.114:0/2072654666 >> 172.21.15.114:0/2072654666 conn(0x55a999d20350 msgr2=0x55a99a0d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:42.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 -- 172.21.15.114:0/2072654666 shutdown_connections 2024-09-21T06:53:42.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab5fe00640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fab440b6560 0x7fab440b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab6d652740 1 -- 172.21.15.114:0/2072654666 wait complete. 2024-09-21T06:53:42.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.080+0000 7fab5fe00640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fab440b6560 0x7fab440b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55a99a0d12f0 tx=0x7fab50009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:42.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 -- start start 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 Processor -- start 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a999e9a5f0 con 0x55a999d2bf60 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a999fd4460 con 0x55a999d0b280 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0e0df0 con 0x55a999d030a0 2024-09-21T06:53:42.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.081+0000 7fab6d652740 10 client.0 _ll_register_callbacks cb 0x55a999fe4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab64a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab57400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab64a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0e40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44214/0 (socket says 172.21.15.114:44214) 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab64a00640 1 -- 172.21.15.114:0/409650409 learned_addr learned my addr 172.21.15.114:0/409650409 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab57400640 1 -- 172.21.15.114:0/409650409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 msgr2=0x55a99a0de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab57400640 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab57400640 1 -- 172.21.15.114:0/409650409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 msgr2=0x55a99a0e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab57400640 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab57400640 1 -- 172.21.15.114:0/409650409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a999d97540 con 0x55a999d0b280 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.082+0000 7fab64a00640 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab57400640 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0de8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fab480ad530 tx=0x7fab480ace60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af35f89543d11ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:42.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab4d600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab4d600640 1 -- 172.21.15.114:0/409650409 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480acfa0 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab4d600640 1 -- 172.21.15.114:0/409650409 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fab480b33f0 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a999fd6a70 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab4d600640 1 -- 172.21.15.114:0/409650409 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab480ad340 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab6d652740 10 client.25424 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a999ff16f0 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.083+0000 7fab4d600640 1 -- 172.21.15.114:0/409650409 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a999fd6a70 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab4d600640 1 -- 172.21.15.114:0/409650409 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fab480b66e0 con 0x55a999d0b280 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab6d652740 10 client.25424 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab6d652740 10 client.25424 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:53:42.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab6d652740 20 client.25424 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:42.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab6d652740 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cea8e0 0x55a99a0f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab6d652740 4 client.25424 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:53:42.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a99a04bca0 con 0x55a999cea8e0 2024-09-21T06:53:42.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.084+0000 7fab5fe00640 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cea8e0 0x55a99a0f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.085+0000 7fab5fe00640 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cea8e0 0x55a99a0f0730 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df385bd0dd49f2df server_cookie=76825eff784df629 in_seq=0 out_seq=0 2024-09-21T06:53:42.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.085+0000 7fab4d600640 10 client.25424 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab4d600640 1 -- 172.21.15.114:0/409650409 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a99a04bca0 con 0x55a999cea8e0 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab4d600640 10 client.25424 handle_command_reply: tid=0 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 client.25424 shutdown 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 msgr2=0x55a99a0de8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0de8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fab480ad530 tx=0x7fab480ace60 comp rx=0 tx=0).stop 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 shutdown_connections 2024-09-21T06:53:42.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d2bf60 0x55a99a0e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d0b280 0x55a99a0de8f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cea8e0 0x55a99a0f0730 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 --2- 172.21.15.114:0/409650409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 >> 172.21.15.114:0/409650409 conn(0x55a999d20350 msgr2=0x55a99a0d5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab4d600640 0 client.25424 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 shutdown_connections 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 1 -- 172.21.15.114:0/409650409 wait complete. 2024-09-21T06:53:42.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.086+0000 7fab6d652740 20 client.25424 trim_cache size 0 max 16384 2024-09-21T06:53:42.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 Processor -- start 2024-09-21T06:53:42.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 -- start start 2024-09-21T06:53:42.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d2bf60 0x55a99a0d7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0eea70 con 0x55a999d0b280 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0ef140 con 0x55a999d2bf60 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.105+0000 7fab6d652740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0ef810 con 0x55a999d030a0 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44230/0 (socket says 172.21.15.114:44230) 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 -- 172.21.15.114:0/1650473245 learned_addr learned my addr 172.21.15.114:0/1650473245 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab64a00640 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d2bf60 0x55a99a0d7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab5fe00640 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 -- 172.21.15.114:0/1650473245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 msgr2=0x55a99a0f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 -- 172.21.15.114:0/1650473245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d2bf60 msgr2=0x55a99a0d7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d2bf60 0x55a99a0d7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.106+0000 7fab57400640 1 -- 172.21.15.114:0/1650473245 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a99a0d6780 con 0x55a999d0b280 2024-09-21T06:53:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.107+0000 7fab57400640 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d7980 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fab480057a0 tx=0x7fab480b7090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd31f4e140c9663 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:42.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.107+0000 7fab4fe00640 1 -- 172.21.15.114:0/1650473245 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480a9e80 con 0x55a999d0b280 2024-09-21T06:53:42.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.107+0000 7fab4fe00640 1 -- 172.21.15.114:0/1650473245 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab480b2260 con 0x55a999d0b280 2024-09-21T06:53:42.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.107+0000 7fab4fe00640 1 -- 172.21.15.114:0/1650473245 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480b33f0 con 0x55a999d0b280 2024-09-21T06:53:42.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 -- 172.21.15.114:0/1650473245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 msgr2=0x55a99a0d7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d7980 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fab480057a0 tx=0x7fab480b7090 comp rx=0 tx=0).stop 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 -- 172.21.15.114:0/1650473245 shutdown_connections 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d2bf60 0x55a99a0d7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d7980 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 --2- 172.21.15.114:0/1650473245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d030a0 0x55a99a0f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 -- 172.21.15.114:0/1650473245 >> 172.21.15.114:0/1650473245 conn(0x55a999d20350 msgr2=0x55a99a0ee1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 -- 172.21.15.114:0/1650473245 shutdown_connections 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 -- 172.21.15.114:0/1650473245 wait complete. 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 -- start start 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.108+0000 7fab6d652740 1 Processor -- start 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d030a0 0x55a99a0d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d2bf60 0x55a99a0e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0f05b0 con 0x55a999d0b280 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0efee0 con 0x55a999d030a0 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a99a0f0c80 con 0x55a999d2bf60 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab6d652740 10 client.0 _ll_register_callbacks cb 0x7fab480a97a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab57400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab5fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d030a0 0x55a99a0d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab57400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44246/0 (socket says 172.21.15.114:44246) 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab57400640 1 -- 172.21.15.114:0/3856217173 learned_addr learned my addr 172.21.15.114:0/3856217173 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.109+0000 7fab64a00640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d2bf60 0x55a99a0e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab5fe00640 1 -- 172.21.15.114:0/3856217173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d2bf60 msgr2=0x55a99a0e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab5fe00640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d2bf60 0x55a99a0e9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab5fe00640 1 -- 172.21.15.114:0/3856217173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 msgr2=0x55a99a0d6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab5fe00640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab5fe00640 1 -- 172.21.15.114:0/3856217173 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a99a0ea290 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab57400640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab5fe00640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d030a0 0x55a99a0d6950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55a99a0f1570 tx=0x7fab50017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c60481d98bda870 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab4d600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab4d600640 1 -- 172.21.15.114:0/3856217173 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5001d670 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab480b2260 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.110+0000 7fab4d600640 1 -- 172.21.15.114:0/3856217173 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fab5001d810 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab4d600640 1 -- 172.21.15.114:0/3856217173 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab5003e050 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 10 client.25430 fetch_fsmap learned FSMap version 28 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a99a15bc00 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab4d600640 1 -- 172.21.15.114:0/3856217173 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab480b2260 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab4d600640 1 -- 172.21.15.114:0/3856217173 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fab5002bc20 con 0x55a999d030a0 2024-09-21T06:53:42.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 10 client.25430 fetch_fsmap finished waiting for FSMap version 28 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 10 client.25430 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 20 client.25430 populate_metadata read hostname 'smithi114' 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cbfc90 0x55a99a15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 4 client.25430 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.111+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "status"}) -- 0x55a99a047750 con 0x55a999cbfc90 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab57400640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cbfc90 0x55a99a15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab57400640 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cbfc90 0x55a99a15f4d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8b68026c870ec8 server_cookie=96a3f747f2f1b8e0 in_seq=0 out_seq=0 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab4d600640 10 client.25430 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab4d600640 1 -- 172.21.15.114:0/3856217173 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a99a047750 con 0x55a999cbfc90 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab4d600640 10 client.25430 handle_command_reply: tid=0 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab6d652740 1 client.25430 shutdown 2024-09-21T06:53:42.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d030a0 msgr2=0x55a99a0d6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.112+0000 7fab6d652740 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d030a0 0x55a99a0d6950 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55a99a0f1570 tx=0x7fab50017000 comp rx=0 tx=0).stop 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 shutdown_connections 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a999cbfc90 0x55a99a15f4d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a999d2bf60 0x55a99a0e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a999d0b280 0x55a99a0d6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 --2- 172.21.15.114:0/3856217173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a999d030a0 0x55a99a0d6950 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 >> 172.21.15.114:0/3856217173 conn(0x55a999d20350 msgr2=0x55a99a0ee1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab4d600640 0 client.25430 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 shutdown_connections 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 1 -- 172.21.15.114:0/3856217173 wait complete. 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab6d652740 20 client.25430 trim_cache size 0 max 16384 2024-09-21T06:53:42.121 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24761, 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 172.896186008, 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 28, 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 87, 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 186.51290745399999 2024-09-21T06:53:42.122 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:53:42.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fab440b6560 msgr2=0x7fab440b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab65400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fab440b6560 0x7fab440b8a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55a99a0d12f0 tx=0x7fab50009500 comp rx=0 tx=0).stop 2024-09-21T06:53:42.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 msgr2=0x7fab6019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:42.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.113+0000 7fab65400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 0x7fab6019ca60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fab6006c8a0 tx=0x7fab4800ee50 comp rx=0 tx=0).stop 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 shutdown_connections 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab6010be50 0x7fab601a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fab440b6560 0x7fab440b8a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab6010b500 0x7fab6019ca60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 --2- 172.21.15.114:0/2626606347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fab60105bc0 0x7fab6019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 >> 172.21.15.114:0/2626606347 conn(0x7fab60101670 msgr2=0x7fab60108ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 shutdown_connections 2024-09-21T06:53:42.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:42.114+0000 7fab65400640 1 -- 172.21.15.114:0/2626606347 wait complete. 2024-09-21T06:53:42.715 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:42 smithi114 ceph-mon[30654]: pgmap v383: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:42.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:42 smithi183 ceph-mon[34219]: pgmap v383: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:42 smithi193 ceph-mon[34601]: pgmap v383: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:43.254 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T06:53:43.267 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T06:53:43.267 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T06:53:43.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:43.267 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-21T06:53:43.303 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T06:53:43.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:43.304 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T06:53:43.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:43.402 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-09-21T06:53:43.734 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-21T06:53:43.734 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T06:53:43.735 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-21T06:53:43.735 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-21T06:53:43.735 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T06:53:44.104 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:44.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 -- 172.21.15.114:0/1572956546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810b6c0 msgr2=0x7ff84810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:44.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 --2- 172.21.15.114:0/1572956546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810b6c0 0x7ff84810bad0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff834009900 tx=0x7ff83402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:44.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 -- 172.21.15.114:0/1572956546 shutdown_connections 2024-09-21T06:53:44.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 --2- 172.21.15.114:0/1572956546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff84810c010 0x7ff848112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:44.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 --2- 172.21.15.114:0/1572956546 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810b6c0 0x7ff84810bad0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:44.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 --2- 172.21.15.114:0/1572956546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff848106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:44.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 -- 172.21.15.114:0/1572956546 >> 172.21.15.114:0/1572956546 conn(0x7ff848101810 msgr2=0x7ff848103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:44.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 -- 172.21.15.114:0/1572956546 shutdown_connections 2024-09-21T06:53:44.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.759+0000 7ff84e800640 1 -- 172.21.15.114:0/1572956546 wait complete. 2024-09-21T06:53:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.760+0000 7ff84e800640 1 Processor -- start 2024-09-21T06:53:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.760+0000 7ff84e800640 1 -- start start 2024-09-21T06:53:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff84e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff84819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff84e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff84810b6c0 0x7ff84819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff84e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810c010 0x7ff8481a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:44.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff84e800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff848115d20 con 0x7ff84810c010 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff84e800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff848115ba0 con 0x7ff848105d80 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff84e800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff848115ea0 con 0x7ff84810b6c0 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff847e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810c010 0x7ff8481a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff847400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff84819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff847e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810c010 0x7ff8481a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44274/0 (socket says 172.21.15.114:44274) 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff847400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff84819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43466/0 (socket says 172.21.15.114:43466) 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff846a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff84810b6c0 0x7ff84819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.761+0000 7ff847e00640 1 -- 172.21.15.114:0/917959197 learned_addr learned my addr 172.21.15.114:0/917959197 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff847400640 1 -- 172.21.15.114:0/917959197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff84810b6c0 msgr2=0x7ff84819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff847400640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff84810b6c0 0x7ff84819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff847400640 1 -- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810c010 msgr2=0x7ff8481a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff847400640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810c010 0x7ff8481a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:44.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff847400640 1 -- 172.21.15.114:0/917959197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481a2660 con 0x7ff848105d80 2024-09-21T06:53:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff847400640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff84819c6f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff83800e9f0 tx=0x7ff83800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83800ccc0 con 0x7ff848105d80 2024-09-21T06:53:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8380108e0 con 0x7ff848105d80 2024-09-21T06:53:44.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838018880 con 0x7ff848105d80 2024-09-21T06:53:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481a2950 con 0x7ff848105d80 2024-09-21T06:53:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.762+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a90b0 con 0x7ff848105d80 2024-09-21T06:53:44.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.764+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff848106190 con 0x7ff848105d80 2024-09-21T06:53:44.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.766+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7ff838010430 con 0x7ff848105d80 2024-09-21T06:53:44.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.767+0000 7ff83fe00640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff82c0b65e0 0x7ff82c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:44.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.768+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8380e7f60 con 0x7ff848105d80 2024-09-21T06:53:44.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.768+0000 7ff846a00640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff82c0b65e0 0x7ff82c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:44.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.768+0000 7ff846a00640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff82c0b65e0 0x7ff82c0b8af0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff83402aa00 tx=0x7ff834005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:44.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.769+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8380b2740 con 0x7ff848105d80 2024-09-21T06:53:44.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:44 smithi114 ceph-mon[30654]: pgmap v384: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:44.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:44 smithi183 ceph-mon[34219]: pgmap v384: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:44.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:44 smithi193 ceph-mon[34601]: pgmap v384: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:53:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.997+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff84802e640 con 0x7ff848105d80 2024-09-21T06:53:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:44.998+0000 7ff83fe00640 1 -- 172.21.15.114:0/917959197 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7ff8380b7680 con 0x7ff848105d80 2024-09-21T06:53:45.001 INFO:teuthology.orchestra.run.smithi114.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T06:53:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.000+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff82c0b65e0 msgr2=0x7ff82c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.000+0000 7ff84e800640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff82c0b65e0 0x7ff82c0b8af0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff83402aa00 tx=0x7ff834005a20 comp rx=0 tx=0).stop 2024-09-21T06:53:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.000+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 msgr2=0x7ff84819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.000+0000 7ff84e800640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff84819c6f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff83800e9f0 tx=0x7ff83800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 shutdown_connections 2024-09-21T06:53:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff84810c010 0x7ff8481a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff84810b6c0 0x7ff84819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff82c0b65e0 0x7ff82c0b8af0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 --2- 172.21.15.114:0/917959197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff848105d80 0x7ff84819c6f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 >> 172.21.15.114:0/917959197 conn(0x7ff848101810 msgr2=0x7ff848103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 shutdown_connections 2024-09-21T06:53:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45.001+0000 7ff84e800640 1 -- 172.21.15.114:0/917959197 wait complete. 2024-09-21T06:53:45.799 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:45 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/917959197' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi114.front.sepia.ceph.com 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T06:53:45.841 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-21T06:53:45.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:45.841 DEBUG:teuthology.orchestra.run.smithi114:> ip addr 2024-09-21T06:53:45.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:45 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/917959197' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:53:45.878 INFO:teuthology.orchestra.run.smithi114.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T06:53:45.878 INFO:teuthology.orchestra.run.smithi114.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 ::1/128 scope host 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:92 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f0 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:d8:a8:93 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: altname enp7s0f1 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:62 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T06:53:45.879 INFO:teuthology.orchestra.run.smithi114.stdout: link/ether 0c:c4:7a:8f:cb:63 brd ff:ff:ff:ff:ff:ff 2024-09-21T06:53:45.880 INFO:teuthology.orchestra.run.smithi114.stdout: inet 172.21.15.114/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T06:53:45.880 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft 41996sec preferred_lft 41996sec 2024-09-21T06:53:45.880 INFO:teuthology.orchestra.run.smithi114.stdout: inet6 fe80::ec4:7aff:fe8f:cb63/64 scope link noprefixroute 2024-09-21T06:53:45.880 INFO:teuthology.orchestra.run.smithi114.stdout: valid_lft forever preferred_lft forever 2024-09-21T06:53:45.880 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T06:53:45.880 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:53:45.880 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:53:45.880 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add name ceph-brx type bridge 2024-09-21T06:53:45.880 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr flush dev ceph-brx 2024-09-21T06:53:45.881 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set ceph-brx up 2024-09-21T06:53:45.881 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T06:53:45.881 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:53:45.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:45 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/917959197' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T06:53:45.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:53:46.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:46.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:46.107 DEBUG:teuthology.orchestra.run.smithi114:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T06:53:46.160 INFO:teuthology.orchestra.run.smithi114.stdout:1 2024-09-21T06:53:46.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:46.164 DEBUG:teuthology.orchestra.run.smithi114:> route 2024-09-21T06:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout:Kernel IP routing table 2024-09-21T06:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T06:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T06:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T06:53:46.200 INFO:teuthology.orchestra.run.smithi114.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T06:53:46.200 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:53:46.200 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:53:46.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T06:53:46.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T06:53:46.200 DEBUG:teuthology.orchestra.run.smithi114:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T06:53:46.201 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:53:46.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:53:46.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:46.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:46.399 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-21T06:53:46.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:46.435 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list-id 2024-09-21T06:53:46.511 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:53:46.511 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:53:46.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-21T06:53:46.511 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-21T06:53:46.511 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:53:46.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:53:46.647 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:46 smithi114 ceph-mon[30654]: pgmap v385: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:46.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:46.690 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-21T06:53:46.690 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:53:46.690 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:53:46.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-21T06:53:46.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T06:53:46.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-21T06:53:46.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-21T06:53:46.691 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-21T06:53:46.691 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:53:46.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:53:46.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:46 smithi183 ceph-mon[34219]: pgmap v385: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:46.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:46.910 DEBUG:teuthology.orchestra.run.smithi114:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T06:53:46.910 DEBUG:teuthology.orchestra.run.smithi114:> set -e 2024-09-21T06:53:46.910 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set brx.0 up 2024-09-21T06:53:46.910 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T06:53:46.910 DEBUG:teuthology.orchestra.run.smithi114:> ') 2024-09-21T06:53:46.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:46 smithi193 ceph-mon[34601]: pgmap v385: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:53:46.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:53:47.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:47.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:53:47.076 DEBUG:teuthology.orchestra.run.smithi114:> mkdir -p -v /tmp/cephfs 2024-09-21T06:53:47.112 INFO:teuthology.orchestra.run.smithi114.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-21T06:53:47.112 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T06:53:47.112 DEBUG:teuthology.orchestra.run.smithi114:> chmod 0000 /tmp/cephfs 2024-09-21T06:53:47.187 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T06:53:47.188 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T06:53:47.188 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644 2024-09-21T06:53:47.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:47.188 DEBUG:teuthology.orchestra.run.smithi114:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-21T06:53:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-21T06:53:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T06:53:47.382 INFO:teuthology.orchestra.run.smithi114.stdout:invalid new device string format 2024-09-21T06:53:48.012 INFO:teuthology.orchestra.run.smithi114.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-21T06:53:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-21T06:53:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:invalid new device string format 2024-09-21T06:53:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: resolved to: "172.21.15.114:6789,172.21.15.183:6789,172.21.15.193:6789" 2024-09-21T06:53:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.114:6789,172.21.15.183:6789,172.21.15.193:6789:/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644 2024-09-21T06:53:48.013 INFO:teuthology.orchestra.run.smithi114.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=5d75913a-77e4-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T06:53:48.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T06:53:48.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:53:48.018 DEBUG:teuthology.orchestra.run.smithi114:> sudo chmod 1777 /tmp/cephfs 2024-09-21T06:53:48.079 DEBUG:teuthology.orchestra.run.smithi114:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:53:48.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:53:48.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:48.172 INFO:teuthology.orchestra.run.smithi114.stdout:client25444 2024-09-21T06:53:48.172 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T06:53:48.172 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:53:48.210 DEBUG:teuthology.orchestra.run.smithi114:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:53:48.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:53:48.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:48.340 INFO:teuthology.orchestra.run.smithi114.stdout:client25444 2024-09-21T06:53:48.340 DEBUG:teuthology.orchestra.run.smithi114:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T06:53:48.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T06:53:48.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:48.430 INFO:teuthology.orchestra.run.smithi114.stdout:client25444 2024-09-21T06:53:48.430 DEBUG:teuthology.orchestra.run.smithi114:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5d75913a-77e4-11ef-baf5-efdc52797490.client25444/status') 2024-09-21T06:53:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T06:53:48.529 INFO:teuthology.orchestra.run.smithi114.stdout:instance: client.25444 (3)172.21.15.114:0/965143239 2024-09-21T06:53:48.529 INFO:teuthology.orchestra.run.smithi114.stdout:blocklisted: false 2024-09-21T06:53:48.529 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records in 2024-09-21T06:53:48.530 INFO:teuthology.orchestra.run.smithi114.stderr:0+1 records out 2024-09-21T06:53:48.530 INFO:teuthology.orchestra.run.smithi114.stderr:71 bytes copied, 6.0056e-05 s, 1.2 MB/s 2024-09-21T06:53:48.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T06:53:48.540 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:53:48.550 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:53:48.550 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:53:48.550 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T06:53:48.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 -- 172.21.15.114:0/1640624814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca8078290 msgr2=0x7faca8114160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:48.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 --2- 172.21.15.114:0/1640624814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca8078290 0x7faca8114160 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fac98009900 tx=0x7fac9802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:48.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 -- 172.21.15.114:0/1640624814 shutdown_connections 2024-09-21T06:53:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 --2- 172.21.15.114:0/1640624814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faca81146a0 0x7faca8116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 --2- 172.21.15.114:0/1640624814 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca8078290 0x7faca8114160 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 --2- 172.21.15.114:0/1640624814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faca8077940 0x7faca8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 -- 172.21.15.114:0/1640624814 >> 172.21.15.114:0/1640624814 conn(0x7faca81016d0 msgr2=0x7faca8103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 -- 172.21.15.114:0/1640624814 shutdown_connections 2024-09-21T06:53:48.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.767+0000 7facb0800640 1 -- 172.21.15.114:0/1640624814 wait complete. 2024-09-21T06:53:48.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.768+0000 7facb0800640 1 Processor -- start 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.768+0000 7facb0800640 1 -- start start 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facb0800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faca8077940 0x7faca81a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facb0800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faca8078290 0x7faca81a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facb0800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 0x7faca81a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facb0800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faca8119fe0 con 0x7faca81146a0 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facb0800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faca8119e60 con 0x7faca8078290 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facb0800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faca811a160 con 0x7faca8077940 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facae800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 0x7faca81a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facad400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faca8078290 0x7faca81a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facae800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 0x7faca81a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51156/0 (socket says 172.21.15.114:51156) 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faca8077940 0x7faca81a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:48.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facae800640 1 -- 172.21.15.114:0/3447059040 learned_addr learned my addr 172.21.15.114:0/3447059040 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facae800640 1 -- 172.21.15.114:0/3447059040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faca8077940 msgr2=0x7faca81a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.769+0000 7facae800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faca8077940 0x7faca81a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7facae800640 1 -- 172.21.15.114:0/3447059040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faca8078290 msgr2=0x7faca81a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7facae800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faca8078290 0x7faca81a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7facae800640 1 -- 172.21.15.114:0/3447059040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81a6990 con 0x7faca81146a0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7facae800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 0x7faca81a6290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faca400eb10 tx=0x7faca400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4004b20 con 0x7faca81146a0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81a6c80 con 0x7faca81146a0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faca4004cc0 con 0x7faca81146a0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4010670 con 0x7faca81146a0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.770+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81ad410 con 0x7faca81146a0 2024-09-21T06:53:48.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.771+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca8077d50 con 0x7faca81146a0 2024-09-21T06:53:48.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.774+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7faca4005000 con 0x7faca81146a0 2024-09-21T06:53:48.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.775+0000 7faca2a00640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fac900b6560 0x7fac900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:48.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.775+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7faca40e7ff0 con 0x7faca81146a0 2024-09-21T06:53:48.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.776+0000 7facade00640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fac900b6560 0x7fac900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:48.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.776+0000 7facade00640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fac900b6560 0x7fac900b8a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009bd0 tx=0x7fac9c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:48.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:48.778+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faca4056710 con 0x7faca81146a0 2024-09-21T06:53:48.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:48 smithi114 ceph-mon[30654]: pgmap v386: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:48 smithi183 ceph-mon[34219]: pgmap v386: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:48 smithi193 ceph-mon[34601]: pgmap v386: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T06:53:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.004+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7faca810f830 con 0x7faca81146a0 2024-09-21T06:53:49.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.580+0000 7faca2a00640 1 -- 172.21.15.114:0/3447059040 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7faca40ae3b0 con 0x7faca81146a0 2024-09-21T06:53:49.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fac900b6560 msgr2=0x7fac900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fac900b6560 0x7fac900b8a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009bd0 tx=0x7fac9c005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 msgr2=0x7faca81a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 0x7faca81a6290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faca400eb10 tx=0x7faca400c2d0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 shutdown_connections 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faca81146a0 0x7faca81a6290 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fac900b6560 0x7fac900b8a70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faca8078290 0x7faca81a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 --2- 172.21.15.114:0/3447059040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faca8077940 0x7faca81a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 >> 172.21.15.114:0/3447059040 conn(0x7faca81016d0 msgr2=0x7faca8103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:49.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 shutdown_connections 2024-09-21T06:53:49.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.583+0000 7facb0800640 1 -- 172.21.15.114:0/3447059040 wait complete. 2024-09-21T06:53:49.609 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T06:53:49.654 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T06:53:49.748 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:53:49.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3447059040' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T06:53:49.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3447059040' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T06:53:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 -- 172.21.15.114:0/2037145105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c105cd0 msgr2=0x7f642c1060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 --2- 172.21.15.114:0/2037145105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c105cd0 0x7f642c1060e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6420009900 tx=0x7f642002a400 comp rx=0 tx=0).stop 2024-09-21T06:53:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 -- 172.21.15.114:0/2037145105 shutdown_connections 2024-09-21T06:53:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 --2- 172.21.15.114:0/2037145105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c10bf50 0x7f642c112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 --2- 172.21.15.114:0/2037145105 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10b600 0x7f642c10ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 --2- 172.21.15.114:0/2037145105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c105cd0 0x7f642c1060e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 -- 172.21.15.114:0/2037145105 >> 172.21.15.114:0/2037145105 conn(0x7f642c1016f0 msgr2=0x7f642c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 -- 172.21.15.114:0/2037145105 shutdown_connections 2024-09-21T06:53:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.966+0000 7f6434400640 1 -- 172.21.15.114:0/2037145105 wait complete. 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 Processor -- start 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 -- start start 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c105cd0 0x7f642c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c10b600 0x7f642c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 0x7f642c1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f642c115c60 con 0x7f642c10bf50 2024-09-21T06:53:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f642c115ae0 con 0x7f642c10b600 2024-09-21T06:53:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.967+0000 7f6434400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f642c115de0 con 0x7f642c105cd0 2024-09-21T06:53:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 0x7f642c1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 0x7f642c1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51182/0 (socket says 172.21.15.114:51182) 2024-09-21T06:53:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 -- 172.21.15.114:0/2877267474 learned_addr learned my addr 172.21.15.114:0/2877267474 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6431a00640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c105cd0 0x7f642c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6431000640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c10b600 0x7f642c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 -- 172.21.15.114:0/2877267474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c105cd0 msgr2=0x7f642c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c105cd0 0x7f642c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 -- 172.21.15.114:0/2877267474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c10b600 msgr2=0x7f642c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c10b600 0x7f642c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6432400640 1 -- 172.21.15.114:0/2877267474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c1a2550 con 0x7f642c10bf50 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.968+0000 7f6431a00640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c105cd0 0x7f642c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.969+0000 7f6432400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 0x7f642c1a1e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f642400efd0 tx=0x7f642400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.969+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424003b70 con 0x7f642c10bf50 2024-09-21T06:53:49.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.969+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6424003d10 con 0x7f642c10bf50 2024-09-21T06:53:49.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.969+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642c1a2840 con 0x7f642c10bf50 2024-09-21T06:53:49.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.969+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64240108a0 con 0x7f642c10bf50 2024-09-21T06:53:49.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.969+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f642c1a8fd0 con 0x7f642c10bf50 2024-09-21T06:53:49.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.970+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f642c106f40 con 0x7f642c10bf50 2024-09-21T06:53:49.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.972+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f6424004b40 con 0x7f642c10bf50 2024-09-21T06:53:49.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.974+0000 7f642a000640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64140b6560 0x7f64140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:49.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.974+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f64240e8080 con 0x7f642c10bf50 2024-09-21T06:53:49.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.974+0000 7f6431a00640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64140b6560 0x7f64140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:49.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.975+0000 7f6431a00640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64140b6560 0x7f64140b8a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6420005ab0 tx=0x7f6420005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:49.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:49.976+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f64240ae420 con 0x7f642c10bf50 2024-09-21T06:53:50.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3447059040' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T06:53:50.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.163+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f642c19d740 con 0x7f642c10bf50 2024-09-21T06:53:50.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.165+0000 7f642a000640 1 -- 172.21.15.114:0/2877267474 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25277 (secure 0 0 0) 0x7f64240b2800 con 0x7f642c10bf50 2024-09-21T06:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:50.167 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:52:31.637070+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.167+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64140b6560 msgr2=0x7f64140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.167+0000 7f6434400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64140b6560 0x7f64140b8a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6420005ab0 tx=0x7f6420005a20 comp rx=0 tx=0).stop 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.167+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 msgr2=0x7f642c1a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.167+0000 7f6434400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 0x7f642c1a1e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f642400efd0 tx=0x7f642400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 shutdown_connections 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f642c10bf50 0x7f642c1a1e50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f64140b6560 0x7f64140b8a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f642c10b600 0x7f642c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 --2- 172.21.15.114:0/2877267474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f642c105cd0 0x7f642c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 >> 172.21.15.114:0/2877267474 conn(0x7f642c1016f0 msgr2=0x7f642c103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:50.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 shutdown_connections 2024-09-21T06:53:50.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.168+0000 7f6434400640 1 -- 172.21.15.114:0/2877267474 wait complete. 2024-09-21T06:53:50.195 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:45:37.711233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:50:42.695843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:50:45.269171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T06:50:54.559397+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T06:53:50.195 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:53:50.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 -- 172.21.15.114:0/2002121644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd684109d10 msgr2=0x7fd684110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 --2- 172.21.15.114:0/2002121644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd684109d10 0x7fd684110540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009be0 tx=0x7fd67c02a6c0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 -- 172.21.15.114:0/2002121644 shutdown_connections 2024-09-21T06:53:50.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 --2- 172.21.15.114:0/2002121644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd684109d10 0x7fd684110540 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 --2- 172.21.15.114:0/2002121644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6841093c0 0x7fd6841097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 --2- 172.21.15.114:0/2002121644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 0x7fd684103ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 -- 172.21.15.114:0/2002121644 >> 172.21.15.114:0/2002121644 conn(0x7fd6840ff4b0 msgr2=0x7fd684101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:50.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 -- 172.21.15.114:0/2002121644 shutdown_connections 2024-09-21T06:53:50.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.414+0000 7fd68b400640 1 -- 172.21.15.114:0/2002121644 wait complete. 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.415+0000 7fd68b400640 1 Processor -- start 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.415+0000 7fd68b400640 1 -- start start 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd68b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 0x7fd68419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd68b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6841093c0 0x7fd68419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd68b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 0x7fd6841a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd68b400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6841139d0 con 0x7fd684109d10 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd68b400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd684113850 con 0x7fd684103a90 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd68b400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd684113b50 con 0x7fd6841093c0 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd689400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 0x7fd6841a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd689400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 0x7fd6841a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51210/0 (socket says 172.21.15.114:51210) 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd689400640 1 -- 172.21.15.114:0/1286412204 learned_addr learned my addr 172.21.15.114:0/1286412204 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd67be00640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6841093c0 0x7fd68419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.416+0000 7fd688a00640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 0x7fd68419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd689400640 1 -- 172.21.15.114:0/1286412204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6841093c0 msgr2=0x7fd68419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd689400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6841093c0 0x7fd68419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd689400640 1 -- 172.21.15.114:0/1286412204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 msgr2=0x7fd68419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd689400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 0x7fd68419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd689400640 1 -- 172.21.15.114:0/1286412204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6841a24c0 con 0x7fd684109d10 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd688a00640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 0x7fd68419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd689400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 0x7fd6841a1dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd67c004af0 tx=0x7fd67c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c041020 con 0x7fd684109d10 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c003e10 con 0x7fd684109d10 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6841a2750 con 0x7fd684109d10 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c036040 con 0x7fd684109d10 2024-09-21T06:53:50.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.417+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841a8f40 con 0x7fd684109d10 2024-09-21T06:53:50.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.418+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd684104d00 con 0x7fd684109d10 2024-09-21T06:53:50.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.421+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fd67c044020 con 0x7fd684109d10 2024-09-21T06:53:50.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.422+0000 7fd682000640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6680b6560 0x7fd6680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.422+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd67c1087a0 con 0x7fd684109d10 2024-09-21T06:53:50.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.422+0000 7fd688a00640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6680b6560 0x7fd6680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.423+0000 7fd688a00640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6680b6560 0x7fd6680b8a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd674009fc0 tx=0x7fd674005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:50.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.424+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd67c04b020 con 0x7fd684109d10 2024-09-21T06:53:50.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.615+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6841a9110 con 0x7fd684109d10 2024-09-21T06:53:50.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.615+0000 7fd682000640 1 -- 172.21.15.114:0/1286412204 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fd67c030970 con 0x7fd684109d10 2024-09-21T06:53:50.618 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-21T06:53:50.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6680b6560 msgr2=0x7fd6680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6680b6560 0x7fd6680b8a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd674009fc0 tx=0x7fd674005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 msgr2=0x7fd6841a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 0x7fd6841a1dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd67c004af0 tx=0x7fd67c003c20 comp rx=0 tx=0).stop 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 shutdown_connections 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd684109d10 0x7fd6841a1dc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6841093c0 0x7fd68419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6680b6560 0x7fd6680b8a70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 --2- 172.21.15.114:0/1286412204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd684103a90 0x7fd68419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 >> 172.21.15.114:0/1286412204 conn(0x7fd6840ff4b0 msgr2=0x7fd684106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:50.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 shutdown_connections 2024-09-21T06:53:50.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.618+0000 7fd68b400640 1 -- 172.21.15.114:0/1286412204 wait complete. 2024-09-21T06:53:50.644 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:53:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:50 smithi183 ceph-mon[34219]: pgmap v387: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T06:53:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:50 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3447059040' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T06:53:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:50 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:53:50.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:50 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2877267474' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 -- 172.21.15.114:0/2217312498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410b600 msgr2=0x7fdff410ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 --2- 172.21.15.114:0/2217312498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410b600 0x7fdff410ba10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009900 tx=0x7fdfdc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 -- 172.21.15.114:0/2217312498 shutdown_connections 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 --2- 172.21.15.114:0/2217312498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410bf50 0x7fdff4112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 --2- 172.21.15.114:0/2217312498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410b600 0x7fdff410ba10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 --2- 172.21.15.114:0/2217312498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff41060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 -- 172.21.15.114:0/2217312498 >> 172.21.15.114:0/2217312498 conn(0x7fdff4101710 msgr2=0x7fdff4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 -- 172.21.15.114:0/2217312498 shutdown_connections 2024-09-21T06:53:50.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.864+0000 7fdff9600640 1 -- 172.21.15.114:0/2217312498 wait complete. 2024-09-21T06:53:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.865+0000 7fdff9600640 1 Processor -- start 2024-09-21T06:53:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.865+0000 7fdff9600640 1 -- start start 2024-09-21T06:53:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff9600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410b600 0x7fdff419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410bf50 0x7fdff41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff9600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4115c10 con 0x7fdff410b600 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff9600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4115a90 con 0x7fdff410bf50 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff9600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4115d90 con 0x7fdff4105cd0 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdfea000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410b600 0x7fdff419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58636/0 (socket says 172.21.15.114:58636) 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 -- 172.21.15.114:0/4133330574 learned_addr learned my addr 172.21.15.114:0/4133330574 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:50.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff3400640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410bf50 0x7fdff41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 -- 172.21.15.114:0/4133330574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410bf50 msgr2=0x7fdff41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410bf50 0x7fdff41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 -- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410b600 msgr2=0x7fdff419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:50.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410b600 0x7fdff419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:50.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdff2a00640 1 -- 172.21.15.114:0/4133330574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff41a2580 con 0x7fdff4105cd0 2024-09-21T06:53:50.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.866+0000 7fdfea000640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410b600 0x7fdff419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:50.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.867+0000 7fdff2a00640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff419c650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e9f0 tx=0x7fdfe400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:50.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.867+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe400ccc0 con 0x7fdff4105cd0 2024-09-21T06:53:50.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.867+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41a2870 con 0x7fdff4105cd0 2024-09-21T06:53:50.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.867+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe400ce60 con 0x7fdff4105cd0 2024-09-21T06:53:50.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.867+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4010430 con 0x7fdff4105cd0 2024-09-21T06:53:50.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.867+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41a9000 con 0x7fdff4105cd0 2024-09-21T06:53:50.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.868+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff41060e0 con 0x7fdff4105cd0 2024-09-21T06:53:50.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.871+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fdfe40046c0 con 0x7fdff4105cd0 2024-09-21T06:53:50.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.872+0000 7fdfebe00640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd80b6560 0x7fdfd80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:50.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.873+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfe40e7e50 con 0x7fdff4105cd0 2024-09-21T06:53:50.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.873+0000 7fdfea000640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd80b6560 0x7fdfd80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:50.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.873+0000 7fdfea000640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd80b6560 0x7fdfd80b8a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009230 tx=0x7fdfdc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:50.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:50.874+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdfe4056710 con 0x7fdff4105cd0 2024-09-21T06:53:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:50 smithi193 ceph-mon[34601]: pgmap v387: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T06:53:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:50 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3447059040' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T06:53:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:50 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:53:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:50 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2877267474' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:51.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.059+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdff402e640 con 0x7fdff4105cd0 2024-09-21T06:53:51.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.059+0000 7fdfebe00640 1 -- 172.21.15.114:0/4133330574 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fdfe40ae1f0 con 0x7fdff4105cd0 2024-09-21T06:53:51.061 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd80b6560 msgr2=0x7fdfd80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd80b6560 0x7fdfd80b8a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009230 tx=0x7fdfdc031400 comp rx=0 tx=0).stop 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 msgr2=0x7fdff419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff419c650 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400e9f0 tx=0x7fdfe400edf0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 shutdown_connections 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd80b6560 0x7fdfd80b8a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff410bf50 0x7fdff41a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdff410b600 0x7fdff419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 --2- 172.21.15.114:0/4133330574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdff4105cd0 0x7fdff419c650 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 >> 172.21.15.114:0/4133330574 conn(0x7fdff4101710 msgr2=0x7fdff4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 shutdown_connections 2024-09-21T06:53:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.062+0000 7fdff9600640 1 -- 172.21.15.114:0/4133330574 wait complete. 2024-09-21T06:53:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:50 smithi114 ceph-mon[30654]: pgmap v387: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-21T06:53:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:50 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3447059040' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T06:53:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:50 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:53:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:50 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2877267474' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:51.086 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:53:51.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 -- 172.21.15.114:0/3181858945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3704103ab0 msgr2=0x7f3704103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 --2- 172.21.15.114:0/3181858945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3704103ab0 0x7f3704103ec0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009860 tx=0x7f36f402a340 comp rx=0 tx=0).stop 2024-09-21T06:53:51.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 -- 172.21.15.114:0/3181858945 shutdown_connections 2024-09-21T06:53:51.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 --2- 172.21.15.114:0/3181858945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704109d30 0x7f3704110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 --2- 172.21.15.114:0/3181858945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37041093e0 0x7f37041097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 --2- 172.21.15.114:0/3181858945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3704103ab0 0x7f3704103ec0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 -- 172.21.15.114:0/3181858945 >> 172.21.15.114:0/3181858945 conn(0x7f37040ff4d0 msgr2=0x7f3704101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 -- 172.21.15.114:0/3181858945 shutdown_connections 2024-09-21T06:53:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.310+0000 7f370ac00640 1 -- 172.21.15.114:0/3181858945 wait complete. 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.311+0000 7f370ac00640 1 Processor -- start 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 -- start start 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 0x7f370419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37041093e0 0x7f370419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704109d30 0x7f37041a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37041139f0 con 0x7f3704109d30 2024-09-21T06:53:51.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3704113870 con 0x7f37041093e0 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f370ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3704113b70 con 0x7f3704103ab0 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f3708c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704109d30 0x7f37041a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f3703e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 0x7f370419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f3703e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 0x7f370419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58656/0 (socket says 172.21.15.114:58656) 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f3703e00640 1 -- 172.21.15.114:0/9051576 learned_addr learned my addr 172.21.15.114:0/9051576 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.312+0000 7f3703400640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37041093e0 0x7f370419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703e00640 1 -- 172.21.15.114:0/9051576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37041093e0 msgr2=0x7f370419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703e00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37041093e0 0x7f370419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703e00640 1 -- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704109d30 msgr2=0x7f37041a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703e00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704109d30 0x7f37041a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703e00640 1 -- 172.21.15.114:0/9051576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37041a2620 con 0x7f3704103ab0 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703400640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37041093e0 0x7f370419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3708c00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704109d30 0x7f37041a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3703e00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 0x7f370419c6f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36f40303f0 tx=0x7f36f4032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4040020 con 0x7f3704103ab0 2024-09-21T06:53:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37041a28b0 con 0x7f3704103ab0 2024-09-21T06:53:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f4030dd0 con 0x7f3704103ab0 2024-09-21T06:53:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.313+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4037040 con 0x7f3704103ab0 2024-09-21T06:53:51.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.314+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37041a90a0 con 0x7f3704103ab0 2024-09-21T06:53:51.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.315+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3704103ec0 con 0x7f3704103ab0 2024-09-21T06:53:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.317+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f36f4043020 con 0x7f3704103ab0 2024-09-21T06:53:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.319+0000 7f3700c00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e80b6560 0x7f36e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.319+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f36f4107b70 con 0x7f3704103ab0 2024-09-21T06:53:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.319+0000 7f3703400640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e80b6560 0x7f36e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.320+0000 7f3703400640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e80b6560 0x7f36e80b8a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36f00096c0 tx=0x7f36f0009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:51.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.320+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36f404a020 con 0x7f3704103ab0 2024-09-21T06:53:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.517+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37041a9300 con 0x7f3704103ab0 2024-09-21T06:53:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.518+0000 7f3700c00640 1 -- 172.21.15.114:0/9051576 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f36f40328c0 con 0x7f3704103ab0 2024-09-21T06:53:51.519 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-21T06:53:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e80b6560 msgr2=0x7f36e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e80b6560 0x7f36e80b8a70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36f00096c0 tx=0x7f36f0009260 comp rx=0 tx=0).stop 2024-09-21T06:53:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 msgr2=0x7f370419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 0x7f370419c6f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36f40303f0 tx=0x7f36f4032da0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 shutdown_connections 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3704109d30 0x7f37041a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e80b6560 0x7f36e80b8a70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37041093e0 0x7f370419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 --2- 172.21.15.114:0/9051576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3704103ab0 0x7f370419c6f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.520+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 >> 172.21.15.114:0/9051576 conn(0x7f37040ff4d0 msgr2=0x7f3704108ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.521+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 shutdown_connections 2024-09-21T06:53:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.521+0000 7f370ac00640 1 -- 172.21.15.114:0/9051576 wait complete. 2024-09-21T06:53:51.545 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 -- 172.21.15.114:0/3592771150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 msgr2=0x7f1dac10fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 --2- 172.21.15.114:0/3592771150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 0x7f1dac10fa80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009900 tx=0x7f1d9402a480 comp rx=0 tx=0).stop 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 -- 172.21.15.114:0/3592771150 shutdown_connections 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 --2- 172.21.15.114:0/3592771150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 0x7f1dac1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 --2- 172.21.15.114:0/3592771150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 0x7f1dac10fa80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 --2- 172.21.15.114:0/3592771150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 -- 172.21.15.114:0/3592771150 >> 172.21.15.114:0/3592771150 conn(0x7f1dac07a4e0 msgr2=0x7f1dac07a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 -- 172.21.15.114:0/3592771150 shutdown_connections 2024-09-21T06:53:51.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.789+0000 7f1dabe00640 1 -- 172.21.15.114:0/3592771150 wait complete. 2024-09-21T06:53:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.790+0000 7f1dabe00640 1 Processor -- start 2024-09-21T06:53:51.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.790+0000 7f1dabe00640 1 -- start start 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 0x7f1dac19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dabe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 0x7f1dac1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dabe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac115a00 con 0x7f1dac102d60 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dabe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac115880 con 0x7f1dac10ffc0 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dabe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac115b80 con 0x7f1dac102410 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dab400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 0x7f1dac1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58676/0 (socket says 172.21.15.114:58676) 2024-09-21T06:53:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 -- 172.21.15.114:0/2366428389 learned_addr learned my addr 172.21.15.114:0/2366428389 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:51.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1da3e00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 0x7f1dac19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 -- 172.21.15.114:0/2366428389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 msgr2=0x7f1dac1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 0x7f1dac1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 -- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 msgr2=0x7f1dac19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 0x7f1dac19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1daaa00640 1 -- 172.21.15.114:0/2366428389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac1a2350 con 0x7f1dac102410 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.791+0000 7f1dab400640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 0x7f1dac1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.792+0000 7f1daaa00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac19c420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00e9f0 tx=0x7f1d9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.792+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00cd80 con 0x7f1dac102410 2024-09-21T06:53:51.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.792+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1a2640 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.792+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9c0108e0 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.792+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c018980 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.792+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a8dd0 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.793+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac102820 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.795+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f1d9c010430 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.797+0000 7f1da3400640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d900b6560 0x7f1d900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.797+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f1d9c0e7b90 con 0x7f1dac102410 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.797+0000 7f1da3e00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d900b6560 0x7f1d900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:51.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.798+0000 7f1da3e00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d900b6560 0x7f1d900b8a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d94031440 tx=0x7f1d94009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:51.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:51.801+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d9c0b2310 con 0x7f1dac102410 2024-09-21T06:53:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1286412204' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4133330574' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:51 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:53:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/9051576' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1286412204' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4133330574' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:51 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:53:51.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/9051576' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1286412204' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4133330574' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:53:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:51 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:53:51.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/9051576' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:53:52.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.005+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1dac02e640 con 0x7f1dac102410 2024-09-21T06:53:52.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.005+0000 7f1da3400640 1 -- 172.21.15.114:0/2366428389 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f1d9c0b7250 con 0x7f1dac102410 2024-09-21T06:53:52.007 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d900b6560 msgr2=0x7f1d900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d900b6560 0x7f1d900b8a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d94031440 tx=0x7f1d94009560 comp rx=0 tx=0).stop 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 msgr2=0x7f1dac19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac19c420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00e9f0 tx=0x7f1d9c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 shutdown_connections 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d900b6560 0x7f1d900b8a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dac10ffc0 0x7f1dac1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1dac102d60 0x7f1dac19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:52.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 --2- 172.21.15.114:0/2366428389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1dac102410 0x7f1dac19c420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:52.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 >> 172.21.15.114:0/2366428389 conn(0x7f1dac07a4e0 msgr2=0x7f1dac111400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:52.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 shutdown_connections 2024-09-21T06:53:52.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:52.008+0000 7f1dabe00640 1 -- 172.21.15.114:0/2366428389 wait complete. 2024-09-21T06:53:52.037 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T06:53:52.037 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:53:52.414 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: pgmap v388: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2366428389' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:53:52.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: pgmap v388: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2366428389' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:53:52.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:53.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 -- 172.21.15.114:0/3265256304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 msgr2=0x7f33a8112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 --2- 172.21.15.114:0/3265256304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a8112680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009ca0 tx=0x7f33a002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 -- 172.21.15.114:0/3265256304 shutdown_connections 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 --2- 172.21.15.114:0/3265256304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a8112680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 --2- 172.21.15.114:0/3265256304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33a810b500 0x7f33a810b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 --2- 172.21.15.114:0/3265256304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 -- 172.21.15.114:0/3265256304 >> 172.21.15.114:0/3265256304 conn(0x7f33a8101670 msgr2=0x7f33a8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 -- 172.21.15.114:0/3265256304 shutdown_connections 2024-09-21T06:53:53.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.061+0000 7f33aec00640 1 -- 172.21.15.114:0/3265256304 wait complete. 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 Processor -- start 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 -- start start 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33a810b500 0x7f33a819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a81a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8115b60 con 0x7f33a810be50 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33a81159e0 con 0x7f33a8105bc0 2024-09-21T06:53:53.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.063+0000 7f33aec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8115ce0 con 0x7f33a810b500 2024-09-21T06:53:53.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:47082/0 (socket says 172.21.15.114:47082) 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 -- 172.21.15.114:0/563955609 learned_addr learned my addr 172.21.15.114:0/563955609 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ae200640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a81a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ace00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33a810b500 0x7f33a819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 -- 172.21.15.114:0/563955609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33a810b500 msgr2=0x7f33a819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33a810b500 0x7f33a819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 -- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 msgr2=0x7f33a81a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a81a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ad800640 1 -- 172.21.15.114:0/563955609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a81a2460 con 0x7f33a8105bc0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.064+0000 7f33ae200640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a81a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.065+0000 7f33ad800640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a819c4f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f339c00e9f0 tx=0x7f339c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.065+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c00cd80 con 0x7f33a8105bc0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.065+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a81a2750 con 0x7f33a8105bc0 2024-09-21T06:53:53.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.065+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f339c0108e0 con 0x7f33a8105bc0 2024-09-21T06:53:53.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.065+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c018980 con 0x7f33a8105bc0 2024-09-21T06:53:53.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.065+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a81a8eb0 con 0x7f33a8105bc0 2024-09-21T06:53:53.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.066+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a8105fd0 con 0x7f33a8105bc0 2024-09-21T06:53:53.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.068+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f339c010430 con 0x7f33a8105bc0 2024-09-21T06:53:53.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.070+0000 7f33a6000640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f33900b65e0 0x7f33900b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:53.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.070+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f339c0e7f80 con 0x7f33a8105bc0 2024-09-21T06:53:53.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.070+0000 7f33ace00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f33900b65e0 0x7f33900b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:53.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.071+0000 7f33ace00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f33900b65e0 0x7f33900b8af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33a8104ec0 tx=0x7f3394005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:53.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.072+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f339c0b2700 con 0x7f33a8105bc0 2024-09-21T06:53:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: pgmap v388: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T06:53:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:53.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:53.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2366428389' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T06:53:53.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:53:53.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:53:53.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:53:53.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.292+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33a819d620 con 0x7f33a8105bc0 2024-09-21T06:53:53.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.293+0000 7f33a6000640 1 -- 172.21.15.114:0/563955609 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f339c0b7640 con 0x7f33a8105bc0 2024-09-21T06:53:53.295 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:53.295 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"btime":"2024-09-21T06:53:49:573848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:53:49.006951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:53:53.295 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-21T06:53:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f33900b65e0 msgr2=0x7f33900b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f33900b65e0 0x7f33900b8af0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33a8104ec0 tx=0x7f3394005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 msgr2=0x7f33a819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a819c4f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f339c00e9f0 tx=0x7f339c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 shutdown_connections 2024-09-21T06:53:53.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f33a810be50 0x7f33a81a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33a810b500 0x7f33a819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f33900b65e0 0x7f33900b8af0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 --2- 172.21.15.114:0/563955609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a8105bc0 0x7f33a819c4f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.296+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 >> 172.21.15.114:0/563955609 conn(0x7f33a8101670 msgr2=0x7f33a810ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.297+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 shutdown_connections 2024-09-21T06:53:53.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:53.297+0000 7f33aec00640 1 -- 172.21.15.114:0/563955609 wait complete. 2024-09-21T06:53:54.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:53 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/563955609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:54.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:53 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/563955609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:54.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:53 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/563955609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:54.264 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T06:53:54.264 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T06:53:54.264 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 -- 172.21.15.114:0/4250997002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec105230 msgr2=0x7f93ec10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 --2- 172.21.15.114:0/4250997002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec105230 0x7f93ec10d770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009900 tx=0x7f93d802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 -- 172.21.15.114:0/4250997002 shutdown_connections 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 --2- 172.21.15.114:0/4250997002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ec10dd40 0x7f93ec110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 --2- 172.21.15.114:0/4250997002 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec105230 0x7f93ec10d770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 --2- 172.21.15.114:0/4250997002 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec1048e0 0x7f93ec104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 -- 172.21.15.114:0/4250997002 >> 172.21.15.114:0/4250997002 conn(0x7f93ec0fe350 msgr2=0x7f93ec1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 -- 172.21.15.114:0/4250997002 shutdown_connections 2024-09-21T06:53:54.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.482+0000 7f93f3e00640 1 -- 172.21.15.114:0/4250997002 wait complete. 2024-09-21T06:53:54.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.483+0000 7f93f3e00640 1 Processor -- start 2024-09-21T06:53:54.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.483+0000 7f93f3e00640 1 -- start start 2024-09-21T06:53:54.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 0x7f93ec19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ec105230 0x7f93ec19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec10dd40 0x7f93ec1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1139a0 con 0x7f93ec1048e0 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec113820 con 0x7f93ec105230 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec113b20 con 0x7f93ec10dd40 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 0x7f93ec19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 0x7f93ec19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51334/0 (socket says 172.21.15.114:51334) 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 -- 172.21.15.114:0/2843072980 learned_addr learned my addr 172.21.15.114:0/2843072980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f0a00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ec105230 0x7f93ec19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec10dd40 0x7f93ec1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 -- 172.21.15.114:0/2843072980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec10dd40 msgr2=0x7f93ec1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec10dd40 0x7f93ec1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 -- 172.21.15.114:0/2843072980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ec105230 msgr2=0x7f93ec19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ec105230 0x7f93ec19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1400640 1 -- 172.21.15.114:0/2843072980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec1a2550 con 0x7f93ec1048e0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.484+0000 7f93f1e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec10dd40 0x7f93ec1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.485+0000 7f93f1400640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 0x7f93ec19c620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f93e000e9f0 tx=0x7f93e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.485+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e000cd80 con 0x7f93ec1048e0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.485+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e00054d0 con 0x7f93ec1048e0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.485+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec1a2840 con 0x7f93ec1048e0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.485+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0010690 con 0x7f93ec1048e0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.485+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec1a8fd0 con 0x7f93ec1048e0 2024-09-21T06:53:54.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.486+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec06c090 con 0x7f93ec1048e0 2024-09-21T06:53:54.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.488+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f93e0004e90 con 0x7f93ec1048e0 2024-09-21T06:53:54.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.490+0000 7f93ea000640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93d40b6560 0x7f93d40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.490+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f93e00e7ce0 con 0x7f93ec1048e0 2024-09-21T06:53:54.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.490+0000 7f93f0a00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93d40b6560 0x7f93d40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.491+0000 7f93f0a00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93d40b6560 0x7f93d40b8a70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009230 tx=0x7f93d8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:54.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.494+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f93e00ae080 con 0x7f93ec1048e0 2024-09-21T06:53:54.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.680+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93ec1032d0 con 0x7f93d40b6560 2024-09-21T06:53:54.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.687+0000 7f93ea000640 1 -- 172.21.15.114:0/2843072980 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340511 (secure 0 0 0) 0x7f93ec1032d0 con 0x7f93d40b6560 2024-09-21T06:53:54.693 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:54.695 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:53:54.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93d40b6560 msgr2=0x7f93d40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93d40b6560 0x7f93d40b8a70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009230 tx=0x7f93d8031400 comp rx=0 tx=0).stop 2024-09-21T06:53:54.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 msgr2=0x7f93ec19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 0x7f93ec19c620 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f93e000e9f0 tx=0x7f93e000edf0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 shutdown_connections 2024-09-21T06:53:54.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93ec10dd40 0x7f93ec1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93d40b6560 0x7f93d40b8a70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93ec105230 0x7f93ec19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 --2- 172.21.15.114:0/2843072980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f93ec1048e0 0x7f93ec19c620 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 >> 172.21.15.114:0/2843072980 conn(0x7f93ec0fe350 msgr2=0x7f93ec102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:54.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 shutdown_connections 2024-09-21T06:53:54.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.692+0000 7f93f3e00640 1 -- 172.21.15.114:0/2843072980 wait complete. 2024-09-21T06:53:54.716 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-21T06:53:53.542113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666558,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1402,"num_read_kb":815,"num_write":394,"num_write_kb":2314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376036,"kb_used_data":14308,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565788,"statfs":{"total":1151940427776,"available":1151555366912,"internally_reserved":0,"allocated":14651392,"data_stored":8351922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":62,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003709"},"pg_stats":[{"pgid":"2.1f","version":"89'15","reported_seq":189,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.825504+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:53:50.825504+0000","last_peered":"2024-09-21T06:53:50.825504+0000","last_clean":"2024-09-21T06:53:50.825504+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:53:50.825504+0000","last_undegraded":"2024-09-21T06:53:50.825504+0000","last_fullsized":"2024-09-21T06:53:50.825504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741199+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:59.741199+0000","last_peered":"2024-09-21T06:52:59.741199+0000","last_clean":"2024-09-21T06:52:59.741199+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:59.741199+0000","last_undegraded":"2024-09-21T06:52:59.741199+0000","last_fullsized":"2024-09-21T06:52:59.741199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703671+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703671+0000","last_peered":"2024-09-21T06:52:31.703671+0000","last_clean":"2024-09-21T06:52:31.703671+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703671+0000","last_undegraded":"2024-09-21T06:52:31.703671+0000","last_fullsized":"2024-09-21T06:52:31.703671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699254+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699254+0000","last_peered":"2024-09-21T06:52:31.699254+0000","last_clean":"2024-09-21T06:52:31.699254+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699254+0000","last_undegraded":"2024-09-21T06:52:31.699254+0000","last_fullsized":"2024-09-21T06:52:31.699254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:16.554226+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:53:16.554226+0000","last_peered":"2024-09-21T06:53:16.554226+0000","last_clean":"2024-09-21T06:53:16.554226+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:53:16.554226+0000","last_undegraded":"2024-09-21T06:53:16.554226+0000","last_fullsized":"2024-09-21T06:53:16.554226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755557+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:59.755557+0000","last_peered":"2024-09-21T06:52:59.755557+0000","last_clean":"2024-09-21T06:52:59.755557+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:59.755557+0000","last_undegraded":"2024-09-21T06:52:59.755557+0000","last_fullsized":"2024-09-21T06:52:59.755557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699992+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699992+0000","last_peered":"2024-09-21T06:52:31.699992+0000","last_clean":"2024-09-21T06:52:31.699992+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699992+0000","last_undegraded":"2024-09-21T06:52:31.699992+0000","last_fullsized":"2024-09-21T06:52:31.699992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891939+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891939+0000","last_peered":"2024-09-21T06:52:31.891939+0000","last_clean":"2024-09-21T06:52:31.891939+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891939+0000","last_undegraded":"2024-09-21T06:52:31.891939+0000","last_fullsized":"2024-09-21T06:52:31.891939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751784+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:59.751784+0000","last_peered":"2024-09-21T06:52:59.751784+0000","last_clean":"2024-09-21T06:52:59.751784+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:59.751784+0000","last_undegraded":"2024-09-21T06:52:59.751784+0000","last_fullsized":"2024-09-21T06:52:59.751784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699375+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699375+0000","last_peered":"2024-09-21T06:52:31.699375+0000","last_clean":"2024-09-21T06:52:31.699375+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699375+0000","last_undegraded":"2024-09-21T06:52:31.699375+0000","last_fullsized":"2024-09-21T06:52:31.699375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755326+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:59.755326+0000","last_peered":"2024-09-21T06:52:59.755326+0000","last_clean":"2024-09-21T06:52:59.755326+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:59.755326+0000","last_undegraded":"2024-09-21T06:52:59.755326+0000","last_fullsized":"2024-09-21T06:52:59.755326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645462+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645462+0000","last_peered":"2024-09-21T06:52:31.645462+0000","last_clean":"2024-09-21T06:52:31.645462+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645462+0000","last_undegraded":"2024-09-21T06:52:31.645462+0000","last_fullsized":"2024-09-21T06:52:31.645462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755915+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:59.755915+0000","last_peered":"2024-09-21T06:52:59.755915+0000","last_clean":"2024-09-21T06:52:59.755915+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:59.755915+0000","last_undegraded":"2024-09-21T06:52:59.755915+0000","last_fullsized":"2024-09-21T06:52:59.755915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704037+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.704037+0000","last_peered":"2024-09-21T06:52:31.704037+0000","last_clean":"2024-09-21T06:52:31.704037+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.704037+0000","last_undegraded":"2024-09-21T06:52:31.704037+0000","last_fullsized":"2024-09-21T06:52:31.704037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699968+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699968+0000","last_peered":"2024-09-21T06:52:31.699968+0000","last_clean":"2024-09-21T06:52:31.699968+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699968+0000","last_undegraded":"2024-09-21T06:52:31.699968+0000","last_fullsized":"2024-09-21T06:52:31.699968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745662+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:59.745662+0000","last_peered":"2024-09-21T06:52:59.745662+0000","last_clean":"2024-09-21T06:52:59.745662+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:59.745662+0000","last_undegraded":"2024-09-21T06:52:59.745662+0000","last_fullsized":"2024-09-21T06:52:59.745662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892935+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892935+0000","last_peered":"2024-09-21T06:52:31.892935+0000","last_clean":"2024-09-21T06:52:31.892935+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892935+0000","last_undegraded":"2024-09-21T06:52:31.892935+0000","last_fullsized":"2024-09-21T06:52:31.892935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646491+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646491+0000","last_peered":"2024-09-21T06:52:31.646491+0000","last_clean":"2024-09-21T06:52:31.646491+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646491+0000","last_undegraded":"2024-09-21T06:52:31.646491+0000","last_fullsized":"2024-09-21T06:52:31.646491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744826+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:59.744826+0000","last_peered":"2024-09-21T06:52:59.744826+0000","last_clean":"2024-09-21T06:52:59.744826+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:59.744826+0000","last_undegraded":"2024-09-21T06:52:59.744826+0000","last_fullsized":"2024-09-21T06:52:59.744826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836870+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836870+0000","last_peered":"2024-09-21T06:52:31.836870+0000","last_clean":"2024-09-21T06:52:31.836870+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836870+0000","last_undegraded":"2024-09-21T06:52:31.836870+0000","last_fullsized":"2024-09-21T06:52:31.836870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755751+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:59.755751+0000","last_peered":"2024-09-21T06:52:59.755751+0000","last_clean":"2024-09-21T06:52:59.755751+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:59.755751+0000","last_undegraded":"2024-09-21T06:52:59.755751+0000","last_fullsized":"2024-09-21T06:52:59.755751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700368+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700368+0000","last_peered":"2024-09-21T06:52:31.700368+0000","last_clean":"2024-09-21T06:52:31.700368+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700368+0000","last_undegraded":"2024-09-21T06:52:31.700368+0000","last_fullsized":"2024-09-21T06:52:31.700368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703897+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703897+0000","last_peered":"2024-09-21T06:52:31.703897+0000","last_clean":"2024-09-21T06:52:31.703897+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703897+0000","last_undegraded":"2024-09-21T06:52:31.703897+0000","last_fullsized":"2024-09-21T06:52:31.703897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741893+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:59.741893+0000","last_peered":"2024-09-21T06:52:59.741893+0000","last_clean":"2024-09-21T06:52:59.741893+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:59.741893+0000","last_undegraded":"2024-09-21T06:52:59.741893+0000","last_fullsized":"2024-09-21T06:52:59.741893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:00.244438+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:53:00.244438+0000","last_peered":"2024-09-21T06:53:00.244438+0000","last_clean":"2024-09-21T06:53:00.244438+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:53:00.244438+0000","last_undegraded":"2024-09-21T06:53:00.244438+0000","last_fullsized":"2024-09-21T06:53:00.244438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891910+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891910+0000","last_peered":"2024-09-21T06:52:31.891910+0000","last_clean":"2024-09-21T06:52:31.891910+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891910+0000","last_undegraded":"2024-09-21T06:52:31.891910+0000","last_fullsized":"2024-09-21T06:52:31.891910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742064+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:59.742064+0000","last_peered":"2024-09-21T06:52:59.742064+0000","last_clean":"2024-09-21T06:52:59.742064+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:59.742064+0000","last_undegraded":"2024-09-21T06:52:59.742064+0000","last_fullsized":"2024-09-21T06:52:59.742064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892041+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892041+0000","last_peered":"2024-09-21T06:52:31.892041+0000","last_clean":"2024-09-21T06:52:31.892041+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892041+0000","last_undegraded":"2024-09-21T06:52:31.892041+0000","last_fullsized":"2024-09-21T06:52:31.892041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645535+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645535+0000","last_peered":"2024-09-21T06:52:31.645535+0000","last_clean":"2024-09-21T06:52:31.645535+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645535+0000","last_undegraded":"2024-09-21T06:52:31.645535+0000","last_fullsized":"2024-09-21T06:52:31.645535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754090+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:59.754090+0000","last_peered":"2024-09-21T06:52:59.754090+0000","last_clean":"2024-09-21T06:52:59.754090+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:59.754090+0000","last_undegraded":"2024-09-21T06:52:59.754090+0000","last_fullsized":"2024-09-21T06:52:59.754090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645977+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645977+0000","last_peered":"2024-09-21T06:52:31.645977+0000","last_clean":"2024-09-21T06:52:31.645977+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645977+0000","last_undegraded":"2024-09-21T06:52:31.645977+0000","last_fullsized":"2024-09-21T06:52:31.645977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891517+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891517+0000","last_peered":"2024-09-21T06:52:31.891517+0000","last_clean":"2024-09-21T06:52:31.891517+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891517+0000","last_undegraded":"2024-09-21T06:52:31.891517+0000","last_fullsized":"2024-09-21T06:52:31.891517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758656+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:59.758656+0000","last_peered":"2024-09-21T06:52:59.758656+0000","last_clean":"2024-09-21T06:52:59.758656+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:59.758656+0000","last_undegraded":"2024-09-21T06:52:59.758656+0000","last_fullsized":"2024-09-21T06:52:59.758656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'8","reported_seq":182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:48.822552+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:53:48.822552+0000","last_peered":"2024-09-21T06:53:48.822552+0000","last_clean":"2024-09-21T06:53:48.822552+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:53:48.822552+0000","last_undegraded":"2024-09-21T06:53:48.822552+0000","last_fullsized":"2024-09-21T06:53:48.822552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1431,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701262+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701262+0000","last_peered":"2024-09-21T06:52:31.701262+0000","last_clean":"2024-09-21T06:52:31.701262+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701262+0000","last_undegraded":"2024-09-21T06:52:31.701262+0000","last_fullsized":"2024-09-21T06:52:31.701262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749255+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:59.749255+0000","last_peered":"2024-09-21T06:52:59.749255+0000","last_clean":"2024-09-21T06:52:59.749255+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:59.749255+0000","last_undegraded":"2024-09-21T06:52:59.749255+0000","last_fullsized":"2024-09-21T06:52:59.749255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.825547+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:53:50.825547+0000","last_peered":"2024-09-21T06:53:50.825547+0000","last_clean":"2024-09-21T06:53:50.825547+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:53:50.825547+0000","last_undegraded":"2024-09-21T06:53:50.825547+0000","last_fullsized":"2024-09-21T06:53:50.825547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701440+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701440+0000","last_peered":"2024-09-21T06:52:31.701440+0000","last_clean":"2024-09-21T06:52:31.701440+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701440+0000","last_undegraded":"2024-09-21T06:52:31.701440+0000","last_fullsized":"2024-09-21T06:52:31.701440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740983+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:59.740983+0000","last_peered":"2024-09-21T06:52:59.740983+0000","last_clean":"2024-09-21T06:52:59.740983+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:59.740983+0000","last_undegraded":"2024-09-21T06:52:59.740983+0000","last_fullsized":"2024-09-21T06:52:59.740983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701878+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701878+0000","last_peered":"2024-09-21T06:52:31.701878+0000","last_clean":"2024-09-21T06:52:31.701878+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701878+0000","last_undegraded":"2024-09-21T06:52:31.701878+0000","last_fullsized":"2024-09-21T06:52:31.701878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749354+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:59.749354+0000","last_peered":"2024-09-21T06:52:59.749354+0000","last_clean":"2024-09-21T06:52:59.749354+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:59.749354+0000","last_undegraded":"2024-09-21T06:52:59.749354+0000","last_fullsized":"2024-09-21T06:52:59.749354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648426+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648426+0000","last_peered":"2024-09-21T06:52:31.648426+0000","last_clean":"2024-09-21T06:52:31.648426+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648426+0000","last_undegraded":"2024-09-21T06:52:31.648426+0000","last_fullsized":"2024-09-21T06:52:31.648426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648721+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648721+0000","last_peered":"2024-09-21T06:52:31.648721+0000","last_clean":"2024-09-21T06:52:31.648721+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648721+0000","last_undegraded":"2024-09-21T06:52:31.648721+0000","last_fullsized":"2024-09-21T06:52:31.648721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753632+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:59.753632+0000","last_peered":"2024-09-21T06:52:59.753632+0000","last_clean":"2024-09-21T06:52:59.753632+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:59.753632+0000","last_undegraded":"2024-09-21T06:52:59.753632+0000","last_fullsized":"2024-09-21T06:52:59.753632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":183,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.826734+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:53:50.826734+0000","last_peered":"2024-09-21T06:53:50.826734+0000","last_clean":"2024-09-21T06:53:50.826734+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:53:50.826734+0000","last_undegraded":"2024-09-21T06:53:50.826734+0000","last_fullsized":"2024-09-21T06:53:50.826734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836829+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836829+0000","last_peered":"2024-09-21T06:52:31.836829+0000","last_clean":"2024-09-21T06:52:31.836829+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836829+0000","last_undegraded":"2024-09-21T06:52:31.836829+0000","last_fullsized":"2024-09-21T06:52:31.836829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744841+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:59.744841+0000","last_peered":"2024-09-21T06:52:59.744841+0000","last_clean":"2024-09-21T06:52:59.744841+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:59.744841+0000","last_undegraded":"2024-09-21T06:52:59.744841+0000","last_fullsized":"2024-09-21T06:52:59.744841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702337+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:52:31.702337+0000","last_peered":"2024-09-21T06:52:31.702337+0000","last_clean":"2024-09-21T06:52:31.702337+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:52:31.702337+0000","last_undegraded":"2024-09-21T06:52:31.702337+0000","last_fullsized":"2024-09-21T06:52:31.702337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891861+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891861+0000","last_peered":"2024-09-21T06:52:31.891861+0000","last_clean":"2024-09-21T06:52:31.891861+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891861+0000","last_undegraded":"2024-09-21T06:52:31.891861+0000","last_fullsized":"2024-09-21T06:52:31.891861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753918+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:59.753918+0000","last_peered":"2024-09-21T06:52:59.753918+0000","last_clean":"2024-09-21T06:52:59.753918+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:59.753918+0000","last_undegraded":"2024-09-21T06:52:59.753918+0000","last_fullsized":"2024-09-21T06:52:59.753918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701640+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701640+0000","last_peered":"2024-09-21T06:52:31.701640+0000","last_clean":"2024-09-21T06:52:31.701640+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701640+0000","last_undegraded":"2024-09-21T06:52:31.701640+0000","last_fullsized":"2024-09-21T06:52:31.701640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701328+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701328+0000","last_peered":"2024-09-21T06:52:31.701328+0000","last_clean":"2024-09-21T06:52:31.701328+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701328+0000","last_undegraded":"2024-09-21T06:52:31.701328+0000","last_fullsized":"2024-09-21T06:52:31.701328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734233+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:59.734233+0000","last_peered":"2024-09-21T06:52:59.734233+0000","last_clean":"2024-09-21T06:52:59.734233+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:59.734233+0000","last_undegraded":"2024-09-21T06:52:59.734233+0000","last_fullsized":"2024-09-21T06:52:59.734233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645941+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645941+0000","last_peered":"2024-09-21T06:52:31.645941+0000","last_clean":"2024-09-21T06:52:31.645941+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645941+0000","last_undegraded":"2024-09-21T06:52:31.645941+0000","last_fullsized":"2024-09-21T06:52:31.645941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701203+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701203+0000","last_peered":"2024-09-21T06:52:31.701203+0000","last_clean":"2024-09-21T06:52:31.701203+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701203+0000","last_undegraded":"2024-09-21T06:52:31.701203+0000","last_fullsized":"2024-09-21T06:52:31.701203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.748492+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:59.748492+0000","last_peered":"2024-09-21T06:52:59.748492+0000","last_clean":"2024-09-21T06:52:59.748492+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:59.748492+0000","last_undegraded":"2024-09-21T06:52:59.748492+0000","last_fullsized":"2024-09-21T06:52:59.748492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891566+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891566+0000","last_peered":"2024-09-21T06:52:31.891566+0000","last_clean":"2024-09-21T06:52:31.891566+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891566+0000","last_undegraded":"2024-09-21T06:52:31.891566+0000","last_fullsized":"2024-09-21T06:52:31.891566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701524+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701524+0000","last_peered":"2024-09-21T06:52:31.701524+0000","last_clean":"2024-09-21T06:52:31.701524+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701524+0000","last_undegraded":"2024-09-21T06:52:31.701524+0000","last_fullsized":"2024-09-21T06:52:31.701524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756438+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:59.756438+0000","last_peered":"2024-09-21T06:52:59.756438+0000","last_clean":"2024-09-21T06:52:59.756438+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:59.756438+0000","last_undegraded":"2024-09-21T06:52:59.756438+0000","last_fullsized":"2024-09-21T06:52:59.756438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:49.824645+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:53:49.824645+0000","last_peered":"2024-09-21T06:53:49.824645+0000","last_clean":"2024-09-21T06:53:49.824645+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:53:49.824645+0000","last_undegraded":"2024-09-21T06:53:49.824645+0000","last_fullsized":"2024-09-21T06:53:49.824645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891921+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891921+0000","last_peered":"2024-09-21T06:52:31.891921+0000","last_clean":"2024-09-21T06:52:31.891921+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891921+0000","last_undegraded":"2024-09-21T06:52:31.891921+0000","last_fullsized":"2024-09-21T06:52:31.891921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758614+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:59.758614+0000","last_peered":"2024-09-21T06:52:59.758614+0000","last_clean":"2024-09-21T06:52:59.758614+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:59.758614+0000","last_undegraded":"2024-09-21T06:52:59.758614+0000","last_fullsized":"2024-09-21T06:52:59.758614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701396+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701396+0000","last_peered":"2024-09-21T06:52:31.701396+0000","last_clean":"2024-09-21T06:52:31.701396+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701396+0000","last_undegraded":"2024-09-21T06:52:31.701396+0000","last_fullsized":"2024-09-21T06:52:31.701396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645499+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645499+0000","last_peered":"2024-09-21T06:52:31.645499+0000","last_clean":"2024-09-21T06:52:31.645499+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645499+0000","last_undegraded":"2024-09-21T06:52:31.645499+0000","last_fullsized":"2024-09-21T06:52:31.645499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749562+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:59.749562+0000","last_peered":"2024-09-21T06:52:59.749562+0000","last_clean":"2024-09-21T06:52:59.749562+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:59.749562+0000","last_undegraded":"2024-09-21T06:52:59.749562+0000","last_fullsized":"2024-09-21T06:52:59.749562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703886+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703886+0000","last_peered":"2024-09-21T06:52:31.703886+0000","last_clean":"2024-09-21T06:52:31.703886+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703886+0000","last_undegraded":"2024-09-21T06:52:31.703886+0000","last_fullsized":"2024-09-21T06:52:31.703886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702060+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.702060+0000","last_peered":"2024-09-21T06:52:31.702060+0000","last_clean":"2024-09-21T06:52:31.702060+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.702060+0000","last_undegraded":"2024-09-21T06:52:31.702060+0000","last_fullsized":"2024-09-21T06:52:31.702060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745772+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:59.745772+0000","last_peered":"2024-09-21T06:52:59.745772+0000","last_clean":"2024-09-21T06:52:59.745772+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:59.745772+0000","last_undegraded":"2024-09-21T06:52:59.745772+0000","last_fullsized":"2024-09-21T06:52:59.745772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:48.824553+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:53:48.824553+0000","last_peered":"2024-09-21T06:53:48.824553+0000","last_clean":"2024-09-21T06:53:48.824553+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:53:48.824553+0000","last_undegraded":"2024-09-21T06:53:48.824553+0000","last_fullsized":"2024-09-21T06:53:48.824553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67627,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":263,"num_read_kb":199,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645656+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645656+0000","last_peered":"2024-09-21T06:52:31.645656+0000","last_clean":"2024-09-21T06:52:31.645656+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645656+0000","last_undegraded":"2024-09-21T06:52:31.645656+0000","last_fullsized":"2024-09-21T06:52:31.645656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.747527+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:59.747527+0000","last_peered":"2024-09-21T06:52:59.747527+0000","last_clean":"2024-09-21T06:52:59.747527+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:59.747527+0000","last_undegraded":"2024-09-21T06:52:59.747527+0000","last_fullsized":"2024-09-21T06:52:59.747527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702309+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702309+0000","last_peered":"2024-09-21T06:52:31.702309+0000","last_clean":"2024-09-21T06:52:31.702309+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702309+0000","last_undegraded":"2024-09-21T06:52:31.702309+0000","last_fullsized":"2024-09-21T06:52:31.702309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699508+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699508+0000","last_peered":"2024-09-21T06:52:31.699508+0000","last_clean":"2024-09-21T06:52:31.699508+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699508+0000","last_undegraded":"2024-09-21T06:52:31.699508+0000","last_fullsized":"2024-09-21T06:52:31.699508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757106+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:59.757106+0000","last_peered":"2024-09-21T06:52:59.757106+0000","last_clean":"2024-09-21T06:52:59.757106+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:59.757106+0000","last_undegraded":"2024-09-21T06:52:59.757106+0000","last_fullsized":"2024-09-21T06:52:59.757106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648918+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648918+0000","last_peered":"2024-09-21T06:52:31.648918+0000","last_clean":"2024-09-21T06:52:31.648918+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648918+0000","last_undegraded":"2024-09-21T06:52:31.648918+0000","last_fullsized":"2024-09-21T06:52:31.648918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645559+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645559+0000","last_peered":"2024-09-21T06:52:31.645559+0000","last_clean":"2024-09-21T06:52:31.645559+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645559+0000","last_undegraded":"2024-09-21T06:52:31.645559+0000","last_fullsized":"2024-09-21T06:52:31.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750255+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:59.750255+0000","last_peered":"2024-09-21T06:52:59.750255+0000","last_clean":"2024-09-21T06:52:59.750255+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:59.750255+0000","last_undegraded":"2024-09-21T06:52:59.750255+0000","last_fullsized":"2024-09-21T06:52:59.750255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:48.822735+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:53:48.822735+0000","last_peered":"2024-09-21T06:53:48.822735+0000","last_clean":"2024-09-21T06:53:48.822735+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:53:48.822735+0000","last_undegraded":"2024-09-21T06:53:48.822735+0000","last_fullsized":"2024-09-21T06:53:48.822735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":252,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892718+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892718+0000","last_peered":"2024-09-21T06:52:31.892718+0000","last_clean":"2024-09-21T06:52:31.892718+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892718+0000","last_undegraded":"2024-09-21T06:52:31.892718+0000","last_fullsized":"2024-09-21T06:52:31.892718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892186+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892186+0000","last_peered":"2024-09-21T06:52:31.892186+0000","last_clean":"2024-09-21T06:52:31.892186+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892186+0000","last_undegraded":"2024-09-21T06:52:31.892186+0000","last_fullsized":"2024-09-21T06:52:31.892186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741381+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:59.741381+0000","last_peered":"2024-09-21T06:52:59.741381+0000","last_clean":"2024-09-21T06:52:59.741381+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:59.741381+0000","last_undegraded":"2024-09-21T06:52:59.741381+0000","last_fullsized":"2024-09-21T06:52:59.741381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701609+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701609+0000","last_peered":"2024-09-21T06:52:31.701609+0000","last_clean":"2024-09-21T06:52:31.701609+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701609+0000","last_undegraded":"2024-09-21T06:52:31.701609+0000","last_fullsized":"2024-09-21T06:52:31.701609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645808+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645808+0000","last_peered":"2024-09-21T06:52:31.645808+0000","last_clean":"2024-09-21T06:52:31.645808+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645808+0000","last_undegraded":"2024-09-21T06:52:31.645808+0000","last_fullsized":"2024-09-21T06:52:31.645808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742667+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:59.742667+0000","last_peered":"2024-09-21T06:52:59.742667+0000","last_clean":"2024-09-21T06:52:59.742667+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:59.742667+0000","last_undegraded":"2024-09-21T06:52:59.742667+0000","last_fullsized":"2024-09-21T06:52:59.742667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700994+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700994+0000","last_peered":"2024-09-21T06:52:31.700994+0000","last_clean":"2024-09-21T06:52:31.700994+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700994+0000","last_undegraded":"2024-09-21T06:52:31.700994+0000","last_fullsized":"2024-09-21T06:52:31.700994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645981+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645981+0000","last_peered":"2024-09-21T06:52:31.645981+0000","last_clean":"2024-09-21T06:52:31.645981+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645981+0000","last_undegraded":"2024-09-21T06:52:31.645981+0000","last_fullsized":"2024-09-21T06:52:31.645981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749130+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:59.749130+0000","last_peered":"2024-09-21T06:52:59.749130+0000","last_clean":"2024-09-21T06:52:59.749130+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:59.749130+0000","last_undegraded":"2024-09-21T06:52:59.749130+0000","last_fullsized":"2024-09-21T06:52:59.749130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703865+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703865+0000","last_peered":"2024-09-21T06:52:31.703865+0000","last_clean":"2024-09-21T06:52:31.703865+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703865+0000","last_undegraded":"2024-09-21T06:52:31.703865+0000","last_fullsized":"2024-09-21T06:52:31.703865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:49.825569+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:53:49.825569+0000","last_peered":"2024-09-21T06:53:49.825569+0000","last_clean":"2024-09-21T06:53:49.825569+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:53:49.825569+0000","last_undegraded":"2024-09-21T06:53:49.825569+0000","last_fullsized":"2024-09-21T06:53:49.825569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.736418+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:59.736418+0000","last_peered":"2024-09-21T06:52:59.736418+0000","last_clean":"2024-09-21T06:52:59.736418+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:59.736418+0000","last_undegraded":"2024-09-21T06:52:59.736418+0000","last_fullsized":"2024-09-21T06:52:59.736418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704051+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.704051+0000","last_peered":"2024-09-21T06:52:31.704051+0000","last_clean":"2024-09-21T06:52:31.704051+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.704051+0000","last_undegraded":"2024-09-21T06:52:31.704051+0000","last_fullsized":"2024-09-21T06:52:31.704051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.739975+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:59.739975+0000","last_peered":"2024-09-21T06:52:59.739975+0000","last_clean":"2024-09-21T06:52:59.739975+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:59.739975+0000","last_undegraded":"2024-09-21T06:52:59.739975+0000","last_fullsized":"2024-09-21T06:52:59.739975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749347+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:59.749347+0000","last_peered":"2024-09-21T06:52:59.749347+0000","last_clean":"2024-09-21T06:52:59.749347+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:59.749347+0000","last_undegraded":"2024-09-21T06:52:59.749347+0000","last_fullsized":"2024-09-21T06:52:59.749347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746716+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:59.746716+0000","last_peered":"2024-09-21T06:52:59.746716+0000","last_clean":"2024-09-21T06:52:59.746716+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:59.746716+0000","last_undegraded":"2024-09-21T06:52:59.746716+0000","last_fullsized":"2024-09-21T06:52:59.746716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757121+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:59.757121+0000","last_peered":"2024-09-21T06:52:59.757121+0000","last_clean":"2024-09-21T06:52:59.757121+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:59.757121+0000","last_undegraded":"2024-09-21T06:52:59.757121+0000","last_fullsized":"2024-09-21T06:52:59.757121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756339+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:59.756339+0000","last_peered":"2024-09-21T06:52:59.756339+0000","last_clean":"2024-09-21T06:52:59.756339+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:59.756339+0000","last_undegraded":"2024-09-21T06:52:59.756339+0000","last_fullsized":"2024-09-21T06:52:59.756339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734085+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:59.734085+0000","last_peered":"2024-09-21T06:52:59.734085+0000","last_clean":"2024-09-21T06:52:59.734085+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:59.734085+0000","last_undegraded":"2024-09-21T06:52:59.734085+0000","last_fullsized":"2024-09-21T06:52:59.734085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749595+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:59.749595+0000","last_peered":"2024-09-21T06:52:59.749595+0000","last_clean":"2024-09-21T06:52:59.749595+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:59.749595+0000","last_undegraded":"2024-09-21T06:52:59.749595+0000","last_fullsized":"2024-09-21T06:52:59.749595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.737176+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:59.737176+0000","last_peered":"2024-09-21T06:52:59.737176+0000","last_clean":"2024-09-21T06:52:59.737176+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:59.737176+0000","last_undegraded":"2024-09-21T06:52:59.737176+0000","last_fullsized":"2024-09-21T06:52:59.737176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746235+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:59.746235+0000","last_peered":"2024-09-21T06:52:59.746235+0000","last_clean":"2024-09-21T06:52:59.746235+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:59.746235+0000","last_undegraded":"2024-09-21T06:52:59.746235+0000","last_fullsized":"2024-09-21T06:52:59.746235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740649+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:59.740649+0000","last_peered":"2024-09-21T06:52:59.740649+0000","last_clean":"2024-09-21T06:52:59.740649+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:59.740649+0000","last_undegraded":"2024-09-21T06:52:59.740649+0000","last_fullsized":"2024-09-21T06:52:59.740649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753696+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:59.753696+0000","last_peered":"2024-09-21T06:52:59.753696+0000","last_clean":"2024-09-21T06:52:59.753696+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:59.753696+0000","last_undegraded":"2024-09-21T06:52:59.753696+0000","last_fullsized":"2024-09-21T06:52:59.753696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756495+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:59.756495+0000","last_peered":"2024-09-21T06:52:59.756495+0000","last_clean":"2024-09-21T06:52:59.756495+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:59.756495+0000","last_undegraded":"2024-09-21T06:52:59.756495+0000","last_fullsized":"2024-09-21T06:52:59.756495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734175+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:59.734175+0000","last_peered":"2024-09-21T06:52:59.734175+0000","last_clean":"2024-09-21T06:52:59.734175+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:59.734175+0000","last_undegraded":"2024-09-21T06:52:59.734175+0000","last_fullsized":"2024-09-21T06:52:59.734175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745394+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:59.745394+0000","last_peered":"2024-09-21T06:52:59.745394+0000","last_clean":"2024-09-21T06:52:59.745394+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:59.745394+0000","last_undegraded":"2024-09-21T06:52:59.745394+0000","last_fullsized":"2024-09-21T06:52:59.745394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751016+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:59.751016+0000","last_peered":"2024-09-21T06:52:59.751016+0000","last_clean":"2024-09-21T06:52:59.751016+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:59.751016+0000","last_undegraded":"2024-09-21T06:52:59.751016+0000","last_fullsized":"2024-09-21T06:52:59.751016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755908+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:59.755908+0000","last_peered":"2024-09-21T06:52:59.755908+0000","last_clean":"2024-09-21T06:52:59.755908+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:59.755908+0000","last_undegraded":"2024-09-21T06:52:59.755908+0000","last_fullsized":"2024-09-21T06:52:59.755908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752146+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:59.752146+0000","last_peered":"2024-09-21T06:52:59.752146+0000","last_clean":"2024-09-21T06:52:59.752146+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:59.752146+0000","last_undegraded":"2024-09-21T06:52:59.752146+0000","last_fullsized":"2024-09-21T06:52:59.752146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754481+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:59.754481+0000","last_peered":"2024-09-21T06:52:59.754481+0000","last_clean":"2024-09-21T06:52:59.754481+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:59.754481+0000","last_undegraded":"2024-09-21T06:52:59.754481+0000","last_fullsized":"2024-09-21T06:52:59.754481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758054+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:59.758054+0000","last_peered":"2024-09-21T06:52:59.758054+0000","last_clean":"2024-09-21T06:52:59.758054+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:59.758054+0000","last_undegraded":"2024-09-21T06:52:59.758054+0000","last_fullsized":"2024-09-21T06:52:59.758054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756767+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:59.756767+0000","last_peered":"2024-09-21T06:52:59.756767+0000","last_clean":"2024-09-21T06:52:59.756767+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:59.756767+0000","last_undegraded":"2024-09-21T06:52:59.756767+0000","last_fullsized":"2024-09-21T06:52:59.756767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755238+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:59.755238+0000","last_peered":"2024-09-21T06:52:59.755238+0000","last_clean":"2024-09-21T06:52:59.755238+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:59.755238+0000","last_undegraded":"2024-09-21T06:52:59.755238+0000","last_fullsized":"2024-09-21T06:52:59.755238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752053+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:59.752053+0000","last_peered":"2024-09-21T06:52:59.752053+0000","last_clean":"2024-09-21T06:52:59.752053+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:59.752053+0000","last_undegraded":"2024-09-21T06:52:59.752053+0000","last_fullsized":"2024-09-21T06:52:59.752053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.759125+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:59.759125+0000","last_peered":"2024-09-21T06:52:59.759125+0000","last_clean":"2024-09-21T06:52:59.759125+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:59.759125+0000","last_undegraded":"2024-09-21T06:52:59.759125+0000","last_fullsized":"2024-09-21T06:52:59.759125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742201+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:59.742201+0000","last_peered":"2024-09-21T06:52:59.742201+0000","last_clean":"2024-09-21T06:52:59.742201+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:59.742201+0000","last_undegraded":"2024-09-21T06:52:59.742201+0000","last_fullsized":"2024-09-21T06:52:59.742201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755485+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:59.755485+0000","last_peered":"2024-09-21T06:52:59.755485+0000","last_clean":"2024-09-21T06:52:59.755485+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:59.755485+0000","last_undegraded":"2024-09-21T06:52:59.755485+0000","last_fullsized":"2024-09-21T06:52:59.755485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752958+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:59.752958+0000","last_peered":"2024-09-21T06:52:59.752958+0000","last_clean":"2024-09-21T06:52:59.752958+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:59.752958+0000","last_undegraded":"2024-09-21T06:52:59.752958+0000","last_fullsized":"2024-09-21T06:52:59.752958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754740+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:59.754740+0000","last_peered":"2024-09-21T06:52:59.754740+0000","last_clean":"2024-09-21T06:52:59.754740+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:59.754740+0000","last_undegraded":"2024-09-21T06:52:59.754740+0000","last_fullsized":"2024-09-21T06:52:59.754740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746351+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:59.746351+0000","last_peered":"2024-09-21T06:52:59.746351+0000","last_clean":"2024-09-21T06:52:59.746351+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:59.746351+0000","last_undegraded":"2024-09-21T06:52:59.746351+0000","last_fullsized":"2024-09-21T06:52:59.746351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749389+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:59.749389+0000","last_peered":"2024-09-21T06:52:59.749389+0000","last_clean":"2024-09-21T06:52:59.749389+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:59.749389+0000","last_undegraded":"2024-09-21T06:52:59.749389+0000","last_fullsized":"2024-09-21T06:52:59.749389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757852+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:59.757852+0000","last_peered":"2024-09-21T06:52:59.757852+0000","last_clean":"2024-09-21T06:52:59.757852+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:59.757852+0000","last_undegraded":"2024-09-21T06:52:59.757852+0000","last_fullsized":"2024-09-21T06:52:59.757852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750926+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:59.750926+0000","last_peered":"2024-09-21T06:52:59.750926+0000","last_clean":"2024-09-21T06:52:59.750926+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:59.750926+0000","last_undegraded":"2024-09-21T06:52:59.750926+0000","last_fullsized":"2024-09-21T06:52:59.750926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75956,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1321,"num_read_kb":746,"num_write":269,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":236566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":531799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906999,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1185688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070525,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":535271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266756,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":531531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462984,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1129034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626511,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":527522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1123773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149793,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":526387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378727,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":599815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542251,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":601682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705777,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:53:54.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 -- 172.21.15.114:0/498254320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac10b600 msgr2=0x7f2fac10ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 --2- 172.21.15.114:0/498254320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac10b600 0x7f2fac10ba10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009900 tx=0x7f2f9402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 -- 172.21.15.114:0/498254320 shutdown_connections 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 --2- 172.21.15.114:0/498254320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 0x7f2fac112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 --2- 172.21.15.114:0/498254320 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac10b600 0x7f2fac10ba10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 --2- 172.21.15.114:0/498254320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fac105cd0 0x7f2fac1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 -- 172.21.15.114:0/498254320 >> 172.21.15.114:0/498254320 conn(0x7f2fac1016f0 msgr2=0x7f2fac103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 -- 172.21.15.114:0/498254320 shutdown_connections 2024-09-21T06:53:54.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.974+0000 7f2fb2400640 1 -- 172.21.15.114:0/498254320 wait complete. 2024-09-21T06:53:54.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 Processor -- start 2024-09-21T06:53:54.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 -- start start 2024-09-21T06:53:54.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 0x7f2fac19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fac10b600 0x7f2fac19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 0x7f2fac1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac115c60 con 0x7f2fac105cd0 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac115ae0 con 0x7f2fac10bf50 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.975+0000 7f2fb2400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac115de0 con 0x7f2fac10b600 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 0x7f2fac19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 0x7f2fac19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47696/0 (socket says 172.21.15.114:47696) 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 -- 172.21.15.114:0/133813365 learned_addr learned my addr 172.21.15.114:0/133813365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fa2a00640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fac10b600 0x7f2fac19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fabe00640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 0x7f2fac1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 -- 172.21.15.114:0/133813365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fac10b600 msgr2=0x7f2fac19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fac10b600 0x7f2fac19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 -- 172.21.15.114:0/133813365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 msgr2=0x7f2fac1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:54.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 0x7f2fac1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 -- 172.21.15.114:0/133813365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fac1a2590 con 0x7f2fac105cd0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fabe00640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 0x7f2fac1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.976+0000 7f2fab400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 0x7f2fac19c660 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00eb90 tx=0x7f2f9c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.977+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004d20 con 0x7f2fac105cd0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.977+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fac1a2880 con 0x7f2fac105cd0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.977+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c00ee90 con 0x7f2fac105cd0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.977+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fac1a9010 con 0x7f2fac105cd0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.977+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c018690 con 0x7f2fac105cd0 2024-09-21T06:53:54.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.978+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fac106f40 con 0x7f2fac105cd0 2024-09-21T06:53:54.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.980+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f2f9c018830 con 0x7f2fac105cd0 2024-09-21T06:53:54.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.982+0000 7f2fa8c00640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2f900b6560 0x7f2f900b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:54.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.982+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2f9c0e86d0 con 0x7f2fac105cd0 2024-09-21T06:53:54.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.982+0000 7f2fa2a00640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2f900b6560 0x7f2f900b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:54.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.983+0000 7f2fa2a00640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2f900b6560 0x7f2f900b8a70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f9402a9e0 tx=0x7f2f94031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:54.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:54.984+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f9c0aead0 con 0x7f2fac105cd0 2024-09-21T06:53:55.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:54 smithi114 ceph-mon[30654]: pgmap v389: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T06:53:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:54 smithi183 ceph-mon[34219]: pgmap v389: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T06:53:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:54 smithi193 ceph-mon[34601]: pgmap v389: 129 pgs: 129 active+clean; 651 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-21T06:53:55.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.168+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fac063800 con 0x7f2f900b6560 2024-09-21T06:53:55.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.173+0000 7f2fa8c00640 1 -- 172.21.15.114:0/133813365 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340511 (secure 0 0 0) 0x7f2fac063800 con 0x7f2f900b6560 2024-09-21T06:53:55.179 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:55.181 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:53:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.178+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2f900b6560 msgr2=0x7f2f900b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.178+0000 7f2fb2400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2f900b6560 0x7f2f900b8a70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2f9402a9e0 tx=0x7f2f94031400 comp rx=0 tx=0).stop 2024-09-21T06:53:55.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 msgr2=0x7f2fac19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 0x7f2fac19c660 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00eb90 tx=0x7f2f9c00c6a0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 shutdown_connections 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2f900b6560 0x7f2f900b8a70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fac10bf50 0x7f2fac1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fac10b600 0x7f2fac19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 --2- 172.21.15.114:0/133813365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fac105cd0 0x7f2fac19c660 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 >> 172.21.15.114:0/133813365 conn(0x7f2fac1016f0 msgr2=0x7f2fac103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 shutdown_connections 2024-09-21T06:53:55.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.179+0000 7f2fb2400640 1 -- 172.21.15.114:0/133813365 wait complete. 2024-09-21T06:53:55.206 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-21T06:53:53.542113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666558,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1402,"num_read_kb":815,"num_write":394,"num_write_kb":2314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":825,"ondisk_log_size":825,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376036,"kb_used_data":14308,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565788,"statfs":{"total":1151940427776,"available":1151555366912,"internally_reserved":0,"allocated":14651392,"data_stored":8351922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":62,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003709"},"pg_stats":[{"pgid":"2.1f","version":"89'15","reported_seq":189,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.825504+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:53:50.825504+0000","last_peered":"2024-09-21T06:53:50.825504+0000","last_clean":"2024-09-21T06:53:50.825504+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:53:50.825504+0000","last_undegraded":"2024-09-21T06:53:50.825504+0000","last_fullsized":"2024-09-21T06:53:50.825504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741199+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:59.741199+0000","last_peered":"2024-09-21T06:52:59.741199+0000","last_clean":"2024-09-21T06:52:59.741199+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:59.741199+0000","last_undegraded":"2024-09-21T06:52:59.741199+0000","last_fullsized":"2024-09-21T06:52:59.741199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703671+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703671+0000","last_peered":"2024-09-21T06:52:31.703671+0000","last_clean":"2024-09-21T06:52:31.703671+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703671+0000","last_undegraded":"2024-09-21T06:52:31.703671+0000","last_fullsized":"2024-09-21T06:52:31.703671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699254+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699254+0000","last_peered":"2024-09-21T06:52:31.699254+0000","last_clean":"2024-09-21T06:52:31.699254+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699254+0000","last_undegraded":"2024-09-21T06:52:31.699254+0000","last_fullsized":"2024-09-21T06:52:31.699254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:16.554226+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:53:16.554226+0000","last_peered":"2024-09-21T06:53:16.554226+0000","last_clean":"2024-09-21T06:53:16.554226+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:53:16.554226+0000","last_undegraded":"2024-09-21T06:53:16.554226+0000","last_fullsized":"2024-09-21T06:53:16.554226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755557+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:59.755557+0000","last_peered":"2024-09-21T06:52:59.755557+0000","last_clean":"2024-09-21T06:52:59.755557+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:59.755557+0000","last_undegraded":"2024-09-21T06:52:59.755557+0000","last_fullsized":"2024-09-21T06:52:59.755557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699992+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699992+0000","last_peered":"2024-09-21T06:52:31.699992+0000","last_clean":"2024-09-21T06:52:31.699992+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699992+0000","last_undegraded":"2024-09-21T06:52:31.699992+0000","last_fullsized":"2024-09-21T06:52:31.699992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891939+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891939+0000","last_peered":"2024-09-21T06:52:31.891939+0000","last_clean":"2024-09-21T06:52:31.891939+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891939+0000","last_undegraded":"2024-09-21T06:52:31.891939+0000","last_fullsized":"2024-09-21T06:52:31.891939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751784+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:59.751784+0000","last_peered":"2024-09-21T06:52:59.751784+0000","last_clean":"2024-09-21T06:52:59.751784+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:59.751784+0000","last_undegraded":"2024-09-21T06:52:59.751784+0000","last_fullsized":"2024-09-21T06:52:59.751784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699375+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699375+0000","last_peered":"2024-09-21T06:52:31.699375+0000","last_clean":"2024-09-21T06:52:31.699375+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699375+0000","last_undegraded":"2024-09-21T06:52:31.699375+0000","last_fullsized":"2024-09-21T06:52:31.699375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755326+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:59.755326+0000","last_peered":"2024-09-21T06:52:59.755326+0000","last_clean":"2024-09-21T06:52:59.755326+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:59.755326+0000","last_undegraded":"2024-09-21T06:52:59.755326+0000","last_fullsized":"2024-09-21T06:52:59.755326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645462+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645462+0000","last_peered":"2024-09-21T06:52:31.645462+0000","last_clean":"2024-09-21T06:52:31.645462+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645462+0000","last_undegraded":"2024-09-21T06:52:31.645462+0000","last_fullsized":"2024-09-21T06:52:31.645462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755915+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:59.755915+0000","last_peered":"2024-09-21T06:52:59.755915+0000","last_clean":"2024-09-21T06:52:59.755915+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:59.755915+0000","last_undegraded":"2024-09-21T06:52:59.755915+0000","last_fullsized":"2024-09-21T06:52:59.755915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704037+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.704037+0000","last_peered":"2024-09-21T06:52:31.704037+0000","last_clean":"2024-09-21T06:52:31.704037+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.704037+0000","last_undegraded":"2024-09-21T06:52:31.704037+0000","last_fullsized":"2024-09-21T06:52:31.704037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699968+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699968+0000","last_peered":"2024-09-21T06:52:31.699968+0000","last_clean":"2024-09-21T06:52:31.699968+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699968+0000","last_undegraded":"2024-09-21T06:52:31.699968+0000","last_fullsized":"2024-09-21T06:52:31.699968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745662+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:59.745662+0000","last_peered":"2024-09-21T06:52:59.745662+0000","last_clean":"2024-09-21T06:52:59.745662+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:59.745662+0000","last_undegraded":"2024-09-21T06:52:59.745662+0000","last_fullsized":"2024-09-21T06:52:59.745662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892935+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892935+0000","last_peered":"2024-09-21T06:52:31.892935+0000","last_clean":"2024-09-21T06:52:31.892935+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892935+0000","last_undegraded":"2024-09-21T06:52:31.892935+0000","last_fullsized":"2024-09-21T06:52:31.892935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646491+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646491+0000","last_peered":"2024-09-21T06:52:31.646491+0000","last_clean":"2024-09-21T06:52:31.646491+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646491+0000","last_undegraded":"2024-09-21T06:52:31.646491+0000","last_fullsized":"2024-09-21T06:52:31.646491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744826+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:59.744826+0000","last_peered":"2024-09-21T06:52:59.744826+0000","last_clean":"2024-09-21T06:52:59.744826+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:59.744826+0000","last_undegraded":"2024-09-21T06:52:59.744826+0000","last_fullsized":"2024-09-21T06:52:59.744826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836870+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836870+0000","last_peered":"2024-09-21T06:52:31.836870+0000","last_clean":"2024-09-21T06:52:31.836870+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836870+0000","last_undegraded":"2024-09-21T06:52:31.836870+0000","last_fullsized":"2024-09-21T06:52:31.836870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755751+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:59.755751+0000","last_peered":"2024-09-21T06:52:59.755751+0000","last_clean":"2024-09-21T06:52:59.755751+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:59.755751+0000","last_undegraded":"2024-09-21T06:52:59.755751+0000","last_fullsized":"2024-09-21T06:52:59.755751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700368+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700368+0000","last_peered":"2024-09-21T06:52:31.700368+0000","last_clean":"2024-09-21T06:52:31.700368+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700368+0000","last_undegraded":"2024-09-21T06:52:31.700368+0000","last_fullsized":"2024-09-21T06:52:31.700368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703897+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703897+0000","last_peered":"2024-09-21T06:52:31.703897+0000","last_clean":"2024-09-21T06:52:31.703897+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703897+0000","last_undegraded":"2024-09-21T06:52:31.703897+0000","last_fullsized":"2024-09-21T06:52:31.703897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741893+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:59.741893+0000","last_peered":"2024-09-21T06:52:59.741893+0000","last_clean":"2024-09-21T06:52:59.741893+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:59.741893+0000","last_undegraded":"2024-09-21T06:52:59.741893+0000","last_fullsized":"2024-09-21T06:52:59.741893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:00.244438+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:53:00.244438+0000","last_peered":"2024-09-21T06:53:00.244438+0000","last_clean":"2024-09-21T06:53:00.244438+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:53:00.244438+0000","last_undegraded":"2024-09-21T06:53:00.244438+0000","last_fullsized":"2024-09-21T06:53:00.244438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891910+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891910+0000","last_peered":"2024-09-21T06:52:31.891910+0000","last_clean":"2024-09-21T06:52:31.891910+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891910+0000","last_undegraded":"2024-09-21T06:52:31.891910+0000","last_fullsized":"2024-09-21T06:52:31.891910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742064+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:59.742064+0000","last_peered":"2024-09-21T06:52:59.742064+0000","last_clean":"2024-09-21T06:52:59.742064+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:59.742064+0000","last_undegraded":"2024-09-21T06:52:59.742064+0000","last_fullsized":"2024-09-21T06:52:59.742064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892041+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892041+0000","last_peered":"2024-09-21T06:52:31.892041+0000","last_clean":"2024-09-21T06:52:31.892041+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892041+0000","last_undegraded":"2024-09-21T06:52:31.892041+0000","last_fullsized":"2024-09-21T06:52:31.892041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645535+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645535+0000","last_peered":"2024-09-21T06:52:31.645535+0000","last_clean":"2024-09-21T06:52:31.645535+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645535+0000","last_undegraded":"2024-09-21T06:52:31.645535+0000","last_fullsized":"2024-09-21T06:52:31.645535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754090+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:59.754090+0000","last_peered":"2024-09-21T06:52:59.754090+0000","last_clean":"2024-09-21T06:52:59.754090+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:59.754090+0000","last_undegraded":"2024-09-21T06:52:59.754090+0000","last_fullsized":"2024-09-21T06:52:59.754090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645977+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645977+0000","last_peered":"2024-09-21T06:52:31.645977+0000","last_clean":"2024-09-21T06:52:31.645977+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645977+0000","last_undegraded":"2024-09-21T06:52:31.645977+0000","last_fullsized":"2024-09-21T06:52:31.645977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891517+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891517+0000","last_peered":"2024-09-21T06:52:31.891517+0000","last_clean":"2024-09-21T06:52:31.891517+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891517+0000","last_undegraded":"2024-09-21T06:52:31.891517+0000","last_fullsized":"2024-09-21T06:52:31.891517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758656+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:59.758656+0000","last_peered":"2024-09-21T06:52:59.758656+0000","last_clean":"2024-09-21T06:52:59.758656+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:59.758656+0000","last_undegraded":"2024-09-21T06:52:59.758656+0000","last_fullsized":"2024-09-21T06:52:59.758656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'8","reported_seq":182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:48.822552+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:53:48.822552+0000","last_peered":"2024-09-21T06:53:48.822552+0000","last_clean":"2024-09-21T06:53:48.822552+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:53:48.822552+0000","last_undegraded":"2024-09-21T06:53:48.822552+0000","last_fullsized":"2024-09-21T06:53:48.822552+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1431,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701262+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701262+0000","last_peered":"2024-09-21T06:52:31.701262+0000","last_clean":"2024-09-21T06:52:31.701262+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701262+0000","last_undegraded":"2024-09-21T06:52:31.701262+0000","last_fullsized":"2024-09-21T06:52:31.701262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749255+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:59.749255+0000","last_peered":"2024-09-21T06:52:59.749255+0000","last_clean":"2024-09-21T06:52:59.749255+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:59.749255+0000","last_undegraded":"2024-09-21T06:52:59.749255+0000","last_fullsized":"2024-09-21T06:52:59.749255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.825547+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:53:50.825547+0000","last_peered":"2024-09-21T06:53:50.825547+0000","last_clean":"2024-09-21T06:53:50.825547+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:53:50.825547+0000","last_undegraded":"2024-09-21T06:53:50.825547+0000","last_fullsized":"2024-09-21T06:53:50.825547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701440+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701440+0000","last_peered":"2024-09-21T06:52:31.701440+0000","last_clean":"2024-09-21T06:52:31.701440+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701440+0000","last_undegraded":"2024-09-21T06:52:31.701440+0000","last_fullsized":"2024-09-21T06:52:31.701440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740983+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:59.740983+0000","last_peered":"2024-09-21T06:52:59.740983+0000","last_clean":"2024-09-21T06:52:59.740983+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:59.740983+0000","last_undegraded":"2024-09-21T06:52:59.740983+0000","last_fullsized":"2024-09-21T06:52:59.740983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701878+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701878+0000","last_peered":"2024-09-21T06:52:31.701878+0000","last_clean":"2024-09-21T06:52:31.701878+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701878+0000","last_undegraded":"2024-09-21T06:52:31.701878+0000","last_fullsized":"2024-09-21T06:52:31.701878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749354+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:59.749354+0000","last_peered":"2024-09-21T06:52:59.749354+0000","last_clean":"2024-09-21T06:52:59.749354+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:59.749354+0000","last_undegraded":"2024-09-21T06:52:59.749354+0000","last_fullsized":"2024-09-21T06:52:59.749354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648426+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648426+0000","last_peered":"2024-09-21T06:52:31.648426+0000","last_clean":"2024-09-21T06:52:31.648426+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648426+0000","last_undegraded":"2024-09-21T06:52:31.648426+0000","last_fullsized":"2024-09-21T06:52:31.648426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648721+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648721+0000","last_peered":"2024-09-21T06:52:31.648721+0000","last_clean":"2024-09-21T06:52:31.648721+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648721+0000","last_undegraded":"2024-09-21T06:52:31.648721+0000","last_fullsized":"2024-09-21T06:52:31.648721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753632+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:59.753632+0000","last_peered":"2024-09-21T06:52:59.753632+0000","last_clean":"2024-09-21T06:52:59.753632+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:59.753632+0000","last_undegraded":"2024-09-21T06:52:59.753632+0000","last_fullsized":"2024-09-21T06:52:59.753632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":183,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.826734+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:53:50.826734+0000","last_peered":"2024-09-21T06:53:50.826734+0000","last_clean":"2024-09-21T06:53:50.826734+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:53:50.826734+0000","last_undegraded":"2024-09-21T06:53:50.826734+0000","last_fullsized":"2024-09-21T06:53:50.826734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836829+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836829+0000","last_peered":"2024-09-21T06:52:31.836829+0000","last_clean":"2024-09-21T06:52:31.836829+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836829+0000","last_undegraded":"2024-09-21T06:52:31.836829+0000","last_fullsized":"2024-09-21T06:52:31.836829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744841+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:59.744841+0000","last_peered":"2024-09-21T06:52:59.744841+0000","last_clean":"2024-09-21T06:52:59.744841+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:59.744841+0000","last_undegraded":"2024-09-21T06:52:59.744841+0000","last_fullsized":"2024-09-21T06:52:59.744841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702337+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:52:31.702337+0000","last_peered":"2024-09-21T06:52:31.702337+0000","last_clean":"2024-09-21T06:52:31.702337+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:52:31.702337+0000","last_undegraded":"2024-09-21T06:52:31.702337+0000","last_fullsized":"2024-09-21T06:52:31.702337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891861+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891861+0000","last_peered":"2024-09-21T06:52:31.891861+0000","last_clean":"2024-09-21T06:52:31.891861+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891861+0000","last_undegraded":"2024-09-21T06:52:31.891861+0000","last_fullsized":"2024-09-21T06:52:31.891861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753918+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:59.753918+0000","last_peered":"2024-09-21T06:52:59.753918+0000","last_clean":"2024-09-21T06:52:59.753918+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:59.753918+0000","last_undegraded":"2024-09-21T06:52:59.753918+0000","last_fullsized":"2024-09-21T06:52:59.753918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701640+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701640+0000","last_peered":"2024-09-21T06:52:31.701640+0000","last_clean":"2024-09-21T06:52:31.701640+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701640+0000","last_undegraded":"2024-09-21T06:52:31.701640+0000","last_fullsized":"2024-09-21T06:52:31.701640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701328+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701328+0000","last_peered":"2024-09-21T06:52:31.701328+0000","last_clean":"2024-09-21T06:52:31.701328+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701328+0000","last_undegraded":"2024-09-21T06:52:31.701328+0000","last_fullsized":"2024-09-21T06:52:31.701328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734233+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:59.734233+0000","last_peered":"2024-09-21T06:52:59.734233+0000","last_clean":"2024-09-21T06:52:59.734233+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:59.734233+0000","last_undegraded":"2024-09-21T06:52:59.734233+0000","last_fullsized":"2024-09-21T06:52:59.734233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645941+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645941+0000","last_peered":"2024-09-21T06:52:31.645941+0000","last_clean":"2024-09-21T06:52:31.645941+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645941+0000","last_undegraded":"2024-09-21T06:52:31.645941+0000","last_fullsized":"2024-09-21T06:52:31.645941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701203+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701203+0000","last_peered":"2024-09-21T06:52:31.701203+0000","last_clean":"2024-09-21T06:52:31.701203+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701203+0000","last_undegraded":"2024-09-21T06:52:31.701203+0000","last_fullsized":"2024-09-21T06:52:31.701203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.748492+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:59.748492+0000","last_peered":"2024-09-21T06:52:59.748492+0000","last_clean":"2024-09-21T06:52:59.748492+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:59.748492+0000","last_undegraded":"2024-09-21T06:52:59.748492+0000","last_fullsized":"2024-09-21T06:52:59.748492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891566+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891566+0000","last_peered":"2024-09-21T06:52:31.891566+0000","last_clean":"2024-09-21T06:52:31.891566+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891566+0000","last_undegraded":"2024-09-21T06:52:31.891566+0000","last_fullsized":"2024-09-21T06:52:31.891566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701524+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701524+0000","last_peered":"2024-09-21T06:52:31.701524+0000","last_clean":"2024-09-21T06:52:31.701524+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701524+0000","last_undegraded":"2024-09-21T06:52:31.701524+0000","last_fullsized":"2024-09-21T06:52:31.701524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756438+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:59.756438+0000","last_peered":"2024-09-21T06:52:59.756438+0000","last_clean":"2024-09-21T06:52:59.756438+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:59.756438+0000","last_undegraded":"2024-09-21T06:52:59.756438+0000","last_fullsized":"2024-09-21T06:52:59.756438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:49.824645+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:53:49.824645+0000","last_peered":"2024-09-21T06:53:49.824645+0000","last_clean":"2024-09-21T06:53:49.824645+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:53:49.824645+0000","last_undegraded":"2024-09-21T06:53:49.824645+0000","last_fullsized":"2024-09-21T06:53:49.824645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891921+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891921+0000","last_peered":"2024-09-21T06:52:31.891921+0000","last_clean":"2024-09-21T06:52:31.891921+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891921+0000","last_undegraded":"2024-09-21T06:52:31.891921+0000","last_fullsized":"2024-09-21T06:52:31.891921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758614+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:59.758614+0000","last_peered":"2024-09-21T06:52:59.758614+0000","last_clean":"2024-09-21T06:52:59.758614+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:59.758614+0000","last_undegraded":"2024-09-21T06:52:59.758614+0000","last_fullsized":"2024-09-21T06:52:59.758614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701396+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701396+0000","last_peered":"2024-09-21T06:52:31.701396+0000","last_clean":"2024-09-21T06:52:31.701396+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701396+0000","last_undegraded":"2024-09-21T06:52:31.701396+0000","last_fullsized":"2024-09-21T06:52:31.701396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645499+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645499+0000","last_peered":"2024-09-21T06:52:31.645499+0000","last_clean":"2024-09-21T06:52:31.645499+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645499+0000","last_undegraded":"2024-09-21T06:52:31.645499+0000","last_fullsized":"2024-09-21T06:52:31.645499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749562+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:59.749562+0000","last_peered":"2024-09-21T06:52:59.749562+0000","last_clean":"2024-09-21T06:52:59.749562+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:59.749562+0000","last_undegraded":"2024-09-21T06:52:59.749562+0000","last_fullsized":"2024-09-21T06:52:59.749562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703886+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703886+0000","last_peered":"2024-09-21T06:52:31.703886+0000","last_clean":"2024-09-21T06:52:31.703886+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703886+0000","last_undegraded":"2024-09-21T06:52:31.703886+0000","last_fullsized":"2024-09-21T06:52:31.703886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702060+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.702060+0000","last_peered":"2024-09-21T06:52:31.702060+0000","last_clean":"2024-09-21T06:52:31.702060+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.702060+0000","last_undegraded":"2024-09-21T06:52:31.702060+0000","last_fullsized":"2024-09-21T06:52:31.702060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745772+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:59.745772+0000","last_peered":"2024-09-21T06:52:59.745772+0000","last_clean":"2024-09-21T06:52:59.745772+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:59.745772+0000","last_undegraded":"2024-09-21T06:52:59.745772+0000","last_fullsized":"2024-09-21T06:52:59.745772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:48.824553+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:53:48.824553+0000","last_peered":"2024-09-21T06:53:48.824553+0000","last_clean":"2024-09-21T06:53:48.824553+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:53:48.824553+0000","last_undegraded":"2024-09-21T06:53:48.824553+0000","last_fullsized":"2024-09-21T06:53:48.824553+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67627,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":263,"num_read_kb":199,"num_write":42,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645656+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645656+0000","last_peered":"2024-09-21T06:52:31.645656+0000","last_clean":"2024-09-21T06:52:31.645656+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645656+0000","last_undegraded":"2024-09-21T06:52:31.645656+0000","last_fullsized":"2024-09-21T06:52:31.645656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.747527+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:59.747527+0000","last_peered":"2024-09-21T06:52:59.747527+0000","last_clean":"2024-09-21T06:52:59.747527+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:59.747527+0000","last_undegraded":"2024-09-21T06:52:59.747527+0000","last_fullsized":"2024-09-21T06:52:59.747527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702309+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702309+0000","last_peered":"2024-09-21T06:52:31.702309+0000","last_clean":"2024-09-21T06:52:31.702309+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702309+0000","last_undegraded":"2024-09-21T06:52:31.702309+0000","last_fullsized":"2024-09-21T06:52:31.702309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699508+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699508+0000","last_peered":"2024-09-21T06:52:31.699508+0000","last_clean":"2024-09-21T06:52:31.699508+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699508+0000","last_undegraded":"2024-09-21T06:52:31.699508+0000","last_fullsized":"2024-09-21T06:52:31.699508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757106+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:59.757106+0000","last_peered":"2024-09-21T06:52:59.757106+0000","last_clean":"2024-09-21T06:52:59.757106+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:59.757106+0000","last_undegraded":"2024-09-21T06:52:59.757106+0000","last_fullsized":"2024-09-21T06:52:59.757106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648918+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648918+0000","last_peered":"2024-09-21T06:52:31.648918+0000","last_clean":"2024-09-21T06:52:31.648918+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648918+0000","last_undegraded":"2024-09-21T06:52:31.648918+0000","last_fullsized":"2024-09-21T06:52:31.648918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645559+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645559+0000","last_peered":"2024-09-21T06:52:31.645559+0000","last_clean":"2024-09-21T06:52:31.645559+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645559+0000","last_undegraded":"2024-09-21T06:52:31.645559+0000","last_fullsized":"2024-09-21T06:52:31.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750255+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:59.750255+0000","last_peered":"2024-09-21T06:52:59.750255+0000","last_clean":"2024-09-21T06:52:59.750255+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:59.750255+0000","last_undegraded":"2024-09-21T06:52:59.750255+0000","last_fullsized":"2024-09-21T06:52:59.750255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":301,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:48.822735+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:53:48.822735+0000","last_peered":"2024-09-21T06:53:48.822735+0000","last_clean":"2024-09-21T06:53:48.822735+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:53:48.822735+0000","last_undegraded":"2024-09-21T06:53:48.822735+0000","last_fullsized":"2024-09-21T06:53:48.822735+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":252,"num_read_kb":252,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892718+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892718+0000","last_peered":"2024-09-21T06:52:31.892718+0000","last_clean":"2024-09-21T06:52:31.892718+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892718+0000","last_undegraded":"2024-09-21T06:52:31.892718+0000","last_fullsized":"2024-09-21T06:52:31.892718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892186+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892186+0000","last_peered":"2024-09-21T06:52:31.892186+0000","last_clean":"2024-09-21T06:52:31.892186+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892186+0000","last_undegraded":"2024-09-21T06:52:31.892186+0000","last_fullsized":"2024-09-21T06:52:31.892186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741381+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:59.741381+0000","last_peered":"2024-09-21T06:52:59.741381+0000","last_clean":"2024-09-21T06:52:59.741381+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:59.741381+0000","last_undegraded":"2024-09-21T06:52:59.741381+0000","last_fullsized":"2024-09-21T06:52:59.741381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701609+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701609+0000","last_peered":"2024-09-21T06:52:31.701609+0000","last_clean":"2024-09-21T06:52:31.701609+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701609+0000","last_undegraded":"2024-09-21T06:52:31.701609+0000","last_fullsized":"2024-09-21T06:52:31.701609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645808+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645808+0000","last_peered":"2024-09-21T06:52:31.645808+0000","last_clean":"2024-09-21T06:52:31.645808+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645808+0000","last_undegraded":"2024-09-21T06:52:31.645808+0000","last_fullsized":"2024-09-21T06:52:31.645808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742667+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:59.742667+0000","last_peered":"2024-09-21T06:52:59.742667+0000","last_clean":"2024-09-21T06:52:59.742667+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:59.742667+0000","last_undegraded":"2024-09-21T06:52:59.742667+0000","last_fullsized":"2024-09-21T06:52:59.742667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700994+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700994+0000","last_peered":"2024-09-21T06:52:31.700994+0000","last_clean":"2024-09-21T06:52:31.700994+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700994+0000","last_undegraded":"2024-09-21T06:52:31.700994+0000","last_fullsized":"2024-09-21T06:52:31.700994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645981+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645981+0000","last_peered":"2024-09-21T06:52:31.645981+0000","last_clean":"2024-09-21T06:52:31.645981+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645981+0000","last_undegraded":"2024-09-21T06:52:31.645981+0000","last_fullsized":"2024-09-21T06:52:31.645981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749130+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:59.749130+0000","last_peered":"2024-09-21T06:52:59.749130+0000","last_clean":"2024-09-21T06:52:59.749130+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:59.749130+0000","last_undegraded":"2024-09-21T06:52:59.749130+0000","last_fullsized":"2024-09-21T06:52:59.749130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703865+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703865+0000","last_peered":"2024-09-21T06:52:31.703865+0000","last_clean":"2024-09-21T06:52:31.703865+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703865+0000","last_undegraded":"2024-09-21T06:52:31.703865+0000","last_fullsized":"2024-09-21T06:52:31.703865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:49.825569+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:53:49.825569+0000","last_peered":"2024-09-21T06:53:49.825569+0000","last_clean":"2024-09-21T06:53:49.825569+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:53:49.825569+0000","last_undegraded":"2024-09-21T06:53:49.825569+0000","last_fullsized":"2024-09-21T06:53:49.825569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.736418+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:59.736418+0000","last_peered":"2024-09-21T06:52:59.736418+0000","last_clean":"2024-09-21T06:52:59.736418+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:59.736418+0000","last_undegraded":"2024-09-21T06:52:59.736418+0000","last_fullsized":"2024-09-21T06:52:59.736418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704051+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.704051+0000","last_peered":"2024-09-21T06:52:31.704051+0000","last_clean":"2024-09-21T06:52:31.704051+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.704051+0000","last_undegraded":"2024-09-21T06:52:31.704051+0000","last_fullsized":"2024-09-21T06:52:31.704051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.739975+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:59.739975+0000","last_peered":"2024-09-21T06:52:59.739975+0000","last_clean":"2024-09-21T06:52:59.739975+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:59.739975+0000","last_undegraded":"2024-09-21T06:52:59.739975+0000","last_fullsized":"2024-09-21T06:52:59.739975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749347+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:59.749347+0000","last_peered":"2024-09-21T06:52:59.749347+0000","last_clean":"2024-09-21T06:52:59.749347+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:59.749347+0000","last_undegraded":"2024-09-21T06:52:59.749347+0000","last_fullsized":"2024-09-21T06:52:59.749347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746716+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:59.746716+0000","last_peered":"2024-09-21T06:52:59.746716+0000","last_clean":"2024-09-21T06:52:59.746716+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:59.746716+0000","last_undegraded":"2024-09-21T06:52:59.746716+0000","last_fullsized":"2024-09-21T06:52:59.746716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757121+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:59.757121+0000","last_peered":"2024-09-21T06:52:59.757121+0000","last_clean":"2024-09-21T06:52:59.757121+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:59.757121+0000","last_undegraded":"2024-09-21T06:52:59.757121+0000","last_fullsized":"2024-09-21T06:52:59.757121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756339+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:59.756339+0000","last_peered":"2024-09-21T06:52:59.756339+0000","last_clean":"2024-09-21T06:52:59.756339+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:59.756339+0000","last_undegraded":"2024-09-21T06:52:59.756339+0000","last_fullsized":"2024-09-21T06:52:59.756339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734085+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:59.734085+0000","last_peered":"2024-09-21T06:52:59.734085+0000","last_clean":"2024-09-21T06:52:59.734085+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:59.734085+0000","last_undegraded":"2024-09-21T06:52:59.734085+0000","last_fullsized":"2024-09-21T06:52:59.734085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749595+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:59.749595+0000","last_peered":"2024-09-21T06:52:59.749595+0000","last_clean":"2024-09-21T06:52:59.749595+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:59.749595+0000","last_undegraded":"2024-09-21T06:52:59.749595+0000","last_fullsized":"2024-09-21T06:52:59.749595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.737176+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:59.737176+0000","last_peered":"2024-09-21T06:52:59.737176+0000","last_clean":"2024-09-21T06:52:59.737176+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:59.737176+0000","last_undegraded":"2024-09-21T06:52:59.737176+0000","last_fullsized":"2024-09-21T06:52:59.737176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746235+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:59.746235+0000","last_peered":"2024-09-21T06:52:59.746235+0000","last_clean":"2024-09-21T06:52:59.746235+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:59.746235+0000","last_undegraded":"2024-09-21T06:52:59.746235+0000","last_fullsized":"2024-09-21T06:52:59.746235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740649+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:59.740649+0000","last_peered":"2024-09-21T06:52:59.740649+0000","last_clean":"2024-09-21T06:52:59.740649+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:59.740649+0000","last_undegraded":"2024-09-21T06:52:59.740649+0000","last_fullsized":"2024-09-21T06:52:59.740649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753696+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:59.753696+0000","last_peered":"2024-09-21T06:52:59.753696+0000","last_clean":"2024-09-21T06:52:59.753696+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:59.753696+0000","last_undegraded":"2024-09-21T06:52:59.753696+0000","last_fullsized":"2024-09-21T06:52:59.753696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756495+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:59.756495+0000","last_peered":"2024-09-21T06:52:59.756495+0000","last_clean":"2024-09-21T06:52:59.756495+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:59.756495+0000","last_undegraded":"2024-09-21T06:52:59.756495+0000","last_fullsized":"2024-09-21T06:52:59.756495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734175+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:59.734175+0000","last_peered":"2024-09-21T06:52:59.734175+0000","last_clean":"2024-09-21T06:52:59.734175+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:59.734175+0000","last_undegraded":"2024-09-21T06:52:59.734175+0000","last_fullsized":"2024-09-21T06:52:59.734175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745394+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:59.745394+0000","last_peered":"2024-09-21T06:52:59.745394+0000","last_clean":"2024-09-21T06:52:59.745394+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:59.745394+0000","last_undegraded":"2024-09-21T06:52:59.745394+0000","last_fullsized":"2024-09-21T06:52:59.745394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751016+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:59.751016+0000","last_peered":"2024-09-21T06:52:59.751016+0000","last_clean":"2024-09-21T06:52:59.751016+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:59.751016+0000","last_undegraded":"2024-09-21T06:52:59.751016+0000","last_fullsized":"2024-09-21T06:52:59.751016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755908+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:59.755908+0000","last_peered":"2024-09-21T06:52:59.755908+0000","last_clean":"2024-09-21T06:52:59.755908+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:59.755908+0000","last_undegraded":"2024-09-21T06:52:59.755908+0000","last_fullsized":"2024-09-21T06:52:59.755908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752146+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:59.752146+0000","last_peered":"2024-09-21T06:52:59.752146+0000","last_clean":"2024-09-21T06:52:59.752146+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:59.752146+0000","last_undegraded":"2024-09-21T06:52:59.752146+0000","last_fullsized":"2024-09-21T06:52:59.752146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754481+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:59.754481+0000","last_peered":"2024-09-21T06:52:59.754481+0000","last_clean":"2024-09-21T06:52:59.754481+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:59.754481+0000","last_undegraded":"2024-09-21T06:52:59.754481+0000","last_fullsized":"2024-09-21T06:52:59.754481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758054+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:59.758054+0000","last_peered":"2024-09-21T06:52:59.758054+0000","last_clean":"2024-09-21T06:52:59.758054+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:59.758054+0000","last_undegraded":"2024-09-21T06:52:59.758054+0000","last_fullsized":"2024-09-21T06:52:59.758054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756767+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:59.756767+0000","last_peered":"2024-09-21T06:52:59.756767+0000","last_clean":"2024-09-21T06:52:59.756767+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:59.756767+0000","last_undegraded":"2024-09-21T06:52:59.756767+0000","last_fullsized":"2024-09-21T06:52:59.756767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755238+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:59.755238+0000","last_peered":"2024-09-21T06:52:59.755238+0000","last_clean":"2024-09-21T06:52:59.755238+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:59.755238+0000","last_undegraded":"2024-09-21T06:52:59.755238+0000","last_fullsized":"2024-09-21T06:52:59.755238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752053+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:59.752053+0000","last_peered":"2024-09-21T06:52:59.752053+0000","last_clean":"2024-09-21T06:52:59.752053+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:59.752053+0000","last_undegraded":"2024-09-21T06:52:59.752053+0000","last_fullsized":"2024-09-21T06:52:59.752053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.759125+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:59.759125+0000","last_peered":"2024-09-21T06:52:59.759125+0000","last_clean":"2024-09-21T06:52:59.759125+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:59.759125+0000","last_undegraded":"2024-09-21T06:52:59.759125+0000","last_fullsized":"2024-09-21T06:52:59.759125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742201+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:59.742201+0000","last_peered":"2024-09-21T06:52:59.742201+0000","last_clean":"2024-09-21T06:52:59.742201+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:59.742201+0000","last_undegraded":"2024-09-21T06:52:59.742201+0000","last_fullsized":"2024-09-21T06:52:59.742201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755485+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:59.755485+0000","last_peered":"2024-09-21T06:52:59.755485+0000","last_clean":"2024-09-21T06:52:59.755485+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:59.755485+0000","last_undegraded":"2024-09-21T06:52:59.755485+0000","last_fullsized":"2024-09-21T06:52:59.755485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752958+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:59.752958+0000","last_peered":"2024-09-21T06:52:59.752958+0000","last_clean":"2024-09-21T06:52:59.752958+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:59.752958+0000","last_undegraded":"2024-09-21T06:52:59.752958+0000","last_fullsized":"2024-09-21T06:52:59.752958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754740+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:59.754740+0000","last_peered":"2024-09-21T06:52:59.754740+0000","last_clean":"2024-09-21T06:52:59.754740+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:59.754740+0000","last_undegraded":"2024-09-21T06:52:59.754740+0000","last_fullsized":"2024-09-21T06:52:59.754740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746351+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:59.746351+0000","last_peered":"2024-09-21T06:52:59.746351+0000","last_clean":"2024-09-21T06:52:59.746351+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:59.746351+0000","last_undegraded":"2024-09-21T06:52:59.746351+0000","last_fullsized":"2024-09-21T06:52:59.746351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749389+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:59.749389+0000","last_peered":"2024-09-21T06:52:59.749389+0000","last_clean":"2024-09-21T06:52:59.749389+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:59.749389+0000","last_undegraded":"2024-09-21T06:52:59.749389+0000","last_fullsized":"2024-09-21T06:52:59.749389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757852+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:59.757852+0000","last_peered":"2024-09-21T06:52:59.757852+0000","last_clean":"2024-09-21T06:52:59.757852+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:59.757852+0000","last_undegraded":"2024-09-21T06:52:59.757852+0000","last_fullsized":"2024-09-21T06:52:59.757852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750926+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:59.750926+0000","last_peered":"2024-09-21T06:52:59.750926+0000","last_clean":"2024-09-21T06:52:59.750926+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:59.750926+0000","last_undegraded":"2024-09-21T06:52:59.750926+0000","last_fullsized":"2024-09-21T06:52:59.750926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75956,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1321,"num_read_kb":746,"num_write":269,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":598016,"data_stored":236566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":531799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906999,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28720,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716432,"statfs":{"total":95995035648,"available":95965626368,"internally_reserved":0,"allocated":1736704,"data_stored":1185688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070525,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":535271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266756,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":531531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462984,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1129034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626511,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":527522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1123773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149793,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":526387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378727,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":599815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542251,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":601682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705777,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:53:55.211 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T06:53:55.212 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T06:53:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.471+0000 7fd5db000640 1 -- 172.21.15.114:0/1707436232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 msgr2=0x7fd5d4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.471+0000 7fd5db000640 1 --2- 172.21.15.114:0/1707436232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4103d00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009900 tx=0x7fd5c802a410 comp rx=0 tx=0).stop 2024-09-21T06:53:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 -- 172.21.15.114:0/1707436232 shutdown_connections 2024-09-21T06:53:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 --2- 172.21.15.114:0/1707436232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5d4109b70 0x7fd5d41103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 --2- 172.21.15.114:0/1707436232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5d4109220 0x7fd5d4109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 --2- 172.21.15.114:0/1707436232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4103d00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 -- 172.21.15.114:0/1707436232 >> 172.21.15.114:0/1707436232 conn(0x7fd5d40ff350 msgr2=0x7fd5d41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 -- 172.21.15.114:0/1707436232 shutdown_connections 2024-09-21T06:53:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.472+0000 7fd5db000640 1 -- 172.21.15.114:0/1707436232 wait complete. 2024-09-21T06:53:55.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 Processor -- start 2024-09-21T06:53:55.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 -- start start 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5d4109220 0x7fd5d4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5d4109b70 0x7fd5d4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4113830 con 0x7fd5d41038f0 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41136b0 con 0x7fd5d4109b70 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5db000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d41139b0 con 0x7fd5d4109220 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.473+0000 7fd5d9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47708/0 (socket says 172.21.15.114:47708) 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 -- 172.21.15.114:0/4126051219 learned_addr learned my addr 172.21.15.114:0/4126051219 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9200640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5d4109220 0x7fd5d4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5da600640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5d4109b70 0x7fd5d4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 -- 172.21.15.114:0/4126051219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5d4109220 msgr2=0x7fd5d4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5d4109220 0x7fd5d4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 -- 172.21.15.114:0/4126051219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5d4109b70 msgr2=0x7fd5d4076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5d4109b70 0x7fd5d4076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 -- 172.21.15.114:0/4126051219 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d41a8540 con 0x7fd5d41038f0 2024-09-21T06:53:55.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.474+0000 7fd5d9c00640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4075710 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005940 tx=0x7fd5c80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:55.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.475+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c803f030 con 0x7fd5d41038f0 2024-09-21T06:53:55.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.475+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c80044c0 con 0x7fd5d41038f0 2024-09-21T06:53:55.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.475+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41a87d0 con 0x7fd5d41038f0 2024-09-21T06:53:55.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.475+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8035070 con 0x7fd5d41038f0 2024-09-21T06:53:55.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.475+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a8cc0 con 0x7fd5d41038f0 2024-09-21T06:53:55.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.476+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d4103d00 con 0x7fd5d41038f0 2024-09-21T06:53:55.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.478+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fd5c8042030 con 0x7fd5d41038f0 2024-09-21T06:53:55.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.479+0000 7fd5d2000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5bc0b6590 0x7fd5bc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:55.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.480+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd5c8107d70 con 0x7fd5d41038f0 2024-09-21T06:53:55.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.480+0000 7fd5d9200640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5bc0b6590 0x7fd5bc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:55.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.480+0000 7fd5d9200640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5bc0b6590 0x7fd5bc0b8aa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4108d80 tx=0x7fd5c0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:55.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.486+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5c80d2550 con 0x7fd5d41038f0 2024-09-21T06:53:55.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.680+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5d4063800 con 0x7fd5bc0b6590 2024-09-21T06:53:55.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.687+0000 7fd5d2000640 1 -- 172.21.15.114:0/4126051219 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340513 (secure 0 0 0) 0x7fd5d4063800 con 0x7fd5bc0b6590 2024-09-21T06:53:55.693 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:55.694 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.691+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5bc0b6590 msgr2=0x7fd5bc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.691+0000 7fd5db000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5bc0b6590 0x7fd5bc0b8aa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4108d80 tx=0x7fd5c0005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.691+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 msgr2=0x7fd5d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.691+0000 7fd5db000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4075710 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005940 tx=0x7fd5c80040c0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 shutdown_connections 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5bc0b6590 0x7fd5bc0b8aa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5d4109b70 0x7fd5d4076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5d4109220 0x7fd5d4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 --2- 172.21.15.114:0/4126051219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5d41038f0 0x7fd5d4075710 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 >> 172.21.15.114:0/4126051219 conn(0x7fd5d40ff350 msgr2=0x7fd5d4101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:55.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 shutdown_connections 2024-09-21T06:53:55.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:55.692+0000 7fd5db000640 1 -- 172.21.15.114:0/4126051219 wait complete. 2024-09-21T06:53:55.719 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-09-21T06:53:55.542755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670907,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1428,"num_read_kb":835,"num_write":395,"num_write_kb":2319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":826,"ondisk_log_size":826,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376040,"kb_used_data":14312,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565784,"statfs":{"total":1151940427776,"available":1151555362816,"internally_reserved":0,"allocated":14655488,"data_stored":8356271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4971,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":72,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.1f","version":"89'15","reported_seq":189,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.825504+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:53:50.825504+0000","last_peered":"2024-09-21T06:53:50.825504+0000","last_clean":"2024-09-21T06:53:50.825504+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:53:50.825504+0000","last_undegraded":"2024-09-21T06:53:50.825504+0000","last_fullsized":"2024-09-21T06:53:50.825504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741199+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:59.741199+0000","last_peered":"2024-09-21T06:52:59.741199+0000","last_clean":"2024-09-21T06:52:59.741199+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:59.741199+0000","last_undegraded":"2024-09-21T06:52:59.741199+0000","last_fullsized":"2024-09-21T06:52:59.741199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703671+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703671+0000","last_peered":"2024-09-21T06:52:31.703671+0000","last_clean":"2024-09-21T06:52:31.703671+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703671+0000","last_undegraded":"2024-09-21T06:52:31.703671+0000","last_fullsized":"2024-09-21T06:52:31.703671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699254+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699254+0000","last_peered":"2024-09-21T06:52:31.699254+0000","last_clean":"2024-09-21T06:52:31.699254+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699254+0000","last_undegraded":"2024-09-21T06:52:31.699254+0000","last_fullsized":"2024-09-21T06:52:31.699254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:16.554226+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:53:16.554226+0000","last_peered":"2024-09-21T06:53:16.554226+0000","last_clean":"2024-09-21T06:53:16.554226+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:53:16.554226+0000","last_undegraded":"2024-09-21T06:53:16.554226+0000","last_fullsized":"2024-09-21T06:53:16.554226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755557+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:59.755557+0000","last_peered":"2024-09-21T06:52:59.755557+0000","last_clean":"2024-09-21T06:52:59.755557+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:59.755557+0000","last_undegraded":"2024-09-21T06:52:59.755557+0000","last_fullsized":"2024-09-21T06:52:59.755557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699992+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699992+0000","last_peered":"2024-09-21T06:52:31.699992+0000","last_clean":"2024-09-21T06:52:31.699992+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699992+0000","last_undegraded":"2024-09-21T06:52:31.699992+0000","last_fullsized":"2024-09-21T06:52:31.699992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891939+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891939+0000","last_peered":"2024-09-21T06:52:31.891939+0000","last_clean":"2024-09-21T06:52:31.891939+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891939+0000","last_undegraded":"2024-09-21T06:52:31.891939+0000","last_fullsized":"2024-09-21T06:52:31.891939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751784+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:59.751784+0000","last_peered":"2024-09-21T06:52:59.751784+0000","last_clean":"2024-09-21T06:52:59.751784+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:59.751784+0000","last_undegraded":"2024-09-21T06:52:59.751784+0000","last_fullsized":"2024-09-21T06:52:59.751784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699375+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699375+0000","last_peered":"2024-09-21T06:52:31.699375+0000","last_clean":"2024-09-21T06:52:31.699375+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699375+0000","last_undegraded":"2024-09-21T06:52:31.699375+0000","last_fullsized":"2024-09-21T06:52:31.699375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755326+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:59.755326+0000","last_peered":"2024-09-21T06:52:59.755326+0000","last_clean":"2024-09-21T06:52:59.755326+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:59.755326+0000","last_undegraded":"2024-09-21T06:52:59.755326+0000","last_fullsized":"2024-09-21T06:52:59.755326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645462+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645462+0000","last_peered":"2024-09-21T06:52:31.645462+0000","last_clean":"2024-09-21T06:52:31.645462+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645462+0000","last_undegraded":"2024-09-21T06:52:31.645462+0000","last_fullsized":"2024-09-21T06:52:31.645462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755915+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:59.755915+0000","last_peered":"2024-09-21T06:52:59.755915+0000","last_clean":"2024-09-21T06:52:59.755915+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:59.755915+0000","last_undegraded":"2024-09-21T06:52:59.755915+0000","last_fullsized":"2024-09-21T06:52:59.755915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704037+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.704037+0000","last_peered":"2024-09-21T06:52:31.704037+0000","last_clean":"2024-09-21T06:52:31.704037+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.704037+0000","last_undegraded":"2024-09-21T06:52:31.704037+0000","last_fullsized":"2024-09-21T06:52:31.704037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699968+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699968+0000","last_peered":"2024-09-21T06:52:31.699968+0000","last_clean":"2024-09-21T06:52:31.699968+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699968+0000","last_undegraded":"2024-09-21T06:52:31.699968+0000","last_fullsized":"2024-09-21T06:52:31.699968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745662+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:59.745662+0000","last_peered":"2024-09-21T06:52:59.745662+0000","last_clean":"2024-09-21T06:52:59.745662+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:59.745662+0000","last_undegraded":"2024-09-21T06:52:59.745662+0000","last_fullsized":"2024-09-21T06:52:59.745662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892935+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892935+0000","last_peered":"2024-09-21T06:52:31.892935+0000","last_clean":"2024-09-21T06:52:31.892935+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892935+0000","last_undegraded":"2024-09-21T06:52:31.892935+0000","last_fullsized":"2024-09-21T06:52:31.892935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646491+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646491+0000","last_peered":"2024-09-21T06:52:31.646491+0000","last_clean":"2024-09-21T06:52:31.646491+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646491+0000","last_undegraded":"2024-09-21T06:52:31.646491+0000","last_fullsized":"2024-09-21T06:52:31.646491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744826+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:59.744826+0000","last_peered":"2024-09-21T06:52:59.744826+0000","last_clean":"2024-09-21T06:52:59.744826+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:59.744826+0000","last_undegraded":"2024-09-21T06:52:59.744826+0000","last_fullsized":"2024-09-21T06:52:59.744826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836870+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836870+0000","last_peered":"2024-09-21T06:52:31.836870+0000","last_clean":"2024-09-21T06:52:31.836870+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836870+0000","last_undegraded":"2024-09-21T06:52:31.836870+0000","last_fullsized":"2024-09-21T06:52:31.836870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755751+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:59.755751+0000","last_peered":"2024-09-21T06:52:59.755751+0000","last_clean":"2024-09-21T06:52:59.755751+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:59.755751+0000","last_undegraded":"2024-09-21T06:52:59.755751+0000","last_fullsized":"2024-09-21T06:52:59.755751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700368+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700368+0000","last_peered":"2024-09-21T06:52:31.700368+0000","last_clean":"2024-09-21T06:52:31.700368+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700368+0000","last_undegraded":"2024-09-21T06:52:31.700368+0000","last_fullsized":"2024-09-21T06:52:31.700368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703897+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703897+0000","last_peered":"2024-09-21T06:52:31.703897+0000","last_clean":"2024-09-21T06:52:31.703897+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703897+0000","last_undegraded":"2024-09-21T06:52:31.703897+0000","last_fullsized":"2024-09-21T06:52:31.703897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741893+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:59.741893+0000","last_peered":"2024-09-21T06:52:59.741893+0000","last_clean":"2024-09-21T06:52:59.741893+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:59.741893+0000","last_undegraded":"2024-09-21T06:52:59.741893+0000","last_fullsized":"2024-09-21T06:52:59.741893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:00.244438+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:53:00.244438+0000","last_peered":"2024-09-21T06:53:00.244438+0000","last_clean":"2024-09-21T06:53:00.244438+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:53:00.244438+0000","last_undegraded":"2024-09-21T06:53:00.244438+0000","last_fullsized":"2024-09-21T06:53:00.244438+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891910+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891910+0000","last_peered":"2024-09-21T06:52:31.891910+0000","last_clean":"2024-09-21T06:52:31.891910+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891910+0000","last_undegraded":"2024-09-21T06:52:31.891910+0000","last_fullsized":"2024-09-21T06:52:31.891910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742064+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:59.742064+0000","last_peered":"2024-09-21T06:52:59.742064+0000","last_clean":"2024-09-21T06:52:59.742064+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:59.742064+0000","last_undegraded":"2024-09-21T06:52:59.742064+0000","last_fullsized":"2024-09-21T06:52:59.742064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892041+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892041+0000","last_peered":"2024-09-21T06:52:31.892041+0000","last_clean":"2024-09-21T06:52:31.892041+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892041+0000","last_undegraded":"2024-09-21T06:52:31.892041+0000","last_fullsized":"2024-09-21T06:52:31.892041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645535+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645535+0000","last_peered":"2024-09-21T06:52:31.645535+0000","last_clean":"2024-09-21T06:52:31.645535+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645535+0000","last_undegraded":"2024-09-21T06:52:31.645535+0000","last_fullsized":"2024-09-21T06:52:31.645535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754090+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:59.754090+0000","last_peered":"2024-09-21T06:52:59.754090+0000","last_clean":"2024-09-21T06:52:59.754090+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:59.754090+0000","last_undegraded":"2024-09-21T06:52:59.754090+0000","last_fullsized":"2024-09-21T06:52:59.754090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645977+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645977+0000","last_peered":"2024-09-21T06:52:31.645977+0000","last_clean":"2024-09-21T06:52:31.645977+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645977+0000","last_undegraded":"2024-09-21T06:52:31.645977+0000","last_fullsized":"2024-09-21T06:52:31.645977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891517+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891517+0000","last_peered":"2024-09-21T06:52:31.891517+0000","last_clean":"2024-09-21T06:52:31.891517+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891517+0000","last_undegraded":"2024-09-21T06:52:31.891517+0000","last_fullsized":"2024-09-21T06:52:31.891517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758656+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:59.758656+0000","last_peered":"2024-09-21T06:52:59.758656+0000","last_clean":"2024-09-21T06:52:59.758656+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:59.758656+0000","last_undegraded":"2024-09-21T06:52:59.758656+0000","last_fullsized":"2024-09-21T06:52:59.758656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'8","reported_seq":187,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:52.829279+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:53:52.829279+0000","last_peered":"2024-09-21T06:53:52.829279+0000","last_clean":"2024-09-21T06:53:52.829279+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:53:52.829279+0000","last_undegraded":"2024-09-21T06:53:52.829279+0000","last_fullsized":"2024-09-21T06:53:52.829279+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1431,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701262+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701262+0000","last_peered":"2024-09-21T06:52:31.701262+0000","last_clean":"2024-09-21T06:52:31.701262+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701262+0000","last_undegraded":"2024-09-21T06:52:31.701262+0000","last_fullsized":"2024-09-21T06:52:31.701262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749255+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:59.749255+0000","last_peered":"2024-09-21T06:52:59.749255+0000","last_clean":"2024-09-21T06:52:59.749255+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:59.749255+0000","last_undegraded":"2024-09-21T06:52:59.749255+0000","last_fullsized":"2024-09-21T06:52:59.749255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.825547+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:53:50.825547+0000","last_peered":"2024-09-21T06:53:50.825547+0000","last_clean":"2024-09-21T06:53:50.825547+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:53:50.825547+0000","last_undegraded":"2024-09-21T06:53:50.825547+0000","last_fullsized":"2024-09-21T06:53:50.825547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701440+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701440+0000","last_peered":"2024-09-21T06:52:31.701440+0000","last_clean":"2024-09-21T06:52:31.701440+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701440+0000","last_undegraded":"2024-09-21T06:52:31.701440+0000","last_fullsized":"2024-09-21T06:52:31.701440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740983+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:59.740983+0000","last_peered":"2024-09-21T06:52:59.740983+0000","last_clean":"2024-09-21T06:52:59.740983+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:59.740983+0000","last_undegraded":"2024-09-21T06:52:59.740983+0000","last_fullsized":"2024-09-21T06:52:59.740983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701878+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701878+0000","last_peered":"2024-09-21T06:52:31.701878+0000","last_clean":"2024-09-21T06:52:31.701878+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701878+0000","last_undegraded":"2024-09-21T06:52:31.701878+0000","last_fullsized":"2024-09-21T06:52:31.701878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749354+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:59.749354+0000","last_peered":"2024-09-21T06:52:59.749354+0000","last_clean":"2024-09-21T06:52:59.749354+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:59.749354+0000","last_undegraded":"2024-09-21T06:52:59.749354+0000","last_fullsized":"2024-09-21T06:52:59.749354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648426+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648426+0000","last_peered":"2024-09-21T06:52:31.648426+0000","last_clean":"2024-09-21T06:52:31.648426+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648426+0000","last_undegraded":"2024-09-21T06:52:31.648426+0000","last_fullsized":"2024-09-21T06:52:31.648426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648721+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648721+0000","last_peered":"2024-09-21T06:52:31.648721+0000","last_clean":"2024-09-21T06:52:31.648721+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648721+0000","last_undegraded":"2024-09-21T06:52:31.648721+0000","last_fullsized":"2024-09-21T06:52:31.648721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753632+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:59.753632+0000","last_peered":"2024-09-21T06:52:59.753632+0000","last_clean":"2024-09-21T06:52:59.753632+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:59.753632+0000","last_undegraded":"2024-09-21T06:52:59.753632+0000","last_fullsized":"2024-09-21T06:52:59.753632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":183,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:50.826734+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:53:50.826734+0000","last_peered":"2024-09-21T06:53:50.826734+0000","last_clean":"2024-09-21T06:53:50.826734+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:53:50.826734+0000","last_undegraded":"2024-09-21T06:53:50.826734+0000","last_fullsized":"2024-09-21T06:53:50.826734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836829+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836829+0000","last_peered":"2024-09-21T06:52:31.836829+0000","last_clean":"2024-09-21T06:52:31.836829+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836829+0000","last_undegraded":"2024-09-21T06:52:31.836829+0000","last_fullsized":"2024-09-21T06:52:31.836829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744841+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:59.744841+0000","last_peered":"2024-09-21T06:52:59.744841+0000","last_clean":"2024-09-21T06:52:59.744841+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:59.744841+0000","last_undegraded":"2024-09-21T06:52:59.744841+0000","last_fullsized":"2024-09-21T06:52:59.744841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702337+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:52:31.702337+0000","last_peered":"2024-09-21T06:52:31.702337+0000","last_clean":"2024-09-21T06:52:31.702337+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:52:31.702337+0000","last_undegraded":"2024-09-21T06:52:31.702337+0000","last_fullsized":"2024-09-21T06:52:31.702337+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891861+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891861+0000","last_peered":"2024-09-21T06:52:31.891861+0000","last_clean":"2024-09-21T06:52:31.891861+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891861+0000","last_undegraded":"2024-09-21T06:52:31.891861+0000","last_fullsized":"2024-09-21T06:52:31.891861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753918+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:59.753918+0000","last_peered":"2024-09-21T06:52:59.753918+0000","last_clean":"2024-09-21T06:52:59.753918+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:59.753918+0000","last_undegraded":"2024-09-21T06:52:59.753918+0000","last_fullsized":"2024-09-21T06:52:59.753918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701640+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701640+0000","last_peered":"2024-09-21T06:52:31.701640+0000","last_clean":"2024-09-21T06:52:31.701640+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701640+0000","last_undegraded":"2024-09-21T06:52:31.701640+0000","last_fullsized":"2024-09-21T06:52:31.701640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701328+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701328+0000","last_peered":"2024-09-21T06:52:31.701328+0000","last_clean":"2024-09-21T06:52:31.701328+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701328+0000","last_undegraded":"2024-09-21T06:52:31.701328+0000","last_fullsized":"2024-09-21T06:52:31.701328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734233+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:59.734233+0000","last_peered":"2024-09-21T06:52:59.734233+0000","last_clean":"2024-09-21T06:52:59.734233+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:59.734233+0000","last_undegraded":"2024-09-21T06:52:59.734233+0000","last_fullsized":"2024-09-21T06:52:59.734233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645941+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645941+0000","last_peered":"2024-09-21T06:52:31.645941+0000","last_clean":"2024-09-21T06:52:31.645941+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645941+0000","last_undegraded":"2024-09-21T06:52:31.645941+0000","last_fullsized":"2024-09-21T06:52:31.645941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701203+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701203+0000","last_peered":"2024-09-21T06:52:31.701203+0000","last_clean":"2024-09-21T06:52:31.701203+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701203+0000","last_undegraded":"2024-09-21T06:52:31.701203+0000","last_fullsized":"2024-09-21T06:52:31.701203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.748492+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:59.748492+0000","last_peered":"2024-09-21T06:52:59.748492+0000","last_clean":"2024-09-21T06:52:59.748492+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:59.748492+0000","last_undegraded":"2024-09-21T06:52:59.748492+0000","last_fullsized":"2024-09-21T06:52:59.748492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891566+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891566+0000","last_peered":"2024-09-21T06:52:31.891566+0000","last_clean":"2024-09-21T06:52:31.891566+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891566+0000","last_undegraded":"2024-09-21T06:52:31.891566+0000","last_fullsized":"2024-09-21T06:52:31.891566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701524+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701524+0000","last_peered":"2024-09-21T06:52:31.701524+0000","last_clean":"2024-09-21T06:52:31.701524+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701524+0000","last_undegraded":"2024-09-21T06:52:31.701524+0000","last_fullsized":"2024-09-21T06:52:31.701524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756438+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:59.756438+0000","last_peered":"2024-09-21T06:52:59.756438+0000","last_clean":"2024-09-21T06:52:59.756438+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:59.756438+0000","last_undegraded":"2024-09-21T06:52:59.756438+0000","last_fullsized":"2024-09-21T06:52:59.756438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:49.824645+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:53:49.824645+0000","last_peered":"2024-09-21T06:53:49.824645+0000","last_clean":"2024-09-21T06:53:49.824645+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:53:49.824645+0000","last_undegraded":"2024-09-21T06:53:49.824645+0000","last_fullsized":"2024-09-21T06:53:49.824645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891921+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891921+0000","last_peered":"2024-09-21T06:52:31.891921+0000","last_clean":"2024-09-21T06:52:31.891921+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891921+0000","last_undegraded":"2024-09-21T06:52:31.891921+0000","last_fullsized":"2024-09-21T06:52:31.891921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758614+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:59.758614+0000","last_peered":"2024-09-21T06:52:59.758614+0000","last_clean":"2024-09-21T06:52:59.758614+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:59.758614+0000","last_undegraded":"2024-09-21T06:52:59.758614+0000","last_fullsized":"2024-09-21T06:52:59.758614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701396+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701396+0000","last_peered":"2024-09-21T06:52:31.701396+0000","last_clean":"2024-09-21T06:52:31.701396+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701396+0000","last_undegraded":"2024-09-21T06:52:31.701396+0000","last_fullsized":"2024-09-21T06:52:31.701396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645499+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645499+0000","last_peered":"2024-09-21T06:52:31.645499+0000","last_clean":"2024-09-21T06:52:31.645499+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645499+0000","last_undegraded":"2024-09-21T06:52:31.645499+0000","last_fullsized":"2024-09-21T06:52:31.645499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749562+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:59.749562+0000","last_peered":"2024-09-21T06:52:59.749562+0000","last_clean":"2024-09-21T06:52:59.749562+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:59.749562+0000","last_undegraded":"2024-09-21T06:52:59.749562+0000","last_fullsized":"2024-09-21T06:52:59.749562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703886+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703886+0000","last_peered":"2024-09-21T06:52:31.703886+0000","last_clean":"2024-09-21T06:52:31.703886+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703886+0000","last_undegraded":"2024-09-21T06:52:31.703886+0000","last_fullsized":"2024-09-21T06:52:31.703886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702060+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.702060+0000","last_peered":"2024-09-21T06:52:31.702060+0000","last_clean":"2024-09-21T06:52:31.702060+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.702060+0000","last_undegraded":"2024-09-21T06:52:31.702060+0000","last_fullsized":"2024-09-21T06:52:31.702060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745772+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:59.745772+0000","last_peered":"2024-09-21T06:52:59.745772+0000","last_clean":"2024-09-21T06:52:59.745772+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:59.745772+0000","last_undegraded":"2024-09-21T06:52:59.745772+0000","last_fullsized":"2024-09-21T06:52:59.745772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:53.831214+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:53:53.831214+0000","last_peered":"2024-09-21T06:53:53.831214+0000","last_clean":"2024-09-21T06:53:53.831214+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:53:53.831214+0000","last_undegraded":"2024-09-21T06:53:53.831214+0000","last_fullsized":"2024-09-21T06:53:53.831214+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71976,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":274,"num_read_kb":209,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645757+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645757+0000","last_peered":"2024-09-21T06:52:31.645757+0000","last_clean":"2024-09-21T06:52:31.645757+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645757+0000","last_undegraded":"2024-09-21T06:52:31.645757+0000","last_fullsized":"2024-09-21T06:52:31.645757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.747527+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:59.747527+0000","last_peered":"2024-09-21T06:52:59.747527+0000","last_clean":"2024-09-21T06:52:59.747527+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:59.747527+0000","last_undegraded":"2024-09-21T06:52:59.747527+0000","last_fullsized":"2024-09-21T06:52:59.747527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702309+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702309+0000","last_peered":"2024-09-21T06:52:31.702309+0000","last_clean":"2024-09-21T06:52:31.702309+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702309+0000","last_undegraded":"2024-09-21T06:52:31.702309+0000","last_fullsized":"2024-09-21T06:52:31.702309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699508+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699508+0000","last_peered":"2024-09-21T06:52:31.699508+0000","last_clean":"2024-09-21T06:52:31.699508+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699508+0000","last_undegraded":"2024-09-21T06:52:31.699508+0000","last_fullsized":"2024-09-21T06:52:31.699508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757106+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:59.757106+0000","last_peered":"2024-09-21T06:52:59.757106+0000","last_clean":"2024-09-21T06:52:59.757106+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:59.757106+0000","last_undegraded":"2024-09-21T06:52:59.757106+0000","last_fullsized":"2024-09-21T06:52:59.757106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648918+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648918+0000","last_peered":"2024-09-21T06:52:31.648918+0000","last_clean":"2024-09-21T06:52:31.648918+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648918+0000","last_undegraded":"2024-09-21T06:52:31.648918+0000","last_fullsized":"2024-09-21T06:52:31.648918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645559+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645559+0000","last_peered":"2024-09-21T06:52:31.645559+0000","last_clean":"2024-09-21T06:52:31.645559+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645559+0000","last_undegraded":"2024-09-21T06:52:31.645559+0000","last_fullsized":"2024-09-21T06:52:31.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750255+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:59.750255+0000","last_peered":"2024-09-21T06:52:59.750255+0000","last_clean":"2024-09-21T06:52:59.750255+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:59.750255+0000","last_undegraded":"2024-09-21T06:52:59.750255+0000","last_fullsized":"2024-09-21T06:52:59.750255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":311,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:53.830828+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:53:53.830828+0000","last_peered":"2024-09-21T06:53:53.830828+0000","last_clean":"2024-09-21T06:53:53.830828+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:53:53.830828+0000","last_undegraded":"2024-09-21T06:53:53.830828+0000","last_fullsized":"2024-09-21T06:53:53.830828+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":262,"num_read_kb":262,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892718+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892718+0000","last_peered":"2024-09-21T06:52:31.892718+0000","last_clean":"2024-09-21T06:52:31.892718+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892718+0000","last_undegraded":"2024-09-21T06:52:31.892718+0000","last_fullsized":"2024-09-21T06:52:31.892718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892186+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892186+0000","last_peered":"2024-09-21T06:52:31.892186+0000","last_clean":"2024-09-21T06:52:31.892186+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892186+0000","last_undegraded":"2024-09-21T06:52:31.892186+0000","last_fullsized":"2024-09-21T06:52:31.892186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741381+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:59.741381+0000","last_peered":"2024-09-21T06:52:59.741381+0000","last_clean":"2024-09-21T06:52:59.741381+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:59.741381+0000","last_undegraded":"2024-09-21T06:52:59.741381+0000","last_fullsized":"2024-09-21T06:52:59.741381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701609+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701609+0000","last_peered":"2024-09-21T06:52:31.701609+0000","last_clean":"2024-09-21T06:52:31.701609+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701609+0000","last_undegraded":"2024-09-21T06:52:31.701609+0000","last_fullsized":"2024-09-21T06:52:31.701609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645808+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645808+0000","last_peered":"2024-09-21T06:52:31.645808+0000","last_clean":"2024-09-21T06:52:31.645808+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645808+0000","last_undegraded":"2024-09-21T06:52:31.645808+0000","last_fullsized":"2024-09-21T06:52:31.645808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742667+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:59.742667+0000","last_peered":"2024-09-21T06:52:59.742667+0000","last_clean":"2024-09-21T06:52:59.742667+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:59.742667+0000","last_undegraded":"2024-09-21T06:52:59.742667+0000","last_fullsized":"2024-09-21T06:52:59.742667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700994+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700994+0000","last_peered":"2024-09-21T06:52:31.700994+0000","last_clean":"2024-09-21T06:52:31.700994+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700994+0000","last_undegraded":"2024-09-21T06:52:31.700994+0000","last_fullsized":"2024-09-21T06:52:31.700994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645981+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645981+0000","last_peered":"2024-09-21T06:52:31.645981+0000","last_clean":"2024-09-21T06:52:31.645981+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645981+0000","last_undegraded":"2024-09-21T06:52:31.645981+0000","last_fullsized":"2024-09-21T06:52:31.645981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749130+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:59.749130+0000","last_peered":"2024-09-21T06:52:59.749130+0000","last_clean":"2024-09-21T06:52:59.749130+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:59.749130+0000","last_undegraded":"2024-09-21T06:52:59.749130+0000","last_fullsized":"2024-09-21T06:52:59.749130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703865+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703865+0000","last_peered":"2024-09-21T06:52:31.703865+0000","last_clean":"2024-09-21T06:52:31.703865+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703865+0000","last_undegraded":"2024-09-21T06:52:31.703865+0000","last_fullsized":"2024-09-21T06:52:31.703865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:49.825569+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:53:49.825569+0000","last_peered":"2024-09-21T06:53:49.825569+0000","last_clean":"2024-09-21T06:53:49.825569+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:53:49.825569+0000","last_undegraded":"2024-09-21T06:53:49.825569+0000","last_fullsized":"2024-09-21T06:53:49.825569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.736418+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:59.736418+0000","last_peered":"2024-09-21T06:52:59.736418+0000","last_clean":"2024-09-21T06:52:59.736418+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:59.736418+0000","last_undegraded":"2024-09-21T06:52:59.736418+0000","last_fullsized":"2024-09-21T06:52:59.736418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704051+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.704051+0000","last_peered":"2024-09-21T06:52:31.704051+0000","last_clean":"2024-09-21T06:52:31.704051+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.704051+0000","last_undegraded":"2024-09-21T06:52:31.704051+0000","last_fullsized":"2024-09-21T06:52:31.704051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.739975+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:59.739975+0000","last_peered":"2024-09-21T06:52:59.739975+0000","last_clean":"2024-09-21T06:52:59.739975+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:59.739975+0000","last_undegraded":"2024-09-21T06:52:59.739975+0000","last_fullsized":"2024-09-21T06:52:59.739975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749347+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:59.749347+0000","last_peered":"2024-09-21T06:52:59.749347+0000","last_clean":"2024-09-21T06:52:59.749347+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:59.749347+0000","last_undegraded":"2024-09-21T06:52:59.749347+0000","last_fullsized":"2024-09-21T06:52:59.749347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746716+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:59.746716+0000","last_peered":"2024-09-21T06:52:59.746716+0000","last_clean":"2024-09-21T06:52:59.746716+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:59.746716+0000","last_undegraded":"2024-09-21T06:52:59.746716+0000","last_fullsized":"2024-09-21T06:52:59.746716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757121+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:59.757121+0000","last_peered":"2024-09-21T06:52:59.757121+0000","last_clean":"2024-09-21T06:52:59.757121+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:59.757121+0000","last_undegraded":"2024-09-21T06:52:59.757121+0000","last_fullsized":"2024-09-21T06:52:59.757121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756339+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:59.756339+0000","last_peered":"2024-09-21T06:52:59.756339+0000","last_clean":"2024-09-21T06:52:59.756339+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:59.756339+0000","last_undegraded":"2024-09-21T06:52:59.756339+0000","last_fullsized":"2024-09-21T06:52:59.756339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734085+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:59.734085+0000","last_peered":"2024-09-21T06:52:59.734085+0000","last_clean":"2024-09-21T06:52:59.734085+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:59.734085+0000","last_undegraded":"2024-09-21T06:52:59.734085+0000","last_fullsized":"2024-09-21T06:52:59.734085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749595+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:59.749595+0000","last_peered":"2024-09-21T06:52:59.749595+0000","last_clean":"2024-09-21T06:52:59.749595+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:59.749595+0000","last_undegraded":"2024-09-21T06:52:59.749595+0000","last_fullsized":"2024-09-21T06:52:59.749595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.737176+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:59.737176+0000","last_peered":"2024-09-21T06:52:59.737176+0000","last_clean":"2024-09-21T06:52:59.737176+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:59.737176+0000","last_undegraded":"2024-09-21T06:52:59.737176+0000","last_fullsized":"2024-09-21T06:52:59.737176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746235+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:59.746235+0000","last_peered":"2024-09-21T06:52:59.746235+0000","last_clean":"2024-09-21T06:52:59.746235+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:59.746235+0000","last_undegraded":"2024-09-21T06:52:59.746235+0000","last_fullsized":"2024-09-21T06:52:59.746235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740649+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:59.740649+0000","last_peered":"2024-09-21T06:52:59.740649+0000","last_clean":"2024-09-21T06:52:59.740649+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:59.740649+0000","last_undegraded":"2024-09-21T06:52:59.740649+0000","last_fullsized":"2024-09-21T06:52:59.740649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753696+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:59.753696+0000","last_peered":"2024-09-21T06:52:59.753696+0000","last_clean":"2024-09-21T06:52:59.753696+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:59.753696+0000","last_undegraded":"2024-09-21T06:52:59.753696+0000","last_fullsized":"2024-09-21T06:52:59.753696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756495+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:59.756495+0000","last_peered":"2024-09-21T06:52:59.756495+0000","last_clean":"2024-09-21T06:52:59.756495+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:59.756495+0000","last_undegraded":"2024-09-21T06:52:59.756495+0000","last_fullsized":"2024-09-21T06:52:59.756495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734175+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:59.734175+0000","last_peered":"2024-09-21T06:52:59.734175+0000","last_clean":"2024-09-21T06:52:59.734175+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:59.734175+0000","last_undegraded":"2024-09-21T06:52:59.734175+0000","last_fullsized":"2024-09-21T06:52:59.734175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745394+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:59.745394+0000","last_peered":"2024-09-21T06:52:59.745394+0000","last_clean":"2024-09-21T06:52:59.745394+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:59.745394+0000","last_undegraded":"2024-09-21T06:52:59.745394+0000","last_fullsized":"2024-09-21T06:52:59.745394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751016+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:59.751016+0000","last_peered":"2024-09-21T06:52:59.751016+0000","last_clean":"2024-09-21T06:52:59.751016+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:59.751016+0000","last_undegraded":"2024-09-21T06:52:59.751016+0000","last_fullsized":"2024-09-21T06:52:59.751016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755908+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:59.755908+0000","last_peered":"2024-09-21T06:52:59.755908+0000","last_clean":"2024-09-21T06:52:59.755908+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:59.755908+0000","last_undegraded":"2024-09-21T06:52:59.755908+0000","last_fullsized":"2024-09-21T06:52:59.755908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752146+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:59.752146+0000","last_peered":"2024-09-21T06:52:59.752146+0000","last_clean":"2024-09-21T06:52:59.752146+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:59.752146+0000","last_undegraded":"2024-09-21T06:52:59.752146+0000","last_fullsized":"2024-09-21T06:52:59.752146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754481+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:59.754481+0000","last_peered":"2024-09-21T06:52:59.754481+0000","last_clean":"2024-09-21T06:52:59.754481+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:59.754481+0000","last_undegraded":"2024-09-21T06:52:59.754481+0000","last_fullsized":"2024-09-21T06:52:59.754481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758054+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:59.758054+0000","last_peered":"2024-09-21T06:52:59.758054+0000","last_clean":"2024-09-21T06:52:59.758054+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:59.758054+0000","last_undegraded":"2024-09-21T06:52:59.758054+0000","last_fullsized":"2024-09-21T06:52:59.758054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756767+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:59.756767+0000","last_peered":"2024-09-21T06:52:59.756767+0000","last_clean":"2024-09-21T06:52:59.756767+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:59.756767+0000","last_undegraded":"2024-09-21T06:52:59.756767+0000","last_fullsized":"2024-09-21T06:52:59.756767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755238+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:59.755238+0000","last_peered":"2024-09-21T06:52:59.755238+0000","last_clean":"2024-09-21T06:52:59.755238+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:59.755238+0000","last_undegraded":"2024-09-21T06:52:59.755238+0000","last_fullsized":"2024-09-21T06:52:59.755238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752053+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:59.752053+0000","last_peered":"2024-09-21T06:52:59.752053+0000","last_clean":"2024-09-21T06:52:59.752053+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:59.752053+0000","last_undegraded":"2024-09-21T06:52:59.752053+0000","last_fullsized":"2024-09-21T06:52:59.752053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.759125+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:59.759125+0000","last_peered":"2024-09-21T06:52:59.759125+0000","last_clean":"2024-09-21T06:52:59.759125+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:59.759125+0000","last_undegraded":"2024-09-21T06:52:59.759125+0000","last_fullsized":"2024-09-21T06:52:59.759125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742201+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:59.742201+0000","last_peered":"2024-09-21T06:52:59.742201+0000","last_clean":"2024-09-21T06:52:59.742201+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:59.742201+0000","last_undegraded":"2024-09-21T06:52:59.742201+0000","last_fullsized":"2024-09-21T06:52:59.742201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755485+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:59.755485+0000","last_peered":"2024-09-21T06:52:59.755485+0000","last_clean":"2024-09-21T06:52:59.755485+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:59.755485+0000","last_undegraded":"2024-09-21T06:52:59.755485+0000","last_fullsized":"2024-09-21T06:52:59.755485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752958+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:59.752958+0000","last_peered":"2024-09-21T06:52:59.752958+0000","last_clean":"2024-09-21T06:52:59.752958+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:59.752958+0000","last_undegraded":"2024-09-21T06:52:59.752958+0000","last_fullsized":"2024-09-21T06:52:59.752958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754740+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:59.754740+0000","last_peered":"2024-09-21T06:52:59.754740+0000","last_clean":"2024-09-21T06:52:59.754740+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:59.754740+0000","last_undegraded":"2024-09-21T06:52:59.754740+0000","last_fullsized":"2024-09-21T06:52:59.754740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746351+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:59.746351+0000","last_peered":"2024-09-21T06:52:59.746351+0000","last_clean":"2024-09-21T06:52:59.746351+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:59.746351+0000","last_undegraded":"2024-09-21T06:52:59.746351+0000","last_fullsized":"2024-09-21T06:52:59.746351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749389+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:59.749389+0000","last_peered":"2024-09-21T06:52:59.749389+0000","last_clean":"2024-09-21T06:52:59.749389+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:59.749389+0000","last_undegraded":"2024-09-21T06:52:59.749389+0000","last_fullsized":"2024-09-21T06:52:59.749389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757852+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:59.757852+0000","last_peered":"2024-09-21T06:52:59.757852+0000","last_clean":"2024-09-21T06:52:59.757852+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:59.757852+0000","last_undegraded":"2024-09-21T06:52:59.757852+0000","last_fullsized":"2024-09-21T06:52:59.757852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750926+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:59.750926+0000","last_peered":"2024-09-21T06:52:59.750926+0000","last_clean":"2024-09-21T06:52:59.750926+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:59.750926+0000","last_undegraded":"2024-09-21T06:52:59.750926+0000","last_fullsized":"2024-09-21T06:52:59.750926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":80305,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1347,"num_read_kb":766,"num_write":270,"num_write_kb":167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":240915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710772,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":531799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907000,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1740800,"data_stored":1190037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070525,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":535271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266756,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":531531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1129034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626511,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":527522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1123773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149793,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":526387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378727,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":599815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542251,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":601682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705777,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:53:55.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:53:56.151 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:56.737 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:56 smithi114 ceph-mon[30654]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:56.737 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:56 smithi114 ceph-mon[30654]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:56.737 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:56 smithi114 ceph-mon[30654]: pgmap v390: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T06:53:56.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 -- 172.21.15.114:0/1473121775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 msgr2=0x7f43e810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 --2- 172.21.15.114:0/1473121775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e810b900 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009900 tx=0x7f43d402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 -- 172.21.15.114:0/1473121775 shutdown_connections 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 --2- 172.21.15.114:0/1473121775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43e810be40 0x7f43e8112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 --2- 172.21.15.114:0/1473121775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e810b900 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 --2- 172.21.15.114:0/1473121775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e8105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 -- 172.21.15.114:0/1473121775 >> 172.21.15.114:0/1473121775 conn(0x7f43e8101680 msgr2=0x7f43e8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 -- 172.21.15.114:0/1473121775 shutdown_connections 2024-09-21T06:53:56.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.821+0000 7f43eda00640 1 -- 172.21.15.114:0/1473121775 wait complete. 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 Processor -- start 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 -- start start 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43e810be40 0x7f43e81a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8115b50 con 0x7f43e810b4f0 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43e81159d0 con 0x7f43e810be40 2024-09-21T06:53:56.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.823+0000 7f43eda00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43e8115cd0 con 0x7f43e8105bb0 2024-09-21T06:53:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:56.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43ed000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43e810be40 0x7f43e81a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55766/0 (socket says 172.21.15.114:55766) 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47722/0 (socket says 172.21.15.114:47722) 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 -- 172.21.15.114:0/1063075201 learned_addr learned my addr 172.21.15.114:0/1063075201 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 -- 172.21.15.114:0/1063075201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43e810be40 msgr2=0x7f43e81a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43e810be40 0x7f43e81a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 -- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 msgr2=0x7f43e819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.824+0000 7f43e7e00640 1 -- 172.21.15.114:0/1063075201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e81a2490 con 0x7f43e8105bb0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.825+0000 7f43e7e00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e819c520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e9f0 tx=0x7f43d800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.825+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d800cd80 con 0x7f43e8105bb0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.825+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e81a2780 con 0x7f43e8105bb0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.825+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d80054d0 con 0x7f43e8105bb0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.825+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8010690 con 0x7f43e8105bb0 2024-09-21T06:53:56.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.825+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e81a8ee0 con 0x7f43e8105bb0 2024-09-21T06:53:56.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.826+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43e8105fc0 con 0x7f43e8105bb0 2024-09-21T06:53:56.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.828+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f43d8004e90 con 0x7f43e8105bb0 2024-09-21T06:53:56.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.830+0000 7f43e4c00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43cc0b65e0 0x7f43cc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:56.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.830+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f43d80e7ee0 con 0x7f43e8105bb0 2024-09-21T06:53:56.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.830+0000 7f43e7400640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43cc0b65e0 0x7f43cc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:56.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.831+0000 7f43e7400640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43cc0b65e0 0x7f43cc0b8af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43d402aa00 tx=0x7f43d4005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:56.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:56.832+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f43d80ae280 con 0x7f43e8105bb0 2024-09-21T06:53:57.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.056+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43e819d650 con 0x7f43e8105bb0 2024-09-21T06:53:57.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.057+0000 7f43e4c00640 1 -- 172.21.15.114:0/1063075201 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f43d80b2660 con 0x7f43e8105bb0 2024-09-21T06:53:57.059 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:57.059 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"btime":"2024-09-21T06:53:49:573848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:53:49.006951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:53:57.059 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-21T06:53:57.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43cc0b65e0 msgr2=0x7f43cc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43cc0b65e0 0x7f43cc0b8af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43d402aa00 tx=0x7f43d4005a20 comp rx=0 tx=0).stop 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 msgr2=0x7f43e819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e819c520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e9f0 tx=0x7f43d800edf0 comp rx=0 tx=0).stop 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 shutdown_connections 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43cc0b65e0 0x7f43cc0b8af0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43e810be40 0x7f43e81a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43e810b4f0 0x7f43e819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 --2- 172.21.15.114:0/1063075201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43e8105bb0 0x7f43e819c520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 >> 172.21.15.114:0/1063075201 conn(0x7f43e8101680 msgr2=0x7f43e8103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 shutdown_connections 2024-09-21T06:53:57.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:57.060+0000 7f43eda00640 1 -- 172.21.15.114:0/1063075201 wait complete. 2024-09-21T06:53:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:56 smithi183 ceph-mon[34219]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:56 smithi183 ceph-mon[34219]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:56 smithi183 ceph-mon[34219]: pgmap v390: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T06:53:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:56 smithi193 ceph-mon[34601]: from='client.15486 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:56 smithi193 ceph-mon[34601]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:56 smithi193 ceph-mon[34601]: pgmap v390: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-21T06:53:57.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:57 smithi114 ceph-mon[30654]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:57.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:57 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1063075201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:58.004 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:53:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:57 smithi183 ceph-mon[34219]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:57 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1063075201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:57 smithi193 ceph-mon[34601]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:53:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:57 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1063075201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:53:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:53:58.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:58 smithi114 ceph-mon[30654]: pgmap v391: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:53:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 -- 172.21.15.114:0/1324742870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b4105d80 msgr2=0x7f16b4106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 --2- 172.21.15.114:0/1324742870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b4105d80 0x7f16b4106190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009900 tx=0x7f16a402a410 comp rx=0 tx=0).stop 2024-09-21T06:53:59.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 -- 172.21.15.114:0/1324742870 shutdown_connections 2024-09-21T06:53:59.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 --2- 172.21.15.114:0/1324742870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b410c010 0x7f16b4112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 --2- 172.21.15.114:0/1324742870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16b410b6c0 0x7f16b410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 --2- 172.21.15.114:0/1324742870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b4105d80 0x7f16b4106190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 -- 172.21.15.114:0/1324742870 >> 172.21.15.114:0/1324742870 conn(0x7f16b4101810 msgr2=0x7f16b4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:59.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 -- 172.21.15.114:0/1324742870 shutdown_connections 2024-09-21T06:53:59.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.062+0000 7f16b9a00640 1 -- 172.21.15.114:0/1324742870 wait complete. 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.063+0000 7f16b9a00640 1 Processor -- start 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 -- start start 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 0x7f16b419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b410b6c0 0x7f16b419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16b410c010 0x7f16b41a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4115d20 con 0x7f16b410b6c0 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4115ba0 con 0x7f16b410c010 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b9a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4115ea0 con 0x7f16b4105d80 2024-09-21T06:53:59.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 0x7f16b419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 0x7f16b419c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55778/0 (socket says 172.21.15.114:55778) 2024-09-21T06:53:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b2a00640 1 -- 172.21.15.114:0/3437503658 learned_addr learned my addr 172.21.15.114:0/3437503658 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:53:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b2000640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b410b6c0 0x7f16b419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.064+0000 7f16b3400640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16b410c010 0x7f16b41a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:59.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b2a00640 1 -- 172.21.15.114:0/3437503658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16b410c010 msgr2=0x7f16b41a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b2a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16b410c010 0x7f16b41a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b2a00640 1 -- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b410b6c0 msgr2=0x7f16b419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b2a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b410b6c0 0x7f16b419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b2a00640 1 -- 172.21.15.114:0/3437503658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41a26e0 con 0x7f16b4105d80 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b2a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 0x7f16b419c770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005ab0 tx=0x7f16a4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a403f070 con 0x7f16b4105d80 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.065+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b41a2970 con 0x7f16b4105d80 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.066+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a4031b60 con 0x7f16b4105d80 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.066+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4035030 con 0x7f16b4105d80 2024-09-21T06:53:59.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.066+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41a9130 con 0x7f16b4105d80 2024-09-21T06:53:59.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.067+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b4106190 con 0x7f16b4105d80 2024-09-21T06:53:59.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.069+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f16a4042070 con 0x7f16b4105d80 2024-09-21T06:53:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.071+0000 7f16ab400640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16980b6560 0x7f16980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:53:59.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.071+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f16a4108710 con 0x7f16b4105d80 2024-09-21T06:53:59.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.071+0000 7f16b2000640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16980b6560 0x7f16980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:53:59.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.072+0000 7f16b2000640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16980b6560 0x7f16980b8a70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16b4104f10 tx=0x7f16a0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:53:59.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.073+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16a40d2e90 con 0x7f16b4105d80 2024-09-21T06:53:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:58 smithi183 ceph-mon[34219]: pgmap v391: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:53:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:58 smithi193 ceph-mon[34601]: pgmap v391: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:53:59.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.266+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16b419d8a0 con 0x7f16b4105d80 2024-09-21T06:53:59.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.268+0000 7f16ab400640 1 -- 172.21.15.114:0/3437503658 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25277 (secure 0 0 0) 0x7f16b419d8a0 con 0x7f16b4105d80 2024-09-21T06:53:59.270 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:53:59.271 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:52:31.637070+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:53:59.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16980b6560 msgr2=0x7f16980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16980b6560 0x7f16980b8a70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f16b4104f10 tx=0x7f16a0005f90 comp rx=0 tx=0).stop 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 msgr2=0x7f16b419c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 0x7f16b419c770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005ab0 tx=0x7f16a4003c10 comp rx=0 tx=0).stop 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 shutdown_connections 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16980b6560 0x7f16980b8a70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16b410c010 0x7f16b41a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f16b410b6c0 0x7f16b419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 --2- 172.21.15.114:0/3437503658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16b4105d80 0x7f16b419c770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 >> 172.21.15.114:0/3437503658 conn(0x7f16b4101810 msgr2=0x7f16b410ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 shutdown_connections 2024-09-21T06:53:59.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:53:59.271+0000 7f16b9a00640 1 -- 172.21.15.114:0/3437503658 wait complete. 2024-09-21T06:53:59.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:59 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3437503658' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:53:59.974 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:53:59 smithi114 ceph-mon[30654]: pgmap v392: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:54:00.014 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T06:54:00.014 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:54:00.023 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:54:00.024 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:54:00.024 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T06:54:00.024 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T06:54:00.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:54:00.025 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:54:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:59 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3437503658' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:54:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:53:59 smithi183 ceph-mon[34219]: pgmap v392: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:54:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:59 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3437503658' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:54:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:53:59 smithi193 ceph-mon[34601]: pgmap v392: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-21T06:54:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 -- 172.21.15.114:0/2071267138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109b90 msgr2=0x7f5c801103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/2071267138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109b90 0x7f5c801103c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009ca0 tx=0x7f5c7802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 -- 172.21.15.114:0/2071267138 shutdown_connections 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/2071267138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109b90 0x7f5c801103c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/2071267138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c80109240 0x7f5c80109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/2071267138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c80103910 0x7f5c80103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 -- 172.21.15.114:0/2071267138 >> 172.21.15.114:0/2071267138 conn(0x7f5c800ff330 msgr2=0x7f5c80101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 -- 172.21.15.114:0/2071267138 shutdown_connections 2024-09-21T06:54:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.246+0000 7f5c7fe00640 1 -- 172.21.15.114:0/2071267138 wait complete. 2024-09-21T06:54:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 Processor -- start 2024-09-21T06:54:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 -- start start 2024-09-21T06:54:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c80103910 0x7f5c80075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 0x7f5c80075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c80109b90 0x7f5c80076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5c801138a0 con 0x7f5c80109240 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c80113720 con 0x7f5c80109b90 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.247+0000 7f5c7fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c80113a20 con 0x7f5c80103910 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 0x7f5c80075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 0x7f5c80075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47744/0 (socket says 172.21.15.114:47744) 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 -- 172.21.15.114:0/3922152362 learned_addr learned my addr 172.21.15.114:0/3922152362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7ea00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c80103910 0x7f5c80075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7f400640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c80109b90 0x7f5c80076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 -- 172.21.15.114:0/3922152362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c80103910 msgr2=0x7f5c80075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c80103910 0x7f5c80075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 -- 172.21.15.114:0/3922152362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c80109b90 msgr2=0x7f5c80076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c80109b90 0x7f5c80076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 -- 172.21.15.114:0/3922152362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c801a8530 con 0x7f5c80109240 2024-09-21T06:54:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.248+0000 7f5c7e000640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 0x7f5c80075c50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e9f0 tx=0x7f5c6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.249+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00ccc0 con 0x7f5c80109240 2024-09-21T06:54:00.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.249+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c0108e0 con 0x7f5c80109240 2024-09-21T06:54:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.249+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c801a8820 con 0x7f5c80109240 2024-09-21T06:54:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.249+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c018860 con 0x7f5c80109240 2024-09-21T06:54:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.249+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c801a8d70 con 0x7f5c80109240 2024-09-21T06:54:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.250+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c80103d20 con 0x7f5c80109240 2024-09-21T06:54:00.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.253+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f5c6c010430 con 0x7f5c80109240 2024-09-21T06:54:00.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.256+0000 7f5c77400640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c640b6590 0x7f5c640b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.256+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5c6c0e7ef0 con 0x7f5c80109240 2024-09-21T06:54:00.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.256+0000 7f5c7ea00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c640b6590 0x7f5c640b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.256+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5c6c0b2670 con 0x7f5c80109240 2024-09-21T06:54:00.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.257+0000 7f5c7ea00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c640b6590 0x7f5c640b8aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c80077160 tx=0x7f5c70005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:00.410 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:00.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.488+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f5c80063800 con 0x7f5c640b6590 2024-09-21T06:54:00.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.671+0000 7f5c77400640 1 -- 172.21.15.114:0/3922152362 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5c80063800 con 0x7f5c640b6590 2024-09-21T06:54:00.673 INFO:teuthology.orchestra.run.smithi114.stdout:Schedule set for path / 2024-09-21T06:54:00.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.673+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c640b6590 msgr2=0x7f5c640b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.673+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c640b6590 0x7f5c640b8aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5c80077160 tx=0x7f5c70005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:00.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.673+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 msgr2=0x7f5c80075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.673+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 0x7f5c80075c50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e9f0 tx=0x7f5c6c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.673+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 shutdown_connections 2024-09-21T06:54:00.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.673+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5c640b6590 0x7f5c640b8aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.674+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c80109b90 0x7f5c80076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.674+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5c80109240 0x7f5c80075c50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.674+0000 7f5c7fe00640 1 --2- 172.21.15.114:0/3922152362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c80103910 0x7f5c80075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.674+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 >> 172.21.15.114:0/3922152362 conn(0x7f5c800ff330 msgr2=0x7f5c801067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.674+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 shutdown_connections 2024-09-21T06:54:00.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.674+0000 7f5c7fe00640 1 -- 172.21.15.114:0/3922152362 wait complete. 2024-09-21T06:54:00.698 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T06:54:00.799 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:00 smithi114 ceph-mon[30654]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 -- 172.21.15.114:0/4289983622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea6810b440 msgr2=0x7fea6810b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 --2- 172.21.15.114:0/4289983622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea6810b440 0x7fea6810b850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fea50009860 tx=0x7fea5002a340 comp rx=0 tx=0).stop 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 -- 172.21.15.114:0/4289983622 shutdown_connections 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 --2- 172.21.15.114:0/4289983622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea6810bd90 0x7fea681125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 --2- 172.21.15.114:0/4289983622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea6810b440 0x7fea6810b850 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 --2- 172.21.15.114:0/4289983622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea68105b10 0x7fea68105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 -- 172.21.15.114:0/4289983622 >> 172.21.15.114:0/4289983622 conn(0x7fea68101550 msgr2=0x7fea681039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 -- 172.21.15.114:0/4289983622 shutdown_connections 2024-09-21T06:54:00.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.924+0000 7fea6d200640 1 -- 172.21.15.114:0/4289983622 wait complete. 2024-09-21T06:54:00.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.925+0000 7fea6d200640 1 Processor -- start 2024-09-21T06:54:00.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.925+0000 7fea6d200640 1 -- start start 2024-09-21T06:54:00.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea6d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 0x7fea6819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea6d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6810b440 0x7fea6819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea6d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea6810bd90 0x7fea681a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea6d200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fea68115a50 con 0x7fea68105b10 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea6d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea681158d0 con 0x7fea6810bd90 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea6d200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea68115bd0 con 0x7fea6810b440 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea67400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 0x7fea6819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea5ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6810b440 0x7fea6819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea67400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 0x7fea6819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47780/0 (socket says 172.21.15.114:47780) 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea67400640 1 -- 172.21.15.114:0/1290781749 learned_addr learned my addr 172.21.15.114:0/1290781749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:00.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea67e00640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea6810bd90 0x7fea681a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea5ea00640 1 -- 172.21.15.114:0/1290781749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea6810bd90 msgr2=0x7fea681a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea5ea00640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea6810bd90 0x7fea681a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea5ea00640 1 -- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 msgr2=0x7fea6819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:00.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea5ea00640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 0x7fea6819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:00.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea5ea00640 1 -- 172.21.15.114:0/1290781749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea681a23c0 con 0x7fea6810b440 2024-09-21T06:54:00.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.926+0000 7fea67400640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 0x7fea6819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.927+0000 7fea5ea00640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6810b440 0x7fea6819ca30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea5002a860 tx=0x7fea50032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.927+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50040020 con 0x7fea6810b440 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.927+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea50030e10 con 0x7fea6810b440 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.927+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea681a2650 con 0x7fea6810b440 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.927+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50037040 con 0x7fea6810b440 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.927+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea681a8e40 con 0x7fea6810b440 2024-09-21T06:54:00.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.928+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea68106d80 con 0x7fea6810b440 2024-09-21T06:54:00.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.931+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fea50043020 con 0x7fea6810b440 2024-09-21T06:54:00.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.932+0000 7fea64c00640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea4c0b6560 0x7fea4c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:00.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.932+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fea501072b0 con 0x7fea6810b440 2024-09-21T06:54:00.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.932+0000 7fea67400640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea4c0b6560 0x7fea4c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:00.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.933+0000 7fea67400640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea4c0b6560 0x7fea4c0b8a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea580096c0 tx=0x7fea58009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:00.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:00.934+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea5004a020 con 0x7fea6810b440 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 -- 172.21.15.114:0/2162326035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 msgr2=0x7fe2400761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 --2- 172.21.15.114:0/2162326035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2400761e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009900 tx=0x7fe22c02a410 comp rx=0 tx=0).stop 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 -- 172.21.15.114:0/2162326035 shutdown_connections 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 --2- 172.21.15.114:0/2162326035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe240114280 0x7fe240116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 --2- 172.21.15.114:0/2162326035 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2400761e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 --2- 172.21.15.114:0/2162326035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe240077380 0x7fe2400757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 -- 172.21.15.114:0/2162326035 >> 172.21.15.114:0/2162326035 conn(0x7fe2400ff440 msgr2=0x7fe240101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 -- 172.21.15.114:0/2162326035 shutdown_connections 2024-09-21T06:54:01.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.112+0000 7fe246400640 1 -- 172.21.15.114:0/2162326035 wait complete. 2024-09-21T06:54:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:00 smithi183 ceph-mon[34219]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:01.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 Processor -- start 2024-09-21T06:54:01.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 -- start start 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2401a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe240077380 0x7fe2401a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe240114280 0x7fe2401a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe240119ed0 con 0x7fe240075dd0 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe240119d50 con 0x7fe240114280 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.114+0000 7fe246400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe24011a050 con 0x7fe240077380 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2401a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2401a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47804/0 (socket says 172.21.15.114:47804) 2024-09-21T06:54:01.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 -- 172.21.15.114:0/2362377473 learned_addr learned my addr 172.21.15.114:0/2362377473 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe240077380 0x7fe2401a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245a00640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe240114280 0x7fe2401a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 -- 172.21.15.114:0/2362377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe240077380 msgr2=0x7fe2401a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe240077380 0x7fe2401a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 -- 172.21.15.114:0/2362377473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe240114280 msgr2=0x7fe2401a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe240114280 0x7fe2401a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.115+0000 7fe245000640 1 -- 172.21.15.114:0/2362377473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2401a68d0 con 0x7fe240075dd0 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.116+0000 7fe245000640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2401a0960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe23400e9f0 tx=0x7fe23400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.116+0000 7fe23d600640 1 -- 172.21.15.114:0/2362377473 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23400cd80 con 0x7fe240075dd0 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.116+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2401a6bc0 con 0x7fe240075dd0 2024-09-21T06:54:01.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.116+0000 7fe23d600640 1 -- 172.21.15.114:0/2362377473 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2340108e0 con 0x7fe240075dd0 2024-09-21T06:54:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.116+0000 7fe23d600640 1 -- 172.21.15.114:0/2362377473 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234018980 con 0x7fe240075dd0 2024-09-21T06:54:01.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.116+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2401ad320 con 0x7fe240075dd0 2024-09-21T06:54:01.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.119+0000 7fe24e628740 1 Processor -- start 2024-09-21T06:54:01.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.119+0000 7fe24e628740 1 -- start start 2024-09-21T06:54:01.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe24e628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe24e628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe24e628740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe24e628740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555cf795fe90 con 0x555cf7621670 2024-09-21T06:54:01.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe24e628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555cf794f2c0 con 0x555cf7619490 2024-09-21T06:54:01.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe24e628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555cf79530d0 con 0x555cf7642350 2024-09-21T06:54:01.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe245a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe245a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79e9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55868/0 (socket says 172.21.15.114:55868) 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe245a00640 1 -- 172.21.15.114:0/212050821 learned_addr learned my addr 172.21.15.114:0/212050821 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.120+0000 7fe23fe00640 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe245a00640 1 -- 172.21.15.114:0/212050821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 msgr2=0x555cf79ea440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe245a00640 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79ea440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe245a00640 1 -- 172.21.15.114:0/212050821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 msgr2=0x555cf79ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe245a00640 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe245a00640 1 -- 172.21.15.114:0/212050821 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf76ad930 con 0x555cf7642350 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe23fe00640 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:01.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe245a00640 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79e9f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2380039f0 tx=0x7fe238003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81dc4c54c4bf9eac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.121+0000 7fe23d600640 1 -- 172.21.15.114:0/2362377473 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7fe234010430 con 0x7fe240075dd0 2024-09-21T06:54:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.122+0000 7fe232000640 1 -- 172.21.15.114:0/212050821 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23801a470 con 0x555cf7642350 2024-09-21T06:54:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.122+0000 7fe232000640 1 -- 172.21.15.114:0/212050821 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe238015070 con 0x555cf7642350 2024-09-21T06:54:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.122+0000 7fe232000640 1 -- 172.21.15.114:0/212050821 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238010780 con 0x555cf7642350 2024-09-21T06:54:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 -- 172.21.15.114:0/212050821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 msgr2=0x555cf79e9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79e9f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe2380039f0 tx=0x7fe238003eb0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 -- 172.21.15.114:0/212050821 shutdown_connections 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79ea440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 --2- 172.21.15.114:0/212050821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79e9f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 -- 172.21.15.114:0/212050821 >> 172.21.15.114:0/212050821 conn(0x555cf7636740 msgr2=0x555cf79e7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 -- 172.21.15.114:0/212050821 shutdown_connections 2024-09-21T06:54:01.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe24e628740 1 -- 172.21.15.114:0/212050821 wait complete. 2024-09-21T06:54:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.123+0000 7fe23d600640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2280b6470 0x7fe2280b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe23d600640 1 -- 172.21.15.114:0/2362377473 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe2340e7b30 con 0x7fe240075dd0 2024-09-21T06:54:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2280b6470 0x7fe2280b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe24e628740 1 -- start start 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe24e628740 1 Processor -- start 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2280b6470 0x7fe2280b8980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe22c031440 tx=0x7fe22c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe24e628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.124+0000 7fe24e628740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe24e628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf7a01a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe24e628740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555cf77b0890 con 0x555cf7621670 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe24e628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555cf78ea160 con 0x555cf7619490 2024-09-21T06:54:01.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe24e628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555cf79f86e0 con 0x555cf7642350 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe24e628740 10 client.0 _ll_register_callbacks cb 0x555cf7906a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47826/0 (socket says 172.21.15.114:47826) 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe23fe00640 1 -- 172.21.15.114:0/2363428453 learned_addr learned my addr 172.21.15.114:0/2363428453 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245000640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245a00640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf7a01a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245000640 1 -- 172.21.15.114:0/2363428453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 msgr2=0x555cf7a01a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245000640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf7a01a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245000640 1 -- 172.21.15.114:0/2363428453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 msgr2=0x555cf79f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245000640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe245000640 1 -- 172.21.15.114:0/2363428453 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf7a02040 con 0x555cf7619490 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.125+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe245000640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f5b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2340e5170 tx=0x7fe2340ad7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb25f1ff83d1861b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe226a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe226a00640 1 -- 172.21.15.114:0/2363428453 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234010720 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe238015070 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe226a00640 1 -- 172.21.15.114:0/2363428453 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe2340aeb60 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe226a00640 1 -- 172.21.15.114:0/2363428453 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe2340ad3f0 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe24e628740 10 client.25535 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555cf7a049c0 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.126+0000 7fe226a00640 1 -- 172.21.15.114:0/2363428453 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe238015070 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe226a00640 1 -- 172.21.15.114:0/2363428453 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe2340b34a0 con 0x555cf7619490 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe24e628740 10 client.25535 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe24e628740 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe24e628740 20 client.25535 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:01.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe24e628740 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf7600cd0 0x555cf7a07e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe24e628740 4 client.25535 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cf7961de0 con 0x555cf7600cd0 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.127+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf7600cd0 0x555cf7a07e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.128+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf7600cd0 0x555cf7a07e30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7793341ee7efc6 server_cookie=a8bfaaeedbe22eb9 in_seq=0 out_seq=0 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.128+0000 7fe226a00640 10 client.25535 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.128+0000 7fe226a00640 1 -- 172.21.15.114:0/2363428453 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555cf7961de0 con 0x555cf7600cd0 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.128+0000 7fe226a00640 10 client.25535 handle_command_reply: tid=0 2024-09-21T06:54:01.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.128+0000 7fe24e628740 1 client.25535 shutdown 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 msgr2=0x555cf79f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f5b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2340e5170 tx=0x7fe2340ad7e0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 shutdown_connections 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf7600cd0 0x555cf7a07e30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf7a01a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 --2- 172.21.15.114:0/2363428453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f5b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe226a00640 0 client.25535 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 >> 172.21.15.114:0/2363428453 conn(0x555cf7636740 msgr2=0x555cf79ed090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 shutdown_connections 2024-09-21T06:54:01.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 1 -- 172.21.15.114:0/2363428453 wait complete. 2024-09-21T06:54:01.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.129+0000 7fe24e628740 20 client.25535 trim_cache size 0 max 16384 2024-09-21T06:54:01.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.147+0000 7fe24e628740 1 Processor -- start 2024-09-21T06:54:01.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 -- start start 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401b7310 con 0x555cf7621670 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401b8e50 con 0x555cf7619490 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe24e628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401b79e0 con 0x555cf7642350 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe245000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe245000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49322/0 (socket says 172.21.15.114:49322) 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe245000640 1 -- 172.21.15.114:0/2963300042 learned_addr learned my addr 172.21.15.114:0/2963300042 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.148+0000 7fe245a00640 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe245000640 1 -- 172.21.15.114:0/2963300042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 msgr2=0x555cf79f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe245000640 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe245000640 1 -- 172.21.15.114:0/2963300042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 msgr2=0x555cf79e7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe245000640 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79e7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe245000640 1 -- 172.21.15.114:0/2963300042 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf79ec460 con 0x555cf7619490 2024-09-21T06:54:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe23fe00640 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe245000640 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2340ad870 tx=0x7fe234005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d915e25cf4fb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe231600640 1 -- 172.21.15.114:0/2963300042 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe234010720 con 0x555cf7619490 2024-09-21T06:54:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe231600640 1 -- 172.21.15.114:0/2963300042 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2340b2310 con 0x555cf7619490 2024-09-21T06:54:01.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.149+0000 7fe231600640 1 -- 172.21.15.114:0/2963300042 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340a9490 con 0x555cf7619490 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 -- 172.21.15.114:0/2963300042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 msgr2=0x555cf79f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2340ad870 tx=0x7fe234005760 comp rx=0 tx=0).stop 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 -- 172.21.15.114:0/2963300042 shutdown_connections 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 --2- 172.21.15.114:0/2963300042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79f51c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 -- 172.21.15.114:0/2963300042 >> 172.21.15.114:0/2963300042 conn(0x555cf7636740 msgr2=0x555cf79e75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.150+0000 7fe24e628740 1 -- 172.21.15.114:0/2963300042 shutdown_connections 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.151+0000 7fe24e628740 1 -- 172.21.15.114:0/2963300042 wait complete. 2024-09-21T06:54:01.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.151+0000 7fe24e628740 1 -- start start 2024-09-21T06:54:01.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.151+0000 7fe24e628740 1 Processor -- start 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79fa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401b8780 con 0x555cf7621670 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401b80b0 con 0x555cf7619490 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401b52f0 con 0x555cf7642350 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.152+0000 7fe24e628740 10 client.0 _ll_register_callbacks cb 0x7fe23400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49328/0 (socket says 172.21.15.114:49328) 2024-09-21T06:54:01.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 -- 172.21.15.114:0/3009490704 learned_addr learned my addr 172.21.15.114:0/3009490704 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245a00640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 -- 172.21.15.114:0/3009490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 msgr2=0x555cf79fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79fa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 -- 172.21.15.114:0/3009490704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 msgr2=0x555cf79fdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79fdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 -- 172.21.15.114:0/3009490704 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf79fe650 con 0x555cf7619490 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe23fe00640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79fdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe245000640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79fd960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2340a9010 tx=0x7fe234000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f31db96e7c80fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.153+0000 7fe226a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:01.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe226a00640 1 -- 172.21.15.114:0/3009490704 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2340c8060 con 0x555cf7619490 2024-09-21T06:54:01.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe226a00640 1 -- 172.21.15.114:0/3009490704 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe2340ac5b0 con 0x555cf7619490 2024-09-21T06:54:01.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2340b2310 con 0x555cf7619490 2024-09-21T06:54:01.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe226a00640 1 -- 172.21.15.114:0/3009490704 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe234010720 con 0x555cf7619490 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe24e628740 10 client.25541 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555cf7a72d50 con 0x555cf7619490 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe226a00640 1 -- 172.21.15.114:0/3009490704 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2340b2310 con 0x555cf7619490 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe226a00640 1 -- 172.21.15.114:0/3009490704 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe2340ad970 con 0x555cf7619490 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe24e628740 10 client.25541 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe24e628740 10 client.25541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.154+0000 7fe24e628740 20 client.25541 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.155+0000 7fe24e628740 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf75d6080 0x555cf7a75220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.155+0000 7fe24e628740 4 client.25541 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.155+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555cf795d890 con 0x555cf75d6080 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.155+0000 7fe23fe00640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf75d6080 0x555cf7a75220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.155+0000 7fe23fe00640 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf75d6080 0x555cf7a75220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4e6d563c9f3b24 server_cookie=f42e3ba3bde5f603 in_seq=0 out_seq=0 2024-09-21T06:54:01.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.155+0000 7fe226a00640 10 client.25541 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:01.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.157+0000 7fe226a00640 1 -- 172.21.15.114:0/3009490704 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555cf795d890 con 0x555cf75d6080 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.157+0000 7fe226a00640 10 client.25541 handle_command_reply: tid=0 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.157+0000 7fe24e628740 1 client.25541 shutdown 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.157+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 msgr2=0x555cf79fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.157+0000 7fe24e628740 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79fd960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe2340a9010 tx=0x7fe234000bc0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 shutdown_connections 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555cf75d6080 0x555cf7a75220 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555cf7642350 0x555cf79fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555cf7621670 0x555cf79fdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe226a00640 0 client.25541 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 --2- 172.21.15.114:0/3009490704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555cf7619490 0x555cf79fd960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 >> 172.21.15.114:0/3009490704 conn(0x555cf7636740 msgr2=0x555cf79f6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 shutdown_connections 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 1 -- 172.21.15.114:0/3009490704 wait complete. 2024-09-21T06:54:01.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.158+0000 7fe24e628740 20 client.25541 trim_cache size 0 max 16384 2024-09-21T06:54:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:01.161 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T06:54:01.161 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "8394af23-1a6b-44ac-a980-7631419f7455", 2024-09-21T06:54:01.161 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T06:54:01.161 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2280b6470 msgr2=0x7fe2280b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2280b6470 0x7fe2280b8980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe22c031440 tx=0x7fe22c005a20 comp rx=0 tx=0).stop 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 msgr2=0x7fe2401a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2401a0960 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe23400e9f0 tx=0x7fe23400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 shutdown_connections 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2280b6470 0x7fe2280b8980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe240114280 0x7fe2401a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe240077380 0x7fe2401a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 --2- 172.21.15.114:0/2362377473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe240075dd0 0x7fe2401a0960 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 >> 172.21.15.114:0/2362377473 conn(0x7fe2400ff440 msgr2=0x7fe240110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 shutdown_connections 2024-09-21T06:54:01.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.159+0000 7fe246400640 1 -- 172.21.15.114:0/2362377473 wait complete. 2024-09-21T06:54:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:00 smithi193 ceph-mon[34601]: from='client.15516 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:01.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.166+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fea68063800 con 0x7fea4c0b6560 2024-09-21T06:54:01.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.241+0000 7fea64c00640 1 -- 172.21.15.114:0/1290781749 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fea68063800 con 0x7fea4c0b6560 2024-09-21T06:54:01.243 INFO:teuthology.orchestra.run.smithi114.stdout:Retention added to path / 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea4c0b6560 msgr2=0x7fea4c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea4c0b6560 0x7fea4c0b8a70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea580096c0 tx=0x7fea58009260 comp rx=0 tx=0).stop 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6810b440 msgr2=0x7fea6819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6810b440 0x7fea6819ca30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea5002a860 tx=0x7fea50032d50 comp rx=0 tx=0).stop 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 shutdown_connections 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea4c0b6560 0x7fea4c0b8a70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea6810bd90 0x7fea681a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea6810b440 0x7fea6819ca30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 --2- 172.21.15.114:0/1290781749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea68105b10 0x7fea6819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.243+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 >> 172.21.15.114:0/1290781749 conn(0x7fea68101550 msgr2=0x7fea68103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.244+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 shutdown_connections 2024-09-21T06:54:01.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.244+0000 7fea6d200640 1 -- 172.21.15.114:0/1290781749 wait complete. 2024-09-21T06:54:01.268 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T06:54:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 -- 172.21.15.114:0/57116218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 msgr2=0x7f15b0109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 --2- 172.21.15.114:0/57116218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b0109840 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15a0009900 tx=0x7f15a002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 -- 172.21.15.114:0/57116218 shutdown_connections 2024-09-21T06:54:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 --2- 172.21.15.114:0/57116218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b0109d80 0x7f15b01105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 --2- 172.21.15.114:0/57116218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b0109840 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 --2- 172.21.15.114:0/57116218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b0103b00 0x7f15b0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 -- 172.21.15.114:0/57116218 >> 172.21.15.114:0/57116218 conn(0x7f15b00ff500 msgr2=0x7f15b0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 -- 172.21.15.114:0/57116218 shutdown_connections 2024-09-21T06:54:01.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.483+0000 7f15b8800640 1 -- 172.21.15.114:0/57116218 wait complete. 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.484+0000 7f15b8800640 1 Processor -- start 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.484+0000 7f15b8800640 1 -- start start 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b8800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b0103b00 0x7f15b019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b8800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b8800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b0109d80 0x7f15b019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b8800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0113a40 con 0x7f15b0109430 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b8800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15b01138c0 con 0x7f15b0109d80 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b8800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0113bc0 con 0x7f15b0103b00 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47848/0 (socket says 172.21.15.114:47848) 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b0103b00 0x7f15b019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b5400640 1 -- 172.21.15.114:0/1655322980 learned_addr learned my addr 172.21.15.114:0/1655322980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.485+0000 7f15b6800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b0109d80 0x7f15b019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b5400640 1 -- 172.21.15.114:0/1655322980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b0103b00 msgr2=0x7f15b019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b5400640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b0103b00 0x7f15b019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b5400640 1 -- 172.21.15.114:0/1655322980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b0109d80 msgr2=0x7f15b019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b5400640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b0109d80 0x7f15b019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b5400640 1 -- 172.21.15.114:0/1655322980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b01a03f0 con 0x7f15b0109430 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b5400640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b019aa60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f15a002a910 tx=0x7f15a0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a003f030 con 0x7f15b0109430 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a00043d0 con 0x7f15b0109430 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01a0680 con 0x7f15b0109430 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a0035070 con 0x7f15b0109430 2024-09-21T06:54:01.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.486+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b01a6e70 con 0x7f15b0109430 2024-09-21T06:54:01.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.487+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b0103f10 con 0x7f15b0109430 2024-09-21T06:54:01.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.491+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 23) ==== 154530+0+0 (secure 0 0 0) 0x7f15a0042030 con 0x7f15b0109430 2024-09-21T06:54:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.493+0000 7f15aaa00640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15980b6560 0x7f15980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.493+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f15a0107f10 con 0x7f15b0109430 2024-09-21T06:54:01.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.493+0000 7f15b5e00640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15980b6560 0x7f15980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.494+0000 7f15b5e00640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15980b6560 0x7f15980b8a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009bd0 tx=0x7f15a4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.495+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15a00ce2b0 con 0x7f15b0109430 2024-09-21T06:54:01.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.712+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f15a00c9e20 con 0x7f15b0109430 2024-09-21T06:54:01.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.717+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f15b0063800 con 0x7f15980b6560 2024-09-21T06:54:01.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.725+0000 7f15aaa00640 1 -- 172.21.15.114:0/1655322980 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f15b0063800 con 0x7f15980b6560 2024-09-21T06:54:01.727 INFO:teuthology.orchestra.run.smithi114.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T06:54:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T06:54:01.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15980b6560 msgr2=0x7f15980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15980b6560 0x7f15980b8a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009bd0 tx=0x7f15a4005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 msgr2=0x7f15b019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b019aa60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f15a002a910 tx=0x7f15a0031a20 comp rx=0 tx=0).stop 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 shutdown_connections 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15980b6560 0x7f15980b8a70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b0109d80 0x7f15b019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15b0109430 0x7f15b019aa60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 --2- 172.21.15.114:0/1655322980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b0103b00 0x7f15b019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 >> 172.21.15.114:0/1655322980 conn(0x7f15b00ff500 msgr2=0x7f15b0106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 shutdown_connections 2024-09-21T06:54:01.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.728+0000 7f15b8800640 1 -- 172.21.15.114:0/1655322980 wait complete. 2024-09-21T06:54:01.755 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T06:54:01.919 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:01 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T06:54:01.919 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:01.919 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:54:01.919 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:01.920 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:01 smithi114 ceph-mon[30654]: from='client.25531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:01.920 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:01 smithi114 ceph-mon[30654]: pgmap v393: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T06:54:01.962 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 -- 172.21.15.114:0/1751239758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 msgr2=0x7f24dc103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 --2- 172.21.15.114:0/1751239758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc103d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009900 tx=0x7f24cc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 -- 172.21.15.114:0/1751239758 shutdown_connections 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 --2- 172.21.15.114:0/1751239758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24dc109bb0 0x7f24dc1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 --2- 172.21.15.114:0/1751239758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc109260 0x7f24dc109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 --2- 172.21.15.114:0/1751239758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc103d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.981+0000 7f24e0c00640 1 -- 172.21.15.114:0/1751239758 >> 172.21.15.114:0/1751239758 conn(0x7f24dc0ff390 msgr2=0x7f24dc1017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:01.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.982+0000 7f24e0c00640 1 -- 172.21.15.114:0/1751239758 shutdown_connections 2024-09-21T06:54:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.982+0000 7f24e0c00640 1 -- 172.21.15.114:0/1751239758 wait complete. 2024-09-21T06:54:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 Processor -- start 2024-09-21T06:54:01.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 -- start start 2024-09-21T06:54:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc19a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc109260 0x7f24dc19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24dc109bb0 0x7f24dc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc1138c0 con 0x7f24dc103930 2024-09-21T06:54:01.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc113740 con 0x7f24dc109260 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24e0c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc113a40 con 0x7f24dc109bb0 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.983+0000 7f24db400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc19a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc19a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47854/0 (socket says 172.21.15.114:47854) 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 -- 172.21.15.114:0/2548300980 learned_addr learned my addr 172.21.15.114:0/2548300980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24d3e00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc109260 0x7f24dc19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24dbe00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24dc109bb0 0x7f24dc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 -- 172.21.15.114:0/2548300980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24dc109bb0 msgr2=0x7f24dc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24dc109bb0 0x7f24dc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 -- 172.21.15.114:0/2548300980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc109260 msgr2=0x7f24dc19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc109260 0x7f24dc19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:01.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 -- 172.21.15.114:0/2548300980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc1a0210 con 0x7f24dc103930 2024-09-21T06:54:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.984+0000 7f24db400640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc19a2e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f24cc005ab0 tx=0x7f24cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.985+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc03f070 con 0x7f24dc103930 2024-09-21T06:54:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.985+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24cc031b60 con 0x7f24dc103930 2024-09-21T06:54:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.985+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc035030 con 0x7f24dc103930 2024-09-21T06:54:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.985+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc1a04a0 con 0x7f24dc103930 2024-09-21T06:54:01.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.985+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc1a0940 con 0x7f24dc103930 2024-09-21T06:54:01.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.986+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc103d40 con 0x7f24dc103930 2024-09-21T06:54:01.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.988+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f24cc042070 con 0x7f24dc103930 2024-09-21T06:54:01.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.990+0000 7f24d8c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24c00b6670 0x7f24c00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:01.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.990+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f24cc107ff0 con 0x7f24dc103930 2024-09-21T06:54:01.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.990+0000 7f24d3e00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24c00b6670 0x7f24c00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:01.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.991+0000 7f24d3e00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24c00b6670 0x7f24c00b8b80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24dc108d30 tx=0x7f24c4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:01.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:01.994+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f24dc1a0940 con 0x7f24dc103930 2024-09-21T06:54:02.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:01 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T06:54:02.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:54:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:01 smithi183 ceph-mon[34219]: from='client.25531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:01 smithi183 ceph-mon[34219]: pgmap v393: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T06:54:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:01 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T06:54:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:54:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:01 smithi193 ceph-mon[34601]: from='client.25531 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:01 smithi193 ceph-mon[34601]: pgmap v393: 129 pgs: 129 active+clean; 655 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T06:54:02.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.212+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f24dc063800 con 0x7f24c00b6670 2024-09-21T06:54:02.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.221+0000 7f24d8c00640 1 -- 172.21.15.114:0/2548300980 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f24dc063800 con 0x7f24c00b6670 2024-09-21T06:54:02.223 INFO:teuthology.orchestra.run.smithi114.stdout:/ 1m 6m3h 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24c00b6670 msgr2=0x7f24c00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24c00b6670 0x7f24c00b8b80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f24dc108d30 tx=0x7f24c4005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 msgr2=0x7f24dc19a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc19a2e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f24cc005ab0 tx=0x7f24cc003c10 comp rx=0 tx=0).stop 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 shutdown_connections 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24dc109bb0 0x7f24dc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24c00b6670 0x7f24c00b8b80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24dc109260 0x7f24dc19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 --2- 172.21.15.114:0/2548300980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24dc103930 0x7f24dc19a2e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 >> 172.21.15.114:0/2548300980 conn(0x7f24dc0ff390 msgr2=0x7f24dc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 shutdown_connections 2024-09-21T06:54:02.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.224+0000 7f24e0c00640 1 -- 172.21.15.114:0/2548300980 wait complete. 2024-09-21T06:54:02.253 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T06:54:02.342 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:02.358 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T06:54:02.368 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T06:54:02.393 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:54:02.393 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T06:54:02.398 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-21T06:54:02.429 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T06:54:02.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:54:02.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 -- 172.21.15.114:0/1154649980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e8109e00 msgr2=0x7f07e8110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 --2- 172.21.15.114:0/1154649980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e8109e00 0x7f07e8110630 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009ca0 tx=0x7f07e002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.950 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:02 smithi114 ceph-mon[30654]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T06:54:02.950 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:02 smithi114 ceph-mon[30654]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:02.950 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:02 smithi114 ceph-mon[30654]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:02.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 -- 172.21.15.114:0/1154649980 shutdown_connections 2024-09-21T06:54:02.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 --2- 172.21.15.114:0/1154649980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e8109e00 0x7f07e8110630 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 --2- 172.21.15.114:0/1154649980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07e81094b0 0x7f07e81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 --2- 172.21.15.114:0/1154649980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.947+0000 7f07ef600640 1 -- 172.21.15.114:0/1154649980 >> 172.21.15.114:0/1154649980 conn(0x7f07e80ff5e0 msgr2=0x7f07e8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:02.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.948+0000 7f07ef600640 1 -- 172.21.15.114:0/1154649980 shutdown_connections 2024-09-21T06:54:02.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.948+0000 7f07ef600640 1 -- 172.21.15.114:0/1154649980 wait complete. 2024-09-21T06:54:02.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.949+0000 7f07ef600640 1 Processor -- start 2024-09-21T06:54:02.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 -- start start 2024-09-21T06:54:02.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e81094b0 0x7f07e8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07e8109e00 0x7f07e8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8113ac0 con 0x7f07e81094b0 2024-09-21T06:54:02.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8113940 con 0x7f07e8103b70 2024-09-21T06:54:02.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.950+0000 7f07ef600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07e8113c40 con 0x7f07e8109e00 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49374/0 (socket says 172.21.15.114:49374) 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 -- 172.21.15.114:0/1822634072 learned_addr learned my addr 172.21.15.114:0/1822634072 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07e7e00640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e81094b0 0x7f07e8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ed600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07e8109e00 0x7f07e8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 -- 172.21.15.114:0/1822634072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07e8109e00 msgr2=0x7f07e8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07e8109e00 0x7f07e8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 -- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e81094b0 msgr2=0x7f07e8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e81094b0 0x7f07e8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 -- 172.21.15.114:0/1822634072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e81a8730 con 0x7f07e8103b70 2024-09-21T06:54:02.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.951+0000 7f07ecc00640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00e9f0 tx=0x7f07dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:02.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.952+0000 7f07e5600640 1 -- 172.21.15.114:0/1822634072 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00cd80 con 0x7f07e8103b70 2024-09-21T06:54:02.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.952+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e81a8a20 con 0x7f07e8103b70 2024-09-21T06:54:02.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.952+0000 7f07e5600640 1 -- 172.21.15.114:0/1822634072 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07dc0108e0 con 0x7f07e8103b70 2024-09-21T06:54:02.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.952+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e81a8fb0 con 0x7f07e8103b70 2024-09-21T06:54:02.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.952+0000 7f07e5600640 1 -- 172.21.15.114:0/1822634072 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0188e0 con 0x7f07e8103b70 2024-09-21T06:54:02.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.955+0000 7f07f63fa740 1 Processor -- start 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 -- start start 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c996d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b93c90df70 con 0x55b93c5cf390 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b93c8fd3a0 con 0x55b93c5c71b0 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07f63fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b93c9011b0 con 0x55b93c5f0070 2024-09-21T06:54:02.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07e5600640 1 -- 172.21.15.114:0/1822634072 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f07dc018a80 con 0x7f07e8103b70 2024-09-21T06:54:02.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07ed600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c996d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07ed600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c996d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55970/0 (socket says 172.21.15.114:55970) 2024-09-21T06:54:02.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.956+0000 7f07ed600640 1 -- 172.21.15.114:0/2624102282 learned_addr learned my addr 172.21.15.114:0/2624102282 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:02.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07ecc00640 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07ecc00640 1 -- 172.21.15.114:0/2624102282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 msgr2=0x55b93c996d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07ecc00640 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c996d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07ecc00640 1 -- 172.21.15.114:0/2624102282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 msgr2=0x55b93c9977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07ecc00640 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07ecc00640 1 -- 172.21.15.114:0/2624102282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93c8c7c90 con 0x55b93c5c71b0 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.957+0000 7f07e7e00640 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:02.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.958+0000 7f07ecc00640 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9972a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0e4f20 tx=0x7f07dc0f0190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d330fa7d1c59c840 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.958+0000 7f07d9c00640 1 -- 172.21.15.114:0/2624102282 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0ee070 con 0x55b93c5c71b0 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.958+0000 7f07d9c00640 1 -- 172.21.15.114:0/2624102282 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07dc0f0da0 con 0x55b93c5c71b0 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.958+0000 7f07d9c00640 1 -- 172.21.15.114:0/2624102282 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0f4690 con 0x55b93c5c71b0 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.958+0000 7f07e5600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07d00b6670 0x7f07d00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.958+0000 7f07e5600640 1 -- 172.21.15.114:0/1822634072 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f07dc0e7a00 con 0x7f07e8103b70 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.959+0000 7f07ed600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07d00b6670 0x7f07d00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.959+0000 7f07f63fa740 1 -- 172.21.15.114:0/2624102282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 msgr2=0x55b93c9972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.959+0000 7f07f63fa740 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9972a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0e4f20 tx=0x7f07dc0f0190 comp rx=0 tx=0).stop 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 -- 172.21.15.114:0/2624102282 shutdown_connections 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9972a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 --2- 172.21.15.114:0/2624102282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c996d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 -- 172.21.15.114:0/2624102282 >> 172.21.15.114:0/2624102282 conn(0x55b93c5e4460 msgr2=0x55b93c994c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07ed600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07d00b6670 0x7f07d00b8b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07e002f500 tx=0x7f07e002f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 -- 172.21.15.114:0/2624102282 shutdown_connections 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 -- 172.21.15.114:0/2624102282 wait complete. 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.960+0000 7f07f63fa740 1 -- start start 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 Processor -- start 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5c71b0 0x55b93c9ab640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5f0070 0x55b93c9b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b93c75e700 con 0x55b93c5cf390 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b93c898540 con 0x55b93c5f0070 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b93c9a5540 con 0x55b93c5c71b0 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07f63fa740 10 client.0 _ll_register_callbacks cb 0x55b93c8a8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07ed600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5f0070 0x55b93c9b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9abc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47892/0 (socket says 172.21.15.114:47892) 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.961+0000 7f07e7e00640 1 -- 172.21.15.114:0/3762651846 learned_addr learned my addr 172.21.15.114:0/3762651846 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07ecc00640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5c71b0 0x55b93c9ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07e7e00640 1 -- 172.21.15.114:0/3762651846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5c71b0 msgr2=0x55b93c9ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07e7e00640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5c71b0 0x55b93c9ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07e7e00640 1 -- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5f0070 msgr2=0x55b93c9b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07e7e00640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5f0070 0x55b93c9b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07e7e00640 1 -- 172.21.15.114:0/3762651846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93c65b650 con 0x55b93c5cf390 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07ed600640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5f0070 0x55b93c9b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07e7e00640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9abc20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07d40126e0 tx=0x7f07d4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38421a42168eaaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07ce000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.962+0000 7f07ce000640 1 -- 172.21.15.114:0/3762651846 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4012190 con 0x55b93c5cf390 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.963+0000 7f07ce000640 1 -- 172.21.15.114:0/3762651846 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f07d400f920 con 0x55b93c5cf390 2024-09-21T06:54:02.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.963+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b93c89ab50 con 0x55b93c5cf390 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.963+0000 7f07ce000640 1 -- 172.21.15.114:0/3762651846 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07d400fb20 con 0x55b93c5cf390 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.963+0000 7f07f63fa740 10 client.15570 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.963+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b93c8b5910 con 0x55b93c5cf390 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.963+0000 7f07ce000640 1 -- 172.21.15.114:0/3762651846 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b93c89ab50 con 0x55b93c5cf390 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07ce000640 1 -- 172.21.15.114:0/3762651846 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f07d40051e0 con 0x55b93c5cf390 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07f63fa740 10 client.15570 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07f63fa740 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07f63fa740 20 client.15570 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07f63fa740 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c5ae9f0 0x55b93c9bcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07f63fa740 4 client.15570 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93c90fec0 con 0x55b93c5ae9f0 2024-09-21T06:54:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.964+0000 7f07ecc00640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c5ae9f0 0x55b93c9bcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.965+0000 7f07ce000640 10 client.15570 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:02.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.965+0000 7f07ecc00640 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c5ae9f0 0x55b93c9bcf90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e06f9a4a002395 server_cookie=f9f2053b98a709dd in_seq=0 out_seq=0 2024-09-21T06:54:02.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07ce000640 1 -- 172.21.15.114:0/3762651846 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b93c90fec0 con 0x55b93c5ae9f0 2024-09-21T06:54:02.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07ce000640 10 client.15570 handle_command_reply: tid=0 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 client.15570 shutdown 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 msgr2=0x55b93c9abc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9abc20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07d40126e0 tx=0x7f07d4012ba0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 shutdown_connections 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5f0070 0x55b93c9b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5cf390 0x55b93c9abc20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c5ae9f0 0x55b93c9bcf90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 --2- 172.21.15.114:0/3762651846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5c71b0 0x55b93c9ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 >> 172.21.15.114:0/3762651846 conn(0x55b93c5e4460 msgr2=0x55b93c999ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07ce000640 0 client.15570 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 shutdown_connections 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.966+0000 7f07f63fa740 1 -- 172.21.15.114:0/3762651846 wait complete. 2024-09-21T06:54:02.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.967+0000 7f07f63fa740 20 client.15570 trim_cache size 0 max 16384 2024-09-21T06:54:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.986+0000 7f07f63fa740 1 Processor -- start 2024-09-21T06:54:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.986+0000 7f07f63fa740 1 -- start start 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 0x55b93c9aa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5cf390 0x55b93c9b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c9a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07f63fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07c800fec0 con 0x55b93c5c71b0 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07f63fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8011a00 con 0x55b93c5cf390 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07f63fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8010590 con 0x55b93c5f0070 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07ecc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 0x55b93c9aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07ecc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 0x55b93c9aa330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47908/0 (socket says 172.21.15.114:47908) 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5cf390 0x55b93c9b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07ecc00640 1 -- 172.21.15.114:0/151369065 learned_addr learned my addr 172.21.15.114:0/151369065 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.987+0000 7f07ed600640 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c9a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07e7e00640 1 -- 172.21.15.114:0/151369065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 msgr2=0x55b93c9a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07e7e00640 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c9a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07e7e00640 1 -- 172.21.15.114:0/151369065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 msgr2=0x55b93c9aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07e7e00640 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 0x55b93c9aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07e7e00640 1 -- 172.21.15.114:0/151369065 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93c999050 con 0x55b93c5cf390 2024-09-21T06:54:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07ecc00640 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 0x55b93c9aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07e7e00640 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5cf390 0x55b93c9b12c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005600 tx=0x7f07d4005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c80750f590b20a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07d9200640 1 -- 172.21.15.114:0/151369065 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4012190 con 0x55b93c5cf390 2024-09-21T06:54:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07d9200640 1 -- 172.21.15.114:0/151369065 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d400fcc0 con 0x55b93c5cf390 2024-09-21T06:54:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.988+0000 7f07d9200640 1 -- 172.21.15.114:0/151369065 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003c30 con 0x55b93c5cf390 2024-09-21T06:54:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 -- 172.21.15.114:0/151369065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5cf390 msgr2=0x55b93c9b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5cf390 0x55b93c9b12c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f07d4005600 tx=0x7f07d4005db0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 -- 172.21.15.114:0/151369065 shutdown_connections 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5f0070 0x55b93c9a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5cf390 0x55b93c9b12c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 --2- 172.21.15.114:0/151369065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5c71b0 0x55b93c9aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 -- 172.21.15.114:0/151369065 >> 172.21.15.114:0/151369065 conn(0x55b93c5e4460 msgr2=0x55b93c9b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.989+0000 7f07f63fa740 1 -- 172.21.15.114:0/151369065 shutdown_connections 2024-09-21T06:54:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.990+0000 7f07f63fa740 1 -- 172.21.15.114:0/151369065 wait complete. 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.990+0000 7f07f63fa740 1 -- start start 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.990+0000 7f07f63fa740 1 Processor -- start 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 0x55b93c9af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5f0070 0x55b93c9b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8011330 con 0x55b93c5f0070 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8010c60 con 0x55b93c5c71b0 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c800dea0 con 0x55b93c5cf390 2024-09-21T06:54:02.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07f63fa740 10 client.0 _ll_register_callbacks cb 0x7f07d4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:02.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 0x55b93c9af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 0x55b93c9af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55980/0 (socket says 172.21.15.114:55980) 2024-09-21T06:54:02.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07e7e00640 1 -- 172.21.15.114:0/755182641 learned_addr learned my addr 172.21.15.114:0/755182641 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:02.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07ed600640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5f0070 0x55b93c9b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.991+0000 7f07ecc00640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07e7e00640 1 -- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 msgr2=0x55b93c9af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07e7e00640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07e7e00640 1 -- 172.21.15.114:0/755182641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5f0070 msgr2=0x55b93c9b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:02.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07e7e00640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5f0070 0x55b93c9b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07e7e00640 1 -- 172.21.15.114:0/755182641 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93c9b7240 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07e7e00640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 0x55b93c9af730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07d401da00 tx=0x7f07d401da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f28b49df649196 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07ce000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.992+0000 7f07ce000640 1 -- 172.21.15.114:0/755182641 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400fe70 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07ce000640 1 -- 172.21.15.114:0/755182641 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f07d4012190 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d400fcc0 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07ce000640 1 -- 172.21.15.114:0/755182641 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f07d403f050 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 10 client.25579 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b93ca27ed0 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07ce000640 1 -- 172.21.15.114:0/755182641 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d400fcc0 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07ce000640 1 -- 172.21.15.114:0/755182641 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f07d4003ba0 con 0x55b93c5cf390 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 10 client.25579 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 20 client.25579 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:02.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c583da0 0x55b93ca2a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:02.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 4 client.25579 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:54:02.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.993+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b93c90b970 con 0x55b93c583da0 2024-09-21T06:54:02.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.994+0000 7f07ecc00640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c583da0 0x55b93ca2a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.994+0000 7f07ecc00640 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c583da0 0x55b93ca2a590 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e0cb8a3522ac2e4 server_cookie=9b5ba585ce33d5a3 in_seq=0 out_seq=0 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.994+0000 7f07ce000640 10 client.25579 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07ce000640 1 -- 172.21.15.114:0/755182641 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55b93c90b970 con 0x55b93c583da0 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07ce000640 10 client.25579 handle_command_reply: tid=0 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 client.25579 shutdown 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 msgr2=0x55b93c9af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 0x55b93c9af730 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07d401da00 tx=0x7f07d401da30 comp rx=0 tx=0).stop 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 shutdown_connections 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b93c5f0070 0x55b93c9b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b93c583da0 0x55b93ca2a590 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b93c5cf390 0x55b93c9af730 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 --2- 172.21.15.114:0/755182641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b93c5c71b0 0x55b93c9af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 >> 172.21.15.114:0/755182641 conn(0x55b93c5e4460 msgr2=0x55b93c999e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07ce000640 0 client.25579 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 shutdown_connections 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 1 -- 172.21.15.114:0/755182641 wait complete. 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.995+0000 7f07f63fa740 20 client.25579 trim_cache size 0 max 16384 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T06:54:02.999 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "8394af23-1a6b-44ac-a980-7631419f7455", 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:03.000 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07d00b6670 msgr2=0x7f07d00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07d00b6670 0x7f07d00b8b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f07e002f500 tx=0x7f07e002f310 comp rx=0 tx=0).stop 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 msgr2=0x7f07e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8075710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00e9f0 tx=0x7f07dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 shutdown_connections 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07e8109e00 0x7f07e8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:03.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07e81094b0 0x7f07e8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:03.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07d00b6670 0x7f07d00b8b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:03.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 --2- 172.21.15.114:0/1822634072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e8103b70 0x7f07e8075710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:03.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 >> 172.21.15.114:0/1822634072 conn(0x7f07e80ff5e0 msgr2=0x7f07e8108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:03.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 shutdown_connections 2024-09-21T06:54:03.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:02.996+0000 7f07ef600640 1 -- 172.21.15.114:0/1822634072 wait complete. 2024-09-21T06:54:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:02 smithi183 ceph-mon[34219]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T06:54:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:02 smithi183 ceph-mon[34219]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:02 smithi183 ceph-mon[34219]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:02 smithi193 ceph-mon[34601]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-09-21T06:54:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:02 smithi193 ceph-mon[34601]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:02 smithi193 ceph-mon[34601]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:54:03.396 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:03.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:03 smithi114 ceph-mon[30654]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T06:54:03.941 INFO:tasks.cephfs.filesystem:scrub status for tag:8394af23-1a6b-44ac-a980-7631419f7455 - {'path': '/', 'tag': '8394af23-1a6b-44ac-a980-7631419f7455', 'options': 'recursive,force'} 2024-09-21T06:54:04.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:03 smithi183 ceph-mon[34219]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T06:54:04.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:03 smithi193 ceph-mon[34601]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T06:54:04.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 -- 172.21.15.114:0/346040014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8810de30 msgr2=0x7fbf88110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 --2- 172.21.15.114:0/346040014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8810de30 0x7fbf88110320 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009ca0 tx=0x7fbf8002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 -- 172.21.15.114:0/346040014 shutdown_connections 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 --2- 172.21.15.114:0/346040014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf8810de30 0x7fbf88110320 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 --2- 172.21.15.114:0/346040014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf88105320 0x7fbf8810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 --2- 172.21.15.114:0/346040014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf881049d0 0x7fbf88104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 -- 172.21.15.114:0/346040014 >> 172.21.15.114:0/346040014 conn(0x7fbf88079740 msgr2=0x7fbf88079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 -- 172.21.15.114:0/346040014 shutdown_connections 2024-09-21T06:54:04.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.471+0000 7fbf8f800640 1 -- 172.21.15.114:0/346040014 wait complete. 2024-09-21T06:54:04.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.472+0000 7fbf8f800640 1 Processor -- start 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 -- start start 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf881049d0 0x7fbf88102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 0x7fbf88102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf8810de30 0x7fbf88103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88113a90 con 0x7fbf88105320 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88113910 con 0x7fbf881049d0 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8f800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88113c10 con 0x7fbf8810de30 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 0x7fbf88102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf87e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 0x7fbf88102c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47948/0 (socket says 172.21.15.114:47948) 2024-09-21T06:54:04.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf881049d0 0x7fbf88102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf87e00640 1 -- 172.21.15.114:0/1631299801 learned_addr learned my addr 172.21.15.114:0/1631299801 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.473+0000 7fbf8d800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf8810de30 0x7fbf88103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf87e00640 1 -- 172.21.15.114:0/1631299801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf8810de30 msgr2=0x7fbf88103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf87e00640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf8810de30 0x7fbf88103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf87e00640 1 -- 172.21.15.114:0/1631299801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf881049d0 msgr2=0x7fbf88102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf87e00640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf881049d0 0x7fbf88102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf87e00640 1 -- 172.21.15.114:0/1631299801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf881a87a0 con 0x7fbf88105320 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf8ce00640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf881049d0 0x7fbf88102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:04.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.474+0000 7fbf87e00640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 0x7fbf88102c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400d740 tx=0x7fbf7400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.475+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74017070 con 0x7fbf88105320 2024-09-21T06:54:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.475+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf7400de50 con 0x7fbf88105320 2024-09-21T06:54:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.475+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf881a8a90 con 0x7fbf88105320 2024-09-21T06:54:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.475+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400f620 con 0x7fbf88105320 2024-09-21T06:54:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.475+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a8fd0 con 0x7fbf88105320 2024-09-21T06:54:04.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.476+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8806c240 con 0x7fbf88105320 2024-09-21T06:54:04.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.478+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fbf7401a070 con 0x7fbf88105320 2024-09-21T06:54:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.480+0000 7fbf85600640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf700b65f0 0x7fbf700b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:04.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.480+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbf740e72c0 con 0x7fbf88105320 2024-09-21T06:54:04.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.480+0000 7fbf8ce00640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf700b65f0 0x7fbf700b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:04.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.481+0000 7fbf8ce00640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf700b65f0 0x7fbf700b8b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf88104160 tx=0x7fbf7c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:04.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.482+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf740b18e0 con 0x7fbf88105320 2024-09-21T06:54:04.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.670+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf88100c90 con 0x7fbf700b65f0 2024-09-21T06:54:04.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.678+0000 7fbf85600640 1 -- 172.21.15.114:0/1631299801 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340574 (secure 0 0 0) 0x7fbf88100c90 con 0x7fbf700b65f0 2024-09-21T06:54:04.683 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:04.684 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:54:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf700b65f0 msgr2=0x7fbf700b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf700b65f0 0x7fbf700b8b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbf88104160 tx=0x7fbf7c00b390 comp rx=0 tx=0).stop 2024-09-21T06:54:04.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 msgr2=0x7fbf88102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 0x7fbf88102c50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400d740 tx=0x7fbf7400dc00 comp rx=0 tx=0).stop 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 shutdown_connections 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf8810de30 0x7fbf88103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf88105320 0x7fbf88102c50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf700b65f0 0x7fbf700b8b00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 --2- 172.21.15.114:0/1631299801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf881049d0 0x7fbf88102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 >> 172.21.15.114:0/1631299801 conn(0x7fbf88079740 msgr2=0x7fbf880ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 shutdown_connections 2024-09-21T06:54:04.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:04.683+0000 7fbf8f800640 1 -- 172.21.15.114:0/1631299801 wait complete. 2024-09-21T06:54:05.063 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:04 smithi114 ceph-mon[30654]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:04 smithi183 ceph-mon[34219]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:04 smithi193 ceph-mon[34601]: from='client.15582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:05.451 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-21T06:54:03.545348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1457899,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1836,"num_read_kb":1168,"num_write":619,"num_write_kb":4407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378028,"kb_used_data":16240,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124563796,"statfs":{"total":1151940427776,"available":1151553327104,"internally_reserved":0,"allocated":16629760,"data_stored":10325747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":791341,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":454,"num_read_kb":363,"num_write":230,"num_write_kb":2096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003868"},"pg_stats":[{"pgid":"2.1f","version":"89'15","reported_seq":200,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:01.157297+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:54:01.157297+0000","last_peered":"2024-09-21T06:54:01.157297+0000","last_clean":"2024-09-21T06:54:01.157297+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:54:01.157297+0000","last_undegraded":"2024-09-21T06:54:01.157297+0000","last_fullsized":"2024-09-21T06:54:01.157297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741199+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:59.741199+0000","last_peered":"2024-09-21T06:52:59.741199+0000","last_clean":"2024-09-21T06:52:59.741199+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:59.741199+0000","last_undegraded":"2024-09-21T06:52:59.741199+0000","last_fullsized":"2024-09-21T06:52:59.741199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703671+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703671+0000","last_peered":"2024-09-21T06:52:31.703671+0000","last_clean":"2024-09-21T06:52:31.703671+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703671+0000","last_undegraded":"2024-09-21T06:52:31.703671+0000","last_fullsized":"2024-09-21T06:52:31.703671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699254+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699254+0000","last_peered":"2024-09-21T06:52:31.699254+0000","last_clean":"2024-09-21T06:52:31.699254+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699254+0000","last_undegraded":"2024-09-21T06:52:31.699254+0000","last_fullsized":"2024-09-21T06:52:31.699254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:16.554226+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:53:16.554226+0000","last_peered":"2024-09-21T06:53:16.554226+0000","last_clean":"2024-09-21T06:53:16.554226+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:53:16.554226+0000","last_undegraded":"2024-09-21T06:53:16.554226+0000","last_fullsized":"2024-09-21T06:53:16.554226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755557+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:59.755557+0000","last_peered":"2024-09-21T06:52:59.755557+0000","last_clean":"2024-09-21T06:52:59.755557+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:59.755557+0000","last_undegraded":"2024-09-21T06:52:59.755557+0000","last_fullsized":"2024-09-21T06:52:59.755557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699992+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699992+0000","last_peered":"2024-09-21T06:52:31.699992+0000","last_clean":"2024-09-21T06:52:31.699992+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699992+0000","last_undegraded":"2024-09-21T06:52:31.699992+0000","last_fullsized":"2024-09-21T06:52:31.699992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891939+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891939+0000","last_peered":"2024-09-21T06:52:31.891939+0000","last_clean":"2024-09-21T06:52:31.891939+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891939+0000","last_undegraded":"2024-09-21T06:52:31.891939+0000","last_fullsized":"2024-09-21T06:52:31.891939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751784+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:59.751784+0000","last_peered":"2024-09-21T06:52:59.751784+0000","last_clean":"2024-09-21T06:52:59.751784+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:59.751784+0000","last_undegraded":"2024-09-21T06:52:59.751784+0000","last_fullsized":"2024-09-21T06:52:59.751784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699375+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699375+0000","last_peered":"2024-09-21T06:52:31.699375+0000","last_clean":"2024-09-21T06:52:31.699375+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699375+0000","last_undegraded":"2024-09-21T06:52:31.699375+0000","last_fullsized":"2024-09-21T06:52:31.699375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755326+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:59.755326+0000","last_peered":"2024-09-21T06:52:59.755326+0000","last_clean":"2024-09-21T06:52:59.755326+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:59.755326+0000","last_undegraded":"2024-09-21T06:52:59.755326+0000","last_fullsized":"2024-09-21T06:52:59.755326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645462+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645462+0000","last_peered":"2024-09-21T06:52:31.645462+0000","last_clean":"2024-09-21T06:52:31.645462+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645462+0000","last_undegraded":"2024-09-21T06:52:31.645462+0000","last_fullsized":"2024-09-21T06:52:31.645462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755915+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:59.755915+0000","last_peered":"2024-09-21T06:52:59.755915+0000","last_clean":"2024-09-21T06:52:59.755915+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:59.755915+0000","last_undegraded":"2024-09-21T06:52:59.755915+0000","last_fullsized":"2024-09-21T06:52:59.755915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704037+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.704037+0000","last_peered":"2024-09-21T06:52:31.704037+0000","last_clean":"2024-09-21T06:52:31.704037+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.704037+0000","last_undegraded":"2024-09-21T06:52:31.704037+0000","last_fullsized":"2024-09-21T06:52:31.704037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699968+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699968+0000","last_peered":"2024-09-21T06:52:31.699968+0000","last_clean":"2024-09-21T06:52:31.699968+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699968+0000","last_undegraded":"2024-09-21T06:52:31.699968+0000","last_fullsized":"2024-09-21T06:52:31.699968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745662+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:59.745662+0000","last_peered":"2024-09-21T06:52:59.745662+0000","last_clean":"2024-09-21T06:52:59.745662+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:59.745662+0000","last_undegraded":"2024-09-21T06:52:59.745662+0000","last_fullsized":"2024-09-21T06:52:59.745662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892935+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892935+0000","last_peered":"2024-09-21T06:52:31.892935+0000","last_clean":"2024-09-21T06:52:31.892935+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892935+0000","last_undegraded":"2024-09-21T06:52:31.892935+0000","last_fullsized":"2024-09-21T06:52:31.892935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646491+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646491+0000","last_peered":"2024-09-21T06:52:31.646491+0000","last_clean":"2024-09-21T06:52:31.646491+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646491+0000","last_undegraded":"2024-09-21T06:52:31.646491+0000","last_fullsized":"2024-09-21T06:52:31.646491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744826+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:59.744826+0000","last_peered":"2024-09-21T06:52:59.744826+0000","last_clean":"2024-09-21T06:52:59.744826+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:59.744826+0000","last_undegraded":"2024-09-21T06:52:59.744826+0000","last_fullsized":"2024-09-21T06:52:59.744826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836870+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836870+0000","last_peered":"2024-09-21T06:52:31.836870+0000","last_clean":"2024-09-21T06:52:31.836870+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836870+0000","last_undegraded":"2024-09-21T06:52:31.836870+0000","last_fullsized":"2024-09-21T06:52:31.836870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755751+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:59.755751+0000","last_peered":"2024-09-21T06:52:59.755751+0000","last_clean":"2024-09-21T06:52:59.755751+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:59.755751+0000","last_undegraded":"2024-09-21T06:52:59.755751+0000","last_fullsized":"2024-09-21T06:52:59.755751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700368+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700368+0000","last_peered":"2024-09-21T06:52:31.700368+0000","last_clean":"2024-09-21T06:52:31.700368+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700368+0000","last_undegraded":"2024-09-21T06:52:31.700368+0000","last_fullsized":"2024-09-21T06:52:31.700368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703897+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703897+0000","last_peered":"2024-09-21T06:52:31.703897+0000","last_clean":"2024-09-21T06:52:31.703897+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703897+0000","last_undegraded":"2024-09-21T06:52:31.703897+0000","last_fullsized":"2024-09-21T06:52:31.703897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741893+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:59.741893+0000","last_peered":"2024-09-21T06:52:59.741893+0000","last_clean":"2024-09-21T06:52:59.741893+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:59.741893+0000","last_undegraded":"2024-09-21T06:52:59.741893+0000","last_fullsized":"2024-09-21T06:52:59.741893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:02.219221+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:54:02.219221+0000","last_peered":"2024-09-21T06:54:02.219221+0000","last_clean":"2024-09-21T06:54:02.219221+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:54:02.219221+0000","last_undegraded":"2024-09-21T06:54:02.219221+0000","last_fullsized":"2024-09-21T06:54:02.219221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891910+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891910+0000","last_peered":"2024-09-21T06:52:31.891910+0000","last_clean":"2024-09-21T06:52:31.891910+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891910+0000","last_undegraded":"2024-09-21T06:52:31.891910+0000","last_fullsized":"2024-09-21T06:52:31.891910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742064+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:59.742064+0000","last_peered":"2024-09-21T06:52:59.742064+0000","last_clean":"2024-09-21T06:52:59.742064+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:59.742064+0000","last_undegraded":"2024-09-21T06:52:59.742064+0000","last_fullsized":"2024-09-21T06:52:59.742064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892041+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892041+0000","last_peered":"2024-09-21T06:52:31.892041+0000","last_clean":"2024-09-21T06:52:31.892041+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892041+0000","last_undegraded":"2024-09-21T06:52:31.892041+0000","last_fullsized":"2024-09-21T06:52:31.892041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645535+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645535+0000","last_peered":"2024-09-21T06:52:31.645535+0000","last_clean":"2024-09-21T06:52:31.645535+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645535+0000","last_undegraded":"2024-09-21T06:52:31.645535+0000","last_fullsized":"2024-09-21T06:52:31.645535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754090+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:59.754090+0000","last_peered":"2024-09-21T06:52:59.754090+0000","last_clean":"2024-09-21T06:52:59.754090+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:59.754090+0000","last_undegraded":"2024-09-21T06:52:59.754090+0000","last_fullsized":"2024-09-21T06:52:59.754090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645977+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645977+0000","last_peered":"2024-09-21T06:52:31.645977+0000","last_clean":"2024-09-21T06:52:31.645977+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645977+0000","last_undegraded":"2024-09-21T06:52:31.645977+0000","last_fullsized":"2024-09-21T06:52:31.645977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891517+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891517+0000","last_peered":"2024-09-21T06:52:31.891517+0000","last_clean":"2024-09-21T06:52:31.891517+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891517+0000","last_undegraded":"2024-09-21T06:52:31.891517+0000","last_fullsized":"2024-09-21T06:52:31.891517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758656+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:59.758656+0000","last_peered":"2024-09-21T06:52:59.758656+0000","last_clean":"2024-09-21T06:52:59.758656+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:59.758656+0000","last_undegraded":"2024-09-21T06:52:59.758656+0000","last_fullsized":"2024-09-21T06:52:59.758656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'8","reported_seq":192,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:57.837491+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:53:57.837491+0000","last_peered":"2024-09-21T06:53:57.837491+0000","last_clean":"2024-09-21T06:53:57.837491+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:53:57.837491+0000","last_undegraded":"2024-09-21T06:53:57.837491+0000","last_fullsized":"2024-09-21T06:53:57.837491+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1431,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701262+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701262+0000","last_peered":"2024-09-21T06:52:31.701262+0000","last_clean":"2024-09-21T06:52:31.701262+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701262+0000","last_undegraded":"2024-09-21T06:52:31.701262+0000","last_fullsized":"2024-09-21T06:52:31.701262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749255+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:59.749255+0000","last_peered":"2024-09-21T06:52:59.749255+0000","last_clean":"2024-09-21T06:52:59.749255+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:59.749255+0000","last_undegraded":"2024-09-21T06:52:59.749255+0000","last_fullsized":"2024-09-21T06:52:59.749255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:00.842041+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:54:00.842041+0000","last_peered":"2024-09-21T06:54:00.842041+0000","last_clean":"2024-09-21T06:54:00.842041+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:54:00.842041+0000","last_undegraded":"2024-09-21T06:54:00.842041+0000","last_fullsized":"2024-09-21T06:54:00.842041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701440+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701440+0000","last_peered":"2024-09-21T06:52:31.701440+0000","last_clean":"2024-09-21T06:52:31.701440+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701440+0000","last_undegraded":"2024-09-21T06:52:31.701440+0000","last_fullsized":"2024-09-21T06:52:31.701440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740983+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:59.740983+0000","last_peered":"2024-09-21T06:52:59.740983+0000","last_clean":"2024-09-21T06:52:59.740983+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:59.740983+0000","last_undegraded":"2024-09-21T06:52:59.740983+0000","last_fullsized":"2024-09-21T06:52:59.740983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701878+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701878+0000","last_peered":"2024-09-21T06:52:31.701878+0000","last_clean":"2024-09-21T06:52:31.701878+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701878+0000","last_undegraded":"2024-09-21T06:52:31.701878+0000","last_fullsized":"2024-09-21T06:52:31.701878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749354+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:59.749354+0000","last_peered":"2024-09-21T06:52:59.749354+0000","last_clean":"2024-09-21T06:52:59.749354+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:59.749354+0000","last_undegraded":"2024-09-21T06:52:59.749354+0000","last_fullsized":"2024-09-21T06:52:59.749354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648426+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648426+0000","last_peered":"2024-09-21T06:52:31.648426+0000","last_clean":"2024-09-21T06:52:31.648426+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648426+0000","last_undegraded":"2024-09-21T06:52:31.648426+0000","last_fullsized":"2024-09-21T06:52:31.648426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648721+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648721+0000","last_peered":"2024-09-21T06:52:31.648721+0000","last_clean":"2024-09-21T06:52:31.648721+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648721+0000","last_undegraded":"2024-09-21T06:52:31.648721+0000","last_fullsized":"2024-09-21T06:52:31.648721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753632+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:59.753632+0000","last_peered":"2024-09-21T06:52:59.753632+0000","last_clean":"2024-09-21T06:52:59.753632+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:59.753632+0000","last_undegraded":"2024-09-21T06:52:59.753632+0000","last_fullsized":"2024-09-21T06:52:59.753632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:00.843109+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:54:00.843109+0000","last_peered":"2024-09-21T06:54:00.843109+0000","last_clean":"2024-09-21T06:54:00.843109+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:54:00.843109+0000","last_undegraded":"2024-09-21T06:54:00.843109+0000","last_fullsized":"2024-09-21T06:54:00.843109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836829+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836829+0000","last_peered":"2024-09-21T06:52:31.836829+0000","last_clean":"2024-09-21T06:52:31.836829+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836829+0000","last_undegraded":"2024-09-21T06:52:31.836829+0000","last_fullsized":"2024-09-21T06:52:31.836829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744841+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:59.744841+0000","last_peered":"2024-09-21T06:52:59.744841+0000","last_clean":"2024-09-21T06:52:59.744841+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:59.744841+0000","last_undegraded":"2024-09-21T06:52:59.744841+0000","last_fullsized":"2024-09-21T06:52:59.744841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'62","reported_seq":135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:01.233444+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:54:01.233444+0000","last_peered":"2024-09-21T06:54:01.233444+0000","last_clean":"2024-09-21T06:54:01.233444+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:54:01.233444+0000","last_undegraded":"2024-09-21T06:54:01.233444+0000","last_fullsized":"2024-09-21T06:54:01.233444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891861+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891861+0000","last_peered":"2024-09-21T06:52:31.891861+0000","last_clean":"2024-09-21T06:52:31.891861+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891861+0000","last_undegraded":"2024-09-21T06:52:31.891861+0000","last_fullsized":"2024-09-21T06:52:31.891861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753918+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:59.753918+0000","last_peered":"2024-09-21T06:52:59.753918+0000","last_clean":"2024-09-21T06:52:59.753918+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:59.753918+0000","last_undegraded":"2024-09-21T06:52:59.753918+0000","last_fullsized":"2024-09-21T06:52:59.753918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701640+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701640+0000","last_peered":"2024-09-21T06:52:31.701640+0000","last_clean":"2024-09-21T06:52:31.701640+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701640+0000","last_undegraded":"2024-09-21T06:52:31.701640+0000","last_fullsized":"2024-09-21T06:52:31.701640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701328+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701328+0000","last_peered":"2024-09-21T06:52:31.701328+0000","last_clean":"2024-09-21T06:52:31.701328+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701328+0000","last_undegraded":"2024-09-21T06:52:31.701328+0000","last_fullsized":"2024-09-21T06:52:31.701328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734233+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:59.734233+0000","last_peered":"2024-09-21T06:52:59.734233+0000","last_clean":"2024-09-21T06:52:59.734233+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:59.734233+0000","last_undegraded":"2024-09-21T06:52:59.734233+0000","last_fullsized":"2024-09-21T06:52:59.734233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645941+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645941+0000","last_peered":"2024-09-21T06:52:31.645941+0000","last_clean":"2024-09-21T06:52:31.645941+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645941+0000","last_undegraded":"2024-09-21T06:52:31.645941+0000","last_fullsized":"2024-09-21T06:52:31.645941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701203+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701203+0000","last_peered":"2024-09-21T06:52:31.701203+0000","last_clean":"2024-09-21T06:52:31.701203+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701203+0000","last_undegraded":"2024-09-21T06:52:31.701203+0000","last_fullsized":"2024-09-21T06:52:31.701203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.748492+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:59.748492+0000","last_peered":"2024-09-21T06:52:59.748492+0000","last_clean":"2024-09-21T06:52:59.748492+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:59.748492+0000","last_undegraded":"2024-09-21T06:52:59.748492+0000","last_fullsized":"2024-09-21T06:52:59.748492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891566+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891566+0000","last_peered":"2024-09-21T06:52:31.891566+0000","last_clean":"2024-09-21T06:52:31.891566+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891566+0000","last_undegraded":"2024-09-21T06:52:31.891566+0000","last_fullsized":"2024-09-21T06:52:31.891566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701524+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701524+0000","last_peered":"2024-09-21T06:52:31.701524+0000","last_clean":"2024-09-21T06:52:31.701524+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701524+0000","last_undegraded":"2024-09-21T06:52:31.701524+0000","last_fullsized":"2024-09-21T06:52:31.701524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756438+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:59.756438+0000","last_peered":"2024-09-21T06:52:59.756438+0000","last_clean":"2024-09-21T06:52:59.756438+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:59.756438+0000","last_undegraded":"2024-09-21T06:52:59.756438+0000","last_fullsized":"2024-09-21T06:52:59.756438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:59.841078+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:53:59.841078+0000","last_peered":"2024-09-21T06:53:59.841078+0000","last_clean":"2024-09-21T06:53:59.841078+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:53:59.841078+0000","last_undegraded":"2024-09-21T06:53:59.841078+0000","last_fullsized":"2024-09-21T06:53:59.841078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891979+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891979+0000","last_peered":"2024-09-21T06:52:31.891979+0000","last_clean":"2024-09-21T06:52:31.891979+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891979+0000","last_undegraded":"2024-09-21T06:52:31.891979+0000","last_fullsized":"2024-09-21T06:52:31.891979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758614+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:59.758614+0000","last_peered":"2024-09-21T06:52:59.758614+0000","last_clean":"2024-09-21T06:52:59.758614+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:59.758614+0000","last_undegraded":"2024-09-21T06:52:59.758614+0000","last_fullsized":"2024-09-21T06:52:59.758614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701396+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701396+0000","last_peered":"2024-09-21T06:52:31.701396+0000","last_clean":"2024-09-21T06:52:31.701396+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701396+0000","last_undegraded":"2024-09-21T06:52:31.701396+0000","last_fullsized":"2024-09-21T06:52:31.701396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645499+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645499+0000","last_peered":"2024-09-21T06:52:31.645499+0000","last_clean":"2024-09-21T06:52:31.645499+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645499+0000","last_undegraded":"2024-09-21T06:52:31.645499+0000","last_fullsized":"2024-09-21T06:52:31.645499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749562+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:59.749562+0000","last_peered":"2024-09-21T06:52:59.749562+0000","last_clean":"2024-09-21T06:52:59.749562+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:59.749562+0000","last_undegraded":"2024-09-21T06:52:59.749562+0000","last_fullsized":"2024-09-21T06:52:59.749562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703886+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703886+0000","last_peered":"2024-09-21T06:52:31.703886+0000","last_clean":"2024-09-21T06:52:31.703886+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703886+0000","last_undegraded":"2024-09-21T06:52:31.703886+0000","last_fullsized":"2024-09-21T06:52:31.703886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702060+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.702060+0000","last_peered":"2024-09-21T06:52:31.702060+0000","last_clean":"2024-09-21T06:52:31.702060+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.702060+0000","last_undegraded":"2024-09-21T06:52:31.702060+0000","last_fullsized":"2024-09-21T06:52:31.702060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745772+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:59.745772+0000","last_peered":"2024-09-21T06:52:59.745772+0000","last_clean":"2024-09-21T06:52:59.745772+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:59.745772+0000","last_undegraded":"2024-09-21T06:52:59.745772+0000","last_fullsized":"2024-09-21T06:52:59.745772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'41","reported_seq":368,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:58.839382+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:53:58.839382+0000","last_peered":"2024-09-21T06:53:58.839382+0000","last_clean":"2024-09-21T06:53:58.839382+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:53:58.839382+0000","last_undegraded":"2024-09-21T06:53:58.839382+0000","last_fullsized":"2024-09-21T06:53:58.839382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71976,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":284,"num_read_kb":214,"num_write":43,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645757+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645757+0000","last_peered":"2024-09-21T06:52:31.645757+0000","last_clean":"2024-09-21T06:52:31.645757+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645757+0000","last_undegraded":"2024-09-21T06:52:31.645757+0000","last_fullsized":"2024-09-21T06:52:31.645757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.747527+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:59.747527+0000","last_peered":"2024-09-21T06:52:59.747527+0000","last_clean":"2024-09-21T06:52:59.747527+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:59.747527+0000","last_undegraded":"2024-09-21T06:52:59.747527+0000","last_fullsized":"2024-09-21T06:52:59.747527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702309+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702309+0000","last_peered":"2024-09-21T06:52:31.702309+0000","last_clean":"2024-09-21T06:52:31.702309+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702309+0000","last_undegraded":"2024-09-21T06:52:31.702309+0000","last_fullsized":"2024-09-21T06:52:31.702309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699508+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699508+0000","last_peered":"2024-09-21T06:52:31.699508+0000","last_clean":"2024-09-21T06:52:31.699508+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699508+0000","last_undegraded":"2024-09-21T06:52:31.699508+0000","last_fullsized":"2024-09-21T06:52:31.699508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757106+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:59.757106+0000","last_peered":"2024-09-21T06:52:59.757106+0000","last_clean":"2024-09-21T06:52:59.757106+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:59.757106+0000","last_undegraded":"2024-09-21T06:52:59.757106+0000","last_fullsized":"2024-09-21T06:52:59.757106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648918+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648918+0000","last_peered":"2024-09-21T06:52:31.648918+0000","last_clean":"2024-09-21T06:52:31.648918+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648918+0000","last_undegraded":"2024-09-21T06:52:31.648918+0000","last_fullsized":"2024-09-21T06:52:31.648918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645559+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645559+0000","last_peered":"2024-09-21T06:52:31.645559+0000","last_clean":"2024-09-21T06:52:31.645559+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645559+0000","last_undegraded":"2024-09-21T06:52:31.645559+0000","last_fullsized":"2024-09-21T06:52:31.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750255+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:59.750255+0000","last_peered":"2024-09-21T06:52:59.750255+0000","last_clean":"2024-09-21T06:52:59.750255+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:59.750255+0000","last_undegraded":"2024-09-21T06:52:59.750255+0000","last_fullsized":"2024-09-21T06:52:59.750255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":321,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:58.839382+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:53:58.839382+0000","last_peered":"2024-09-21T06:53:58.839382+0000","last_clean":"2024-09-21T06:53:58.839382+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:53:58.839382+0000","last_undegraded":"2024-09-21T06:53:58.839382+0000","last_fullsized":"2024-09-21T06:53:58.839382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":272,"num_read_kb":272,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892718+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892718+0000","last_peered":"2024-09-21T06:52:31.892718+0000","last_clean":"2024-09-21T06:52:31.892718+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892718+0000","last_undegraded":"2024-09-21T06:52:31.892718+0000","last_fullsized":"2024-09-21T06:52:31.892718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892186+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892186+0000","last_peered":"2024-09-21T06:52:31.892186+0000","last_clean":"2024-09-21T06:52:31.892186+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892186+0000","last_undegraded":"2024-09-21T06:52:31.892186+0000","last_fullsized":"2024-09-21T06:52:31.892186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741381+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:59.741381+0000","last_peered":"2024-09-21T06:52:59.741381+0000","last_clean":"2024-09-21T06:52:59.741381+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:59.741381+0000","last_undegraded":"2024-09-21T06:52:59.741381+0000","last_fullsized":"2024-09-21T06:52:59.741381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701609+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701609+0000","last_peered":"2024-09-21T06:52:31.701609+0000","last_clean":"2024-09-21T06:52:31.701609+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701609+0000","last_undegraded":"2024-09-21T06:52:31.701609+0000","last_fullsized":"2024-09-21T06:52:31.701609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645808+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645808+0000","last_peered":"2024-09-21T06:52:31.645808+0000","last_clean":"2024-09-21T06:52:31.645808+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645808+0000","last_undegraded":"2024-09-21T06:52:31.645808+0000","last_fullsized":"2024-09-21T06:52:31.645808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742667+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:59.742667+0000","last_peered":"2024-09-21T06:52:59.742667+0000","last_clean":"2024-09-21T06:52:59.742667+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:59.742667+0000","last_undegraded":"2024-09-21T06:52:59.742667+0000","last_fullsized":"2024-09-21T06:52:59.742667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700994+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700994+0000","last_peered":"2024-09-21T06:52:31.700994+0000","last_clean":"2024-09-21T06:52:31.700994+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700994+0000","last_undegraded":"2024-09-21T06:52:31.700994+0000","last_fullsized":"2024-09-21T06:52:31.700994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645981+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645981+0000","last_peered":"2024-09-21T06:52:31.645981+0000","last_clean":"2024-09-21T06:52:31.645981+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645981+0000","last_undegraded":"2024-09-21T06:52:31.645981+0000","last_fullsized":"2024-09-21T06:52:31.645981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749130+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:59.749130+0000","last_peered":"2024-09-21T06:52:59.749130+0000","last_clean":"2024-09-21T06:52:59.749130+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:59.749130+0000","last_undegraded":"2024-09-21T06:52:59.749130+0000","last_fullsized":"2024-09-21T06:52:59.749130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703865+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703865+0000","last_peered":"2024-09-21T06:52:31.703865+0000","last_clean":"2024-09-21T06:52:31.703865+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703865+0000","last_undegraded":"2024-09-21T06:52:31.703865+0000","last_fullsized":"2024-09-21T06:52:31.703865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:59.841987+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:53:59.841987+0000","last_peered":"2024-09-21T06:53:59.841987+0000","last_clean":"2024-09-21T06:53:59.841987+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:53:59.841987+0000","last_undegraded":"2024-09-21T06:53:59.841987+0000","last_fullsized":"2024-09-21T06:53:59.841987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.736418+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:59.736418+0000","last_peered":"2024-09-21T06:52:59.736418+0000","last_clean":"2024-09-21T06:52:59.736418+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:59.736418+0000","last_undegraded":"2024-09-21T06:52:59.736418+0000","last_fullsized":"2024-09-21T06:52:59.736418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704051+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.704051+0000","last_peered":"2024-09-21T06:52:31.704051+0000","last_clean":"2024-09-21T06:52:31.704051+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.704051+0000","last_undegraded":"2024-09-21T06:52:31.704051+0000","last_fullsized":"2024-09-21T06:52:31.704051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.739975+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:59.739975+0000","last_peered":"2024-09-21T06:52:59.739975+0000","last_clean":"2024-09-21T06:52:59.739975+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:59.739975+0000","last_undegraded":"2024-09-21T06:52:59.739975+0000","last_fullsized":"2024-09-21T06:52:59.739975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749347+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:59.749347+0000","last_peered":"2024-09-21T06:52:59.749347+0000","last_clean":"2024-09-21T06:52:59.749347+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:59.749347+0000","last_undegraded":"2024-09-21T06:52:59.749347+0000","last_fullsized":"2024-09-21T06:52:59.749347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746716+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:59.746716+0000","last_peered":"2024-09-21T06:52:59.746716+0000","last_clean":"2024-09-21T06:52:59.746716+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:59.746716+0000","last_undegraded":"2024-09-21T06:52:59.746716+0000","last_fullsized":"2024-09-21T06:52:59.746716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757121+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:59.757121+0000","last_peered":"2024-09-21T06:52:59.757121+0000","last_clean":"2024-09-21T06:52:59.757121+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:59.757121+0000","last_undegraded":"2024-09-21T06:52:59.757121+0000","last_fullsized":"2024-09-21T06:52:59.757121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756339+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:59.756339+0000","last_peered":"2024-09-21T06:52:59.756339+0000","last_clean":"2024-09-21T06:52:59.756339+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:59.756339+0000","last_undegraded":"2024-09-21T06:52:59.756339+0000","last_fullsized":"2024-09-21T06:52:59.756339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734085+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:59.734085+0000","last_peered":"2024-09-21T06:52:59.734085+0000","last_clean":"2024-09-21T06:52:59.734085+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:59.734085+0000","last_undegraded":"2024-09-21T06:52:59.734085+0000","last_fullsized":"2024-09-21T06:52:59.734085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749595+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:59.749595+0000","last_peered":"2024-09-21T06:52:59.749595+0000","last_clean":"2024-09-21T06:52:59.749595+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:59.749595+0000","last_undegraded":"2024-09-21T06:52:59.749595+0000","last_fullsized":"2024-09-21T06:52:59.749595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.737176+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:59.737176+0000","last_peered":"2024-09-21T06:52:59.737176+0000","last_clean":"2024-09-21T06:52:59.737176+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:59.737176+0000","last_undegraded":"2024-09-21T06:52:59.737176+0000","last_fullsized":"2024-09-21T06:52:59.737176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746235+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:59.746235+0000","last_peered":"2024-09-21T06:52:59.746235+0000","last_clean":"2024-09-21T06:52:59.746235+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:59.746235+0000","last_undegraded":"2024-09-21T06:52:59.746235+0000","last_fullsized":"2024-09-21T06:52:59.746235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740649+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:59.740649+0000","last_peered":"2024-09-21T06:52:59.740649+0000","last_clean":"2024-09-21T06:52:59.740649+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:59.740649+0000","last_undegraded":"2024-09-21T06:52:59.740649+0000","last_fullsized":"2024-09-21T06:52:59.740649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753696+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:59.753696+0000","last_peered":"2024-09-21T06:52:59.753696+0000","last_clean":"2024-09-21T06:52:59.753696+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:59.753696+0000","last_undegraded":"2024-09-21T06:52:59.753696+0000","last_fullsized":"2024-09-21T06:52:59.753696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756495+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:59.756495+0000","last_peered":"2024-09-21T06:52:59.756495+0000","last_clean":"2024-09-21T06:52:59.756495+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:59.756495+0000","last_undegraded":"2024-09-21T06:52:59.756495+0000","last_fullsized":"2024-09-21T06:52:59.756495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734175+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:59.734175+0000","last_peered":"2024-09-21T06:52:59.734175+0000","last_clean":"2024-09-21T06:52:59.734175+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:59.734175+0000","last_undegraded":"2024-09-21T06:52:59.734175+0000","last_fullsized":"2024-09-21T06:52:59.734175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745394+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:59.745394+0000","last_peered":"2024-09-21T06:52:59.745394+0000","last_clean":"2024-09-21T06:52:59.745394+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:59.745394+0000","last_undegraded":"2024-09-21T06:52:59.745394+0000","last_fullsized":"2024-09-21T06:52:59.745394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751016+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:59.751016+0000","last_peered":"2024-09-21T06:52:59.751016+0000","last_clean":"2024-09-21T06:52:59.751016+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:59.751016+0000","last_undegraded":"2024-09-21T06:52:59.751016+0000","last_fullsized":"2024-09-21T06:52:59.751016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755908+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:59.755908+0000","last_peered":"2024-09-21T06:52:59.755908+0000","last_clean":"2024-09-21T06:52:59.755908+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:59.755908+0000","last_undegraded":"2024-09-21T06:52:59.755908+0000","last_fullsized":"2024-09-21T06:52:59.755908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752146+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:59.752146+0000","last_peered":"2024-09-21T06:52:59.752146+0000","last_clean":"2024-09-21T06:52:59.752146+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:59.752146+0000","last_undegraded":"2024-09-21T06:52:59.752146+0000","last_fullsized":"2024-09-21T06:52:59.752146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754481+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:59.754481+0000","last_peered":"2024-09-21T06:52:59.754481+0000","last_clean":"2024-09-21T06:52:59.754481+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:59.754481+0000","last_undegraded":"2024-09-21T06:52:59.754481+0000","last_fullsized":"2024-09-21T06:52:59.754481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758054+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:59.758054+0000","last_peered":"2024-09-21T06:52:59.758054+0000","last_clean":"2024-09-21T06:52:59.758054+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:59.758054+0000","last_undegraded":"2024-09-21T06:52:59.758054+0000","last_fullsized":"2024-09-21T06:52:59.758054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756767+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:59.756767+0000","last_peered":"2024-09-21T06:52:59.756767+0000","last_clean":"2024-09-21T06:52:59.756767+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:59.756767+0000","last_undegraded":"2024-09-21T06:52:59.756767+0000","last_fullsized":"2024-09-21T06:52:59.756767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755238+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:59.755238+0000","last_peered":"2024-09-21T06:52:59.755238+0000","last_clean":"2024-09-21T06:52:59.755238+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:59.755238+0000","last_undegraded":"2024-09-21T06:52:59.755238+0000","last_fullsized":"2024-09-21T06:52:59.755238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752053+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:59.752053+0000","last_peered":"2024-09-21T06:52:59.752053+0000","last_clean":"2024-09-21T06:52:59.752053+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:59.752053+0000","last_undegraded":"2024-09-21T06:52:59.752053+0000","last_fullsized":"2024-09-21T06:52:59.752053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.759125+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:59.759125+0000","last_peered":"2024-09-21T06:52:59.759125+0000","last_clean":"2024-09-21T06:52:59.759125+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:59.759125+0000","last_undegraded":"2024-09-21T06:52:59.759125+0000","last_fullsized":"2024-09-21T06:52:59.759125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742201+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:59.742201+0000","last_peered":"2024-09-21T06:52:59.742201+0000","last_clean":"2024-09-21T06:52:59.742201+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:59.742201+0000","last_undegraded":"2024-09-21T06:52:59.742201+0000","last_fullsized":"2024-09-21T06:52:59.742201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755485+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:59.755485+0000","last_peered":"2024-09-21T06:52:59.755485+0000","last_clean":"2024-09-21T06:52:59.755485+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:59.755485+0000","last_undegraded":"2024-09-21T06:52:59.755485+0000","last_fullsized":"2024-09-21T06:52:59.755485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752958+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:59.752958+0000","last_peered":"2024-09-21T06:52:59.752958+0000","last_clean":"2024-09-21T06:52:59.752958+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:59.752958+0000","last_undegraded":"2024-09-21T06:52:59.752958+0000","last_fullsized":"2024-09-21T06:52:59.752958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754740+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:59.754740+0000","last_peered":"2024-09-21T06:52:59.754740+0000","last_clean":"2024-09-21T06:52:59.754740+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:59.754740+0000","last_undegraded":"2024-09-21T06:52:59.754740+0000","last_fullsized":"2024-09-21T06:52:59.754740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746351+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:59.746351+0000","last_peered":"2024-09-21T06:52:59.746351+0000","last_clean":"2024-09-21T06:52:59.746351+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:59.746351+0000","last_undegraded":"2024-09-21T06:52:59.746351+0000","last_fullsized":"2024-09-21T06:52:59.746351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749389+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:59.749389+0000","last_peered":"2024-09-21T06:52:59.749389+0000","last_clean":"2024-09-21T06:52:59.749389+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:59.749389+0000","last_undegraded":"2024-09-21T06:52:59.749389+0000","last_fullsized":"2024-09-21T06:52:59.749389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757852+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:59.757852+0000","last_peered":"2024-09-21T06:52:59.757852+0000","last_clean":"2024-09-21T06:52:59.757852+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:59.757852+0000","last_undegraded":"2024-09-21T06:52:59.757852+0000","last_fullsized":"2024-09-21T06:52:59.757852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750926+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:59.750926+0000","last_peered":"2024-09-21T06:52:59.750926+0000","last_clean":"2024-09-21T06:52:59.750926+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:59.750926+0000","last_undegraded":"2024-09-21T06:52:59.750926+0000","last_fullsized":"2024-09-21T06:52:59.750926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":867297,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1755,"num_read_kb":1099,"num_write":494,"num_write_kb":2255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2576384,"data_stored":2210391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":375,"ondisk_log_size":375,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":531799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907001,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1740800,"data_stored":1190037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":928487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1445888,"data_stored":924747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462986,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1129034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":527522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822743,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29036,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716116,"statfs":{"total":95995035648,"available":95965302784,"internally_reserved":0,"allocated":2060288,"data_stored":1517549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28880,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716272,"statfs":{"total":95995035648,"available":95965462528,"internally_reserved":0,"allocated":1839104,"data_stored":1313379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378729,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":600953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":602820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705779,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:54:05.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:54:05.868 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:06.007 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:05 smithi114 ceph-mon[30654]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:54:06.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:05 smithi183 ceph-mon[34219]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:54:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:05 smithi193 ceph-mon[34601]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T06:54:06.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 -- 172.21.15.114:0/398301802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e38105320 msgr2=0x7f5e3810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:06.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 --2- 172.21.15.114:0/398301802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e38105320 0x7f5e3810d870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009900 tx=0x7f5e2402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 -- 172.21.15.114:0/398301802 shutdown_connections 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 --2- 172.21.15.114:0/398301802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e3810de30 0x7f5e38110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 --2- 172.21.15.114:0/398301802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e38105320 0x7f5e3810d870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 --2- 172.21.15.114:0/398301802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 0x7f5e38104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 -- 172.21.15.114:0/398301802 >> 172.21.15.114:0/398301802 conn(0x7f5e38079740 msgr2=0x7f5e38079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 -- 172.21.15.114:0/398301802 shutdown_connections 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.522+0000 7f5e40000640 1 -- 172.21.15.114:0/398301802 wait complete. 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.524+0000 7f5e40000640 1 Processor -- start 2024-09-21T06:54:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.524+0000 7f5e40000640 1 -- start start 2024-09-21T06:54:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e40000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 0x7f5e3819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e40000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e38105320 0x7f5e3819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e40000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e3810de30 0x7f5e381a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e40000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38113ae0 con 0x7f5e3810de30 2024-09-21T06:54:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e40000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38113960 con 0x7f5e381049d0 2024-09-21T06:54:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e40000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e38113c60 con 0x7f5e38105320 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e3810de30 0x7f5e381a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e38105320 0x7f5e3819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 0x7f5e3819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e38105320 0x7f5e3819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48674/0 (socket says 172.21.15.114:48674) 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e3810de30 0x7f5e381a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52330/0 (socket says 172.21.15.114:52330) 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3cc00640 1 -- 172.21.15.114:0/4122871586 learned_addr learned my addr 172.21.15.114:0/4122871586 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3d600640 1 -- 172.21.15.114:0/4122871586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e38105320 msgr2=0x7f5e3819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3d600640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e38105320 0x7f5e3819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3d600640 1 -- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e3810de30 msgr2=0x7f5e381a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3d600640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e3810de30 0x7f5e381a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.525+0000 7f5e3d600640 1 -- 172.21.15.114:0/4122871586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e381a25e0 con 0x7f5e381049d0 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.526+0000 7f5e3d600640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 0x7f5e3819c670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00e9c0 tx=0x7f5e2c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.526+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00ccc0 con 0x7f5e381049d0 2024-09-21T06:54:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.526+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e381a28d0 con 0x7f5e381049d0 2024-09-21T06:54:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.526+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e2c0035f0 con 0x7f5e381049d0 2024-09-21T06:54:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.526+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00ce60 con 0x7f5e381049d0 2024-09-21T06:54:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.526+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e381a9030 con 0x7f5e381049d0 2024-09-21T06:54:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.527+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3806c240 con 0x7f5e381049d0 2024-09-21T06:54:06.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.530+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f5e2c004000 con 0x7f5e381049d0 2024-09-21T06:54:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.531+0000 7f5e36000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e200b65f0 0x7f5e200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.531+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5e2c0e7f00 con 0x7f5e381049d0 2024-09-21T06:54:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.531+0000 7f5e3cc00640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e200b65f0 0x7f5e200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.532+0000 7f5e3cc00640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e200b65f0 0x7f5e200b8b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e24006010 tx=0x7f5e24005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:06.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.533+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5e2c0ae1d0 con 0x7f5e381049d0 2024-09-21T06:54:06.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.716+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e38103dd0 con 0x7f5e200b65f0 2024-09-21T06:54:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.722+0000 7f5e36000640 1 -- 172.21.15.114:0/4122871586 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340575 (secure 0 0 0) 0x7f5e38103dd0 con 0x7f5e200b65f0 2024-09-21T06:54:06.729 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:06.731 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:54:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e200b65f0 msgr2=0x7f5e200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:06.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e200b65f0 0x7f5e200b8b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e24006010 tx=0x7f5e24005a20 comp rx=0 tx=0).stop 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 msgr2=0x7f5e3819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 0x7f5e3819c670 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00e9c0 tx=0x7f5e2c00ee80 comp rx=0 tx=0).stop 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 shutdown_connections 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e3810de30 0x7f5e381a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e38105320 0x7f5e3819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e200b65f0 0x7f5e200b8b00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 --2- 172.21.15.114:0/4122871586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e381049d0 0x7f5e3819c670 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 >> 172.21.15.114:0/4122871586 conn(0x7f5e38079740 msgr2=0x7f5e381036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 shutdown_connections 2024-09-21T06:54:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:06.728+0000 7f5e40000640 1 -- 172.21.15.114:0/4122871586 wait complete. 2024-09-21T06:54:07.510 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-21T06:54:05.545948+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1459037,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1863,"num_read_kb":1186,"num_write":620,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378476,"kb_used_data":16628,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124563348,"statfs":{"total":1151940427776,"available":1151552868352,"internally_reserved":0,"allocated":17027072,"data_stored":10720661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":792479,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":461,"num_read_kb":371,"num_write":226,"num_write_kb":2095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003835"},"pg_stats":[{"pgid":"2.1f","version":"89'15","reported_seq":200,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:01.157297+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:54:01.157297+0000","last_peered":"2024-09-21T06:54:01.157297+0000","last_clean":"2024-09-21T06:54:01.157297+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:54:01.157297+0000","last_undegraded":"2024-09-21T06:54:01.157297+0000","last_fullsized":"2024-09-21T06:54:01.157297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741199+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:59.741199+0000","last_peered":"2024-09-21T06:52:59.741199+0000","last_clean":"2024-09-21T06:52:59.741199+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:59.741199+0000","last_undegraded":"2024-09-21T06:52:59.741199+0000","last_fullsized":"2024-09-21T06:52:59.741199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703671+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703671+0000","last_peered":"2024-09-21T06:52:31.703671+0000","last_clean":"2024-09-21T06:52:31.703671+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703671+0000","last_undegraded":"2024-09-21T06:52:31.703671+0000","last_fullsized":"2024-09-21T06:52:31.703671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699254+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699254+0000","last_peered":"2024-09-21T06:52:31.699254+0000","last_clean":"2024-09-21T06:52:31.699254+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699254+0000","last_undegraded":"2024-09-21T06:52:31.699254+0000","last_fullsized":"2024-09-21T06:52:31.699254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:16.554226+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:53:16.554226+0000","last_peered":"2024-09-21T06:53:16.554226+0000","last_clean":"2024-09-21T06:53:16.554226+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:53:16.554226+0000","last_undegraded":"2024-09-21T06:53:16.554226+0000","last_fullsized":"2024-09-21T06:53:16.554226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755557+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:59.755557+0000","last_peered":"2024-09-21T06:52:59.755557+0000","last_clean":"2024-09-21T06:52:59.755557+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:59.755557+0000","last_undegraded":"2024-09-21T06:52:59.755557+0000","last_fullsized":"2024-09-21T06:52:59.755557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699992+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699992+0000","last_peered":"2024-09-21T06:52:31.699992+0000","last_clean":"2024-09-21T06:52:31.699992+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699992+0000","last_undegraded":"2024-09-21T06:52:31.699992+0000","last_fullsized":"2024-09-21T06:52:31.699992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891939+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891939+0000","last_peered":"2024-09-21T06:52:31.891939+0000","last_clean":"2024-09-21T06:52:31.891939+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891939+0000","last_undegraded":"2024-09-21T06:52:31.891939+0000","last_fullsized":"2024-09-21T06:52:31.891939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751784+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:59.751784+0000","last_peered":"2024-09-21T06:52:59.751784+0000","last_clean":"2024-09-21T06:52:59.751784+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:59.751784+0000","last_undegraded":"2024-09-21T06:52:59.751784+0000","last_fullsized":"2024-09-21T06:52:59.751784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699375+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699375+0000","last_peered":"2024-09-21T06:52:31.699375+0000","last_clean":"2024-09-21T06:52:31.699375+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699375+0000","last_undegraded":"2024-09-21T06:52:31.699375+0000","last_fullsized":"2024-09-21T06:52:31.699375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755326+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:59.755326+0000","last_peered":"2024-09-21T06:52:59.755326+0000","last_clean":"2024-09-21T06:52:59.755326+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:59.755326+0000","last_undegraded":"2024-09-21T06:52:59.755326+0000","last_fullsized":"2024-09-21T06:52:59.755326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645462+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645462+0000","last_peered":"2024-09-21T06:52:31.645462+0000","last_clean":"2024-09-21T06:52:31.645462+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645462+0000","last_undegraded":"2024-09-21T06:52:31.645462+0000","last_fullsized":"2024-09-21T06:52:31.645462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755915+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:59.755915+0000","last_peered":"2024-09-21T06:52:59.755915+0000","last_clean":"2024-09-21T06:52:59.755915+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:59.755915+0000","last_undegraded":"2024-09-21T06:52:59.755915+0000","last_fullsized":"2024-09-21T06:52:59.755915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704037+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.704037+0000","last_peered":"2024-09-21T06:52:31.704037+0000","last_clean":"2024-09-21T06:52:31.704037+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.704037+0000","last_undegraded":"2024-09-21T06:52:31.704037+0000","last_fullsized":"2024-09-21T06:52:31.704037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699968+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699968+0000","last_peered":"2024-09-21T06:52:31.699968+0000","last_clean":"2024-09-21T06:52:31.699968+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699968+0000","last_undegraded":"2024-09-21T06:52:31.699968+0000","last_fullsized":"2024-09-21T06:52:31.699968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745662+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:59.745662+0000","last_peered":"2024-09-21T06:52:59.745662+0000","last_clean":"2024-09-21T06:52:59.745662+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:59.745662+0000","last_undegraded":"2024-09-21T06:52:59.745662+0000","last_fullsized":"2024-09-21T06:52:59.745662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892935+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892935+0000","last_peered":"2024-09-21T06:52:31.892935+0000","last_clean":"2024-09-21T06:52:31.892935+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892935+0000","last_undegraded":"2024-09-21T06:52:31.892935+0000","last_fullsized":"2024-09-21T06:52:31.892935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646491+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646491+0000","last_peered":"2024-09-21T06:52:31.646491+0000","last_clean":"2024-09-21T06:52:31.646491+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646491+0000","last_undegraded":"2024-09-21T06:52:31.646491+0000","last_fullsized":"2024-09-21T06:52:31.646491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744826+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:59.744826+0000","last_peered":"2024-09-21T06:52:59.744826+0000","last_clean":"2024-09-21T06:52:59.744826+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:59.744826+0000","last_undegraded":"2024-09-21T06:52:59.744826+0000","last_fullsized":"2024-09-21T06:52:59.744826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836870+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836870+0000","last_peered":"2024-09-21T06:52:31.836870+0000","last_clean":"2024-09-21T06:52:31.836870+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836870+0000","last_undegraded":"2024-09-21T06:52:31.836870+0000","last_fullsized":"2024-09-21T06:52:31.836870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755751+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:59.755751+0000","last_peered":"2024-09-21T06:52:59.755751+0000","last_clean":"2024-09-21T06:52:59.755751+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:59.755751+0000","last_undegraded":"2024-09-21T06:52:59.755751+0000","last_fullsized":"2024-09-21T06:52:59.755751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700368+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700368+0000","last_peered":"2024-09-21T06:52:31.700368+0000","last_clean":"2024-09-21T06:52:31.700368+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700368+0000","last_undegraded":"2024-09-21T06:52:31.700368+0000","last_fullsized":"2024-09-21T06:52:31.700368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703897+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703897+0000","last_peered":"2024-09-21T06:52:31.703897+0000","last_clean":"2024-09-21T06:52:31.703897+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703897+0000","last_undegraded":"2024-09-21T06:52:31.703897+0000","last_fullsized":"2024-09-21T06:52:31.703897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741893+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:59.741893+0000","last_peered":"2024-09-21T06:52:59.741893+0000","last_clean":"2024-09-21T06:52:59.741893+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:59.741893+0000","last_undegraded":"2024-09-21T06:52:59.741893+0000","last_fullsized":"2024-09-21T06:52:59.741893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:02.219221+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:54:02.219221+0000","last_peered":"2024-09-21T06:54:02.219221+0000","last_clean":"2024-09-21T06:54:02.219221+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:54:02.219221+0000","last_undegraded":"2024-09-21T06:54:02.219221+0000","last_fullsized":"2024-09-21T06:54:02.219221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891910+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891910+0000","last_peered":"2024-09-21T06:52:31.891910+0000","last_clean":"2024-09-21T06:52:31.891910+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891910+0000","last_undegraded":"2024-09-21T06:52:31.891910+0000","last_fullsized":"2024-09-21T06:52:31.891910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742064+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:59.742064+0000","last_peered":"2024-09-21T06:52:59.742064+0000","last_clean":"2024-09-21T06:52:59.742064+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:59.742064+0000","last_undegraded":"2024-09-21T06:52:59.742064+0000","last_fullsized":"2024-09-21T06:52:59.742064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892041+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892041+0000","last_peered":"2024-09-21T06:52:31.892041+0000","last_clean":"2024-09-21T06:52:31.892041+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892041+0000","last_undegraded":"2024-09-21T06:52:31.892041+0000","last_fullsized":"2024-09-21T06:52:31.892041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645535+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645535+0000","last_peered":"2024-09-21T06:52:31.645535+0000","last_clean":"2024-09-21T06:52:31.645535+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645535+0000","last_undegraded":"2024-09-21T06:52:31.645535+0000","last_fullsized":"2024-09-21T06:52:31.645535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754090+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:59.754090+0000","last_peered":"2024-09-21T06:52:59.754090+0000","last_clean":"2024-09-21T06:52:59.754090+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:59.754090+0000","last_undegraded":"2024-09-21T06:52:59.754090+0000","last_fullsized":"2024-09-21T06:52:59.754090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645977+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645977+0000","last_peered":"2024-09-21T06:52:31.645977+0000","last_clean":"2024-09-21T06:52:31.645977+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645977+0000","last_undegraded":"2024-09-21T06:52:31.645977+0000","last_fullsized":"2024-09-21T06:52:31.645977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891517+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891517+0000","last_peered":"2024-09-21T06:52:31.891517+0000","last_clean":"2024-09-21T06:52:31.891517+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891517+0000","last_undegraded":"2024-09-21T06:52:31.891517+0000","last_fullsized":"2024-09-21T06:52:31.891517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758656+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:59.758656+0000","last_peered":"2024-09-21T06:52:59.758656+0000","last_clean":"2024-09-21T06:52:59.758656+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:59.758656+0000","last_undegraded":"2024-09-21T06:52:59.758656+0000","last_fullsized":"2024-09-21T06:52:59.758656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'8","reported_seq":197,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:02.845800+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:54:02.845800+0000","last_peered":"2024-09-21T06:54:02.845800+0000","last_clean":"2024-09-21T06:54:02.845800+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:54:02.845800+0000","last_undegraded":"2024-09-21T06:54:02.845800+0000","last_fullsized":"2024-09-21T06:54:02.845800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1431,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701262+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701262+0000","last_peered":"2024-09-21T06:52:31.701262+0000","last_clean":"2024-09-21T06:52:31.701262+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701262+0000","last_undegraded":"2024-09-21T06:52:31.701262+0000","last_fullsized":"2024-09-21T06:52:31.701262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749255+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:59.749255+0000","last_peered":"2024-09-21T06:52:59.749255+0000","last_clean":"2024-09-21T06:52:59.749255+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:59.749255+0000","last_undegraded":"2024-09-21T06:52:59.749255+0000","last_fullsized":"2024-09-21T06:52:59.749255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:00.842041+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:54:00.842041+0000","last_peered":"2024-09-21T06:54:00.842041+0000","last_clean":"2024-09-21T06:54:00.842041+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:54:00.842041+0000","last_undegraded":"2024-09-21T06:54:00.842041+0000","last_fullsized":"2024-09-21T06:54:00.842041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701440+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701440+0000","last_peered":"2024-09-21T06:52:31.701440+0000","last_clean":"2024-09-21T06:52:31.701440+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701440+0000","last_undegraded":"2024-09-21T06:52:31.701440+0000","last_fullsized":"2024-09-21T06:52:31.701440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740983+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:59.740983+0000","last_peered":"2024-09-21T06:52:59.740983+0000","last_clean":"2024-09-21T06:52:59.740983+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:59.740983+0000","last_undegraded":"2024-09-21T06:52:59.740983+0000","last_fullsized":"2024-09-21T06:52:59.740983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701878+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701878+0000","last_peered":"2024-09-21T06:52:31.701878+0000","last_clean":"2024-09-21T06:52:31.701878+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701878+0000","last_undegraded":"2024-09-21T06:52:31.701878+0000","last_fullsized":"2024-09-21T06:52:31.701878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749354+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:59.749354+0000","last_peered":"2024-09-21T06:52:59.749354+0000","last_clean":"2024-09-21T06:52:59.749354+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:59.749354+0000","last_undegraded":"2024-09-21T06:52:59.749354+0000","last_fullsized":"2024-09-21T06:52:59.749354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648487+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648487+0000","last_peered":"2024-09-21T06:52:31.648487+0000","last_clean":"2024-09-21T06:52:31.648487+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648487+0000","last_undegraded":"2024-09-21T06:52:31.648487+0000","last_fullsized":"2024-09-21T06:52:31.648487+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648721+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648721+0000","last_peered":"2024-09-21T06:52:31.648721+0000","last_clean":"2024-09-21T06:52:31.648721+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648721+0000","last_undegraded":"2024-09-21T06:52:31.648721+0000","last_fullsized":"2024-09-21T06:52:31.648721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753632+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:59.753632+0000","last_peered":"2024-09-21T06:52:59.753632+0000","last_clean":"2024-09-21T06:52:59.753632+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:59.753632+0000","last_undegraded":"2024-09-21T06:52:59.753632+0000","last_fullsized":"2024-09-21T06:52:59.753632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":193,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:00.843109+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:54:00.843109+0000","last_peered":"2024-09-21T06:54:00.843109+0000","last_clean":"2024-09-21T06:54:00.843109+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:54:00.843109+0000","last_undegraded":"2024-09-21T06:54:00.843109+0000","last_fullsized":"2024-09-21T06:54:00.843109+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836829+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836829+0000","last_peered":"2024-09-21T06:52:31.836829+0000","last_clean":"2024-09-21T06:52:31.836829+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836829+0000","last_undegraded":"2024-09-21T06:52:31.836829+0000","last_fullsized":"2024-09-21T06:52:31.836829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744841+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:59.744841+0000","last_peered":"2024-09-21T06:52:59.744841+0000","last_clean":"2024-09-21T06:52:59.744841+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:59.744841+0000","last_undegraded":"2024-09-21T06:52:59.744841+0000","last_fullsized":"2024-09-21T06:52:59.744841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'62","reported_seq":135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:01.233444+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:54:01.233444+0000","last_peered":"2024-09-21T06:54:01.233444+0000","last_clean":"2024-09-21T06:54:01.233444+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:54:01.233444+0000","last_undegraded":"2024-09-21T06:54:01.233444+0000","last_fullsized":"2024-09-21T06:54:01.233444+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":30,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891861+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891861+0000","last_peered":"2024-09-21T06:52:31.891861+0000","last_clean":"2024-09-21T06:52:31.891861+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891861+0000","last_undegraded":"2024-09-21T06:52:31.891861+0000","last_fullsized":"2024-09-21T06:52:31.891861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753918+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:59.753918+0000","last_peered":"2024-09-21T06:52:59.753918+0000","last_clean":"2024-09-21T06:52:59.753918+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:59.753918+0000","last_undegraded":"2024-09-21T06:52:59.753918+0000","last_fullsized":"2024-09-21T06:52:59.753918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701640+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701640+0000","last_peered":"2024-09-21T06:52:31.701640+0000","last_clean":"2024-09-21T06:52:31.701640+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701640+0000","last_undegraded":"2024-09-21T06:52:31.701640+0000","last_fullsized":"2024-09-21T06:52:31.701640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701328+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701328+0000","last_peered":"2024-09-21T06:52:31.701328+0000","last_clean":"2024-09-21T06:52:31.701328+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701328+0000","last_undegraded":"2024-09-21T06:52:31.701328+0000","last_fullsized":"2024-09-21T06:52:31.701328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734233+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:59.734233+0000","last_peered":"2024-09-21T06:52:59.734233+0000","last_clean":"2024-09-21T06:52:59.734233+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:59.734233+0000","last_undegraded":"2024-09-21T06:52:59.734233+0000","last_fullsized":"2024-09-21T06:52:59.734233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645941+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645941+0000","last_peered":"2024-09-21T06:52:31.645941+0000","last_clean":"2024-09-21T06:52:31.645941+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645941+0000","last_undegraded":"2024-09-21T06:52:31.645941+0000","last_fullsized":"2024-09-21T06:52:31.645941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701203+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701203+0000","last_peered":"2024-09-21T06:52:31.701203+0000","last_clean":"2024-09-21T06:52:31.701203+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701203+0000","last_undegraded":"2024-09-21T06:52:31.701203+0000","last_fullsized":"2024-09-21T06:52:31.701203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.748492+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:59.748492+0000","last_peered":"2024-09-21T06:52:59.748492+0000","last_clean":"2024-09-21T06:52:59.748492+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:59.748492+0000","last_undegraded":"2024-09-21T06:52:59.748492+0000","last_fullsized":"2024-09-21T06:52:59.748492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891566+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891566+0000","last_peered":"2024-09-21T06:52:31.891566+0000","last_clean":"2024-09-21T06:52:31.891566+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891566+0000","last_undegraded":"2024-09-21T06:52:31.891566+0000","last_fullsized":"2024-09-21T06:52:31.891566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701524+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701524+0000","last_peered":"2024-09-21T06:52:31.701524+0000","last_clean":"2024-09-21T06:52:31.701524+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701524+0000","last_undegraded":"2024-09-21T06:52:31.701524+0000","last_fullsized":"2024-09-21T06:52:31.701524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756438+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:59.756438+0000","last_peered":"2024-09-21T06:52:59.756438+0000","last_clean":"2024-09-21T06:52:59.756438+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:59.756438+0000","last_undegraded":"2024-09-21T06:52:59.756438+0000","last_fullsized":"2024-09-21T06:52:59.756438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:59.841078+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:53:59.841078+0000","last_peered":"2024-09-21T06:53:59.841078+0000","last_clean":"2024-09-21T06:53:59.841078+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:53:59.841078+0000","last_undegraded":"2024-09-21T06:53:59.841078+0000","last_fullsized":"2024-09-21T06:53:59.841078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891979+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891979+0000","last_peered":"2024-09-21T06:52:31.891979+0000","last_clean":"2024-09-21T06:52:31.891979+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891979+0000","last_undegraded":"2024-09-21T06:52:31.891979+0000","last_fullsized":"2024-09-21T06:52:31.891979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758614+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:59.758614+0000","last_peered":"2024-09-21T06:52:59.758614+0000","last_clean":"2024-09-21T06:52:59.758614+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:59.758614+0000","last_undegraded":"2024-09-21T06:52:59.758614+0000","last_fullsized":"2024-09-21T06:52:59.758614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701396+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701396+0000","last_peered":"2024-09-21T06:52:31.701396+0000","last_clean":"2024-09-21T06:52:31.701396+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701396+0000","last_undegraded":"2024-09-21T06:52:31.701396+0000","last_fullsized":"2024-09-21T06:52:31.701396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645499+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645499+0000","last_peered":"2024-09-21T06:52:31.645499+0000","last_clean":"2024-09-21T06:52:31.645499+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645499+0000","last_undegraded":"2024-09-21T06:52:31.645499+0000","last_fullsized":"2024-09-21T06:52:31.645499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749562+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:59.749562+0000","last_peered":"2024-09-21T06:52:59.749562+0000","last_clean":"2024-09-21T06:52:59.749562+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:59.749562+0000","last_undegraded":"2024-09-21T06:52:59.749562+0000","last_fullsized":"2024-09-21T06:52:59.749562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703886+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703886+0000","last_peered":"2024-09-21T06:52:31.703886+0000","last_clean":"2024-09-21T06:52:31.703886+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703886+0000","last_undegraded":"2024-09-21T06:52:31.703886+0000","last_fullsized":"2024-09-21T06:52:31.703886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702060+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.702060+0000","last_peered":"2024-09-21T06:52:31.702060+0000","last_clean":"2024-09-21T06:52:31.702060+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.702060+0000","last_undegraded":"2024-09-21T06:52:31.702060+0000","last_fullsized":"2024-09-21T06:52:31.702060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745772+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:59.745772+0000","last_peered":"2024-09-21T06:52:59.745772+0000","last_clean":"2024-09-21T06:52:59.745772+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:59.745772+0000","last_undegraded":"2024-09-21T06:52:59.745772+0000","last_fullsized":"2024-09-21T06:52:59.745772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":381,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:03.847696+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:54:03.847696+0000","last_peered":"2024-09-21T06:54:03.847696+0000","last_clean":"2024-09-21T06:54:03.847696+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:54:03.847696+0000","last_undegraded":"2024-09-21T06:54:03.847696+0000","last_fullsized":"2024-09-21T06:54:03.847696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73114,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":222,"num_write":44,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645757+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645757+0000","last_peered":"2024-09-21T06:52:31.645757+0000","last_clean":"2024-09-21T06:52:31.645757+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645757+0000","last_undegraded":"2024-09-21T06:52:31.645757+0000","last_fullsized":"2024-09-21T06:52:31.645757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.747527+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:59.747527+0000","last_peered":"2024-09-21T06:52:59.747527+0000","last_clean":"2024-09-21T06:52:59.747527+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:59.747527+0000","last_undegraded":"2024-09-21T06:52:59.747527+0000","last_fullsized":"2024-09-21T06:52:59.747527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702309+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702309+0000","last_peered":"2024-09-21T06:52:31.702309+0000","last_clean":"2024-09-21T06:52:31.702309+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702309+0000","last_undegraded":"2024-09-21T06:52:31.702309+0000","last_fullsized":"2024-09-21T06:52:31.702309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699508+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699508+0000","last_peered":"2024-09-21T06:52:31.699508+0000","last_clean":"2024-09-21T06:52:31.699508+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699508+0000","last_undegraded":"2024-09-21T06:52:31.699508+0000","last_fullsized":"2024-09-21T06:52:31.699508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757106+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:59.757106+0000","last_peered":"2024-09-21T06:52:59.757106+0000","last_clean":"2024-09-21T06:52:59.757106+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:59.757106+0000","last_undegraded":"2024-09-21T06:52:59.757106+0000","last_fullsized":"2024-09-21T06:52:59.757106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648918+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648918+0000","last_peered":"2024-09-21T06:52:31.648918+0000","last_clean":"2024-09-21T06:52:31.648918+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648918+0000","last_undegraded":"2024-09-21T06:52:31.648918+0000","last_fullsized":"2024-09-21T06:52:31.648918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645559+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645559+0000","last_peered":"2024-09-21T06:52:31.645559+0000","last_clean":"2024-09-21T06:52:31.645559+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645559+0000","last_undegraded":"2024-09-21T06:52:31.645559+0000","last_fullsized":"2024-09-21T06:52:31.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750255+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:59.750255+0000","last_peered":"2024-09-21T06:52:59.750255+0000","last_clean":"2024-09-21T06:52:59.750255+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:59.750255+0000","last_undegraded":"2024-09-21T06:52:59.750255+0000","last_fullsized":"2024-09-21T06:52:59.750255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":331,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:03.847571+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:54:03.847571+0000","last_peered":"2024-09-21T06:54:03.847571+0000","last_clean":"2024-09-21T06:54:03.847571+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:54:03.847571+0000","last_undegraded":"2024-09-21T06:54:03.847571+0000","last_fullsized":"2024-09-21T06:54:03.847571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892718+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892718+0000","last_peered":"2024-09-21T06:52:31.892718+0000","last_clean":"2024-09-21T06:52:31.892718+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892718+0000","last_undegraded":"2024-09-21T06:52:31.892718+0000","last_fullsized":"2024-09-21T06:52:31.892718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892186+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892186+0000","last_peered":"2024-09-21T06:52:31.892186+0000","last_clean":"2024-09-21T06:52:31.892186+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892186+0000","last_undegraded":"2024-09-21T06:52:31.892186+0000","last_fullsized":"2024-09-21T06:52:31.892186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741381+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:59.741381+0000","last_peered":"2024-09-21T06:52:59.741381+0000","last_clean":"2024-09-21T06:52:59.741381+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:59.741381+0000","last_undegraded":"2024-09-21T06:52:59.741381+0000","last_fullsized":"2024-09-21T06:52:59.741381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701609+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701609+0000","last_peered":"2024-09-21T06:52:31.701609+0000","last_clean":"2024-09-21T06:52:31.701609+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701609+0000","last_undegraded":"2024-09-21T06:52:31.701609+0000","last_fullsized":"2024-09-21T06:52:31.701609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645808+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645808+0000","last_peered":"2024-09-21T06:52:31.645808+0000","last_clean":"2024-09-21T06:52:31.645808+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645808+0000","last_undegraded":"2024-09-21T06:52:31.645808+0000","last_fullsized":"2024-09-21T06:52:31.645808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742667+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:59.742667+0000","last_peered":"2024-09-21T06:52:59.742667+0000","last_clean":"2024-09-21T06:52:59.742667+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:59.742667+0000","last_undegraded":"2024-09-21T06:52:59.742667+0000","last_fullsized":"2024-09-21T06:52:59.742667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700994+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700994+0000","last_peered":"2024-09-21T06:52:31.700994+0000","last_clean":"2024-09-21T06:52:31.700994+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700994+0000","last_undegraded":"2024-09-21T06:52:31.700994+0000","last_fullsized":"2024-09-21T06:52:31.700994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645981+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645981+0000","last_peered":"2024-09-21T06:52:31.645981+0000","last_clean":"2024-09-21T06:52:31.645981+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645981+0000","last_undegraded":"2024-09-21T06:52:31.645981+0000","last_fullsized":"2024-09-21T06:52:31.645981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749130+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:59.749130+0000","last_peered":"2024-09-21T06:52:59.749130+0000","last_clean":"2024-09-21T06:52:59.749130+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:59.749130+0000","last_undegraded":"2024-09-21T06:52:59.749130+0000","last_fullsized":"2024-09-21T06:52:59.749130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703865+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703865+0000","last_peered":"2024-09-21T06:52:31.703865+0000","last_clean":"2024-09-21T06:52:31.703865+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703865+0000","last_undegraded":"2024-09-21T06:52:31.703865+0000","last_fullsized":"2024-09-21T06:52:31.703865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":201,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:59.841987+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:53:59.841987+0000","last_peered":"2024-09-21T06:53:59.841987+0000","last_clean":"2024-09-21T06:53:59.841987+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:53:59.841987+0000","last_undegraded":"2024-09-21T06:53:59.841987+0000","last_fullsized":"2024-09-21T06:53:59.841987+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.736418+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:59.736418+0000","last_peered":"2024-09-21T06:52:59.736418+0000","last_clean":"2024-09-21T06:52:59.736418+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:59.736418+0000","last_undegraded":"2024-09-21T06:52:59.736418+0000","last_fullsized":"2024-09-21T06:52:59.736418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704051+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.704051+0000","last_peered":"2024-09-21T06:52:31.704051+0000","last_clean":"2024-09-21T06:52:31.704051+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.704051+0000","last_undegraded":"2024-09-21T06:52:31.704051+0000","last_fullsized":"2024-09-21T06:52:31.704051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.739975+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:59.739975+0000","last_peered":"2024-09-21T06:52:59.739975+0000","last_clean":"2024-09-21T06:52:59.739975+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:59.739975+0000","last_undegraded":"2024-09-21T06:52:59.739975+0000","last_fullsized":"2024-09-21T06:52:59.739975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749347+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:59.749347+0000","last_peered":"2024-09-21T06:52:59.749347+0000","last_clean":"2024-09-21T06:52:59.749347+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:59.749347+0000","last_undegraded":"2024-09-21T06:52:59.749347+0000","last_fullsized":"2024-09-21T06:52:59.749347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746716+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:59.746716+0000","last_peered":"2024-09-21T06:52:59.746716+0000","last_clean":"2024-09-21T06:52:59.746716+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:59.746716+0000","last_undegraded":"2024-09-21T06:52:59.746716+0000","last_fullsized":"2024-09-21T06:52:59.746716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757121+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:59.757121+0000","last_peered":"2024-09-21T06:52:59.757121+0000","last_clean":"2024-09-21T06:52:59.757121+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:59.757121+0000","last_undegraded":"2024-09-21T06:52:59.757121+0000","last_fullsized":"2024-09-21T06:52:59.757121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756339+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:59.756339+0000","last_peered":"2024-09-21T06:52:59.756339+0000","last_clean":"2024-09-21T06:52:59.756339+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:59.756339+0000","last_undegraded":"2024-09-21T06:52:59.756339+0000","last_fullsized":"2024-09-21T06:52:59.756339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734085+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:59.734085+0000","last_peered":"2024-09-21T06:52:59.734085+0000","last_clean":"2024-09-21T06:52:59.734085+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:59.734085+0000","last_undegraded":"2024-09-21T06:52:59.734085+0000","last_fullsized":"2024-09-21T06:52:59.734085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749595+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:59.749595+0000","last_peered":"2024-09-21T06:52:59.749595+0000","last_clean":"2024-09-21T06:52:59.749595+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:59.749595+0000","last_undegraded":"2024-09-21T06:52:59.749595+0000","last_fullsized":"2024-09-21T06:52:59.749595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.737176+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:59.737176+0000","last_peered":"2024-09-21T06:52:59.737176+0000","last_clean":"2024-09-21T06:52:59.737176+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:59.737176+0000","last_undegraded":"2024-09-21T06:52:59.737176+0000","last_fullsized":"2024-09-21T06:52:59.737176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746235+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:59.746235+0000","last_peered":"2024-09-21T06:52:59.746235+0000","last_clean":"2024-09-21T06:52:59.746235+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:59.746235+0000","last_undegraded":"2024-09-21T06:52:59.746235+0000","last_fullsized":"2024-09-21T06:52:59.746235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740649+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:59.740649+0000","last_peered":"2024-09-21T06:52:59.740649+0000","last_clean":"2024-09-21T06:52:59.740649+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:59.740649+0000","last_undegraded":"2024-09-21T06:52:59.740649+0000","last_fullsized":"2024-09-21T06:52:59.740649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753696+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:59.753696+0000","last_peered":"2024-09-21T06:52:59.753696+0000","last_clean":"2024-09-21T06:52:59.753696+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:59.753696+0000","last_undegraded":"2024-09-21T06:52:59.753696+0000","last_fullsized":"2024-09-21T06:52:59.753696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756495+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:59.756495+0000","last_peered":"2024-09-21T06:52:59.756495+0000","last_clean":"2024-09-21T06:52:59.756495+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:59.756495+0000","last_undegraded":"2024-09-21T06:52:59.756495+0000","last_fullsized":"2024-09-21T06:52:59.756495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734175+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:59.734175+0000","last_peered":"2024-09-21T06:52:59.734175+0000","last_clean":"2024-09-21T06:52:59.734175+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:59.734175+0000","last_undegraded":"2024-09-21T06:52:59.734175+0000","last_fullsized":"2024-09-21T06:52:59.734175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745394+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:59.745394+0000","last_peered":"2024-09-21T06:52:59.745394+0000","last_clean":"2024-09-21T06:52:59.745394+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:59.745394+0000","last_undegraded":"2024-09-21T06:52:59.745394+0000","last_fullsized":"2024-09-21T06:52:59.745394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751016+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:59.751016+0000","last_peered":"2024-09-21T06:52:59.751016+0000","last_clean":"2024-09-21T06:52:59.751016+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:59.751016+0000","last_undegraded":"2024-09-21T06:52:59.751016+0000","last_fullsized":"2024-09-21T06:52:59.751016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755908+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:59.755908+0000","last_peered":"2024-09-21T06:52:59.755908+0000","last_clean":"2024-09-21T06:52:59.755908+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:59.755908+0000","last_undegraded":"2024-09-21T06:52:59.755908+0000","last_fullsized":"2024-09-21T06:52:59.755908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752146+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:59.752146+0000","last_peered":"2024-09-21T06:52:59.752146+0000","last_clean":"2024-09-21T06:52:59.752146+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:59.752146+0000","last_undegraded":"2024-09-21T06:52:59.752146+0000","last_fullsized":"2024-09-21T06:52:59.752146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754481+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:59.754481+0000","last_peered":"2024-09-21T06:52:59.754481+0000","last_clean":"2024-09-21T06:52:59.754481+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:59.754481+0000","last_undegraded":"2024-09-21T06:52:59.754481+0000","last_fullsized":"2024-09-21T06:52:59.754481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758054+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:59.758054+0000","last_peered":"2024-09-21T06:52:59.758054+0000","last_clean":"2024-09-21T06:52:59.758054+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:59.758054+0000","last_undegraded":"2024-09-21T06:52:59.758054+0000","last_fullsized":"2024-09-21T06:52:59.758054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756767+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:59.756767+0000","last_peered":"2024-09-21T06:52:59.756767+0000","last_clean":"2024-09-21T06:52:59.756767+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:59.756767+0000","last_undegraded":"2024-09-21T06:52:59.756767+0000","last_fullsized":"2024-09-21T06:52:59.756767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755238+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:59.755238+0000","last_peered":"2024-09-21T06:52:59.755238+0000","last_clean":"2024-09-21T06:52:59.755238+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:59.755238+0000","last_undegraded":"2024-09-21T06:52:59.755238+0000","last_fullsized":"2024-09-21T06:52:59.755238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752053+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:59.752053+0000","last_peered":"2024-09-21T06:52:59.752053+0000","last_clean":"2024-09-21T06:52:59.752053+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:59.752053+0000","last_undegraded":"2024-09-21T06:52:59.752053+0000","last_fullsized":"2024-09-21T06:52:59.752053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.759125+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:59.759125+0000","last_peered":"2024-09-21T06:52:59.759125+0000","last_clean":"2024-09-21T06:52:59.759125+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:59.759125+0000","last_undegraded":"2024-09-21T06:52:59.759125+0000","last_fullsized":"2024-09-21T06:52:59.759125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742201+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:59.742201+0000","last_peered":"2024-09-21T06:52:59.742201+0000","last_clean":"2024-09-21T06:52:59.742201+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:59.742201+0000","last_undegraded":"2024-09-21T06:52:59.742201+0000","last_fullsized":"2024-09-21T06:52:59.742201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755485+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:59.755485+0000","last_peered":"2024-09-21T06:52:59.755485+0000","last_clean":"2024-09-21T06:52:59.755485+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:59.755485+0000","last_undegraded":"2024-09-21T06:52:59.755485+0000","last_fullsized":"2024-09-21T06:52:59.755485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752958+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:59.752958+0000","last_peered":"2024-09-21T06:52:59.752958+0000","last_clean":"2024-09-21T06:52:59.752958+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:59.752958+0000","last_undegraded":"2024-09-21T06:52:59.752958+0000","last_fullsized":"2024-09-21T06:52:59.752958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754740+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:59.754740+0000","last_peered":"2024-09-21T06:52:59.754740+0000","last_clean":"2024-09-21T06:52:59.754740+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:59.754740+0000","last_undegraded":"2024-09-21T06:52:59.754740+0000","last_fullsized":"2024-09-21T06:52:59.754740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746351+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:59.746351+0000","last_peered":"2024-09-21T06:52:59.746351+0000","last_clean":"2024-09-21T06:52:59.746351+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:59.746351+0000","last_undegraded":"2024-09-21T06:52:59.746351+0000","last_fullsized":"2024-09-21T06:52:59.746351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749389+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:59.749389+0000","last_peered":"2024-09-21T06:52:59.749389+0000","last_clean":"2024-09-21T06:52:59.749389+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:59.749389+0000","last_undegraded":"2024-09-21T06:52:59.749389+0000","last_fullsized":"2024-09-21T06:52:59.749389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757852+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:59.757852+0000","last_peered":"2024-09-21T06:52:59.757852+0000","last_clean":"2024-09-21T06:52:59.757852+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:59.757852+0000","last_undegraded":"2024-09-21T06:52:59.757852+0000","last_fullsized":"2024-09-21T06:52:59.757852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750926+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:59.750926+0000","last_peered":"2024-09-21T06:52:59.750926+0000","last_clean":"2024-09-21T06:52:59.750926+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:59.750926+0000","last_undegraded":"2024-09-21T06:52:59.750926+0000","last_fullsized":"2024-09-21T06:52:59.750926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":868435,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1782,"num_read_kb":1117,"num_write":495,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2973696,"data_stored":2605305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":376,"ondisk_log_size":376,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710774,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":531799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29172,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715980,"statfs":{"total":95995035648,"available":95965163520,"internally_reserved":0,"allocated":2138112,"data_stored":1584951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":928487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266758,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1445888,"data_stored":924747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462987,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1129034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":527522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822743,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29036,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716116,"statfs":{"total":95995035648,"available":95965302784,"internally_reserved":0,"allocated":2060288,"data_stored":1517549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953565,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28880,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716272,"statfs":{"total":95995035648,"available":95965462528,"internally_reserved":0,"allocated":1839104,"data_stored":1313379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378729,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":600953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":602820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705779,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":468626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:54:07.515 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T06:54:07.515 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T06:54:07.888 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 -- 172.21.15.114:0/4029994011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc114740 msgr2=0x7f37cc116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 --2- 172.21.15.114:0/4029994011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc114740 0x7f37cc116b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009ca0 tx=0x7f37c402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 -- 172.21.15.114:0/4029994011 shutdown_connections 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 --2- 172.21.15.114:0/4029994011 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc114740 0x7f37cc116b60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 --2- 172.21.15.114:0/4029994011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37cc078290 0x7f37cc114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 --2- 172.21.15.114:0/4029994011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc077940 0x7f37cc077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 -- 172.21.15.114:0/4029994011 >> 172.21.15.114:0/4029994011 conn(0x7f37cc101810 msgr2=0x7f37cc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:08.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 -- 172.21.15.114:0/4029994011 shutdown_connections 2024-09-21T06:54:08.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.621+0000 7f37d3000640 1 -- 172.21.15.114:0/4029994011 wait complete. 2024-09-21T06:54:08.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.622+0000 7f37d3000640 1 Processor -- start 2024-09-21T06:54:08.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 -- start start 2024-09-21T06:54:08.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 0x7f37cc1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:08.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37cc078290 0x7f37cc1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:08.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc114740 0x7f37cc1a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:08.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc11a110 con 0x7f37cc077940 2024-09-21T06:54:08.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc119f90 con 0x7f37cc078290 2024-09-21T06:54:08.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37cc11a290 con 0x7f37cc114740 2024-09-21T06:54:08.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 0x7f37cc1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:08.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37cbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 0x7f37cc1a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52350/0 (socket says 172.21.15.114:52350) 2024-09-21T06:54:08.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37cbe00640 1 -- 172.21.15.114:0/125510579 learned_addr learned my addr 172.21.15.114:0/125510579 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:08.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37c3e00640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37cc078290 0x7f37cc1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.623+0000 7f37d1000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc114740 0x7f37cc1a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37cbe00640 1 -- 172.21.15.114:0/125510579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc114740 msgr2=0x7f37cc1a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37cbe00640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc114740 0x7f37cc1a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37cbe00640 1 -- 172.21.15.114:0/125510579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37cc078290 msgr2=0x7f37cc1a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37cbe00640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37cc078290 0x7f37cc1a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37cbe00640 1 -- 172.21.15.114:0/125510579 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37cc1a6a70 con 0x7f37cc077940 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37cbe00640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 0x7f37cc1a0b00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9f0 tx=0x7f37bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cd80 con 0x7f37cc077940 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37bc0054d0 con 0x7f37cc077940 2024-09-21T06:54:08.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37cc1a6d60 con 0x7f37cc077940 2024-09-21T06:54:08.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc010690 con 0x7f37cc077940 2024-09-21T06:54:08.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.624+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37cc1ad4c0 con 0x7f37cc077940 2024-09-21T06:54:08.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.626+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37cc077d50 con 0x7f37cc077940 2024-09-21T06:54:08.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.628+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f37bc004e30 con 0x7f37cc077940 2024-09-21T06:54:08.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.630+0000 7f37c9600640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37b00b65f0 0x7f37b00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:08.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.631+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f37bc0e7fa0 con 0x7f37cc077940 2024-09-21T06:54:08.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.631+0000 7f37c3e00640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37b00b65f0 0x7f37b00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:08.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.631+0000 7f37c3e00640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37b00b65f0 0x7f37b00b8b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37cc07b200 tx=0x7f37b4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:08.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.633+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f37bc0ae1e0 con 0x7f37cc077940 2024-09-21T06:54:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:08 smithi114 ceph-mon[30654]: from='client.25583 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:08.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:08 smithi114 ceph-mon[30654]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:08.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.820+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37cc063800 con 0x7f37b00b65f0 2024-09-21T06:54:08.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.826+0000 7f37c9600640 1 -- 172.21.15.114:0/125510579 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340575 (secure 0 0 0) 0x7f37cc063800 con 0x7f37b00b65f0 2024-09-21T06:54:08.831 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:08.833 INFO:teuthology.orchestra.run.smithi114.stderr:dumped all 2024-09-21T06:54:08.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37b00b65f0 msgr2=0x7f37b00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:08.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37b00b65f0 0x7f37b00b8b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f37cc07b200 tx=0x7f37b4005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:08.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 msgr2=0x7f37cc1a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:08.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 0x7f37cc1a0b00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e9f0 tx=0x7f37bc00edf0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 shutdown_connections 2024-09-21T06:54:08.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc114740 0x7f37cc1a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37b00b65f0 0x7f37b00b8b00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37cc078290 0x7f37cc1a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 --2- 172.21.15.114:0/125510579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f37cc077940 0x7f37cc1a0b00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:08.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 >> 172.21.15.114:0/125510579 conn(0x7f37cc101810 msgr2=0x7f37cc116740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:08.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 shutdown_connections 2024-09-21T06:54:08.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:08.831+0000 7f37d3000640 1 -- 172.21.15.114:0/125510579 wait complete. 2024-09-21T06:54:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:08 smithi183 ceph-mon[34219]: from='client.25583 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:08.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:08 smithi183 ceph-mon[34219]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:08 smithi193 ceph-mon[34601]: from='client.25583 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:08 smithi193 ceph-mon[34601]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:09.577 INFO:teuthology.orchestra.run.smithi114.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-21T06:54:07.546386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1459037,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1880,"num_read_kb":1188,"num_write":620,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":970,"ondisk_log_size":970,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378536,"kb_used_data":16628,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124563288,"statfs":{"total":1151940427776,"available":1151552806912,"internally_reserved":0,"allocated":17027072,"data_stored":10720661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":788130,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":452,"num_read_kb":353,"num_write":225,"num_write_kb":2090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003631"},"pg_stats":[{"pgid":"2.1f","version":"89'15","reported_seq":200,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:01.157297+0000","last_change":"2024-09-21T06:50:44.198606+0000","last_active":"2024-09-21T06:54:01.157297+0000","last_peered":"2024-09-21T06:54:01.157297+0000","last_clean":"2024-09-21T06:54:01.157297+0000","last_became_active":"2024-09-21T06:50:44.198075+0000","last_became_peered":"2024-09-21T06:50:44.198075+0000","last_unstale":"2024-09-21T06:54:01.157297+0000","last_undegraded":"2024-09-21T06:54:01.157297+0000","last_fullsized":"2024-09-21T06:54:01.157297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:01.824797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":141,"num_read_kb":141,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741199+0000","last_change":"2024-09-21T06:50:47.228479+0000","last_active":"2024-09-21T06:52:59.741199+0000","last_peered":"2024-09-21T06:52:59.741199+0000","last_clean":"2024-09-21T06:52:59.741199+0000","last_became_active":"2024-09-21T06:50:47.228150+0000","last_became_peered":"2024-09-21T06:50:47.228150+0000","last_unstale":"2024-09-21T06:52:59.741199+0000","last_undegraded":"2024-09-21T06:52:59.741199+0000","last_fullsized":"2024-09-21T06:52:59.741199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:07.704422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703671+0000","last_change":"2024-09-21T06:50:56.576341+0000","last_active":"2024-09-21T06:52:31.703671+0000","last_peered":"2024-09-21T06:52:31.703671+0000","last_clean":"2024-09-21T06:52:31.703671+0000","last_became_active":"2024-09-21T06:50:56.575894+0000","last_became_peered":"2024-09-21T06:50:56.575894+0000","last_unstale":"2024-09-21T06:52:31.703671+0000","last_undegraded":"2024-09-21T06:52:31.703671+0000","last_fullsized":"2024-09-21T06:52:31.703671+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:16.787409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892550+0000","last_change":"2024-09-21T06:50:44.686578+0000","last_active":"2024-09-21T06:52:31.892550+0000","last_peered":"2024-09-21T06:52:31.892550+0000","last_clean":"2024-09-21T06:52:31.892550+0000","last_became_active":"2024-09-21T06:50:44.686166+0000","last_became_peered":"2024-09-21T06:50:44.686166+0000","last_unstale":"2024-09-21T06:52:31.892550+0000","last_undegraded":"2024-09-21T06:52:31.892550+0000","last_fullsized":"2024-09-21T06:52:31.892550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:11.458415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.224763+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.224365+0000","last_became_peered":"2024-09-21T06:50:47.224365+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:47.486317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699254+0000","last_change":"2024-09-21T06:51:30.754958+0000","last_active":"2024-09-21T06:52:31.699254+0000","last_peered":"2024-09-21T06:52:31.699254+0000","last_clean":"2024-09-21T06:52:31.699254+0000","last_became_active":"2024-09-21T06:51:30.754215+0000","last_became_peered":"2024-09-21T06:51:30.754215+0000","last_unstale":"2024-09-21T06:52:31.699254+0000","last_undegraded":"2024-09-21T06:52:31.699254+0000","last_fullsized":"2024-09-21T06:52:31.699254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.181627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:53:16.554226+0000","last_change":"2024-09-21T06:50:44.198765+0000","last_active":"2024-09-21T06:53:16.554226+0000","last_peered":"2024-09-21T06:53:16.554226+0000","last_clean":"2024-09-21T06:53:16.554226+0000","last_became_active":"2024-09-21T06:50:44.198295+0000","last_became_peered":"2024-09-21T06:50:44.198295+0000","last_unstale":"2024-09-21T06:53:16.554226+0000","last_undegraded":"2024-09-21T06:53:16.554226+0000","last_fullsized":"2024-09-21T06:53:16.554226+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:46.642101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755557+0000","last_change":"2024-09-21T06:50:47.225614+0000","last_active":"2024-09-21T06:52:59.755557+0000","last_peered":"2024-09-21T06:52:59.755557+0000","last_clean":"2024-09-21T06:52:59.755557+0000","last_became_active":"2024-09-21T06:50:47.225220+0000","last_became_peered":"2024-09-21T06:50:47.225220+0000","last_unstale":"2024-09-21T06:52:59.755557+0000","last_undegraded":"2024-09-21T06:52:59.755557+0000","last_fullsized":"2024-09-21T06:52:59.755557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:56.019902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699992+0000","last_change":"2024-09-21T06:50:56.582116+0000","last_active":"2024-09-21T06:52:31.699992+0000","last_peered":"2024-09-21T06:52:31.699992+0000","last_clean":"2024-09-21T06:52:31.699992+0000","last_became_active":"2024-09-21T06:50:56.581600+0000","last_became_peered":"2024-09-21T06:50:56.581600+0000","last_unstale":"2024-09-21T06:52:31.699992+0000","last_undegraded":"2024-09-21T06:52:31.699992+0000","last_fullsized":"2024-09-21T06:52:31.699992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:14.423663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891939+0000","last_change":"2024-09-21T06:50:44.201431+0000","last_active":"2024-09-21T06:52:31.891939+0000","last_peered":"2024-09-21T06:52:31.891939+0000","last_clean":"2024-09-21T06:52:31.891939+0000","last_became_active":"2024-09-21T06:50:44.201032+0000","last_became_peered":"2024-09-21T06:50:44.201032+0000","last_unstale":"2024-09-21T06:52:31.891939+0000","last_undegraded":"2024-09-21T06:52:31.891939+0000","last_fullsized":"2024-09-21T06:52:31.891939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:29.926686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751784+0000","last_change":"2024-09-21T06:50:47.224147+0000","last_active":"2024-09-21T06:52:59.751784+0000","last_peered":"2024-09-21T06:52:59.751784+0000","last_clean":"2024-09-21T06:52:59.751784+0000","last_became_active":"2024-09-21T06:50:47.223782+0000","last_became_peered":"2024-09-21T06:50:47.223782+0000","last_unstale":"2024-09-21T06:52:59.751784+0000","last_undegraded":"2024-09-21T06:52:59.751784+0000","last_fullsized":"2024-09-21T06:52:59.751784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:52.951324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699375+0000","last_change":"2024-09-21T06:50:56.569396+0000","last_active":"2024-09-21T06:52:31.699375+0000","last_peered":"2024-09-21T06:52:31.699375+0000","last_clean":"2024-09-21T06:52:31.699375+0000","last_became_active":"2024-09-21T06:50:56.568949+0000","last_became_peered":"2024-09-21T06:50:56.568949+0000","last_unstale":"2024-09-21T06:52:31.699375+0000","last_undegraded":"2024-09-21T06:52:31.699375+0000","last_fullsized":"2024-09-21T06:52:31.699375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:50.082822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755326+0000","last_change":"2024-09-21T06:50:47.227779+0000","last_active":"2024-09-21T06:52:59.755326+0000","last_peered":"2024-09-21T06:52:59.755326+0000","last_clean":"2024-09-21T06:52:59.755326+0000","last_became_active":"2024-09-21T06:50:47.226275+0000","last_became_peered":"2024-09-21T06:50:47.226275+0000","last_unstale":"2024-09-21T06:52:59.755326+0000","last_undegraded":"2024-09-21T06:52:59.755326+0000","last_fullsized":"2024-09-21T06:52:59.755326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:01.721102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645462+0000","last_change":"2024-09-21T06:51:30.680342+0000","last_active":"2024-09-21T06:52:31.645462+0000","last_peered":"2024-09-21T06:52:31.645462+0000","last_clean":"2024-09-21T06:52:31.645462+0000","last_became_active":"2024-09-21T06:51:30.674594+0000","last_became_peered":"2024-09-21T06:51:30.674594+0000","last_unstale":"2024-09-21T06:52:31.645462+0000","last_undegraded":"2024-09-21T06:52:31.645462+0000","last_fullsized":"2024-09-21T06:52:31.645462+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:37.443077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.574937+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.574101+0000","last_became_peered":"2024-09-21T06:50:56.574101+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:15.524301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755915+0000","last_change":"2024-09-21T06:50:47.227342+0000","last_active":"2024-09-21T06:52:59.755915+0000","last_peered":"2024-09-21T06:52:59.755915+0000","last_clean":"2024-09-21T06:52:59.755915+0000","last_became_active":"2024-09-21T06:50:47.226893+0000","last_became_peered":"2024-09-21T06:50:47.226893+0000","last_unstale":"2024-09-21T06:52:59.755915+0000","last_undegraded":"2024-09-21T06:52:59.755915+0000","last_fullsized":"2024-09-21T06:52:59.755915+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:50.468899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704037+0000","last_change":"2024-09-21T06:50:44.198548+0000","last_active":"2024-09-21T06:52:31.704037+0000","last_peered":"2024-09-21T06:52:31.704037+0000","last_clean":"2024-09-21T06:52:31.704037+0000","last_became_active":"2024-09-21T06:50:44.198171+0000","last_became_peered":"2024-09-21T06:50:44.198171+0000","last_unstale":"2024-09-21T06:52:31.704037+0000","last_undegraded":"2024-09-21T06:52:31.704037+0000","last_fullsized":"2024-09-21T06:52:31.704037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:03.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699968+0000","last_change":"2024-09-21T06:50:56.582397+0000","last_active":"2024-09-21T06:52:31.699968+0000","last_peered":"2024-09-21T06:52:31.699968+0000","last_clean":"2024-09-21T06:52:31.699968+0000","last_became_active":"2024-09-21T06:50:56.581460+0000","last_became_peered":"2024-09-21T06:50:56.581460+0000","last_unstale":"2024-09-21T06:52:31.699968+0000","last_undegraded":"2024-09-21T06:52:31.699968+0000","last_fullsized":"2024-09-21T06:52:31.699968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:55.912856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745662+0000","last_change":"2024-09-21T06:50:47.222456+0000","last_active":"2024-09-21T06:52:59.745662+0000","last_peered":"2024-09-21T06:52:59.745662+0000","last_clean":"2024-09-21T06:52:59.745662+0000","last_became_active":"2024-09-21T06:50:47.222094+0000","last_became_peered":"2024-09-21T06:50:47.222094+0000","last_unstale":"2024-09-21T06:52:59.745662+0000","last_undegraded":"2024-09-21T06:52:59.745662+0000","last_fullsized":"2024-09-21T06:52:59.745662+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:52.035081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892935+0000","last_change":"2024-09-21T06:50:44.192578+0000","last_active":"2024-09-21T06:52:31.892935+0000","last_peered":"2024-09-21T06:52:31.892935+0000","last_clean":"2024-09-21T06:52:31.892935+0000","last_became_active":"2024-09-21T06:50:44.192189+0000","last_became_peered":"2024-09-21T06:50:44.192189+0000","last_unstale":"2024-09-21T06:52:31.892935+0000","last_undegraded":"2024-09-21T06:52:31.892935+0000","last_fullsized":"2024-09-21T06:52:31.892935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:41.608920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.646491+0000","last_change":"2024-09-21T06:50:56.588576+0000","last_active":"2024-09-21T06:52:31.646491+0000","last_peered":"2024-09-21T06:52:31.646491+0000","last_clean":"2024-09-21T06:52:31.646491+0000","last_became_active":"2024-09-21T06:50:56.587232+0000","last_became_peered":"2024-09-21T06:50:56.587232+0000","last_unstale":"2024-09-21T06:52:31.646491+0000","last_undegraded":"2024-09-21T06:52:31.646491+0000","last_fullsized":"2024-09-21T06:52:31.646491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:40.019860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744826+0000","last_change":"2024-09-21T06:50:47.216878+0000","last_active":"2024-09-21T06:52:59.744826+0000","last_peered":"2024-09-21T06:52:59.744826+0000","last_clean":"2024-09-21T06:52:59.744826+0000","last_became_active":"2024-09-21T06:50:47.216452+0000","last_became_peered":"2024-09-21T06:50:47.216452+0000","last_unstale":"2024-09-21T06:52:59.744826+0000","last_undegraded":"2024-09-21T06:52:59.744826+0000","last_fullsized":"2024-09-21T06:52:59.744826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:50.888284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836870+0000","last_change":"2024-09-21T06:50:44.688636+0000","last_active":"2024-09-21T06:52:31.836870+0000","last_peered":"2024-09-21T06:52:31.836870+0000","last_clean":"2024-09-21T06:52:31.836870+0000","last_became_active":"2024-09-21T06:50:44.688290+0000","last_became_peered":"2024-09-21T06:50:44.688290+0000","last_unstale":"2024-09-21T06:52:31.836870+0000","last_undegraded":"2024-09-21T06:52:31.836870+0000","last_fullsized":"2024-09-21T06:52:31.836870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:22.430498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700150+0000","last_change":"2024-09-21T06:50:56.582126+0000","last_active":"2024-09-21T06:52:31.700150+0000","last_peered":"2024-09-21T06:52:31.700150+0000","last_clean":"2024-09-21T06:52:31.700150+0000","last_became_active":"2024-09-21T06:50:56.581420+0000","last_became_peered":"2024-09-21T06:50:56.581420+0000","last_unstale":"2024-09-21T06:52:31.700150+0000","last_undegraded":"2024-09-21T06:52:31.700150+0000","last_fullsized":"2024-09-21T06:52:31.700150+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:17.234847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755751+0000","last_change":"2024-09-21T06:51:30.669240+0000","last_active":"2024-09-21T06:52:59.755751+0000","last_peered":"2024-09-21T06:52:59.755751+0000","last_clean":"2024-09-21T06:52:59.755751+0000","last_became_active":"2024-09-21T06:51:30.668774+0000","last_became_peered":"2024-09-21T06:51:30.668774+0000","last_unstale":"2024-09-21T06:52:59.755751+0000","last_undegraded":"2024-09-21T06:52:59.755751+0000","last_fullsized":"2024-09-21T06:52:59.755751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:39:25.659026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700368+0000","last_change":"2024-09-21T06:50:44.941956+0000","last_active":"2024-09-21T06:52:31.700368+0000","last_peered":"2024-09-21T06:52:31.700368+0000","last_clean":"2024-09-21T06:52:31.700368+0000","last_became_active":"2024-09-21T06:50:44.941574+0000","last_became_peered":"2024-09-21T06:50:44.941574+0000","last_unstale":"2024-09-21T06:52:31.700368+0000","last_undegraded":"2024-09-21T06:52:31.700368+0000","last_fullsized":"2024-09-21T06:52:31.700368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:06.832175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703897+0000","last_change":"2024-09-21T06:50:56.566291+0000","last_active":"2024-09-21T06:52:31.703897+0000","last_peered":"2024-09-21T06:52:31.703897+0000","last_clean":"2024-09-21T06:52:31.703897+0000","last_became_active":"2024-09-21T06:50:56.565716+0000","last_became_peered":"2024-09-21T06:50:56.565716+0000","last_unstale":"2024-09-21T06:52:31.703897+0000","last_undegraded":"2024-09-21T06:52:31.703897+0000","last_fullsized":"2024-09-21T06:52:31.703897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:26.027032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741893+0000","last_change":"2024-09-21T06:50:47.232559+0000","last_active":"2024-09-21T06:52:59.741893+0000","last_peered":"2024-09-21T06:52:59.741893+0000","last_clean":"2024-09-21T06:52:59.741893+0000","last_became_active":"2024-09-21T06:50:47.229357+0000","last_became_peered":"2024-09-21T06:50:47.229357+0000","last_unstale":"2024-09-21T06:52:59.741893+0000","last_undegraded":"2024-09-21T06:52:59.741893+0000","last_fullsized":"2024-09-21T06:52:59.741893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:10.738484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":234,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:02.219221+0000","last_change":"2024-09-21T06:50:44.200191+0000","last_active":"2024-09-21T06:54:02.219221+0000","last_peered":"2024-09-21T06:54:02.219221+0000","last_clean":"2024-09-21T06:54:02.219221+0000","last_became_active":"2024-09-21T06:50:44.199462+0000","last_became_peered":"2024-09-21T06:50:44.199462+0000","last_unstale":"2024-09-21T06:54:02.219221+0000","last_undegraded":"2024-09-21T06:54:02.219221+0000","last_fullsized":"2024-09-21T06:54:02.219221+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:36.550080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":185,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891910+0000","last_change":"2024-09-21T06:50:56.571608+0000","last_active":"2024-09-21T06:52:31.891910+0000","last_peered":"2024-09-21T06:52:31.891910+0000","last_clean":"2024-09-21T06:52:31.891910+0000","last_became_active":"2024-09-21T06:50:56.570776+0000","last_became_peered":"2024-09-21T06:50:56.570776+0000","last_unstale":"2024-09-21T06:52:31.891910+0000","last_undegraded":"2024-09-21T06:52:31.891910+0000","last_fullsized":"2024-09-21T06:52:31.891910+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:21.279676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742064+0000","last_change":"2024-09-21T06:50:47.216425+0000","last_active":"2024-09-21T06:52:59.742064+0000","last_peered":"2024-09-21T06:52:59.742064+0000","last_clean":"2024-09-21T06:52:59.742064+0000","last_became_active":"2024-09-21T06:50:47.215971+0000","last_became_peered":"2024-09-21T06:50:47.215971+0000","last_unstale":"2024-09-21T06:52:59.742064+0000","last_undegraded":"2024-09-21T06:52:59.742064+0000","last_fullsized":"2024-09-21T06:52:59.742064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:56.408845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892041+0000","last_change":"2024-09-21T06:50:44.689534+0000","last_active":"2024-09-21T06:52:31.892041+0000","last_peered":"2024-09-21T06:52:31.892041+0000","last_clean":"2024-09-21T06:52:31.892041+0000","last_became_active":"2024-09-21T06:50:44.688887+0000","last_became_peered":"2024-09-21T06:50:44.688887+0000","last_unstale":"2024-09-21T06:52:31.892041+0000","last_undegraded":"2024-09-21T06:52:31.892041+0000","last_fullsized":"2024-09-21T06:52:31.892041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:45.841243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645535+0000","last_change":"2024-09-21T06:50:56.562617+0000","last_active":"2024-09-21T06:52:31.645535+0000","last_peered":"2024-09-21T06:52:31.645535+0000","last_clean":"2024-09-21T06:52:31.645535+0000","last_became_active":"2024-09-21T06:50:56.562059+0000","last_became_peered":"2024-09-21T06:50:56.562059+0000","last_unstale":"2024-09-21T06:52:31.645535+0000","last_undegraded":"2024-09-21T06:52:31.645535+0000","last_fullsized":"2024-09-21T06:52:31.645535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:31.287653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754090+0000","last_change":"2024-09-21T06:52:31.701866+0000","last_active":"2024-09-21T06:52:59.754090+0000","last_peered":"2024-09-21T06:52:59.754090+0000","last_clean":"2024-09-21T06:52:59.754090+0000","last_became_active":"2024-09-21T06:52:31.701476+0000","last_became_peered":"2024-09-21T06:52:31.701476+0000","last_unstale":"2024-09-21T06:52:59.754090+0000","last_undegraded":"2024-09-21T06:52:59.754090+0000","last_fullsized":"2024-09-21T06:52:59.754090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:46.430092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645977+0000","last_change":"2024-09-21T06:50:44.686579+0000","last_active":"2024-09-21T06:52:31.645977+0000","last_peered":"2024-09-21T06:52:31.645977+0000","last_clean":"2024-09-21T06:52:31.645977+0000","last_became_active":"2024-09-21T06:50:44.686087+0000","last_became_peered":"2024-09-21T06:50:44.686087+0000","last_unstale":"2024-09-21T06:52:31.645977+0000","last_undegraded":"2024-09-21T06:52:31.645977+0000","last_fullsized":"2024-09-21T06:52:31.645977+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:07.267335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891517+0000","last_change":"2024-09-21T06:50:56.576987+0000","last_active":"2024-09-21T06:52:31.891517+0000","last_peered":"2024-09-21T06:52:31.891517+0000","last_clean":"2024-09-21T06:52:31.891517+0000","last_became_active":"2024-09-21T06:50:56.575928+0000","last_became_peered":"2024-09-21T06:50:56.575928+0000","last_unstale":"2024-09-21T06:52:31.891517+0000","last_undegraded":"2024-09-21T06:52:31.891517+0000","last_fullsized":"2024-09-21T06:52:31.891517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:50.929693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758656+0000","last_change":"2024-09-21T06:50:47.222431+0000","last_active":"2024-09-21T06:52:59.758656+0000","last_peered":"2024-09-21T06:52:59.758656+0000","last_clean":"2024-09-21T06:52:59.758656+0000","last_became_active":"2024-09-21T06:50:47.222071+0000","last_became_peered":"2024-09-21T06:50:47.222071+0000","last_unstale":"2024-09-21T06:52:59.758656+0000","last_undegraded":"2024-09-21T06:52:59.758656+0000","last_fullsized":"2024-09-21T06:52:59.758656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:13.172130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'8","reported_seq":197,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:02.845800+0000","last_change":"2024-09-21T06:50:44.193380+0000","last_active":"2024-09-21T06:54:02.845800+0000","last_peered":"2024-09-21T06:54:02.845800+0000","last_clean":"2024-09-21T06:54:02.845800+0000","last_became_active":"2024-09-21T06:50:44.192938+0000","last_became_peered":"2024-09-21T06:50:44.192938+0000","last_unstale":"2024-09-21T06:54:02.845800+0000","last_undegraded":"2024-09-21T06:54:02.845800+0000","last_fullsized":"2024-09-21T06:54:02.845800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:41.102162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1431,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701262+0000","last_change":"2024-09-21T06:50:56.566786+0000","last_active":"2024-09-21T06:52:31.701262+0000","last_peered":"2024-09-21T06:52:31.701262+0000","last_clean":"2024-09-21T06:52:31.701262+0000","last_became_active":"2024-09-21T06:50:56.566270+0000","last_became_peered":"2024-09-21T06:50:56.566270+0000","last_unstale":"2024-09-21T06:52:31.701262+0000","last_undegraded":"2024-09-21T06:52:31.701262+0000","last_fullsized":"2024-09-21T06:52:31.701262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.314593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749255+0000","last_change":"2024-09-21T06:50:47.219008+0000","last_active":"2024-09-21T06:52:59.749255+0000","last_peered":"2024-09-21T06:52:59.749255+0000","last_clean":"2024-09-21T06:52:59.749255+0000","last_became_active":"2024-09-21T06:50:47.218659+0000","last_became_peered":"2024-09-21T06:50:47.218659+0000","last_unstale":"2024-09-21T06:52:59.749255+0000","last_undegraded":"2024-09-21T06:52:59.749255+0000","last_fullsized":"2024-09-21T06:52:59.749255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:02.702400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":191,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:00.842041+0000","last_change":"2024-09-21T06:50:44.686495+0000","last_active":"2024-09-21T06:54:00.842041+0000","last_peered":"2024-09-21T06:54:00.842041+0000","last_clean":"2024-09-21T06:54:00.842041+0000","last_became_active":"2024-09-21T06:50:44.686004+0000","last_became_peered":"2024-09-21T06:50:44.686004+0000","last_unstale":"2024-09-21T06:54:00.842041+0000","last_undegraded":"2024-09-21T06:54:00.842041+0000","last_fullsized":"2024-09-21T06:54:00.842041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:59.702327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701440+0000","last_change":"2024-09-21T06:50:56.573028+0000","last_active":"2024-09-21T06:52:31.701440+0000","last_peered":"2024-09-21T06:52:31.701440+0000","last_clean":"2024-09-21T06:52:31.701440+0000","last_became_active":"2024-09-21T06:50:56.571714+0000","last_became_peered":"2024-09-21T06:50:56.571714+0000","last_unstale":"2024-09-21T06:52:31.701440+0000","last_undegraded":"2024-09-21T06:52:31.701440+0000","last_fullsized":"2024-09-21T06:52:31.701440+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:40.060982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740983+0000","last_change":"2024-09-21T06:50:47.217682+0000","last_active":"2024-09-21T06:52:59.740983+0000","last_peered":"2024-09-21T06:52:59.740983+0000","last_clean":"2024-09-21T06:52:59.740983+0000","last_became_active":"2024-09-21T06:50:47.215453+0000","last_became_peered":"2024-09-21T06:50:47.215453+0000","last_unstale":"2024-09-21T06:52:59.740983+0000","last_undegraded":"2024-09-21T06:52:59.740983+0000","last_fullsized":"2024-09-21T06:52:59.740983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:12.275709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648234+0000","last_change":"2024-09-21T06:50:44.202937+0000","last_active":"2024-09-21T06:52:31.648234+0000","last_peered":"2024-09-21T06:52:31.648234+0000","last_clean":"2024-09-21T06:52:31.648234+0000","last_became_active":"2024-09-21T06:50:44.202505+0000","last_became_peered":"2024-09-21T06:50:44.202505+0000","last_unstale":"2024-09-21T06:52:31.648234+0000","last_undegraded":"2024-09-21T06:52:31.648234+0000","last_fullsized":"2024-09-21T06:52:31.648234+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:43.860641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701878+0000","last_change":"2024-09-21T06:50:56.581567+0000","last_active":"2024-09-21T06:52:31.701878+0000","last_peered":"2024-09-21T06:52:31.701878+0000","last_clean":"2024-09-21T06:52:31.701878+0000","last_became_active":"2024-09-21T06:50:56.581241+0000","last_became_peered":"2024-09-21T06:50:56.581241+0000","last_unstale":"2024-09-21T06:52:31.701878+0000","last_undegraded":"2024-09-21T06:52:31.701878+0000","last_fullsized":"2024-09-21T06:52:31.701878+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:34.467612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749354+0000","last_change":"2024-09-21T06:50:47.221600+0000","last_active":"2024-09-21T06:52:59.749354+0000","last_peered":"2024-09-21T06:52:59.749354+0000","last_clean":"2024-09-21T06:52:59.749354+0000","last_became_active":"2024-09-21T06:50:47.221184+0000","last_became_peered":"2024-09-21T06:50:47.221184+0000","last_unstale":"2024-09-21T06:52:59.749354+0000","last_undegraded":"2024-09-21T06:52:59.749354+0000","last_fullsized":"2024-09-21T06:52:59.749354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:06.969424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699872+0000","last_change":"2024-09-21T06:50:44.192451+0000","last_active":"2024-09-21T06:52:31.699872+0000","last_peered":"2024-09-21T06:52:31.699872+0000","last_clean":"2024-09-21T06:52:31.699872+0000","last_became_active":"2024-09-21T06:50:44.192010+0000","last_became_peered":"2024-09-21T06:50:44.192010+0000","last_unstale":"2024-09-21T06:52:31.699872+0000","last_undegraded":"2024-09-21T06:52:31.699872+0000","last_fullsized":"2024-09-21T06:52:31.699872+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:44.425080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648487+0000","last_change":"2024-09-21T06:49:26.486211+0000","last_active":"2024-09-21T06:52:31.648487+0000","last_peered":"2024-09-21T06:52:31.648487+0000","last_clean":"2024-09-21T06:52:31.648487+0000","last_became_active":"2024-09-21T06:49:26.480563+0000","last_became_peered":"2024-09-21T06:49:26.480563+0000","last_unstale":"2024-09-21T06:52:31.648487+0000","last_undegraded":"2024-09-21T06:52:31.648487+0000","last_fullsized":"2024-09-21T06:52:31.648487+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:45:38.191185+0000","last_clean_scrub_stamp":"2024-09-21T06:45:38.191185+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:48.855630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648721+0000","last_change":"2024-09-21T06:50:56.577439+0000","last_active":"2024-09-21T06:52:31.648721+0000","last_peered":"2024-09-21T06:52:31.648721+0000","last_clean":"2024-09-21T06:52:31.648721+0000","last_became_active":"2024-09-21T06:50:56.577066+0000","last_became_peered":"2024-09-21T06:50:56.577066+0000","last_unstale":"2024-09-21T06:52:31.648721+0000","last_undegraded":"2024-09-21T06:52:31.648721+0000","last_fullsized":"2024-09-21T06:52:31.648721+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:06.099346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753632+0000","last_change":"2024-09-21T06:50:47.229504+0000","last_active":"2024-09-21T06:52:59.753632+0000","last_peered":"2024-09-21T06:52:59.753632+0000","last_clean":"2024-09-21T06:52:59.753632+0000","last_became_active":"2024-09-21T06:50:47.229079+0000","last_became_peered":"2024-09-21T06:50:47.229079+0000","last_unstale":"2024-09-21T06:52:59.753632+0000","last_undegraded":"2024-09-21T06:52:59.753632+0000","last_fullsized":"2024-09-21T06:52:59.753632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.101640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'7","reported_seq":198,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:05.851175+0000","last_change":"2024-09-21T06:50:44.941825+0000","last_active":"2024-09-21T06:54:05.851175+0000","last_peered":"2024-09-21T06:54:05.851175+0000","last_clean":"2024-09-21T06:54:05.851175+0000","last_became_active":"2024-09-21T06:50:44.941301+0000","last_became_peered":"2024-09-21T06:50:44.941301+0000","last_unstale":"2024-09-21T06:54:05.851175+0000","last_undegraded":"2024-09-21T06:54:05.851175+0000","last_fullsized":"2024-09-21T06:54:05.851175+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:15.688467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.836829+0000","last_change":"2024-09-21T06:51:30.754203+0000","last_active":"2024-09-21T06:52:31.836829+0000","last_peered":"2024-09-21T06:52:31.836829+0000","last_clean":"2024-09-21T06:52:31.836829+0000","last_became_active":"2024-09-21T06:51:30.753611+0000","last_became_peered":"2024-09-21T06:51:30.753611+0000","last_unstale":"2024-09-21T06:52:31.836829+0000","last_undegraded":"2024-09-21T06:52:31.836829+0000","last_fullsized":"2024-09-21T06:52:31.836829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:04.292920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6,7],"acting":[1,8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.744841+0000","last_change":"2024-09-21T06:50:47.233154+0000","last_active":"2024-09-21T06:52:59.744841+0000","last_peered":"2024-09-21T06:52:59.744841+0000","last_clean":"2024-09-21T06:52:59.744841+0000","last_became_active":"2024-09-21T06:50:47.232582+0000","last_became_peered":"2024-09-21T06:50:47.232582+0000","last_unstale":"2024-09-21T06:52:59.744841+0000","last_undegraded":"2024-09-21T06:52:59.744841+0000","last_fullsized":"2024-09-21T06:52:59.744841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:28.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'62","reported_seq":137,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:01.723508+0000","last_change":"2024-09-21T06:50:44.200049+0000","last_active":"2024-09-21T06:54:01.723508+0000","last_peered":"2024-09-21T06:54:01.723508+0000","last_clean":"2024-09-21T06:54:01.723508+0000","last_became_active":"2024-09-21T06:50:44.199390+0000","last_became_peered":"2024-09-21T06:50:44.199390+0000","last_unstale":"2024-09-21T06:54:01.723508+0000","last_undegraded":"2024-09-21T06:54:01.723508+0000","last_fullsized":"2024-09-21T06:54:01.723508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:46.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891861+0000","last_change":"2024-09-21T06:50:56.578250+0000","last_active":"2024-09-21T06:52:31.891861+0000","last_peered":"2024-09-21T06:52:31.891861+0000","last_clean":"2024-09-21T06:52:31.891861+0000","last_became_active":"2024-09-21T06:50:56.577783+0000","last_became_peered":"2024-09-21T06:50:56.577783+0000","last_unstale":"2024-09-21T06:52:31.891861+0000","last_undegraded":"2024-09-21T06:52:31.891861+0000","last_fullsized":"2024-09-21T06:52:31.891861+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:19.199048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753918+0000","last_change":"2024-09-21T06:50:47.219721+0000","last_active":"2024-09-21T06:52:59.753918+0000","last_peered":"2024-09-21T06:52:59.753918+0000","last_clean":"2024-09-21T06:52:59.753918+0000","last_became_active":"2024-09-21T06:50:47.219052+0000","last_became_peered":"2024-09-21T06:50:47.219052+0000","last_unstale":"2024-09-21T06:52:59.753918+0000","last_undegraded":"2024-09-21T06:52:59.753918+0000","last_fullsized":"2024-09-21T06:52:59.753918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:57.733071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701640+0000","last_change":"2024-09-21T06:50:44.190496+0000","last_active":"2024-09-21T06:52:31.701640+0000","last_peered":"2024-09-21T06:52:31.701640+0000","last_clean":"2024-09-21T06:52:31.701640+0000","last_became_active":"2024-09-21T06:50:44.190099+0000","last_became_peered":"2024-09-21T06:50:44.190099+0000","last_unstale":"2024-09-21T06:52:31.701640+0000","last_undegraded":"2024-09-21T06:52:31.701640+0000","last_fullsized":"2024-09-21T06:52:31.701640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:41.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701328+0000","last_change":"2024-09-21T06:50:56.566356+0000","last_active":"2024-09-21T06:52:31.701328+0000","last_peered":"2024-09-21T06:52:31.701328+0000","last_clean":"2024-09-21T06:52:31.701328+0000","last_became_active":"2024-09-21T06:50:56.565900+0000","last_became_peered":"2024-09-21T06:50:56.565900+0000","last_unstale":"2024-09-21T06:52:31.701328+0000","last_undegraded":"2024-09-21T06:52:31.701328+0000","last_fullsized":"2024-09-21T06:52:31.701328+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:31.474793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734233+0000","last_change":"2024-09-21T06:50:47.210334+0000","last_active":"2024-09-21T06:52:59.734233+0000","last_peered":"2024-09-21T06:52:59.734233+0000","last_clean":"2024-09-21T06:52:59.734233+0000","last_became_active":"2024-09-21T06:50:47.209930+0000","last_became_peered":"2024-09-21T06:50:47.209930+0000","last_unstale":"2024-09-21T06:52:59.734233+0000","last_undegraded":"2024-09-21T06:52:59.734233+0000","last_fullsized":"2024-09-21T06:52:59.734233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:57.590592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645941+0000","last_change":"2024-09-21T06:51:30.676883+0000","last_active":"2024-09-21T06:52:31.645941+0000","last_peered":"2024-09-21T06:52:31.645941+0000","last_clean":"2024-09-21T06:52:31.645941+0000","last_became_active":"2024-09-21T06:51:30.672351+0000","last_became_peered":"2024-09-21T06:51:30.672351+0000","last_unstale":"2024-09-21T06:52:31.645941+0000","last_undegraded":"2024-09-21T06:52:31.645941+0000","last_fullsized":"2024-09-21T06:52:31.645941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.459255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701203+0000","last_change":"2024-09-21T06:50:56.574619+0000","last_active":"2024-09-21T06:52:31.701203+0000","last_peered":"2024-09-21T06:52:31.701203+0000","last_clean":"2024-09-21T06:52:31.701203+0000","last_became_active":"2024-09-21T06:50:56.574199+0000","last_became_peered":"2024-09-21T06:50:56.574199+0000","last_unstale":"2024-09-21T06:52:31.701203+0000","last_undegraded":"2024-09-21T06:52:31.701203+0000","last_fullsized":"2024-09-21T06:52:31.701203+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:32.984348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.748492+0000","last_change":"2024-09-21T06:52:31.838201+0000","last_active":"2024-09-21T06:52:59.748492+0000","last_peered":"2024-09-21T06:52:59.748492+0000","last_clean":"2024-09-21T06:52:59.748492+0000","last_became_active":"2024-09-21T06:52:31.837829+0000","last_became_peered":"2024-09-21T06:52:31.837829+0000","last_unstale":"2024-09-21T06:52:59.748492+0000","last_undegraded":"2024-09-21T06:52:59.748492+0000","last_fullsized":"2024-09-21T06:52:59.748492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:40.499193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891566+0000","last_change":"2024-09-21T06:50:44.685625+0000","last_active":"2024-09-21T06:52:31.891566+0000","last_peered":"2024-09-21T06:52:31.891566+0000","last_clean":"2024-09-21T06:52:31.891566+0000","last_became_active":"2024-09-21T06:50:44.685184+0000","last_became_peered":"2024-09-21T06:50:44.685184+0000","last_unstale":"2024-09-21T06:52:31.891566+0000","last_undegraded":"2024-09-21T06:52:31.891566+0000","last_fullsized":"2024-09-21T06:52:31.891566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:15.439341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701524+0000","last_change":"2024-09-21T06:50:56.579733+0000","last_active":"2024-09-21T06:52:31.701524+0000","last_peered":"2024-09-21T06:52:31.701524+0000","last_clean":"2024-09-21T06:52:31.701524+0000","last_became_active":"2024-09-21T06:50:56.579269+0000","last_became_peered":"2024-09-21T06:50:56.579269+0000","last_unstale":"2024-09-21T06:52:31.701524+0000","last_undegraded":"2024-09-21T06:52:31.701524+0000","last_fullsized":"2024-09-21T06:52:31.701524+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:07.900251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756438+0000","last_change":"2024-09-21T06:52:31.649738+0000","last_active":"2024-09-21T06:52:59.756438+0000","last_peered":"2024-09-21T06:52:59.756438+0000","last_clean":"2024-09-21T06:52:59.756438+0000","last_became_active":"2024-09-21T06:52:31.649321+0000","last_became_peered":"2024-09-21T06:52:31.649321+0000","last_unstale":"2024-09-21T06:52:59.756438+0000","last_undegraded":"2024-09-21T06:52:59.756438+0000","last_fullsized":"2024-09-21T06:52:59.756438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:43.249870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:04.849294+0000","last_change":"2024-09-21T06:50:44.192086+0000","last_active":"2024-09-21T06:54:04.849294+0000","last_peered":"2024-09-21T06:54:04.849294+0000","last_clean":"2024-09-21T06:54:04.849294+0000","last_became_active":"2024-09-21T06:50:44.191576+0000","last_became_peered":"2024-09-21T06:50:44.191576+0000","last_unstale":"2024-09-21T06:54:04.849294+0000","last_undegraded":"2024-09-21T06:54:04.849294+0000","last_fullsized":"2024-09-21T06:54:04.849294+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:44.264027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.891979+0000","last_change":"2024-09-21T06:51:30.670004+0000","last_active":"2024-09-21T06:52:31.891979+0000","last_peered":"2024-09-21T06:52:31.891979+0000","last_clean":"2024-09-21T06:52:31.891979+0000","last_became_active":"2024-09-21T06:51:30.669502+0000","last_became_peered":"2024-09-21T06:51:30.669502+0000","last_unstale":"2024-09-21T06:52:31.891979+0000","last_undegraded":"2024-09-21T06:52:31.891979+0000","last_fullsized":"2024-09-21T06:52:31.891979+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:47.283053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758614+0000","last_change":"2024-09-21T06:50:47.223507+0000","last_active":"2024-09-21T06:52:59.758614+0000","last_peered":"2024-09-21T06:52:59.758614+0000","last_clean":"2024-09-21T06:52:59.758614+0000","last_became_active":"2024-09-21T06:50:47.221584+0000","last_became_peered":"2024-09-21T06:50:47.221584+0000","last_unstale":"2024-09-21T06:52:59.758614+0000","last_undegraded":"2024-09-21T06:52:59.758614+0000","last_fullsized":"2024-09-21T06:52:59.758614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:18.711392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701396+0000","last_change":"2024-09-21T06:50:44.202806+0000","last_active":"2024-09-21T06:52:31.701396+0000","last_peered":"2024-09-21T06:52:31.701396+0000","last_clean":"2024-09-21T06:52:31.701396+0000","last_became_active":"2024-09-21T06:50:44.201979+0000","last_became_peered":"2024-09-21T06:50:44.201979+0000","last_unstale":"2024-09-21T06:52:31.701396+0000","last_undegraded":"2024-09-21T06:52:31.701396+0000","last_fullsized":"2024-09-21T06:52:31.701396+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:12.556346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645499+0000","last_change":"2024-09-21T06:50:56.577607+0000","last_active":"2024-09-21T06:52:31.645499+0000","last_peered":"2024-09-21T06:52:31.645499+0000","last_clean":"2024-09-21T06:52:31.645499+0000","last_became_active":"2024-09-21T06:50:56.577188+0000","last_became_peered":"2024-09-21T06:50:56.577188+0000","last_unstale":"2024-09-21T06:52:31.645499+0000","last_undegraded":"2024-09-21T06:52:31.645499+0000","last_fullsized":"2024-09-21T06:52:31.645499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:26.746168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749562+0000","last_change":"2024-09-21T06:50:47.223006+0000","last_active":"2024-09-21T06:52:59.749562+0000","last_peered":"2024-09-21T06:52:59.749562+0000","last_clean":"2024-09-21T06:52:59.749562+0000","last_became_active":"2024-09-21T06:50:47.222664+0000","last_became_peered":"2024-09-21T06:50:47.222664+0000","last_unstale":"2024-09-21T06:52:59.749562+0000","last_undegraded":"2024-09-21T06:52:59.749562+0000","last_fullsized":"2024-09-21T06:52:59.749562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.175567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703886+0000","last_change":"2024-09-21T06:50:44.192337+0000","last_active":"2024-09-21T06:52:31.703886+0000","last_peered":"2024-09-21T06:52:31.703886+0000","last_clean":"2024-09-21T06:52:31.703886+0000","last_became_active":"2024-09-21T06:50:44.191810+0000","last_became_peered":"2024-09-21T06:50:44.191810+0000","last_unstale":"2024-09-21T06:52:31.703886+0000","last_undegraded":"2024-09-21T06:52:31.703886+0000","last_fullsized":"2024-09-21T06:52:31.703886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:10.787856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702060+0000","last_change":"2024-09-21T06:50:56.569827+0000","last_active":"2024-09-21T06:52:31.702060+0000","last_peered":"2024-09-21T06:52:31.702060+0000","last_clean":"2024-09-21T06:52:31.702060+0000","last_became_active":"2024-09-21T06:50:56.569402+0000","last_became_peered":"2024-09-21T06:50:56.569402+0000","last_unstale":"2024-09-21T06:52:31.702060+0000","last_undegraded":"2024-09-21T06:52:31.702060+0000","last_fullsized":"2024-09-21T06:52:31.702060+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:40.473951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745772+0000","last_change":"2024-09-21T06:50:47.220222+0000","last_active":"2024-09-21T06:52:59.745772+0000","last_peered":"2024-09-21T06:52:59.745772+0000","last_clean":"2024-09-21T06:52:59.745772+0000","last_became_active":"2024-09-21T06:50:47.219836+0000","last_became_peered":"2024-09-21T06:50:47.219836+0000","last_unstale":"2024-09-21T06:52:59.745772+0000","last_undegraded":"2024-09-21T06:52:59.745772+0000","last_fullsized":"2024-09-21T06:52:59.745772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:08.384627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":381,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:03.847696+0000","last_change":"2024-09-21T06:50:44.193272+0000","last_active":"2024-09-21T06:54:03.847696+0000","last_peered":"2024-09-21T06:54:03.847696+0000","last_clean":"2024-09-21T06:54:03.847696+0000","last_became_active":"2024-09-21T06:50:44.192835+0000","last_became_peered":"2024-09-21T06:50:44.192835+0000","last_unstale":"2024-09-21T06:54:03.847696+0000","last_undegraded":"2024-09-21T06:54:03.847696+0000","last_fullsized":"2024-09-21T06:54:03.847696+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:28.053537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73114,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":296,"num_read_kb":222,"num_write":44,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645757+0000","last_change":"2024-09-21T06:51:30.671254+0000","last_active":"2024-09-21T06:52:31.645757+0000","last_peered":"2024-09-21T06:52:31.645757+0000","last_clean":"2024-09-21T06:52:31.645757+0000","last_became_active":"2024-09-21T06:51:30.670805+0000","last_became_peered":"2024-09-21T06:51:30.670805+0000","last_unstale":"2024-09-21T06:52:31.645757+0000","last_undegraded":"2024-09-21T06:52:31.645757+0000","last_fullsized":"2024-09-21T06:52:31.645757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:54.339804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.747527+0000","last_change":"2024-09-21T06:50:47.221829+0000","last_active":"2024-09-21T06:52:59.747527+0000","last_peered":"2024-09-21T06:52:59.747527+0000","last_clean":"2024-09-21T06:52:59.747527+0000","last_became_active":"2024-09-21T06:50:47.221202+0000","last_became_peered":"2024-09-21T06:50:47.221202+0000","last_unstale":"2024-09-21T06:52:59.747527+0000","last_undegraded":"2024-09-21T06:52:59.747527+0000","last_fullsized":"2024-09-21T06:52:59.747527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:34.268137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.702309+0000","last_change":"2024-09-21T06:50:44.689387+0000","last_active":"2024-09-21T06:52:31.702309+0000","last_peered":"2024-09-21T06:52:31.702309+0000","last_clean":"2024-09-21T06:52:31.702309+0000","last_became_active":"2024-09-21T06:50:44.688895+0000","last_became_peered":"2024-09-21T06:50:44.688895+0000","last_unstale":"2024-09-21T06:52:31.702309+0000","last_undegraded":"2024-09-21T06:52:31.702309+0000","last_fullsized":"2024-09-21T06:52:31.702309+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:15.033743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.699508+0000","last_change":"2024-09-21T06:50:56.581833+0000","last_active":"2024-09-21T06:52:31.699508+0000","last_peered":"2024-09-21T06:52:31.699508+0000","last_clean":"2024-09-21T06:52:31.699508+0000","last_became_active":"2024-09-21T06:50:56.581305+0000","last_became_peered":"2024-09-21T06:50:56.581305+0000","last_unstale":"2024-09-21T06:52:31.699508+0000","last_undegraded":"2024-09-21T06:52:31.699508+0000","last_fullsized":"2024-09-21T06:52:31.699508+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:31.693487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757106+0000","last_change":"2024-09-21T06:50:47.223769+0000","last_active":"2024-09-21T06:52:59.757106+0000","last_peered":"2024-09-21T06:52:59.757106+0000","last_clean":"2024-09-21T06:52:59.757106+0000","last_became_active":"2024-09-21T06:50:47.222804+0000","last_became_peered":"2024-09-21T06:50:47.222804+0000","last_unstale":"2024-09-21T06:52:59.757106+0000","last_undegraded":"2024-09-21T06:52:59.757106+0000","last_fullsized":"2024-09-21T06:52:59.757106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:47.408222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.648918+0000","last_change":"2024-09-21T06:50:44.191650+0000","last_active":"2024-09-21T06:52:31.648918+0000","last_peered":"2024-09-21T06:52:31.648918+0000","last_clean":"2024-09-21T06:52:31.648918+0000","last_became_active":"2024-09-21T06:50:44.191242+0000","last_became_peered":"2024-09-21T06:50:44.191242+0000","last_unstale":"2024-09-21T06:52:31.648918+0000","last_undegraded":"2024-09-21T06:52:31.648918+0000","last_fullsized":"2024-09-21T06:52:31.648918+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:31.724512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645559+0000","last_change":"2024-09-21T06:50:56.586705+0000","last_active":"2024-09-21T06:52:31.645559+0000","last_peered":"2024-09-21T06:52:31.645559+0000","last_clean":"2024-09-21T06:52:31.645559+0000","last_became_active":"2024-09-21T06:50:56.580568+0000","last_became_peered":"2024-09-21T06:50:56.580568+0000","last_unstale":"2024-09-21T06:52:31.645559+0000","last_undegraded":"2024-09-21T06:52:31.645559+0000","last_fullsized":"2024-09-21T06:52:31.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:07.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750255+0000","last_change":"2024-09-21T06:50:47.224203+0000","last_active":"2024-09-21T06:52:59.750255+0000","last_peered":"2024-09-21T06:52:59.750255+0000","last_clean":"2024-09-21T06:52:59.750255+0000","last_became_active":"2024-09-21T06:50:47.223854+0000","last_became_peered":"2024-09-21T06:50:47.223854+0000","last_unstale":"2024-09-21T06:52:59.750255+0000","last_undegraded":"2024-09-21T06:52:59.750255+0000","last_fullsized":"2024-09-21T06:52:59.750255+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:55.388128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'6","reported_seq":331,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:03.847571+0000","last_change":"2024-09-21T06:50:44.203068+0000","last_active":"2024-09-21T06:54:03.847571+0000","last_peered":"2024-09-21T06:54:03.847571+0000","last_clean":"2024-09-21T06:54:03.847571+0000","last_became_active":"2024-09-21T06:50:44.202655+0000","last_became_peered":"2024-09-21T06:50:44.202655+0000","last_unstale":"2024-09-21T06:54:03.847571+0000","last_undegraded":"2024-09-21T06:54:03.847571+0000","last_fullsized":"2024-09-21T06:54:03.847571+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:54.726101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892718+0000","last_change":"2024-09-21T06:50:56.578646+0000","last_active":"2024-09-21T06:52:31.892718+0000","last_peered":"2024-09-21T06:52:31.892718+0000","last_clean":"2024-09-21T06:52:31.892718+0000","last_became_active":"2024-09-21T06:50:56.576638+0000","last_became_peered":"2024-09-21T06:50:56.576638+0000","last_unstale":"2024-09-21T06:52:31.892718+0000","last_undegraded":"2024-09-21T06:52:31.892718+0000","last_fullsized":"2024-09-21T06:52:31.892718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:45.339085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.892186+0000","last_change":"2024-09-21T06:50:56.570573+0000","last_active":"2024-09-21T06:52:31.892186+0000","last_peered":"2024-09-21T06:52:31.892186+0000","last_clean":"2024-09-21T06:52:31.892186+0000","last_became_active":"2024-09-21T06:50:56.570141+0000","last_became_peered":"2024-09-21T06:50:56.570141+0000","last_unstale":"2024-09-21T06:52:31.892186+0000","last_undegraded":"2024-09-21T06:52:31.892186+0000","last_fullsized":"2024-09-21T06:52:31.892186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:19.586909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.741381+0000","last_change":"2024-09-21T06:50:47.213509+0000","last_active":"2024-09-21T06:52:59.741381+0000","last_peered":"2024-09-21T06:52:59.741381+0000","last_clean":"2024-09-21T06:52:59.741381+0000","last_became_active":"2024-09-21T06:50:47.213149+0000","last_became_peered":"2024-09-21T06:50:47.213149+0000","last_unstale":"2024-09-21T06:52:59.741381+0000","last_undegraded":"2024-09-21T06:52:59.741381+0000","last_fullsized":"2024-09-21T06:52:59.741381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:49.974646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.701609+0000","last_change":"2024-09-21T06:50:44.201991+0000","last_active":"2024-09-21T06:52:31.701609+0000","last_peered":"2024-09-21T06:52:31.701609+0000","last_clean":"2024-09-21T06:52:31.701609+0000","last_became_active":"2024-09-21T06:50:44.201290+0000","last_became_peered":"2024-09-21T06:50:44.201290+0000","last_unstale":"2024-09-21T06:52:31.701609+0000","last_undegraded":"2024-09-21T06:52:31.701609+0000","last_fullsized":"2024-09-21T06:52:31.701609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:43.861873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645808+0000","last_change":"2024-09-21T06:50:56.581825+0000","last_active":"2024-09-21T06:52:31.645808+0000","last_peered":"2024-09-21T06:52:31.645808+0000","last_clean":"2024-09-21T06:52:31.645808+0000","last_became_active":"2024-09-21T06:50:56.581300+0000","last_became_peered":"2024-09-21T06:50:56.581300+0000","last_unstale":"2024-09-21T06:52:31.645808+0000","last_undegraded":"2024-09-21T06:52:31.645808+0000","last_fullsized":"2024-09-21T06:52:31.645808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:15.767114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742667+0000","last_change":"2024-09-21T06:50:47.214234+0000","last_active":"2024-09-21T06:52:59.742667+0000","last_peered":"2024-09-21T06:52:59.742667+0000","last_clean":"2024-09-21T06:52:59.742667+0000","last_became_active":"2024-09-21T06:50:47.213278+0000","last_became_peered":"2024-09-21T06:50:47.213278+0000","last_unstale":"2024-09-21T06:52:59.742667+0000","last_undegraded":"2024-09-21T06:52:59.742667+0000","last_fullsized":"2024-09-21T06:52:59.742667+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:11.754131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.700994+0000","last_change":"2024-09-21T06:50:44.193291+0000","last_active":"2024-09-21T06:52:31.700994+0000","last_peered":"2024-09-21T06:52:31.700994+0000","last_clean":"2024-09-21T06:52:31.700994+0000","last_became_active":"2024-09-21T06:50:44.192581+0000","last_became_peered":"2024-09-21T06:50:44.192581+0000","last_unstale":"2024-09-21T06:52:31.700994+0000","last_undegraded":"2024-09-21T06:52:31.700994+0000","last_fullsized":"2024-09-21T06:52:31.700994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:28.684879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.645981+0000","last_change":"2024-09-21T06:50:44.199875+0000","last_active":"2024-09-21T06:52:31.645981+0000","last_peered":"2024-09-21T06:52:31.645981+0000","last_clean":"2024-09-21T06:52:31.645981+0000","last_became_active":"2024-09-21T06:50:44.199430+0000","last_became_peered":"2024-09-21T06:50:44.199430+0000","last_unstale":"2024-09-21T06:52:31.645981+0000","last_undegraded":"2024-09-21T06:52:31.645981+0000","last_fullsized":"2024-09-21T06:52:31.645981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:25.686273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749130+0000","last_change":"2024-09-21T06:50:47.232957+0000","last_active":"2024-09-21T06:52:59.749130+0000","last_peered":"2024-09-21T06:52:59.749130+0000","last_clean":"2024-09-21T06:52:59.749130+0000","last_became_active":"2024-09-21T06:50:47.232529+0000","last_became_peered":"2024-09-21T06:50:47.232529+0000","last_unstale":"2024-09-21T06:52:59.749130+0000","last_undegraded":"2024-09-21T06:52:59.749130+0000","last_fullsized":"2024-09-21T06:52:59.749130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.512331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.703865+0000","last_change":"2024-09-21T06:50:56.576339+0000","last_active":"2024-09-21T06:52:31.703865+0000","last_peered":"2024-09-21T06:52:31.703865+0000","last_clean":"2024-09-21T06:52:31.703865+0000","last_became_active":"2024-09-21T06:50:56.575917+0000","last_became_peered":"2024-09-21T06:50:56.575917+0000","last_unstale":"2024-09-21T06:52:31.703865+0000","last_undegraded":"2024-09-21T06:52:31.703865+0000","last_fullsized":"2024-09-21T06:52:31.703865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.198745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":206,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:54:04.850014+0000","last_change":"2024-09-21T06:51:30.672629+0000","last_active":"2024-09-21T06:54:04.850014+0000","last_peered":"2024-09-21T06:54:04.850014+0000","last_clean":"2024-09-21T06:54:04.850014+0000","last_became_active":"2024-09-21T06:51:30.672051+0000","last_became_peered":"2024-09-21T06:51:30.672051+0000","last_unstale":"2024-09-21T06:54:04.850014+0000","last_undegraded":"2024-09-21T06:54:04.850014+0000","last_fullsized":"2024-09-21T06:54:04.850014+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:43.163249+0000","last_clean_scrub_stamp":"2024-09-21T06:50:43.163249+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:44.080056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":147,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.736418+0000","last_change":"2024-09-21T06:50:47.217972+0000","last_active":"2024-09-21T06:52:59.736418+0000","last_peered":"2024-09-21T06:52:59.736418+0000","last_clean":"2024-09-21T06:52:59.736418+0000","last_became_active":"2024-09-21T06:50:47.217590+0000","last_became_peered":"2024-09-21T06:50:47.217590+0000","last_unstale":"2024-09-21T06:52:59.736418+0000","last_undegraded":"2024-09-21T06:52:59.736418+0000","last_fullsized":"2024-09-21T06:52:59.736418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:27:54.272521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:31.704051+0000","last_change":"2024-09-21T06:50:56.566755+0000","last_active":"2024-09-21T06:52:31.704051+0000","last_peered":"2024-09-21T06:52:31.704051+0000","last_clean":"2024-09-21T06:52:31.704051+0000","last_became_active":"2024-09-21T06:50:56.566031+0000","last_became_peered":"2024-09-21T06:50:56.566031+0000","last_unstale":"2024-09-21T06:52:31.704051+0000","last_undegraded":"2024-09-21T06:52:31.704051+0000","last_fullsized":"2024-09-21T06:52:31.704051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:55.541496+0000","last_clean_scrub_stamp":"2024-09-21T06:50:55.541496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:06.517105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.739975+0000","last_change":"2024-09-21T06:50:47.223596+0000","last_active":"2024-09-21T06:52:59.739975+0000","last_peered":"2024-09-21T06:52:59.739975+0000","last_clean":"2024-09-21T06:52:59.739975+0000","last_became_active":"2024-09-21T06:50:47.223209+0000","last_became_peered":"2024-09-21T06:50:47.223209+0000","last_unstale":"2024-09-21T06:52:59.739975+0000","last_undegraded":"2024-09-21T06:52:59.739975+0000","last_fullsized":"2024-09-21T06:52:59.739975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:18:29.339648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749347+0000","last_change":"2024-09-21T06:50:47.219914+0000","last_active":"2024-09-21T06:52:59.749347+0000","last_peered":"2024-09-21T06:52:59.749347+0000","last_clean":"2024-09-21T06:52:59.749347+0000","last_became_active":"2024-09-21T06:50:47.218970+0000","last_became_peered":"2024-09-21T06:50:47.218970+0000","last_unstale":"2024-09-21T06:52:59.749347+0000","last_undegraded":"2024-09-21T06:52:59.749347+0000","last_fullsized":"2024-09-21T06:52:59.749347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:05.646986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746716+0000","last_change":"2024-09-21T06:50:47.230096+0000","last_active":"2024-09-21T06:52:59.746716+0000","last_peered":"2024-09-21T06:52:59.746716+0000","last_clean":"2024-09-21T06:52:59.746716+0000","last_became_active":"2024-09-21T06:50:47.229604+0000","last_became_peered":"2024-09-21T06:50:47.229604+0000","last_unstale":"2024-09-21T06:52:59.746716+0000","last_undegraded":"2024-09-21T06:52:59.746716+0000","last_fullsized":"2024-09-21T06:52:59.746716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:09.140303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757121+0000","last_change":"2024-09-21T06:50:47.222727+0000","last_active":"2024-09-21T06:52:59.757121+0000","last_peered":"2024-09-21T06:52:59.757121+0000","last_clean":"2024-09-21T06:52:59.757121+0000","last_became_active":"2024-09-21T06:50:47.222363+0000","last_became_peered":"2024-09-21T06:50:47.222363+0000","last_unstale":"2024-09-21T06:52:59.757121+0000","last_undegraded":"2024-09-21T06:52:59.757121+0000","last_fullsized":"2024-09-21T06:52:59.757121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:24.114714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756339+0000","last_change":"2024-09-21T06:50:47.222340+0000","last_active":"2024-09-21T06:52:59.756339+0000","last_peered":"2024-09-21T06:52:59.756339+0000","last_clean":"2024-09-21T06:52:59.756339+0000","last_became_active":"2024-09-21T06:50:47.221023+0000","last_became_peered":"2024-09-21T06:50:47.221023+0000","last_unstale":"2024-09-21T06:52:59.756339+0000","last_undegraded":"2024-09-21T06:52:59.756339+0000","last_fullsized":"2024-09-21T06:52:59.756339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:40.560942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734085+0000","last_change":"2024-09-21T06:50:47.214185+0000","last_active":"2024-09-21T06:52:59.734085+0000","last_peered":"2024-09-21T06:52:59.734085+0000","last_clean":"2024-09-21T06:52:59.734085+0000","last_became_active":"2024-09-21T06:50:47.213793+0000","last_became_peered":"2024-09-21T06:50:47.213793+0000","last_unstale":"2024-09-21T06:52:59.734085+0000","last_undegraded":"2024-09-21T06:52:59.734085+0000","last_fullsized":"2024-09-21T06:52:59.734085+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:05:22.372881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749595+0000","last_change":"2024-09-21T06:50:47.220952+0000","last_active":"2024-09-21T06:52:59.749595+0000","last_peered":"2024-09-21T06:52:59.749595+0000","last_clean":"2024-09-21T06:52:59.749595+0000","last_became_active":"2024-09-21T06:50:47.219100+0000","last_became_peered":"2024-09-21T06:50:47.219100+0000","last_unstale":"2024-09-21T06:52:59.749595+0000","last_undegraded":"2024-09-21T06:52:59.749595+0000","last_fullsized":"2024-09-21T06:52:59.749595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:23:46.617112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.737176+0000","last_change":"2024-09-21T06:50:47.231750+0000","last_active":"2024-09-21T06:52:59.737176+0000","last_peered":"2024-09-21T06:52:59.737176+0000","last_clean":"2024-09-21T06:52:59.737176+0000","last_became_active":"2024-09-21T06:50:47.231413+0000","last_became_peered":"2024-09-21T06:50:47.231413+0000","last_unstale":"2024-09-21T06:52:59.737176+0000","last_undegraded":"2024-09-21T06:52:59.737176+0000","last_fullsized":"2024-09-21T06:52:59.737176+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:28.999496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746235+0000","last_change":"2024-09-21T06:50:47.232990+0000","last_active":"2024-09-21T06:52:59.746235+0000","last_peered":"2024-09-21T06:52:59.746235+0000","last_clean":"2024-09-21T06:52:59.746235+0000","last_became_active":"2024-09-21T06:50:47.232542+0000","last_became_peered":"2024-09-21T06:50:47.232542+0000","last_unstale":"2024-09-21T06:52:59.746235+0000","last_undegraded":"2024-09-21T06:52:59.746235+0000","last_fullsized":"2024-09-21T06:52:59.746235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:32.601276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.740649+0000","last_change":"2024-09-21T06:50:47.214680+0000","last_active":"2024-09-21T06:52:59.740649+0000","last_peered":"2024-09-21T06:52:59.740649+0000","last_clean":"2024-09-21T06:52:59.740649+0000","last_became_active":"2024-09-21T06:50:47.214369+0000","last_became_peered":"2024-09-21T06:50:47.214369+0000","last_unstale":"2024-09-21T06:52:59.740649+0000","last_undegraded":"2024-09-21T06:52:59.740649+0000","last_fullsized":"2024-09-21T06:52:59.740649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.414829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.753696+0000","last_change":"2024-09-21T06:50:47.236291+0000","last_active":"2024-09-21T06:52:59.753696+0000","last_peered":"2024-09-21T06:52:59.753696+0000","last_clean":"2024-09-21T06:52:59.753696+0000","last_became_active":"2024-09-21T06:50:47.235941+0000","last_became_peered":"2024-09-21T06:50:47.235941+0000","last_unstale":"2024-09-21T06:52:59.753696+0000","last_undegraded":"2024-09-21T06:52:59.753696+0000","last_fullsized":"2024-09-21T06:52:59.753696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:54.729289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756495+0000","last_change":"2024-09-21T06:50:47.224545+0000","last_active":"2024-09-21T06:52:59.756495+0000","last_peered":"2024-09-21T06:52:59.756495+0000","last_clean":"2024-09-21T06:52:59.756495+0000","last_became_active":"2024-09-21T06:50:47.223756+0000","last_became_peered":"2024-09-21T06:50:47.223756+0000","last_unstale":"2024-09-21T06:52:59.756495+0000","last_undegraded":"2024-09-21T06:52:59.756495+0000","last_fullsized":"2024-09-21T06:52:59.756495+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:16.627572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.734175+0000","last_change":"2024-09-21T06:50:47.213419+0000","last_active":"2024-09-21T06:52:59.734175+0000","last_peered":"2024-09-21T06:52:59.734175+0000","last_clean":"2024-09-21T06:52:59.734175+0000","last_became_active":"2024-09-21T06:50:47.212979+0000","last_became_peered":"2024-09-21T06:50:47.212979+0000","last_unstale":"2024-09-21T06:52:59.734175+0000","last_undegraded":"2024-09-21T06:52:59.734175+0000","last_fullsized":"2024-09-21T06:52:59.734175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:16.271673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.745394+0000","last_change":"2024-09-21T06:50:47.230998+0000","last_active":"2024-09-21T06:52:59.745394+0000","last_peered":"2024-09-21T06:52:59.745394+0000","last_clean":"2024-09-21T06:52:59.745394+0000","last_became_active":"2024-09-21T06:50:47.230223+0000","last_became_peered":"2024-09-21T06:50:47.230223+0000","last_unstale":"2024-09-21T06:52:59.745394+0000","last_undegraded":"2024-09-21T06:52:59.745394+0000","last_fullsized":"2024-09-21T06:52:59.745394+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.738659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.751016+0000","last_change":"2024-09-21T06:50:47.220479+0000","last_active":"2024-09-21T06:52:59.751016+0000","last_peered":"2024-09-21T06:52:59.751016+0000","last_clean":"2024-09-21T06:52:59.751016+0000","last_became_active":"2024-09-21T06:50:47.220049+0000","last_became_peered":"2024-09-21T06:50:47.220049+0000","last_unstale":"2024-09-21T06:52:59.751016+0000","last_undegraded":"2024-09-21T06:52:59.751016+0000","last_fullsized":"2024-09-21T06:52:59.751016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:46.924969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755908+0000","last_change":"2024-09-21T06:50:47.226342+0000","last_active":"2024-09-21T06:52:59.755908+0000","last_peered":"2024-09-21T06:52:59.755908+0000","last_clean":"2024-09-21T06:52:59.755908+0000","last_became_active":"2024-09-21T06:50:47.225988+0000","last_became_peered":"2024-09-21T06:50:47.225988+0000","last_unstale":"2024-09-21T06:52:59.755908+0000","last_undegraded":"2024-09-21T06:52:59.755908+0000","last_fullsized":"2024-09-21T06:52:59.755908+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:19.725448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752146+0000","last_change":"2024-09-21T06:50:47.225378+0000","last_active":"2024-09-21T06:52:59.752146+0000","last_peered":"2024-09-21T06:52:59.752146+0000","last_clean":"2024-09-21T06:52:59.752146+0000","last_became_active":"2024-09-21T06:50:47.224996+0000","last_became_peered":"2024-09-21T06:50:47.224996+0000","last_unstale":"2024-09-21T06:52:59.752146+0000","last_undegraded":"2024-09-21T06:52:59.752146+0000","last_fullsized":"2024-09-21T06:52:59.752146+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:22.170353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754481+0000","last_change":"2024-09-21T06:51:30.673147+0000","last_active":"2024-09-21T06:52:59.754481+0000","last_peered":"2024-09-21T06:52:59.754481+0000","last_clean":"2024-09-21T06:52:59.754481+0000","last_became_active":"2024-09-21T06:51:30.672703+0000","last_became_peered":"2024-09-21T06:51:30.672703+0000","last_unstale":"2024-09-21T06:52:59.754481+0000","last_undegraded":"2024-09-21T06:52:59.754481+0000","last_fullsized":"2024-09-21T06:52:59.754481+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:03.708545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.758054+0000","last_change":"2024-09-21T06:50:47.220210+0000","last_active":"2024-09-21T06:52:59.758054+0000","last_peered":"2024-09-21T06:52:59.758054+0000","last_clean":"2024-09-21T06:52:59.758054+0000","last_became_active":"2024-09-21T06:50:47.218607+0000","last_became_peered":"2024-09-21T06:50:47.218607+0000","last_unstale":"2024-09-21T06:52:59.758054+0000","last_undegraded":"2024-09-21T06:52:59.758054+0000","last_fullsized":"2024-09-21T06:52:59.758054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:18.503403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.756767+0000","last_change":"2024-09-21T06:50:47.215960+0000","last_active":"2024-09-21T06:52:59.756767+0000","last_peered":"2024-09-21T06:52:59.756767+0000","last_clean":"2024-09-21T06:52:59.756767+0000","last_became_active":"2024-09-21T06:50:47.215569+0000","last_became_peered":"2024-09-21T06:50:47.215569+0000","last_unstale":"2024-09-21T06:52:59.756767+0000","last_undegraded":"2024-09-21T06:52:59.756767+0000","last_fullsized":"2024-09-21T06:52:59.756767+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:57.096117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755238+0000","last_change":"2024-09-21T06:50:47.219208+0000","last_active":"2024-09-21T06:52:59.755238+0000","last_peered":"2024-09-21T06:52:59.755238+0000","last_clean":"2024-09-21T06:52:59.755238+0000","last_became_active":"2024-09-21T06:50:47.218255+0000","last_became_peered":"2024-09-21T06:50:47.218255+0000","last_unstale":"2024-09-21T06:52:59.755238+0000","last_undegraded":"2024-09-21T06:52:59.755238+0000","last_fullsized":"2024-09-21T06:52:59.755238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:39.115449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752053+0000","last_change":"2024-09-21T06:50:47.229988+0000","last_active":"2024-09-21T06:52:59.752053+0000","last_peered":"2024-09-21T06:52:59.752053+0000","last_clean":"2024-09-21T06:52:59.752053+0000","last_became_active":"2024-09-21T06:50:47.229585+0000","last_became_peered":"2024-09-21T06:50:47.229585+0000","last_unstale":"2024-09-21T06:52:59.752053+0000","last_undegraded":"2024-09-21T06:52:59.752053+0000","last_fullsized":"2024-09-21T06:52:59.752053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:35.341292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.759125+0000","last_change":"2024-09-21T06:50:47.220308+0000","last_active":"2024-09-21T06:52:59.759125+0000","last_peered":"2024-09-21T06:52:59.759125+0000","last_clean":"2024-09-21T06:52:59.759125+0000","last_became_active":"2024-09-21T06:50:47.218258+0000","last_became_peered":"2024-09-21T06:50:47.218258+0000","last_unstale":"2024-09-21T06:52:59.759125+0000","last_undegraded":"2024-09-21T06:52:59.759125+0000","last_fullsized":"2024-09-21T06:52:59.759125+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:38.948320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.742201+0000","last_change":"2024-09-21T06:50:47.227401+0000","last_active":"2024-09-21T06:52:59.742201+0000","last_peered":"2024-09-21T06:52:59.742201+0000","last_clean":"2024-09-21T06:52:59.742201+0000","last_became_active":"2024-09-21T06:50:47.227024+0000","last_became_peered":"2024-09-21T06:50:47.227024+0000","last_unstale":"2024-09-21T06:52:59.742201+0000","last_undegraded":"2024-09-21T06:52:59.742201+0000","last_fullsized":"2024-09-21T06:52:59.742201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:14.130292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755485+0000","last_change":"2024-09-21T06:50:47.227136+0000","last_active":"2024-09-21T06:52:59.755485+0000","last_peered":"2024-09-21T06:52:59.755485+0000","last_clean":"2024-09-21T06:52:59.755485+0000","last_became_active":"2024-09-21T06:50:47.226541+0000","last_became_peered":"2024-09-21T06:50:47.226541+0000","last_unstale":"2024-09-21T06:52:59.755485+0000","last_undegraded":"2024-09-21T06:52:59.755485+0000","last_fullsized":"2024-09-21T06:52:59.755485+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:39.062192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.752958+0000","last_change":"2024-09-21T06:50:47.226509+0000","last_active":"2024-09-21T06:52:59.752958+0000","last_peered":"2024-09-21T06:52:59.752958+0000","last_clean":"2024-09-21T06:52:59.752958+0000","last_became_active":"2024-09-21T06:50:47.225962+0000","last_became_peered":"2024-09-21T06:50:47.225962+0000","last_unstale":"2024-09-21T06:52:59.752958+0000","last_undegraded":"2024-09-21T06:52:59.752958+0000","last_fullsized":"2024-09-21T06:52:59.752958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:36.376080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.754740+0000","last_change":"2024-09-21T06:52:31.705756+0000","last_active":"2024-09-21T06:52:59.754740+0000","last_peered":"2024-09-21T06:52:59.754740+0000","last_clean":"2024-09-21T06:52:59.754740+0000","last_became_active":"2024-09-21T06:52:31.705416+0000","last_became_peered":"2024-09-21T06:52:31.705416+0000","last_unstale":"2024-09-21T06:52:59.754740+0000","last_undegraded":"2024-09-21T06:52:59.754740+0000","last_fullsized":"2024-09-21T06:52:59.754740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:27.703930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.746351+0000","last_change":"2024-09-21T06:50:47.223318+0000","last_active":"2024-09-21T06:52:59.746351+0000","last_peered":"2024-09-21T06:52:59.746351+0000","last_clean":"2024-09-21T06:52:59.746351+0000","last_became_active":"2024-09-21T06:50:47.222817+0000","last_became_peered":"2024-09-21T06:50:47.222817+0000","last_unstale":"2024-09-21T06:52:59.746351+0000","last_undegraded":"2024-09-21T06:52:59.746351+0000","last_fullsized":"2024-09-21T06:52:59.746351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:10.457352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.749389+0000","last_change":"2024-09-21T06:50:47.220563+0000","last_active":"2024-09-21T06:52:59.749389+0000","last_peered":"2024-09-21T06:52:59.749389+0000","last_clean":"2024-09-21T06:52:59.749389+0000","last_became_active":"2024-09-21T06:50:47.220156+0000","last_became_peered":"2024-09-21T06:50:47.220156+0000","last_unstale":"2024-09-21T06:52:59.749389+0000","last_undegraded":"2024-09-21T06:52:59.749389+0000","last_fullsized":"2024-09-21T06:52:59.749389+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:10.448013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.755271+0000","last_change":"2024-09-21T06:50:47.226109+0000","last_active":"2024-09-21T06:52:59.755271+0000","last_peered":"2024-09-21T06:52:59.755271+0000","last_clean":"2024-09-21T06:52:59.755271+0000","last_became_active":"2024-09-21T06:50:47.225772+0000","last_became_peered":"2024-09-21T06:50:47.225772+0000","last_unstale":"2024-09-21T06:52:59.755271+0000","last_undegraded":"2024-09-21T06:52:59.755271+0000","last_fullsized":"2024-09-21T06:52:59.755271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:12.345790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.757852+0000","last_change":"2024-09-21T06:50:47.232236+0000","last_active":"2024-09-21T06:52:59.757852+0000","last_peered":"2024-09-21T06:52:59.757852+0000","last_clean":"2024-09-21T06:52:59.757852+0000","last_became_active":"2024-09-21T06:50:47.231885+0000","last_became_peered":"2024-09-21T06:50:47.231885+0000","last_unstale":"2024-09-21T06:52:59.757852+0000","last_undegraded":"2024-09-21T06:52:59.757852+0000","last_fullsized":"2024-09-21T06:52:59.757852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:12.003935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-21T06:52:59.750926+0000","last_change":"2024-09-21T06:50:47.226653+0000","last_active":"2024-09-21T06:52:59.750926+0000","last_peered":"2024-09-21T06:52:59.750926+0000","last_clean":"2024-09-21T06:52:59.750926+0000","last_became_active":"2024-09-21T06:50:47.226286+0000","last_became_peered":"2024-09-21T06:50:47.226286+0000","last_unstale":"2024-09-21T06:52:59.750926+0000","last_undegraded":"2024-09-21T06:52:59.750926+0000","last_fullsized":"2024-09-21T06:52:59.750926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T06:50:46.187795+0000","last_clean_scrub_stamp":"2024-09-21T06:50:46.187795+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:10.443315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":868435,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1799,"num_read_kb":1119,"num_write":495,"num_write_kb":2257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2973696,"data_stored":2605305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":376,"ondisk_log_size":376,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710775,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":531799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907002,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29172,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715980,"statfs":{"total":95995035648,"available":95965163520,"internally_reserved":0,"allocated":2138112,"data_stored":1584951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070527,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1458176,"data_stored":928487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266759,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1445888,"data_stored":924747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462987,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1654784,"data_stored":1129034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626513,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":527522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822744,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29096,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716056,"statfs":{"total":95995035648,"available":95965241344,"internally_reserved":0,"allocated":2060288,"data_stored":1517549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953566,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":530753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28880,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716272,"statfs":{"total":95995035648,"available":95965462528,"internally_reserved":0,"allocated":1839104,"data_stored":1313379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378730,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1126400,"data_stored":600953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542254,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":602820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705780,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1060864,"data_stored":528667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":74996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":819200,"data_stored":787422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":397128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":468626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T06:54:09.584 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:54:09.956 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:10.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 -- 172.21.15.114:0/813975013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 msgr2=0x7fb0cc114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/813975013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc114030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009900 tx=0x7fb0b802a410 comp rx=0 tx=0).stop 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 -- 172.21.15.114:0/813975013 shutdown_connections 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/813975013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0cc114570 0x7fb0cc116990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/813975013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc114030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/813975013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 0x7fb0cc077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 -- 172.21.15.114:0/813975013 >> 172.21.15.114:0/813975013 conn(0x7fb0cc101680 msgr2=0x7fb0cc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 -- 172.21.15.114:0/813975013 shutdown_connections 2024-09-21T06:54:10.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.580+0000 7fb0cbe00640 1 -- 172.21.15.114:0/813975013 wait complete. 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 Processor -- start 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 -- start start 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 0x7fb0cc1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc1a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0cc114570 0x7fb0cc1a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc119f20 con 0x7fb0cc078290 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc119da0 con 0x7fb0cc077940 2024-09-21T06:54:10.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0cbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0cc11a0a0 con 0x7fb0cc114570 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.582+0000 7fb0ca000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc1a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 0x7fb0cc1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0ca000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc1a0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52370/0 (socket says 172.21.15.114:52370) 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0ca000640 1 -- 172.21.15.114:0/860106166 learned_addr learned my addr 172.21.15.114:0/860106166 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0cb400640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0cc114570 0x7fb0cc1a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 -- 172.21.15.114:0/860106166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0cc114570 msgr2=0x7fb0cc1a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0cc114570 0x7fb0cc1a6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 -- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 msgr2=0x7fb0cc1a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc1a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 -- 172.21.15.114:0/860106166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0cc1a6880 con 0x7fb0cc077940 2024-09-21T06:54:10.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.583+0000 7fb0caa00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 0x7fb0cc1a0910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00eb00 tx=0x7fb0bc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:10.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.584+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc010430 con 0x7fb0cc077940 2024-09-21T06:54:10.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.584+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0bc010a80 con 0x7fb0cc077940 2024-09-21T06:54:10.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.584+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0cc1a6b70 con 0x7fb0cc077940 2024-09-21T06:54:10.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.584+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc017a90 con 0x7fb0cc077940 2024-09-21T06:54:10.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.584+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0cc1ad2d0 con 0x7fb0cc077940 2024-09-21T06:54:10.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.585+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0cc077d50 con 0x7fb0cc077940 2024-09-21T06:54:10.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.587+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fb0bc017c50 con 0x7fb0cc077940 2024-09-21T06:54:10.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.589+0000 7fb0c3400640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0b00b6670 0x7fb0b00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.589+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb0bc0e8480 con 0x7fb0cc077940 2024-09-21T06:54:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.589+0000 7fb0ca000640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0b00b6670 0x7fb0b00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:10.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.590+0000 7fb0ca000640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0b00b6670 0x7fb0b00b8b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8006010 tx=0x7fb0b8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:10.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.591+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb0bc0ae750 con 0x7fb0cc077940 2024-09-21T06:54:10.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:10 smithi114 ceph-mon[30654]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:10.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:10 smithi114 ceph-mon[30654]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:10.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.817+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0cc1a1a40 con 0x7fb0cc077940 2024-09-21T06:54:10.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.818+0000 7fb0c3400640 1 -- 172.21.15.114:0/860106166 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7fb0bc0b5e20 con 0x7fb0cc077940 2024-09-21T06:54:10.820 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:10.820 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"btime":"2024-09-21T06:53:49:573848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:53:49.006951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:54:10.820 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-21T06:54:10.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0b00b6670 msgr2=0x7fb0b00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:10.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0b00b6670 0x7fb0b00b8b80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8006010 tx=0x7fb0b8005a20 comp rx=0 tx=0).stop 2024-09-21T06:54:10.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 msgr2=0x7fb0cc1a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:10.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 0x7fb0cc1a0910 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00eb00 tx=0x7fb0bc00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 shutdown_connections 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb0cc114570 0x7fb0cc1a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb0cc078290 0x7fb0cc1a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0b00b6670 0x7fb0b00b8b80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 --2- 172.21.15.114:0/860106166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0cc077940 0x7fb0cc1a0910 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 >> 172.21.15.114:0/860106166 conn(0x7fb0cc101680 msgr2=0x7fb0cc116600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 shutdown_connections 2024-09-21T06:54:10.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:10.821+0000 7fb0cbe00640 1 -- 172.21.15.114:0/860106166 wait complete. 2024-09-21T06:54:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:10 smithi183 ceph-mon[34219]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:10.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:10 smithi183 ceph-mon[34219]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:10 smithi193 ceph-mon[34601]: from='client.15594 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T06:54:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:10 smithi193 ceph-mon[34601]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:11.624 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:54:11.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:11 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/860106166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:11 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/860106166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:11.994 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:12.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:11 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/860106166' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:12.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 -- 172.21.15.114:0/3550433767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10b4f0 msgr2=0x7faecc10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:12.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 --2- 172.21.15.114:0/3550433767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10b4f0 0x7faecc10b900 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009900 tx=0x7faeb402a410 comp rx=0 tx=0).stop 2024-09-21T06:54:12.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 -- 172.21.15.114:0/3550433767 shutdown_connections 2024-09-21T06:54:12.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 --2- 172.21.15.114:0/3550433767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10be40 0x7faecc112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 --2- 172.21.15.114:0/3550433767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10b4f0 0x7faecc10b900 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 --2- 172.21.15.114:0/3550433767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faecc105bb0 0x7faecc105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 -- 172.21.15.114:0/3550433767 >> 172.21.15.114:0/3550433767 conn(0x7faecc101680 msgr2=0x7faecc103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:12.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 -- 172.21.15.114:0/3550433767 shutdown_connections 2024-09-21T06:54:12.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.641+0000 7faed1c00640 1 -- 172.21.15.114:0/3550433767 wait complete. 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.642+0000 7faed1c00640 1 Processor -- start 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 -- start start 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faecc105bb0 0x7faecc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 0x7faecc19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10be40 0x7faecc1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7faecc115b50 con 0x7faecc10be40 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1159d0 con 0x7faecc10b4f0 2024-09-21T06:54:12.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faecc115cd0 con 0x7faecc105bb0 2024-09-21T06:54:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faec3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 0x7faecc19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faed1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10be40 0x7faecc1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faec3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 0x7faecc19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33586/0 (socket says 172.21.15.114:33586) 2024-09-21T06:54:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faec3400640 1 -- 172.21.15.114:0/2436324471 learned_addr learned my addr 172.21.15.114:0/2436324471 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.643+0000 7faecbe00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faecc105bb0 0x7faecc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:12.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec3400640 1 -- 172.21.15.114:0/2436324471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faecc105bb0 msgr2=0x7faecc19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec3400640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faecc105bb0 0x7faecc19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec3400640 1 -- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10be40 msgr2=0x7faecc1a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec3400640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10be40 0x7faecc1a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec3400640 1 -- 172.21.15.114:0/2436324471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1a24b0 con 0x7faecc10b4f0 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faed1200640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10be40 0x7faecc1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec3400640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 0x7faecc19cae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faeb4005ab0 tx=0x7faeb4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.644+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb403f070 con 0x7faecc10b4f0 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.645+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1a2740 con 0x7faecc10b4f0 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.645+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb4031b50 con 0x7faecc10b4f0 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.645+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4035030 con 0x7faecc10b4f0 2024-09-21T06:54:12.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.645+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1a8f00 con 0x7faecc10b4f0 2024-09-21T06:54:12.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.646+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc106e30 con 0x7faecc10b4f0 2024-09-21T06:54:12.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.648+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7faeb4042070 con 0x7faecc10b4f0 2024-09-21T06:54:12.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.649+0000 7faec9600640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faeb00b65f0 0x7faeb00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:12.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.650+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7faeb4107f30 con 0x7faecc10b4f0 2024-09-21T06:54:12.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.650+0000 7faecbe00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faeb00b65f0 0x7faeb00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:12.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.650+0000 7faecbe00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faeb00b65f0 0x7faeb00b8b00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faebc009bd0 tx=0x7faebc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:12.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.652+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faeb4139710 con 0x7faecc10b4f0 2024-09-21T06:54:12.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:12 smithi114 ceph-mon[30654]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-09-21T06:54:12.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.880+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faecc105fc0 con 0x7faecc10b4f0 2024-09-21T06:54:12.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.881+0000 7faec9600640 1 -- 172.21.15.114:0/2436324471 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7faeb40d2550 con 0x7faecc10b4f0 2024-09-21T06:54:12.883 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:12.883 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"btime":"2024-09-21T06:53:49:573848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:53:49.006951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:54:12.883 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-21T06:54:12.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faeb00b65f0 msgr2=0x7faeb00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:12.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faeb00b65f0 0x7faeb00b8b00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faebc009bd0 tx=0x7faebc005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:12.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 msgr2=0x7faecc19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:12.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 0x7faecc19cae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faeb4005ab0 tx=0x7faeb4003b90 comp rx=0 tx=0).stop 2024-09-21T06:54:12.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 shutdown_connections 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7faecc10be40 0x7faecc1a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7faeb00b65f0 0x7faeb00b8b00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faecc10b4f0 0x7faecc19cae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 --2- 172.21.15.114:0/2436324471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faecc105bb0 0x7faecc19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.883+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 >> 172.21.15.114:0/2436324471 conn(0x7faecc101680 msgr2=0x7faecc103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.884+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 shutdown_connections 2024-09-21T06:54:12.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:12.884+0000 7faed1c00640 1 -- 172.21.15.114:0/2436324471 wait complete. 2024-09-21T06:54:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:12 smithi193 ceph-mon[34601]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-09-21T06:54:13.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:12 smithi183 ceph-mon[34219]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 174 KiB/s wr, 57 op/s 2024-09-21T06:54:13.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:54:13.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:13 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2436324471' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:14.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:13 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2436324471' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:14.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:13 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2436324471' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:14.149 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 -- 172.21.15.114:0/2218726636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e6010b6c0 msgr2=0x7f9e6010bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 --2- 172.21.15.114:0/2218726636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e6010b6c0 0x7f9e6010bad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009900 tx=0x7f9e4802a400 comp rx=0 tx=0).stop 2024-09-21T06:54:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 -- 172.21.15.114:0/2218726636 shutdown_connections 2024-09-21T06:54:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 --2- 172.21.15.114:0/2218726636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6010c010 0x7f9e60112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 --2- 172.21.15.114:0/2218726636 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e6010b6c0 0x7f9e6010bad0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:14.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 --2- 172.21.15.114:0/2218726636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e60105d80 0x7f9e60106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 -- 172.21.15.114:0/2218726636 >> 172.21.15.114:0/2218726636 conn(0x7f9e60101810 msgr2=0x7f9e60103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 -- 172.21.15.114:0/2218726636 shutdown_connections 2024-09-21T06:54:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.826+0000 7f9e66600640 1 -- 172.21.15.114:0/2218726636 wait complete. 2024-09-21T06:54:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 Processor -- start 2024-09-21T06:54:14.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 -- start start 2024-09-21T06:54:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 0x7f9e6019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e6010b6c0 0x7f9e6019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6010c010 0x7f9e601a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60115d20 con 0x7f9e60105d80 2024-09-21T06:54:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60115ba0 con 0x7f9e6010b6c0 2024-09-21T06:54:14.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.828+0000 7f9e66600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60115ea0 con 0x7f9e6010c010 2024-09-21T06:54:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 0x7f9e6019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 0x7f9e6019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33152/0 (socket says 172.21.15.114:33152) 2024-09-21T06:54:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 -- 172.21.15.114:0/1232818726 learned_addr learned my addr 172.21.15.114:0/1232818726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e56a00640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e6010b6c0 0x7f9e6019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:14.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5fe00640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6010c010 0x7f9e601a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 -- 172.21.15.114:0/1232818726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6010c010 msgr2=0x7f9e601a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6010c010 0x7f9e601a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 -- 172.21.15.114:0/1232818726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e6010b6c0 msgr2=0x7f9e6019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e6010b6c0 0x7f9e6019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.829+0000 7f9e5f400640 1 -- 172.21.15.114:0/1232818726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e601a2660 con 0x7f9e60105d80 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.830+0000 7f9e5f400640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 0x7f9e6019c6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e9f0 tx=0x7f9e5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.830+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000cd80 con 0x7f9e60105d80 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.830+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e500108e0 con 0x7f9e60105d80 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.830+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e601a2950 con 0x7f9e60105d80 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.830+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50018980 con 0x7f9e60105d80 2024-09-21T06:54:14.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.830+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a90b0 con 0x7f9e60105d80 2024-09-21T06:54:14.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.831+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e60106190 con 0x7f9e60105d80 2024-09-21T06:54:14.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.835+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f9e50010430 con 0x7f9e60105d80 2024-09-21T06:54:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.836+0000 7f9e5cc00640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e440b65f0 0x7f9e440b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.837+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9e500e8140 con 0x7f9e60105d80 2024-09-21T06:54:14.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.837+0000 7f9e56a00640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e440b65f0 0x7f9e440b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:14.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.838+0000 7f9e56a00640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e440b65f0 0x7f9e440b8b00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e4802a9f0 tx=0x7f9e48005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:14.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:14.839+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9e500b27c0 con 0x7f9e60105d80 2024-09-21T06:54:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:14 smithi193 ceph-mon[34601]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:15.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.028+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e6019d820 con 0x7f9e60105d80 2024-09-21T06:54:15.030 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:14 smithi114 ceph-mon[30654]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:15.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.029+0000 7f9e5cc00640 1 -- 172.21.15.114:0/1232818726 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25277 (secure 0 0 0) 0x7f9e500b7700 con 0x7f9e60105d80 2024-09-21T06:54:15.031 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:15.031 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":89,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T06:52:31.637070+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e440b65f0 msgr2=0x7f9e440b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e440b65f0 0x7f9e440b8b00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e4802a9f0 tx=0x7f9e48005a20 comp rx=0 tx=0).stop 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 msgr2=0x7f9e6019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 0x7f9e6019c6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e9f0 tx=0x7f9e5000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 shutdown_connections 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e6010c010 0x7f9e601a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e440b65f0 0x7f9e440b8b00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e6010b6c0 0x7f9e6019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 --2- 172.21.15.114:0/1232818726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9e60105d80 0x7f9e6019c6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.032+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 >> 172.21.15.114:0/1232818726 conn(0x7f9e60101810 msgr2=0x7f9e60103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.033+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 shutdown_connections 2024-09-21T06:54:15.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:15.033+0000 7f9e66600640 1 -- 172.21.15.114:0/1232818726 wait complete. 2024-09-21T06:54:15.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:14 smithi183 ceph-mon[34219]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 174 KiB/s wr, 56 op/s 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 -- 172.21.15.114:0/3255047760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c109b90 msgr2=0x7fda0c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 --2- 172.21.15.114:0/3255047760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c109b90 0x7fda0c1103c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fda04009ca0 tx=0x7fda0402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 -- 172.21.15.114:0/3255047760 shutdown_connections 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 --2- 172.21.15.114:0/3255047760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c109b90 0x7fda0c1103c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 --2- 172.21.15.114:0/3255047760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda0c109240 0x7fda0c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 --2- 172.21.15.114:0/3255047760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda0c103910 0x7fda0c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 -- 172.21.15.114:0/3255047760 >> 172.21.15.114:0/3255047760 conn(0x7fda0c0ff330 msgr2=0x7fda0c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 -- 172.21.15.114:0/3255047760 shutdown_connections 2024-09-21T06:54:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.652+0000 7fda12400640 1 -- 172.21.15.114:0/3255047760 wait complete. 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.653+0000 7fda12400640 1 Processor -- start 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.653+0000 7fda12400640 1 -- start start 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda12400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 0x7fda0c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda12400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda0c109240 0x7fda0c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda12400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda0c109b90 0x7fda0c1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda12400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c113850 con 0x7fda0c103910 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda12400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1136d0 con 0x7fda0c109240 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda12400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1139d0 con 0x7fda0c109b90 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda11000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 0x7fda0c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda0c109240 0x7fda0c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda11000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 0x7fda0c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33170/0 (socket says 172.21.15.114:33170) 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda11000640 1 -- 172.21.15.114:0/2407203248 learned_addr learned my addr 172.21.15.114:0/2407203248 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.654+0000 7fda11a00640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda0c109b90 0x7fda0c1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda11000640 1 -- 172.21.15.114:0/2407203248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda0c109b90 msgr2=0x7fda0c1a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda11000640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda0c109b90 0x7fda0c1a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda11000640 1 -- 172.21.15.114:0/2407203248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda0c109240 msgr2=0x7fda0c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda11000640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda0c109240 0x7fda0c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda11000640 1 -- 172.21.15.114:0/2407203248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1a2380 con 0x7fda0c103910 2024-09-21T06:54:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda11000640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 0x7fda0c19c450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d950 tx=0x7fda0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda09600640 1 -- 172.21.15.114:0/2407203248 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00017030 con 0x7fda0c103910 2024-09-21T06:54:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda09600640 1 -- 172.21.15.114:0/2407203248 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda00004020 con 0x7fda0c103910 2024-09-21T06:54:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c1a2670 con 0x7fda0c103910 2024-09-21T06:54:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda09600640 1 -- 172.21.15.114:0/2407203248 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000047a0 con 0x7fda0c103910 2024-09-21T06:54:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.655+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c1a8e00 con 0x7fda0c103910 2024-09-21T06:54:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.658+0000 7fda1aa1e740 1 Processor -- start 2024-09-21T06:54:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.658+0000 7fda1aa1e740 1 -- start start 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 0x555a50e72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50aad4d0 0x555a50e73890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555a50de86b0 con 0x555a50aa52f0 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a50dd7ae0 con 0x555a50aad4d0 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555a50ddb8f0 con 0x555a50ace1b0 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda09600640 1 -- 172.21.15.114:0/2407203248 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fda00009a30 con 0x7fda0c103910 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50aad4d0 0x555a50e73890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda11a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 0x555a50e72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50aad4d0 0x555a50e73890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54302/0 (socket says 172.21.15.114:54302) 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda0be00640 1 -- 172.21.15.114:0/2022030273 learned_addr learned my addr 172.21.15.114:0/2022030273 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda11a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 0x555a50e72e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51908/0 (socket says 172.21.15.114:51908) 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda11000640 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda11a00640 1 -- 172.21.15.114:0/2022030273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50aad4d0 msgr2=0x555a50e73890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda11a00640 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50aad4d0 0x555a50e73890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.659+0000 7fda11a00640 1 -- 172.21.15.114:0/2022030273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 msgr2=0x555a50e73350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.660+0000 7fda11a00640 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e73350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.660+0000 7fda11a00640 1 -- 172.21.15.114:0/2022030273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a50da4780 con 0x555a50ace1b0 2024-09-21T06:54:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.660+0000 7fda11a00640 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 0x555a50e72e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda0402ade0 tx=0x7fda0402f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4f1a8af5752698 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.660+0000 7fd9fe000640 1 -- 172.21.15.114:0/2022030273 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04031cf0 con 0x555a50ace1b0 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.660+0000 7fd9fe000640 1 -- 172.21.15.114:0/2022030273 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda04036070 con 0x555a50ace1b0 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.660+0000 7fd9fe000640 1 -- 172.21.15.114:0/2022030273 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040042a0 con 0x555a50ace1b0 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda09600640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9f40b65f0 0x7fd9f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda11000640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9f40b65f0 0x7fd9f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda09600640 1 -- 172.21.15.114:0/2407203248 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fda000e7470 con 0x7fda0c103910 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2022030273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 msgr2=0x555a50e72e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 0x555a50e72e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fda0402ade0 tx=0x7fda0402f4c0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2022030273 shutdown_connections 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50aad4d0 0x555a50e73890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e73350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2022030273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50ace1b0 0x555a50e72e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.661+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2022030273 >> 172.21.15.114:0/2022030273 conn(0x555a50ac2300 msgr2=0x555a50e70d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.662+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2022030273 shutdown_connections 2024-09-21T06:54:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.662+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2022030273 wait complete. 2024-09-21T06:54:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.662+0000 7fda11000640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9f40b65f0 0x7fd9f40b8b00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda000e4960 tx=0x7fda00000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.662+0000 7fda1aa1e740 1 -- start start 2024-09-21T06:54:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 Processor -- start 2024-09-21T06:54:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aa52f0 0x555a50e7e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 0x555a50e7e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555a50d0fab0 con 0x555a50aad4d0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a50d7e570 con 0x555a50ace1b0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555a50e813b0 con 0x555a50aa52f0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda1aa1e740 10 client.0 _ll_register_callbacks cb 0x555a50d8d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 0x555a50e7e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 0x555a50e7e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33188/0 (socket says 172.21.15.114:33188) 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda0be00640 1 -- 172.21.15.114:0/2973500487 learned_addr learned my addr 172.21.15.114:0/2973500487 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.663+0000 7fda11a00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aa52f0 0x555a50e7e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda11a00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda0be00640 1 -- 172.21.15.114:0/2973500487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aa52f0 msgr2=0x555a50e7e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda0be00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aa52f0 0x555a50e7e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda0be00640 1 -- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 msgr2=0x555a50e7eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda0be00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda0be00640 1 -- 172.21.15.114:0/2973500487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a50b39790 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda11a00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda0be00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 0x555a50e7e960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8012760 tx=0x7fd9f8012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe734c108d4a6d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fd9f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fd9f2a00640 1 -- 172.21.15.114:0/2973500487 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800f920 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda04036070 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.664+0000 7fd9f2a00640 1 -- 172.21.15.114:0/2973500487 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd9f800fac0 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fd9f2a00640 1 -- 172.21.15.114:0/2973500487 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9f800fcc0 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 10 client.15624 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a50bb2620 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fd9f2a00640 1 -- 172.21.15.114:0/2973500487 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda04036070 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fd9f2a00640 1 -- 172.21.15.114:0/2973500487 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd9f801c420 con 0x555a50aad4d0 2024-09-21T06:54:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 10 client.15624 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 20 client.15624 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a8cd80 0x555a50e90430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 4 client.15624 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.665+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a50dea600 con 0x555a50a8cd80 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.666+0000 7fda11a00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a8cd80 0x555a50e90430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.666+0000 7fda11a00640 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a8cd80 0x555a50e90430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e09ce3df3601adb server_cookie=5e95de83883145c7 in_seq=0 out_seq=0 2024-09-21T06:54:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.666+0000 7fd9f2a00640 10 client.15624 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fd9f2a00640 1 -- 172.21.15.114:0/2973500487 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a50dea600 con 0x555a50a8cd80 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fd9f2a00640 10 client.15624 handle_command_reply: tid=0 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 client.15624 shutdown 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 msgr2=0x555a50e7e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 0x555a50e7e960 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8012760 tx=0x7fd9f8012c20 comp rx=0 tx=0).stop 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 shutdown_connections 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aad4d0 0x555a50e7e960 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a8cd80 0x555a50e90430 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.667+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/2973500487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aa52f0 0x555a50e7e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.668+0000 7fd9f2a00640 0 client.15624 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.668+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 >> 172.21.15.114:0/2973500487 conn(0x555a50ac2300 msgr2=0x555a50e73e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.668+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 shutdown_connections 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.668+0000 7fda1aa1e740 1 -- 172.21.15.114:0/2973500487 wait complete. 2024-09-21T06:54:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.668+0000 7fda1aa1e740 20 client.15624 trim_cache size 0 max 16384 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.683+0000 7fda1aa1e740 1 Processor -- start 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 -- start start 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e74550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e7b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1b2d00 con 0x555a50aa52f0 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1b4840 con 0x555a50ace1b0 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1b33d0 con 0x555a50aad4d0 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda11a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e74550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda11a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e74550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33202/0 (socket says 172.21.15.114:33202) 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda11a00640 1 -- 172.21.15.114:0/4280130940 learned_addr learned my addr 172.21.15.114:0/4280130940 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.684+0000 7fda0be00640 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e7b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 -- 172.21.15.114:0/4280130940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 msgr2=0x555a50e7b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e7b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 -- 172.21.15.114:0/4280130940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 msgr2=0x555a50e7bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 -- 172.21.15.114:0/4280130940 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a50e753b0 con 0x555a50aa52f0 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fda11a00640 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e74550 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda04000f80 tx=0x7fda0402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985a3ed0383d4197 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fd9fd600640 1 -- 172.21.15.114:0/4280130940 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04031bf0 con 0x555a50aa52f0 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fd9fd600640 1 -- 172.21.15.114:0/4280130940 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda04036040 con 0x555a50aa52f0 2024-09-21T06:54:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.685+0000 7fd9fd600640 1 -- 172.21.15.114:0/4280130940 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040041c0 con 0x555a50aa52f0 2024-09-21T06:54:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4280130940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 msgr2=0x555a50e74550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e74550 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fda04000f80 tx=0x7fda0402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4280130940 shutdown_connections 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50e7bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e7b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4280130940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e74550 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4280130940 >> 172.21.15.114:0/4280130940 conn(0x555a50ac2300 msgr2=0x555a50e73e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4280130940 shutdown_connections 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.686+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4280130940 wait complete. 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 -- start start 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 Processor -- start 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e83f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50ef2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1b4170 con 0x555a50aa52f0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1b3aa0 con 0x555a50ace1b0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c1b0ce0 con 0x555a50aad4d0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.687+0000 7fda1aa1e740 10 client.0 _ll_register_callbacks cb 0x7fd9f8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e83f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e83f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33208/0 (socket says 172.21.15.114:33208) 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 -- 172.21.15.114:0/4007086968 learned_addr learned my addr 172.21.15.114:0/4007086968 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda0be00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50ef2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 -- 172.21.15.114:0/4007086968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 msgr2=0x555a50e844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e844c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 -- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 msgr2=0x555a50ef2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50ef2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 -- 172.21.15.114:0/4007086968 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a50ef3430 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fda11a00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e83f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fda04009bc0 tx=0x7fda0402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f36fa66285ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.688+0000 7fd9f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fd9f2a00640 1 -- 172.21.15.114:0/4007086968 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04063060 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fd9f2a00640 1 -- 172.21.15.114:0/4007086968 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fda04031450 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda04036040 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fd9f2a00640 1 -- 172.21.15.114:0/4007086968 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda0406a050 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fda1aa1e740 10 client.15630 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555a50efa780 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fd9f2a00640 1 -- 172.21.15.114:0/4007086968 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda04036040 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fd9f2a00640 1 -- 172.21.15.114:0/4007086968 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fda04043020 con 0x555a50aa52f0 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fda1aa1e740 10 client.15630 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.689+0000 7fda1aa1e740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fda1aa1e740 20 client.15630 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a621c0 0x555a50efcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fda1aa1e740 4 client.15630 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:54:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a50de60b0 con 0x555a50a621c0 2024-09-21T06:54:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fda0be00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a621c0 0x555a50efcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fda0be00640 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a621c0 0x555a50efcaf0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f82a724974db8ad server_cookie=f7cdc8716f11d5a1 in_seq=0 out_seq=0 2024-09-21T06:54:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.690+0000 7fd9f2a00640 10 client.15630 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fd9f2a00640 1 -- 172.21.15.114:0/4007086968 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a50de60b0 con 0x555a50a621c0 2024-09-21T06:54:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fd9f2a00640 10 client.15630 handle_command_reply: tid=0 2024-09-21T06:54:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 client.15630 shutdown 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 msgr2=0x555a50e83f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e83f20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fda04009bc0 tx=0x7fda0402f3a0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 shutdown_connections 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a50ace1b0 0x555a50ef2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555a50a621c0 0x555a50efcaf0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555a50aad4d0 0x555a50e844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 --2- 172.21.15.114:0/4007086968 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555a50aa52f0 0x555a50e83f20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fd9f2a00640 0 client.15630 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 >> 172.21.15.114:0/4007086968 conn(0x555a50ac2300 msgr2=0x555a50e895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 shutdown_connections 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 1 -- 172.21.15.114:0/4007086968 wait complete. 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.703+0000 7fda1aa1e740 20 client.15630 trim_cache size 0 max 16384 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:54:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9f40b65f0 msgr2=0x7fd9f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9f40b65f0 0x7fd9f40b8b00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda000e4960 tx=0x7fda00000bc0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 msgr2=0x7fda0c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 0x7fda0c19c450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fda0000d950 tx=0x7fda0000de10 comp rx=0 tx=0).stop 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 shutdown_connections 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda0c109b90 0x7fda0c1a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9f40b65f0 0x7fd9f40b8b00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda0c109240 0x7fda0c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 --2- 172.21.15.114:0/2407203248 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fda0c103910 0x7fda0c19c450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 >> 172.21.15.114:0/2407203248 conn(0x7fda0c0ff330 msgr2=0x7fda0c1088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 shutdown_connections 2024-09-21T06:54:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:15.704+0000 7fda12400640 1 -- 172.21.15.114:0/2407203248 wait complete. 2024-09-21T06:54:15.810 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:15 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1232818726' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:54:15.851 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mon.a --format=json config get run_dir 2024-09-21T06:54:15.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:15 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1232818726' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:54:16.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:15 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1232818726' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:54:16.219 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:16 smithi193 ceph-mon[34601]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:54:16.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:54:16.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:16 smithi114 ceph-mon[30654]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:54:16.957 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:54:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:16 smithi183 ceph-mon[34219]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:54:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:54:17.673 INFO:teuthology.orchestra.run.smithi114.stdout:{"run_dir":"/var/run/ceph"} 2024-09-21T06:54:17.674 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-21T06:54:17.674 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7864 2024-09-21T06:54:17.674 DEBUG:tasks.quiescer:Yielding 2024-09-21T06:54:17.674 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T06:54:17.684 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:54:17.684 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T06:54:17.684 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-21T06:54:17.685 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:54:17.764 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:17.794 INFO:teuthology.orchestra.run.smithi114.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-21T06:54:17.794 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6789,172.21.15.183:6789,172.21.15.193:6789:/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644 1.1T 372M 1.1T 1% /tmp/cephfs 2024-09-21T06:54:17.799 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-21T06:54:17.951 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-21T06:54:18.038 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-21T06:54:18.071 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 -- 172.21.15.114:0/2289433537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 msgr2=0x7ffb00110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2289433537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 0x7ffb00110470 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009ca0 tx=0x7ffaf802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 -- 172.21.15.114:0/2289433537 shutdown_connections 2024-09-21T06:54:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2289433537 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 0x7ffb00110470 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2289433537 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 0x7ffb00109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2289433537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb00103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 -- 172.21.15.114:0/2289433537 >> 172.21.15.114:0/2289433537 conn(0x7ffb000ff440 msgr2=0x7ffb00101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 -- 172.21.15.114:0/2289433537 shutdown_connections 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.745+0000 7ffb06a00640 1 -- 172.21.15.114:0/2289433537 wait complete. 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.746+0000 7ffb06a00640 1 Processor -- start 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.746+0000 7ffb06a00640 1 -- start start 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb0019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 0x7ffb0019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 0x7ffb001a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00113900 con 0x7ffb00109c40 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00113780 con 0x7ffb001039b0 2024-09-21T06:54:18.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffb00113a80 con 0x7ffb001092f0 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb0019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb04c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 0x7ffb0019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb06000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 0x7ffb001a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb0019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54372/0 (socket says 172.21.15.114:54372) 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 -- 172.21.15.114:0/2222473118 learned_addr learned my addr 172.21.15.114:0/2222473118 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 -- 172.21.15.114:0/2222473118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 msgr2=0x7ffb0019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 0x7ffb0019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 -- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 msgr2=0x7ffb001a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 0x7ffb001a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb05600640 1 -- 172.21.15.114:0/2222473118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb001a2420 con 0x7ffb001039b0 2024-09-21T06:54:18.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.747+0000 7ffb04c00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 0x7ffb0019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.748+0000 7ffb05600640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb0019c4b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e9f0 tx=0x7ffaf400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.748+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400cd80 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.748+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001a2710 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.748+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaf40108e0 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.748+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4018980 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.748+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001a8e70 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.749+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb00103dc0 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.753+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7ffaf4010430 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.755+0000 7ffafe000640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffae80b65f0 0x7ffae80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.755+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ffaf40e8140 con 0x7ffb001039b0 2024-09-21T06:54:18.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.755+0000 7ffb04c00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffae80b65f0 0x7ffae80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:18.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.755+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffaf40b27c0 con 0x7ffb001039b0 2024-09-21T06:54:18.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.756+0000 7ffb04c00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffae80b65f0 0x7ffae80b8b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffb00107d10 tx=0x7ffaec005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:18.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:18 smithi114 ceph-mon[30654]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T06:54:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:18 smithi193 ceph-mon[34601]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T06:54:18.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.992+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb0019d5e0 con 0x7ffb001039b0 2024-09-21T06:54:18.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.994+0000 7ffafe000640 1 -- 172.21.15.114:0/2222473118 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7ffaf40b7700 con 0x7ffb001039b0 2024-09-21T06:54:18.995 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:18.996 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"btime":"2024-09-21T06:53:49:573848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:53:49.006951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:54:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffae80b65f0 msgr2=0x7ffae80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffae80b65f0 0x7ffae80b8b00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffb00107d10 tx=0x7ffaec005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 msgr2=0x7ffb0019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb0019c4b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e9f0 tx=0x7ffaf400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 shutdown_connections 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ffb00109c40 0x7ffb001a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb001092f0 0x7ffb0019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ffae80b65f0 0x7ffae80b8b00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 --2- 172.21.15.114:0/2222473118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb001039b0 0x7ffb0019c4b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 >> 172.21.15.114:0/2222473118 conn(0x7ffb000ff440 msgr2=0x7ffb001089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:18.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.996+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 shutdown_connections 2024-09-21T06:54:18.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:18.997+0000 7ffb06a00640 1 -- 172.21.15.114:0/2222473118 wait complete. 2024-09-21T06:54:19.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:18 smithi183 ceph-mon[34219]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-21T06:54:19.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:19 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2222473118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:19.969 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:19 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2222473118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:20.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:19 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2222473118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:20.441 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-21T06:54:20.448 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T06:50:48.407210+0000', 'modified': '2024-09-21T06:53:49.006951+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24761, 'mds_1': 24760, 'mds_2': 24742, 'mds_3': 24727, 'mds_4': 24725}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14742': {'gid': 14742, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.114:6839/244481807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6838', 'nonce': 244481807}, {'type': 'v1', 'addr': '172.21.15.114:6839', 'nonce': 244481807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24667': {'gid': 24667, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6833/2086656361', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6832', 'nonce': 2086656361}, {'type': 'v1', 'addr': '172.21.15.193:6833', 'nonce': 2086656361}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24668': {'gid': 24668, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6833/919969847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 919969847}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 919969847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2729823804', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2729823804}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2729823804}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24715': {'gid': 24715, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/2057077471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2057077471}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2057077471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24725': {'gid': 24725, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.183:6837/841299119', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 841299119}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 841299119}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24727': {'gid': 24727, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.193:6837/1013898612', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1013898612}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1013898612}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24742': {'gid': 24742, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.193:6839/3226153327', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3226153327}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3226153327}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24760': {'gid': 24760, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.114:6841/2055898271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.114:6840', 'nonce': 2055898271}, {'type': 'v1', 'addr': '172.21.15.114:6841', 'nonce': 2055898271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24761': {'gid': 24761, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.183:6839/42004963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 42004963}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 42004963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24761, 'qdb_cluster': [24761, 24760, 24742, 24727, 24725]} 2024-09-21T06:54:20.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T06:54:20.449 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T06:54:20.567 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-21T06:54:20.572 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-21T06:54:20.699 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/: 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 .. 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 data 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data: 2024-09-21T06:54:20.781 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:20.782 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:20.782 INFO:teuthology.orchestra.run.smithi114.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 .. 2024-09-21T06:54:20.788 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-21T06:54:20.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:20 smithi114 ceph-mon[30654]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T06:54:20.887 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:20.895 INFO:teuthology.orchestra.run.smithi114.stdout:[Service] 2024-09-21T06:54:20.895 INFO:teuthology.orchestra.run.smithi114.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-21T06:54:20.895 INFO:teuthology.orchestra.run.smithi114.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-21T06:54:20.900 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-21T06:54:20.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:20 smithi193 ceph-mon[34601]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T06:54:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:20 smithi183 ceph-mon[34219]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-21T06:54:21.125 INFO:teuthology.orchestra.run.smithi114.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-21T06:54:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 -- 172.21.15.114:0/3699332427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620103b60 msgr2=0x7fc620103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 --2- 172.21.15.114:0/3699332427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620103b60 0x7fc620103f70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc610009ca0 tx=0x7fc61002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 -- 172.21.15.114:0/3699332427 shutdown_connections 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 --2- 172.21.15.114:0/3699332427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc620109df0 0x7fc620110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 --2- 172.21.15.114:0/3699332427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6201094a0 0x7fc6201098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 --2- 172.21.15.114:0/3699332427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc620103b60 0x7fc620103f70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 -- 172.21.15.114:0/3699332427 >> 172.21.15.114:0/3699332427 conn(0x7fc62007b850 msgr2=0x7fc62007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 -- 172.21.15.114:0/3699332427 shutdown_connections 2024-09-21T06:54:21.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.586+0000 7fc626200640 1 -- 172.21.15.114:0/3699332427 wait complete. 2024-09-21T06:54:21.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 Processor -- start 2024-09-21T06:54:21.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 -- start start 2024-09-21T06:54:21.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc620103b60 0x7fc62019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 0x7fc62019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc620109df0 0x7fc62019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc620113ab0 con 0x7fc620103b60 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc620113930 con 0x7fc6201094a0 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.588+0000 7fc626200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc620113c30 con 0x7fc620109df0 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc61f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc620103b60 0x7fc62019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 0x7fc62019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 0x7fc62019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54384/0 (socket says 172.21.15.114:54384) 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 -- 172.21.15.114:0/2406538867 learned_addr learned my addr 172.21.15.114:0/2406538867 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc61fe00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc620109df0 0x7fc62019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 -- 172.21.15.114:0/2406538867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc620109df0 msgr2=0x7fc62019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc620109df0 0x7fc62019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 -- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc620103b60 msgr2=0x7fc62019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc620103b60 0x7fc62019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 -- 172.21.15.114:0/2406538867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6201a03b0 con 0x7fc6201094a0 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.589+0000 7fc616a00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 0x7fc62019a9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9f0 tx=0x7fc60800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.590+0000 7fc61cc00640 1 -- 172.21.15.114:0/2406538867 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60800ccc0 con 0x7fc6201094a0 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.590+0000 7fc61cc00640 1 -- 172.21.15.114:0/2406538867 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6080108e0 con 0x7fc6201094a0 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.590+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201a06a0 con 0x7fc6201094a0 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.590+0000 7fc61cc00640 1 -- 172.21.15.114:0/2406538867 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608018860 con 0x7fc6201094a0 2024-09-21T06:54:21.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.590+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6200755f0 con 0x7fc6201094a0 2024-09-21T06:54:21.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 Processor -- start 2024-09-21T06:54:21.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 -- start start 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d0274a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d026c1d50 con 0x560d023a3f60 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d026b1180 con 0x560d0237b0a0 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.593+0000 7fc62ce00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d026b4f90 con 0x560d02383280 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54390/0 (socket says 172.21.15.114:54390) 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 -- 172.21.15.114:0/2267085767 learned_addr learned my addr 172.21.15.114:0/2267085767 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61fe00640 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d0274a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61cc00640 1 -- 172.21.15.114:0/2406538867 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fc608010430 con 0x7fc6201094a0 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 -- 172.21.15.114:0/2267085767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 msgr2=0x560d0274af10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 -- 172.21.15.114:0/2267085767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 msgr2=0x560d0274a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d0274a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 -- 172.21.15.114:0/2267085767 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0267ba70 con 0x560d0237b0a0 2024-09-21T06:54:21.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.594+0000 7fc61f400640 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274a9d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc610035000 tx=0x7fc6100093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea16ae1945e93f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:21.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.595+0000 7fc60fe00640 1 -- 172.21.15.114:0/2267085767 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610031cf0 con 0x560d0237b0a0 2024-09-21T06:54:21.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.595+0000 7fc60fe00640 1 -- 172.21.15.114:0/2267085767 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc610036030 con 0x560d0237b0a0 2024-09-21T06:54:21.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.595+0000 7fc60fe00640 1 -- 172.21.15.114:0/2267085767 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100042a0 con 0x560d0237b0a0 2024-09-21T06:54:21.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc61cc00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6040b65f0 0x7fc6040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc61cc00640 1 -- 172.21.15.114:0/2406538867 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc6080e7d40 con 0x7fc6201094a0 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc61fe00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6040b65f0 0x7fc6040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 -- 172.21.15.114:0/2267085767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 msgr2=0x560d0274a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274a9d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc610035000 tx=0x7fc6100093f0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 -- 172.21.15.114:0/2267085767 shutdown_connections 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274a9d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 --2- 172.21.15.114:0/2267085767 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d0274a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 -- 172.21.15.114:0/2267085767 >> 172.21.15.114:0/2267085767 conn(0x560d02398350 msgr2=0x560d02748380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.596+0000 7fc62ce00740 1 -- 172.21.15.114:0/2267085767 shutdown_connections 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.597+0000 7fc62ce00740 1 -- 172.21.15.114:0/2267085767 wait complete. 2024-09-21T06:54:21.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.597+0000 7fc61fe00640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6040b65f0 0x7fc6040b8b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc618009c00 tx=0x7fc618005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.597+0000 7fc62ce00740 1 -- start start 2024-09-21T06:54:21.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.597+0000 7fc62ce00740 1 Processor -- start 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d027560d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d02756630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02761d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d025125f0 con 0x560d023a3f60 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d0264c460 con 0x560d0237b0a0 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d02758a90 con 0x560d02383280 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc62ce00740 10 client.0 _ll_register_callbacks cb 0x560d0265c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc61f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d027560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc61f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d027560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54400/0 (socket says 172.21.15.114:54400) 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc61f400640 1 -- 172.21.15.114:0/4094388864 learned_addr learned my addr 172.21.15.114:0/4094388864 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.598+0000 7fc616a00640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d02756630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61fe00640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02761d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61f400640 1 -- 172.21.15.114:0/4094388864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 msgr2=0x560d02756630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61f400640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d02756630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61f400640 1 -- 172.21.15.114:0/4094388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 msgr2=0x560d02761d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61f400640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02761d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61f400640 1 -- 172.21.15.114:0/4094388864 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d0240f540 con 0x560d0237b0a0 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61fe00640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02761d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc61f400640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d027560d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc610005e30 tx=0x7fc610003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57fe49fd8f5f7ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc60cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc60cc00640 1 -- 172.21.15.114:0/4094388864 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610003970 con 0x560d0237b0a0 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc60cc00640 1 -- 172.21.15.114:0/4094388864 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc61002f660 con 0x560d0237b0a0 2024-09-21T06:54:21.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.599+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d0264ea70 con 0x560d0237b0a0 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc60cc00640 1 -- 172.21.15.114:0/4094388864 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc61002f860 con 0x560d0237b0a0 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc62ce00740 10 client.25643 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d026696f0 con 0x560d0237b0a0 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc60cc00640 1 -- 172.21.15.114:0/4094388864 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560d0264ea70 con 0x560d0237b0a0 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc60cc00640 1 -- 172.21.15.114:0/4094388864 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc61004e070 con 0x560d0237b0a0 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc62ce00740 10 client.25643 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc62ce00740 10 client.25643 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:54:21.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc62ce00740 20 client.25643 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.600+0000 7fc62ce00740 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d023628e0 0x560d02768010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.601+0000 7fc62ce00740 4 client.25643 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.601+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d026c3ca0 con 0x560d023628e0 2024-09-21T06:54:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.601+0000 7fc616a00640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d023628e0 0x560d02768010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.601+0000 7fc616a00640 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d023628e0 0x560d02768010 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=91db4d82e210d206 server_cookie=c58982aff7a331fd in_seq=0 out_seq=0 2024-09-21T06:54:21.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.601+0000 7fc60cc00640 10 client.25643 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc60cc00640 1 -- 172.21.15.114:0/4094388864 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d026c3ca0 con 0x560d023628e0 2024-09-21T06:54:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc60cc00640 10 client.25643 handle_command_reply: tid=0 2024-09-21T06:54:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 client.25643 shutdown 2024-09-21T06:54:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 msgr2=0x560d027560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d027560d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc610005e30 tx=0x7fc610003ce0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 shutdown_connections 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02761d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d02756630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d023628e0 0x560d02768010 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 --2- 172.21.15.114:0/4094388864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d027560d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 >> 172.21.15.114:0/4094388864 conn(0x560d02398350 msgr2=0x560d0274d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.602+0000 7fc60cc00640 0 client.25643 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.603+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 shutdown_connections 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.603+0000 7fc62ce00740 1 -- 172.21.15.114:0/4094388864 wait complete. 2024-09-21T06:54:21.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.603+0000 7fc62ce00740 20 client.25643 trim_cache size 0 max 16384 2024-09-21T06:54:21.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 Processor -- start 2024-09-21T06:54:21.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 -- start start 2024-09-21T06:54:21.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02751010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d02766350 con 0x560d023a3f60 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d02766a20 con 0x560d0237b0a0 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.622+0000 7fc62ce00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d027670f0 con 0x560d02383280 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54404/0 (socket says 172.21.15.114:54404) 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 -- 172.21.15.114:0/3093227404 learned_addr learned my addr 172.21.15.114:0/3093227404 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61fe00640 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02751010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc616a00640 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 -- 172.21.15.114:0/3093227404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 msgr2=0x560d0274fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 -- 172.21.15.114:0/3093227404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 msgr2=0x560d02751010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02751010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 -- 172.21.15.114:0/3093227404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d0274d2f0 con 0x560d0237b0a0 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.623+0000 7fc61f400640 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274f710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc610002230 tx=0x7fc6100099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caeeb917d1bbe7f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.624+0000 7fc60f400640 1 -- 172.21.15.114:0/3093227404 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610031660 con 0x560d0237b0a0 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.624+0000 7fc60f400640 1 -- 172.21.15.114:0/3093227404 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc610054070 con 0x560d0237b0a0 2024-09-21T06:54:21.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.624+0000 7fc60f400640 1 -- 172.21.15.114:0/3093227404 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610003970 con 0x560d0237b0a0 2024-09-21T06:54:21.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 -- 172.21.15.114:0/3093227404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 msgr2=0x560d0274f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274f710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc610002230 tx=0x7fc6100099c0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 -- 172.21.15.114:0/3093227404 shutdown_connections 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d023a3f60 0x560d02751010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d02383280 0x560d0274fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 --2- 172.21.15.114:0/3093227404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d0237b0a0 0x560d0274f710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 -- 172.21.15.114:0/3093227404 >> 172.21.15.114:0/3093227404 conn(0x560d02398350 msgr2=0x560d027560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 -- 172.21.15.114:0/3093227404 shutdown_connections 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.625+0000 7fc62ce00740 1 -- 172.21.15.114:0/3093227404 wait complete. 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 -- start start 2024-09-21T06:54:21.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 Processor -- start 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d0237b0a0 0x560d02770750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 0x560d02770cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d023a3f60 0x560d02747240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560d02767e90 con 0x560d0237b0a0 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d027677c0 con 0x560d02383280 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560d02768560 con 0x560d023a3f60 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.626+0000 7fc62ce00740 10 client.0 _ll_register_callbacks cb 0x7fc610031ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 0x560d02770cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc61f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d0237b0a0 0x560d02770750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 0x560d02770cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54418/0 (socket says 172.21.15.114:54418) 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 -- 172.21.15.114:0/1906635580 learned_addr learned my addr 172.21.15.114:0/1906635580 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc61fe00640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d023a3f60 0x560d02747240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 -- 172.21.15.114:0/1906635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d023a3f60 msgr2=0x560d02747240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d023a3f60 0x560d02747240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 -- 172.21.15.114:0/1906635580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d0237b0a0 msgr2=0x560d02770750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d0237b0a0 0x560d02770750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 -- 172.21.15.114:0/1906635580 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d027479d0 con 0x560d02383280 2024-09-21T06:54:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc61f400640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d0237b0a0 0x560d02770750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc616a00640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 0x560d02770cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ad810 tx=0x7fc6080096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=303c971b72d7753a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.627+0000 7fc60cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc60cc00640 1 -- 172.21.15.114:0/1906635580 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080ad390 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc60cc00640 1 -- 172.21.15.114:0/1906635580 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc6080aebb0 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc610054070 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc60cc00640 1 -- 172.21.15.114:0/1906635580 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6080adf80 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc62ce00740 10 client.25649 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560d0276de60 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.628+0000 7fc60cc00640 1 -- 172.21.15.114:0/1906635580 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc610054070 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc60cc00640 1 -- 172.21.15.114:0/1906635580 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fc6080ac550 con 0x560d02383280 2024-09-21T06:54:21.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc62ce00740 10 client.25649 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc62ce00740 10 client.25649 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc62ce00740 20 client.25649 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc62ce00740 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d02337c90 0x560d027d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc62ce00740 4 client.25649 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "status"}) -- 0x560d026bf750 con 0x560d02337c90 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.629+0000 7fc61f400640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d02337c90 0x560d027d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc61f400640 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d02337c90 0x560d027d6900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b53cf9c9977517d7 server_cookie=3a06feccda694d14 in_seq=0 out_seq=0 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc60cc00640 10 client.25649 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc60cc00640 1 -- 172.21.15.114:0/1906635580 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x560d026bf750 con 0x560d02337c90 2024-09-21T06:54:21.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc60cc00640 10 client.25649 handle_command_reply: tid=0 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 client.25649 shutdown 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 msgr2=0x560d02770cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 0x560d02770cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ad810 tx=0x7fc6080096a0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 shutdown_connections 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d023a3f60 0x560d02747240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x560d02337c90 0x560d027d6900 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d02383280 0x560d02770cf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 --2- 172.21.15.114:0/1906635580 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560d0237b0a0 0x560d02770750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc60cc00640 0 client.25649 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.630+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 >> 172.21.15.114:0/1906635580 conn(0x560d02398350 msgr2=0x560d02751010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:21.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 shutdown_connections 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc62ce00740 1 -- 172.21.15.114:0/1906635580 wait complete. 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc62ce00740 20 client.25649 trim_cache size 0 max 16384 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 4, 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24725, 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:54:21.637 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:54:21.638 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 159.030713405, 2024-09-21T06:54:21.638 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-21T06:54:21.638 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:54:21.638 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 89, 2024-09-21T06:54:21.638 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 238.30104813 2024-09-21T06:54:21.638 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6040b65f0 msgr2=0x7fc6040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc626200640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6040b65f0 0x7fc6040b8b00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc618009c00 tx=0x7fc618005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 msgr2=0x7fc62019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.631+0000 7fc626200640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 0x7fc62019a9e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9f0 tx=0x7fc60800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 shutdown_connections 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc620109df0 0x7fc62019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6040b65f0 0x7fc6040b8b00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6201094a0 0x7fc62019a9e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 --2- 172.21.15.114:0/2406538867 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc620103b60 0x7fc62019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 >> 172.21.15.114:0/2406538867 conn(0x7fc62007b850 msgr2=0x7fc6201074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 shutdown_connections 2024-09-21T06:54:21.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:21.632+0000 7fc626200640 1 -- 172.21.15.114:0/2406538867 wait complete. 2024-09-21T06:54:22.480 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T06:54:22.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:22 smithi114 ceph-mon[30654]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-21T06:54:22.876 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:22.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:22 smithi193 ceph-mon[34601]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-21T06:54:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:22 smithi183 ceph-mon[34219]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 15 op/s 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 -- 172.21.15.114:0/1891024644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1904105bc0 msgr2=0x7f1904105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 --2- 172.21.15.114:0/1891024644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1904105bc0 0x7f1904105fd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009900 tx=0x7f18f402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 -- 172.21.15.114:0/1891024644 shutdown_connections 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 --2- 172.21.15.114:0/1891024644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f190410be50 0x7f1904112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 --2- 172.21.15.114:0/1891024644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190410b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 --2- 172.21.15.114:0/1891024644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1904105bc0 0x7f1904105fd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 -- 172.21.15.114:0/1891024644 >> 172.21.15.114:0/1891024644 conn(0x7f1904101670 msgr2=0x7f1904103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 -- 172.21.15.114:0/1891024644 shutdown_connections 2024-09-21T06:54:23.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.557+0000 7f1903e00640 1 -- 172.21.15.114:0/1891024644 wait complete. 2024-09-21T06:54:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.558+0000 7f1903e00640 1 Processor -- start 2024-09-21T06:54:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.558+0000 7f1903e00640 1 -- start start 2024-09-21T06:54:23.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1904105bc0 0x7f190419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 0x7f19041a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1904115b60 con 0x7f190410be50 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19041159e0 con 0x7f190410b500 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1904115ce0 con 0x7f1904105bc0 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 0x7f19041a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1902a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1904105bc0 0x7f190419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 0x7f19041a1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33312/0 (socket says 172.21.15.114:33312) 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54438/0 (socket says 172.21.15.114:54438) 2024-09-21T06:54:23.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.559+0000 7f1903400640 1 -- 172.21.15.114:0/2355165648 learned_addr learned my addr 172.21.15.114:0/2355165648 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:23.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903400640 1 -- 172.21.15.114:0/2355165648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1904105bc0 msgr2=0x7f190419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903400640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1904105bc0 0x7f190419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903400640 1 -- 172.21.15.114:0/2355165648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 msgr2=0x7f190419cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903400640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190419cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903400640 1 -- 172.21.15.114:0/2355165648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041a24a0 con 0x7f190410be50 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1902000640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1902a00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1904105bc0 0x7f190419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903400640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 0x7f19041a1d70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00eb10 tx=0x7f18fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f18fb400640 1 -- 172.21.15.114:0/2355165648 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc004b20 con 0x7f190410be50 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f18fb400640 1 -- 172.21.15.114:0/2355165648 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc004cc0 con 0x7f190410be50 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19041a2790 con 0x7f190410be50 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f18fb400640 1 -- 172.21.15.114:0/2355165648 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc010670 con 0x7f190410be50 2024-09-21T06:54:23.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.560+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19041a8ef0 con 0x7f190410be50 2024-09-21T06:54:23.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.564+0000 7f191084b740 1 Processor -- start 2024-09-21T06:54:23.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.564+0000 7f18fb400640 1 -- 172.21.15.114:0/2355165648 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f18fc005020 con 0x7f190410be50 2024-09-21T06:54:23.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.564+0000 7f191084b740 1 -- start start 2024-09-21T06:54:23.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f191084b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53f97670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f191084b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53f97bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f191084b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f191084b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cd53f0ed50 con 0x55cd53bc80a0 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f191084b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd53efe180 con 0x55cd53bd0280 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f191084b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd53f01f90 con 0x55cd53bf0f60 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53f97bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54440/0 (socket says 172.21.15.114:54440) 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 -- 172.21.15.114:0/4117686401 learned_addr learned my addr 172.21.15.114:0/4117686401 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:23.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53f97670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 -- 172.21.15.114:0/4117686401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 msgr2=0x55cd53f97670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53f97670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 -- 172.21.15.114:0/4117686401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 msgr2=0x55cd53f97bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53f97bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.565+0000 7f1902000640 1 -- 172.21.15.114:0/4117686401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd53ec8a70 con 0x55cd53bd0280 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f1902000640 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f980f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f18f0011b70 tx=0x7f18f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dd05e1b5e7b907 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f18fb400640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18e80b6670 0x7f18e80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f18efe00640 1 -- 172.21.15.114:0/4117686401 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f000fc50 con 0x55cd53bd0280 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f18efe00640 1 -- 172.21.15.114:0/4117686401 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f000fdf0 con 0x55cd53bd0280 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f18efe00640 1 -- 172.21.15.114:0/4117686401 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0003a10 con 0x55cd53bd0280 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f18fb400640 1 -- 172.21.15.114:0/2355165648 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f18fc0e7ef0 con 0x7f190410be50 2024-09-21T06:54:23.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.566+0000 7f1902a00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18e80b6670 0x7f18e80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 -- 172.21.15.114:0/4117686401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 msgr2=0x55cd53f980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f980f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f18f0011b70 tx=0x7f18f00046c0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f1902a00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18e80b6670 0x7f18e80b8b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18f4005ab0 tx=0x7f18f4005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:23.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 -- 172.21.15.114:0/4117686401 shutdown_connections 2024-09-21T06:54:23.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f980f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53f97bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 --2- 172.21.15.114:0/4117686401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53f97670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.567+0000 7f191084b740 1 -- 172.21.15.114:0/4117686401 >> 172.21.15.114:0/4117686401 conn(0x55cd53be5350 msgr2=0x55cd53f95560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:23.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.568+0000 7f191084b740 1 -- 172.21.15.114:0/4117686401 shutdown_connections 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.568+0000 7f191084b740 1 -- 172.21.15.114:0/4117686401 wait complete. 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.568+0000 7f191084b740 1 -- start start 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.568+0000 7f191084b740 1 Processor -- start 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 0x55cd53fa32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bd0280 0x55cd53fa3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53faf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cd53d5f5f0 con 0x55cd53bd0280 2024-09-21T06:54:23.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd53e99460 con 0x55cd53bc80a0 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fa5e50 con 0x55cd53bf0f60 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f191084b740 10 client.0 _ll_register_callbacks cb 0x55cd53ea9210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f1902a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 0x55cd53fa32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f1902a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 0x55cd53fa32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54446/0 (socket says 172.21.15.114:54446) 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f1902a00640 1 -- 172.21.15.114:0/4288998985 learned_addr learned my addr 172.21.15.114:0/4288998985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f1902000640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bd0280 0x55cd53fa3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.569+0000 7f1903400640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53faf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902a00640 1 -- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 msgr2=0x55cd53faf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902a00640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53faf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902a00640 1 -- 172.21.15.114:0/4288998985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bd0280 msgr2=0x55cd53fa3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902a00640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bd0280 0x55cd53fa3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902a00640 1 -- 172.21.15.114:0/4288998985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd53c5c540 con 0x55cd53bc80a0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902000640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bd0280 0x55cd53fa3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f1902a00640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 0x55cd53fa32d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009900 tx=0x7f18f4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c7348efd7e1917 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f18ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f18ecc00640 1 -- 172.21.15.114:0/4288998985 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4035040 con 0x55cd53bc80a0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f18ecc00640 1 -- 172.21.15.114:0/4288998985 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f18f40037f0 con 0x55cd53bc80a0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.570+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cd53e9ba70 con 0x55cd53bc80a0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f18ecc00640 1 -- 172.21.15.114:0/4288998985 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18f4004e50 con 0x55cd53bc80a0 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 10 client.25661 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:23.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd53eb66f0 con 0x55cd53bc80a0 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f18ecc00640 1 -- 172.21.15.114:0/4288998985 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cd53e9ba70 con 0x55cd53bc80a0 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f18ecc00640 1 -- 172.21.15.114:0/4288998985 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f18f4044030 con 0x55cd53bc80a0 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 10 client.25661 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 10 client.25661 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 20 client.25661 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53baf8e0 0x55cd53fb57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 4 client.25661 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:23.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.571+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd53f10ca0 con 0x55cd53baf8e0 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.572+0000 7f1902000640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53baf8e0 0x55cd53fb57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.572+0000 7f1902000640 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53baf8e0 0x55cd53fb57c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=58ed97f74972f5dc server_cookie=93271ca98cfb67fb in_seq=0 out_seq=0 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.572+0000 7f18ecc00640 10 client.25661 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f18ecc00640 1 -- 172.21.15.114:0/4288998985 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd53f10ca0 con 0x55cd53baf8e0 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f18ecc00640 10 client.25661 handle_command_reply: tid=0 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 client.25661 shutdown 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 msgr2=0x55cd53fa32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 0x55cd53fa32d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009900 tx=0x7f18f4009260 comp rx=0 tx=0).stop 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 shutdown_connections 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53baf8e0 0x55cd53fb57c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53faf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bd0280 0x55cd53fa3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 --2- 172.21.15.114:0/4288998985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bc80a0 0x55cd53fa32d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f18ecc00640 0 client.25661 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:23.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.573+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 >> 172.21.15.114:0/4288998985 conn(0x55cd53be5350 msgr2=0x55cd53f9a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:23.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.574+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 shutdown_connections 2024-09-21T06:54:23.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.574+0000 7f191084b740 1 -- 172.21.15.114:0/4288998985 wait complete. 2024-09-21T06:54:23.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.574+0000 7f191084b740 20 client.25661 trim_cache size 0 max 16384 2024-09-21T06:54:23.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 Processor -- start 2024-09-21T06:54:23.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 -- start start 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bc80a0 0x55cd53f9e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f9e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bf0f60 0x55cd53f9c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fb3b00 con 0x55cd53bf0f60 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fb41d0 con 0x55cd53bd0280 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f191084b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fb48a0 con 0x55cd53bc80a0 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f9e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f9e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54452/0 (socket says 172.21.15.114:54452) 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.594+0000 7f1902000640 1 -- 172.21.15.114:0/3481104616 learned_addr learned my addr 172.21.15.114:0/3481104616 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:23.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1903400640 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bf0f60 0x55cd53f9c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902a00640 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bc80a0 0x55cd53f9e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902000640 1 -- 172.21.15.114:0/3481104616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bc80a0 msgr2=0x55cd53f9e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902000640 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bc80a0 0x55cd53f9e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902000640 1 -- 172.21.15.114:0/3481104616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bf0f60 msgr2=0x55cd53f9c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902000640 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bf0f60 0x55cd53f9c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902000640 1 -- 172.21.15.114:0/3481104616 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd53fa4390 con 0x55cd53bd0280 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f1902000640 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f9e7c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18f0003660 tx=0x7f18f0001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a2d125921a10f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.595+0000 7f18ef400640 1 -- 172.21.15.114:0/3481104616 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0005230 con 0x55cd53bd0280 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.596+0000 7f18ef400640 1 -- 172.21.15.114:0/3481104616 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f0004be0 con 0x55cd53bd0280 2024-09-21T06:54:23.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.596+0000 7f18ef400640 1 -- 172.21.15.114:0/3481104616 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f001f690 con 0x55cd53bd0280 2024-09-21T06:54:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.596+0000 7f191084b740 1 -- 172.21.15.114:0/3481104616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 msgr2=0x55cd53f9e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.596+0000 7f191084b740 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f9e7c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18f0003660 tx=0x7f18f0001e50 comp rx=0 tx=0).stop 2024-09-21T06:54:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 -- 172.21.15.114:0/3481104616 shutdown_connections 2024-09-21T06:54:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bf0f60 0x55cd53f9c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53f9e7c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 --2- 172.21.15.114:0/3481104616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bc80a0 0x55cd53f9e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 -- 172.21.15.114:0/3481104616 >> 172.21.15.114:0/3481104616 conn(0x55cd53be5350 msgr2=0x55cd53fb31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 -- 172.21.15.114:0/3481104616 shutdown_connections 2024-09-21T06:54:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 -- 172.21.15.114:0/3481104616 wait complete. 2024-09-21T06:54:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 -- start start 2024-09-21T06:54:23.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.597+0000 7f191084b740 1 Processor -- start 2024-09-21T06:54:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53fbdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53fba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53fba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fb5640 con 0x55cd53bc80a0 2024-09-21T06:54:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fb4f70 con 0x55cd53bd0280 2024-09-21T06:54:23.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55cd53fb5d10 con 0x55cd53bf0f60 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f191084b740 10 client.0 _ll_register_callbacks cb 0x7f18f4005340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53fba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f1902a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53fbdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f1902000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53fba080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54468/0 (socket says 172.21.15.114:54468) 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f1902000640 1 -- 172.21.15.114:0/1359824427 learned_addr learned my addr 172.21.15.114:0/1359824427 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.598+0000 7f1903400640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53fba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1902000640 1 -- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 msgr2=0x55cd53fba5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1902000640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53fba5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1902000640 1 -- 172.21.15.114:0/1359824427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 msgr2=0x55cd53fbdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1902000640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53fbdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1902000640 1 -- 172.21.15.114:0/1359824427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd53fbab00 con 0x55cd53bd0280 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1903400640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53fba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f1902000640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53fba080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18f0011520 tx=0x7f18f0005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8917a8ba3c304bee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:23.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f18ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f18ecc00640 1 -- 172.21.15.114:0/1359824427 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0004f90 con 0x55cd53bd0280 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f0004be0 con 0x55cd53bd0280 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.599+0000 7f18ecc00640 1 -- 172.21.15.114:0/1359824427 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f18f0005130 con 0x55cd53bd0280 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f18ecc00640 1 -- 172.21.15.114:0/1359824427 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18f0043050 con 0x55cd53bd0280 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 10 client.25667 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd54020c90 con 0x55cd53bd0280 2024-09-21T06:54:23.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f18ecc00640 1 -- 172.21.15.114:0/1359824427 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18f0004be0 con 0x55cd53bd0280 2024-09-21T06:54:23.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f18ecc00640 1 -- 172.21.15.114:0/1359824427 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f18f001d020 con 0x55cd53bd0280 2024-09-21T06:54:23.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:23.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 10 client.25667 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 20 client.25667 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53b84c90 0x55cd54023df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 4 client.25667 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "status"}) -- 0x55cd53f0c750 con 0x55cd53b84c90 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.600+0000 7f1902a00640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53b84c90 0x55cd54023df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f1902a00640 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53b84c90 0x55cd54023df0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5baaedf0f75710f server_cookie=124708c822073636 in_seq=0 out_seq=0 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f18ecc00640 10 client.25667 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f18ecc00640 1 -- 172.21.15.114:0/1359824427 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cd53f0c750 con 0x55cd53b84c90 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f18ecc00640 10 client.25667 handle_command_reply: tid=0 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f191084b740 1 client.25667 shutdown 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 msgr2=0x55cd53fba080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f191084b740 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53fba080 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f18f0011520 tx=0x7f18f0005d30 comp rx=0 tx=0).stop 2024-09-21T06:54:23.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 shutdown_connections 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.601+0000 7f191084b740 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55cd53b84c90 0x55cd54023df0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55cd53bf0f60 0x55cd53fba5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd53bd0280 0x55cd53fba080 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 1 --2- 172.21.15.114:0/1359824427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55cd53bc80a0 0x55cd53fbdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f18ecc00640 0 client.25667 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 >> 172.21.15.114:0/1359824427 conn(0x55cd53be5350 msgr2=0x55cd53fa32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 shutdown_connections 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 1 -- 172.21.15.114:0/1359824427 wait complete. 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.602+0000 7f191084b740 20 client.25667 trim_cache size 0 max 16384 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:23.607 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 3, 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24727, 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 162.01410691300001, 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 89, 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 242.84828228699999 2024-09-21T06:54:23.608 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18e80b6670 msgr2=0x7f18e80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18e80b6670 0x7f18e80b8b80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18f4005ab0 tx=0x7f18f4005a20 comp rx=0 tx=0).stop 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 msgr2=0x7f19041a1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 0x7f19041a1d70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00eb10 tx=0x7f18fc00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 shutdown_connections 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f190410be50 0x7f19041a1d70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18e80b6670 0x7f18e80b8b80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f190410b500 0x7f190419cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 --2- 172.21.15.114:0/2355165648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1904105bc0 0x7f190419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:23.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 >> 172.21.15.114:0/2355165648 conn(0x7f1904101670 msgr2=0x7f190410ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:23.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 shutdown_connections 2024-09-21T06:54:23.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:23.603+0000 7f1903e00640 1 -- 172.21.15.114:0/2355165648 wait complete. 2024-09-21T06:54:24.390 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T06:54:24.775 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:24.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:24 smithi193 ceph-mon[34601]: pgmap v404: 129 pgs: 129 active+clean; 1.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-21T06:54:24.913 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:24 smithi114 ceph-mon[30654]: pgmap v404: 129 pgs: 129 active+clean; 1.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-21T06:54:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:24 smithi183 ceph-mon[34219]: pgmap v404: 129 pgs: 129 active+clean; 1.9 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-09-21T06:54:25.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 -- 172.21.15.114:0/3519996185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec076100 msgr2=0x7fdfec10fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 --2- 172.21.15.114:0/3519996185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec076100 0x7fdfec10fba0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009900 tx=0x7fdfd802a410 comp rx=0 tx=0).stop 2024-09-21T06:54:25.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 -- 172.21.15.114:0/3519996185 shutdown_connections 2024-09-21T06:54:25.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 --2- 172.21.15.114:0/3519996185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfec1100e0 0x7fdfec112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 --2- 172.21.15.114:0/3519996185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec076100 0x7fdfec10fba0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 --2- 172.21.15.114:0/3519996185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 -- 172.21.15.114:0/3519996185 >> 172.21.15.114:0/3519996185 conn(0x7fdfec0ff590 msgr2=0x7fdfec1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:25.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 -- 172.21.15.114:0/3519996185 shutdown_connections 2024-09-21T06:54:25.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.429+0000 7fdff2a00640 1 -- 172.21.15.114:0/3519996185 wait complete. 2024-09-21T06:54:25.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.431+0000 7fdff2a00640 1 Processor -- start 2024-09-21T06:54:25.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.431+0000 7fdff2a00640 1 -- start start 2024-09-21T06:54:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfec076100 0x7fdfec19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec1100e0 0x7fdfec1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec115c90 con 0x7fdfec1100e0 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec115b10 con 0x7fdfec0757b0 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfec115e10 con 0x7fdfec076100 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60062/0 (socket says 172.21.15.114:60062) 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdfebe00640 1 -- 172.21.15.114:0/2535220935 learned_addr learned my addr 172.21.15.114:0/2535220935 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdfeb400640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfec076100 0x7fdfec19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.432+0000 7fdff0a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec1100e0 0x7fdfec1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfebe00640 1 -- 172.21.15.114:0/2535220935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfec076100 msgr2=0x7fdfec19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfebe00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfec076100 0x7fdfec19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfebe00640 1 -- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec1100e0 msgr2=0x7fdfec1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfebe00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec1100e0 0x7fdfec1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfebe00640 1 -- 172.21.15.114:0/2535220935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfec1a25d0 con 0x7fdfec0757b0 2024-09-21T06:54:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfebe00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec19c660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfe8c00640 1 -- 172.21.15.114:0/2535220935 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc017030 con 0x7fdfec0757b0 2024-09-21T06:54:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfe8c00640 1 -- 172.21.15.114:0/2535220935 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc004020 con 0x7fdfec0757b0 2024-09-21T06:54:25.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdfe8c00640 1 -- 172.21.15.114:0/2535220935 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0047a0 con 0x7fdfec0757b0 2024-09-21T06:54:25.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfec1a28c0 con 0x7fdfec0757b0 2024-09-21T06:54:25.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.433+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfec1a9020 con 0x7fdfec0757b0 2024-09-21T06:54:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.436+0000 7fdff97e5740 1 Processor -- start 2024-09-21T06:54:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 -- start start 2024-09-21T06:54:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53be2f60 0x559d53f89490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bba0a0 0x559d53f899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 0x559d53f89f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559d53f00d50 con 0x559d53be2f60 2024-09-21T06:54:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d53ef0180 con 0x559d53bc2280 2024-09-21T06:54:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff97e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559d53ef3f90 con 0x559d53bba0a0 2024-09-21T06:54:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdff0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53be2f60 0x559d53f89490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdfe8c00640 1 -- 172.21.15.114:0/2535220935 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fdfdc009a30 con 0x7fdfec0757b0 2024-09-21T06:54:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.437+0000 7fdfeb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 0x559d53f89f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 0x559d53f89f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60070/0 (socket says 172.21.15.114:60070) 2024-09-21T06:54:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 -- 172.21.15.114:0/1489017615 learned_addr learned my addr 172.21.15.114:0/1489017615 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 -- 172.21.15.114:0/1489017615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bba0a0 msgr2=0x559d53f899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bba0a0 0x559d53f899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 -- 172.21.15.114:0/1489017615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53be2f60 msgr2=0x559d53f89490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53be2f60 0x559d53f89490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 -- 172.21.15.114:0/1489017615 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d53ebaa70 con 0x559d53bc2280 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.438+0000 7fdfeb400640 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 0x559d53f89f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdfec06a800 tx=0x7fdfd8032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e89c9c81616b0e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.439+0000 7fdfe1200640 1 -- 172.21.15.114:0/1489017615 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8040070 con 0x559d53bc2280 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.439+0000 7fdfe1200640 1 -- 172.21.15.114:0/1489017615 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd8004030 con 0x559d53bc2280 2024-09-21T06:54:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.439+0000 7fdfe1200640 1 -- 172.21.15.114:0/1489017615 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8035040 con 0x559d53bc2280 2024-09-21T06:54:25.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.439+0000 7fdff97e5740 1 -- 172.21.15.114:0/1489017615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 msgr2=0x559d53f89f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 0x559d53f89f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdfec06a800 tx=0x7fdfd8032bd0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 -- 172.21.15.114:0/1489017615 shutdown_connections 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bc2280 0x559d53f89f10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bba0a0 0x559d53f899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 --2- 172.21.15.114:0/1489017615 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53be2f60 0x559d53f89490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 -- 172.21.15.114:0/1489017615 >> 172.21.15.114:0/1489017615 conn(0x559d53bd7350 msgr2=0x559d53f87380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 -- 172.21.15.114:0/1489017615 shutdown_connections 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdff97e5740 1 -- 172.21.15.114:0/1489017615 wait complete. 2024-09-21T06:54:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.440+0000 7fdfe8c00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd00b6500 0x7fdfd00b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 -- start start 2024-09-21T06:54:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdfe8c00640 1 -- 172.21.15.114:0/2535220935 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfdc0e7520 con 0x7fdfec0757b0 2024-09-21T06:54:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff0a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd00b6500 0x7fdfd00b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 Processor -- start 2024-09-21T06:54:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53f95050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53f95590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff0a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd00b6500 0x7fdfd00b8a10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fa0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559d53d515f0 con 0x559d53bba0a0 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d53e8b460 con 0x559d53be2f60 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.441+0000 7fdff97e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559d53f97c10 con 0x559d53bc2280 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdff97e5740 10 client.0 _ll_register_callbacks cb 0x559d53e9b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53f95050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfeb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53f95590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdff0a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fa0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53f95050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38492/0 (socket says 172.21.15.114:38492) 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 -- 172.21.15.114:0/2105714274 learned_addr learned my addr 172.21.15.114:0/2105714274 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 -- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 msgr2=0x559d53f95590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53f95590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 -- 172.21.15.114:0/2105714274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 msgr2=0x559d53fa0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fa0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfebe00640 1 -- 172.21.15.114:0/2105714274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d53c4e540 con 0x559d53bba0a0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.442+0000 7fdfeb400640 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53f95590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdfebe00640 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53f95050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0a9630 tx=0x7fdfdc004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30b6d55fa77a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdfd5600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdfd5600640 1 -- 172.21.15.114:0/2105714274 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0ad760 con 0x559d53bba0a0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdfd5600640 1 -- 172.21.15.114:0/2105714274 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfdc0e70e0 con 0x559d53bba0a0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d53e8da70 con 0x559d53bba0a0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdfd5600640 1 -- 172.21.15.114:0/2105714274 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfdc0e72e0 con 0x559d53bba0a0 2024-09-21T06:54:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdff97e5740 10 client.15684 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.443+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d53ea86f0 con 0x559d53bba0a0 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdfd5600640 1 -- 172.21.15.114:0/2105714274 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559d53e8da70 con 0x559d53bba0a0 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdfd5600640 1 -- 172.21.15.114:0/2105714274 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdfdc05c1a0 con 0x559d53bba0a0 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdff97e5740 10 client.15684 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdff97e5740 10 client.15684 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdff97e5740 20 client.15684 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdff97e5740 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53ba18e0 0x559d53fa7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdff97e5740 4 client.15684 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.444+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d53f02ca0 con 0x559d53ba18e0 2024-09-21T06:54:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.445+0000 7fdfeb400640 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53ba18e0 0x559d53fa7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.445+0000 7fdfd5600640 10 client.15684 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.445+0000 7fdfeb400640 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53ba18e0 0x559d53fa7530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e5b9e9ae8a7fa5 server_cookie=4ff850e72f2ef066 in_seq=0 out_seq=0 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.446+0000 7fdfd5600640 1 -- 172.21.15.114:0/2105714274 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d53f02ca0 con 0x559d53ba18e0 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.446+0000 7fdfd5600640 10 client.15684 handle_command_reply: tid=0 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.446+0000 7fdff97e5740 1 client.15684 shutdown 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 msgr2=0x559d53f95050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53f95050 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0a9630 tx=0x7fdfdc004400 comp rx=0 tx=0).stop 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 shutdown_connections 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53ba18e0 0x559d53fa7530 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fa0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53f95590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 --2- 172.21.15.114:0/2105714274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53f95050 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 >> 172.21.15.114:0/2105714274 conn(0x559d53bd7350 msgr2=0x559d53f8c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:25.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdfd5600640 0 client.15684 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:25.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 shutdown_connections 2024-09-21T06:54:25.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 1 -- 172.21.15.114:0/2105714274 wait complete. 2024-09-21T06:54:25.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.447+0000 7fdff97e5740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T06:54:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.484+0000 7fdff97e5740 1 Processor -- start 2024-09-21T06:54:25.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 -- start start 2024-09-21T06:54:25.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 0x559d53f95030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bc2280 0x559d53f95570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53be2f60 0x559d53fa0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559d53fa5870 con 0x559d53bc2280 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d53fa5f40 con 0x559d53bba0a0 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.485+0000 7fdff97e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559d53fa6610 con 0x559d53be2f60 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 0x559d53f95030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 0x559d53f95030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60090/0 (socket says 172.21.15.114:60090) 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 -- 172.21.15.114:0/407237018 learned_addr learned my addr 172.21.15.114:0/407237018 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfeb400640 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bc2280 0x559d53f95570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdff0a00640 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53be2f60 0x559d53fa0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 -- 172.21.15.114:0/407237018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53be2f60 msgr2=0x559d53fa0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53be2f60 0x559d53fa0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 -- 172.21.15.114:0/407237018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bc2280 msgr2=0x559d53f95570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bc2280 0x559d53f95570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 -- 172.21.15.114:0/407237018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d53f8c2f0 con 0x559d53bba0a0 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.486+0000 7fdfebe00640 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 0x559d53f95030 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0aa2d0 tx=0x7fdfdc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=547a2113ea3c2908 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.487+0000 7fdfd7e00640 1 -- 172.21.15.114:0/407237018 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0ad760 con 0x559d53bba0a0 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.487+0000 7fdfd7e00640 1 -- 172.21.15.114:0/407237018 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc0b1b40 con 0x559d53bba0a0 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.487+0000 7fdfd7e00640 1 -- 172.21.15.114:0/407237018 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0abdc0 con 0x559d53bba0a0 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 -- 172.21.15.114:0/407237018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 msgr2=0x559d53f95030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 0x559d53f95030 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc0aa2d0 tx=0x7fdfdc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 -- 172.21.15.114:0/407237018 shutdown_connections 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53be2f60 0x559d53fa0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bc2280 0x559d53f95570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 --2- 172.21.15.114:0/407237018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53bba0a0 0x559d53f95030 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 -- 172.21.15.114:0/407237018 >> 172.21.15.114:0/407237018 conn(0x559d53bd7350 msgr2=0x559d53f82010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 -- 172.21.15.114:0/407237018 shutdown_connections 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 -- 172.21.15.114:0/407237018 wait complete. 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.488+0000 7fdff97e5740 1 -- start start 2024-09-21T06:54:25.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 Processor -- start 2024-09-21T06:54:25.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53faff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53fab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fabf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559d53fa73b0 con 0x559d53bba0a0 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d53fa6ce0 con 0x559d53be2f60 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559d53fa7a80 con 0x559d53bc2280 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff97e5740 10 client.0 _ll_register_callbacks cb 0x7fdfdc0ace30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53faff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdfebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53faff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38518/0 (socket says 172.21.15.114:38518) 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdfebe00640 1 -- 172.21.15.114:0/3044289872 learned_addr learned my addr 172.21.15.114:0/3044289872 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.489+0000 7fdff0a00640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fabf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfeb400640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53fab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfebe00640 1 -- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 msgr2=0x559d53fab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfebe00640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53fab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfebe00640 1 -- 172.21.15.114:0/3044289872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 msgr2=0x559d53fabf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfebe00640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fabf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfebe00640 1 -- 172.21.15.114:0/3044289872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d53fac600 con 0x559d53bba0a0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfebe00640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53faff80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc01e040 tx=0x7fdfdc0ac050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4099d2886333bc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfd5600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfd5600640 1 -- 172.21.15.114:0/3044289872 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0ad760 con 0x559d53bba0a0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdfd5600640 1 -- 172.21.15.114:0/3044289872 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfdc098200 con 0x559d53bba0a0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.490+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfdc0b1b40 con 0x559d53bba0a0 2024-09-21T06:54:25.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdfd5600640 1 -- 172.21.15.114:0/3044289872 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfdc0a89b0 con 0x559d53bba0a0 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 10 client.15690 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d53fac600 con 0x559d53bba0a0 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdfd5600640 1 -- 172.21.15.114:0/3044289872 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc0b1b40 con 0x559d53bba0a0 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdfd5600640 1 -- 172.21.15.114:0/3044289872 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fdfdc0940d0 con 0x559d53bba0a0 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 10 client.15690 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 10 client.15690 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 20 client.15690 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53b76c90 0x559d54014b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 4 client.15690 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.491+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "status"}) -- 0x559d53efe750 con 0x559d53b76c90 2024-09-21T06:54:25.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.492+0000 7fdfeb400640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53b76c90 0x559d54014b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:25.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.492+0000 7fdfd5600640 10 client.15690 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.492+0000 7fdfeb400640 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53b76c90 0x559d54014b70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e020f8e228e74f5d server_cookie=12d51f3080912a7d in_seq=0 out_seq=0 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdfd5600640 1 -- 172.21.15.114:0/3044289872 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559d53efe750 con 0x559d53b76c90 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdfd5600640 10 client.15690 handle_command_reply: tid=0 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 client.15690 shutdown 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 msgr2=0x559d53faff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53faff80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc01e040 tx=0x7fdfdc0ac050 comp rx=0 tx=0).stop 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 shutdown_connections 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x559d53b76c90 0x559d54014b70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d53be2f60 0x559d53fabf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559d53bc2280 0x559d53fab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 --2- 172.21.15.114:0/3044289872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559d53bba0a0 0x559d53faff80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 >> 172.21.15.114:0/3044289872 conn(0x559d53bd7350 msgr2=0x559d53f95030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdfd5600640 0 client.15690 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 shutdown_connections 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.493+0000 7fdff97e5740 1 -- 172.21.15.114:0/3044289872 wait complete. 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff97e5740 20 client.15690 trim_cache size 0 max 16384 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 2, 2024-09-21T06:54:25.500 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24742, 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 164.91552810300001, 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 89, 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 239.51763947000001 2024-09-21T06:54:25.501 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd00b6500 msgr2=0x7fdfd00b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd00b6500 0x7fdfd00b8a10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe4009500 comp rx=0 tx=0).stop 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 msgr2=0x7fdfec19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec19c660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00de10 comp rx=0 tx=0).stop 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 shutdown_connections 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfec1100e0 0x7fdfec1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfec076100 0x7fdfec19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfd00b6500 0x7fdfd00b8a10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 --2- 172.21.15.114:0/2535220935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfec0757b0 0x7fdfec19c660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:25.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.494+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 >> 172.21.15.114:0/2535220935 conn(0x7fdfec0ff590 msgr2=0x7fdfec109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:25.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.495+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 shutdown_connections 2024-09-21T06:54:25.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:25.495+0000 7fdff2a00640 1 -- 172.21.15.114:0/2535220935 wait complete. 2024-09-21T06:54:26.412 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.a status 2024-09-21T06:54:26.450 INFO:teuthology.orchestra.run.smithi114.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-21T06:54:26.455 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T06:54:26.591 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/: 2024-09-21T06:54:26.591 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.591 INFO:teuthology.orchestra.run.smithi114.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T06:54:26.591 INFO:teuthology.orchestra.run.smithi114.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 .. 2024-09-21T06:54:26.591 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 data 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data: 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:total 35 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 . 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 .. 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 base 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 06:54 global 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 log 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_commit_ts 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_dynshmem 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 06:54 pg_hba.conf 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 06:54 pg_ident.conf 2024-09-21T06:54:26.592 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 pg_logical 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 pg_multixact 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_notify 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_replslot 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_serial 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_snapshots 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_stat 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_stat_tmp 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 pg_subtrans 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_tblspc 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_twophase 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 pg_wal 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 pg_xact 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 06:54 postgresql.auto.conf 2024-09-21T06:54:26.593 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 06:54 postgresql.conf 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 . 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 1 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 13493 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 13494 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:total 7777 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 . 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 .. 2024-09-21T06:54:26.594 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 112 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 113 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 1247 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1247_fsm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1247_vm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:54 1249 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1249_fsm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1249_vm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:54 1255 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1255_fsm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1255_vm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:54 1259 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1259_fsm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1259_vm 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:54 13344 2024-09-21T06:54:26.595 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13344_fsm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13344_vm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13346 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13348 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13349_fsm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349_vm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13351 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13353 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13354_fsm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354_vm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13356 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13358 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13359_fsm 2024-09-21T06:54:26.596 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359_vm 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13361 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13363 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1417 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1418 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 174 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 175 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2187 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2224 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2328 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2336 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2337 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2579 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2600 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2600_fsm 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2600_vm 2024-09-21T06:54:26.597 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2601_fsm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601_vm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2602 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2602_fsm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2602_vm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2603 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2603_fsm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2603_vm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2604 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2605 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2605_fsm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2605_vm 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606 2024-09-21T06:54:26.598 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2606_fsm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606_vm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2607 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2607_fsm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2607_vm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:54 2608 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2608_fsm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2608_vm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:54 2609 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2609_fsm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2609_vm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2610 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2610_fsm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2610_vm 2024-09-21T06:54:26.599 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2611 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2612_fsm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612_vm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2613 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2615_fsm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615_vm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2616 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2616_fsm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2616_vm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:54 2617 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2617_fsm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2617_vm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:54 2618 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2618_fsm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2618_vm 2024-09-21T06:54:26.600 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:54 2619 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2619_fsm 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2619_vm 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2620 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2650 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2651 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2652 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2653 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2654 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2655 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2656 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2657 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:54 2658 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 2659 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2660 2024-09-21T06:54:26.601 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2661 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2662 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2663 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2664 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2665 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2666 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2667 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2668 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2669 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2670 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:54 2673 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:54 2674 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:54 2675 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2678 2024-09-21T06:54:26.602 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2679 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2680 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2681 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2682 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2683 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2684 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2685 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2686 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2687 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2688 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2689 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:54 2690 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:54 2691 2024-09-21T06:54:26.603 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2692 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2693 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2696 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2699 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2701 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2702 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2703 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2704 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2753 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2753_fsm 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2753_vm 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2754 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2755 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2756 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2757 2024-09-21T06:54:26.604 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2830 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2831 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2832 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2833 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2834 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2835 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2836 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2837 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:54 2838 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2838_fsm 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2838_vm 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2839 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840_fsm 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2840_vm 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2841 2024-09-21T06:54:26.605 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2995 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2996 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3079_fsm 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079_vm 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3080 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3081 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3085 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3118 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3119 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:54 3164 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3256 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3257 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3258 2024-09-21T06:54:26.606 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3350 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3351 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3379 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3380 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3381 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3394 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3394_fsm 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3394_vm 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3395 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3429 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3430 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3431 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3433 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3439 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3440 2024-09-21T06:54:26.607 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3455 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:54 3456 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3456_fsm 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3456_vm 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3466 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3467 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3468 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3501 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3502 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3503 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3534 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3541_fsm 2024-09-21T06:54:26.608 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541_vm 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3542 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3574 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3575 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3576 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3596 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3597 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3598 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3599 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3600_fsm 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600_vm 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3601_fsm 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601_vm 2024-09-21T06:54:26.609 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3602_fsm 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602_vm 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603_fsm 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3603_vm 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3604 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3605 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3606 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3607 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3608 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 3609 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3712 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764 2024-09-21T06:54:26.610 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3764_fsm 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764_vm 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3766 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3767 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3997 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4143 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4144 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4145 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4146 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4147 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4148 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4149 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4150 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4151 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4152 2024-09-21T06:54:26.611 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4153 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4154 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4155 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4156 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4157 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4158 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4159 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4160 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4161 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4162 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4163 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4164 2024-09-21T06:54:26.612 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4165 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4166 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4167 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4168 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4169 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4170 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4171 2024-09-21T06:54:26.613 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4172 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4173 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4174 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 5002 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 548 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 549 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6102 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6104 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6106 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6110 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6111 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6112 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6113 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6117 2024-09-21T06:54:26.614 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 826 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 827 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 828 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:total 7777 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 . 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 .. 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 112 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 113 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 1247 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1247_fsm 2024-09-21T06:54:26.615 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1247_vm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:54 1249 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1249_fsm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1249_vm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:54 1255 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1255_fsm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1255_vm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:54 1259 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1259_fsm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1259_vm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:54 13344 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13344_fsm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13344_vm 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13346 2024-09-21T06:54:26.616 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13348 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13349_fsm 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349_vm 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13351 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13353 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13354_fsm 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354_vm 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13356 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13358 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13359_fsm 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359_vm 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13361 2024-09-21T06:54:26.617 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13363 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1417 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1418 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 174 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 175 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2187 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2224 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2328 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2336 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2337 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2579 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2600 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2600_fsm 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2600_vm 2024-09-21T06:54:26.618 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2601_fsm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601_vm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2602 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2602_fsm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2602_vm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2603 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2603_fsm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2603_vm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2604 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2605 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2605_fsm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2605_vm 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606 2024-09-21T06:54:26.619 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2606_fsm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606_vm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2607 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2607_fsm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2607_vm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:54 2608 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2608_fsm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2608_vm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:54 2609 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2609_fsm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2609_vm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2610 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2610_fsm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2610_vm 2024-09-21T06:54:26.620 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2611 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2612_fsm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612_vm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2613 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2615_fsm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615_vm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2616 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2616_fsm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2616_vm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:54 2617 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2617_fsm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2617_vm 2024-09-21T06:54:26.621 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:54 2618 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2618_fsm 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2618_vm 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:54 2619 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2619_fsm 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2619_vm 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2620 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2650 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2651 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2652 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2653 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2654 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2655 2024-09-21T06:54:26.622 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2656 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2657 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:54 2658 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 2659 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2660 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2661 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2662 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2663 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2664 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2665 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2666 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2667 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2668 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2669 2024-09-21T06:54:26.623 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2670 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:54 2673 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:54 2674 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:54 2675 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2678 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2679 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2680 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2681 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2682 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2683 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2684 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2685 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2686 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2687 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2688 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2689 2024-09-21T06:54:26.624 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:54 2690 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:54 2691 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2692 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2693 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2696 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2699 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2701 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2702 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2703 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2704 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2753 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2753_fsm 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2753_vm 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2754 2024-09-21T06:54:26.625 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2755 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2756 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2757 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2830 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2831 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2832 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2833 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2834 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2835 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2836 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2837 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:54 2838 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2838_fsm 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2838_vm 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2839 2024-09-21T06:54:26.626 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840_fsm 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2840_vm 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2841 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2995 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2996 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3079_fsm 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079_vm 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3080 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3081 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3085 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3118 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3119 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:54 3164 2024-09-21T06:54:26.627 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3256 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3257 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3258 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3350 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3351 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3379 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3380 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3381 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3394 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3394_fsm 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3394_vm 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3395 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3429 2024-09-21T06:54:26.628 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3430 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3431 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3433 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3439 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3440 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3455 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:54 3456 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3456_fsm 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3456_vm 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3466 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3467 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3468 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3501 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3502 2024-09-21T06:54:26.629 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3503 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3534 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3541_fsm 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541_vm 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3542 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3574 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3575 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3576 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3596 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3597 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3598 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3599 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600 2024-09-21T06:54:26.630 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3600_fsm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600_vm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3601_fsm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601_vm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3602_fsm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602_vm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603_fsm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3603_vm 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3604 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3605 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3606 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3607 2024-09-21T06:54:26.631 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3608 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 3609 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3712 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3764_fsm 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764_vm 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3766 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3767 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3997 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4143 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4144 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4145 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4146 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4147 2024-09-21T06:54:26.632 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4148 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4149 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4150 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4151 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4152 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4153 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4154 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4155 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4156 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4157 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4158 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4159 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4160 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4161 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4162 2024-09-21T06:54:26.633 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4163 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4164 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4165 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4166 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4167 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4168 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4169 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4170 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4171 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4172 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4173 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4174 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 5002 2024-09-21T06:54:26.634 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 548 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 549 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6102 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6104 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6106 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6110 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6111 2024-09-21T06:54:26.636 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6112 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6113 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6117 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 826 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 827 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 828 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:total 7777 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 . 2024-09-21T06:54:26.637 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 .. 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 112 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 113 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 1247 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1247_fsm 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1247_vm 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:54 1249 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1249_fsm 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1249_vm 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:54 1255 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1255_fsm 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1255_vm 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:54 1259 2024-09-21T06:54:26.638 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1259_fsm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1259_vm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:54 13344 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13344_fsm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13344_vm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13346 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13348 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13349_fsm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349_vm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13351 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13353 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13354_fsm 2024-09-21T06:54:26.639 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354_vm 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13356 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13358 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13359_fsm 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359_vm 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13361 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13363 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1417 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1418 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 174 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 175 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2187 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2224 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2328 2024-09-21T06:54:26.640 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2336 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2337 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2579 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2600 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2600_fsm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2600_vm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2601_fsm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601_vm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2602 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2602_fsm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2602_vm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2603 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2603_fsm 2024-09-21T06:54:26.641 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2603_vm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2604 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2605 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2605_fsm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2605_vm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2606_fsm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606_vm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2607 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2607_fsm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2607_vm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:54 2608 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2608_fsm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2608_vm 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:54 2609 2024-09-21T06:54:26.642 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2609_fsm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2609_vm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2610 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2610_fsm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2610_vm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2611 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2612_fsm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612_vm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2613 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2615_fsm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615_vm 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2616 2024-09-21T06:54:26.643 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2616_fsm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2616_vm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:54 2617 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2617_fsm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2617_vm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:54 2618 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2618_fsm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2618_vm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:54 2619 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2619_fsm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2619_vm 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2620 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2650 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2651 2024-09-21T06:54:26.644 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2652 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2653 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2654 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2655 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2656 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2657 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:54 2658 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 2659 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2660 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2661 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2662 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2663 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2664 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2665 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2666 2024-09-21T06:54:26.645 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2667 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2668 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2669 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2670 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:54 2673 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:54 2674 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:54 2675 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2678 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2679 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2680 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2681 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2682 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2683 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2684 2024-09-21T06:54:26.646 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2685 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2686 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2687 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2688 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2689 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:54 2690 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:54 2691 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2692 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2693 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2696 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2699 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2701 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2702 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2703 2024-09-21T06:54:26.647 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2704 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2753 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2753_fsm 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2753_vm 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2754 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2755 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2756 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2757 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2830 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2831 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2832 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2833 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2834 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2835 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2836 2024-09-21T06:54:26.648 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2837 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:54 2838 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2838_fsm 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2838_vm 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2839 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840_fsm 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2840_vm 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2841 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2995 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2996 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3079_fsm 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079_vm 2024-09-21T06:54:26.649 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3080 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3081 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3085 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3118 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3119 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:54 3164 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3256 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3257 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3258 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3350 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3351 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3379 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3380 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3381 2024-09-21T06:54:26.650 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3394 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3394_fsm 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3394_vm 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3395 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3429 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3430 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3431 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3433 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3439 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3440 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3455 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:54 3456 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3456_fsm 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3456_vm 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3466 2024-09-21T06:54:26.651 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3467 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3468 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3501 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3502 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3503 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3534 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3541_fsm 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541_vm 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3542 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3574 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3575 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3576 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3596 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3597 2024-09-21T06:54:26.652 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3598 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3599 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3600_fsm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600_vm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3601_fsm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601_vm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3602_fsm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602_vm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603_fsm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3603_vm 2024-09-21T06:54:26.653 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3604 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3605 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3606 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3607 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3608 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 3609 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3712 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3764_fsm 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764_vm 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3766 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3767 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3997 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4143 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4144 2024-09-21T06:54:26.654 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4145 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4146 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4147 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4148 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4149 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4150 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4151 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4152 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4153 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4154 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4155 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4156 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4157 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4158 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4159 2024-09-21T06:54:26.655 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4160 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4161 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4162 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4163 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4164 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4165 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4166 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4167 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4168 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4169 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4170 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4171 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4172 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4173 2024-09-21T06:54:26.656 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4174 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 5002 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 548 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 549 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6102 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6104 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6106 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6110 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6111 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6112 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6113 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6117 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 826 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 827 2024-09-21T06:54:26.657 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 828 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:total 537 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 21 06:54 . 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1213 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1213_fsm 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1213_vm 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1214 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1214_fsm 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1214_vm 2024-09-21T06:54:26.658 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 1232 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 1233 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1260 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1260_fsm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1260_vm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1261 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1261_fsm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1261_vm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1262 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1262_fsm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1262_vm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2396 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2396_fsm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2396_vm 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2397 2024-09-21T06:54:26.659 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2671 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2672 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2676 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2677 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2694 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2695 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2697 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2698 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2846 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2847 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2964 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2965 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2966 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2967 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3592 2024-09-21T06:54:26.660 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3593 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4060 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4061 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4175 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4176 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4177 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4178 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4181 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4182 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4183 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4184 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4185 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4186 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6000 2024-09-21T06:54:26.661 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6001 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6002 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6100 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6114 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6115 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 pg_control 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.662 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:total 1 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 . 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.663 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 mappings 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 06:54 replorigin_checkpoint 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 snapshots 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 .. 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 .. 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.664 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 . 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 members 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 offsets 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:total 8 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 .. 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 0000 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.665 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:total 8 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 .. 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 0000 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.666 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.667 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:total 8 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 0000 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.668 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:total 16384 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 . 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:54 000000010000000000000001 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 archive_status 2024-09-21T06:54:26.669 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 .. 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:total 8 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 21 06:54 .. 2024-09-21T06:54:26.670 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 0000 2024-09-21T06:54:26.670 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-21T06:54:26.811 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:26.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:26 smithi193 ceph-mon[34601]: pgmap v405: 129 pgs: 129 active+clean; 8.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 653 KiB/s wr, 24 op/s 2024-09-21T06:54:26.913 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:26 smithi114 ceph-mon[30654]: pgmap v405: 129 pgs: 129 active+clean; 8.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 653 KiB/s wr, 24 op/s 2024-09-21T06:54:26.920 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-21T06:54:27.011 INFO:teuthology.orchestra.run.smithi114.stderr:dropping old tables... 2024-09-21T06:54:27.012 INFO:teuthology.orchestra.run.smithi114.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-21T06:54:27.012 INFO:teuthology.orchestra.run.smithi114.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-21T06:54:27.012 INFO:teuthology.orchestra.run.smithi114.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-21T06:54:27.012 INFO:teuthology.orchestra.run.smithi114.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-21T06:54:27.012 INFO:teuthology.orchestra.run.smithi114.stderr:creating tables... 2024-09-21T06:54:27.028 INFO:teuthology.orchestra.run.smithi114.stderr:generating data (client-side)... 2024-09-21T06:54:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:26 smithi183 ceph-mon[34219]: pgmap v405: 129 pgs: 129 active+clean; 8.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 653 KiB/s wr, 24 op/s 2024-09-21T06:54:27.184 INFO:teuthology.orchestra.run.smithi114.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.13 s, remaining 3.96 s) 2024-09-21T06:54:27.461 INFO:teuthology.orchestra.run.smithi114.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.40 s, remaining 6.07 s) 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 -- 172.21.15.114:0/1870341837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370114280 msgr2=0x7f9370116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 --2- 172.21.15.114:0/1870341837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370114280 0x7f9370116750 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9368009be0 tx=0x7f936802a6c0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 -- 172.21.15.114:0/1870341837 shutdown_connections 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 --2- 172.21.15.114:0/1870341837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370114280 0x7f9370116750 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 --2- 172.21.15.114:0/1870341837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9370075dd0 0x7f93700761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 --2- 172.21.15.114:0/1870341837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370077380 0x7f93700757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.523+0000 7f936f200640 1 -- 172.21.15.114:0/1870341837 >> 172.21.15.114:0/1870341837 conn(0x7f93700ff440 msgr2=0x7f9370101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.524+0000 7f936f200640 1 -- 172.21.15.114:0/1870341837 shutdown_connections 2024-09-21T06:54:27.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.524+0000 7f936f200640 1 -- 172.21.15.114:0/1870341837 wait complete. 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 Processor -- start 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 -- start start 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 0x7f93701a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 0x7f93701a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9370114280 0x7f93701a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9370119ec0 con 0x7f9370077380 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9370119d40 con 0x7f9370075dd0 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.525+0000 7f936f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f937011a040 con 0x7f9370114280 2024-09-21T06:54:27.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 0x7f93701a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 0x7f93701a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 0x7f93701a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60124/0 (socket says 172.21.15.114:60124) 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 0x7f93701a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38528/0 (socket says 172.21.15.114:38528) 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 -- 172.21.15.114:0/106465313 learned_addr learned my addr 172.21.15.114:0/106465313 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936e800640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9370114280 0x7f93701a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 -- 172.21.15.114:0/106465313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9370114280 msgr2=0x7f93701a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9370114280 0x7f93701a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 -- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 msgr2=0x7f93701a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 0x7f93701a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936de00640 1 -- 172.21.15.114:0/106465313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93701a67e0 con 0x7f9370075dd0 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.526+0000 7f936d400640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 0x7f93701a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.527+0000 7f936de00640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 0x7f93701a0870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f936000e9c0 tx=0x7f936000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.527+0000 7f9366a00640 1 -- 172.21.15.114:0/106465313 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936000cde0 con 0x7f9370075dd0 2024-09-21T06:54:27.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.527+0000 7f9366a00640 1 -- 172.21.15.114:0/106465313 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93600035f0 con 0x7f9370075dd0 2024-09-21T06:54:27.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.527+0000 7f9366a00640 1 -- 172.21.15.114:0/106465313 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360010690 con 0x7f9370075dd0 2024-09-21T06:54:27.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.527+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93701a6ad0 con 0x7f9370075dd0 2024-09-21T06:54:27.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.528+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93701ad230 con 0x7f9370075dd0 2024-09-21T06:54:27.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.532+0000 7f937bc47740 1 Processor -- start 2024-09-21T06:54:27.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.532+0000 7f937bc47740 1 -- start start 2024-09-21T06:54:27.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f937bc47740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4971f60 0x5569e4d18670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f937bc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d18bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f937bc47740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f937bc47740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569e4c8fd50 con 0x5569e4951280 2024-09-21T06:54:27.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f937bc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569e4c7f180 con 0x5569e49490a0 2024-09-21T06:54:27.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f937bc47740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569e4c82f90 con 0x5569e4971f60 2024-09-21T06:54:27.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f9366a00640 1 -- 172.21.15.114:0/106465313 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f9360004000 con 0x7f9370075dd0 2024-09-21T06:54:27.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f936de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d18bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f936de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d18bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60136/0 (socket says 172.21.15.114:60136) 2024-09-21T06:54:27.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.533+0000 7f936de00640 1 -- 172.21.15.114:0/408134003 learned_addr learned my addr 172.21.15.114:0/408134003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936e800640 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4971f60 0x5569e4d18670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936de00640 1 -- 172.21.15.114:0/408134003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4971f60 msgr2=0x5569e4d18670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936de00640 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4971f60 0x5569e4d18670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936de00640 1 -- 172.21.15.114:0/408134003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 msgr2=0x5569e4d190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936de00640 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936de00640 1 -- 172.21.15.114:0/408134003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e4c49a70 con 0x5569e49490a0 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f936de00640 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d18bb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93600ebbf0 tx=0x7f93600f0730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7571308fde937a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.534+0000 7f935b400640 1 -- 172.21.15.114:0/408134003 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600f6070 con 0x5569e49490a0 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.535+0000 7f935b400640 1 -- 172.21.15.114:0/408134003 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93600f2af0 con 0x5569e49490a0 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.535+0000 7f935b400640 1 -- 172.21.15.114:0/408134003 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600f1a80 con 0x5569e49490a0 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 -- 172.21.15.114:0/408134003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 msgr2=0x5569e4d18bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d18bb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93600ebbf0 tx=0x7f93600f0730 comp rx=0 tx=0).stop 2024-09-21T06:54:27.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 -- 172.21.15.114:0/408134003 shutdown_connections 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d18bb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 --2- 172.21.15.114:0/408134003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4971f60 0x5569e4d18670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 -- 172.21.15.114:0/408134003 >> 172.21.15.114:0/408134003 conn(0x5569e4966350 msgr2=0x5569e4d16560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f9366a00640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93540b66f0 0x7f93540b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f937bc47740 1 -- 172.21.15.114:0/408134003 shutdown_connections 2024-09-21T06:54:27.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.536+0000 7f9366a00640 1 -- 172.21.15.114:0/106465313 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f93600e7c90 con 0x7f9370075dd0 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.537+0000 7f936d400640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93540b66f0 0x7f93540b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.537+0000 7f937bc47740 1 -- 172.21.15.114:0/408134003 wait complete. 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.538+0000 7f937bc47740 1 -- start start 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 Processor -- start 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f936d400640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93540b66f0 0x7f93540b8c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f93701161a0 tx=0x7f935c0046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 0x5569e4d241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d30170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569e4ae05f0 con 0x5569e4951280 2024-09-21T06:54:27.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569e4c1a460 con 0x5569e4971f60 2024-09-21T06:54:27.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d26e50 con 0x5569e49490a0 2024-09-21T06:54:27.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.539+0000 7f937bc47740 10 client.0 _ll_register_callbacks cb 0x5569e4c2a210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:27.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 0x5569e4d241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 0x5569e4d241b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52018/0 (socket says 172.21.15.114:52018) 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 -- 172.21.15.114:0/2891169909 learned_addr learned my addr 172.21.15.114:0/2891169909 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936d400640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936e800640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d30170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 -- 172.21.15.114:0/2891169909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 msgr2=0x5569e4d30170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d30170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 -- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 msgr2=0x5569e4d246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936de00640 1 -- 172.21.15.114:0/2891169909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e49dd540 con 0x5569e49490a0 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.540+0000 7f936d400640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.541+0000 7f936de00640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 0x5569e4d241b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f93600f0fa0 tx=0x7f9360003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f0c01d73e74c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:27.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.541+0000 7f934fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.541+0000 7f934fe00640 1 -- 172.21.15.114:0/2891169909 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600f6030 con 0x5569e49490a0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.541+0000 7f934fe00640 1 -- 172.21.15.114:0/2891169909 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f93600094f0 con 0x5569e49490a0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.541+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5569e4c1ca70 con 0x5569e49490a0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.542+0000 7f934fe00640 1 -- 172.21.15.114:0/2891169909 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93600a9e20 con 0x5569e49490a0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.542+0000 7f937bc47740 10 client.25699 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.542+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569e4c376f0 con 0x5569e49490a0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.542+0000 7f934fe00640 1 -- 172.21.15.114:0/2891169909 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5569e4c1ca70 con 0x5569e49490a0 2024-09-21T06:54:27.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.542+0000 7f934fe00640 1 -- 172.21.15.114:0/2891169909 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f93600a9a40 con 0x5569e49490a0 2024-09-21T06:54:27.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f937bc47740 10 client.25699 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:27.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f937bc47740 10 client.25699 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:54:27.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f937bc47740 20 client.25699 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f937bc47740 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e49308e0 0x5569e4d360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f937bc47740 4 client.25699 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569e4c91ca0 con 0x5569e49308e0 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.543+0000 7f936d400640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e49308e0 0x5569e4d360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.544+0000 7f936d400640 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e49308e0 0x5569e4d360f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=627391ce19063cc7 server_cookie=98f9df311572ff9f in_seq=0 out_seq=0 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.544+0000 7f934fe00640 10 client.25699 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.545+0000 7f934fe00640 1 -- 172.21.15.114:0/2891169909 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5569e4c91ca0 con 0x5569e49308e0 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.545+0000 7f934fe00640 10 client.25699 handle_command_reply: tid=0 2024-09-21T06:54:27.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.545+0000 7f937bc47740 1 client.25699 shutdown 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.545+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 msgr2=0x5569e4d241b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.545+0000 7f937bc47740 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 0x5569e4d241b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f93600f0fa0 tx=0x7f9360003cd0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.545+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 shutdown_connections 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e49308e0 0x5569e4d360f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d30170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4951280 0x5569e4d246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 --2- 172.21.15.114:0/2891169909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e49490a0 0x5569e4d241b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 >> 172.21.15.114:0/2891169909 conn(0x5569e4966350 msgr2=0x5569e4d1b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:27.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f934fe00640 0 client.25699 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:27.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 shutdown_connections 2024-09-21T06:54:27.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.546+0000 7f937bc47740 1 -- 172.21.15.114:0/2891169909 wait complete. 2024-09-21T06:54:27.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.547+0000 7f937bc47740 20 client.25699 trim_cache size 0 max 16384 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 Processor -- start 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 -- start start 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d34c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4971f60 0x5569e4d16500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d34430 con 0x5569e4971f60 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d34b00 con 0x5569e49490a0 2024-09-21T06:54:27.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.586+0000 7f937bc47740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d351d0 con 0x5569e4951280 2024-09-21T06:54:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4971f60 0x5569e4d16500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d34c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d34c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60156/0 (socket says 172.21.15.114:60156) 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 -- 172.21.15.114:0/1214804279 learned_addr learned my addr 172.21.15.114:0/1214804279 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936e800640 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4971f60 0x5569e4d16500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38542/0 (socket says 172.21.15.114:38542) 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936d400640 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 -- 172.21.15.114:0/1214804279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 msgr2=0x5569e4d112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 -- 172.21.15.114:0/1214804279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4971f60 msgr2=0x5569e4d16500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4971f60 0x5569e4d16500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 -- 172.21.15.114:0/1214804279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e4d1f860 con 0x5569e49490a0 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.587+0000 7f936de00640 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d34c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f93600f1650 tx=0x7f93600f0730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed9fb8976bc80bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.588+0000 7f935aa00640 1 -- 172.21.15.114:0/1214804279 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600f6070 con 0x5569e49490a0 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.588+0000 7f935aa00640 1 -- 172.21.15.114:0/1214804279 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93600ad2e0 con 0x5569e49490a0 2024-09-21T06:54:27.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.588+0000 7f935aa00640 1 -- 172.21.15.114:0/1214804279 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360098070 con 0x5569e49490a0 2024-09-21T06:54:27.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 -- 172.21.15.114:0/1214804279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 msgr2=0x5569e4d34c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d34c80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f93600f1650 tx=0x7f93600f0730 comp rx=0 tx=0).stop 2024-09-21T06:54:27.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 -- 172.21.15.114:0/1214804279 shutdown_connections 2024-09-21T06:54:27.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e4971f60 0x5569e4d16500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 --2- 172.21.15.114:0/1214804279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e49490a0 0x5569e4d34c80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 -- 172.21.15.114:0/1214804279 >> 172.21.15.114:0/1214804279 conn(0x5569e4966350 msgr2=0x5569e4d15df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:27.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 -- 172.21.15.114:0/1214804279 shutdown_connections 2024-09-21T06:54:27.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.589+0000 7f937bc47740 1 -- 172.21.15.114:0/1214804279 wait complete. 2024-09-21T06:54:27.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.590+0000 7f937bc47740 1 -- start start 2024-09-21T06:54:27.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.590+0000 7f937bc47740 1 Processor -- start 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.590+0000 7f937bc47740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e49490a0 0x5569e4d3eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.590+0000 7f937bc47740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d37fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f937bc47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d38520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f937bc47740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d35f70 con 0x5569e49490a0 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f937bc47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d358a0 con 0x5569e4971f60 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f937bc47740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569e4d36640 con 0x5569e4951280 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f937bc47740 10 client.0 _ll_register_callbacks cb 0x7f9360003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d37fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d37fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52040/0 (socket says 172.21.15.114:52040) 2024-09-21T06:54:27.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936d400640 1 -- 172.21.15.114:0/2274669510 learned_addr learned my addr 172.21.15.114:0/2274669510 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:27.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936de00640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e49490a0 0x5569e4d3eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936e800640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d38520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936d400640 1 -- 172.21.15.114:0/2274669510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 msgr2=0x5569e4d38520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.591+0000 7f936d400640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d38520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f936d400640 1 -- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e49490a0 msgr2=0x5569e4d3eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f936d400640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e49490a0 0x5569e4d3eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f936d400640 1 -- 172.21.15.114:0/2274669510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e4d38c60 con 0x5569e4951280 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f936d400640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d37fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5569e4d25150 tx=0x7f935c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95efed411dee034 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f934fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f934fe00640 1 -- 172.21.15.114:0/2274669510 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c034060 con 0x5569e4951280 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f934fe00640 1 -- 172.21.15.114:0/2274669510 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f935c01f840 con 0x5569e4951280 2024-09-21T06:54:27.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.592+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93600ad2e0 con 0x5569e4951280 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f934fe00640 1 -- 172.21.15.114:0/2274669510 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f935c03b050 con 0x5569e4951280 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 10 client.25705 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5569e4da15c0 con 0x5569e4951280 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f934fe00640 1 -- 172.21.15.114:0/2274669510 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93600ad2e0 con 0x5569e4951280 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f934fe00640 1 -- 172.21.15.114:0/2274669510 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f935c02fb70 con 0x5569e4951280 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 10 client.25705 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 10 client.25705 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 20 client.25705 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e4905c90 0x5569e4da4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 4 client.25705 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:54:27.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.593+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "status"}) -- 0x5569e4c8d750 con 0x5569e4905c90 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.594+0000 7f936de00640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e4905c90 0x5569e4da4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.594+0000 7f936de00640 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e4905c90 0x5569e4da4e40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee98c8e326fa017e server_cookie=7fe3e4b515eac865 in_seq=0 out_seq=0 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.595+0000 7f934fe00640 10 client.25705 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.595+0000 7f934fe00640 1 -- 172.21.15.114:0/2274669510 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5569e4c8d750 con 0x5569e4905c90 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.595+0000 7f934fe00640 10 client.25705 handle_command_reply: tid=0 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.595+0000 7f937bc47740 1 client.25705 shutdown 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.595+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 msgr2=0x5569e4d37fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.595+0000 7f937bc47740 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d37fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x5569e4d25150 tx=0x7f935c005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:27.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 shutdown_connections 2024-09-21T06:54:27.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5569e4905c90 0x5569e4da4e40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569e4971f60 0x5569e4d38520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569e4951280 0x5569e4d37fe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 --2- 172.21.15.114:0/2274669510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569e49490a0 0x5569e4d3eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f934fe00640 0 client.25705 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 >> 172.21.15.114:0/2274669510 conn(0x5569e4966350 msgr2=0x5569e4d1b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 shutdown_connections 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.596+0000 7f937bc47740 1 -- 172.21.15.114:0/2274669510 wait complete. 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.597+0000 7f937bc47740 20 client.25705 trim_cache size 0 max 16384 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 1, 2024-09-21T06:54:27.600 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24760, 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 168.028906326, 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 89, 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 234.71288147300001 2024-09-21T06:54:27.601 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.597+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93540b66f0 msgr2=0x7f93540b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.597+0000 7f936f200640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93540b66f0 0x7f93540b8c00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f93701161a0 tx=0x7f935c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.597+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 msgr2=0x7f93701a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.597+0000 7f936f200640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 0x7f93701a0870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f936000e9c0 tx=0x7f936000ee80 comp rx=0 tx=0).stop 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 shutdown_connections 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9370114280 0x7f93701a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9370077380 0x7f93701a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f93540b66f0 0x7f93540b8c00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 --2- 172.21.15.114:0/106465313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9370075dd0 0x7f93701a0870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 >> 172.21.15.114:0/106465313 conn(0x7f93700ff440 msgr2=0x7f9370110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:27.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 shutdown_connections 2024-09-21T06:54:27.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:27.598+0000 7f936f200640 1 -- 172.21.15.114:0/106465313 wait complete. 2024-09-21T06:54:27.751 INFO:teuthology.orchestra.run.smithi114.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.69 s, remaining 6.72 s) 2024-09-21T06:54:27.889 INFO:teuthology.orchestra.run.smithi114.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.83 s, remaining 5.83 s) 2024-09-21T06:54:28.103 INFO:teuthology.orchestra.run.smithi114.stderr:500000 of 3200000 tuples (15%) done (elapsed 1.05 s, remaining 5.65 s) 2024-09-21T06:54:28.278 INFO:teuthology.orchestra.run.smithi114.stderr:600000 of 3200000 tuples (18%) done (elapsed 1.22 s, remaining 5.29 s) 2024-09-21T06:54:28.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T06:54:28.594 INFO:teuthology.orchestra.run.smithi114.stderr:700000 of 3200000 tuples (21%) done (elapsed 1.54 s, remaining 5.49 s) 2024-09-21T06:54:28.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:28 smithi114 ceph-mon[30654]: pgmap v406: 129 pgs: 129 active+clean; 29 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 70 op/s 2024-09-21T06:54:28.905 INFO:teuthology.orchestra.run.smithi114.stderr:800000 of 3200000 tuples (25%) done (elapsed 1.85 s, remaining 5.55 s) 2024-09-21T06:54:28.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:28 smithi193 ceph-mon[34601]: pgmap v406: 129 pgs: 129 active+clean; 29 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 70 op/s 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 -- 172.21.15.114:0/2541130747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b8076100 msgr2=0x7f27b810faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 --2- 172.21.15.114:0/2541130747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b8076100 0x7f27b810faf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009860 tx=0x7f27a402a340 comp rx=0 tx=0).stop 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 -- 172.21.15.114:0/2541130747 shutdown_connections 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 --2- 172.21.15.114:0/2541130747 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 0x7f27b8112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 --2- 172.21.15.114:0/2541130747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b8076100 0x7f27b810faf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 --2- 172.21.15.114:0/2541130747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b80757b0 0x7f27b8075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 -- 172.21.15.114:0/2541130747 >> 172.21.15.114:0/2541130747 conn(0x7f27b80ff520 msgr2=0x7f27b8101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 -- 172.21.15.114:0/2541130747 shutdown_connections 2024-09-21T06:54:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.931+0000 7f27c0400640 1 -- 172.21.15.114:0/2541130747 wait complete. 2024-09-21T06:54:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.932+0000 7f27c0400640 1 Processor -- start 2024-09-21T06:54:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.932+0000 7f27c0400640 1 -- start start 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27c0400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 0x7f27b819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27c0400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 0x7f27b819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27c0400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 0x7f27b81a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27c0400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8115bd0 con 0x7f27b8110030 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27c0400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8115a50 con 0x7f27b80757b0 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27c0400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8115d50 con 0x7f27b8076100 2024-09-21T06:54:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27bd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 0x7f27b819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27bda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 0x7f27b819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27bd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 0x7f27b819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52066/0 (socket says 172.21.15.114:52066) 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27bda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 0x7f27b819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60182/0 (socket says 172.21.15.114:60182) 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27bd000640 1 -- 172.21.15.114:0/543187226 learned_addr learned my addr 172.21.15.114:0/543187226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.933+0000 7f27be400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 0x7f27b81a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bd000640 1 -- 172.21.15.114:0/543187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 msgr2=0x7f27b819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bd000640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 0x7f27b819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bd000640 1 -- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 msgr2=0x7f27b81a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bd000640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 0x7f27b81a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bd000640 1 -- 172.21.15.114:0/543187226 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b81a25e0 con 0x7f27b8076100 2024-09-21T06:54:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bda00640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 0x7f27b819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27be400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 0x7f27b81a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27bd000640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 0x7f27b819cc50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27a4035040 tx=0x7f27a4032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27b6000640 1 -- 172.21.15.114:0/543187226 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4040020 con 0x7f27b8076100 2024-09-21T06:54:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27b6000640 1 -- 172.21.15.114:0/543187226 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a4030e10 con 0x7f27b8076100 2024-09-21T06:54:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27b6000640 1 -- 172.21.15.114:0/543187226 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4037040 con 0x7f27b8076100 2024-09-21T06:54:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b81a2870 con 0x7f27b8076100 2024-09-21T06:54:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.934+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a9060 con 0x7f27b8076100 2024-09-21T06:54:28.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.938+0000 7f27c7461740 1 Processor -- start 2024-09-21T06:54:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.938+0000 7f27c7461740 1 -- start start 2024-09-21T06:54:28.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27c7461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff61dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27c7461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff61e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27b6000640 1 -- 172.21.15.114:0/543187226 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f27a4043020 con 0x7f27b8076100 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27c7461740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff61e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27c7461740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575ff5936b0 con 0x5575ff2584d0 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27c7461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575ff582ae0 con 0x5575ff2791b0 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27c7461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575ff5868f0 con 0x5575ff2502f0 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27be400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff61dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27bd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff61e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27be400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff61dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60196/0 (socket says 172.21.15.114:60196) 2024-09-21T06:54:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27bd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff61e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38578/0 (socket says 172.21.15.114:38578) 2024-09-21T06:54:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27be400640 1 -- 172.21.15.114:0/2099707740 learned_addr learned my addr 172.21.15.114:0/2099707740 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.939+0000 7f27bda00640 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff61e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.940+0000 7f27be400640 1 -- 172.21.15.114:0/2099707740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 msgr2=0x5575ff61e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.940+0000 7f27be400640 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff61e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.940+0000 7f27be400640 1 -- 172.21.15.114:0/2099707740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 msgr2=0x5575ff61e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.940+0000 7f27be400640 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff61e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.940+0000 7f27be400640 1 -- 172.21.15.114:0/2099707740 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ff54f780 con 0x5575ff2791b0 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.940+0000 7f27be400640 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff61dc30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27b000f400 tx=0x7f27b000f8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37672fb1d13c9b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.941+0000 7f27aaa00640 1 -- 172.21.15.114:0/2099707740 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0003970 con 0x5575ff2791b0 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.941+0000 7f27aaa00640 1 -- 172.21.15.114:0/2099707740 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b0004020 con 0x5575ff2791b0 2024-09-21T06:54:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.941+0000 7f27aaa00640 1 -- 172.21.15.114:0/2099707740 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b000c970 con 0x5575ff2791b0 2024-09-21T06:54:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.941+0000 7f27b6000640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27a00b65f0 0x7f27a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.941+0000 7f27b6000640 1 -- 172.21.15.114:0/543187226 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f27a4107560 con 0x7f27b8076100 2024-09-21T06:54:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.941+0000 7f27bda00640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27a00b65f0 0x7f27a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 -- 172.21.15.114:0/2099707740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 msgr2=0x5575ff61dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff61dc30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27b000f400 tx=0x7f27b000f8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 -- 172.21.15.114:0/2099707740 shutdown_connections 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff61e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff61e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 --2- 172.21.15.114:0/2099707740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff61dc30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 -- 172.21.15.114:0/2099707740 >> 172.21.15.114:0/2099707740 conn(0x5575ff26d300 msgr2=0x5575ff61bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 -- 172.21.15.114:0/2099707740 shutdown_connections 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27bda00640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27a00b65f0 0x7f27a00b8b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5575ff61c8b0 tx=0x7f27ac005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.942+0000 7f27c7461740 1 -- 172.21.15.114:0/2099707740 wait complete. 2024-09-21T06:54:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.943+0000 7f27c7461740 1 -- start start 2024-09-21T06:54:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.943+0000 7f27c7461740 1 Processor -- start 2024-09-21T06:54:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff6292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff629830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6356d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575ff4baab0 con 0x5575ff2584d0 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575ff529570 con 0x5575ff2791b0 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575ff62c3b0 con 0x5575ff2502f0 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.944+0000 7f27c7461740 10 client.0 _ll_register_callbacks cb 0x5575ff538100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff6292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff6292f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52084/0 (socket says 172.21.15.114:52084) 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 -- 172.21.15.114:0/137493123 learned_addr learned my addr 172.21.15.114:0/137493123 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bd000640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff629830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27be400640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 -- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 msgr2=0x5575ff6356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 -- 172.21.15.114:0/137493123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 msgr2=0x5575ff629830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff629830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.945+0000 7f27bda00640 1 -- 172.21.15.114:0/137493123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ff2e4790 con 0x5575ff2502f0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f27bda00640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff6292f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009f70 tx=0x7f27ac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6461e11d9a7eda3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f279f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f279f400640 1 -- 172.21.15.114:0/137493123 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00f8b0 con 0x5575ff2502f0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f279f400640 1 -- 172.21.15.114:0/137493123 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f27ac00fa50 con 0x5575ff2502f0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b0004020 con 0x5575ff2502f0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f279f400640 1 -- 172.21.15.114:0/137493123 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27ac00fc50 con 0x5575ff2502f0 2024-09-21T06:54:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f27c7461740 10 client.25717 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.946+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575ff35d620 con 0x5575ff2502f0 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f279f400640 1 -- 172.21.15.114:0/137493123 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b0004020 con 0x5575ff2502f0 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f279f400640 1 -- 172.21.15.114:0/137493123 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f27ac027070 con 0x5575ff2502f0 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27c7461740 10 client.25717 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27c7461740 10 client.25717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27c7461740 20 client.25717 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27c7461740 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff237d80 0x5575ff63b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27c7461740 4 client.25717 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ff595600 con 0x5575ff237d80 2024-09-21T06:54:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.947+0000 7f27bd000640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff237d80 0x5575ff63b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.948+0000 7f27bd000640 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff237d80 0x5575ff63b4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d3cd75495d3ae0 server_cookie=386dce0d802f3595 in_seq=0 out_seq=0 2024-09-21T06:54:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.948+0000 7f279f400640 10 client.25717 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.948+0000 7f279f400640 1 -- 172.21.15.114:0/137493123 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575ff595600 con 0x5575ff237d80 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.948+0000 7f279f400640 10 client.25717 handle_command_reply: tid=0 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 client.25717 shutdown 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 msgr2=0x5575ff6292f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff6292f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009f70 tx=0x7f27ac004730 comp rx=0 tx=0).stop 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 shutdown_connections 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2584d0 0x5575ff629830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff237d80 0x5575ff63b4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 --2- 172.21.15.114:0/137493123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2502f0 0x5575ff6292f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f279f400640 0 client.25717 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 >> 172.21.15.114:0/137493123 conn(0x5575ff26d300 msgr2=0x5575ff61ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 shutdown_connections 2024-09-21T06:54:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 1 -- 172.21.15.114:0/137493123 wait complete. 2024-09-21T06:54:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.949+0000 7f27c7461740 20 client.25717 trim_cache size 0 max 16384 2024-09-21T06:54:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.965+0000 7f27c7461740 1 Processor -- start 2024-09-21T06:54:28.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.965+0000 7f27c7461740 1 -- start start 2024-09-21T06:54:28.968 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27c7461740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff61f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27c7461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff6264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27c7461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff626a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27c7461740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81b3070 con 0x5575ff2502f0 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27c7461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81b4bb0 con 0x5575ff2791b0 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27c7461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81b3740 con 0x5575ff2584d0 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff61f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff6264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff61f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38596/0 (socket says 172.21.15.114:38596) 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff6264c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52086/0 (socket says 172.21.15.114:52086) 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bda00640 1 -- 172.21.15.114:0/2132033984 learned_addr learned my addr 172.21.15.114:0/2132033984 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27be400640 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff626a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bd000640 1 -- 172.21.15.114:0/2132033984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 msgr2=0x5575ff626a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bd000640 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff626a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.966+0000 7f27bd000640 1 -- 172.21.15.114:0/2132033984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 msgr2=0x5575ff61f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.967+0000 7f27bd000640 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff61f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.967+0000 7f27bd000640 1 -- 172.21.15.114:0/2132033984 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ff620a40 con 0x5575ff2584d0 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.967+0000 7f27bd000640 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff6264c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27a4104a80 tx=0x7f27a410d110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1f6d99362da68f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.967+0000 7f27aa000640 1 -- 172.21.15.114:0/2132033984 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40c8fa0 con 0x5575ff2584d0 2024-09-21T06:54:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.967+0000 7f27aa000640 1 -- 172.21.15.114:0/2132033984 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a40ccc10 con 0x5575ff2584d0 2024-09-21T06:54:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.967+0000 7f27aa000640 1 -- 172.21.15.114:0/2132033984 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40cc870 con 0x5575ff2584d0 2024-09-21T06:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 -- 172.21.15.114:0/2132033984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 msgr2=0x5575ff6264c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff6264c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f27a4104a80 tx=0x7f27a410d110 comp rx=0 tx=0).stop 2024-09-21T06:54:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 -- 172.21.15.114:0/2132033984 shutdown_connections 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff626a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff6264c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 --2- 172.21.15.114:0/2132033984 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff61f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 -- 172.21.15.114:0/2132033984 >> 172.21.15.114:0/2132033984 conn(0x5575ff26d300 msgr2=0x5575ff61ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 -- 172.21.15.114:0/2132033984 shutdown_connections 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.968+0000 7f27c7461740 1 -- 172.21.15.114:0/2132033984 wait complete. 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.969+0000 7f27c7461740 1 -- start start 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.969+0000 7f27c7461740 1 Processor -- start 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff62fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff62ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81b44e0 con 0x5575ff2502f0 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81b3e10 con 0x5575ff2791b0 2024-09-21T06:54:28.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81b1050 con 0x5575ff2584d0 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27c7461740 10 client.0 _ll_register_callbacks cb 0x7f27ac009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27bda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff62fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27bda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff62fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38610/0 (socket says 172.21.15.114:38610) 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27bda00640 1 -- 172.21.15.114:0/4079939067 learned_addr learned my addr 172.21.15.114:0/4079939067 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60230/0 (socket says 172.21.15.114:60230) 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27bd000640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff62ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 -- 172.21.15.114:0/4079939067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 msgr2=0x5575ff62ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff62ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 -- 172.21.15.114:0/4079939067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 msgr2=0x5575ff62fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff62fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27be400640 1 -- 172.21.15.114:0/4079939067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ff6a5420 con 0x5575ff2791b0 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.970+0000 7f27bd000640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff62ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.971+0000 7f27be400640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6a4ee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009820 tx=0x7f27b0004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ebc5159403bc68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.971+0000 7f279f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.971+0000 7f279f400640 1 -- 172.21.15.114:0/4079939067 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b000c9f0 con 0x5575ff2791b0 2024-09-21T06:54:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.971+0000 7f279f400640 1 -- 172.21.15.114:0/4079939067 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f27b000cb90 con 0x5575ff2791b0 2024-09-21T06:54:28.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.971+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a40ccc10 con 0x5575ff2791b0 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.971+0000 7f279f400640 1 -- 172.21.15.114:0/4079939067 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27b0033050 con 0x5575ff2791b0 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 10 client.25721 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575ff6a6500 con 0x5575ff2791b0 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f279f400640 1 -- 172.21.15.114:0/4079939067 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a40ccc10 con 0x5575ff2791b0 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f279f400640 1 -- 172.21.15.114:0/4079939067 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f27b000b040 con 0x5575ff2791b0 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 10 client.25721 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 20 client.25721 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff20d1c0 0x5575ff6a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 4 client.25721 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:54:28.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575ff5910b0 con 0x5575ff20d1c0 2024-09-21T06:54:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.972+0000 7f27bda00640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff20d1c0 0x5575ff6a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.973+0000 7f279f400640 10 client.25721 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:28.973+0000 7f27bda00640 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff20d1c0 0x5575ff6a8900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d707dba5adaedd0b server_cookie=c2863bff6b842cbb in_seq=0 out_seq=0 2024-09-21T06:54:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:28 smithi183 ceph-mon[34219]: pgmap v406: 129 pgs: 129 active+clean; 29 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 70 op/s 2024-09-21T06:54:29.333 INFO:teuthology.orchestra.run.smithi114.stderr:900000 of 3200000 tuples (28%) done (elapsed 2.28 s, remaining 5.82 s) 2024-09-21T06:54:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f279f400640 1 -- 172.21.15.114:0/4079939067 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575ff5910b0 con 0x5575ff20d1c0 2024-09-21T06:54:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f279f400640 10 client.25721 handle_command_reply: tid=0 2024-09-21T06:54:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 client.25721 shutdown 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 msgr2=0x5575ff6a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6a4ee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27b0009820 tx=0x7f27b0004f90 comp rx=0 tx=0).stop 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 shutdown_connections 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575ff2791b0 0x5575ff6a4ee0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575ff20d1c0 0x5575ff6a8900 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575ff2584d0 0x5575ff62ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 --2- 172.21.15.114:0/4079939067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575ff2502f0 0x5575ff62fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 >> 172.21.15.114:0/4079939067 conn(0x5575ff26d300 msgr2=0x5575ff61ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.369+0000 7f279f400640 0 client.25721 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 shutdown_connections 2024-09-21T06:54:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c7461740 1 -- 172.21.15.114:0/4079939067 wait complete. 2024-09-21T06:54:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c7461740 20 client.25721 trim_cache size 0 max 16384 2024-09-21T06:54:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:54:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:54:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:54:29.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27a00b65f0 msgr2=0x7f27a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27a00b65f0 0x7f27a00b8b00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5575ff61c8b0 tx=0x7f27ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 msgr2=0x7f27b819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 0x7f27b819cc50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f27a4035040 tx=0x7f27a4032d50 comp rx=0 tx=0).stop 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 shutdown_connections 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f27b8110030 0x7f27b81a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27b8076100 0x7f27b819cc50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.370+0000 7f27c0400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f27a00b65f0 0x7f27a00b8b00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.371+0000 7f27c0400640 1 --2- 172.21.15.114:0/543187226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27b80757b0 0x7f27b819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.371+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 >> 172.21.15.114:0/543187226 conn(0x7f27b80ff520 msgr2=0x7f27b8109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.371+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 shutdown_connections 2024-09-21T06:54:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:29.371+0000 7f27c0400640 1 -- 172.21.15.114:0/543187226 wait complete. 2024-09-21T06:54:29.587 INFO:teuthology.orchestra.run.smithi114.stderr:1000000 of 3200000 tuples (31%) done (elapsed 2.53 s, remaining 5.57 s) 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 -- 172.21.15.114:0/1403489760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109c40 msgr2=0x7f3a0c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 --2- 172.21.15.114:0/1403489760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109c40 0x7f3a0c110470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009ca0 tx=0x7f3a0402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 -- 172.21.15.114:0/1403489760 shutdown_connections 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 --2- 172.21.15.114:0/1403489760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109c40 0x7f3a0c110470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 --2- 172.21.15.114:0/1403489760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 --2- 172.21.15.114:0/1403489760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1039b0 0x7f3a0c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 -- 172.21.15.114:0/1403489760 >> 172.21.15.114:0/1403489760 conn(0x7f3a0c0ff400 msgr2=0x7f3a0c101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 -- 172.21.15.114:0/1403489760 shutdown_connections 2024-09-21T06:54:29.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.696+0000 7f3a12200640 1 -- 172.21.15.114:0/1403489760 wait complete. 2024-09-21T06:54:29.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.698+0000 7f3a12200640 1 Processor -- start 2024-09-21T06:54:29.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.698+0000 7f3a12200640 1 -- start start 2024-09-21T06:54:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a12200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c1039b0 0x7f3a0c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a12200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a12200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c109c40 0x7f3a0c1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a12200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c113950 con 0x7f3a0c109c40 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a12200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c1137d0 con 0x7f3a0c1039b0 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a12200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c113ad0 con 0x7f3a0c1092f0 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a10e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c1039b0 0x7f3a0c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c19ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52118/0 (socket says 172.21.15.114:52118) 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 -- 172.21.15.114:0/2127235698 learned_addr learned my addr 172.21.15.114:0/2127235698 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a11800640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c109c40 0x7f3a0c1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 -- 172.21.15.114:0/2127235698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c1039b0 msgr2=0x7f3a0c19c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c1039b0 0x7f3a0c19c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 -- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c109c40 msgr2=0x7f3a0c1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c109c40 0x7f3a0c1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a0be00640 1 -- 172.21.15.114:0/2127235698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c1a23d0 con 0x7f3a0c1092f0 2024-09-21T06:54:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.699+0000 7f3a11800640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c109c40 0x7f3a0c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:29.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.700+0000 7f3a0be00640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c19ca00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39f800e9f0 tx=0x7f39f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:29.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.700+0000 7f3a09600640 1 -- 172.21.15.114:0/2127235698 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f800ccc0 con 0x7f3a0c1092f0 2024-09-21T06:54:29.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.700+0000 7f3a09600640 1 -- 172.21.15.114:0/2127235698 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f80108e0 con 0x7f3a0c1092f0 2024-09-21T06:54:29.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.700+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c1a26c0 con 0x7f3a0c1092f0 2024-09-21T06:54:29.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.700+0000 7f3a09600640 1 -- 172.21.15.114:0/2127235698 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8018860 con 0x7f3a0c1092f0 2024-09-21T06:54:29.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.700+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c1a8e20 con 0x7f3a0c1092f0 2024-09-21T06:54:29.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.703+0000 7f3a1a4b8740 1 Processor -- start 2024-09-21T06:54:29.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.703+0000 7f3a1a4b8740 1 -- start start 2024-09-21T06:54:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 0x55be49dd8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dd8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a11280 0x55be49dd90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55be49d4fd50 con 0x55be49a090a0 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be49d3f180 con 0x55be49a31f60 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be49d42f90 con 0x55be49a11280 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a09600640 1 -- 172.21.15.114:0/2127235698 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f39f8010430 con 0x7f3a0c1092f0 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a11800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 0x55be49dd8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a11800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 0x55be49dd8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60254/0 (socket says 172.21.15.114:60254) 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a11800640 1 -- 172.21.15.114:0/240395207 learned_addr learned my addr 172.21.15.114:0/240395207 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a10e00640 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dd8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.704+0000 7f3a0be00640 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a11280 0x55be49dd90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a11800640 1 -- 172.21.15.114:0/240395207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a11280 msgr2=0x55be49dd90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a11800640 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a11280 0x55be49dd90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a11800640 1 -- 172.21.15.114:0/240395207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 msgr2=0x55be49dd8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a11800640 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dd8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a11800640 1 -- 172.21.15.114:0/240395207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be49d09a70 con 0x55be49a31f60 2024-09-21T06:54:29.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a10e00640 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dd8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f3a11800640 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 0x55be49dd8670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009870 tx=0x7f3a040098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523083d9957fc446 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.705+0000 7f39fe000640 1 -- 172.21.15.114:0/240395207 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04031d70 con 0x55be49a31f60 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.706+0000 7f39fe000640 1 -- 172.21.15.114:0/240395207 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a04036040 con 0x55be49a31f60 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.706+0000 7f39fe000640 1 -- 172.21.15.114:0/240395207 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04004330 con 0x55be49a31f60 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.706+0000 7f3a09600640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b6670 0x7f39f40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.706+0000 7f3a09600640 1 -- 172.21.15.114:0/2127235698 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f39f80e7ca0 con 0x7f3a0c1092f0 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.706+0000 7f3a10e00640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b6670 0x7f39f40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/240395207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 msgr2=0x55be49dd8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 0x55be49dd8670 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009870 tx=0x7f3a040098a0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/240395207 shutdown_connections 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a11280 0x55be49dd90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dd8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/240395207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a31f60 0x55be49dd8670 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/240395207 >> 172.21.15.114:0/240395207 conn(0x55be49a26350 msgr2=0x55be49dd6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/240395207 shutdown_connections 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a10e00640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b6670 0x7f39f40b8b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55be49dd72f0 tx=0x7f3a00005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:29.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.707+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/240395207 wait complete. 2024-09-21T06:54:29.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.708+0000 7f3a1a4b8740 1 -- start start 2024-09-21T06:54:29.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.708+0000 7f3a1a4b8740 1 Processor -- start 2024-09-21T06:54:29.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.708+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49de4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.708+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49de4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55be49ba05f0 con 0x55be49a090a0 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be49cda460 con 0x55be49a11280 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be49de6df0 con 0x55be49a31f60 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a1a4b8740 10 client.0 _ll_register_callbacks cb 0x55be49cea210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49de4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a10e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49de4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49de4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60260/0 (socket says 172.21.15.114:60260) 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 -- 172.21.15.114:0/2707766861 learned_addr learned my addr 172.21.15.114:0/2707766861 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a11800640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 -- 172.21.15.114:0/2707766861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 msgr2=0x55be49df00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 -- 172.21.15.114:0/2707766861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 msgr2=0x55be49de4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49de4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a0be00640 1 -- 172.21.15.114:0/2707766861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be49a9d540 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.709+0000 7f3a11800640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f3a0be00640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49de4790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ad770 tx=0x7f39f80096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f97fd7f18a78d25e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f39f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f39f2a00640 1 -- 172.21.15.114:0/2707766861 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80ad5b0 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be49cdca70 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f39f2a00640 1 -- 172.21.15.114:0/2707766861 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f39f80a9340 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f39f2a00640 1 -- 172.21.15.114:0/2707766861 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39f80acf50 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f3a1a4b8740 10 client.25733 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.710+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be49cf76f0 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f39f2a00640 1 -- 172.21.15.114:0/2707766861 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be49cdca70 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f39f2a00640 1 -- 172.21.15.114:0/2707766861 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f39f80ed020 con 0x55be49a11280 2024-09-21T06:54:29.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a1a4b8740 10 client.25733 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:29.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a1a4b8740 10 client.25733 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:54:29.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a1a4b8740 20 client.25733 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:29.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499f08e0 0x55be49df6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a1a4b8740 4 client.25733 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:29.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be49d51ca0 con 0x55be499f08e0 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.711+0000 7f3a10e00640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499f08e0 0x55be49df6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.712+0000 7f39f2a00640 10 client.25733 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.712+0000 7f3a10e00640 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499f08e0 0x55be49df6660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b29b86813a851a server_cookie=4df35efd91f68455 in_seq=0 out_seq=0 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f39f2a00640 1 -- 172.21.15.114:0/2707766861 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be49d51ca0 con 0x55be499f08e0 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f39f2a00640 10 client.25733 handle_command_reply: tid=0 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 client.25733 shutdown 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 msgr2=0x55be49de4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49de4790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ad770 tx=0x7f39f80096a0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 shutdown_connections 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499f08e0 0x55be49df6660 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49de4790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f39f2a00640 0 client.25733 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/2707766861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49de4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 >> 172.21.15.114:0/2707766861 conn(0x55be49a26350 msgr2=0x55be49ddb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 shutdown_connections 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.713+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/2707766861 wait complete. 2024-09-21T06:54:29.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.714+0000 7f3a1a4b8740 20 client.25733 trim_cache size 0 max 16384 2024-09-21T06:54:29.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 Processor -- start 2024-09-21T06:54:29.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 -- start start 2024-09-21T06:54:29.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 0x55be49ddf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49ddf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a31f60 0x55be49de28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55be49df49a0 con 0x55be49a31f60 2024-09-21T06:54:29.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be49df5070 con 0x55be49a11280 2024-09-21T06:54:29.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.733+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be49df5740 con 0x55be49a090a0 2024-09-21T06:54:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49ddf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 0x55be49ddf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 0x55be49ddf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52130/0 (socket says 172.21.15.114:52130) 2024-09-21T06:54:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49ddf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60270/0 (socket says 172.21.15.114:60270) 2024-09-21T06:54:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 -- 172.21.15.114:0/3505659446 learned_addr learned my addr 172.21.15.114:0/3505659446 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:29.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a11800640 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a31f60 0x55be49de28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 -- 172.21.15.114:0/3505659446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 msgr2=0x55be49ddf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49ddf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 -- 172.21.15.114:0/3505659446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a31f60 msgr2=0x55be49de28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a31f60 0x55be49de28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 -- 172.21.15.114:0/3505659446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be49df47d0 con 0x55be49a090a0 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.734+0000 7f3a10e00640 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 0x55be49ddf200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009730 tx=0x7f3a00022040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b82b7aca7c69e823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.735+0000 7f39fd600640 1 -- 172.21.15.114:0/3505659446 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00003c40 con 0x55be49a090a0 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.735+0000 7f39fd600640 1 -- 172.21.15.114:0/3505659446 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a00003de0 con 0x55be49a090a0 2024-09-21T06:54:29.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.735+0000 7f39fd600640 1 -- 172.21.15.114:0/3505659446 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0001f690 con 0x55be49a090a0 2024-09-21T06:54:29.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/3505659446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 msgr2=0x55be49ddf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 0x55be49ddf200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009730 tx=0x7f3a00022040 comp rx=0 tx=0).stop 2024-09-21T06:54:29.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/3505659446 shutdown_connections 2024-09-21T06:54:29.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a31f60 0x55be49de28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49ddf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/3505659446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a090a0 0x55be49ddf200 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/3505659446 >> 172.21.15.114:0/3505659446 conn(0x55be49a26350 msgr2=0x55be49df40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/3505659446 shutdown_connections 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.736+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/3505659446 wait complete. 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 -- start start 2024-09-21T06:54:29.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 Processor -- start 2024-09-21T06:54:29.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dfeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49dff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55be49df64e0 con 0x55be49a090a0 2024-09-21T06:54:29.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55be49df5e10 con 0x55be49a11280 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55be49df6bb0 con 0x55be49a31f60 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.737+0000 7f3a1a4b8740 10 client.0 _ll_register_callbacks cb 0x7f39f80a9e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a10e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dfeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49dff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49dff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60284/0 (socket says 172.21.15.114:60284) 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 -- 172.21.15.114:0/4153395990 learned_addr learned my addr 172.21.15.114:0/4153395990 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a11800640 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 -- 172.21.15.114:0/4153395990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 msgr2=0x55be49df8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 -- 172.21.15.114:0/4153395990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 msgr2=0x55be49dfeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dfeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 -- 172.21.15.114:0/4153395990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be49df8740 con 0x55be49a11280 2024-09-21T06:54:29.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f3a0be00640 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49dff130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39f800e9c0 tx=0x7f39f8009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80cc88e9e559e42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.738+0000 7f39f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f39f2a00640 1 -- 172.21.15.114:0/4153395990 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f80ad130 con 0x55be49a11280 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f39f2a00640 1 -- 172.21.15.114:0/4153395990 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f39f80ac4b0 con 0x55be49a11280 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a00003de0 con 0x55be49a11280 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f39f2a00640 1 -- 172.21.15.114:0/4153395990 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39f80c5e60 con 0x55be49a11280 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f3a1a4b8740 10 client.25739 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:29.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be49dfe180 con 0x55be49a11280 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f39f2a00640 1 -- 172.21.15.114:0/4153395990 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a00003de0 con 0x55be49a11280 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f39f2a00640 1 -- 172.21.15.114:0/4153395990 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f39f80ed070 con 0x55be49a11280 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.739+0000 7f3a1a4b8740 10 client.25739 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a1a4b8740 10 client.25739 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a1a4b8740 20 client.25739 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499c5c90 0x55be49e64ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a1a4b8740 4 client.25739 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "status"}) -- 0x55be49d4d750 con 0x55be499c5c90 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a10e00640 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499c5c90 0x55be49e64ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f3a10e00640 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499c5c90 0x55be49e64ca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b62a027228bc61 server_cookie=caaa13021ce6e4f3 in_seq=0 out_seq=0 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.740+0000 7f39f2a00640 10 client.25739 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:29.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f39f2a00640 1 -- 172.21.15.114:0/4153395990 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55be49d4d750 con 0x55be499c5c90 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f39f2a00640 10 client.25739 handle_command_reply: tid=0 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 client.25739 shutdown 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 msgr2=0x55be49dff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49dff130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39f800e9c0 tx=0x7f39f8009740 comp rx=0 tx=0).stop 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 shutdown_connections 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55be499c5c90 0x55be49e64ca0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55be49a31f60 0x55be49df8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55be49a11280 0x55be49dff130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 --2- 172.21.15.114:0/4153395990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55be49a090a0 0x55be49dfeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 >> 172.21.15.114:0/4153395990 conn(0x55be49a26350 msgr2=0x55be49df40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f39f2a00640 0 client.25739 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 shutdown_connections 2024-09-21T06:54:29.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 1 -- 172.21.15.114:0/4153395990 wait complete. 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.741+0000 7f3a1a4b8740 20 client.25739 trim_cache size 0 max 16384 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "cluster_fsid": "5d75913a-77e4-11ef-baf5-efdc52797490", 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "whoami": 0, 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "id": 24761, 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "want_state": "up:active", 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "state": "up:active", 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "fs_name": "cephfs", 2024-09-21T06:54:29.747 INFO:teuthology.orchestra.run.smithi114.stdout: "rank_uptime": 220.52459444199999, 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stdout: "mdsmap_epoch": 29, 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch": 89, 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_epoch_barrier": 89, 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stdout: "uptime": 234.141315292 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b6670 msgr2=0x7f39f40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b6670 0x7f39f40b8b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55be49dd72f0 tx=0x7f3a00005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 msgr2=0x7f3a0c19ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c19ca00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39f800e9f0 tx=0x7f39f800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 shutdown_connections 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c109c40 0x7f3a0c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c1092f0 0x7f3a0c19ca00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b6670 0x7f39f40b8b80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 --2- 172.21.15.114:0/2127235698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c1039b0 0x7f3a0c19c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 >> 172.21.15.114:0/2127235698 conn(0x7f3a0c0ff400 msgr2=0x7f3a0c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 shutdown_connections 2024-09-21T06:54:29.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:29.742+0000 7f3a12200640 1 -- 172.21.15.114:0/2127235698 wait complete. 2024-09-21T06:54:30.000 INFO:teuthology.orchestra.run.smithi114.stderr:1100000 of 3200000 tuples (34%) done (elapsed 2.94 s, remaining 5.62 s) 2024-09-21T06:54:30.145 INFO:teuthology.orchestra.run.smithi114.stderr:1200000 of 3200000 tuples (37%) done (elapsed 3.09 s, remaining 5.15 s) 2024-09-21T06:54:30.378 INFO:teuthology.orchestra.run.smithi114.stderr:1300000 of 3200000 tuples (40%) done (elapsed 3.32 s, remaining 4.86 s) 2024-09-21T06:54:30.551 INFO:teuthology.orchestra.run.smithi114.stderr:1400000 of 3200000 tuples (43%) done (elapsed 3.50 s, remaining 4.49 s) 2024-09-21T06:54:30.791 INFO:teuthology.orchestra.run.smithi114.stderr:1500000 of 3200000 tuples (46%) done (elapsed 3.74 s, remaining 4.23 s) 2024-09-21T06:54:31.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:30 smithi114 ceph-mon[30654]: pgmap v407: 129 pgs: 129 active+clean; 61 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 7.4 MiB/s wr, 101 op/s 2024-09-21T06:54:31.097 INFO:teuthology.orchestra.run.smithi114.stderr:1600000 of 3200000 tuples (50%) done (elapsed 4.04 s, remaining 4.04 s) 2024-09-21T06:54:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:30 smithi183 ceph-mon[34219]: pgmap v407: 129 pgs: 129 active+clean; 61 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 7.4 MiB/s wr, 101 op/s 2024-09-21T06:54:31.156 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-21T06:54:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:30 smithi193 ceph-mon[34601]: pgmap v407: 129 pgs: 129 active+clean; 61 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 7.4 MiB/s wr, 101 op/s 2024-09-21T06:54:31.278 INFO:teuthology.orchestra.run.smithi114.stderr:1700000 of 3200000 tuples (53%) done (elapsed 4.22 s, remaining 3.73 s) 2024-09-21T06:54:31.518 INFO:teuthology.orchestra.run.smithi114.stderr:1800000 of 3200000 tuples (56%) done (elapsed 4.46 s, remaining 3.47 s) 2024-09-21T06:54:31.799 INFO:teuthology.orchestra.run.smithi114.stderr:1900000 of 3200000 tuples (59%) done (elapsed 4.74 s, remaining 3.25 s) 2024-09-21T06:54:31.952 INFO:teuthology.orchestra.run.smithi114.stderr:2000000 of 3200000 tuples (62%) done (elapsed 4.90 s, remaining 2.94 s) 2024-09-21T06:54:32.165 INFO:teuthology.orchestra.run.smithi114.stderr:2100000 of 3200000 tuples (65%) done (elapsed 5.11 s, remaining 2.68 s) 2024-09-21T06:54:32.506 INFO:teuthology.orchestra.run.smithi114.stderr:2200000 of 3200000 tuples (68%) done (elapsed 5.45 s, remaining 2.48 s) 2024-09-21T06:54:32.799 INFO:teuthology.orchestra.run.smithi114.stderr:2300000 of 3200000 tuples (71%) done (elapsed 5.74 s, remaining 2.25 s) 2024-09-21T06:54:32.879 INFO:teuthology.orchestra.run.smithi114.stderr:2400000 of 3200000 tuples (75%) done (elapsed 5.82 s, remaining 1.94 s) 2024-09-21T06:54:33.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:32 smithi114 ceph-mon[30654]: pgmap v408: 129 pgs: 129 active+clean; 129 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 18 MiB/s wr, 238 op/s 2024-09-21T06:54:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:32 smithi183 ceph-mon[34219]: pgmap v408: 129 pgs: 129 active+clean; 129 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 18 MiB/s wr, 238 op/s 2024-09-21T06:54:33.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:32 smithi193 ceph-mon[34601]: pgmap v408: 129 pgs: 129 active+clean; 129 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 18 MiB/s wr, 238 op/s 2024-09-21T06:54:33.168 INFO:teuthology.orchestra.run.smithi114.stderr:2500000 of 3200000 tuples (78%) done (elapsed 6.11 s, remaining 1.71 s) 2024-09-21T06:54:33.316 INFO:teuthology.orchestra.run.smithi114.stderr:2600000 of 3200000 tuples (81%) done (elapsed 6.26 s, remaining 1.44 s) 2024-09-21T06:54:33.581 INFO:teuthology.orchestra.run.smithi114.stderr:2700000 of 3200000 tuples (84%) done (elapsed 6.53 s, remaining 1.21 s) 2024-09-21T06:54:33.849 INFO:teuthology.orchestra.run.smithi114.stderr:2800000 of 3200000 tuples (87%) done (elapsed 6.79 s, remaining 0.97 s) 2024-09-21T06:54:33.942 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:54:34.011 INFO:teuthology.orchestra.run.smithi114.stderr:2900000 of 3200000 tuples (90%) done (elapsed 6.95 s, remaining 0.72 s) 2024-09-21T06:54:34.332 INFO:teuthology.orchestra.run.smithi114.stderr:3000000 of 3200000 tuples (93%) done (elapsed 7.28 s, remaining 0.49 s) 2024-09-21T06:54:34.400 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:34.444 INFO:teuthology.orchestra.run.smithi114.stderr:3100000 of 3200000 tuples (96%) done (elapsed 7.39 s, remaining 0.24 s) 2024-09-21T06:54:34.764 INFO:teuthology.orchestra.run.smithi114.stderr:3200000 of 3200000 tuples (100%) done (elapsed 7.71 s, remaining 0.00 s) 2024-09-21T06:54:34.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:34 smithi114 ceph-mon[30654]: pgmap v409: 129 pgs: 129 active+clean; 233 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 35 MiB/s wr, 393 op/s 2024-09-21T06:54:34.817 INFO:teuthology.orchestra.run.smithi114.stderr:vacuuming... 2024-09-21T06:54:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:34 smithi183 ceph-mon[34219]: pgmap v409: 129 pgs: 129 active+clean; 233 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 35 MiB/s wr, 393 op/s 2024-09-21T06:54:35.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:34 smithi193 ceph-mon[34601]: pgmap v409: 129 pgs: 129 active+clean; 233 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 35 MiB/s wr, 393 op/s 2024-09-21T06:54:35.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 -- 172.21.15.114:0/3605918594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a41049d0 msgr2=0x7f58a4104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3605918594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a41049d0 0x7f58a4104de0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5894009900 tx=0x7f589402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 -- 172.21.15.114:0/3605918594 shutdown_connections 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3605918594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a410de30 0x7f58a4110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3605918594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a4105320 0x7f58a410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3605918594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a41049d0 0x7f58a4104de0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 -- 172.21.15.114:0/3605918594 >> 172.21.15.114:0/3605918594 conn(0x7f58a4079740 msgr2=0x7f58a4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 -- 172.21.15.114:0/3605918594 shutdown_connections 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.052+0000 7f58a8a00640 1 -- 172.21.15.114:0/3605918594 wait complete. 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.053+0000 7f58a8a00640 1 Processor -- start 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.053+0000 7f58a8a00640 1 -- start start 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.053+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a41049d0 0x7f58a419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a4105320 0x7f58a419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a410de30 0x7f58a419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a8a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4113a90 con 0x7f58a41049d0 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4113910 con 0x7f58a4105320 2024-09-21T06:54:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4113c10 con 0x7f58a410de30 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a41049d0 0x7f58a419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a410de30 0x7f58a419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a41049d0 0x7f58a419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47596/0 (socket says 172.21.15.114:47596) 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a1e00640 1 -- 172.21.15.114:0/3113758312 learned_addr learned my addr 172.21.15.114:0/3113758312 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a1400640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a4105320 0x7f58a419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a2800640 1 -- 172.21.15.114:0/3113758312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a4105320 msgr2=0x7f58a419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a2800640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a4105320 0x7f58a419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a2800640 1 -- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a41049d0 msgr2=0x7f58a419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a2800640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a41049d0 0x7f58a419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.054+0000 7f58a2800640 1 -- 172.21.15.114:0/3113758312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a41a0430 con 0x7f58a410de30 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.055+0000 7f58a2800640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a410de30 0x7f58a419fd00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f589c009fd0 tx=0x7f589c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.055+0000 7f589aa00640 1 -- 172.21.15.114:0/3113758312 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c00ce20 con 0x7f58a410de30 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.055+0000 7f589aa00640 1 -- 172.21.15.114:0/3113758312 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f589c005020 con 0x7f58a410de30 2024-09-21T06:54:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.055+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a41a0720 con 0x7f58a410de30 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.055+0000 7f589aa00640 1 -- 172.21.15.114:0/3113758312 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c010690 con 0x7f58a410de30 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.055+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a6e80 con 0x7f58a410de30 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.060+0000 7f58af613740 1 Processor -- start 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.060+0000 7f589aa00640 1 -- 172.21.15.114:0/3113758312 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f589c010830 con 0x7f58a410de30 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.060+0000 7f58af613740 1 -- start start 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58af613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58af613740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19d80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58af613740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58af613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650b194ef70 con 0x5650b1631070 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58af613740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650b193e3a0 con 0x5650b1610390 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58af613740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650b19421b0 con 0x5650b16081b0 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58a1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19d80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58a1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47602/0 (socket says 172.21.15.114:47602) 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.061+0000 7f58a1400640 1 -- 172.21.15.114:0/1813992922 learned_addr learned my addr 172.21.15.114:0/1813992922 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1400640 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1e00640 1 -- 172.21.15.114:0/1813992922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 msgr2=0x5650b19d85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1e00640 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19d85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1e00640 1 -- 172.21.15.114:0/1813992922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 msgr2=0x5650b19d7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1e00640 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19d7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1e00640 1 -- 172.21.15.114:0/1813992922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b1908c90 con 0x5650b16081b0 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f58a1e00640 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19d80a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5894004770 tx=0x7f5894031b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48dce372983ea7ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f589aa00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58880b6670 0x7f58880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f588f000640 1 -- 172.21.15.114:0/1813992922 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894040070 con 0x5650b16081b0 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f588f000640 1 -- 172.21.15.114:0/1813992922 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5894004030 con 0x5650b16081b0 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f588f000640 1 -- 172.21.15.114:0/1813992922 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894035040 con 0x5650b16081b0 2024-09-21T06:54:35.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.062+0000 7f589aa00640 1 -- 172.21.15.114:0/3113758312 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f589c0e8070 con 0x7f58a410de30 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.063+0000 7f58a1400640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58880b6670 0x7f58880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.063+0000 7f58af613740 1 -- 172.21.15.114:0/1813992922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 msgr2=0x5650b19d80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.063+0000 7f58a1400640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58880b6670 0x7f58880b8b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5890009400 tx=0x7f5890009370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.063+0000 7f58af613740 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19d80a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5894004770 tx=0x7f5894031b10 comp rx=0 tx=0).stop 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 -- 172.21.15.114:0/1813992922 shutdown_connections 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19d80a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 --2- 172.21.15.114:0/1813992922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 -- 172.21.15.114:0/1813992922 >> 172.21.15.114:0/1813992922 conn(0x5650b1625460 msgr2=0x5650b19d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 -- 172.21.15.114:0/1813992922 shutdown_connections 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 -- 172.21.15.114:0/1813992922 wait complete. 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 -- start start 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.064+0000 7f58af613740 1 Processor -- start 2024-09-21T06:54:35.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19e8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650b179f700 con 0x5650b1631070 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650b18d9540 con 0x5650b1610390 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650b19ea640 con 0x5650b16081b0 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58af613740 10 client.0 _ll_register_callbacks cb 0x5650b18e9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60436/0 (socket says 172.21.15.114:60436) 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58a1e00640 1 -- 172.21.15.114:0/2191515253 learned_addr learned my addr 172.21.15.114:0/2191515253 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.065+0000 7f58a1400640 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58a1e00640 1 -- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 msgr2=0x5650b19e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58a1e00640 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19e8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58a1e00640 1 -- 172.21.15.114:0/2191515253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 msgr2=0x5650b19f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58a1e00640 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58a1e00640 1 -- 172.21.15.114:0/2191515253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b169c650 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58a1e00640 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e7ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5894005160 tx=0x7f589402ab70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f220a93cb49926 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f5883400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f5883400640 1 -- 172.21.15.114:0/2191515253 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589404b020 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650b18dbb50 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f5883400640 1 -- 172.21.15.114:0/2191515253 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f58940314a0 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f5883400640 1 -- 172.21.15.114:0/2191515253 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5894056050 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58af613740 10 client.25753 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.066+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650b18f6910 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f5883400640 1 -- 172.21.15.114:0/2191515253 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5650b18dbb50 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f5883400640 1 -- 172.21.15.114:0/2191515253 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5894004b90 con 0x5650b16081b0 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f58af613740 10 client.25753 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f58af613740 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f58af613740 20 client.25753 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f58af613740 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b15ef9f0 0x5650b19f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f58af613740 4 client.25753 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.067+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b1950ec0 con 0x5650b15ef9f0 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.068+0000 7f58a1400640 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b15ef9f0 0x5650b19f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.068+0000 7f58a1400640 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b15ef9f0 0x5650b19f9800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79fdebea3358de1 server_cookie=22163407b70b86ee in_seq=0 out_seq=0 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.068+0000 7f5883400640 10 client.25753 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f5883400640 1 -- 172.21.15.114:0/2191515253 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650b1950ec0 con 0x5650b15ef9f0 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f5883400640 10 client.25753 handle_command_reply: tid=0 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 client.25753 shutdown 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 msgr2=0x5650b19e7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e7ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5894005160 tx=0x7f589402ab70 comp rx=0 tx=0).stop 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 shutdown_connections 2024-09-21T06:54:35.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1631070 0x5650b19f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b15ef9f0 0x5650b19f9800 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 --2- 172.21.15.114:0/2191515253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e7ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f5883400640 0 client.25753 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 >> 172.21.15.114:0/2191515253 conn(0x5650b1625460 msgr2=0x5650b19d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 shutdown_connections 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.069+0000 7f58af613740 1 -- 172.21.15.114:0/2191515253 wait complete. 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.070+0000 7f58af613740 20 client.25753 trim_cache size 0 max 16384 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 Processor -- start 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 -- start start 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1625460 0x5650b19d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 0x5650b19e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41aea60 con 0x5650b1610390 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41b05a0 con 0x5650b1625460 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58af613740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41af130 con 0x5650b16081b0 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 0x5650b19e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 0x5650b19e5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47618/0 (socket says 172.21.15.114:47618) 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58a2800640 1 -- 172.21.15.114:0/1179306759 learned_addr learned my addr 172.21.15.114:0/1179306759 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.088+0000 7f58a1400640 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1e00640 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1625460 0x5650b19d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1400640 1 -- 172.21.15.114:0/1179306759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1625460 msgr2=0x5650b19d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1400640 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1625460 0x5650b19d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1400640 1 -- 172.21.15.114:0/1179306759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 msgr2=0x5650b19e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1400640 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 0x5650b19e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1400640 1 -- 172.21.15.114:0/1179306759 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b19da470 con 0x5650b16081b0 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a2800640 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 0x5650b19e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:35.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f58a1400640 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e4ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5650b18d95a0 tx=0x7f5890015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f06775df6f3d39d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.089+0000 7f588e600640 1 -- 172.21.15.114:0/1179306759 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5890023920 con 0x5650b16081b0 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.090+0000 7f588e600640 1 -- 172.21.15.114:0/1179306759 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58900035f0 con 0x5650b16081b0 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.090+0000 7f588e600640 1 -- 172.21.15.114:0/1179306759 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589002e640 con 0x5650b16081b0 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.090+0000 7f58af613740 1 -- 172.21.15.114:0/1179306759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 msgr2=0x5650b19e4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.090+0000 7f58af613740 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e4ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5650b18d95a0 tx=0x7f5890015000 comp rx=0 tx=0).stop 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 -- 172.21.15.114:0/1179306759 shutdown_connections 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1610390 0x5650b19e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19e4ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 --2- 172.21.15.114:0/1179306759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1625460 0x5650b19d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 -- 172.21.15.114:0/1179306759 >> 172.21.15.114:0/1179306759 conn(0x5650b19f40f0 msgr2=0x5650b19d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 -- 172.21.15.114:0/1179306759 shutdown_connections 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 -- 172.21.15.114:0/1179306759 wait complete. 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 -- start start 2024-09-21T06:54:35.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.091+0000 7f58af613740 1 Processor -- start 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19f9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1625460 0x5650b1a5d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41afed0 con 0x5650b1625460 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41af800 con 0x5650b1610390 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f58a41aca40 con 0x5650b16081b0 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58af613740 10 client.0 _ll_register_callbacks cb 0x7f5894005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19f9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60448/0 (socket says 172.21.15.114:60448) 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 -- 172.21.15.114:0/192768361 learned_addr learned my addr 172.21.15.114:0/192768361 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a2800640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1625460 0x5650b1a5d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 -- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 msgr2=0x5650b19f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 -- 172.21.15.114:0/192768361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1625460 msgr2=0x5650b1a5d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.092+0000 7f58a1e00640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1625460 0x5650b1a5d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f58a1e00640 1 -- 172.21.15.114:0/192768361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b1a5d690 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f58a2800640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1625460 0x5650b1a5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f58a1e00640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19f9280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5894006010 tx=0x7f5894031c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648c3b35d2d72d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f5883400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f5883400640 1 -- 172.21.15.114:0/192768361 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589405f060 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f5883400640 1 -- 172.21.15.114:0/192768361 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f58940316a0 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58900035f0 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f5883400640 1 -- 172.21.15.114:0/192768361 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5894066050 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f58af613740 10 client.25759 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.093+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650b1a64a70 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f5883400640 1 -- 172.21.15.114:0/192768361 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58900035f0 con 0x5650b16081b0 2024-09-21T06:54:35.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f5883400640 1 -- 172.21.15.114:0/192768361 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f5894003b60 con 0x5650b16081b0 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58af613740 10 client.25759 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58af613740 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58af613740 20 client.25759 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58af613740 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b1631070 0x5650b1a67170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58af613740 4 client.25759 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650b194c970 con 0x5650b1631070 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.094+0000 7f58a1400640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b1631070 0x5650b1a67170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58a1400640 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b1631070 0x5650b1a67170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875e461c5f22ee7d server_cookie=cb9ce1faa2ffe341 in_seq=0 out_seq=0 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f5883400640 10 client.25759 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f5883400640 1 -- 172.21.15.114:0/192768361 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x5650b194c970 con 0x5650b1631070 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f5883400640 10 client.25759 handle_command_reply: tid=0 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 client.25759 shutdown 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 msgr2=0x5650b19f9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19f9280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5894006010 tx=0x7f5894031c00 comp rx=0 tx=0).stop 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 shutdown_connections 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650b1625460 0x5650b1a5d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650b1610390 0x5650b19f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650b1631070 0x5650b1a67170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 --2- 172.21.15.114:0/192768361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650b16081b0 0x5650b19f9280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 >> 172.21.15.114:0/192768361 conn(0x5650b19f40f0 msgr2=0x5650b19d8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.095+0000 7f5883400640 0 client.25759 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 shutdown_connections 2024-09-21T06:54:35.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58af613740 1 -- 172.21.15.114:0/192768361 wait complete. 2024-09-21T06:54:35.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58af613740 20 client.25759 trim_cache size 0 max 16384 2024-09-21T06:54:35.186 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:35.186 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T06:54:35.186 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T06:54:35.186 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:54:35.186 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:35.187 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58880b6670 msgr2=0x7f58880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58880b6670 0x7f58880b8b80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5890009400 tx=0x7f5890009370 comp rx=0 tx=0).stop 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a410de30 msgr2=0x7f58a419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a410de30 0x7f58a419fd00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f589c009fd0 tx=0x7f589c00ef10 comp rx=0 tx=0).stop 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 shutdown_connections 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58a410de30 0x7f58a419fd00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58880b6670 0x7f58880b8b80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58a4105320 0x7f58a419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.096+0000 7f58a8a00640 1 --2- 172.21.15.114:0/3113758312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f58a41049d0 0x7f58a419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.097+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 >> 172.21.15.114:0/3113758312 conn(0x7f58a4079740 msgr2=0x7f58a4100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.097+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 shutdown_connections 2024-09-21T06:54:35.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:35.097+0000 7f58a8a00640 1 -- 172.21.15.114:0/3113758312 wait complete. 2024-09-21T06:54:35.733 INFO:teuthology.orchestra.run.smithi114.stderr:creating primary keys... 2024-09-21T06:54:36.029 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:54:36.029 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:54:36.435 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:36.930 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:36 smithi114 ceph-mon[30654]: pgmap v410: 129 pgs: 129 active+clean; 237 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 36 MiB/s wr, 389 op/s 2024-09-21T06:54:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:36 smithi183 ceph-mon[34219]: pgmap v410: 129 pgs: 129 active+clean; 237 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 36 MiB/s wr, 389 op/s 2024-09-21T06:54:37.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 -- 172.21.15.114:0/3098640329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3410de30 msgr2=0x7f4e34110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3098640329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3410de30 0x7f4e34110320 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009ca0 tx=0x7f4e2c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 -- 172.21.15.114:0/3098640329 shutdown_connections 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3098640329 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e3410de30 0x7f4e34110320 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3098640329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3098640329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e341049d0 0x7f4e34104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.132+0000 7f4e3a200640 1 -- 172.21.15.114:0/3098640329 >> 172.21.15.114:0/3098640329 conn(0x7f4e34079740 msgr2=0x7f4e34079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.133+0000 7f4e3a200640 1 -- 172.21.15.114:0/3098640329 shutdown_connections 2024-09-21T06:54:37.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.133+0000 7f4e3a200640 1 -- 172.21.15.114:0/3098640329 wait complete. 2024-09-21T06:54:37.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.134+0000 7f4e3a200640 1 Processor -- start 2024-09-21T06:54:37.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 -- start start 2024-09-21T06:54:37.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e341049d0 0x7f4e3419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:37.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:37.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3410de30 0x7f4e341a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:37.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34113ab0 con 0x7f4e341049d0 2024-09-21T06:54:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34113930 con 0x7f4e34105320 2024-09-21T06:54:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.135+0000 7f4e3a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34113c30 con 0x7f4e3410de30 2024-09-21T06:54:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33092/0 (socket says 172.21.15.114:33092) 2024-09-21T06:54:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 -- 172.21.15.114:0/3970504472 learned_addr learned my addr 172.21.15.114:0/3970504472 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:37.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e33400640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e341049d0 0x7f4e3419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e33e00640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3410de30 0x7f4e341a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 -- 172.21.15.114:0/3970504472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3410de30 msgr2=0x7f4e341a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3410de30 0x7f4e341a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 -- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e341049d0 msgr2=0x7f4e3419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e341049d0 0x7f4e3419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.136+0000 7f4e2aa00640 1 -- 172.21.15.114:0/3970504472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e341a25d0 con 0x7f4e34105320 2024-09-21T06:54:37.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.137+0000 7f4e2aa00640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3419cc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9f0 tx=0x7f4e1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:37.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.137+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00ccc0 con 0x7f4e34105320 2024-09-21T06:54:37.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.137+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c0108e0 con 0x7f4e34105320 2024-09-21T06:54:37.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.137+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e341a28c0 con 0x7f4e34105320 2024-09-21T06:54:37.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.137+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c018860 con 0x7f4e34105320 2024-09-21T06:54:37.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.137+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e341a9020 con 0x7f4e34105320 2024-09-21T06:54:37.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.138+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3406c240 con 0x7f4e34105320 2024-09-21T06:54:37.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.144+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f4e1c010430 con 0x7f4e34105320 2024-09-21T06:54:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.147+0000 7f4e30c00640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4e180b65f0 0x7f4e180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:37.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.148+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4e1c0e80a0 con 0x7f4e34105320 2024-09-21T06:54:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.148+0000 7f4e33400640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4e180b65f0 0x7f4e180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.148+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4e1c0e85a0 con 0x7f4e34105320 2024-09-21T06:54:37.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.149+0000 7f4e33400640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4e180b65f0 0x7f4e180b8b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e3406c940 tx=0x7f4e24005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:37.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:36 smithi193 ceph-mon[34601]: pgmap v410: 129 pgs: 129 active+clean; 237 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 36 MiB/s wr, 389 op/s 2024-09-21T06:54:37.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.413+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e34104de0 con 0x7f4e34105320 2024-09-21T06:54:37.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.414+0000 7f4e30c00640 1 -- 172.21.15.114:0/3970504472 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11989 (secure 0 0 0) 0x7f4e1c0b2720 con 0x7f4e34105320 2024-09-21T06:54:37.416 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:37.416 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":29,"btime":"2024-09-21T06:53:49:573848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:53:49.006951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:54:37.416 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 29 2024-09-21T06:54:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.417+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4e180b65f0 msgr2=0x7f4e180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.417+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4e180b65f0 0x7f4e180b8b00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e3406c940 tx=0x7f4e24005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.417+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 msgr2=0x7f4e3419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.417+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3419cc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9f0 tx=0x7f4e1c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.417+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 shutdown_connections 2024-09-21T06:54:37.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.417+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e3410de30 0x7f4e341a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.418+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4e180b65f0 0x7f4e180b8b00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.418+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e34105320 0x7f4e3419cc00 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00e9f0 tx=0x7f4e1c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.418+0000 7f4e3a200640 1 --2- 172.21.15.114:0/3970504472 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4e341049d0 0x7f4e3419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:37.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.418+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 >> 172.21.15.114:0/3970504472 conn(0x7f4e34079740 msgr2=0x7f4e341036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:37.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.419+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 shutdown_connections 2024-09-21T06:54:37.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:37.419+0000 7f4e3a200640 1 -- 172.21.15.114:0/3970504472 wait complete. 2024-09-21T06:54:37.718 INFO:teuthology.orchestra.run.smithi114.stderr:done in 10.71 s (drop tables 0.00 s, create tables 0.02 s, client-side generate 7.79 s, vacuum 0.92 s, primary keys 1.99 s). 2024-09-21T06:54:37.725 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-21T06:54:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:starting vacuum...end. 2024-09-21T06:54:38.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:37 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3970504472' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:38.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:37 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3970504472' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:38.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:37 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3970504472' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:38.321 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:54:38.965 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:39.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:38 smithi114 ceph-mon[30654]: pgmap v411: 129 pgs: 129 active+clean; 373 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 57 MiB/s wr, 399 op/s 2024-09-21T06:54:39.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:38 smithi183 ceph-mon[34219]: pgmap v411: 129 pgs: 129 active+clean; 373 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 57 MiB/s wr, 399 op/s 2024-09-21T06:54:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:38 smithi193 ceph-mon[34601]: pgmap v411: 129 pgs: 129 active+clean; 373 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 57 MiB/s wr, 399 op/s 2024-09-21T06:54:39.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:39 smithi114 ceph-mon[30654]: pgmap v412: 129 pgs: 129 active+clean; 417 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 61 MiB/s wr, 355 op/s 2024-09-21T06:54:39.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 -- 172.21.15.114:0/1821027881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 msgr2=0x7fd5e01085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/1821027881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e01085c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009900 tx=0x7fd5c802a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 -- 172.21.15.114:0/1821027881 shutdown_connections 2024-09-21T06:54:39.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/1821027881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 0x7fd5e010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/1821027881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e01085c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/1821027881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 0x7fd5e0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.822+0000 7fd5dfe00640 1 -- 172.21.15.114:0/1821027881 >> 172.21.15.114:0/1821027881 conn(0x7fd5e00fe2c0 msgr2=0x7fd5e0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.823+0000 7fd5dfe00640 1 -- 172.21.15.114:0/1821027881 shutdown_connections 2024-09-21T06:54:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.823+0000 7fd5dfe00640 1 -- 172.21.15.114:0/1821027881 wait complete. 2024-09-21T06:54:39.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 Processor -- start 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 -- start start 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 0x7fd5e010e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e010ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 0x7fd5e01a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e006b1f0 con 0x7fd5e0108b00 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e006b070 con 0x7fd5e0102870 2024-09-21T06:54:39.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.825+0000 7fd5dfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e006b370 con 0x7fd5e01081b0 2024-09-21T06:54:39.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.826+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 0x7fd5e010e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.826+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 0x7fd5e010e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33106/0 (socket says 172.21.15.114:33106) 2024-09-21T06:54:39.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.826+0000 7fd5dea00640 1 -- 172.21.15.114:0/366509146 learned_addr learned my addr 172.21.15.114:0/366509146 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:39.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.826+0000 7fd5df400640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 0x7fd5e01a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.826+0000 7fd5de000640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e010ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.827+0000 7fd5de000640 1 -- 172.21.15.114:0/366509146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 msgr2=0x7fd5e010e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.827+0000 7fd5de000640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 0x7fd5e010e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.827+0000 7fd5de000640 1 -- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 msgr2=0x7fd5e01a2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.827+0000 7fd5de000640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 0x7fd5e01a2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.827+0000 7fd5de000640 1 -- 172.21.15.114:0/366509146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e01a3660 con 0x7fd5e01081b0 2024-09-21T06:54:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.827+0000 7fd5df400640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 0x7fd5e01a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.828+0000 7fd5de000640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e010ebe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009230 tx=0x7fd5c8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.828+0000 7fd5e4a00640 1 -- 172.21.15.114:0/366509146 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c803f070 con 0x7fd5e01081b0 2024-09-21T06:54:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.828+0000 7fd5e4a00640 1 -- 172.21.15.114:0/366509146 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c8031b60 con 0x7fd5e01081b0 2024-09-21T06:54:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.828+0000 7fd5e4a00640 1 -- 172.21.15.114:0/366509146 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8035030 con 0x7fd5e01081b0 2024-09-21T06:54:39.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.828+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e01a38f0 con 0x7fd5e01081b0 2024-09-21T06:54:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.829+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e01a3d50 con 0x7fd5e01081b0 2024-09-21T06:54:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.833+0000 7fd5ec9ca740 1 Processor -- start 2024-09-21T06:54:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.834+0000 7fd5ec9ca740 1 -- start start 2024-09-21T06:54:39.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.834+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 0x56286e932f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e5631b0 0x56286e9334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e9339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56286e8a9fb0 con 0x56286e56b390 2024-09-21T06:54:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56286e8993e0 con 0x56286e5631b0 2024-09-21T06:54:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56286e89d1f0 con 0x56286e58c070 2024-09-21T06:54:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 0x56286e932f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 0x56286e932f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60488/0 (socket says 172.21.15.114:60488) 2024-09-21T06:54:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.835+0000 7fd5df400640 1 -- 172.21.15.114:0/2515209295 learned_addr learned my addr 172.21.15.114:0/2515209295 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.836+0000 7fd5e4a00640 1 -- 172.21.15.114:0/366509146 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fd5c8042070 con 0x7fd5e01081b0 2024-09-21T06:54:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.836+0000 7fd5dea00640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e5631b0 0x56286e9334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.837+0000 7fd5df400640 1 -- 172.21.15.114:0/2515209295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e5631b0 msgr2=0x56286e9334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.839+0000 7fd5df400640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e5631b0 0x56286e9334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.839+0000 7fd5df400640 1 -- 172.21.15.114:0/2515209295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 msgr2=0x56286e9339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.837+0000 7fd5de000640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e9339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.839+0000 7fd5df400640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e9339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.840+0000 7fd5e4a00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5c40b66f0 0x7fd5c40b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.840+0000 7fd5df400640 1 -- 172.21.15.114:0/2515209295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56286e863cd0 con 0x56286e58c070 2024-09-21T06:54:39.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.840+0000 7fd5dea00640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e5631b0 0x56286e9334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.840+0000 7fd5de000640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e9339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.840+0000 7fd5e4a00640 1 -- 172.21.15.114:0/366509146 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd5c8107c20 con 0x7fd5e01081b0 2024-09-21T06:54:39.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.841+0000 7fd5dea00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5c40b66f0 0x7fd5c40b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.841+0000 7fd5df400640 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 0x56286e932f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0010400 tx=0x7fd5d00108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65ae2759027b9b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.842+0000 7fd5cfe00640 1 -- 172.21.15.114:0/2515209295 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0011760 con 0x56286e58c070 2024-09-21T06:54:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.842+0000 7fd5cfe00640 1 -- 172.21.15.114:0/2515209295 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d0011db0 con 0x56286e58c070 2024-09-21T06:54:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.842+0000 7fd5cfe00640 1 -- 172.21.15.114:0/2515209295 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0018380 con 0x56286e58c070 2024-09-21T06:54:39.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/2515209295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 msgr2=0x56286e932f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 0x56286e932f70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0010400 tx=0x7fd5d00108c0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5dea00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5c40b66f0 0x7fd5c40b8c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x56286e931bf0 tx=0x7fd5d8009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/2515209295 shutdown_connections 2024-09-21T06:54:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e9339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e5631b0 0x56286e9334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/2515209295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e58c070 0x56286e932f70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.844+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/2515209295 >> 172.21.15.114:0/2515209295 conn(0x56286e580460 msgr2=0x56286e930e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.845+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/2515209295 shutdown_connections 2024-09-21T06:54:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.845+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/2515209295 wait complete. 2024-09-21T06:54:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.846+0000 7fd5ec9ca740 1 -- start start 2024-09-21T06:54:39.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.846+0000 7fd5ec9ca740 1 Processor -- start 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.846+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e93e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.846+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e93ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.846+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e94aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56286e6fa700 con 0x56286e56b390 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56286e834580 con 0x56286e58c070 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56286e941750 con 0x56286e5631b0 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e94aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e94aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33126/0 (socket says 172.21.15.114:33126) 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 -- 172.21.15.114:0/3059646304 learned_addr learned my addr 172.21.15.114:0/3059646304 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5ec9ca740 10 client.0 _ll_register_callbacks cb 0x56286e844470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5de000640 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e93ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 -- 172.21.15.114:0/3059646304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 msgr2=0x56286e93e770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e93e770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 -- 172.21.15.114:0/3059646304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 msgr2=0x56286e93ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e93ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.847+0000 7fd5df400640 1 -- 172.21.15.114:0/3059646304 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286e5f7650 con 0x56286e58c070 2024-09-21T06:54:39.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.848+0000 7fd5df400640 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e94aa40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0019c30 tx=0x7fd5d0019c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1fb35071ec3407d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.848+0000 7fd5ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.849+0000 7fd5ccc00640 1 -- 172.21.15.114:0/3059646304 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0027020 con 0x56286e58c070 2024-09-21T06:54:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.849+0000 7fd5ccc00640 1 -- 172.21.15.114:0/3059646304 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd5d00195d0 con 0x56286e58c070 2024-09-21T06:54:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.849+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56286e836b90 con 0x56286e58c070 2024-09-21T06:54:39.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.849+0000 7fd5ccc00640 1 -- 172.21.15.114:0/3059646304 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5d001f040 con 0x56286e58c070 2024-09-21T06:54:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.850+0000 7fd5ec9ca740 10 client.25772 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.850+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56286e851950 con 0x56286e58c070 2024-09-21T06:54:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.850+0000 7fd5ccc00640 1 -- 172.21.15.114:0/3059646304 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56286e836b90 con 0x56286e58c070 2024-09-21T06:54:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ccc00640 1 -- 172.21.15.114:0/3059646304 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd5d0022030 con 0x56286e58c070 2024-09-21T06:54:39.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ec9ca740 10 client.25772 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:39.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ec9ca740 10 client.25772 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:54:39.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ec9ca740 20 client.25772 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:39.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e54a9f0 0x56286e9506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ec9ca740 4 client.25772 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:39.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.851+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56286e8abf00 con 0x56286e54a9f0 2024-09-21T06:54:39.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.852+0000 7fd5dea00640 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e54a9f0 0x56286e9506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.853+0000 7fd5ccc00640 10 client.25772 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:39.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.853+0000 7fd5dea00640 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e54a9f0 0x56286e9506b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1617cdbc912ad04 server_cookie=b43ae80f3dc9c90b in_seq=0 out_seq=0 2024-09-21T06:54:39.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.854+0000 7fd5ccc00640 1 -- 172.21.15.114:0/3059646304 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56286e8abf00 con 0x56286e54a9f0 2024-09-21T06:54:39.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.854+0000 7fd5ccc00640 10 client.25772 handle_command_reply: tid=0 2024-09-21T06:54:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.854+0000 7fd5ec9ca740 1 client.25772 shutdown 2024-09-21T06:54:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 msgr2=0x56286e94aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e94aa40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0019c30 tx=0x7fd5d0019c60 comp rx=0 tx=0).stop 2024-09-21T06:54:39.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 shutdown_connections 2024-09-21T06:54:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e54a9f0 0x56286e9506b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e94aa40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e93ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/3059646304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e93e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 >> 172.21.15.114:0/3059646304 conn(0x56286e580460 msgr2=0x56286e92f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:39.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ccc00640 0 client.25772 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.855+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 shutdown_connections 2024-09-21T06:54:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.856+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/3059646304 wait complete. 2024-09-21T06:54:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.856+0000 7fd5ec9ca740 20 client.25772 trim_cache size 0 max 16384 2024-09-21T06:54:39.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.881+0000 7fd5ec9ca740 1 Processor -- start 2024-09-21T06:54:39.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.881+0000 7fd5ec9ca740 1 -- start start 2024-09-21T06:54:39.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.882+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e9346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.882+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e930610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.882+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e930b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.882+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01bfb00 con 0x56286e56b390 2024-09-21T06:54:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.882+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01c1640 con 0x56286e58c070 2024-09-21T06:54:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.882+0000 7fd5ec9ca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01c01d0 con 0x56286e5631b0 2024-09-21T06:54:39.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e9346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e9346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60514/0 (socket says 172.21.15.114:60514) 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 -- 172.21.15.114:0/578744088 learned_addr learned my addr 172.21.15.114:0/578744088 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5df400640 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e930b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5de000640 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e930610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 -- 172.21.15.114:0/578744088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 msgr2=0x56286e930b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e930b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 -- 172.21.15.114:0/578744088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 msgr2=0x56286e930610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e930610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.883+0000 7fd5dea00640 1 -- 172.21.15.114:0/578744088 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56286e54ac20 con 0x56286e5631b0 2024-09-21T06:54:39.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.884+0000 7fd5dea00640 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e9346b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8013fd0 tx=0x7fd5d8019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d996adc902075e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:39.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.884+0000 7fd5cf400640 1 -- 172.21.15.114:0/578744088 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8004b20 con 0x56286e5631b0 2024-09-21T06:54:39.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.884+0000 7fd5cf400640 1 -- 172.21.15.114:0/578744088 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d802c030 con 0x56286e5631b0 2024-09-21T06:54:39.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.884+0000 7fd5cf400640 1 -- 172.21.15.114:0/578744088 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8005580 con 0x56286e5631b0 2024-09-21T06:54:39.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/578744088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 msgr2=0x56286e9346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e9346b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8013fd0 tx=0x7fd5d8019000 comp rx=0 tx=0).stop 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/578744088 shutdown_connections 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e58c070 0x56286e930b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e56b390 0x56286e930610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/578744088 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e9346b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/578744088 >> 172.21.15.114:0/578744088 conn(0x56286e580460 msgr2=0x56286e933fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/578744088 shutdown_connections 2024-09-21T06:54:39.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.886+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/578744088 wait complete. 2024-09-21T06:54:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.887+0000 7fd5ec9ca740 1 -- start start 2024-09-21T06:54:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.887+0000 7fd5ec9ca740 1 Processor -- start 2024-09-21T06:54:39.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.888+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e953eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.888+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 0x56286e951380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.888+0000 7fd5ec9ca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e58c070 0x56286e9518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.888+0000 7fd5de000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 0x56286e951380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.888+0000 7fd5de000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 0x56286e951380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33146/0 (socket says 172.21.15.114:33146) 2024-09-21T06:54:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e953eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5de000640 1 -- 172.21.15.114:0/1176990112 learned_addr learned my addr 172.21.15.114:0/1176990112 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01c0f70 con 0x56286e58c070 2024-09-21T06:54:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01c08a0 con 0x56286e56b390 2024-09-21T06:54:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e01a1b80 con 0x56286e5631b0 2024-09-21T06:54:39.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5ec9ca740 10 client.0 _ll_register_callbacks cb 0x7fd5d00056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:39.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.889+0000 7fd5df400640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e58c070 0x56286e9518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5de000640 1 -- 172.21.15.114:0/1176990112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 msgr2=0x56286e953eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5de000640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e953eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5de000640 1 -- 172.21.15.114:0/1176990112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e58c070 msgr2=0x56286e9518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:39.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5de000640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e58c070 0x56286e9518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5de000640 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56286e952030 con 0x56286e56b390 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5dea00640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e953eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5de000640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 0x56286e951380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80caa80 tx=0x7fd5c8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843f602b288a20a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5ccc00640 1 -- 172.21.15.114:0/1176990112 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8049040 con 0x56286e56b390 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.890+0000 7fd5ccc00640 1 -- 172.21.15.114:0/1176990112 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd5c80316b0 con 0x56286e56b390 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d802c030 con 0x56286e56b390 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ccc00640 1 -- 172.21.15.114:0/1176990112 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5c80ccf80 con 0x56286e56b390 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 10 client.25778 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:39.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56286e9badb0 con 0x56286e56b390 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ccc00640 1 -- 172.21.15.114:0/1176990112 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5d802c030 con 0x56286e56b390 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ccc00640 1 -- 172.21.15.114:0/1176990112 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd5c804c060 con 0x56286e56b390 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 10 client.25778 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 10 client.25778 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 20 client.25778 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e51fda0 0x56286e9bd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 4 client.25778 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.891+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56286e8a79b0 con 0x56286e51fda0 2024-09-21T06:54:39.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.892+0000 7fd5dea00640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e51fda0 0x56286e9bd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:39.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.893+0000 7fd5dea00640 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e51fda0 0x56286e9bd520 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb7452cc5601f5cc server_cookie=57fb202cf00fb4da in_seq=0 out_seq=0 2024-09-21T06:54:39.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.893+0000 7fd5ccc00640 10 client.25778 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:39.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.893+0000 7fd5ccc00640 1 -- 172.21.15.114:0/1176990112 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56286e8a79b0 con 0x56286e51fda0 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.893+0000 7fd5ccc00640 10 client.25778 handle_command_reply: tid=0 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.893+0000 7fd5ec9ca740 1 client.25778 shutdown 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 msgr2=0x56286e951380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 0x56286e951380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80caa80 tx=0x7fd5c8009260 comp rx=0 tx=0).stop 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 shutdown_connections 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56286e58c070 0x56286e9518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x56286e51fda0 0x56286e9bd520 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56286e56b390 0x56286e951380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 --2- 172.21.15.114:0/1176990112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56286e5631b0 0x56286e953eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ccc00640 0 client.25778 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 >> 172.21.15.114:0/1176990112 conn(0x56286e580460 msgr2=0x56286e9405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 shutdown_connections 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.894+0000 7fd5ec9ca740 1 -- 172.21.15.114:0/1176990112 wait complete. 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.895+0000 7fd5ec9ca740 20 client.25778 trim_cache size 0 max 16384 2024-09-21T06:54:39.897 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:54:39.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5c40b66f0 msgr2=0x7fd5c40b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5c40b66f0 0x7fd5c40b8c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x56286e931bf0 tx=0x7fd5d8009500 comp rx=0 tx=0).stop 2024-09-21T06:54:39.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 msgr2=0x7fd5e010ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:39.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e010ebe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8009230 tx=0x7fd5c8003c10 comp rx=0 tx=0).stop 2024-09-21T06:54:39.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 shutdown_connections 2024-09-21T06:54:39.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd5e0108b00 0x7fd5e01a2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5e01081b0 0x7fd5e010ebe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5c40b66f0 0x7fd5c40b8c00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 --2- 172.21.15.114:0/366509146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5e0102870 0x7fd5e010e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:39.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.896+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 >> 172.21.15.114:0/366509146 conn(0x7fd5e00fe2c0 msgr2=0x7fd5e0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:39.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.897+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 shutdown_connections 2024-09-21T06:54:39.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:39.897+0000 7fd5dfe00640 1 -- 172.21.15.114:0/366509146 wait complete. 2024-09-21T06:54:40.015 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:39 smithi183 ceph-mon[34219]: pgmap v412: 129 pgs: 129 active+clean; 417 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 61 MiB/s wr, 355 op/s 2024-09-21T06:54:40.062 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:39 smithi193 ceph-mon[34601]: pgmap v412: 129 pgs: 129 active+clean; 417 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 61 MiB/s wr, 355 op/s 2024-09-21T06:54:40.900 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:54:41.539 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:42.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.433+0000 7f805c000640 1 -- 172.21.15.114:0/111114621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054102a60 msgr2=0x7f8054102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.433+0000 7f805c000640 1 --2- 172.21.15.114:0/111114621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054102a60 0x7f8054102e70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f804800dc90 tx=0x7f804802e390 comp rx=0 tx=0).stop 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.434+0000 7f805c000640 1 -- 172.21.15.114:0/111114621 shutdown_connections 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.434+0000 7f805c000640 1 --2- 172.21.15.114:0/111114621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054108cf0 0x7f805410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.434+0000 7f805c000640 1 --2- 172.21.15.114:0/111114621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 0x7f80541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.434+0000 7f805c000640 1 --2- 172.21.15.114:0/111114621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054102a60 0x7f8054102e70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.434+0000 7f805c000640 1 -- 172.21.15.114:0/111114621 >> 172.21.15.114:0/111114621 conn(0x7f8054079730 msgr2=0x7f8054079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.435+0000 7f805c000640 1 -- 172.21.15.114:0/111114621 shutdown_connections 2024-09-21T06:54:42.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.435+0000 7f805c000640 1 -- 172.21.15.114:0/111114621 wait complete. 2024-09-21T06:54:42.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.436+0000 7f805c000640 1 Processor -- start 2024-09-21T06:54:42.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.438+0000 7f805c000640 1 -- start start 2024-09-21T06:54:42.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.439+0000 7f805c000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 0x7f805419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.439+0000 7f805c000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 0x7f8054198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.439+0000 7f805c000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054108cf0 0x7f8054198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f8059600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 0x7f805419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f805a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054108cf0 0x7f8054198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f8059600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 0x7f805419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33168/0 (socket says 172.21.15.114:33168) 2024-09-21T06:54:42.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f8059600640 1 -- 172.21.15.114:0/956125840 learned_addr learned my addr 172.21.15.114:0/956125840 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8054111920 con 0x7f80541083a0 2024-09-21T06:54:42.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80541117a0 con 0x7f8054102a60 2024-09-21T06:54:42.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8054111aa0 con 0x7f8054108cf0 2024-09-21T06:54:42.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.440+0000 7f8058c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 0x7f8054198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.442+0000 7f8059600640 1 -- 172.21.15.114:0/956125840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054108cf0 msgr2=0x7f8054198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.442+0000 7f8059600640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054108cf0 0x7f8054198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.442+0000 7f8059600640 1 -- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 msgr2=0x7f8054198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.443+0000 7f8059600640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 0x7f8054198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.443+0000 7f8059600640 1 -- 172.21.15.114:0/956125840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8054198e60 con 0x7f8054102a60 2024-09-21T06:54:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.443+0000 7f8058c00640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 0x7f8054198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:42.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.445+0000 7f8059600640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 0x7f805419ee30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8048000c00 tx=0x7f8048036a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.445+0000 7f8052000640 1 -- 172.21.15.114:0/956125840 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804803e5f0 con 0x7f8054102a60 2024-09-21T06:54:42.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.445+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805419daa0 con 0x7f8054102a60 2024-09-21T06:54:42.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.445+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805419dff0 con 0x7f8054102a60 2024-09-21T06:54:42.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.445+0000 7f8052000640 1 -- 172.21.15.114:0/956125840 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8048036e10 con 0x7f8054102a60 2024-09-21T06:54:42.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.446+0000 7f8052000640 1 -- 172.21.15.114:0/956125840 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804803dc30 con 0x7f8054102a60 2024-09-21T06:54:42.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.451+0000 7f8062de7740 1 Processor -- start 2024-09-21T06:54:42.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.452+0000 7f8062de7740 1 -- start start 2024-09-21T06:54:42.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.456+0000 7f8062de7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c52924da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.456+0000 7f8062de7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c529252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.456+0000 7f8062de7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52925820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.456+0000 7f8062de7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c5289bfb0 con 0x557c5257e070 2024-09-21T06:54:42.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.456+0000 7f8062de7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c5288b3e0 con 0x557c5255d390 2024-09-21T06:54:42.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.456+0000 7f8062de7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c5288f1f0 con 0x557c525551b0 2024-09-21T06:54:42.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.457+0000 7f8052000640 1 -- 172.21.15.114:0/956125840 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f804803d3a0 con 0x7f8054102a60 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.458+0000 7f805a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c52924da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.458+0000 7f805a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c52924da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47736/0 (socket says 172.21.15.114:47736) 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.459+0000 7f805a000640 1 -- 172.21.15.114:0/739980100 learned_addr learned my addr 172.21.15.114:0/739980100 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.459+0000 7f805a000640 1 -- 172.21.15.114:0/739980100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 msgr2=0x557c529252e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.459+0000 7f805a000640 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c529252e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.460+0000 7f805a000640 1 -- 172.21.15.114:0/739980100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 msgr2=0x557c52925820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.460+0000 7f805a000640 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52925820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.460+0000 7f805a000640 1 -- 172.21.15.114:0/739980100 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c52855cd0 con 0x557c5257e070 2024-09-21T06:54:42.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.461+0000 7f805a000640 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c52924da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f804c00a4d0 tx=0x7f804c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01438c45ee80a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.461+0000 7f8046600640 1 -- 172.21.15.114:0/739980100 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c018020 con 0x557c5257e070 2024-09-21T06:54:42.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.462+0000 7f8046600640 1 -- 172.21.15.114:0/739980100 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f804c014cc0 con 0x557c5257e070 2024-09-21T06:54:42.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.462+0000 7f8046600640 1 -- 172.21.15.114:0/739980100 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c013d80 con 0x557c5257e070 2024-09-21T06:54:42.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.463+0000 7f8062de7740 1 -- 172.21.15.114:0/739980100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 msgr2=0x557c52924da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.463+0000 7f8062de7740 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c52924da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f804c00a4d0 tx=0x7f804c009650 comp rx=0 tx=0).stop 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.464+0000 7f8052000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f803c0b6670 0x7f803c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f8052000640 1 -- 172.21.15.114:0/956125840 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f80481077e0 con 0x7f8054102a60 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f805a000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f803c0b6670 0x7f803c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f8062de7740 1 -- 172.21.15.114:0/739980100 shutdown_connections 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f8062de7740 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52925820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f8062de7740 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c529252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f8062de7740 1 --2- 172.21.15.114:0/739980100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c52924da0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.465+0000 7f8062de7740 1 -- 172.21.15.114:0/739980100 >> 172.21.15.114:0/739980100 conn(0x557c52572460 msgr2=0x557c52922c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.466+0000 7f8062de7740 1 -- 172.21.15.114:0/739980100 shutdown_connections 2024-09-21T06:54:42.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.466+0000 7f805a000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f803c0b6670 0x7f803c0b8b80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f804c009e60 tx=0x7f804c0138c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.467+0000 7f8062de7740 1 -- 172.21.15.114:0/739980100 wait complete. 2024-09-21T06:54:42.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.468+0000 7f8062de7740 1 -- start start 2024-09-21T06:54:42.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.469+0000 7f8062de7740 1 Processor -- start 2024-09-21T06:54:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.474+0000 7f8062de7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c52930500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.475+0000 7f8062de7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52930a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.475+0000 7f8062de7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5293c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8062de7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557c526ec700 con 0x557c5257e070 2024-09-21T06:54:42.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8062de7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557c52826580 con 0x557c5255d390 2024-09-21T06:54:42.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8062de7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557c52933580 con 0x557c525551b0 2024-09-21T06:54:42.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8062de7740 10 client.0 _ll_register_callbacks cb 0x557c52836470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:42.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8059600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c52930500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8059600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c52930500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60566/0 (socket says 172.21.15.114:60566) 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8059600640 1 -- 172.21.15.114:0/2887895157 learned_addr learned my addr 172.21.15.114:0/2887895157 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f805a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5293c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.476+0000 7f8058c00640 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52930a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.478+0000 7f805a000640 1 -- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 msgr2=0x557c52930500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.478+0000 7f805a000640 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c52930500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.478+0000 7f805a000640 1 -- 172.21.15.114:0/2887895157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 msgr2=0x557c52930a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.478+0000 7f805a000640 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52930a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.478+0000 7f805a000640 1 -- 172.21.15.114:0/2887895157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c525e9650 con 0x557c5257e070 2024-09-21T06:54:42.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.479+0000 7f805a000640 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5293c900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f804c014040 tx=0x7f804c013a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ec3caf3b5a7004 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.479+0000 7f803aa00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:42.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.480+0000 7f803aa00640 1 -- 172.21.15.114:0/2887895157 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c018040 con 0x557c5257e070 2024-09-21T06:54:42.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.482+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c52828b90 con 0x557c5257e070 2024-09-21T06:54:42.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.482+0000 7f803aa00640 1 -- 172.21.15.114:0/2887895157 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f804c01d8f0 con 0x557c5257e070 2024-09-21T06:54:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.483+0000 7f803aa00640 1 -- 172.21.15.114:0/2887895157 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f804c01daf0 con 0x557c5257e070 2024-09-21T06:54:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.484+0000 7f8062de7740 10 client.15786 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:42.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.484+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c52843950 con 0x557c5257e070 2024-09-21T06:54:42.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.486+0000 7f803aa00640 1 -- 172.21.15.114:0/2887895157 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557c52828b90 con 0x557c5257e070 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f803aa00640 1 -- 172.21.15.114:0/2887895157 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f804c027040 con 0x557c5257e070 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f8062de7740 10 client.15786 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f8062de7740 10 client.15786 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f8062de7740 20 client.15786 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f8062de7740 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c5253c9f0 0x557c529425c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f8062de7740 4 client.15786 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:42.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.488+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c5289df00 con 0x557c5253c9f0 2024-09-21T06:54:42.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.489+0000 7f8059600640 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c5253c9f0 0x557c529425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.490+0000 7f803aa00640 10 client.15786 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:42.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.490+0000 7f8059600640 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c5253c9f0 0x557c529425c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b477d4dd7cb7cf5 server_cookie=fd4028c0896a1f0d in_seq=0 out_seq=0 2024-09-21T06:54:42.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.491+0000 7f803aa00640 1 -- 172.21.15.114:0/2887895157 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c5289df00 con 0x557c5253c9f0 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.491+0000 7f803aa00640 10 client.15786 handle_command_reply: tid=0 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.491+0000 7f8062de7740 1 client.15786 shutdown 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.491+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 msgr2=0x557c5293c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.491+0000 7f8062de7740 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5293c900 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f804c014040 tx=0x7f804c013a00 comp rx=0 tx=0).stop 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 shutdown_connections 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c5253c9f0 0x557c529425c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5293c900 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5255d390 0x557c52930a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 --2- 172.21.15.114:0/2887895157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c525551b0 0x557c52930500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 >> 172.21.15.114:0/2887895157 conn(0x557c52572460 msgr2=0x557c52925dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f803aa00640 0 client.15786 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 shutdown_connections 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.492+0000 7f8062de7740 1 -- 172.21.15.114:0/2887895157 wait complete. 2024-09-21T06:54:42.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.493+0000 7f8062de7740 20 client.15786 trim_cache size 0 max 16384 2024-09-21T06:54:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.526+0000 7f8062de7740 1 Processor -- start 2024-09-21T06:54:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.526+0000 7f8062de7740 1 -- start start 2024-09-21T06:54:42.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.526+0000 7f8062de7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c525551b0 0x557c52922cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.526+0000 7f8062de7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c5293c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.526+0000 7f8062de7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5292d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.527+0000 7f8062de7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f803400ff70 con 0x557c5257e070 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.527+0000 7f8062de7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8034011ab0 con 0x557c525551b0 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.527+0000 7f8062de7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8034010640 con 0x557c5255d390 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.527+0000 7f8058c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c5293c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8059600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c525551b0 0x557c52922cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8058c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c5293c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60576/0 (socket says 172.21.15.114:60576) 2024-09-21T06:54:42.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8058c00640 1 -- 172.21.15.114:0/603923540 learned_addr learned my addr 172.21.15.114:0/603923540 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f805a000640 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5292d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8059600640 1 -- 172.21.15.114:0/603923540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 msgr2=0x557c5293c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8059600640 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c5293c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8059600640 1 -- 172.21.15.114:0/603923540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 msgr2=0x557c5292d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8059600640 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5292d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.528+0000 7f8059600640 1 -- 172.21.15.114:0/603923540 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c52933090 con 0x557c525551b0 2024-09-21T06:54:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.529+0000 7f8059600640 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c525551b0 0x557c52922cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80480c9950 tx=0x7f804802e390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dcaf5535835fb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.529+0000 7f8045c00640 1 -- 172.21.15.114:0/603923540 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480f4070 con 0x557c525551b0 2024-09-21T06:54:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.529+0000 7f8045c00640 1 -- 172.21.15.114:0/603923540 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80480cce90 con 0x557c525551b0 2024-09-21T06:54:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.529+0000 7f8045c00640 1 -- 172.21.15.114:0/603923540 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048105030 con 0x557c525551b0 2024-09-21T06:54:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.530+0000 7f8062de7740 1 -- 172.21.15.114:0/603923540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c525551b0 msgr2=0x557c52922cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.530+0000 7f8062de7740 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c525551b0 0x557c52922cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f80480c9950 tx=0x7f804802e390 comp rx=0 tx=0).stop 2024-09-21T06:54:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 -- 172.21.15.114:0/603923540 shutdown_connections 2024-09-21T06:54:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c5257e070 0x557c5292d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c5293c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 --2- 172.21.15.114:0/603923540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c525551b0 0x557c52922cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 -- 172.21.15.114:0/603923540 >> 172.21.15.114:0/603923540 conn(0x557c52572460 msgr2=0x557c529225b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 -- 172.21.15.114:0/603923540 shutdown_connections 2024-09-21T06:54:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.531+0000 7f8062de7740 1 -- 172.21.15.114:0/603923540 wait complete. 2024-09-21T06:54:42.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.532+0000 7f8062de7740 1 -- start start 2024-09-21T06:54:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.532+0000 7f8062de7740 1 Processor -- start 2024-09-21T06:54:42.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.533+0000 7f8062de7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c525551b0 0x557c52947020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.533+0000 7f8062de7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c529424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.533+0000 7f8062de7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 0x557c52942be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f8062de7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f80340113e0 con 0x557c525551b0 2024-09-21T06:54:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f8062de7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8034010d10 con 0x557c5257e070 2024-09-21T06:54:42.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f8062de7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f803400def0 con 0x557c5255d390 2024-09-21T06:54:42.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f8062de7740 10 client.0 _ll_register_callbacks cb 0x7f804c013740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:42.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f805a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 0x557c52942be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f8059600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c525551b0 0x557c52947020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f805a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 0x557c52942be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33196/0 (socket says 172.21.15.114:33196) 2024-09-21T06:54:42.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f805a000640 1 -- 172.21.15.114:0/525786530 learned_addr learned my addr 172.21.15.114:0/525786530 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f8058c00640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c529424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.534+0000 7f805a000640 1 -- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 msgr2=0x557c529424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.535+0000 7f805a000640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c529424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.535+0000 7f805a000640 1 -- 172.21.15.114:0/525786530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c525551b0 msgr2=0x557c52947020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.535+0000 7f805a000640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c525551b0 0x557c52947020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.535+0000 7f805a000640 1 -- 172.21.15.114:0/525786530 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c529bcd40 con 0x557c5257e070 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.536+0000 7f805a000640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 0x557c52942be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f804c00a420 tx=0x7f804c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb26d21df6c1ea6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.536+0000 7f803aa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:42.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.536+0000 7f8058c00640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c529424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.538+0000 7f803aa00640 1 -- 172.21.15.114:0/525786530 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c018040 con 0x557c5257e070 2024-09-21T06:54:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.538+0000 7f803aa00640 1 -- 172.21.15.114:0/525786530 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f804c03c4a0 con 0x557c5257e070 2024-09-21T06:54:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.538+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80480cce90 con 0x557c5257e070 2024-09-21T06:54:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.538+0000 7f803aa00640 1 -- 172.21.15.114:0/525786530 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f804c04e050 con 0x557c5257e070 2024-09-21T06:54:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.538+0000 7f8062de7740 10 client.25790 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:42.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.538+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c529bde60 con 0x557c5257e070 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f803aa00640 1 -- 172.21.15.114:0/525786530 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80480cce90 con 0x557c5257e070 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f803aa00640 1 -- 172.21.15.114:0/525786530 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f804c02a450 con 0x557c5257e070 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f8062de7740 10 client.25790 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f8062de7740 10 client.25790 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f8062de7740 20 client.25790 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f8062de7740 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c52511da0 0x557c529c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f8062de7740 4 client.25790 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:54:42.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.539+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c528999b0 con 0x557c52511da0 2024-09-21T06:54:42.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.540+0000 7f8059600640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c52511da0 0x557c529c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.544+0000 7f803aa00640 10 client.25790 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:42.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.544+0000 7f8059600640 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c52511da0 0x557c529c0500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c68409eaf3417b0c server_cookie=2bad1e20b67606fe in_seq=0 out_seq=0 2024-09-21T06:54:42.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.544+0000 7f803aa00640 1 -- 172.21.15.114:0/525786530 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c528999b0 con 0x557c52511da0 2024-09-21T06:54:42.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.544+0000 7f803aa00640 10 client.25790 handle_command_reply: tid=0 2024-09-21T06:54:42.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.544+0000 7f8062de7740 1 client.25790 shutdown 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 msgr2=0x557c52942be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 0x557c52942be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f804c00a420 tx=0x7f804c009790 comp rx=0 tx=0).stop 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 shutdown_connections 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557c52511da0 0x557c529c0500 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557c5257e070 0x557c52942be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557c5255d390 0x557c529424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 --2- 172.21.15.114:0/525786530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557c525551b0 0x557c52947020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 >> 172.21.15.114:0/525786530 conn(0x557c52572460 msgr2=0x557c529302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f803aa00640 0 client.25790 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 shutdown_connections 2024-09-21T06:54:42.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.545+0000 7f8062de7740 1 -- 172.21.15.114:0/525786530 wait complete. 2024-09-21T06:54:42.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.546+0000 7f8062de7740 20 client.25790 trim_cache size 0 max 16384 2024-09-21T06:54:42.548 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:54:42.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f803c0b6670 msgr2=0x7f803c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f803c0b6670 0x7f803c0b8b80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f804c009e60 tx=0x7f804c0138c0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 msgr2=0x7f805419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 0x7f805419ee30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8048000c00 tx=0x7f8048036a10 comp rx=0 tx=0).stop 2024-09-21T06:54:42.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 shutdown_connections 2024-09-21T06:54:42.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8054108cf0 0x7f8054198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80541083a0 0x7f8054198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f803c0b6670 0x7f803c0b8b80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 --2- 172.21.15.114:0/956125840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8054102a60 0x7f805419ee30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.547+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 >> 172.21.15.114:0/956125840 conn(0x7f8054079730 msgr2=0x7f8054106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.548+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 shutdown_connections 2024-09-21T06:54:42.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:42.548+0000 7f805c000640 1 -- 172.21.15.114:0/956125840 wait complete. 2024-09-21T06:54:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 -- 172.21.15.114:0/311884004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010b440 msgr2=0x7fb07010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 --2- 172.21.15.114:0/311884004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010b440 0x7fb07010b850 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009900 tx=0x7fb05c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 -- 172.21.15.114:0/311884004 shutdown_connections 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 --2- 172.21.15.114:0/311884004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07010bd90 0x7fb0701125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 --2- 172.21.15.114:0/311884004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010b440 0x7fb07010b850 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 --2- 172.21.15.114:0/311884004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb070105b10 0x7fb070105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 -- 172.21.15.114:0/311884004 >> 172.21.15.114:0/311884004 conn(0x7fb0701014f0 msgr2=0x7fb070103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 -- 172.21.15.114:0/311884004 shutdown_connections 2024-09-21T06:54:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.707+0000 7fb074c00640 1 -- 172.21.15.114:0/311884004 wait complete. 2024-09-21T06:54:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.711+0000 7fb074c00640 1 Processor -- start 2024-09-21T06:54:42.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.712+0000 7fb074c00640 1 -- start start 2024-09-21T06:54:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.713+0000 7fb074c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb070105b10 0x7fb070111430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.713+0000 7fb074c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07010b440 0x7fb0701119d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.713+0000 7fb074c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 0x7fb0701a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.713+0000 7fb074c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb070078850 con 0x7fb070105b10 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.713+0000 7fb074c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0700786d0 con 0x7fb07010bd90 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.713+0000 7fb074c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0700789d0 con 0x7fb07010b440 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.714+0000 7fb06fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 0x7fb0701a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.714+0000 7fb06fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 0x7fb0701a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33216/0 (socket says 172.21.15.114:33216) 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.714+0000 7fb06fe00640 1 -- 172.21.15.114:0/297622896 learned_addr learned my addr 172.21.15.114:0/297622896 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.714+0000 7fb06ea00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07010b440 0x7fb0701119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.715+0000 7fb06fe00640 1 -- 172.21.15.114:0/297622896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07010b440 msgr2=0x7fb0701119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.715+0000 7fb06fe00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07010b440 0x7fb0701119d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.715+0000 7fb06fe00640 1 -- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb070105b10 msgr2=0x7fb070111430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:54:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.716+0000 7fb06f400640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb070105b10 0x7fb070111430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.716+0000 7fb06fe00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb070105b10 0x7fb070111430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.717+0000 7fb06fe00640 1 -- 172.21.15.114:0/297622896 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0701a5610 con 0x7fb07010bd90 2024-09-21T06:54:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.718+0000 7fb06fe00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 0x7fb0701a4fa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb06800cce0 tx=0x7fb06800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.718+0000 7fb067e00640 1 -- 172.21.15.114:0/297622896 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06800f450 con 0x7fb07010bd90 2024-09-21T06:54:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.718+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0701a5900 con 0x7fb07010bd90 2024-09-21T06:54:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.718+0000 7fb067e00640 1 -- 172.21.15.114:0/297622896 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06800faa0 con 0x7fb07010bd90 2024-09-21T06:54:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.718+0000 7fb067e00640 1 -- 172.21.15.114:0/297622896 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068018b70 con 0x7fb07010bd90 2024-09-21T06:54:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.719+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0701ad250 con 0x7fb07010bd90 2024-09-21T06:54:42.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.725+0000 7fb067e00640 1 -- 172.21.15.114:0/297622896 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fb068018d10 con 0x7fb07010bd90 2024-09-21T06:54:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.729+0000 7fb07d400740 1 Processor -- start 2024-09-21T06:54:42.731 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb067e00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0540b6630 0x7fb0540b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 -- start start 2024-09-21T06:54:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb067e00640 1 -- 172.21.15.114:0/297622896 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb0680e7e70 con 0x7fb07010bd90 2024-09-21T06:54:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d30c1b0 0x56362d6b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56362d6266b0 con 0x56362d2eb4d0 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56362d615ae0 con 0x56362d2e32f0 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.730+0000 7fb07d400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56362d6198f0 con 0x56362d30c1b0 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.731+0000 7fb06ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.731+0000 7fb06ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33222/0 (socket says 172.21.15.114:33222) 2024-09-21T06:54:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.731+0000 7fb06ea00640 1 -- 172.21.15.114:0/2001634657 learned_addr learned my addr 172.21.15.114:0/2001634657 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.731+0000 7fb06ea00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0540b6630 0x7fb0540b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06f400640 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d30c1b0 0x56362d6b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06ea00640 1 -- 172.21.15.114:0/2001634657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d30c1b0 msgr2=0x56362d6b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06ea00640 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d30c1b0 0x56362d6b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06ea00640 1 -- 172.21.15.114:0/2001634657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 msgr2=0x56362d6b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06ea00640 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06ea00640 1 -- 172.21.15.114:0/2001634657 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362d5e2780 con 0x56362d2e32f0 2024-09-21T06:54:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06f400640 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d30c1b0 0x56362d6b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.732+0000 7fb06ea00640 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6b1350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009900 tx=0x7fb05c003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9a5401339c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.733+0000 7fb06ea00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0540b6630 0x7fb0540b8b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb05c006010 tx=0x7fb05c03a000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.733+0000 7fb05be00640 1 -- 172.21.15.114:0/2001634657 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c035040 con 0x56362d2e32f0 2024-09-21T06:54:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.733+0000 7fb05be00640 1 -- 172.21.15.114:0/2001634657 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c04e070 con 0x56362d2e32f0 2024-09-21T06:54:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.733+0000 7fb05be00640 1 -- 172.21.15.114:0/2001634657 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0047a0 con 0x56362d2e32f0 2024-09-21T06:54:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.736+0000 7fb07d400740 1 -- 172.21.15.114:0/2001634657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 msgr2=0x56362d6b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.736+0000 7fb07d400740 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6b1350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009900 tx=0x7fb05c003c90 comp rx=0 tx=0).stop 2024-09-21T06:54:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 -- 172.21.15.114:0/2001634657 shutdown_connections 2024-09-21T06:54:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6b1350 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 --2- 172.21.15.114:0/2001634657 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d30c1b0 0x56362d6b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 -- 172.21.15.114:0/2001634657 >> 172.21.15.114:0/2001634657 conn(0x56362d300300 msgr2=0x56362d6aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 -- 172.21.15.114:0/2001634657 shutdown_connections 2024-09-21T06:54:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.737+0000 7fb07d400740 1 -- 172.21.15.114:0/2001634657 wait complete. 2024-09-21T06:54:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.739+0000 7fb07d400740 1 -- start start 2024-09-21T06:54:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.740+0000 7fb07d400740 1 Processor -- start 2024-09-21T06:54:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.743+0000 7fb07d400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb07d400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2eb4d0 0x56362d6bcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb07d400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb07d400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56362d54dab0 con 0x56362d30c1b0 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb07d400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56362d5bc570 con 0x56362d2e32f0 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb07d400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56362d6bf530 con 0x56362d2eb4d0 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb06f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb06f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6bc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33226/0 (socket says 172.21.15.114:33226) 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb06f400640 1 -- 172.21.15.114:0/876214515 learned_addr learned my addr 172.21.15.114:0/876214515 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.744+0000 7fb07d400740 10 client.0 _ll_register_callbacks cb 0x56362d5cb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.745+0000 7fb06ea00640 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2eb4d0 0x56362d6bcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.745+0000 7fb06f400640 1 -- 172.21.15.114:0/876214515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2eb4d0 msgr2=0x56362d6bcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.745+0000 7fb06f400640 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2eb4d0 0x56362d6bcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.745+0000 7fb06f400640 1 -- 172.21.15.114:0/876214515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 msgr2=0x56362d6c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.745+0000 7fb06f400640 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.745+0000 7fb06f400640 1 -- 172.21.15.114:0/876214515 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362d377790 con 0x56362d2e32f0 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.746+0000 7fb06f400640 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6bc5d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb06000e660 tx=0x7fb06000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d130ddfc79bbc78e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.746+0000 7fb058c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.747+0000 7fb058c00640 1 -- 172.21.15.114:0/876214515 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06001c060 con 0x56362d2e32f0 2024-09-21T06:54:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.747+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05c04e070 con 0x56362d2e32f0 2024-09-21T06:54:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.747+0000 7fb058c00640 1 -- 172.21.15.114:0/876214515 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb060009a80 con 0x56362d2e32f0 2024-09-21T06:54:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.748+0000 7fb058c00640 1 -- 172.21.15.114:0/876214515 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb060023050 con 0x56362d2e32f0 2024-09-21T06:54:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.748+0000 7fb07d400740 10 client.25802 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56362d3f0620 con 0x56362d2e32f0 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb058c00640 1 -- 172.21.15.114:0/876214515 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb05c04e070 con 0x56362d2e32f0 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb058c00640 1 -- 172.21.15.114:0/876214515 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb060015340 con 0x56362d2e32f0 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb07d400740 10 client.25802 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb07d400740 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb07d400740 20 client.25802 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.749+0000 7fb07d400740 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d2cad80 0x56362d6ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.750+0000 7fb07d400740 4 client.25802 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.750+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56362d628600 con 0x56362d2cad80 2024-09-21T06:54:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.750+0000 7fb06ea00640 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d2cad80 0x56362d6ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.751+0000 7fb06ea00640 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d2cad80 0x56362d6ce770 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93b1feccc704cb3 server_cookie=179a56464cc35dbe in_seq=0 out_seq=0 2024-09-21T06:54:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.751+0000 7fb058c00640 10 client.25802 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.752+0000 7fb058c00640 1 -- 172.21.15.114:0/876214515 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56362d628600 con 0x56362d2cad80 2024-09-21T06:54:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.752+0000 7fb058c00640 10 client.25802 handle_command_reply: tid=0 2024-09-21T06:54:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.752+0000 7fb07d400740 1 client.25802 shutdown 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 msgr2=0x56362d6bc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6bc5d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb06000e660 tx=0x7fb06000eb20 comp rx=0 tx=0).stop 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 shutdown_connections 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d2cad80 0x56362d6ce770 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2eb4d0 0x56362d6bcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 --2- 172.21.15.114:0/876214515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2e32f0 0x56362d6bc5d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 >> 172.21.15.114:0/876214515 conn(0x56362d300300 msgr2=0x56362d6b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb058c00640 0 client.25802 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.753+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 shutdown_connections 2024-09-21T06:54:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.754+0000 7fb07d400740 1 -- 172.21.15.114:0/876214515 wait complete. 2024-09-21T06:54:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.754+0000 7fb07d400740 20 client.25802 trim_cache size 0 max 16384 2024-09-21T06:54:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.784+0000 7fb07d400740 1 Processor -- start 2024-09-21T06:54:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.785+0000 7fb07d400740 1 -- start start 2024-09-21T06:54:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb07d400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 0x56362d6b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb07d400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb07d400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d30c1b0 0x56362d6c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb06f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 0x56362d6b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb06f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 0x56362d6b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:60636/0 (socket says 172.21.15.114:60636) 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb06f400640 1 -- 172.21.15.114:0/826583305 learned_addr learned my addr 172.21.15.114:0/826583305 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701b7160 con 0x56362d2eb4d0 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701b8ca0 con 0x56362d30c1b0 2024-09-21T06:54:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701b7830 con 0x56362d2e32f0 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.786+0000 7fb06ea00640 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06f400640 1 -- 172.21.15.114:0/826583305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d30c1b0 msgr2=0x56362d6c8750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06f400640 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d30c1b0 0x56362d6c8750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06f400640 1 -- 172.21.15.114:0/826583305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 msgr2=0x56362d6b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06f400640 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06f400640 1 -- 172.21.15.114:0/826583305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56362d6cb220 con 0x56362d2e32f0 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06ea00640 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.787+0000 7fb06f400640 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 0x56362d6b7d70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb060015710 tx=0x7fb060009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca4f9e753926483 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.788+0000 7fb05b400640 1 -- 172.21.15.114:0/826583305 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06002b030 con 0x56362d2e32f0 2024-09-21T06:54:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.788+0000 7fb05b400640 1 -- 172.21.15.114:0/826583305 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb060009d30 con 0x56362d2e32f0 2024-09-21T06:54:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.788+0000 7fb05b400640 1 -- 172.21.15.114:0/826583305 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060005040 con 0x56362d2e32f0 2024-09-21T06:54:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.789+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 msgr2=0x56362d6b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.789+0000 7fb07d400740 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 0x56362d6b7d70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb060015710 tx=0x7fb060009670 comp rx=0 tx=0).stop 2024-09-21T06:54:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 shutdown_connections 2024-09-21T06:54:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d30c1b0 0x56362d6c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d2eb4d0 0x56362d6b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 --2- 172.21.15.114:0/826583305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2e32f0 0x56362d6b7d70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 >> 172.21.15.114:0/826583305 conn(0x56362d300300 msgr2=0x56362d6b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 shutdown_connections 2024-09-21T06:54:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.790+0000 7fb07d400740 1 -- 172.21.15.114:0/826583305 wait complete. 2024-09-21T06:54:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.791+0000 7fb07d400740 1 -- start start 2024-09-21T06:54:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.792+0000 7fb07d400740 1 Processor -- start 2024-09-21T06:54:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.793+0000 7fb07d400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 0x56362d6c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.793+0000 7fb07d400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.793+0000 7fb06f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 0x56362d6c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.793+0000 7fb07d400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2a01c0 0x56362d6c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb06f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 0x56362d6c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33250/0 (socket says 172.21.15.114:33250) 2024-09-21T06:54:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb07d400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701b85d0 con 0x56362d30c1b0 2024-09-21T06:54:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb07d400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701b7f00 con 0x56362d2eb4d0 2024-09-21T06:54:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb07d400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb0701b5140 con 0x56362d2a01c0 2024-09-21T06:54:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb07d400740 10 client.0 _ll_register_callbacks cb 0x7fb06000c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb06f400640 1 -- 172.21.15.114:0/1220196154 learned_addr learned my addr 172.21.15.114:0/1220196154 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb06ea00640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.794+0000 7fb06fe00640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2a01c0 0x56362d6c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06fe00640 1 -- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 msgr2=0x56362d6c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06fe00640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 0x56362d6c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06fe00640 1 -- 172.21.15.114:0/1220196154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 msgr2=0x56362d6c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06fe00640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06fe00640 1 -- 172.21.15.114:0/1220196154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56362d6bef40 con 0x56362d2a01c0 2024-09-21T06:54:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06ea00640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.795+0000 7fb06f400640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 0x56362d6c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.796+0000 7fb06fe00640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2a01c0 0x56362d6c2880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb0680e5330 tx=0x7fb0680a9910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2733381c8e3b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.796+0000 7fb058c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.797+0000 7fb058c00640 1 -- 172.21.15.114:0/1220196154 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0680ad120 con 0x56362d2a01c0 2024-09-21T06:54:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.797+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb060009d30 con 0x56362d2a01c0 2024-09-21T06:54:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.797+0000 7fb058c00640 1 -- 172.21.15.114:0/1220196154 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb06800f800 con 0x56362d2a01c0 2024-09-21T06:54:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.798+0000 7fb058c00640 1 -- 172.21.15.114:0/1220196154 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0680ad4c0 con 0x56362d2a01c0 2024-09-21T06:54:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.798+0000 7fb07d400740 10 client.25804 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.798+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56362d738e00 con 0x56362d2a01c0 2024-09-21T06:54:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb058c00640 1 -- 172.21.15.114:0/1220196154 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb060009d30 con 0x56362d2a01c0 2024-09-21T06:54:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb058c00640 1 -- 172.21.15.114:0/1220196154 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fb0680aece0 con 0x56362d2a01c0 2024-09-21T06:54:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb07d400740 10 client.25804 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb07d400740 10 client.25804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb07d400740 20 client.25804 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb07d400740 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d739530 0x56362d73b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb07d400740 4 client.25804 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:54:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.799+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56362d6240b0 con 0x56362d739530 2024-09-21T06:54:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.800+0000 7fb06f400640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d739530 0x56362d73b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.800+0000 7fb06f400640 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d739530 0x56362d73b950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ecb557c53928ec server_cookie=1261756aaf41c4ca in_seq=0 out_seq=0 2024-09-21T06:54:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.800+0000 7fb058c00640 10 client.25804 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:42.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:42 smithi114 ceph-mon[30654]: pgmap v413: 129 pgs: 129 active+clean; 498 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 67 MiB/s wr, 389 op/s 2024-09-21T06:54:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 5.0 s, 2350.0 tps, lat 40.643 ms stddev 45.312 2024-09-21T06:54:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.852+0000 7fb058c00640 1 -- 172.21.15.114:0/1220196154 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56362d6240b0 con 0x56362d739530 2024-09-21T06:54:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.852+0000 7fb058c00640 10 client.25804 handle_command_reply: tid=0 2024-09-21T06:54:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.853+0000 7fb07d400740 1 client.25804 shutdown 2024-09-21T06:54:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2a01c0 msgr2=0x56362d6c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2a01c0 0x56362d6c2880 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb0680e5330 tx=0x7fb0680a9910 comp rx=0 tx=0).stop 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 shutdown_connections 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56362d739530 0x56362d73b950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56362d2a01c0 0x56362d6c2880 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56362d30c1b0 0x56362d6c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 --2- 172.21.15.114:0/1220196154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56362d2eb4d0 0x56362d6c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 >> 172.21.15.114:0/1220196154 conn(0x56362d300300 msgr2=0x56362d6c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.854+0000 7fb058c00640 0 client.25804 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.855+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 shutdown_connections 2024-09-21T06:54:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.855+0000 7fb07d400740 1 -- 172.21.15.114:0/1220196154 wait complete. 2024-09-21T06:54:42.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.855+0000 7fb07d400740 20 client.25804 trim_cache size 0 max 16384 2024-09-21T06:54:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:54:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:54:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:54:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:54:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.857+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0540b6630 msgr2=0x7fb0540b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.857+0000 7fb074c00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0540b6630 0x7fb0540b8b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb05c006010 tx=0x7fb05c03a000 comp rx=0 tx=0).stop 2024-09-21T06:54:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.857+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 msgr2=0x7fb0701a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.857+0000 7fb074c00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 0x7fb0701a4fa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb06800cce0 tx=0x7fb06800a620 comp rx=0 tx=0).stop 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 shutdown_connections 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb0540b6630 0x7fb0540b8b40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb07010bd90 0x7fb0701a4fa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb07010b440 0x7fb0701119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 --2- 172.21.15.114:0/297622896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb070105b10 0x7fb070111430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 >> 172.21.15.114:0/297622896 conn(0x7fb0701014f0 msgr2=0x7fb070107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 shutdown_connections 2024-09-21T06:54:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:42.858+0000 7fb074c00640 1 -- 172.21.15.114:0/297622896 wait complete. 2024-09-21T06:54:42.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:42 smithi193 ceph-mon[34601]: pgmap v413: 129 pgs: 129 active+clean; 498 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 67 MiB/s wr, 389 op/s 2024-09-21T06:54:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:42 smithi183 ceph-mon[34219]: pgmap v413: 129 pgs: 129 active+clean; 498 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 67 MiB/s wr, 389 op/s 2024-09-21T06:54:43.435 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:54:44.117 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:44.803 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:44 smithi114 ceph-mon[30654]: pgmap v414: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 MiB/s wr, 289 op/s 2024-09-21T06:54:44.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:44 smithi193 ceph-mon[34601]: pgmap v414: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 MiB/s wr, 289 op/s 2024-09-21T06:54:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 -- 172.21.15.114:0/1529186171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810b6f0 msgr2=0x7f8f4810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 --2- 172.21.15.114:0/1529186171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810b6f0 0x7f8f4810bb00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009900 tx=0x7f8f3002a400 comp rx=0 tx=0).stop 2024-09-21T06:54:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 -- 172.21.15.114:0/1529186171 shutdown_connections 2024-09-21T06:54:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 --2- 172.21.15.114:0/1529186171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f4810c040 0x7f8f48112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 --2- 172.21.15.114:0/1529186171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810b6f0 0x7f8f4810bb00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 --2- 172.21.15.114:0/1529186171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f48105db0 0x7f8f481061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.970+0000 7f8f4e000640 1 -- 172.21.15.114:0/1529186171 >> 172.21.15.114:0/1529186171 conn(0x7f8f481017c0 msgr2=0x7f8f48103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:44.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.971+0000 7f8f4e000640 1 -- 172.21.15.114:0/1529186171 shutdown_connections 2024-09-21T06:54:44.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.971+0000 7f8f4e000640 1 -- 172.21.15.114:0/1529186171 wait complete. 2024-09-21T06:54:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.973+0000 7f8f4e000640 1 Processor -- start 2024-09-21T06:54:44.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.973+0000 7f8f4e000640 1 -- start start 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f4e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f48105db0 0x7f8f4810fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f4e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f4810b6f0 0x7f8f48110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f4e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810c040 0x7f8f48110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f4e000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48115d00 con 0x7f8f48105db0 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f4e000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48115b80 con 0x7f8f4810b6f0 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f4e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48115e80 con 0x7f8f4810c040 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810c040 0x7f8f48110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810c040 0x7f8f48110790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38418/0 (socket says 172.21.15.114:38418) 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f47e00640 1 -- 172.21.15.114:0/360408341 learned_addr learned my addr 172.21.15.114:0/360408341 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f4810b6f0 0x7f8f48110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.974+0000 7f8f47400640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f48105db0 0x7f8f4810fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f3ea00640 1 -- 172.21.15.114:0/360408341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810c040 msgr2=0x7f8f48110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810c040 0x7f8f48110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f3ea00640 1 -- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f48105db0 msgr2=0x7f8f4810fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f48105db0 0x7f8f4810fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f3ea00640 1 -- 172.21.15.114:0/360408341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f481acb70 con 0x7f8f4810b6f0 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f4810b6f0 0x7f8f48110250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8f3002a920 tx=0x7f8f30004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.975+0000 7f8f44c00640 1 -- 172.21.15.114:0/360408341 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30035070 con 0x7f8f4810b6f0 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.976+0000 7f8f44c00640 1 -- 172.21.15.114:0/360408341 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f30050070 con 0x7f8f4810b6f0 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.976+0000 7f8f44c00640 1 -- 172.21.15.114:0/360408341 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3003bb00 con 0x7f8f4810b6f0 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.976+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4819b170 con 0x7f8f4810b6f0 2024-09-21T06:54:44.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.977+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4819b5d0 con 0x7f8f4810b6f0 2024-09-21T06:54:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.986+0000 7f8f44c00640 1 -- 172.21.15.114:0/360408341 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f8f300037f0 con 0x7f8f4810b6f0 2024-09-21T06:54:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.987+0000 7f8f54c6e740 1 Processor -- start 2024-09-21T06:54:44.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.987+0000 7f8f54c6e740 1 -- start start 2024-09-21T06:54:44.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a56da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645716871b0 0x564571a572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56457168f390 0x564571a57820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645719cdfb0 con 0x56457168f390 2024-09-21T06:54:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645719bd3e0 con 0x5645716b0070 2024-09-21T06:54:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5645719c11f0 con 0x5645716871b0 2024-09-21T06:54:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a56da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37016/0 (socket says 172.21.15.114:37016) 2024-09-21T06:54:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f47e00640 1 -- 172.21.15.114:0/3229711139 learned_addr learned my addr 172.21.15.114:0/3229711139 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.988+0000 7f8f47400640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645716871b0 0x564571a572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.989+0000 7f8f47400640 1 -- 172.21.15.114:0/3229711139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 msgr2=0x564571a56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.989+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56457168f390 0x564571a57820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.990+0000 7f8f47400640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a56da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.990+0000 7f8f47400640 1 -- 172.21.15.114:0/3229711139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56457168f390 msgr2=0x564571a57820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.990+0000 7f8f47400640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56457168f390 0x564571a57820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.990+0000 7f8f47400640 1 -- 172.21.15.114:0/3229711139 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564571987cd0 con 0x5645716871b0 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.990+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56457168f390 0x564571a57820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.990+0000 7f8f47e00640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.991+0000 7f8f44c00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8f2c0b6670 0x7f8f2c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.991+0000 7f8f47400640 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645716871b0 0x564571a572e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009f60 tx=0x7f8f38009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=843e0ab9796e5950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.991+0000 7f8f44c00640 1 -- 172.21.15.114:0/360408341 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8f30118dd0 con 0x7f8f4810b6f0 2024-09-21T06:54:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.991+0000 7f8f37e00640 1 -- 172.21.15.114:0/3229711139 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38011640 con 0x5645716871b0 2024-09-21T06:54:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.992+0000 7f8f37e00640 1 -- 172.21.15.114:0/3229711139 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f38011c90 con 0x5645716871b0 2024-09-21T06:54:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.992+0000 7f8f37e00640 1 -- 172.21.15.114:0/3229711139 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38010ca0 con 0x5645716871b0 2024-09-21T06:54:44.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3229711139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645716871b0 msgr2=0x564571a572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:44.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645716871b0 0x564571a572e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009f60 tx=0x7f8f38009f90 comp rx=0 tx=0).stop 2024-09-21T06:54:44.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3229711139 shutdown_connections 2024-09-21T06:54:44.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56457168f390 0x564571a57820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645716871b0 0x564571a572e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3229711139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3229711139 >> 172.21.15.114:0/3229711139 conn(0x5645716a4460 msgr2=0x564571a54c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.993+0000 7f8f47e00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8f2c0b6670 0x7f8f2c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.994+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3229711139 shutdown_connections 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.994+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3229711139 wait complete. 2024-09-21T06:54:44.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.995+0000 7f8f54c6e740 1 -- start start 2024-09-21T06:54:44.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.995+0000 7f8f54c6e740 1 Processor -- start 2024-09-21T06:54:44.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.996+0000 7f8f47e00640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8f2c0b6670 0x7f8f2c0b8b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005f60 tx=0x7f8f40005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:44.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.996+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 0x564571a625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.996+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a62b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56457181e700 con 0x5645716871b0 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564571958580 con 0x5645716b0070 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564571a65580 con 0x56457168f390 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f54c6e740 10 client.0 _ll_register_callbacks cb 0x564571968470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f47400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 0x564571a625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f47400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 0x564571a625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52398/0 (socket says 172.21.15.114:52398) 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f47e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:44.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.997+0000 7f8f47400640 1 -- 172.21.15.114:0/3155349700 learned_addr learned my addr 172.21.15.114:0/3155349700 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.998+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a62b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.998+0000 7f8f47e00640 1 -- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 msgr2=0x564571a62b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.998+0000 7f8f47e00640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a62b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.998+0000 7f8f47e00640 1 -- 172.21.15.114:0/3155349700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 msgr2=0x564571a625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.999+0000 7f8f47e00640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 0x564571a625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.999+0000 7f8f47e00640 1 -- 172.21.15.114:0/3155349700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457171b650 con 0x5645716b0070 2024-09-21T06:54:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.999+0000 7f8f47400640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 0x564571a625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:45.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.999+0000 7f8f47e00640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a6e870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005f10 tx=0x7f8f40004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8510f1a0252913 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:44.999+0000 7f8f34c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.000+0000 7f8f34c00640 1 -- 172.21.15.114:0/3155349700 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40005020 con 0x5645716b0070 2024-09-21T06:54:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.000+0000 7f8f34c00640 1 -- 172.21.15.114:0/3155349700 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8f40018030 con 0x5645716b0070 2024-09-21T06:54:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.000+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56457195ab90 con 0x5645716b0070 2024-09-21T06:54:45.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.001+0000 7f8f34c00640 1 -- 172.21.15.114:0/3155349700 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f40034050 con 0x5645716b0070 2024-09-21T06:54:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.002+0000 7f8f54c6e740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:45.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.002+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564571975950 con 0x5645716b0070 2024-09-21T06:54:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.002+0000 7f8f34c00640 1 -- 172.21.15.114:0/3155349700 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56457195ab90 con 0x5645716b0070 2024-09-21T06:54:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.002+0000 7f8f34c00640 1 -- 172.21.15.114:0/3155349700 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8f40029420 con 0x5645716b0070 2024-09-21T06:54:45.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f54c6e740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:45.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f54c6e740 10 client.25817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:54:45.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f54c6e740 20 client.25817 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:45.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56457166e9f0 0x564571a74800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f54c6e740 4 client.25817 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:45.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645719cff00 con 0x56457166e9f0 2024-09-21T06:54:45.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.003+0000 7f8f47400640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56457166e9f0 0x564571a74800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.004+0000 7f8f47400640 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56457166e9f0 0x564571a74800 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2efffcf1146f242c server_cookie=e1de3a0781f2567d in_seq=0 out_seq=0 2024-09-21T06:54:45.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.004+0000 7f8f34c00640 10 client.25817 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:45.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.005+0000 7f8f34c00640 1 -- 172.21.15.114:0/3155349700 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645719cff00 con 0x56457166e9f0 2024-09-21T06:54:45.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.005+0000 7f8f34c00640 10 client.25817 handle_command_reply: tid=0 2024-09-21T06:54:45.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.005+0000 7f8f54c6e740 1 client.25817 shutdown 2024-09-21T06:54:45.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.007+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 msgr2=0x564571a6e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:45.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.007+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a6e870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005f10 tx=0x7f8f40004730 comp rx=0 tx=0).stop 2024-09-21T06:54:45.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 shutdown_connections 2024-09-21T06:54:45.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56457166e9f0 0x564571a74800 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716b0070 0x564571a6e870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a62b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/3155349700 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716871b0 0x564571a625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 >> 172.21.15.114:0/3155349700 conn(0x5645716a4460 msgr2=0x564571a59f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:45.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f34c00640 0 client.25817 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:45.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 shutdown_connections 2024-09-21T06:54:45.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.008+0000 7f8f54c6e740 1 -- 172.21.15.114:0/3155349700 wait complete. 2024-09-21T06:54:45.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.009+0000 7f8f54c6e740 20 client.25817 trim_cache size 0 max 16384 2024-09-21T06:54:45.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.034+0000 7f8f54c6e740 1 Processor -- start 2024-09-21T06:54:45.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.034+0000 7f8f54c6e740 1 -- start start 2024-09-21T06:54:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a5dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571a5f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481aec30 con 0x5645716b0070 2024-09-21T06:54:45.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481b0770 con 0x5645716871b0 2024-09-21T06:54:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481af300 con 0x56457168f390 2024-09-21T06:54:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f3ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f47400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a5dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f3ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38452/0 (socket says 172.21.15.114:38452) 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f47400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a5dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37028/0 (socket says 172.21.15.114:37028) 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571a5f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 -- 172.21.15.114:0/1420901583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 msgr2=0x564571a584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 -- 172.21.15.114:0/1420901583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 msgr2=0x564571a5dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a5dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 -- 172.21.15.114:0/1420901583 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564571a71290 con 0x5645716b0070 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.036+0000 7f8f47e00640 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571a5f790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8f40029870 tx=0x7f8f40029640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a4d9f71f32b7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.035+0000 7f8f3ea00640 1 -- 172.21.15.114:0/1420901583 learned_addr learned my addr 172.21.15.114:0/1420901583 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.037+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.037+0000 7f8f47400640 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a5dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.037+0000 7f8f37400640 1 -- 172.21.15.114:0/1420901583 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4003c070 con 0x5645716b0070 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.037+0000 7f8f37400640 1 -- 172.21.15.114:0/1420901583 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f400054d0 con 0x5645716b0070 2024-09-21T06:54:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.037+0000 7f8f37400640 1 -- 172.21.15.114:0/1420901583 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40004a00 con 0x5645716b0070 2024-09-21T06:54:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.038+0000 7f8f54c6e740 1 -- 172.21.15.114:0/1420901583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 msgr2=0x564571a5f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.038+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571a5f790 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8f40029870 tx=0x7f8f40029640 comp rx=0 tx=0).stop 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 -- 172.21.15.114:0/1420901583 shutdown_connections 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571a5f790 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/1420901583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a5dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 -- 172.21.15.114:0/1420901583 >> 172.21.15.114:0/1420901583 conn(0x5645716a4460 msgr2=0x564571a57dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 -- 172.21.15.114:0/1420901583 shutdown_connections 2024-09-21T06:54:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.039+0000 7f8f54c6e740 1 -- 172.21.15.114:0/1420901583 wait complete. 2024-09-21T06:54:45.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.040+0000 7f8f54c6e740 1 -- start start 2024-09-21T06:54:45.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.040+0000 7f8f54c6e740 1 Processor -- start 2024-09-21T06:54:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.041+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a66a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.041+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a66fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f54c6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571ade200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f47400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a66a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f47400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a66a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37038/0 (socket says 172.21.15.114:37038) 2024-09-21T06:54:45.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481b00a0 con 0x5645716b0070 2024-09-21T06:54:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481af9d0 con 0x5645716871b0 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f54c6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48111010 con 0x56457168f390 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f54c6e740 10 client.0 _ll_register_callbacks cb 0x7f8f4001e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.042+0000 7f8f47400640 1 -- 172.21.15.114:0/2450086638 learned_addr learned my addr 172.21.15.114:0/2450086638 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f47400640 1 -- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 msgr2=0x564571a66fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f47400640 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a66fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f47400640 1 -- 172.21.15.114:0/2450086638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 msgr2=0x564571ade200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f47400640 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571ade200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f47400640 1 -- 172.21.15.114:0/2450086638 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564571a67800 con 0x5645716871b0 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f47400640 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a66a70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f38010910 tx=0x7f8f380106d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55a34b2f50e9223 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f34c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:45.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f34c00640 1 -- 172.21.15.114:0/2450086638 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f380103d0 con 0x5645716871b0 2024-09-21T06:54:45.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.043+0000 7f8f34c00640 1 -- 172.21.15.114:0/2450086638 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8f38010b70 con 0x5645716871b0 2024-09-21T06:54:45.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.044+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f400054d0 con 0x5645716871b0 2024-09-21T06:54:45.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.044+0000 7f8f34c00640 1 -- 172.21.15.114:0/2450086638 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f38010d70 con 0x5645716871b0 2024-09-21T06:54:45.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.044+0000 7f8f54c6e740 10 client.25820 fetch_fsmap learned FSMap version 29 2024-09-21T06:54:45.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.044+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564571adf6f0 con 0x5645716871b0 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f34c00640 1 -- 172.21.15.114:0/2450086638 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f400054d0 con 0x5645716871b0 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f34c00640 1 -- 172.21.15.114:0/2450086638 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8f3802d020 con 0x5645716871b0 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f54c6e740 10 client.25820 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f54c6e740 10 client.25820 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f54c6e740 20 client.25820 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564571643da0 0x564571ae1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f54c6e740 4 client.25820 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645719cb9b0 con 0x564571643da0 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.045+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564571643da0 0x564571ae1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.046+0000 7f8f34c00640 10 client.25820 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.046+0000 7f8f3ea00640 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564571643da0 0x564571ae1c00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5efb6df11df1c456 server_cookie=ce55b9420297b737 in_seq=0 out_seq=0 2024-09-21T06:54:45.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.046+0000 7f8f34c00640 1 -- 172.21.15.114:0/2450086638 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645719cb9b0 con 0x564571643da0 2024-09-21T06:54:45.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.046+0000 7f8f34c00640 10 client.25820 handle_command_reply: tid=0 2024-09-21T06:54:45.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.046+0000 7f8f54c6e740 1 client.25820 shutdown 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 msgr2=0x564571a66a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a66a70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f38010910 tx=0x7f8f380106d0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 shutdown_connections 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564571643da0 0x564571ae1c00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f34c00640 0 client.25820 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645716b0070 0x564571ade200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56457168f390 0x564571a66fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 --2- 172.21.15.114:0/2450086638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645716871b0 0x564571a66a70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 >> 172.21.15.114:0/2450086638 conn(0x5645716a4460 msgr2=0x564571a64410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:45.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 shutdown_connections 2024-09-21T06:54:45.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.048+0000 7f8f54c6e740 1 -- 172.21.15.114:0/2450086638 wait complete. 2024-09-21T06:54:45.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.049+0000 7f8f54c6e740 20 client.25820 trim_cache size 0 max 16384 2024-09-21T06:54:45.051 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:54:45.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.050+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8f2c0b6670 msgr2=0x7f8f2c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:45.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.050+0000 7f8f4e000640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8f2c0b6670 0x7f8f2c0b8b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8f40005f60 tx=0x7f8f40005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:45.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.050+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f4810b6f0 msgr2=0x7f8f48110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:45.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.050+0000 7f8f4e000640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f4810b6f0 0x7f8f48110250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8f3002a920 tx=0x7f8f30004060 comp rx=0 tx=0).stop 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.051+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 shutdown_connections 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.051+0000 7f8f4e000640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f4810c040 0x7f8f48110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.051+0000 7f8f4e000640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8f2c0b6670 0x7f8f2c0b8b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.051+0000 7f8f4e000640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f4810b6f0 0x7f8f48110250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.051+0000 7f8f4e000640 1 --2- 172.21.15.114:0/360408341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8f48105db0 0x7f8f4810fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.051+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 >> 172.21.15.114:0/360408341 conn(0x7f8f481017c0 msgr2=0x7f8f48109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.052+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 shutdown_connections 2024-09-21T06:54:45.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:45.052+0000 7f8f4e000640 1 -- 172.21.15.114:0/360408341 wait complete. 2024-09-21T06:54:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:44 smithi183 ceph-mon[34219]: pgmap v414: 129 pgs: 129 active+clean; 630 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 59 MiB/s wr, 289 op/s 2024-09-21T06:54:46.157 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-21T06:54:46.158 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:54:46.693 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:54:46.807 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:46 smithi193 ceph-mon[34601]: pgmap v415: 129 pgs: 129 active+clean; 646 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 MiB/s wr, 141 op/s 2024-09-21T06:54:46.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:46 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:46.987 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:46 smithi114 ceph-mon[30654]: pgmap v415: 129 pgs: 129 active+clean; 646 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 MiB/s wr, 141 op/s 2024-09-21T06:54:46.987 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:46 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:46 smithi183 ceph-mon[34219]: pgmap v415: 129 pgs: 129 active+clean; 646 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 42 MiB/s wr, 141 op/s 2024-09-21T06:54:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:46 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:47.639 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:47.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.685+0000 7f10dde00640 1 -- 172.21.15.114:0/1067904691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 msgr2=0x7f10d009a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:47.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.685+0000 7f10dde00640 1 --2- 172.21.15.114:0/1067904691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 0x7f10d009a680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f10d8066480 tx=0x7f10d80981a0 comp rx=0 tx=0).stop 2024-09-21T06:54:47.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.688+0000 7f10dde00640 1 -- 172.21.15.114:0/1067904691 shutdown_connections 2024-09-21T06:54:47.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.688+0000 7f10dde00640 1 --2- 172.21.15.114:0/1067904691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 0x7f10d009a680 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:47.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.688+0000 7f10dde00640 1 --2- 172.21.15.114:0/1067904691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 0x7f10d0099be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:47.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.688+0000 7f10dde00640 1 --2- 172.21.15.114:0/1067904691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 0x7f10d0095170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:47.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.688+0000 7f10dde00640 1 -- 172.21.15.114:0/1067904691 >> 172.21.15.114:0/1067904691 conn(0x7f10d000cd20 msgr2=0x7f10d000d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:47.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.689+0000 7f10dde00640 1 -- 172.21.15.114:0/1067904691 shutdown_connections 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.690+0000 7f10dde00640 1 -- 172.21.15.114:0/1067904691 wait complete. 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.692+0000 7f10dde00640 1 Processor -- start 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.692+0000 7f10dde00640 1 -- start start 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10dde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 0x7f10d0143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10dde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 0x7f10d0144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10dde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 0x7f10d0149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10dde00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10d00ac5b0 con 0x7f10d009a1a0 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10dde00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10d00ac430 con 0x7f10d00956b0 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10dde00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10d00ac730 con 0x7f10d0094d60 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10d7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 0x7f10d0144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10d7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 0x7f10d0144590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37052/0 (socket says 172.21.15.114:37052) 2024-09-21T06:54:47.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.693+0000 7f10d7e00640 1 -- 172.21.15.114:0/3447638454 learned_addr learned my addr 172.21.15.114:0/3447638454 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.694+0000 7f10dca00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 0x7f10d0143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.695+0000 7f10dca00640 1 -- 172.21.15.114:0/3447638454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 msgr2=0x7f10d0144590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.695+0000 7f10dca00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 0x7f10d0144590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.695+0000 7f10dca00640 1 -- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 msgr2=0x7f10d0149830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.695+0000 7f10dca00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 0x7f10d0149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:47.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.695+0000 7f10dca00640 1 -- 172.21.15.114:0/3447638454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d0149ea0 con 0x7f10d0094d60 2024-09-21T06:54:47.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.698+0000 7f10dca00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 0x7f10d0143ff0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c970 tx=0x7f10cc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:47.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.699+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc007bb0 con 0x7f10d0094d60 2024-09-21T06:54:47.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.699+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc007d50 con 0x7f10d0094d60 2024-09-21T06:54:47.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.699+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc018780 con 0x7f10d0094d60 2024-09-21T06:54:47.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.699+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d014a190 con 0x7f10d0094d60 2024-09-21T06:54:47.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.700+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d01509b0 con 0x7f10d0094d60 2024-09-21T06:54:47.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.706+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d0099be0 con 0x7f10d0094d60 2024-09-21T06:54:47.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.709+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f10cc00f450 con 0x7f10d0094d60 2024-09-21T06:54:47.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.712+0000 7f10d5600640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10c00b6770 0x7f10c00b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:47.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.712+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f10cc0e7860 con 0x7f10d0094d60 2024-09-21T06:54:47.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.713+0000 7f10d7e00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10c00b6770 0x7f10c00b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.715+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10cc014030 con 0x7f10d0094d60 2024-09-21T06:54:47.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:47.716+0000 7f10d7e00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10c00b6770 0x7f10c00b8c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f10d0094930 tx=0x7f10c4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 10.0 s, 2188.3 tps, lat 45.936 ms stddev 52.345 2024-09-21T06:54:48.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.161+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10d0150d30 con 0x7f10d0094d60 2024-09-21T06:54:48.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.162+0000 7f10d5600640 1 -- 172.21.15.114:0/3447638454 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11992 (secure 0 0 0) 0x7f10cc0adb80 con 0x7f10d0094d60 2024-09-21T06:54:48.168 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:54:48.168 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":30,"btime":"2024-09-21T06:54:45:621706+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:45.608923+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:54:48.168 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 30 2024-09-21T06:54:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10c00b6770 msgr2=0x7f10c00b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10c00b6770 0x7f10c00b8c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f10d0094930 tx=0x7f10c4005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 msgr2=0x7f10d0143ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 0x7f10d0143ff0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00c970 tx=0x7f10cc00ce30 comp rx=0 tx=0).stop 2024-09-21T06:54:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 shutdown_connections 2024-09-21T06:54:48.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10d009a1a0 0x7f10d0149830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10c00b6770 0x7f10c00b8c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10d00956b0 0x7f10d0144590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 --2- 172.21.15.114:0/3447638454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10d0094d60 0x7f10d0143ff0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.168+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 >> 172.21.15.114:0/3447638454 conn(0x7f10d000cd20 msgr2=0x7f10d0097bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.169+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 shutdown_connections 2024-09-21T06:54:48.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.169+0000 7f10dde00640 1 -- 172.21.15.114:0/3447638454 wait complete. 2024-09-21T06:54:48.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 -- 172.21.15.114:0/2347658424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 msgr2=0x7f667c116a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 --2- 172.21.15.114:0/2347658424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c116a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6674009c10 tx=0x7f667402a710 comp rx=0 tx=0).stop 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 -- 172.21.15.114:0/2347658424 shutdown_connections 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 --2- 172.21.15.114:0/2347658424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c116a90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 --2- 172.21.15.114:0/2347658424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 --2- 172.21.15.114:0/2347658424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 0x7f667c10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 -- 172.21.15.114:0/2347658424 >> 172.21.15.114:0/2347658424 conn(0x7f667c079420 msgr2=0x7f667c07b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 -- 172.21.15.114:0/2347658424 shutdown_connections 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.507+0000 7f6680a00640 1 -- 172.21.15.114:0/2347658424 wait complete. 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.509+0000 7f6680a00640 1 Processor -- start 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.509+0000 7f6680a00640 1 -- start start 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.510+0000 7f6680a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 0x7f667c074e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.510+0000 7f6680a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.510+0000 7f6680a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.510+0000 7f667b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 0x7f667c074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.511+0000 7f667b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 0x7f667c074e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37070/0 (socket says 172.21.15.114:37070) 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.512+0000 7f667aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.512+0000 7f667aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c06e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52436/0 (socket says 172.21.15.114:52436) 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.512+0000 7f667be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.512+0000 7f667be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c06e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38478/0 (socket says 172.21.15.114:38478) 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.513+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f667c119ed0 con 0x7f667c10c6b0 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.513+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f667c119d50 con 0x7f667c10a490 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.513+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f667c11a050 con 0x7f667c10d000 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.513+0000 7f667b400640 1 -- 172.21.15.114:0/1000074791 learned_addr learned my addr 172.21.15.114:0/1000074791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.514+0000 7f667be00640 1 -- 172.21.15.114:0/1000074791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 msgr2=0x7f667c074e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.514+0000 7f667be00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 0x7f667c074e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.514+0000 7f667be00640 1 -- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 msgr2=0x7f667c06e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.514+0000 7f667be00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c06e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.514+0000 7f667aa00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.515+0000 7f667be00640 1 -- 172.21.15.114:0/1000074791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c06ee40 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.516+0000 7f667be00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c06e6b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6674009b30 tx=0x7f6674030b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.516+0000 7f6673e00640 1 -- 172.21.15.114:0/1000074791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674041020 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.517+0000 7f6673e00640 1 -- 172.21.15.114:0/1000074791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6674030d60 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.517+0000 7f6673e00640 1 -- 172.21.15.114:0/1000074791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674036040 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.517+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c073ad0 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.517+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c073f30 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.527+0000 7f6688cca740 1 Processor -- start 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.528+0000 7f6673e00640 1 -- 172.21.15.114:0/1000074791 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f6674044020 con 0x7f667c10d000 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.528+0000 7f6688cca740 1 -- start start 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.530+0000 7f6688cca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x560553492d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.530+0000 7f6688cca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553493270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.530+0000 7f6688cca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530cb390 0x5605534937b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.530+0000 7f6688cca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560553409fb0 con 0x5605530ec070 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.530+0000 7f6688cca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605533f93e0 con 0x5605530cb390 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f6688cca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605533fd1f0 con 0x5605530c31b0 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553493270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553493270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38492/0 (socket says 172.21.15.114:38492) 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 -- 172.21.15.114:0/2264905346 learned_addr learned my addr 172.21.15.114:0/2264905346 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 -- 172.21.15.114:0/2264905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530cb390 msgr2=0x5605534937b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530cb390 0x5605534937b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 -- 172.21.15.114:0/2264905346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 msgr2=0x560553492d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x560553492d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f667b400640 1 -- 172.21.15.114:0/2264905346 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605533c3cd0 con 0x5605530c31b0 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.531+0000 7f6673e00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66600b6670 0x7f66600b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.532+0000 7f6673e00640 1 -- 172.21.15.114:0/1000074791 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f66741081b0 con 0x7f667c10d000 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.532+0000 7f667b400640 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553493270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f666c009db0 tx=0x7f666c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e953ef2f65d9da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.533+0000 7f6667e00640 1 -- 172.21.15.114:0/2264905346 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c01e070 con 0x5605530c31b0 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.533+0000 7f667aa00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66600b6670 0x7f66600b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.534+0000 7f6667e00640 1 -- 172.21.15.114:0/2264905346 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f666c019ae0 con 0x5605530c31b0 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.534+0000 7f6667e00640 1 -- 172.21.15.114:0/2264905346 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c00ea20 con 0x5605530c31b0 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.535+0000 7f6688cca740 1 -- 172.21.15.114:0/2264905346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 msgr2=0x560553493270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.535+0000 7f6688cca740 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553493270 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f666c009db0 tx=0x7f666c011810 comp rx=0 tx=0).stop 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.536+0000 7f6688cca740 1 -- 172.21.15.114:0/2264905346 shutdown_connections 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.536+0000 7f6688cca740 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530cb390 0x5605534937b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.536+0000 7f6688cca740 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553493270 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.536+0000 7f6688cca740 1 --2- 172.21.15.114:0/2264905346 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x560553492d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.536+0000 7f6688cca740 1 -- 172.21.15.114:0/2264905346 >> 172.21.15.114:0/2264905346 conn(0x5605530e0460 msgr2=0x560553490bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.537+0000 7f667aa00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66600b6670 0x7f66600b8b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f667c07ccd0 tx=0x7f6668005d10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:48.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.537+0000 7f6688cca740 1 -- 172.21.15.114:0/2264905346 shutdown_connections 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.537+0000 7f6688cca740 1 -- 172.21.15.114:0/2264905346 wait complete. 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.538+0000 7f6688cca740 1 -- start start 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.538+0000 7f6688cca740 1 Processor -- start 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.539+0000 7f6688cca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x56055349e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.540+0000 7f6688cca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x56055349ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.540+0000 7f6688cca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534aa800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f6688cca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56055325a700 con 0x5605530ec070 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f6688cca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560553394580 con 0x5605530c31b0 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f6688cca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605534a1510 con 0x5605530cb390 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f6688cca740 10 client.0 _ll_register_callbacks cb 0x5605533a4470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f667aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x56055349ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.540+0000 7f667b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x56055349e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f667b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x56055349e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37082/0 (socket says 172.21.15.114:37082) 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f667b400640 1 -- 172.21.15.114:0/845946794 learned_addr learned my addr 172.21.15.114:0/845946794 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:48.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.541+0000 7f667be00640 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.542+0000 7f667b400640 1 -- 172.21.15.114:0/845946794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 msgr2=0x56055349ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.542+0000 7f667b400640 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x56055349ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.542+0000 7f667b400640 1 -- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 msgr2=0x5605534aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.542+0000 7f667b400640 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.542+0000 7f667b400640 1 -- 172.21.15.114:0/845946794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560553157650 con 0x5605530c31b0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.543+0000 7f667b400640 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x56055349e8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f666c009fa0 tx=0x7f666c0055d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1bfced8884f233 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.544+0000 7f6664c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.544+0000 7f6664c00640 1 -- 172.21.15.114:0/845946794 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c026020 con 0x5605530c31b0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.544+0000 7f6664c00640 1 -- 172.21.15.114:0/845946794 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f666c00eb70 con 0x5605530c31b0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.545+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560553396b90 con 0x5605530c31b0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.546+0000 7f6664c00640 1 -- 172.21.15.114:0/845946794 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f666c01e040 con 0x5605530c31b0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.548+0000 7f6688cca740 10 client.25835 fetch_fsmap learned FSMap version 30 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.548+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5605533b1950 con 0x5605530c31b0 2024-09-21T06:54:48.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6664c00640 1 -- 172.21.15.114:0/845946794 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560553396b90 con 0x5605530c31b0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6664c00640 1 -- 172.21.15.114:0/845946794 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f666c019430 con 0x5605530c31b0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6688cca740 10 client.25835 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6688cca740 10 client.25835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6688cca740 20 client.25835 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6688cca740 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5605530aa9f0 0x5605534b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6688cca740 4 client.25835 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.549+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055340bf00 con 0x5605530aa9f0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.550+0000 7f667aa00640 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5605530aa9f0 0x5605534b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.552+0000 7f6664c00640 10 client.25835 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.553+0000 7f667aa00640 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5605530aa9f0 0x5605534b06c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f796d8dc42975557 server_cookie=2602f0057dacdc61 in_seq=0 out_seq=0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.555+0000 7f6664c00640 1 -- 172.21.15.114:0/845946794 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56055340bf00 con 0x5605530aa9f0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.557+0000 7f6664c00640 10 client.25835 handle_command_reply: tid=0 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.557+0000 7f6688cca740 1 client.25835 shutdown 2024-09-21T06:54:48.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.560+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 msgr2=0x56055349e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.560+0000 7f6688cca740 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x56055349e8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f666c009fa0 tx=0x7f666c0055d0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 shutdown_connections 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6688cca740 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5605530aa9f0 0x5605534b06c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6688cca740 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6688cca740 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x56055349ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6688cca740 1 --2- 172.21.15.114:0/845946794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x56055349e8f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 >> 172.21.15.114:0/845946794 conn(0x5605530e0460 msgr2=0x560553495f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.561+0000 7f6664c00640 0 client.25835 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.562+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 shutdown_connections 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.562+0000 7f6688cca740 1 -- 172.21.15.114:0/845946794 wait complete. 2024-09-21T06:54:48.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.563+0000 7f6688cca740 20 client.25835 trim_cache size 0 max 16384 2024-09-21T06:54:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.596+0000 7f6688cca740 1 Processor -- start 2024-09-21T06:54:48.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 -- start start 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553497e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530cb390 0x560553498380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 0x560553490510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1bfb80 con 0x5605530cb390 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1c1720 con 0x5605530ec070 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.597+0000 7f6688cca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1c0250 con 0x5605530c31b0 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 0x560553490510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 0x560553490510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37084/0 (socket says 172.21.15.114:37084) 2024-09-21T06:54:48.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 -- 172.21.15.114:0/562105766 learned_addr learned my addr 172.21.15.114:0/562105766 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667aa00640 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530cb390 0x560553498380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 -- 172.21.15.114:0/562105766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 msgr2=0x560553497e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:54:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553497e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 -- 172.21.15.114:0/562105766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530cb390 msgr2=0x560553498380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530cb390 0x560553498380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.598+0000 7f667be00640 1 -- 172.21.15.114:0/562105766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055349f290 con 0x5605530ec070 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.599+0000 7f667be00640 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 0x560553490510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6674105700 tx=0x7f667402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd6dad3b615d5bca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.599+0000 7f6667400640 1 -- 172.21.15.114:0/562105766 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740ce450 con 0x5605530ec070 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.599+0000 7f6667400640 1 -- 172.21.15.114:0/562105766 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66740d2830 con 0x5605530ec070 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.599+0000 7f6667400640 1 -- 172.21.15.114:0/562105766 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740d5bb0 con 0x5605530ec070 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 -- 172.21.15.114:0/562105766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 msgr2=0x560553490510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 0x560553490510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6674105700 tx=0x7f667402a710 comp rx=0 tx=0).stop 2024-09-21T06:54:48.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 -- 172.21.15.114:0/562105766 shutdown_connections 2024-09-21T06:54:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530ec070 0x560553490510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530cb390 0x560553498380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 --2- 172.21.15.114:0/562105766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530c31b0 0x560553497e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.601+0000 7f6688cca740 1 -- 172.21.15.114:0/562105766 >> 172.21.15.114:0/562105766 conn(0x5605530e0460 msgr2=0x56055349cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.602+0000 7f6688cca740 1 -- 172.21.15.114:0/562105766 shutdown_connections 2024-09-21T06:54:48.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.603+0000 7f6688cca740 1 -- 172.21.15.114:0/562105766 wait complete. 2024-09-21T06:54:48.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.603+0000 7f6688cca740 1 -- start start 2024-09-21T06:54:48.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.604+0000 7f6688cca740 1 Processor -- start 2024-09-21T06:54:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.604+0000 7f6688cca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x5605534a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.604+0000 7f6688cca740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x5605534a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f6688cca740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f6688cca740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1c0ff0 con 0x5605530ec070 2024-09-21T06:54:48.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f6688cca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1c0920 con 0x5605530c31b0 2024-09-21T06:54:48.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f6688cca740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1bdb60 con 0x5605530cb390 2024-09-21T06:54:48.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f6688cca740 10 client.0 _ll_register_callbacks cb 0x7f666c005470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:48.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f667aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x5605534a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f667b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x5605534a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f667b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x5605534a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37100/0 (socket says 172.21.15.114:37100) 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f667be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f667b400640 1 -- 172.21.15.114:0/3491095571 learned_addr learned my addr 172.21.15.114:0/3491095571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.605+0000 7f667aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x5605534a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38528/0 (socket says 172.21.15.114:38528) 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.606+0000 7f667be00640 1 -- 172.21.15.114:0/3491095571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 msgr2=0x5605534a70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.606+0000 7f667be00640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x5605534a70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.606+0000 7f667be00640 1 -- 172.21.15.114:0/3491095571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 msgr2=0x5605534a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.606+0000 7f667be00640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x5605534a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.606+0000 7f667be00640 1 -- 172.21.15.114:0/3491095571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605534a1fc0 con 0x5605530ec070 2024-09-21T06:54:48.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.607+0000 7f667aa00640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x5605534a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:48.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.607+0000 7f667b400640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x5605534a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:48.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.607+0000 7f667be00640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534a1830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6674105750 tx=0x7f66740d7510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463c44fb24b9b032 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:48.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.607+0000 7f6664c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:48.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.608+0000 7f6664c00640 1 -- 172.21.15.114:0/3491095571 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66740b9020 con 0x5605530ec070 2024-09-21T06:54:48.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.608+0000 7f6664c00640 1 -- 172.21.15.114:0/3491095571 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f66740cd1b0 con 0x5605530ec070 2024-09-21T06:54:48.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.608+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66740d2830 con 0x5605530ec070 2024-09-21T06:54:48.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.608+0000 7f6664c00640 1 -- 172.21.15.114:0/3491095571 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f66740d7770 con 0x5605530ec070 2024-09-21T06:54:48.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.608+0000 7f6688cca740 10 client.15822 fetch_fsmap learned FSMap version 30 2024-09-21T06:54:48.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.608+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56055351ba10 con 0x5605530ec070 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6664c00640 1 -- 172.21.15.114:0/3491095571 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66740d2830 con 0x5605530ec070 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6664c00640 1 -- 172.21.15.114:0/3491095571 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6674105ce0 con 0x5605530ec070 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6688cca740 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6688cca740 10 client.15822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6688cca740 20 client.15822 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6688cca740 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56055307fda0 0x56055351e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6688cca740 4 client.15822 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:54:48.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.609+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605534079b0 con 0x56055307fda0 2024-09-21T06:54:48.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.610+0000 7f667b400640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56055307fda0 0x56055351e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:48.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.610+0000 7f6664c00640 10 client.15822 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:48.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.610+0000 7f667b400640 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56055307fda0 0x56055351e0f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f303c7012e126cb server_cookie=1320130e710642f6 in_seq=0 out_seq=0 2024-09-21T06:54:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.611+0000 7f6664c00640 1 -- 172.21.15.114:0/3491095571 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605534079b0 con 0x56055307fda0 2024-09-21T06:54:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.611+0000 7f6664c00640 10 client.15822 handle_command_reply: tid=0 2024-09-21T06:54:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.611+0000 7f6688cca740 1 client.15822 shutdown 2024-09-21T06:54:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.611+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 msgr2=0x5605534a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.611+0000 7f6688cca740 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534a1830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6674105750 tx=0x7f66740d7510 comp rx=0 tx=0).stop 2024-09-21T06:54:48.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 shutdown_connections 2024-09-21T06:54:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56055307fda0 0x56055351e0f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605530ec070 0x5605534a1830 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605530cb390 0x5605534a70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 --2- 172.21.15.114:0/3491095571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605530c31b0 0x5605534a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 >> 172.21.15.114:0/3491095571 conn(0x5605530e0460 msgr2=0x560553497e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6664c00640 0 client.15822 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:54:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 shutdown_connections 2024-09-21T06:54:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.612+0000 7f6688cca740 1 -- 172.21.15.114:0/3491095571 wait complete. 2024-09-21T06:54:48.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.613+0000 7f6688cca740 20 client.15822 trim_cache size 0 max 16384 2024-09-21T06:54:48.615 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:54:48.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66600b6670 msgr2=0x7f66600b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66600b6670 0x7f66600b8b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f667c07ccd0 tx=0x7f6668005d10 comp rx=0 tx=0).stop 2024-09-21T06:54:48.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 msgr2=0x7f667c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c06e6b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6674009b30 tx=0x7f6674030b70 comp rx=0 tx=0).stop 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 shutdown_connections 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f667c10d000 0x7f667c06e6b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f667c10c6b0 0x7f667c06e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66600b6670 0x7f66600b8b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 --2- 172.21.15.114:0/1000074791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f667c10a490 0x7f667c074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:48.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.615+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 >> 172.21.15.114:0/1000074791 conn(0x7f667c079420 msgr2=0x7f667c10de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:48.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.616+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 shutdown_connections 2024-09-21T06:54:48.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:48.616+0000 7f6680a00640 1 -- 172.21.15.114:0/1000074791 wait complete. 2024-09-21T06:54:48.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:48 smithi193 ceph-mon[34601]: pgmap v416: 129 pgs: 129 active+clean; 877 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 43 MiB/s wr, 342 op/s 2024-09-21T06:54:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:48 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3447638454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:49.061 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:48 smithi114 ceph-mon[30654]: pgmap v416: 129 pgs: 129 active+clean; 877 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 43 MiB/s wr, 342 op/s 2024-09-21T06:54:49.061 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:48 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3447638454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:48 smithi183 ceph-mon[34219]: pgmap v416: 129 pgs: 129 active+clean; 877 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 43 MiB/s wr, 342 op/s 2024-09-21T06:54:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:48 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3447638454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:54:49.319 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-21T06:54:49.320 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-21T06:54:50.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:54:50.221 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:50.684 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:50 smithi114 ceph-mon[30654]: pgmap v417: 129 pgs: 129 active+clean; 902 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 375 op/s 2024-09-21T06:54:50.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:50 smithi193 ceph-mon[34601]: pgmap v417: 129 pgs: 129 active+clean; 902 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 375 op/s 2024-09-21T06:54:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:50 smithi183 ceph-mon[34219]: pgmap v417: 129 pgs: 129 active+clean; 902 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 21 MiB/s wr, 375 op/s 2024-09-21T06:54:51.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:51 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:51 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:51.918 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 -- 172.21.15.114:0/2793366839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 msgr2=0x7f44c80761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 --2- 172.21.15.114:0/2793366839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 0x7f44c80761e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 -- 172.21.15.114:0/2793366839 shutdown_connections 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 --2- 172.21.15.114:0/2793366839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c8116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 --2- 172.21.15.114:0/2793366839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 0x7f44c80761e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 --2- 172.21.15.114:0/2793366839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44c8077380 0x7f44c80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.957+0000 7f44ce000640 1 -- 172.21.15.114:0/2793366839 >> 172.21.15.114:0/2793366839 conn(0x7f44c80ff430 msgr2=0x7f44c8101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.958+0000 7f44ce000640 1 -- 172.21.15.114:0/2793366839 shutdown_connections 2024-09-21T06:54:51.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.958+0000 7f44ce000640 1 -- 172.21.15.114:0/2793366839 wait complete. 2024-09-21T06:54:51.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.959+0000 7f44ce000640 1 Processor -- start 2024-09-21T06:54:51.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.959+0000 7f44ce000640 1 -- start start 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.959+0000 7f44ce000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 0x7f44c81a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.959+0000 7f44ce000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44c8077380 0x7f44c81a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44ce000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c81a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44ce000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8119f60 con 0x7f44c8077380 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44ce000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44c8119de0 con 0x7f44c8114290 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44ce000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44c811a0e0 con 0x7f44c8075dd0 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44cd600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c81a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44cd600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c81a61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37124/0 (socket says 172.21.15.114:37124) 2024-09-21T06:54:51.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44cd600640 1 -- 172.21.15.114:0/2270827976 learned_addr learned my addr 172.21.15.114:0/2270827976 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:51.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44ccc00640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 0x7f44c81a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.960+0000 7f44c7e00640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44c8077380 0x7f44c81a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.961+0000 7f44cd600640 1 -- 172.21.15.114:0/2270827976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 msgr2=0x7f44c81a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:51.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.961+0000 7f44cd600640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 0x7f44c81a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.961+0000 7f44cd600640 1 -- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44c8077380 msgr2=0x7f44c81a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.961+0000 7f44cd600640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44c8077380 0x7f44c81a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.961+0000 7f44cd600640 1 -- 172.21.15.114:0/2270827976 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c81a68f0 con 0x7f44c8114290 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.962+0000 7f44cd600640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c81a61f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f44c000b9a0 tx=0x7f44c000be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.962+0000 7f44c5600640 1 -- 172.21.15.114:0/2270827976 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0019070 con 0x7f44c8114290 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.962+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c81a6be0 con 0x7f44c8114290 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.962+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c806cde0 con 0x7f44c8114290 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.962+0000 7f44c5600640 1 -- 172.21.15.114:0/2270827976 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c000cba0 con 0x7f44c8114290 2024-09-21T06:54:51.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.962+0000 7f44c5600640 1 -- 172.21.15.114:0/2270827976 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c00075d0 con 0x7f44c8114290 2024-09-21T06:54:51.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.967+0000 7f44d62c0740 1 Processor -- start 2024-09-21T06:54:51.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44c5600640 1 -- 172.21.15.114:0/2270827976 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f44c001c070 con 0x7f44c8114290 2024-09-21T06:54:51.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 -- start start 2024-09-21T06:54:51.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb98b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 0x5621fb98ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb98bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621fb900ca0 con 0x5621fb5ba980 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621fb8f00d0 con 0x5621fb5c2b60 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.968+0000 7f44d62c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621fb8f3ee0 con 0x5621fb5e3840 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb98b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb98b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38568/0 (socket says 172.21.15.114:38568) 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 -- 172.21.15.114:0/526033083 learned_addr learned my addr 172.21.15.114:0/526033083 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb98bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 -- 172.21.15.114:0/526033083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 msgr2=0x5621fb98bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb98bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 -- 172.21.15.114:0/526033083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 msgr2=0x5621fb98ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:51.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 0x5621fb98ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.969+0000 7f44c7e00640 1 -- 172.21.15.114:0/526033083 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621fb8a2400 con 0x5621fb5e3840 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.970+0000 7f44c7e00640 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb98b510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005f70 tx=0x7f44b4005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83cdfc5301abbbbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.970+0000 7f44b9c00640 1 -- 172.21.15.114:0/526033083 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4035030 con 0x5621fb5e3840 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.970+0000 7f44b9c00640 1 -- 172.21.15.114:0/526033083 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b40314a0 con 0x5621fb5e3840 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.970+0000 7f44b9c00640 1 -- 172.21.15.114:0/526033083 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b403d6b0 con 0x5621fb5e3840 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.970+0000 7f44c5600640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44b00b6660 0x7f44b00b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.971+0000 7f44c5600640 1 -- 172.21.15.114:0/2270827976 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f44c00e6d20 con 0x7f44c8114290 2024-09-21T06:54:51.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.971+0000 7f44ccc00640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44b00b6660 0x7f44b00b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 -- 172.21.15.114:0/526033083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 msgr2=0x5621fb98b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:51.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb98b510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005f70 tx=0x7f44b4005780 comp rx=0 tx=0).stop 2024-09-21T06:54:51.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 -- 172.21.15.114:0/526033083 shutdown_connections 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb98bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 0x5621fb98ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 --2- 172.21.15.114:0/526033083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb98b510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 -- 172.21.15.114:0/526033083 >> 172.21.15.114:0/526033083 conn(0x5621fb5d7990 msgr2=0x5621fb989400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 -- 172.21.15.114:0/526033083 shutdown_connections 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.972+0000 7f44d62c0740 1 -- 172.21.15.114:0/526033083 wait complete. 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.973+0000 7f44ccc00640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44b00b6660 0x7f44b00b8b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5621fb98a190 tx=0x7f44bc0046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:51.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.973+0000 7f44d62c0740 1 -- start start 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.973+0000 7f44d62c0740 1 Processor -- start 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5ba980 0x5621fb996c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5c2b60 0x5621fb997150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb9a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5621fb7517a0 con 0x5621fb5c2b60 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621fb895a10 con 0x5621fb5ba980 2024-09-21T06:54:51.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5621fb999c90 con 0x5621fb5e3840 2024-09-21T06:54:51.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.974+0000 7f44d62c0740 10 client.0 _ll_register_callbacks cb 0x5621fb8a5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:51.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.975+0000 7f44cd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb9a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.975+0000 7f44cd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb9a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38572/0 (socket says 172.21.15.114:38572) 2024-09-21T06:54:51.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.975+0000 7f44cd600640 1 -- 172.21.15.114:0/880159776 learned_addr learned my addr 172.21.15.114:0/880159776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:51.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.975+0000 7f44ccc00640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5ba980 0x5621fb996c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44cd600640 1 -- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5ba980 msgr2=0x5621fb996c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:51.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44cd600640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5ba980 0x5621fb996c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44c7e00640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5c2b60 0x5621fb997150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44cd600640 1 -- 172.21.15.114:0/880159776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5c2b60 msgr2=0x5621fb997150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:51.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44cd600640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5c2b60 0x5621fb997150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44cd600640 1 -- 172.21.15.114:0/880159776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621fb65db90 con 0x5621fb5e3840 2024-09-21T06:54:51.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.976+0000 7f44c7e00640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5c2b60 0x5621fb997150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:51.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.977+0000 7f44cd600640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb9a2fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44c00ac930 tx=0x7f44c00e4660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f91660ebf41499db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:51.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.977+0000 7f44ae000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:51.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.977+0000 7f44ae000640 1 -- 172.21.15.114:0/880159776 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c00b6280 con 0x5621fb5e3840 2024-09-21T06:54:51.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.978+0000 7f44ae000640 1 -- 172.21.15.114:0/880159776 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f44c00abe10 con 0x5621fb5e3840 2024-09-21T06:54:51.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.978+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b40314a0 con 0x5621fb5e3840 2024-09-21T06:54:51.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.978+0000 7f44ae000640 1 -- 172.21.15.114:0/880159776 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f44c00a82f0 con 0x5621fb5e3840 2024-09-21T06:54:51.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.978+0000 7f44d62c0740 10 client.25840 fetch_fsmap learned FSMap version 31 2024-09-21T06:54:51.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.978+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621fb9a5a40 con 0x5621fb5e3840 2024-09-21T06:54:51.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.978+0000 7f44ae000640 1 -- 172.21.15.114:0/880159776 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b40314a0 con 0x5621fb5e3840 2024-09-21T06:54:51.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44ae000640 1 -- 172.21.15.114:0/880159776 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f44c00c2a10 con 0x5621fb5e3840 2024-09-21T06:54:51.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44d62c0740 10 client.25840 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:54:51.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44d62c0740 10 client.25840 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:54:51.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44d62c0740 20 client.25840 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:51.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44d62c0740 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb5a21c0 0x5621fb9a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:51.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44d62c0740 4 client.25840 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:51.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621fb902bf0 con 0x5621fb5a21c0 2024-09-21T06:54:51.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.980+0000 7f44ccc00640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb5a21c0 0x5621fb9a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:51.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.981+0000 7f44ccc00640 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb5a21c0 0x5621fb9a9050 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4553e056f60cabdc server_cookie=afb3075a66863113 in_seq=0 out_seq=0 2024-09-21T06:54:51.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.981+0000 7f44ae000640 10 client.25840 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:51.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.981+0000 7f44ae000640 1 -- 172.21.15.114:0/880159776 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5621fb902bf0 con 0x5621fb5a21c0 2024-09-21T06:54:51.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.981+0000 7f44ae000640 10 client.25840 handle_command_reply: tid=0 2024-09-21T06:54:51.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.982+0000 7f44d62c0740 1 client.25840 shutdown 2024-09-21T06:54:51.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.982+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 msgr2=0x5621fb9a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:51.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.982+0000 7f44d62c0740 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb9a2fe0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44c00ac930 tx=0x7f44c00e4660 comp rx=0 tx=0).stop 2024-09-21T06:54:51.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 shutdown_connections 2024-09-21T06:54:51.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb5a21c0 0x5621fb9a9050 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5e3840 0x5621fb9a2fe0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5c2b60 0x5621fb997150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 --2- 172.21.15.114:0/880159776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5ba980 0x5621fb996c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:51.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 >> 172.21.15.114:0/880159776 conn(0x5621fb5d7990 msgr2=0x5621fb98e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:51.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44ae000640 0 client.25840 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:51.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 shutdown_connections 2024-09-21T06:54:51.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 1 -- 172.21.15.114:0/880159776 wait complete. 2024-09-21T06:54:51.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:51.983+0000 7f44d62c0740 20 client.25840 trim_cache size 0 max 16384 2024-09-21T06:54:52.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 Processor -- start 2024-09-21T06:54:52.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 -- start start 2024-09-21T06:54:52.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 0x5621fb998af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 0x5621fb98cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5e3840 0x5621fb9a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f44a800fec0 con 0x5621fb5ba980 2024-09-21T06:54:52.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8011a90 con 0x5621fb5e3840 2024-09-21T06:54:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.021+0000 7f44d62c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8010590 con 0x5621fb5c2b60 2024-09-21T06:54:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.022+0000 7f44c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 0x5621fb98cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.022+0000 7f44c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 0x5621fb98cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38586/0 (socket says 172.21.15.114:38586) 2024-09-21T06:54:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.022+0000 7f44c7e00640 1 -- 172.21.15.114:0/591200545 learned_addr learned my addr 172.21.15.114:0/591200545 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.023+0000 7f44cd600640 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5e3840 0x5621fb9a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.023+0000 7f44c7e00640 1 -- 172.21.15.114:0/591200545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5e3840 msgr2=0x5621fb9a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.023+0000 7f44c7e00640 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5e3840 0x5621fb9a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.023+0000 7f44c7e00640 1 -- 172.21.15.114:0/591200545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 msgr2=0x5621fb998af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.023+0000 7f44c7e00640 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 0x5621fb998af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.023+0000 7f44c7e00640 1 -- 172.21.15.114:0/591200545 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621fb991840 con 0x5621fb5c2b60 2024-09-21T06:54:52.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.024+0000 7f44c7e00640 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 0x5621fb98cc50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006010 tx=0x7f44b4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61a26a66559dc25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.024+0000 7f44b9200640 1 -- 172.21.15.114:0/591200545 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b404e070 con 0x5621fb5c2b60 2024-09-21T06:54:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.024+0000 7f44b9200640 1 -- 172.21.15.114:0/591200545 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b40314a0 con 0x5621fb5c2b60 2024-09-21T06:54:52.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.024+0000 7f44b9200640 1 -- 172.21.15.114:0/591200545 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4035070 con 0x5621fb5c2b60 2024-09-21T06:54:52.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 -- 172.21.15.114:0/591200545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 msgr2=0x5621fb98cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 0x5621fb98cc50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f44b4006010 tx=0x7f44b4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 -- 172.21.15.114:0/591200545 shutdown_connections 2024-09-21T06:54:52.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5e3840 0x5621fb9a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5c2b60 0x5621fb98cc50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 --2- 172.21.15.114:0/591200545 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5ba980 0x5621fb998af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.026+0000 7f44d62c0740 1 -- 172.21.15.114:0/591200545 >> 172.21.15.114:0/591200545 conn(0x5621fb5d7990 msgr2=0x5621fb98c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.027+0000 7f44d62c0740 1 -- 172.21.15.114:0/591200545 shutdown_connections 2024-09-21T06:54:52.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.028+0000 7f44d62c0740 1 -- 172.21.15.114:0/591200545 wait complete. 2024-09-21T06:54:52.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.028+0000 7f44d62c0740 1 -- start start 2024-09-21T06:54:52.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.029+0000 7f44d62c0740 1 Processor -- start 2024-09-21T06:54:52.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5ba980 0x5621fb99af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb99b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5e3840 0x5621fb99ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8011330 con 0x5621fb5e3840 2024-09-21T06:54:52.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8010c60 con 0x5621fb5c2b60 2024-09-21T06:54:52.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44a800dea0 con 0x5621fb5ba980 2024-09-21T06:54:52.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.031+0000 7f44d62c0740 10 client.0 _ll_register_callbacks cb 0x7f44c0009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:52.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.032+0000 7f44c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb99b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.032+0000 7f44c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb99b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37184/0 (socket says 172.21.15.114:37184) 2024-09-21T06:54:52.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.032+0000 7f44c7e00640 1 -- 172.21.15.114:0/3812628721 learned_addr learned my addr 172.21.15.114:0/3812628721 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.032+0000 7f44ccc00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5ba980 0x5621fb99af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ccc00640 1 -- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 msgr2=0x5621fb99b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ccc00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb99b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ccc00640 1 -- 172.21.15.114:0/3812628721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5e3840 msgr2=0x5621fb99ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ccc00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5e3840 0x5621fb99ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ccc00640 1 -- 172.21.15.114:0/3812628721 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621fba12a10 con 0x5621fb5ba980 2024-09-21T06:54:52.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44c7e00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb99b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:52.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ccc00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5ba980 0x5621fb99af90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5621fb9a9780 tx=0x7f44bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d71f03f110b868 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.033+0000 7f44ae000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:52.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.034+0000 7f44ae000640 1 -- 172.21.15.114:0/3812628721 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc01f840 con 0x5621fb5ba980 2024-09-21T06:54:52.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.034+0000 7f44ae000640 1 -- 172.21.15.114:0/3812628721 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f44bc01f9e0 con 0x5621fb5ba980 2024-09-21T06:54:52.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.034+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b40314a0 con 0x5621fb5ba980 2024-09-21T06:54:52.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.035+0000 7f44ae000640 1 -- 172.21.15.114:0/3812628721 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f44bc038050 con 0x5621fb5ba980 2024-09-21T06:54:52.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.035+0000 7f44d62c0740 10 client.25846 fetch_fsmap learned FSMap version 31 2024-09-21T06:54:52.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.035+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5621fba13ca0 con 0x5621fb5ba980 2024-09-21T06:54:52.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.035+0000 7f44ae000640 1 -- 172.21.15.114:0/3812628721 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b40314a0 con 0x5621fb5ba980 2024-09-21T06:54:52.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44ae000640 1 -- 172.21.15.114:0/3812628721 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f44bc02fce0 con 0x5621fb5ba980 2024-09-21T06:54:52.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44d62c0740 10 client.25846 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:54:52.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44d62c0740 10 client.25846 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:54:52.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44d62c0740 20 client.25846 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:52.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44d62c0740 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb577570 0x5621fba16510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44d62c0740 4 client.25846 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-21T06:54:52.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x5621fb8fe6a0 con 0x5621fb577570 2024-09-21T06:54:52.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.036+0000 7f44c7e00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb577570 0x5621fba16510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.040+0000 7f44c7e00640 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb577570 0x5621fba16510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b155daa831b7efa7 server_cookie=b104a1d40a29060c in_seq=0 out_seq=0 2024-09-21T06:54:52.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.040+0000 7f44ae000640 10 client.25846 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:52.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:51 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 -- 172.21.15.114:0/3228420468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 msgr2=0x7f0b70110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 --2- 172.21.15.114:0/3228420468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b70110320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009be0 tx=0x7f0b6802a6c0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 -- 172.21.15.114:0/3228420468 shutdown_connections 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 --2- 172.21.15.114:0/3228420468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b70110320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 --2- 172.21.15.114:0/3228420468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 --2- 172.21.15.114:0/3228420468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b70104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 -- 172.21.15.114:0/3228420468 >> 172.21.15.114:0/3228420468 conn(0x7f0b70079740 msgr2=0x7f0b70079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 -- 172.21.15.114:0/3228420468 shutdown_connections 2024-09-21T06:54:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.714+0000 7f0b78600640 1 -- 172.21.15.114:0/3228420468 wait complete. 2024-09-21T06:54:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 Processor -- start 2024-09-21T06:54:52.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 -- start start 2024-09-21T06:54:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b7019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b7019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70113a90 con 0x7f0b701049d0 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70113910 con 0x7f0b70105320 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b78600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b70113c10 con 0x7f0b7010de30 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.716+0000 7f0b75c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b7019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37200/0 (socket says 172.21.15.114:37200) 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 -- 172.21.15.114:0/1797263624 learned_addr learned my addr 172.21.15.114:0/1797263624 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b7019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52544/0 (socket says 172.21.15.114:52544) 2024-09-21T06:54:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b76600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b7019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 -- 172.21.15.114:0/1797263624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 msgr2=0x7f0b7019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b7019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 -- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 msgr2=0x7f0b7019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b7019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 -- 172.21.15.114:0/1797263624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b701a0450 con 0x7f0b70105320 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b76600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b7019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75c00640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b7019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.717+0000 7f0b75200640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7019aa80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0058c0 tx=0x7f0b5c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.718+0000 7f0b6e000640 1 -- 172.21.15.114:0/1797263624 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00cd30 con 0x7f0b70105320 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.718+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b701a0740 con 0x7f0b70105320 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.718+0000 7f0b6e000640 1 -- 172.21.15.114:0/1797263624 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0035f0 con 0x7f0b70105320 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.718+0000 7f0b6e000640 1 -- 172.21.15.114:0/1797263624 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0044e0 con 0x7f0b70105320 2024-09-21T06:54:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.718+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b701025f0 con 0x7f0b70105320 2024-09-21T06:54:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.721+0000 7f0b7f1cb740 1 Processor -- start 2024-09-21T06:54:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 -- start start 2024-09-21T06:54:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9072da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc90732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9073820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55adc8fe9fb0 con 0x55adc8ca31b0 2024-09-21T06:54:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55adc8fd93e0 con 0x55adc8cab390 2024-09-21T06:54:52.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55adc8fdd1f0 con 0x55adc8ccc070 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.722+0000 7f0b76600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc90732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc90732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52546/0 (socket says 172.21.15.114:52546) 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 -- 172.21.15.114:0/401576042 learned_addr learned my addr 172.21.15.114:0/401576042 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 -- 172.21.15.114:0/401576042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 msgr2=0x55adc9072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9072da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 -- 172.21.15.114:0/401576042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 msgr2=0x55adc9073820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:54:52.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9073820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b75c00640 1 -- 172.21.15.114:0/401576042 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc8fa3cd0 con 0x55adc8ca31b0 2024-09-21T06:54:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.723+0000 7f0b6e000640 1 -- 172.21.15.114:0/1797263624 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f0b5c003a50 con 0x7f0b70105320 2024-09-21T06:54:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.724+0000 7f0b75c00640 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc90732e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0b640096c0 tx=0x7f0b64005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6d369b9a870fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.724+0000 7f0b62a00640 1 -- 172.21.15.114:0/401576042 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6401e030 con 0x55adc8ca31b0 2024-09-21T06:54:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.724+0000 7f0b62a00640 1 -- 172.21.15.114:0/401576042 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b64009d60 con 0x55adc8ca31b0 2024-09-21T06:54:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.724+0000 7f0b62a00640 1 -- 172.21.15.114:0/401576042 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b640047a0 con 0x55adc8ca31b0 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/401576042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 msgr2=0x55adc90732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc90732e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0b640096c0 tx=0x7f0b64005f90 comp rx=0 tx=0).stop 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/401576042 shutdown_connections 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9073820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc90732e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/401576042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.725+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/401576042 >> 172.21.15.114:0/401576042 conn(0x55adc8cc0460 msgr2=0x55adc9070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.726+0000 7f0b6e000640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b580b66f0 0x7f0b580b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.726+0000 7f0b6e000640 1 -- 172.21.15.114:0/1797263624 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b5c0e7bb0 con 0x7f0b70105320 2024-09-21T06:54:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.726+0000 7f0b75c00640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b580b66f0 0x7f0b580b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.727+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/401576042 shutdown_connections 2024-09-21T06:54:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.727+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/401576042 wait complete. 2024-09-21T06:54:52.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.727+0000 7f0b75c00640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b580b66f0 0x7f0b580b8c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b64005480 tx=0x7f0b64010870 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.727+0000 7f0b7f1cb740 1 -- start start 2024-09-21T06:54:52.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.727+0000 7f0b7f1cb740 1 Processor -- start 2024-09-21T06:54:52.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc907e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc907ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc908a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55adc8e3a700 con 0x55adc8ca31b0 2024-09-21T06:54:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55adc8f74580 con 0x55adc8cab390 2024-09-21T06:54:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55adc9081580 con 0x55adc8ccc070 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b7f1cb740 10 client.0 _ll_register_callbacks cb 0x55adc8f84470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b76600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc908a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b75c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc907e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b75c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc907e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52556/0 (socket says 172.21.15.114:52556) 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b75c00640 1 -- 172.21.15.114:0/48837402 learned_addr learned my addr 172.21.15.114:0/48837402 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.728+0000 7f0b75200640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc907ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b76600640 1 -- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 msgr2=0x55adc907ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b76600640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc907ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b76600640 1 -- 172.21.15.114:0/48837402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 msgr2=0x55adc907e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b76600640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc907e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b76600640 1 -- 172.21.15.114:0/48837402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc8d37650 con 0x55adc8ccc070 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b75c00640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc907e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b76600640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc908a8d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b6802abe0 tx=0x7f0b680307d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb016ac5480eba9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.729+0000 7f0b57400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b57400640 1 -- 172.21.15.114:0/48837402 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68042070 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b57400640 1 -- 172.21.15.114:0/48837402 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b68003e00 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55adc8f76b90 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b57400640 1 -- 172.21.15.114:0/48837402 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0b68036040 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 10 client.25858 fetch_fsmap learned FSMap version 31 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55adc8f91950 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b57400640 1 -- 172.21.15.114:0/48837402 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55adc8f76b90 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b57400640 1 -- 172.21.15.114:0/48837402 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0b68004660 con 0x55adc8ccc070 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 10 client.25858 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 10 client.25858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 20 client.25858 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.730+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c8a9f0 0x55adc90906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.731+0000 7f0b7f1cb740 4 client.25858 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.731+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc8febf00 con 0x55adc8c8a9f0 2024-09-21T06:54:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.731+0000 7f0b75c00640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c8a9f0 0x55adc90906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.731+0000 7f0b75c00640 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c8a9f0 0x55adc90906b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e179963c6846ed server_cookie=de1b3a60a2c1f75b in_seq=0 out_seq=0 2024-09-21T06:54:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.731+0000 7f0b57400640 10 client.25858 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:52.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b57400640 1 -- 172.21.15.114:0/48837402 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55adc8febf00 con 0x55adc8c8a9f0 2024-09-21T06:54:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b57400640 10 client.25858 handle_command_reply: tid=0 2024-09-21T06:54:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 client.25858 shutdown 2024-09-21T06:54:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 msgr2=0x55adc908a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc908a8d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0b6802abe0 tx=0x7f0b680307d0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 shutdown_connections 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c8a9f0 0x55adc90906b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc908a8d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.735+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc907ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.736+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/48837402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc907e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.736+0000 7f0b57400640 0 client.25858 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.736+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 >> 172.21.15.114:0/48837402 conn(0x55adc8cc0460 msgr2=0x55adc9075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.736+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 shutdown_connections 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.736+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/48837402 wait complete. 2024-09-21T06:54:52.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.736+0000 7f0b7f1cb740 20 client.25858 trim_cache size 0 max 16384 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 Processor -- start 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 -- start start 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8ca31b0 0x55adc907eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 0x55adc9075ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ccc070 0x55adc90763e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701aece0 con 0x55adc8ccc070 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701b08b0 con 0x55adc8ca31b0 2024-09-21T06:54:52.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701af3b0 con 0x55adc8cab390 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.750+0000 7f0b75200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 0x55adc9075ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 0x55adc9075ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38648/0 (socket says 172.21.15.114:38648) 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 -- 172.21.15.114:0/729599897 learned_addr learned my addr 172.21.15.114:0/729599897 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75c00640 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8ca31b0 0x55adc907eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b76600640 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ccc070 0x55adc90763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 -- 172.21.15.114:0/729599897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8ca31b0 msgr2=0x55adc907eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8ca31b0 0x55adc907eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 -- 172.21.15.114:0/729599897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ccc070 msgr2=0x55adc90763e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ccc070 0x55adc90763e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 -- 172.21.15.114:0/729599897 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc907ca20 con 0x55adc8cab390 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b76600640 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ccc070 0x55adc90763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b75200640 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 0x55adc9075ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0ad3f0 tx=0x7f0b5c0ad580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e5427bf14cfeae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.751+0000 7f0b62000640 1 -- 172.21.15.114:0/729599897 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c009530 con 0x55adc8cab390 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b62000640 1 -- 172.21.15.114:0/729599897 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0ad200 con 0x55adc8cab390 2024-09-21T06:54:52.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b62000640 1 -- 172.21.15.114:0/729599897 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0a9ab0 con 0x55adc8cab390 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/729599897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 msgr2=0x55adc9075ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 0x55adc9075ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0ad3f0 tx=0x7f0b5c0ad580 comp rx=0 tx=0).stop 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/729599897 shutdown_connections 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ccc070 0x55adc90763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8cab390 0x55adc9075ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/729599897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8ca31b0 0x55adc907eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.752+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/729599897 >> 172.21.15.114:0/729599897 conn(0x55adc8cc0460 msgr2=0x55adc907e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.753+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/729599897 shutdown_connections 2024-09-21T06:54:52.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.753+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/729599897 wait complete. 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.753+0000 7f0b7f1cb740 1 -- start start 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.753+0000 7f0b7f1cb740 1 Processor -- start 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc9087080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9083540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9083ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701b0150 con 0x55adc8ca31b0 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701afa80 con 0x55adc8cab390 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b701accc0 con 0x55adc8ccc070 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b7f1cb740 10 client.0 _ll_register_callbacks cb 0x7f0b68009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b76600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9083ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b76600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9083ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38664/0 (socket says 172.21.15.114:38664) 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b76600640 1 -- 172.21.15.114:0/3200916158 learned_addr learned my addr 172.21.15.114:0/3200916158 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b75c00640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc9087080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b75200640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9083540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b76600640 1 -- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 msgr2=0x55adc9083540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.754+0000 7f0b76600640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9083540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b76600640 1 -- 172.21.15.114:0/3200916158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 msgr2=0x55adc9087080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:52.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b76600640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc9087080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b76600640 1 -- 172.21.15.114:0/3200916158 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc9084270 con 0x55adc8ccc070 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b76600640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9083ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b68031440 tx=0x7f0b68033c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41b65779cd79bf9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b57400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b57400640 1 -- 172.21.15.114:0/3200916158 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68058060 con 0x55adc8ccc070 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b57400640 1 -- 172.21.15.114:0/3200916158 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b680319e0 con 0x55adc8ccc070 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5c0ad200 con 0x55adc8ccc070 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b57400640 1 -- 172.21.15.114:0/3200916158 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0b6805f050 con 0x55adc8ccc070 2024-09-21T06:54:52.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b7f1cb740 10 client.25864 fetch_fsmap learned FSMap version 31 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.755+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55adc90fbfe0 con 0x55adc8ccc070 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b57400640 1 -- 172.21.15.114:0/3200916158 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0ad200 con 0x55adc8ccc070 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b57400640 1 -- 172.21.15.114:0/3200916158 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0b68033290 con 0x55adc8ccc070 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b7f1cb740 10 client.25864 fetch_fsmap finished waiting for FSMap version 31 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b7f1cb740 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b7f1cb740 20 client.25864 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c5fda0 0x55adc90fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b7f1cb740 4 client.25864 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adc8fe79b0 con 0x55adc8c5fda0 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.756+0000 7f0b75c00640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c5fda0 0x55adc90fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:52.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.757+0000 7f0b75c00640 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c5fda0 0x55adc90fe860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849a52d94d035149 server_cookie=770e2b1aaa29092c in_seq=0 out_seq=0 2024-09-21T06:54:52.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.757+0000 7f0b57400640 10 client.25864 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.760+0000 7f0b57400640 1 -- 172.21.15.114:0/3200916158 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adc8fe79b0 con 0x55adc8c5fda0 2024-09-21T06:54:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.760+0000 7f0b57400640 10 client.25864 handle_command_reply: tid=0 2024-09-21T06:54:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.760+0000 7f0b7f1cb740 1 client.25864 shutdown 2024-09-21T06:54:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 msgr2=0x55adc9083ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9083ae0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b68031440 tx=0x7f0b68033c50 comp rx=0 tx=0).stop 2024-09-21T06:54:52.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 shutdown_connections 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55adc8c5fda0 0x55adc90fe860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55adc8ccc070 0x55adc9083ae0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adc8cab390 0x55adc9083540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 --2- 172.21.15.114:0/3200916158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55adc8ca31b0 0x55adc9087080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b57400640 0 client.25864 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 >> 172.21.15.114:0/3200916158 conn(0x55adc8cc0460 msgr2=0x55adc9075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 shutdown_connections 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 1 -- 172.21.15.114:0/3200916158 wait complete. 2024-09-21T06:54:52.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.761+0000 7f0b7f1cb740 20 client.25864 trim_cache size 0 max 16384 2024-09-21T06:54:52.764 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:54:52.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b580b66f0 msgr2=0x7f0b580b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b580b66f0 0x7f0b580b8c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b64005480 tx=0x7f0b64010870 comp rx=0 tx=0).stop 2024-09-21T06:54:52.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 msgr2=0x7f0b7019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:52.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7019aa80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0058c0 tx=0x7f0b5c00ee50 comp rx=0 tx=0).stop 2024-09-21T06:54:52.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 shutdown_connections 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b7010de30 0x7f0b7019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b580b66f0 0x7f0b580b8c00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b70105320 0x7f0b7019aa80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 --2- 172.21.15.114:0/1797263624 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b701049d0 0x7f0b7019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 >> 172.21.15.114:0/1797263624 conn(0x7f0b70079740 msgr2=0x7f0b700ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 shutdown_connections 2024-09-21T06:54:52.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:52.762+0000 7f0b78600640 1 -- 172.21.15.114:0/1797263624 wait complete. 2024-09-21T06:54:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 15.0 s, 1481.0 tps, lat 53.253 ms stddev 57.651 2024-09-21T06:54:53.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44ae000640 1 -- 172.21.15.114:0/3812628721 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5621fb8fe6a0 con 0x5621fb577570 2024-09-21T06:54:53.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44ae000640 10 client.25846 handle_command_reply: tid=0 2024-09-21T06:54:53.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 client.25846 shutdown 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5ba980 msgr2=0x5621fb99af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5ba980 0x5621fb99af90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5621fb9a9780 tx=0x7f44bc004730 comp rx=0 tx=0).stop 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 shutdown_connections 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5621fb5e3840 0x5621fb99ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621fb5c2b60 0x5621fb99b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5621fb577570 0x5621fba16510 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 --2- 172.21.15.114:0/3812628721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5621fb5ba980 0x5621fb99af90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44ae000640 0 client.25846 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 >> 172.21.15.114:0/3812628721 conn(0x5621fb5d7990 msgr2=0x5621fb9891a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:53.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.014+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 shutdown_connections 2024-09-21T06:54:53.018 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:52 smithi114 ceph-mon[30654]: pgmap v418: 129 pgs: 129 active+clean; 909 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 502 op/s 2024-09-21T06:54:53.018 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:52 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:54:53.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.015+0000 7f44d62c0740 1 -- 172.21.15.114:0/3812628721 wait complete. 2024-09-21T06:54:53.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.015+0000 7f44d62c0740 20 client.25846 trim_cache size 0 max 16384 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 31, 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 2, 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "f894fe5": { 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 2, 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:54:53.019 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 134.5, 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:54:53.020 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:53.021 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:53.021 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:53.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.017+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44b00b6660 msgr2=0x7f44b00b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.017+0000 7f44ce000640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44b00b6660 0x7f44b00b8b70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5621fb98a190 tx=0x7f44bc0046c0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.017+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 msgr2=0x7f44c81a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.017+0000 7f44ce000640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c81a61f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f44c000b9a0 tx=0x7f44c000be60 comp rx=0 tx=0).stop 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 shutdown_connections 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f44b00b6660 0x7f44b00b8b70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44c8114290 0x7f44c81a61f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f44c8077380 0x7f44c81a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 --2- 172.21.15.114:0/2270827976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44c8075dd0 0x7f44c81a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 >> 172.21.15.114:0/2270827976 conn(0x7f44c80ff430 msgr2=0x7f44c8110190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 shutdown_connections 2024-09-21T06:54:53.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:53.018+0000 7f44ce000640 1 -- 172.21.15.114:0/2270827976 wait complete. 2024-09-21T06:54:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:52 smithi183 ceph-mon[34219]: pgmap v418: 129 pgs: 129 active+clean; 909 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 502 op/s 2024-09-21T06:54:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:52 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:54:53.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:52 smithi193 ceph-mon[34601]: pgmap v418: 129 pgs: 129 active+clean; 909 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 502 op/s 2024-09-21T06:54:53.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:52 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:54:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:54:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:54:53.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:54:54.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:54:54.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:54:54.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:54:54.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:54:54.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:54:54.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:54:54.574 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f894fe5', quiesce took 8.4 seconds. Will release after: 6.1 2024-09-21T06:54:54.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:54 smithi114 ceph-mon[30654]: pgmap v419: 129 pgs: 129 active+clean; 916 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2024-09-21T06:54:55.012 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:54:55.012 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:54:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:54 smithi183 ceph-mon[34219]: pgmap v419: 129 pgs: 129 active+clean; 916 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2024-09-21T06:54:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:54 smithi193 ceph-mon[34601]: pgmap v419: 129 pgs: 129 active+clean; 916 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 7.2 MiB/s wr, 562 op/s 2024-09-21T06:54:55.428 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 -- 172.21.15.114:0/3723306529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b4076100 msgr2=0x7f07b410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 --2- 172.21.15.114:0/3723306529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b4076100 0x7f07b410f920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009900 tx=0x7f07a002a410 comp rx=0 tx=0).stop 2024-09-21T06:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 -- 172.21.15.114:0/3723306529 shutdown_connections 2024-09-21T06:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 --2- 172.21.15.114:0/3723306529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410fe60 0x7f07b41122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 --2- 172.21.15.114:0/3723306529 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b4076100 0x7f07b410f920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 --2- 172.21.15.114:0/3723306529 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07b40757b0 0x7f07b4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.067+0000 7f07bb000640 1 -- 172.21.15.114:0/3723306529 >> 172.21.15.114:0/3723306529 conn(0x7f07b40ff330 msgr2=0x7f07b4101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.068+0000 7f07bb000640 1 -- 172.21.15.114:0/3723306529 shutdown_connections 2024-09-21T06:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.068+0000 7f07bb000640 1 -- 172.21.15.114:0/3723306529 wait complete. 2024-09-21T06:54:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 Processor -- start 2024-09-21T06:54:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 -- start start 2024-09-21T06:54:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 0x7f07b419c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07b4076100 0x7f07b419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410fe60 0x7f07b41a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4115a30 con 0x7f07b40757b0 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41158b0 con 0x7f07b410fe60 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07bb000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4115bb0 con 0x7f07b4076100 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07b9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 0x7f07b419c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07b9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 0x7f07b419c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47554/0 (socket says 172.21.15.114:47554) 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.069+0000 7f07b9c00640 1 -- 172.21.15.114:0/1212386825 learned_addr learned my addr 172.21.15.114:0/1212386825 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9200640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07b4076100 0x7f07b419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07ba600640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410fe60 0x7f07b41a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9c00640 1 -- 172.21.15.114:0/1212386825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07b4076100 msgr2=0x7f07b419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9c00640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07b4076100 0x7f07b419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9c00640 1 -- 172.21.15.114:0/1212386825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410fe60 msgr2=0x7f07b41a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9c00640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410fe60 0x7f07b41a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9c00640 1 -- 172.21.15.114:0/1212386825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b41a23c0 con 0x7f07b40757b0 2024-09-21T06:54:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b9c00640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 0x7f07b419c490 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f07a800e9f0 tx=0x7f07a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.070+0000 7f07b2000640 1 -- 172.21.15.114:0/1212386825 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a800cd80 con 0x7f07b40757b0 2024-09-21T06:54:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.071+0000 7f07b2000640 1 -- 172.21.15.114:0/1212386825 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a80054d0 con 0x7f07b40757b0 2024-09-21T06:54:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.071+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b41a26b0 con 0x7f07b40757b0 2024-09-21T06:54:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.071+0000 7f07b2000640 1 -- 172.21.15.114:0/1212386825 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8010690 con 0x7f07b40757b0 2024-09-21T06:54:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.071+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41a8e40 con 0x7f07b40757b0 2024-09-21T06:54:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.074+0000 7f07c37ee740 1 Processor -- start 2024-09-21T06:54:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.074+0000 7f07c37ee740 1 -- start start 2024-09-21T06:54:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.074+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de651b0 0x55673e209e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.074+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e20a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e20a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07c37ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55673e17f6b0 con 0x55673de444d0 2024-09-21T06:54:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07c37ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55673e16eae0 con 0x55673de651b0 2024-09-21T06:54:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07c37ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55673e1728f0 con 0x55673de3c2f0 2024-09-21T06:54:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07b9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e20a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07ba600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de651b0 0x55673e209e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07b9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e20a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47564/0 (socket says 172.21.15.114:47564) 2024-09-21T06:54:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07b9200640 1 -- 172.21.15.114:0/1533007758 learned_addr learned my addr 172.21.15.114:0/1533007758 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07ba600640 1 -- 172.21.15.114:0/1533007758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 msgr2=0x55673e20a350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07ba600640 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e20a350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07ba600640 1 -- 172.21.15.114:0/1533007758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 msgr2=0x55673e20a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07ba600640 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e20a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.075+0000 7f07ba600640 1 -- 172.21.15.114:0/1533007758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673e13b780 con 0x55673de651b0 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.076+0000 7f07ba600640 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de651b0 0x55673e209e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009f00 tx=0x7f07ac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779c0e876e8d4c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.076+0000 7f07b2000640 1 -- 172.21.15.114:0/1212386825 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f07a8004e90 con 0x7f07b40757b0 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.076+0000 7f07a6a00640 1 -- 172.21.15.114:0/1533007758 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac003b70 con 0x55673de651b0 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.076+0000 7f07a6a00640 1 -- 172.21.15.114:0/1533007758 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07ac015070 con 0x55673de651b0 2024-09-21T06:54:56.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.076+0000 7f07a6a00640 1 -- 172.21.15.114:0/1533007758 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac011770 con 0x55673de651b0 2024-09-21T06:54:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 -- 172.21.15.114:0/1533007758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de651b0 msgr2=0x55673e209e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de651b0 0x55673e209e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009f00 tx=0x7f07ac0107a0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 -- 172.21.15.114:0/1533007758 shutdown_connections 2024-09-21T06:54:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e20a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e20a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1533007758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de651b0 0x55673e209e10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 -- 172.21.15.114:0/1533007758 >> 172.21.15.114:0/1533007758 conn(0x55673de59300 msgr2=0x55673e207d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 -- 172.21.15.114:0/1533007758 shutdown_connections 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.077+0000 7f07c37ee740 1 -- 172.21.15.114:0/1533007758 wait complete. 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.078+0000 7f07c37ee740 1 -- start start 2024-09-21T06:54:56.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.078+0000 7f07c37ee740 1 Processor -- start 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.078+0000 7f07b2000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f079c0b6500 0x7f079c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e215550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e215a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de651b0 0x55673e2218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55673e0a6ab0 con 0x55673de651b0 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55673e115570 con 0x55673de444d0 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55673e218590 con 0x55673de3c2f0 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b2000640 1 -- 172.21.15.114:0/1212386825 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f07a80e7e10 con 0x7f07b40757b0 2024-09-21T06:54:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07c37ee740 10 client.0 _ll_register_callbacks cb 0x55673e124100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07ba600640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f079c0b6500 0x7f079c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e215550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e215550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53368/0 (socket says 172.21.15.114:53368) 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 -- 172.21.15.114:0/1239088817 learned_addr learned my addr 172.21.15.114:0/1239088817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07ba600640 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de651b0 0x55673e2218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 -- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 msgr2=0x55673e215a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e215a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 -- 172.21.15.114:0/1239088817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de651b0 msgr2=0x55673e2218b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de651b0 0x55673e2218b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.079+0000 7f07b9c00640 1 -- 172.21.15.114:0/1239088817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673ded0790 con 0x55673de3c2f0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f07ba600640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f079c0b6500 0x7f079c0b8a10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0049d0 tx=0x7f07ac005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f07b9c00640 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e215550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f07a80e5360 tx=0x7f07a80ad840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c90e9a67173eb57f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f079b400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f079b400640 1 -- 172.21.15.114:0/1239088817 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a80ad120 con 0x55673de3c2f0 2024-09-21T06:54:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ac015070 con 0x55673de3c2f0 2024-09-21T06:54:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f079b400640 1 -- 172.21.15.114:0/1239088817 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f07a80acc10 con 0x55673de3c2f0 2024-09-21T06:54:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f079b400640 1 -- 172.21.15.114:0/1239088817 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f07a80a9120 con 0x55673de3c2f0 2024-09-21T06:54:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f07c37ee740 10 client.25873 fetch_fsmap learned FSMap version 32 2024-09-21T06:54:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.080+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55673df49620 con 0x55673de3c2f0 2024-09-21T06:54:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f079b400640 1 -- 172.21.15.114:0/1239088817 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07ac015070 con 0x55673de3c2f0 2024-09-21T06:54:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f079b400640 1 -- 172.21.15.114:0/1239088817 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f07a80aece0 con 0x55673de3c2f0 2024-09-21T06:54:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07c37ee740 10 client.25873 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:54:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07c37ee740 10 client.25873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07c37ee740 20 client.25873 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673de23d80 0x55673e227740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07c37ee740 4 client.25873 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55673e181600 con 0x55673de23d80 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.081+0000 7f07b9200640 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673de23d80 0x55673e227740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.082+0000 7f07b9200640 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673de23d80 0x55673e227740 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88cd8b3939aac158 server_cookie=60d3cd21df331bbf in_seq=0 out_seq=0 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.082+0000 7f079b400640 10 client.25873 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f079b400640 1 -- 172.21.15.114:0/1239088817 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55673e181600 con 0x55673de23d80 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f079b400640 10 client.25873 handle_command_reply: tid=0 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 client.25873 shutdown 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 msgr2=0x55673e215550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e215550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f07a80e5360 tx=0x7f07a80ad840 comp rx=0 tx=0).stop 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 shutdown_connections 2024-09-21T06:54:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de651b0 0x55673e2218b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e215a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673de23d80 0x55673e227740 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 --2- 172.21.15.114:0/1239088817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de3c2f0 0x55673e215550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f079b400640 0 client.25873 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 >> 172.21.15.114:0/1239088817 conn(0x55673de59300 msgr2=0x55673e20ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 shutdown_connections 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 1 -- 172.21.15.114:0/1239088817 wait complete. 2024-09-21T06:54:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.083+0000 7f07c37ee740 20 client.25873 trim_cache size 0 max 16384 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.099+0000 7f07c37ee740 1 Processor -- start 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.099+0000 7f07c37ee740 1 -- start start 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.099+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 0x55673e215a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.099+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e216a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e212720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07c37ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41b2e10 con 0x55673de3c2f0 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07c37ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41b4a10 con 0x55673de444d0 2024-09-21T06:54:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07c37ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41b34e0 con 0x55673de651b0 2024-09-21T06:54:56.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 0x55673e215a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e216a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 0x55673e215a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47594/0 (socket says 172.21.15.114:47594) 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 -- 172.21.15.114:0/2237553929 learned_addr learned my addr 172.21.15.114:0/2237553929 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07ba600640 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e212720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 -- 172.21.15.114:0/2237553929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 msgr2=0x55673e212720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e212720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 -- 172.21.15.114:0/2237553929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 msgr2=0x55673e216a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e216a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9c00640 1 -- 172.21.15.114:0/2237553929 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673e217920 con 0x55673de3c2f0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.100+0000 7f07b9200640 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e216a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.101+0000 7f07b9c00640 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 0x55673e215a30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009380 tx=0x7f07a80d11a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730496b89875b90e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.101+0000 7f07a6000640 1 -- 172.21.15.114:0/2237553929 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8005180 con 0x55673de3c2f0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.101+0000 7f07a6000640 1 -- 172.21.15.114:0/2237553929 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a80ad4c0 con 0x55673de3c2f0 2024-09-21T06:54:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.101+0000 7f07a6000640 1 -- 172.21.15.114:0/2237553929 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a80a9560 con 0x55673de3c2f0 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 -- 172.21.15.114:0/2237553929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 msgr2=0x55673e215a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 0x55673e215a30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009380 tx=0x7f07a80d11a0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 -- 172.21.15.114:0/2237553929 shutdown_connections 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e212720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de444d0 0x55673e216a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 --2- 172.21.15.114:0/2237553929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de3c2f0 0x55673e215a30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 -- 172.21.15.114:0/2237553929 >> 172.21.15.114:0/2237553929 conn(0x55673de59300 msgr2=0x55673e20b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 -- 172.21.15.114:0/2237553929 shutdown_connections 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 -- 172.21.15.114:0/2237553929 wait complete. 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.102+0000 7f07c37ee740 1 -- start start 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 Processor -- start 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de3c2f0 0x55673e21df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e21b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e21b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41b4280 con 0x55673de444d0 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41b3bb0 con 0x55673de3c2f0 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41b0df0 con 0x55673de651b0 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07c37ee740 10 client.0 _ll_register_callbacks cb 0x7f07a800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07b9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e21b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07b9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e21b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47608/0 (socket says 172.21.15.114:47608) 2024-09-21T06:54:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07b9200640 1 -- 172.21.15.114:0/4165252549 learned_addr learned my addr 172.21.15.114:0/4165252549 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.103+0000 7f07b9c00640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de3c2f0 0x55673e21df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07ba600640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e21b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07b9200640 1 -- 172.21.15.114:0/4165252549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 msgr2=0x55673e21b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07b9200640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e21b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07b9200640 1 -- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de3c2f0 msgr2=0x55673e21df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07b9200640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de3c2f0 0x55673e21df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07b9200640 1 -- 172.21.15.114:0/4165252549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673e21bfb0 con 0x55673de444d0 2024-09-21T06:54:56.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f07b9200640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e21b3d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009850 tx=0x7f07a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b70e95496432cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f079b400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.104+0000 7f079b400640 1 -- 172.21.15.114:0/4165252549 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0035040 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a80ad4c0 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f079b400640 1 -- 172.21.15.114:0/4165252549 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f07a0005240 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f079b400640 1 -- 172.21.15.114:0/4165252549 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f07a0054050 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f07c37ee740 10 client.15861 fetch_fsmap learned FSMap version 32 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55673e292020 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f079b400640 1 -- 172.21.15.114:0/4165252549 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07a80ad4c0 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f079b400640 1 -- 172.21.15.114:0/4165252549 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f07a00035f0 con 0x55673de444d0 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f07c37ee740 10 client.15861 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.105+0000 7f07c37ee740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f07c37ee740 20 client.15861 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f07c37ee740 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673ddf91c0 0x55673e2948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:56.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f07c37ee740 4 client.15861 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:54:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55673e17d0b0 con 0x55673ddf91c0 2024-09-21T06:54:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f07b9c00640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673ddf91c0 0x55673e2948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f07b9c00640 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673ddf91c0 0x55673e2948d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98229d8fb0d9353 server_cookie=cf42dee921690f70 in_seq=0 out_seq=0 2024-09-21T06:54:56.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.106+0000 7f079b400640 10 client.15861 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f079b400640 1 -- 172.21.15.114:0/4165252549 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55673e17d0b0 con 0x55673ddf91c0 2024-09-21T06:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f079b400640 10 client.15861 handle_command_reply: tid=0 2024-09-21T06:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 client.15861 shutdown 2024-09-21T06:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 msgr2=0x55673e21b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e21b3d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009850 tx=0x7f07a0004270 comp rx=0 tx=0).stop 2024-09-21T06:54:56.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 shutdown_connections 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55673ddf91c0 0x55673e2948d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55673de651b0 0x55673e21b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55673de444d0 0x55673e21b3d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f079b400640 0 client.15861 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 --2- 172.21.15.114:0/4165252549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55673de3c2f0 0x55673e21df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 >> 172.21.15.114:0/4165252549 conn(0x55673de59300 msgr2=0x55673e216850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 shutdown_connections 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 1 -- 172.21.15.114:0/4165252549 wait complete. 2024-09-21T06:54:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.147+0000 7f07c37ee740 20 client.15861 trim_cache size 0 max 16384 2024-09-21T06:54:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:54:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:54:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:54:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:54:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f079c0b6500 msgr2=0x7f079c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f079c0b6500 0x7f079c0b8a10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0049d0 tx=0x7f07ac005bd0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 msgr2=0x7f07b419c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 0x7f07b419c490 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f07a800e9f0 tx=0x7f07a800edf0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 shutdown_connections 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f079c0b6500 0x7f079c0b8a10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07b410fe60 0x7f07b41a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07b4076100 0x7f07b419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 --2- 172.21.15.114:0/1212386825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07b40757b0 0x7f07b419c490 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 >> 172.21.15.114:0/1212386825 conn(0x7f07b40ff330 msgr2=0x7f07b4109a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 shutdown_connections 2024-09-21T06:54:56.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:54:56.148+0000 7f07bb000640 1 -- 172.21.15.114:0/1212386825 wait complete. 2024-09-21T06:54:57.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:56 smithi114 ceph-mon[30654]: pgmap v420: 129 pgs: 129 active+clean; 917 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-09-21T06:54:57.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:56 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:56 smithi183 ceph-mon[34219]: pgmap v420: 129 pgs: 129 active+clean; 917 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-09-21T06:54:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:56 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:56 smithi193 ceph-mon[34601]: pgmap v420: 129 pgs: 129 active+clean; 917 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.2 MiB/s wr, 541 op/s 2024-09-21T06:54:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:56 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 -- 172.21.15.114:0/757086389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c98103b80 msgr2=0x7f2c98103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 --2- 172.21.15.114:0/757086389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c98103b80 0x7f2c98103f90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009900 tx=0x7f2c8802a410 comp rx=0 tx=0).stop 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 -- 172.21.15.114:0/757086389 shutdown_connections 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 --2- 172.21.15.114:0/757086389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98109e10 0x7f2c98110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 --2- 172.21.15.114:0/757086389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c981094c0 0x7f2c981098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 --2- 172.21.15.114:0/757086389 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c98103b80 0x7f2c98103f90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.751+0000 7f2c9f400640 1 -- 172.21.15.114:0/757086389 >> 172.21.15.114:0/757086389 conn(0x7f2c980ff5d0 msgr2=0x7f2c98101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:57.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.752+0000 7f2c9f400640 1 -- 172.21.15.114:0/757086389 shutdown_connections 2024-09-21T06:54:57.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.752+0000 7f2c9f400640 1 -- 172.21.15.114:0/757086389 wait complete. 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 Processor -- start 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 -- start start 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98103b80 0x7f2c9819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 0x7f2c9819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c98109e10 0x7f2c9819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113ad0 con 0x7f2c981094c0 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113950 con 0x7f2c98109e10 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.753+0000 7f2c9f400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98113c50 con 0x7f2c98103b80 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 0x7f2c9819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98103b80 0x7f2c9819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 0x7f2c9819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47632/0 (socket says 172.21.15.114:47632) 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 -- 172.21.15.114:0/668607766 learned_addr learned my addr 172.21.15.114:0/668607766 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c9d400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c98109e10 0x7f2c9819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 -- 172.21.15.114:0/668607766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98103b80 msgr2=0x7f2c9819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98103b80 0x7f2c9819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 -- 172.21.15.114:0/668607766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c98109e10 msgr2=0x7f2c9819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c98109e10 0x7f2c9819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c8fe00640 1 -- 172.21.15.114:0/668607766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c981a0410 con 0x7f2c981094c0 2024-09-21T06:54:57.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.754+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98103b80 0x7f2c9819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:57.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.755+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 0x7f2c9819aa40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e9f0 tx=0x7f2c8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:57.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.755+0000 7f2c96000640 1 -- 172.21.15.114:0/668607766 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000ccc0 con 0x7f2c981094c0 2024-09-21T06:54:57.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.755+0000 7f2c96000640 1 -- 172.21.15.114:0/668607766 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c800108e0 con 0x7f2c981094c0 2024-09-21T06:54:57.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.755+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981a0700 con 0x7f2c981094c0 2024-09-21T06:54:57.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.755+0000 7f2c96000640 1 -- 172.21.15.114:0/668607766 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80018860 con 0x7f2c981094c0 2024-09-21T06:54:57.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.755+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981a6e60 con 0x7f2c981094c0 2024-09-21T06:54:57.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.758+0000 7f2ca601a740 1 Processor -- start 2024-09-21T06:54:57.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.758+0000 7f2ca601a740 1 -- start start 2024-09-21T06:54:57.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2c96000640 1 -- 172.21.15.114:0/668607766 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f2c80010430 con 0x7f2c981094c0 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7100d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 0x561ad7101250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d38670 0x561ad7101790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2ca601a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ad7076e90 con 0x561ad6d59350 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2ca601a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ad70662c0 con 0x561ad6d30490 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2ca601a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ad706a0d0 con 0x561ad6d38670 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2c9d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 0x561ad7101250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 0x561ad7101250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50766/0 (socket says 172.21.15.114:50766) 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2c9ca00640 1 -- 172.21.15.114:0/3748329493 learned_addr learned my addr 172.21.15.114:0/3748329493 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:57.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.759+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d38670 0x561ad7101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c9ca00640 1 -- 172.21.15.114:0/3748329493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d38670 msgr2=0x561ad7101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d38670 0x561ad7101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c9ca00640 1 -- 172.21.15.114:0/3748329493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 msgr2=0x561ad7100d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7100d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c9ca00640 1 -- 172.21.15.114:0/3748329493 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad6dc4930 con 0x561ad6d30490 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 0x561ad7101250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009900 tx=0x7f2c88005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab66f3693a5664e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c8dc00640 1 -- 172.21.15.114:0/3748329493 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88040070 con 0x561ad6d30490 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c8dc00640 1 -- 172.21.15.114:0/3748329493 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c88031720 con 0x561ad6d30490 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c96000640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2c7c0b65f0 0x7f2c7c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.760+0000 7f2c8dc00640 1 -- 172.21.15.114:0/3748329493 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88035040 con 0x561ad6d30490 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2c96000640 1 -- 172.21.15.114:0/668607766 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2c800e7b20 con 0x7f2c981094c0 2024-09-21T06:54:57.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2c9d400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2c7c0b65f0 0x7f2c7c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 -- 172.21.15.114:0/3748329493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 msgr2=0x561ad7101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 0x561ad7101250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009900 tx=0x7f2c88005e20 comp rx=0 tx=0).stop 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2c9d400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2c7c0b65f0 0x7f2c7c0b8b00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005fd0 tx=0x7f2c90009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 -- 172.21.15.114:0/3748329493 shutdown_connections 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d38670 0x561ad7101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d30490 0x561ad7101250 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 --2- 172.21.15.114:0/3748329493 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.761+0000 7f2ca601a740 1 -- 172.21.15.114:0/3748329493 >> 172.21.15.114:0/3748329493 conn(0x561ad6d4d740 msgr2=0x561ad70fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:57.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.762+0000 7f2ca601a740 1 -- 172.21.15.114:0/3748329493 shutdown_connections 2024-09-21T06:54:57.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.762+0000 7f2ca601a740 1 -- 172.21.15.114:0/3748329493 wait complete. 2024-09-21T06:54:57.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.762+0000 7f2ca601a740 1 -- start start 2024-09-21T06:54:57.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.762+0000 7f2ca601a740 1 Processor -- start 2024-09-21T06:54:57.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7110c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d38670 0x561ad7111160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d59350 0x561ad711cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561ad6ec7890 con 0x561ad6d38670 2024-09-21T06:54:57.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ad7001160 con 0x561ad6d59350 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ad7113790 con 0x561ad6d30490 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2ca601a740 10 client.0 _ll_register_callbacks cb 0x561ad701da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d38670 0x561ad7111160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d38670 0x561ad7111160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47656/0 (socket says 172.21.15.114:47656) 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2c8fe00640 1 -- 172.21.15.114:0/443601334 learned_addr learned my addr 172.21.15.114:0/443601334 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7110c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.763+0000 7f2c9d400640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d59350 0x561ad711cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c9ca00640 1 -- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d59350 msgr2=0x561ad711cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d59350 0x561ad711cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c9ca00640 1 -- 172.21.15.114:0/443601334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d38670 msgr2=0x561ad7111160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d38670 0x561ad7111160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c9ca00640 1 -- 172.21.15.114:0/443601334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad711d090 con 0x561ad6d30490 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7110c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c8802a930 tx=0x7f2c880053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5f0f7aec69df24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:57.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c86000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:57.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c86000640 1 -- 172.21.15.114:0/443601334 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88051060 con 0x561ad6d30490 2024-09-21T06:54:57.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2c86000640 1 -- 172.21.15.114:0/443601334 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2c880035f0 con 0x561ad6d30490 2024-09-21T06:54:57.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.764+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c88031720 con 0x561ad6d30490 2024-09-21T06:54:57.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2c86000640 1 -- 172.21.15.114:0/443601334 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2c88058050 con 0x561ad6d30490 2024-09-21T06:54:57.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2ca601a740 10 client.25891 fetch_fsmap learned FSMap version 32 2024-09-21T06:54:57.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ad711f950 con 0x561ad6d30490 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2c86000640 1 -- 172.21.15.114:0/443601334 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c88031720 con 0x561ad6d30490 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2c86000640 1 -- 172.21.15.114:0/443601334 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2c8803c4d0 con 0x561ad6d30490 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2ca601a740 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2ca601a740 10 client.25891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.765+0000 7f2ca601a740 20 client.25891 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.766+0000 7f2ca601a740 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6d17cd0 0x561ad7122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.766+0000 7f2ca601a740 4 client.25891 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.766+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ad7078de0 con 0x561ad6d17cd0 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.766+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6d17cd0 0x561ad7122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.766+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6d17cd0 0x561ad7122cb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606cf58b2126c655 server_cookie=96347068c0b09ef8 in_seq=0 out_seq=0 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.766+0000 7f2c86000640 10 client.25891 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.767+0000 7f2c86000640 1 -- 172.21.15.114:0/443601334 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561ad7078de0 con 0x561ad6d17cd0 2024-09-21T06:54:57.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.767+0000 7f2c86000640 10 client.25891 handle_command_reply: tid=0 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.767+0000 7f2ca601a740 1 client.25891 shutdown 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 msgr2=0x561ad7110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7110c20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c8802a930 tx=0x7f2c880053f0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 shutdown_connections 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d59350 0x561ad711cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d38670 0x561ad7111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6d17cd0 0x561ad7122cb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 --2- 172.21.15.114:0/443601334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7110c20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 >> 172.21.15.114:0/443601334 conn(0x561ad6d4d740 msgr2=0x561ad70fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2c86000640 0 client.25891 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 shutdown_connections 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 1 -- 172.21.15.114:0/443601334 wait complete. 2024-09-21T06:54:57.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.768+0000 7f2ca601a740 20 client.25891 trim_cache size 0 max 16384 2024-09-21T06:54:57.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.786+0000 7f2ca601a740 1 Processor -- start 2024-09-21T06:54:57.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.786+0000 7f2ca601a740 1 -- start start 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.786+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 0x561ad70fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.786+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad70fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.786+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d59350 0x561ad70fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2ca601a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7400fed0 con 0x561ad6d30490 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2ca601a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74011ad0 con 0x561ad6d38670 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2ca601a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c740105a0 con 0x561ad6d59350 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 0x561ad70fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 0x561ad70fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47666/0 (socket says 172.21.15.114:47666) 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 -- 172.21.15.114:0/2416327003 learned_addr learned my addr 172.21.15.114:0/2416327003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad70fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9d400640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d59350 0x561ad70fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 -- 172.21.15.114:0/2416327003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d59350 msgr2=0x561ad70fea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d59350 0x561ad70fea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 -- 172.21.15.114:0/2416327003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 msgr2=0x561ad70fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad70fe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9ca00640 1 -- 172.21.15.114:0/2416327003 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ad7111790 con 0x561ad6d30490 2024-09-21T06:54:57.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c9d400640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d59350 0x561ad70fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:54:57.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.787+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad70fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.788+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 0x561ad70fd990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2c8803c8a0 tx=0x7f2c8802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee52a6d1af4533a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.788+0000 7f2c8d200640 1 -- 172.21.15.114:0/2416327003 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88051070 con 0x561ad6d30490 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.788+0000 7f2c8d200640 1 -- 172.21.15.114:0/2416327003 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c880055a0 con 0x561ad6d30490 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.788+0000 7f2c8d200640 1 -- 172.21.15.114:0/2416327003 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8803ce60 con 0x561ad6d30490 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 -- 172.21.15.114:0/2416327003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 msgr2=0x561ad70fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 0x561ad70fd990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2c8803c8a0 tx=0x7f2c8802a410 comp rx=0 tx=0).stop 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 -- 172.21.15.114:0/2416327003 shutdown_connections 2024-09-21T06:54:57.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d59350 0x561ad70fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad70fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 --2- 172.21.15.114:0/2416327003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d30490 0x561ad70fd990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 -- 172.21.15.114:0/2416327003 >> 172.21.15.114:0/2416327003 conn(0x561ad6d4d740 msgr2=0x561ad70fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 -- 172.21.15.114:0/2416327003 shutdown_connections 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 -- 172.21.15.114:0/2416327003 wait complete. 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 -- start start 2024-09-21T06:54:57.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.789+0000 7f2ca601a740 1 Processor -- start 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7118ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad7119000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7113a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74011340 con 0x561ad6d59350 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c74010c70 con 0x561ad6d38670 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7400deb0 con 0x561ad6d30490 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2ca601a740 10 client.0 _ll_register_callbacks cb 0x7f2c90009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2c9d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2c9d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47674/0 (socket says 172.21.15.114:47674) 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2c9d400640 1 -- 172.21.15.114:0/111453348 learned_addr learned my addr 172.21.15.114:0/111453348 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7118ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.790+0000 7f2c8fe00640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad7119000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c9d400640 1 -- 172.21.15.114:0/111453348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 msgr2=0x561ad7118ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c9d400640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7118ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c9d400640 1 -- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 msgr2=0x561ad7119000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c9d400640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad7119000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c9d400640 1 -- 172.21.15.114:0/111453348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ad71197d0 con 0x561ad6d59350 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c9d400640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7113a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2c90004130 tx=0x7f2c90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e3e8d57e0a5d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c86000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c86000640 1 -- 172.21.15.114:0/111453348 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9002d060 con 0x561ad6d59350 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2c86000640 1 -- 172.21.15.114:0/111453348 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2c9000f8b0 con 0x561ad6d59350 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.791+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c880055a0 con 0x561ad6d59350 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2c86000640 1 -- 172.21.15.114:0/111453348 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2c90034050 con 0x561ad6d59350 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 10 client.15879 fetch_fsmap learned FSMap version 32 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ad718da70 con 0x561ad6d59350 2024-09-21T06:54:57.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2c86000640 1 -- 172.21.15.114:0/111453348 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c880055a0 con 0x561ad6d59350 2024-09-21T06:54:57.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2c86000640 1 -- 172.21.15.114:0/111453348 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2c9001e400 con 0x561ad6d59350 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 10 client.15879 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 20 client.15879 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6ced080 0x561ad7190350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 4 client.15879 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.792+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561ad7074890 con 0x561ad6ced080 2024-09-21T06:54:57.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.793+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6ced080 0x561ad7190350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.793+0000 7f2c9ca00640 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6ced080 0x561ad7190350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3de45aabbe96a1f server_cookie=8d0a90142cca3dff in_seq=0 out_seq=0 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.793+0000 7f2c86000640 10 client.15879 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.794+0000 7f2c86000640 1 -- 172.21.15.114:0/111453348 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561ad7074890 con 0x561ad6ced080 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.794+0000 7f2c86000640 10 client.15879 handle_command_reply: tid=0 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.794+0000 7f2ca601a740 1 client.15879 shutdown 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 msgr2=0x561ad7113a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7113a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2c90004130 tx=0x7f2c90004a40 comp rx=0 tx=0).stop 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 shutdown_connections 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561ad6d59350 0x561ad7113a70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561ad6d38670 0x561ad7119000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561ad6ced080 0x561ad7190350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 --2- 172.21.15.114:0/111453348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561ad6d30490 0x561ad7118ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 >> 172.21.15.114:0/111453348 conn(0x561ad6d4d740 msgr2=0x561ad70fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:57.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2c86000640 0 client.15879 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 shutdown_connections 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 1 -- 172.21.15.114:0/111453348 wait complete. 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.795+0000 7f2ca601a740 20 client.15879 trim_cache size 0 max 16384 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "baea4e7c-934e-4b5b-8adf-2cbac8995b1e", 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T06:54:57.799 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2c7c0b65f0 msgr2=0x7f2c7c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2c7c0b65f0 0x7f2c7c0b8b00 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2c90005fd0 tx=0x7f2c90009500 comp rx=0 tx=0).stop 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 msgr2=0x7f2c9819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 0x7f2c9819aa40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000e9f0 tx=0x7f2c8000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 shutdown_connections 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2c7c0b65f0 0x7f2c7c0b8b00 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c98109e10 0x7f2c9819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2c981094c0 0x7f2c9819aa40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 --2- 172.21.15.114:0/668607766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c98103b80 0x7f2c9819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 >> 172.21.15.114:0/668607766 conn(0x7f2c980ff5d0 msgr2=0x7f2c981019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 shutdown_connections 2024-09-21T06:54:57.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:57.796+0000 7f2c9f400640 1 -- 172.21.15.114:0/668607766 wait complete. 2024-09-21T06:54:57.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 20.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:54:58.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: pgmap v421: 129 pgs: 129 active+clean; 931 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.4 MiB/s wr, 620 op/s 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:58.975 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:58 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.019 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:54:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: pgmap v421: 129 pgs: 129 active+clean; 931 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.4 MiB/s wr, 620 op/s 2024-09-21T06:54:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T06:54:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:54:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:54:59.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:54:59.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:58 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: pgmap v421: 129 pgs: 129 active+clean; 931 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 6.4 MiB/s wr, 620 op/s 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:58 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 -- 172.21.15.114:0/2956764043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 msgr2=0x7fe0640761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 --2- 172.21.15.114:0/2956764043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0640761e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe050009900 tx=0x7fe05002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 -- 172.21.15.114:0/2956764043 shutdown_connections 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 --2- 172.21.15.114:0/2956764043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe064114260 0x7fe064116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 --2- 172.21.15.114:0/2956764043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0640761e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 --2- 172.21.15.114:0/2956764043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe064077380 0x7fe0640757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 -- 172.21.15.114:0/2956764043 >> 172.21.15.114:0/2956764043 conn(0x7fe0640ff3e0 msgr2=0x7fe064101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 -- 172.21.15.114:0/2956764043 shutdown_connections 2024-09-21T06:54:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.752+0000 7fe063200640 1 -- 172.21.15.114:0/2956764043 wait complete. 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 Processor -- start 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 -- start start 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0641a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe064077380 0x7fe0641a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe064114260 0x7fe0641a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe064119ea0 con 0x7fe064075dd0 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe064119d20 con 0x7fe064077380 2024-09-21T06:54:59.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.753+0000 7fe063200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe06411a020 con 0x7fe064114260 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0641a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0641a08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47704/0 (socket says 172.21.15.114:47704) 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 -- 172.21.15.114:0/3731633752 learned_addr learned my addr 172.21.15.114:0/3731633752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061400640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe064077380 0x7fe0641a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe062800640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe064114260 0x7fe0641a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 -- 172.21.15.114:0/3731633752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe064114260 msgr2=0x7fe0641a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe064114260 0x7fe0641a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 -- 172.21.15.114:0/3731633752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe064077380 msgr2=0x7fe0641a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe064077380 0x7fe0641a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.754+0000 7fe061e00640 1 -- 172.21.15.114:0/3731633752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0641a6840 con 0x7fe064075dd0 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.755+0000 7fe061e00640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0641a08d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe05400ea60 tx=0x7fe05400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:59.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.755+0000 7fe05aa00640 1 -- 172.21.15.114:0/3731633752 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400cde0 con 0x7fe064075dd0 2024-09-21T06:54:59.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.755+0000 7fe05aa00640 1 -- 172.21.15.114:0/3731633752 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0540035f0 con 0x7fe064075dd0 2024-09-21T06:54:59.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.755+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0641a6b30 con 0x7fe064075dd0 2024-09-21T06:54:59.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.755+0000 7fe05aa00640 1 -- 172.21.15.114:0/3731633752 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054010690 con 0x7fe064075dd0 2024-09-21T06:54:59.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.755+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0641ad290 con 0x7fe064075dd0 2024-09-21T06:54:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.758+0000 7fe06fcaf740 1 Processor -- start 2024-09-21T06:54:59.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.758+0000 7fe06fcaf740 1 -- start start 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe05aa00640 1 -- 172.21.15.114:0/3731633752 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7fe0540037a0 con 0x7fe064075dd0 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc4b070 0x55c81dff1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81dff2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c81df68f70 con 0x55c81dc2a390 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c81df583a0 con 0x55c81dc4b070 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c81df5c1b0 con 0x55c81dc221b0 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe061e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81dff2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe061400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe061e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81dff2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53470/0 (socket says 172.21.15.114:53470) 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe061e00640 1 -- 172.21.15.114:0/3998960896 learned_addr learned my addr 172.21.15.114:0/3998960896 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:59.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.759+0000 7fe062800640 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc4b070 0x55c81dff1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe061e00640 1 -- 172.21.15.114:0/3998960896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc4b070 msgr2=0x55c81dff1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe061e00640 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc4b070 0x55c81dff1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe061e00640 1 -- 172.21.15.114:0/3998960896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 msgr2=0x55c81dff29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe061e00640 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe061e00640 1 -- 172.21.15.114:0/3998960896 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81df22c90 con 0x55c81dc221b0 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe061e00640 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81dff2470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ebcb0 tx=0x7fe0540f0790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15d33e95684f5a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe04f400640 1 -- 172.21.15.114:0/3998960896 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540fa670 con 0x55c81dc221b0 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe04f400640 1 -- 172.21.15.114:0/3998960896 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0540f5070 con 0x55c81dc221b0 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.760+0000 7fe04f400640 1 -- 172.21.15.114:0/3998960896 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540eeba0 con 0x55c81dc221b0 2024-09-21T06:54:59.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe05aa00640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0480b65f0 0x7fe0480b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe05aa00640 1 -- 172.21.15.114:0/3731633752 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe0540e7cd0 con 0x7fe064075dd0 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe062800640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0480b65f0 0x7fe0480b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 -- 172.21.15.114:0/3998960896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 msgr2=0x55c81dff2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81dff2470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ebcb0 tx=0x7fe0540f0790 comp rx=0 tx=0).stop 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 -- 172.21.15.114:0/3998960896 shutdown_connections 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81dff2470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/3998960896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc4b070 0x55c81dff1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 -- 172.21.15.114:0/3998960896 >> 172.21.15.114:0/3998960896 conn(0x55c81dc3f460 msgr2=0x55c81dfefe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.761+0000 7fe06fcaf740 1 -- 172.21.15.114:0/3998960896 shutdown_connections 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.762+0000 7fe06fcaf740 1 -- 172.21.15.114:0/3998960896 wait complete. 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.762+0000 7fe062800640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0480b65f0 0x7fe0480b8b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005fd0 tx=0x7fe05c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.762+0000 7fe06fcaf740 1 -- start start 2024-09-21T06:54:59.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.762+0000 7fe06fcaf740 1 Processor -- start 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dffd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc2a390 0x55c81dffdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e009a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c81ddb9700 con 0x55c81dc4b070 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c81def3540 con 0x55c81dc221b0 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c81e000710 con 0x55c81dc2a390 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe06fcaf740 10 client.0 _ll_register_callbacks cb 0x55c81df03430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe061e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dffd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe061e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dffd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50844/0 (socket says 172.21.15.114:50844) 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe061e00640 1 -- 172.21.15.114:0/477978720 learned_addr learned my addr 172.21.15.114:0/477978720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe062800640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e009a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe061400640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc2a390 0x55c81dffdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.768 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe061e00640 1 -- 172.21.15.114:0/477978720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc2a390 msgr2=0x55c81dffdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.763+0000 7fe061e00640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc2a390 0x55c81dffdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe061e00640 1 -- 172.21.15.114:0/477978720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 msgr2=0x55c81e009a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe061e00640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e009a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe061e00640 1 -- 172.21.15.114:0/477978720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81dcb6650 con 0x55c81dc221b0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe062800640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e009a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe061e00640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dffd6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ad7a0 tx=0x7fe054003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c7c5fd6812b85a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe043e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe043e00640 1 -- 172.21.15.114:0/477978720 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540a9e60 con 0x55c81dc221b0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe043e00640 1 -- 172.21.15.114:0/477978720 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe0540e53e0 con 0x55c81dc221b0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c81def5b50 con 0x55c81dc221b0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe043e00640 1 -- 172.21.15.114:0/477978720 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0540acf80 con 0x55c81dc221b0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.764+0000 7fe06fcaf740 10 client.25916 fetch_fsmap learned FSMap version 32 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c81df10910 con 0x55c81dc221b0 2024-09-21T06:54:59.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe043e00640 1 -- 172.21.15.114:0/477978720 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c81def5b50 con 0x55c81dc221b0 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe043e00640 1 -- 172.21.15.114:0/477978720 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe0540b6da0 con 0x55c81dc221b0 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 10 client.25916 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 10 client.25916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 20 client.25916 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dc099f0 0x55c81e00f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 4 client.25916 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c81df6aec0 con 0x55c81dc099f0 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.765+0000 7fe061400640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dc099f0 0x55c81e00f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.766+0000 7fe061400640 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dc099f0 0x55c81e00f6e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89d78bc924da67a server_cookie=e222bd86250a14d4 in_seq=0 out_seq=0 2024-09-21T06:54:59.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.766+0000 7fe043e00640 10 client.25916 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe043e00640 1 -- 172.21.15.114:0/477978720 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c81df6aec0 con 0x55c81dc099f0 2024-09-21T06:54:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe043e00640 10 client.25916 handle_command_reply: tid=0 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 client.25916 shutdown 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 msgr2=0x55c81dffd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dffd6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ad7a0 tx=0x7fe054003e90 comp rx=0 tx=0).stop 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 shutdown_connections 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e009a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dc099f0 0x55c81e00f6e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc2a390 0x55c81dffdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe043e00640 0 client.25916 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/477978720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dffd6b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 >> 172.21.15.114:0/477978720 conn(0x55c81dc3f460 msgr2=0x55c81dfee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 shutdown_connections 2024-09-21T06:54:59.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 1 -- 172.21.15.114:0/477978720 wait complete. 2024-09-21T06:54:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.767+0000 7fe06fcaf740 20 client.25916 trim_cache size 0 max 16384 2024-09-21T06:54:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.785+0000 7fe06fcaf740 1 Processor -- start 2024-09-21T06:54:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 -- start start 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dfff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 0x55c81e0099d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b7240 con 0x55c81dc2a390 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b8e40 con 0x55c81dc221b0 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b7910 con 0x55c81dc4b070 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe061400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe062800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 0x55c81e0099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe062800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 0x55c81e0099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53488/0 (socket says 172.21.15.114:53488) 2024-09-21T06:54:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe062800640 1 -- 172.21.15.114:0/417336907 learned_addr learned my addr 172.21.15.114:0/417336907 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.786+0000 7fe061e00640 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dfff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe062800640 1 -- 172.21.15.114:0/417336907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 msgr2=0x55c81dfff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe062800640 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dfff590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe062800640 1 -- 172.21.15.114:0/417336907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 msgr2=0x55c81dff3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe062800640 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe062800640 1 -- 172.21.15.114:0/417336907 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81dff0710 con 0x55c81dc4b070 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe062800640 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 0x55c81e0099d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe05c004390 tx=0x7fe05c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b87241d7c8bd681 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe04ea00640 1 -- 172.21.15.114:0/417336907 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c004850 con 0x55c81dc4b070 2024-09-21T06:54:59.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe04ea00640 1 -- 172.21.15.114:0/417336907 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c00f9e0 con 0x55c81dc4b070 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.787+0000 7fe04ea00640 1 -- 172.21.15.114:0/417336907 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0056c0 con 0x55c81dc4b070 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 -- 172.21.15.114:0/417336907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 msgr2=0x55c81e0099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 0x55c81e0099d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe05c004390 tx=0x7fe05c01a000 comp rx=0 tx=0).stop 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 -- 172.21.15.114:0/417336907 shutdown_connections 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc4b070 0x55c81e0099d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc2a390 0x55c81dff3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/417336907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc221b0 0x55c81dfff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 -- 172.21.15.114:0/417336907 >> 172.21.15.114:0/417336907 conn(0x55c81dc3f460 msgr2=0x55c81dff2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 -- 172.21.15.114:0/417336907 shutdown_connections 2024-09-21T06:54:59.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.788+0000 7fe06fcaf740 1 -- 172.21.15.114:0/417336907 wait complete. 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 -- start start 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 Processor -- start 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81e010f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc2a390 0x55c81e011530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e078e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b86b0 con 0x55c81dc4b070 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b7fe0 con 0x55c81dc2a390 2024-09-21T06:54:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.789+0000 7fe06fcaf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641b5220 con 0x55c81dc221b0 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe06fcaf740 10 client.0 _ll_register_callbacks cb 0x7fe0540ebb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe062800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e078e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe062800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e078e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47752/0 (socket says 172.21.15.114:47752) 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe062800640 1 -- 172.21.15.114:0/2790841852 learned_addr learned my addr 172.21.15.114:0/2790841852 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061e00640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81e010f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061400640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc2a390 0x55c81e011530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061e00640 1 -- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc2a390 msgr2=0x55c81e011530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061e00640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc2a390 0x55c81e011530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061e00640 1 -- 172.21.15.114:0/2790841852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 msgr2=0x55c81e078e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061e00640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e078e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061e00640 1 -- 172.21.15.114:0/2790841852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81e0794a0 con 0x55c81dc221b0 2024-09-21T06:54:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe061400640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc2a390 0x55c81e011530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.790+0000 7fe062800640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e078e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe061e00640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81e010f90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe05400c760 tx=0x7fe0540ad460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25817b321978dc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe043e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe043e00640 1 -- 172.21.15.114:0/2790841852 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540ad160 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe043e00640 1 -- 172.21.15.114:0/2790841852 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe0540aeb40 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c00f9e0 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe043e00640 1 -- 172.21.15.114:0/2790841852 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0540c8e60 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe06fcaf740 10 client.25915 fetch_fsmap learned FSMap version 32 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c81e07a730 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.791+0000 7fe043e00640 1 -- 172.21.15.114:0/2790841852 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05c00f9e0 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe043e00640 1 -- 172.21.15.114:0/2790841852 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe0540aca70 con 0x55c81dc221b0 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe06fcaf740 10 client.25915 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe06fcaf740 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:54:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe06fcaf740 20 client.25915 populate_metadata read hostname 'smithi114' 2024-09-21T06:54:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dbdeda0 0x55c81e07cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:54:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe06fcaf740 4 client.25915 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:54:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c81df66970 con 0x55c81dbdeda0 2024-09-21T06:54:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.792+0000 7fe061400640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dbdeda0 0x55c81e07cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:54:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe043e00640 10 client.25915 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:59.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe061400640 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dbdeda0 0x55c81e07cd80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a277b2ec398b77 server_cookie=637272004fad01 in_seq=0 out_seq=0 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe043e00640 1 -- 172.21.15.114:0/2790841852 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x55c81df66970 con 0x55c81dbdeda0 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe043e00640 10 client.25915 handle_command_reply: tid=0 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 client.25915 shutdown 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 msgr2=0x55c81e010f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81e010f90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe05400c760 tx=0x7fe0540ad460 comp rx=0 tx=0).stop 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 shutdown_connections 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c81dc4b070 0x55c81e078e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c81dc2a390 0x55c81e011530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c81dbdeda0 0x55c81e07cd80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 --2- 172.21.15.114:0/2790841852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c81dc221b0 0x55c81e010f90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe043e00640 0 client.25915 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 >> 172.21.15.114:0/2790841852 conn(0x55c81dc3f460 msgr2=0x55c81dfff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 shutdown_connections 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 1 -- 172.21.15.114:0/2790841852 wait complete. 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.793+0000 7fe06fcaf740 20 client.25915 trim_cache size 0 max 16384 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:54:59.799 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "baea4e7c-934e-4b5b-8adf-2cbac8995b1e", 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:54:59.800 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:54:59.801 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.794+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0480b65f0 msgr2=0x7fe0480b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.794+0000 7fe063200640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0480b65f0 0x7fe0480b8b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe05c005fd0 tx=0x7fe05c009500 comp rx=0 tx=0).stop 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.794+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 msgr2=0x7fe0641a08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.794+0000 7fe063200640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0641a08d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe05400ea60 tx=0x7fe05400ef20 comp rx=0 tx=0).stop 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 shutdown_connections 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe064114260 0x7fe0641a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0480b65f0 0x7fe0480b8b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe064077380 0x7fe0641a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 --2- 172.21.15.114:0/3731633752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe064075dd0 0x7fe0641a08d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 >> 172.21.15.114:0/3731633752 conn(0x7fe0640ff3e0 msgr2=0x7fe064110160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:54:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 shutdown_connections 2024-09-21T06:54:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:54:59.795+0000 7fe063200640 1 -- 172.21.15.114:0/3731633752 wait complete. 2024-09-21T06:55:00.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:59 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:55:00.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:54:59 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:55:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:59 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:55:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:54:59 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:55:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:59 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:55:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:54:59 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:55:00.565 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:55:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:55:00.253+0000 7f54aa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:55:00.565 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:55:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:55:00.253+0000 7f54aa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:55:00.566 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:55:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:55:00.253+0000 7f54aa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:55:00.566 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:55:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:55:00.253+0000 7f54aa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:55:00.566 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 06:55:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T06:55:00.253+0000 7f54aa000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T06:55:00.603 INFO:tasks.cephfs.filesystem:scrub status for tag:baea4e7c-934e-4b5b-8adf-2cbac8995b1e - {'path': '/', 'tag': 'baea4e7c-934e-4b5b-8adf-2cbac8995b1e', 'options': 'recursive,force'} 2024-09-21T06:55:00.676 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:55:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:01.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:00 smithi114 ceph-mon[30654]: pgmap v422: 129 pgs: 129 active+clean; 931 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-09-21T06:55:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:00 smithi183 ceph-mon[34219]: pgmap v422: 129 pgs: 129 active+clean; 931 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-09-21T06:55:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:00 smithi193 ceph-mon[34601]: pgmap v422: 129 pgs: 129 active+clean; 931 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 4.7 MiB/s wr, 453 op/s 2024-09-21T06:55:01.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:01 smithi114 ceph-mon[30654]: pgmap v423: 129 pgs: 129 active+clean; 962 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 8.7 MiB/s wr, 543 op/s 2024-09-21T06:55:01.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:01 smithi114 ceph-mon[30654]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-09-21T06:55:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:01 smithi183 ceph-mon[34219]: pgmap v423: 129 pgs: 129 active+clean; 962 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 8.7 MiB/s wr, 543 op/s 2024-09-21T06:55:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:01 smithi183 ceph-mon[34219]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-09-21T06:55:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:01 smithi193 ceph-mon[34601]: pgmap v423: 129 pgs: 129 active+clean; 962 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 8.7 MiB/s wr, 543 op/s 2024-09-21T06:55:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:01 smithi193 ceph-mon[34601]: mgrmap e25: x(active, since 11m), standbys: z, y 2024-09-21T06:55:02.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 25.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:55:02.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 -- 172.21.15.114:0/3643701221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010c010 msgr2=0x7f95d0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 --2- 172.21.15.114:0/3643701221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010c010 0x7f95d0112840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009ca0 tx=0x7f95c802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 -- 172.21.15.114:0/3643701221 shutdown_connections 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 --2- 172.21.15.114:0/3643701221 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010c010 0x7f95d0112840 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 --2- 172.21.15.114:0/3643701221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d010b6c0 0x7f95d010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 --2- 172.21.15.114:0/3643701221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 -- 172.21.15.114:0/3643701221 >> 172.21.15.114:0/3643701221 conn(0x7f95d0101810 msgr2=0x7f95d0103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 -- 172.21.15.114:0/3643701221 shutdown_connections 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.695+0000 7f95d7000640 1 -- 172.21.15.114:0/3643701221 wait complete. 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.696+0000 7f95d7000640 1 Processor -- start 2024-09-21T06:55:02.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.696+0000 7f95d7000640 1 -- start start 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d7000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d7000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010b6c0 0x7f95d019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d7000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d010c010 0x7f95d01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d7000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0115cd0 con 0x7f95d010b6c0 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d7000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0115b50 con 0x7f95d010c010 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d7000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0115e50 con 0x7f95d0105d80 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53522/0 (socket says 172.21.15.114:53522) 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95cfe00640 1 -- 172.21.15.114:0/1641713514 learned_addr learned my addr 172.21.15.114:0/1641713514 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95d5000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d010c010 0x7f95d01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.697+0000 7f95cf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010b6c0 0x7f95d019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95cfe00640 1 -- 172.21.15.114:0/1641713514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d010c010 msgr2=0x7f95d01a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:02.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95cfe00640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d010c010 0x7f95d01a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95cfe00640 1 -- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010b6c0 msgr2=0x7f95d019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95cfe00640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010b6c0 0x7f95d019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95cfe00640 1 -- 172.21.15.114:0/1641713514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d01a25d0 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95cfe00640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d019c660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f95c000e9f0 tx=0x7f95c000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c000cd80 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c00054d0 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0010690 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d01a28c0 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.698+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d01a9020 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.699+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0106190 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.702+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 24) ==== 154587+0+0 (secure 0 0 0) 0x7f95c0004e90 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.704+0000 7f95ccc00640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f95b40b6670 0x7f95b40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.704+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f95c00e7fe0 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.705+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95c00ae280 con 0x7f95d0105d80 2024-09-21T06:55:02.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.707+0000 7f95cf400640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f95b40b6670 0x7f95b40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.709+0000 7f95cf400640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f95b40b6670 0x7f95b40b8b80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95d0105060 tx=0x7f95bc00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.718+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f95c00a9df0 con 0x7f95d0105d80 2024-09-21T06:55:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.939+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95d019d790 con 0x7f95d0105d80 2024-09-21T06:55:02.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.940+0000 7f95ccc00640 1 -- 172.21.15.114:0/1641713514 <== mon.1 v2:172.21.15.193:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f95c0064760 con 0x7f95d0105d80 2024-09-21T06:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:55:02.991 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f95b40b6670 msgr2=0x7f95b40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f95b40b6670 0x7f95b40b8b80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f95d0105060 tx=0x7f95bc00b410 comp rx=0 tx=0).stop 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 msgr2=0x7f95d019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d019c660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f95c000e9f0 tx=0x7f95c000edf0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 shutdown_connections 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f95b40b6670 0x7f95b40b8b80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95d010c010 0x7f95d01a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95d010b6c0 0x7f95d019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 --2- 172.21.15.114:0/1641713514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95d0105d80 0x7f95d019c660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 >> 172.21.15.114:0/1641713514 conn(0x7f95d0101810 msgr2=0x7f95d0108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 shutdown_connections 2024-09-21T06:55:02.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:01.943+0000 7f95d7000640 1 -- 172.21.15.114:0/1641713514 wait complete. 2024-09-21T06:55:03.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:02 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1641713514' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:02 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1641713514' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:02 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1641713514' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:03.854 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id f894fe5 --release --await' 2024-09-21T06:55:03.854 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id f894fe5 --release --await 2024-09-21T06:55:04.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:03 smithi114 ceph-mon[30654]: pgmap v424: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 9.5 MiB/s wr, 500 op/s 2024-09-21T06:55:04.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:03 smithi183 ceph-mon[34219]: pgmap v424: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 9.5 MiB/s wr, 500 op/s 2024-09-21T06:55:04.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:03 smithi193 ceph-mon[34601]: pgmap v424: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 9.5 MiB/s wr, 500 op/s 2024-09-21T06:55:04.229 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:04.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 -- 172.21.15.114:0/740716935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc07810b6d0 msgr2=0x7fc07810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:04.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 --2- 172.21.15.114:0/740716935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc07810b6d0 0x7fc07810bae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc064009900 tx=0x7fc06402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 -- 172.21.15.114:0/740716935 shutdown_connections 2024-09-21T06:55:04.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 --2- 172.21.15.114:0/740716935 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc07810c020 0x7fc078112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 --2- 172.21.15.114:0/740716935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc07810b6d0 0x7fc07810bae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 --2- 172.21.15.114:0/740716935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc078105d90 0x7fc0781061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 -- 172.21.15.114:0/740716935 >> 172.21.15.114:0/740716935 conn(0x7fc078101860 msgr2=0x7fc078103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:04.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 -- 172.21.15.114:0/740716935 shutdown_connections 2024-09-21T06:55:04.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.951+0000 7fc07e600640 1 -- 172.21.15.114:0/740716935 wait complete. 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.952+0000 7fc07e600640 1 Processor -- start 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 -- start start 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 0x7fc07819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc07810b6d0 0x7fc07819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc07810c020 0x7fc0781a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc078115ce0 con 0x7fc07810b6d0 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc078115b60 con 0x7fc078105d90 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc07e600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc078115e60 con 0x7fc07810c020 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc077400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 0x7fc07819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc076a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc07810b6d0 0x7fc07819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc077400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 0x7fc07819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:51836/0 (socket says 172.21.15.114:51836) 2024-09-21T06:55:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc077400640 1 -- 172.21.15.114:0/3897214339 learned_addr learned my addr 172.21.15.114:0/3897214339 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.953+0000 7fc077e00640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc07810c020 0x7fc0781a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077400640 1 -- 172.21.15.114:0/3897214339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc07810c020 msgr2=0x7fc0781a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077400640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc07810c020 0x7fc0781a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077400640 1 -- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc07810b6d0 msgr2=0x7fc07819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077400640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc07810b6d0 0x7fc07819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077400640 1 -- 172.21.15.114:0/3897214339 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0781a2660 con 0x7fc078105d90 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077e00640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc07810c020 0x7fc0781a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.954+0000 7fc077400640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 0x7fc07819c6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e9f0 tx=0x7fc06800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.955+0000 7fc06fe00640 1 -- 172.21.15.114:0/3897214339 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800cd80 con 0x7fc078105d90 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.955+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0781a2950 con 0x7fc078105d90 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.955+0000 7fc06fe00640 1 -- 172.21.15.114:0/3897214339 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680054d0 con 0x7fc078105d90 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.955+0000 7fc06fe00640 1 -- 172.21.15.114:0/3897214339 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068010690 con 0x7fc078105d90 2024-09-21T06:55:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.955+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781a90b0 con 0x7fc078105d90 2024-09-21T06:55:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.958+0000 7fc085236740 1 Processor -- start 2024-09-21T06:55:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.958+0000 7fc085236740 1 -- start start 2024-09-21T06:55:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.958+0000 7fc06fe00640 1 -- 172.21.15.114:0/3897214339 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc068004e90 con 0x7fc078105d90 2024-09-21T06:55:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc085236740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6834520 0x55a1a6bdb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc085236740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a680b660 0x55a1a6bdb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc085236740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6813840 0x55a1a6bdbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc085236740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6b52290 con 0x55a1a6813840 2024-09-21T06:55:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc085236740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6b416c0 con 0x55a1a680b660 2024-09-21T06:55:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc085236740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6b454d0 con 0x55a1a6834520 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc077400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a680b660 0x55a1a6bdb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc077e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6834520 0x55a1a6bdb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc077e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6834520 0x55a1a6bdb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44380/0 (socket says 172.21.15.114:44380) 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc077e00640 1 -- 172.21.15.114:0/1758702679 learned_addr learned my addr 172.21.15.114:0/1758702679 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.959+0000 7fc076a00640 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6813840 0x55a1a6bdbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc077400640 1 -- 172.21.15.114:0/1758702679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6834520 msgr2=0x55a1a6bdb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc077400640 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6834520 0x55a1a6bdb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc077400640 1 -- 172.21.15.114:0/1758702679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6813840 msgr2=0x55a1a6bdbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc077400640 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6813840 0x55a1a6bdbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc077400640 1 -- 172.21.15.114:0/1758702679 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a6ace460 con 0x55a1a680b660 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc077400640 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a680b660 0x55a1a6bdb700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0680e5240 tx=0x7fc0680ebf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccba27e0703acae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:04.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.960+0000 7fc06fe00640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc05c0b6680 0x7fc05c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc06cc00640 1 -- 172.21.15.114:0/1758702679 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680fb070 con 0x55a1a680b660 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc06cc00640 1 -- 172.21.15.114:0/1758702679 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680cf700 con 0x55a1a680b660 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc06cc00640 1 -- 172.21.15.114:0/1758702679 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680ad0b0 con 0x55a1a680b660 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc06fe00640 1 -- 172.21.15.114:0/3897214339 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc0680e7e60 con 0x7fc078105d90 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc077e00640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc05c0b6680 0x7fc05c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc085236740 1 -- 172.21.15.114:0/1758702679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a680b660 msgr2=0x55a1a6bdb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.961+0000 7fc077e00640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc05c0b6680 0x7fc05c0b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc070009fd0 tx=0x7fc070009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a680b660 0x55a1a6bdb700 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc0680e5240 tx=0x7fc0680ebf10 comp rx=0 tx=0).stop 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 -- 172.21.15.114:0/1758702679 shutdown_connections 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6813840 0x55a1a6bdbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a680b660 0x55a1a6bdb700 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 --2- 172.21.15.114:0/1758702679 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6834520 0x55a1a6bdb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 -- 172.21.15.114:0/1758702679 >> 172.21.15.114:0/1758702679 conn(0x55a1a6828910 msgr2=0x55a1a6bd90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 -- 172.21.15.114:0/1758702679 shutdown_connections 2024-09-21T06:55:04.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 -- 172.21.15.114:0/1758702679 wait complete. 2024-09-21T06:55:04.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.962+0000 7fc085236740 1 -- start start 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 Processor -- start 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6be6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6be7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6bf2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a1a69a2bb0 con 0x55a1a6834520 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6adc6c0 con 0x55a1a6813840 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6be99a0 con 0x55a1a680b660 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc085236740 10 client.0 _ll_register_callbacks cb 0x55a1a6af7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc077e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6bf2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc076a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6be7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc077400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6be6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.963+0000 7fc077e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6bf2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55014/0 (socket says 172.21.15.114:55014) 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc077e00640 1 -- 172.21.15.114:0/309768360 learned_addr learned my addr 172.21.15.114:0/309768360 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc076a00640 1 -- 172.21.15.114:0/309768360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 msgr2=0x55a1a6be6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc076a00640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6be6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc076a00640 1 -- 172.21.15.114:0/309768360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 msgr2=0x55a1a6bf2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc076a00640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6bf2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc076a00640 1 -- 172.21.15.114:0/309768360 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a689fb00 con 0x55a1a6813840 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc077400640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6be6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc077e00640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6bf2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc076a00640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6be7260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc06402a910 tx=0x7fc064032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c542553df6d770d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.964+0000 7fc061600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc061600640 1 -- 172.21.15.114:0/309768360 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064035040 con 0x55a1a6813840 2024-09-21T06:55:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc061600640 1 -- 172.21.15.114:0/309768360 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc064031760 con 0x55a1a6813840 2024-09-21T06:55:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0680cf700 con 0x55a1a6813840 2024-09-21T06:55:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc061600640 1 -- 172.21.15.114:0/309768360 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc06404f050 con 0x55a1a6813840 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc085236740 10 client.25946 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a1a6916960 con 0x55a1a6813840 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.965+0000 7fc061600640 1 -- 172.21.15.114:0/309768360 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680cf700 con 0x55a1a6813840 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc061600640 1 -- 172.21.15.114:0/309768360 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc064004750 con 0x55a1a6813840 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc085236740 10 client.25946 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc085236740 10 client.25946 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc085236740 20 client.25946 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc085236740 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67f2ea0 0x55a1a6bf8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc085236740 4 client.25946 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1a6b541e0 con 0x55a1a67f2ea0 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc077400640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67f2ea0 0x55a1a6bf8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc077400640 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67f2ea0 0x55a1a6bf8d70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0618f048865faa0 server_cookie=6c17407ba4b98d51 in_seq=0 out_seq=0 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.966+0000 7fc061600640 10 client.25946 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.967+0000 7fc061600640 1 -- 172.21.15.114:0/309768360 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1a6b541e0 con 0x55a1a67f2ea0 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.967+0000 7fc061600640 10 client.25946 handle_command_reply: tid=0 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.967+0000 7fc085236740 1 client.25946 shutdown 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 msgr2=0x55a1a6be7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6be7260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc06402a910 tx=0x7fc064032e90 comp rx=0 tx=0).stop 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 shutdown_connections 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6bf2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6be7260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67f2ea0 0x55a1a6bf8d70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 --2- 172.21.15.114:0/309768360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6be6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc061600640 0 client.25946 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 >> 172.21.15.114:0/309768360 conn(0x55a1a6828910 msgr2=0x55a1a6bdc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 shutdown_connections 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 1 -- 172.21.15.114:0/309768360 wait complete. 2024-09-21T06:55:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.968+0000 7fc085236740 20 client.25946 trim_cache size 0 max 16384 2024-09-21T06:55:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 Processor -- start 2024-09-21T06:55:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 -- start start 2024-09-21T06:55:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 0x55a1a6bdcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6813840 0x55a1a6be12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6834520 0x55a1a6be8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781b2f40 con 0x55a1a680b660 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781b4b40 con 0x55a1a6834520 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc085236740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781b3610 con 0x55a1a6813840 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc077400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 0x55a1a6bdcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc077400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 0x55a1a6bdcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55024/0 (socket says 172.21.15.114:55024) 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.986+0000 7fc077400640 1 -- 172.21.15.114:0/707215930 learned_addr learned my addr 172.21.15.114:0/707215930 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077e00640 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6834520 0x55a1a6be8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc076a00640 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6813840 0x55a1a6be12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077400640 1 -- 172.21.15.114:0/707215930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6813840 msgr2=0x55a1a6be12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077400640 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6813840 0x55a1a6be12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077400640 1 -- 172.21.15.114:0/707215930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6834520 msgr2=0x55a1a6be8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077400640 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6834520 0x55a1a6be8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077400640 1 -- 172.21.15.114:0/707215930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a6be7de0 con 0x55a1a680b660 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc077400640 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 0x55a1a6bdcbb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0680ad640 tx=0x7fc068004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d71bf1637ab2674 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc063e00640 1 -- 172.21.15.114:0/707215930 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06807e030 con 0x55a1a680b660 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc063e00640 1 -- 172.21.15.114:0/707215930 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680eb780 con 0x55a1a680b660 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.987+0000 7fc063e00640 1 -- 172.21.15.114:0/707215930 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06807e220 con 0x55a1a680b660 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 -- 172.21.15.114:0/707215930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 msgr2=0x55a1a6bdcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 0x55a1a6bdcbb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0680ad640 tx=0x7fc068004a90 comp rx=0 tx=0).stop 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 -- 172.21.15.114:0/707215930 shutdown_connections 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6834520 0x55a1a6be8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a6813840 0x55a1a6be12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 --2- 172.21.15.114:0/707215930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a680b660 0x55a1a6bdcbb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 -- 172.21.15.114:0/707215930 >> 172.21.15.114:0/707215930 conn(0x55a1a6828910 msgr2=0x55a1a6be8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.988+0000 7fc085236740 1 -- 172.21.15.114:0/707215930 shutdown_connections 2024-09-21T06:55:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.989+0000 7fc085236740 1 -- 172.21.15.114:0/707215930 wait complete. 2024-09-21T06:55:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.989+0000 7fc085236740 1 -- start start 2024-09-21T06:55:04.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.989+0000 7fc085236740 1 Processor -- start 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6beae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6beb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6beb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781b43b0 con 0x55a1a6834520 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781b3ce0 con 0x55a1a6813840 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc0781b0f20 con 0x55a1a680b660 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc085236740 10 client.0 _ll_register_callbacks cb 0x7fc064009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6beae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6beae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44408/0 (socket says 172.21.15.114:44408) 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc076a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6beb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 -- 172.21.15.114:0/1323154579 learned_addr learned my addr 172.21.15.114:0/1323154579 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:04.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077e00640 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6beb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 -- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 msgr2=0x55a1a6beb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6beb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 -- 172.21.15.114:0/1323154579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 msgr2=0x55a1a6beb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6beb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.990+0000 7fc077400640 1 -- 172.21.15.114:0/1323154579 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a6c62960 con 0x55a1a680b660 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc077400640 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6beae40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc06807dfd0 tx=0x7fc06800e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa10afd1b2d42bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc061600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc061600640 1 -- 172.21.15.114:0/1323154579 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06807e7a0 con 0x55a1a680b660 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0680eb780 con 0x55a1a680b660 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc061600640 1 -- 172.21.15.114:0/1323154579 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc0680a9bb0 con 0x55a1a680b660 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc061600640 1 -- 172.21.15.114:0/1323154579 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc06807e9a0 con 0x55a1a680b660 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc085236740 10 client.25945 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a1a6c63c20 con 0x55a1a680b660 2024-09-21T06:55:04.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc061600640 1 -- 172.21.15.114:0/1323154579 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680eb780 con 0x55a1a680b660 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.991+0000 7fc061600640 1 -- 172.21.15.114:0/1323154579 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc0680eb040 con 0x55a1a680b660 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc085236740 10 client.25945 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc085236740 10 client.25945 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc085236740 20 client.25945 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc085236740 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67c8250 0x55a1a6c65e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc085236740 4 client.25945 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f894fe5", "release": true, "await": true}] 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f894fe5", "release": true, "await": true}) -- 0x55a1a6b4fc90 con 0x55a1a67c8250 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc076a00640 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67c8250 0x55a1a6c65e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc076a00640 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67c8250 0x55a1a6c65e50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5b4ef5a9ccc4de server_cookie=aac8de5ca7dc589c in_seq=0 out_seq=0 2024-09-21T06:55:04.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:04.992+0000 7fc061600640 10 client.25945 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:05.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.222+0000 7fc061600640 1 -- 172.21.15.114:0/1323154579 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55a1a6b4fc90 con 0x55a1a67c8250 2024-09-21T06:55:05.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc061600640 10 client.25945 handle_command_reply: tid=0 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 client.25945 shutdown 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 msgr2=0x55a1a6beae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6beae40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc06807dfd0 tx=0x7fc06800e940 comp rx=0 tx=0).stop 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 shutdown_connections 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a1a6834520 0x55a1a6beb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a1a6813840 0x55a1a6beb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a1a67c8250 0x55a1a6c65e50 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 --2- 172.21.15.114:0/1323154579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a1a680b660 0x55a1a6beae40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc061600640 0 client.25945 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.223+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 >> 172.21.15.114:0/1323154579 conn(0x55a1a6828910 msgr2=0x55a1a6bdc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.224+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 shutdown_connections 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.224+0000 7fc085236740 1 -- 172.21.15.114:0/1323154579 wait complete. 2024-09-21T06:55:05.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.224+0000 7fc085236740 20 client.25945 trim_cache size 0 max 16384 2024-09-21T06:55:05.228 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:55:05.228 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 32, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 4, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "f894fe5": { 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 4, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 134.5, 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:55:05.229 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:05.230 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc05c0b6680 msgr2=0x7fc05c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc05c0b6680 0x7fc05c0b8b90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc070009fd0 tx=0x7fc070009500 comp rx=0 tx=0).stop 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 msgr2=0x7fc07819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 0x7fc07819c6f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc06800e9f0 tx=0x7fc06800edf0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 shutdown_connections 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc07810c020 0x7fc0781a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc07810b6d0 0x7fc07819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc05c0b6680 0x7fc05c0b8b90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 --2- 172.21.15.114:0/3897214339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc078105d90 0x7fc07819c6f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 >> 172.21.15.114:0/3897214339 conn(0x7fc078101860 msgr2=0x7fc078103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 shutdown_connections 2024-09-21T06:55:05.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:05.226+0000 7fc07e600640 1 -- 172.21.15.114:0/3897214339 wait complete. 2024-09-21T06:55:06.349 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f894fe5', total seconds elapsed: 20.2 2024-09-21T06:55:06.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:06 smithi193 ceph-mon[34601]: pgmap v425: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 8.0 MiB/s wr, 372 op/s 2024-09-21T06:55:07.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:06 smithi114 ceph-mon[30654]: pgmap v425: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 8.0 MiB/s wr, 372 op/s 2024-09-21T06:55:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:06 smithi183 ceph-mon[34219]: pgmap v425: 129 pgs: 129 active+clean; 967 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 8.0 MiB/s wr, 372 op/s 2024-09-21T06:55:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 30.0 s, 672.9 tps, lat 477.868 ms stddev 2330.005 2024-09-21T06:55:08.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:08 smithi193 ceph-mon[34601]: pgmap v426: 129 pgs: 129 active+clean; 942 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 29 MiB/s wr, 574 op/s 2024-09-21T06:55:09.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:08 smithi114 ceph-mon[30654]: pgmap v426: 129 pgs: 129 active+clean; 942 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 29 MiB/s wr, 574 op/s 2024-09-21T06:55:09.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:08 smithi183 ceph-mon[34219]: pgmap v426: 129 pgs: 129 active+clean; 942 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 29 MiB/s wr, 574 op/s 2024-09-21T06:55:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 -- 172.21.15.114:0/2818105805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c077390 msgr2=0x7f1c4c0757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 --2- 172.21.15.114:0/2818105805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c077390 0x7f1c4c0757b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009900 tx=0x7f1c3c02a410 comp rx=0 tx=0).stop 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 -- 172.21.15.114:0/2818105805 shutdown_connections 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 --2- 172.21.15.114:0/2818105805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c4c1143a0 0x7f1c4c116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 --2- 172.21.15.114:0/2818105805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c0761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 --2- 172.21.15.114:0/2818105805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c077390 0x7f1c4c0757b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 -- 172.21.15.114:0/2818105805 >> 172.21.15.114:0/2818105805 conn(0x7f1c4c0ff520 msgr2=0x7f1c4c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 -- 172.21.15.114:0/2818105805 shutdown_connections 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.421+0000 7f1c51800640 1 -- 172.21.15.114:0/2818105805 wait complete. 2024-09-21T06:55:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.422+0000 7f1c51800640 1 Processor -- start 2024-09-21T06:55:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.422+0000 7f1c51800640 1 -- start start 2024-09-21T06:55:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.423+0000 7f1c51800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c1a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c51800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c4c077390 0x7f1c4c1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c51800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c1143a0 0x7f1c4c1a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c1a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c1a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44416/0 (socket says 172.21.15.114:44416) 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c4aa00640 1 -- 172.21.15.114:0/2583228183 learned_addr learned my addr 172.21.15.114:0/2583228183 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c11a000 con 0x7f1c4c077390 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c119e80 con 0x7f1c4c1143a0 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.424+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c11a180 con 0x7f1c4c075dd0 2024-09-21T06:55:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4b400640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c1143a0 0x7f1c4c1a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4aa00640 1 -- 172.21.15.114:0/2583228183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c1143a0 msgr2=0x7f1c4c1a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c1143a0 0x7f1c4c1a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4aa00640 1 -- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c4c077390 msgr2=0x7f1c4c1a0fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4a000640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c4c077390 0x7f1c4c1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c4c077390 0x7f1c4c1a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.425+0000 7f1c4aa00640 1 -- 172.21.15.114:0/2583228183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c1a6950 con 0x7f1c4c075dd0 2024-09-21T06:55:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.426+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c1a0a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.426+0000 7f1c43400640 1 -- 172.21.15.114:0/2583228183 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c040030 con 0x7f1c4c075dd0 2024-09-21T06:55:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.426+0000 7f1c43400640 1 -- 172.21.15.114:0/2583228183 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c3c003660 con 0x7f1c4c075dd0 2024-09-21T06:55:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.426+0000 7f1c43400640 1 -- 172.21.15.114:0/2583228183 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c031900 con 0x7f1c4c075dd0 2024-09-21T06:55:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.426+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c1a6be0 con 0x7f1c4c075dd0 2024-09-21T06:55:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.426+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a7080 con 0x7f1c4c075dd0 2024-09-21T06:55:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.431+0000 7f1c588d7740 1 Processor -- start 2024-09-21T06:55:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.431+0000 7f1c43400640 1 -- 172.21.15.114:0/2583228183 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1c3c043030 con 0x7f1c4c075dd0 2024-09-21T06:55:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.434+0000 7f1c43400640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c300b6700 0x7f1c300b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.434+0000 7f1c43400640 1 -- 172.21.15.114:0/2583228183 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f1c3c108200 con 0x7f1c4c075dd0 2024-09-21T06:55:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.434+0000 7f1c588d7740 1 -- start start 2024-09-21T06:55:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.434+0000 7f1c4b400640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c300b6700 0x7f1c300b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636477211b0 0x563647ac5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636476f82f0 0x563647ac6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c588d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563647a3b6b0 con 0x5636477211b0 2024-09-21T06:55:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c588d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563647a2aae0 con 0x5636476f82f0 2024-09-21T06:55:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c588d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563647a2e8f0 con 0x5636477004d0 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.435+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44432/0 (socket says 172.21.15.114:44432) 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 -- 172.21.15.114:0/871286416 learned_addr learned my addr 172.21.15.114:0/871286416 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 -- 172.21.15.114:0/871286416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636476f82f0 msgr2=0x563647ac6170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636476f82f0 0x563647ac6170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 -- 172.21.15.114:0/871286416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636477211b0 msgr2=0x563647ac5c30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636477211b0 0x563647ac5c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.436+0000 7f1c4b400640 1 -- 172.21.15.114:0/871286416 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636479f7780 con 0x5636477004d0 2024-09-21T06:55:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.437+0000 7f1c4b400640 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac66b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c44010d90 tx=0x7f1c4401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0fa755952b3486 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.437+0000 7f1c4b400640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c300b6700 0x7f1c300b8c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c44014bd0 tx=0x7f1c44014750 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.438+0000 7f1c37e00640 1 -- 172.21.15.114:0/871286416 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44026070 con 0x5636477004d0 2024-09-21T06:55:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.438+0000 7f1c37e00640 1 -- 172.21.15.114:0/871286416 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c44014230 con 0x5636477004d0 2024-09-21T06:55:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.438+0000 7f1c37e00640 1 -- 172.21.15.114:0/871286416 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44026070 con 0x5636477004d0 2024-09-21T06:55:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.439+0000 7f1c588d7740 1 -- 172.21.15.114:0/871286416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 msgr2=0x563647ac66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.439+0000 7f1c588d7740 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac66b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c44010d90 tx=0x7f1c4401c000 comp rx=0 tx=0).stop 2024-09-21T06:55:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 -- 172.21.15.114:0/871286416 shutdown_connections 2024-09-21T06:55:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac66b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636476f82f0 0x563647ac6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 --2- 172.21.15.114:0/871286416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636477211b0 0x563647ac5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 -- 172.21.15.114:0/871286416 >> 172.21.15.114:0/871286416 conn(0x563647715300 msgr2=0x563647ac3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 -- 172.21.15.114:0/871286416 shutdown_connections 2024-09-21T06:55:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.440+0000 7f1c588d7740 1 -- 172.21.15.114:0/871286416 wait complete. 2024-09-21T06:55:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.442+0000 7f1c588d7740 1 -- start start 2024-09-21T06:55:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.442+0000 7f1c588d7740 1 Processor -- start 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.443+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ad1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.443+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477004d0 0x563647ad1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.443+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 0x563647add600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.443+0000 7f1c588d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563647962ab0 con 0x5636476f82f0 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c588d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5636479d1570 con 0x5636477004d0 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c588d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563647ad4350 con 0x5636477211b0 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c588d7740 10 client.0 _ll_register_callbacks cb 0x5636479e0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ad1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 0x563647add600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 0x563647add600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44436/0 (socket says 172.21.15.114:44436) 2024-09-21T06:55:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 -- 172.21.15.114:0/3333332331 learned_addr learned my addr 172.21.15.114:0/3333332331 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4a000640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477004d0 0x563647ad1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 -- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477004d0 msgr2=0x563647ad1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477004d0 0x563647ad1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 -- 172.21.15.114:0/3333332331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 msgr2=0x563647ad1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ad1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4b400640 1 -- 172.21.15.114:0/3333332331 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364778c790 con 0x5636477211b0 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.444+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ad1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:55:09.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.445+0000 7f1c4b400640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 0x563647add600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009e70 tx=0x7f1c44004c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddc88f27bd40de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.446+0000 7f1c34c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.446+0000 7f1c34c00640 1 -- 172.21.15.114:0/3333332331 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4402e020 con 0x5636477211b0 2024-09-21T06:55:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.446+0000 7f1c34c00640 1 -- 172.21.15.114:0/3333332331 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f1c44005040 con 0x5636477211b0 2024-09-21T06:55:09.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.446+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c44014230 con 0x5636477211b0 2024-09-21T06:55:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.447+0000 7f1c34c00640 1 -- 172.21.15.114:0/3333332331 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1c44039050 con 0x5636477211b0 2024-09-21T06:55:09.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.448+0000 7f1c588d7740 10 client.25957 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.448+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563647805620 con 0x5636477211b0 2024-09-21T06:55:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.449+0000 7f1c34c00640 1 -- 172.21.15.114:0/3333332331 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c44014230 con 0x5636477211b0 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.449+0000 7f1c34c00640 1 -- 172.21.15.114:0/3333332331 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1c44029030 con 0x5636477211b0 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c588d7740 10 client.25957 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c588d7740 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c588d7740 20 client.25957 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c588d7740 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476dfd80 0x563647ae32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c588d7740 4 client.25957 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:09.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563647a3d600 con 0x5636476dfd80 2024-09-21T06:55:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.450+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476dfd80 0x563647ae32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.451+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476dfd80 0x563647ae32a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d7c1d95891c46f server_cookie=21b48e62230df433 in_seq=0 out_seq=0 2024-09-21T06:55:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.451+0000 7f1c34c00640 10 client.25957 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.451+0000 7f1c34c00640 1 -- 172.21.15.114:0/3333332331 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563647a3d600 con 0x5636476dfd80 2024-09-21T06:55:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.451+0000 7f1c34c00640 10 client.25957 handle_command_reply: tid=0 2024-09-21T06:55:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.452+0000 7f1c588d7740 1 client.25957 shutdown 2024-09-21T06:55:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.452+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 msgr2=0x563647add600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.452+0000 7f1c588d7740 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 0x563647add600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009e70 tx=0x7f1c44004c10 comp rx=0 tx=0).stop 2024-09-21T06:55:09.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 shutdown_connections 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476dfd80 0x563647ae32a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477211b0 0x563647add600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477004d0 0x563647ad1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 --2- 172.21.15.114:0/3333332331 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ad1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 >> 172.21.15.114:0/3333332331 conn(0x563647715300 msgr2=0x563647ac6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c34c00640 0 client.25957 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:09.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 shutdown_connections 2024-09-21T06:55:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.453+0000 7f1c588d7740 1 -- 172.21.15.114:0/3333332331 wait complete. 2024-09-21T06:55:09.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.454+0000 7f1c588d7740 20 client.25957 trim_cache size 0 max 16384 2024-09-21T06:55:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.482+0000 7f1c588d7740 1 Processor -- start 2024-09-21T06:55:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 -- start start 2024-09-21T06:55:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647accab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647ac77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b73a0 con 0x5636476f82f0 2024-09-21T06:55:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b8fa0 con 0x5636477211b0 2024-09-21T06:55:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c588d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b7a70 con 0x5636477004d0 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647ac77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c4aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647accab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647ac77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:51896/0 (socket says 172.21.15.114:51896) 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.483+0000 7f1c4b400640 1 -- 172.21.15.114:0/2989990718 learned_addr learned my addr 172.21.15.114:0/2989990718 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4a000640 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4b400640 1 -- 172.21.15.114:0/2989990718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 msgr2=0x563647ac72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4b400640 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4b400640 1 -- 172.21.15.114:0/2989990718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 msgr2=0x563647accab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4b400640 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647accab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4b400640 1 -- 172.21.15.114:0/2989990718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563647ad4290 con 0x5636477211b0 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647accab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4a000640 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c4b400640 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647ac77f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009530 tx=0x7f1c4400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b125c32887cae61b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c37400640 1 -- 172.21.15.114:0/2989990718 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44018040 con 0x5636477211b0 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c37400640 1 -- 172.21.15.114:0/2989990718 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4400ebc0 con 0x5636477211b0 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.484+0000 7f1c37400640 1 -- 172.21.15.114:0/2989990718 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4402e040 con 0x5636477211b0 2024-09-21T06:55:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 -- 172.21.15.114:0/2989990718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 msgr2=0x563647ac77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647ac77f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009530 tx=0x7f1c4400eef0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 -- 172.21.15.114:0/2989990718 shutdown_connections 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647ac77f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ac72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 --2- 172.21.15.114:0/2989990718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647accab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 -- 172.21.15.114:0/2989990718 >> 172.21.15.114:0/2989990718 conn(0x563647715300 msgr2=0x563647ad3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.486+0000 7f1c588d7740 1 -- 172.21.15.114:0/2989990718 shutdown_connections 2024-09-21T06:55:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.487+0000 7f1c588d7740 1 -- 172.21.15.114:0/2989990718 wait complete. 2024-09-21T06:55:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.487+0000 7f1c588d7740 1 -- start start 2024-09-21T06:55:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.488+0000 7f1c588d7740 1 Processor -- start 2024-09-21T06:55:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ae5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ae6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647b42aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b8810 con 0x5636476f82f0 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b8140 con 0x5636477211b0 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1b5380 con 0x5636477004d0 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c588d7740 10 client.0 _ll_register_callbacks cb 0x7f1c44024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c4a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ae6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c4a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ae6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44450/0 (socket says 172.21.15.114:44450) 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c4a000640 1 -- 172.21.15.114:0/695100654 learned_addr learned my addr 172.21.15.114:0/695100654 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.493+0000 7f1c4b400640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647b42aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ae5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4a000640 1 -- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 msgr2=0x563647b42aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4a000640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647b42aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4a000640 1 -- 172.21.15.114:0/695100654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 msgr2=0x563647ae5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4a000640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ae5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4a000640 1 -- 172.21.15.114:0/695100654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563647b43110 con 0x5636477004d0 2024-09-21T06:55:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4b400640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647b42aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:55:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.494+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ae5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.495+0000 7f1c4a000640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ae6520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1c38013710 tx=0x7f1c38013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0cc639d1211db1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.495+0000 7f1c34c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.495+0000 7f1c34c00640 1 -- 172.21.15.114:0/695100654 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c38021060 con 0x5636477004d0 2024-09-21T06:55:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.495+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4400ebc0 con 0x5636477004d0 2024-09-21T06:55:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.496+0000 7f1c34c00640 1 -- 172.21.15.114:0/695100654 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f1c38019070 con 0x5636477004d0 2024-09-21T06:55:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.496+0000 7f1c34c00640 1 -- 172.21.15.114:0/695100654 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1c38021060 con 0x5636477004d0 2024-09-21T06:55:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.496+0000 7f1c588d7740 10 client.25963 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.496+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563647b4d770 con 0x5636477004d0 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c34c00640 1 -- 172.21.15.114:0/695100654 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4400ebc0 con 0x5636477004d0 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c34c00640 1 -- 172.21.15.114:0/695100654 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1c3800f040 con 0x5636477004d0 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c588d7740 10 client.25963 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c588d7740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c588d7740 20 client.25963 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c588d7740 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476b51c0 0x563647b50000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c588d7740 4 client.25963 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:55:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.497+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563647a390b0 con 0x5636476b51c0 2024-09-21T06:55:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.498+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476b51c0 0x563647b50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.499+0000 7f1c34c00640 10 client.25963 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.499+0000 7f1c4aa00640 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476b51c0 0x563647b50000 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aff426eacc9c77f server_cookie=a3f3fe57bfb10128 in_seq=0 out_seq=0 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c34c00640 1 -- 172.21.15.114:0/695100654 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563647a390b0 con 0x5636476b51c0 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c34c00640 10 client.25963 handle_command_reply: tid=0 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 client.25963 shutdown 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 msgr2=0x563647ae6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ae6520 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1c38013710 tx=0x7f1c38013bd0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 shutdown_connections 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5636477211b0 0x563647b42aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5636476b51c0 0x563647b50000 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5636477004d0 0x563647ae6520 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c34c00640 0 client.25963 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 --2- 172.21.15.114:0/695100654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5636476f82f0 0x563647ae5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.516+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 >> 172.21.15.114:0/695100654 conn(0x563647715300 msgr2=0x563647ad3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.517+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 shutdown_connections 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.517+0000 7f1c588d7740 1 -- 172.21.15.114:0/695100654 wait complete. 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.517+0000 7f1c588d7740 20 client.25963 trim_cache size 0 max 16384 2024-09-21T06:55:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.518+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c300b6700 msgr2=0x7f1c300b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.518+0000 7f1c51800640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c300b6700 0x7f1c300b8c10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1c44014bd0 tx=0x7f1c44014750 comp rx=0 tx=0).stop 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 msgr2=0x7f1c4c1a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c1a0a20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c003f10 comp rx=0 tx=0).stop 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 shutdown_connections 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c300b6700 0x7f1c300b8c10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c4c1143a0 0x7f1c4c1a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c4c077390 0x7f1c4c1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 --2- 172.21.15.114:0/2583228183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c4c075dd0 0x7f1c4c1a0a20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.519+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 >> 172.21.15.114:0/2583228183 conn(0x7f1c4c0ff520 msgr2=0x7f1c4c1102a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.520+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 shutdown_connections 2024-09-21T06:55:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:09.520+0000 7f1c51800640 1 -- 172.21.15.114:0/2583228183 wait complete. 2024-09-21T06:55:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:10 smithi193 ceph-mon[34601]: pgmap v427: 129 pgs: 129 active+clean; 949 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 38 MiB/s wr, 601 op/s 2024-09-21T06:55:10.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:10 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:55:10.915 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:10 smithi183 ceph-mon[34219]: pgmap v427: 129 pgs: 129 active+clean; 949 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 38 MiB/s wr, 601 op/s 2024-09-21T06:55:10.915 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:10 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:55:11.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:10 smithi114 ceph-mon[30654]: pgmap v427: 129 pgs: 129 active+clean; 949 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 38 MiB/s wr, 601 op/s 2024-09-21T06:55:11.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:10 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:55:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:55:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:55:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:55:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:55:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:55:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:55:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:55:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:55:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:55:12.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 35.0 s, 2413.8 tps, lat 41.213 ms stddev 46.343 2024-09-21T06:55:12.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:12 smithi193 ceph-mon[34601]: pgmap v428: 129 pgs: 129 active+clean; 962 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 40 MiB/s wr, 728 op/s 2024-09-21T06:55:13.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:12 smithi114 ceph-mon[30654]: pgmap v428: 129 pgs: 129 active+clean; 962 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 40 MiB/s wr, 728 op/s 2024-09-21T06:55:13.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:12 smithi183 ceph-mon[34219]: pgmap v428: 129 pgs: 129 active+clean; 962 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 40 MiB/s wr, 728 op/s 2024-09-21T06:55:15.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:14 smithi114 ceph-mon[30654]: pgmap v429: 129 pgs: 129 active+clean; 973 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 37 MiB/s wr, 715 op/s 2024-09-21T06:55:15.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:14 smithi183 ceph-mon[34219]: pgmap v429: 129 pgs: 129 active+clean; 973 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 37 MiB/s wr, 715 op/s 2024-09-21T06:55:15.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:14 smithi193 ceph-mon[34601]: pgmap v429: 129 pgs: 129 active+clean; 973 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 37 MiB/s wr, 715 op/s 2024-09-21T06:55:17.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:16 smithi114 ceph-mon[30654]: pgmap v430: 129 pgs: 129 active+clean; 974 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 35 MiB/s wr, 643 op/s 2024-09-21T06:55:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:16 smithi183 ceph-mon[34219]: pgmap v430: 129 pgs: 129 active+clean; 974 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 35 MiB/s wr, 643 op/s 2024-09-21T06:55:17.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:16 smithi193 ceph-mon[34601]: pgmap v430: 129 pgs: 129 active+clean; 974 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 35 MiB/s wr, 643 op/s 2024-09-21T06:55:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 40.0 s, 2299.2 tps, lat 43.590 ms stddev 52.924 2024-09-21T06:55:19.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:18 smithi114 ceph-mon[30654]: pgmap v431: 129 pgs: 129 active+clean; 991 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 MiB/s wr, 837 op/s 2024-09-21T06:55:19.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:18 smithi183 ceph-mon[34219]: pgmap v431: 129 pgs: 129 active+clean; 991 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 MiB/s wr, 837 op/s 2024-09-21T06:55:19.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:18 smithi193 ceph-mon[34601]: pgmap v431: 129 pgs: 129 active+clean; 991 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 37 MiB/s wr, 837 op/s 2024-09-21T06:55:21.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:20 smithi114 ceph-mon[30654]: pgmap v432: 129 pgs: 129 active+clean; 996 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 677 op/s 2024-09-21T06:55:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:20 smithi183 ceph-mon[34219]: pgmap v432: 129 pgs: 129 active+clean; 996 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 677 op/s 2024-09-21T06:55:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:20 smithi193 ceph-mon[34601]: pgmap v432: 129 pgs: 129 active+clean; 996 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 677 op/s 2024-09-21T06:55:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.821+0000 7f7934000640 1 -- 172.21.15.114:0/2511609046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c105cf0 msgr2=0x7f792c106100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.821+0000 7f7934000640 1 --2- 172.21.15.114:0/2511609046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c105cf0 0x7f792c106100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7920009ca0 tx=0x7f792002a8d0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 -- 172.21.15.114:0/2511609046 shutdown_connections 2024-09-21T06:55:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 --2- 172.21.15.114:0/2511609046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792c10bf70 0x7f792c1127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 --2- 172.21.15.114:0/2511609046 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 0x7f792c10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 --2- 172.21.15.114:0/2511609046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c105cf0 0x7f792c106100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 -- 172.21.15.114:0/2511609046 >> 172.21.15.114:0/2511609046 conn(0x7f792c1016f0 msgr2=0x7f792c103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 -- 172.21.15.114:0/2511609046 shutdown_connections 2024-09-21T06:55:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.822+0000 7f7934000640 1 -- 172.21.15.114:0/2511609046 wait complete. 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.824+0000 7f7934000640 1 Processor -- start 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.824+0000 7f7934000640 1 -- start start 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7934000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792c105cf0 0x7f792c10fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7934000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 0x7f792c110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7934000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 0x7f792c110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7934000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f792c115cc0 con 0x7f792c10b620 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7934000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f792c115b40 con 0x7f792c105cf0 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7934000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f792c115e40 con 0x7f792c10bf70 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7932000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 0x7f792c110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7932000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 0x7f792c110760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54564/0 (socket says 172.21.15.114:54564) 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7932000640 1 -- 172.21.15.114:0/2880000801 learned_addr learned my addr 172.21.15.114:0/2880000801 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.825+0000 7f7931600640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792c105cf0 0x7f792c10fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7932000640 1 -- 172.21.15.114:0/2880000801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792c105cf0 msgr2=0x7f792c10fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7930c00640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 0x7f792c110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7932000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792c105cf0 0x7f792c10fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7932000640 1 -- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 msgr2=0x7f792c110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7932000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 0x7f792c110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7932000640 1 -- 172.21.15.114:0/2880000801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c1a2420 con 0x7f792c10bf70 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.826+0000 7f7930c00640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 0x7f792c110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.827+0000 7f7932000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 0x7f792c110760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f792400e9b0 tx=0x7f792400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.827+0000 7f792a000640 1 -- 172.21.15.114:0/2880000801 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792400cd80 con 0x7f792c10bf70 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.827+0000 7f792a000640 1 -- 172.21.15.114:0/2880000801 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79240035f0 con 0x7f792c10bf70 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.827+0000 7f792a000640 1 -- 172.21.15.114:0/2880000801 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792400cd80 con 0x7f792c10bf70 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.827+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c1a2710 con 0x7f792c10bf70 2024-09-21T06:55:22.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.827+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c1a2bb0 con 0x7f792c10bf70 2024-09-21T06:55:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 45.0 s, 2563.3 tps, lat 38.578 ms stddev 47.601 2024-09-21T06:55:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.833+0000 7f793b04a740 1 Processor -- start 2024-09-21T06:55:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.833+0000 7f792a000640 1 -- 172.21.15.114:0/2880000801 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7924003a80 con 0x7f792c10bf70 2024-09-21T06:55:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.835+0000 7f793b04a740 1 -- start start 2024-09-21T06:55:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.835+0000 7f793b04a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305701b0 0x55af30914c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.835+0000 7f793b04a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 0x55af30915170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.835+0000 7f793b04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.835+0000 7f793b04a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55af3088a6b0 con 0x55af305701b0 2024-09-21T06:55:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.835+0000 7f793b04a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af30879ae0 con 0x55af305472f0 2024-09-21T06:55:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.836+0000 7f793b04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af3087d8f0 con 0x55af3054f4d0 2024-09-21T06:55:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.836+0000 7f7931600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 0x55af30915170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.836+0000 7f7931600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 0x55af30915170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52732/0 (socket says 172.21.15.114:52732) 2024-09-21T06:55:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.836+0000 7f7931600640 1 -- 172.21.15.114:0/3976147593 learned_addr learned my addr 172.21.15.114:0/3976147593 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.836+0000 7f792a000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6900 0x7f79140b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7930c00640 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7931600640 1 -- 172.21.15.114:0/3976147593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 msgr2=0x55af309156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7931600640 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309156b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f792a000640 1 -- 172.21.15.114:0/2880000801 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f79240e7aa0 con 0x7f792c10bf70 2024-09-21T06:55:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7931600640 1 -- 172.21.15.114:0/3976147593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305701b0 msgr2=0x55af30914c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7931600640 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305701b0 0x55af30914c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7931600640 1 -- 172.21.15.114:0/3976147593 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af30846780 con 0x55af305472f0 2024-09-21T06:55:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7930c00640 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.837+0000 7f7931600640 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 0x55af30915170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f792002f4d0 tx=0x7f792002f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f14e2f74c65118 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.838+0000 7f7932000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6900 0x7f79140b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.838+0000 7f791e600640 1 -- 172.21.15.114:0/3976147593 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920031d00 con 0x55af305472f0 2024-09-21T06:55:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.838+0000 7f791e600640 1 -- 172.21.15.114:0/3976147593 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7920040070 con 0x55af305472f0 2024-09-21T06:55:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.838+0000 7f791e600640 1 -- 172.21.15.114:0/3976147593 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920031d00 con 0x55af305472f0 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f793b04a740 1 -- 172.21.15.114:0/3976147593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 msgr2=0x55af30915170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f793b04a740 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 0x55af30915170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f792002f4d0 tx=0x7f792002f500 comp rx=0 tx=0).stop 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f7932000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6900 0x7f79140b8e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f792400c6a0 tx=0x7f7924005bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f793b04a740 1 -- 172.21.15.114:0/3976147593 shutdown_connections 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f793b04a740 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f793b04a740 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305472f0 0x55af30915170 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.840+0000 7f793b04a740 1 --2- 172.21.15.114:0/3976147593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305701b0 0x55af30914c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.841+0000 7f793b04a740 1 -- 172.21.15.114:0/3976147593 >> 172.21.15.114:0/3976147593 conn(0x55af30564300 msgr2=0x55af30912b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.841+0000 7f793b04a740 1 -- 172.21.15.114:0/3976147593 shutdown_connections 2024-09-21T06:55:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.841+0000 7f793b04a740 1 -- 172.21.15.114:0/3976147593 wait complete. 2024-09-21T06:55:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.843+0000 7f793b04a740 1 -- start start 2024-09-21T06:55:22.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.844+0000 7f793b04a740 1 Processor -- start 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af309202b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af3092c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55af307b1ab0 con 0x55af305472f0 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af30820570 con 0x55af305701b0 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af30923350 con 0x55af3054f4d0 2024-09-21T06:55:22.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.846+0000 7f793b04a740 10 client.0 _ll_register_callbacks cb 0x55af3082f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.847+0000 7f7931600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.847+0000 7f7930c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af3092c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.847+0000 7f7930c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af3092c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52746/0 (socket says 172.21.15.114:52746) 2024-09-21T06:55:22.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.847+0000 7f7930c00640 1 -- 172.21.15.114:0/30248786 learned_addr learned my addr 172.21.15.114:0/30248786 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.847+0000 7f7930c00640 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af309202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7930c00640 1 -- 172.21.15.114:0/30248786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 msgr2=0x55af309207f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7930c00640 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309207f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7930c00640 1 -- 172.21.15.114:0/30248786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 msgr2=0x55af309202b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7930c00640 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af309202b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7930c00640 1 -- 172.21.15.114:0/30248786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af305db790 con 0x55af305701b0 2024-09-21T06:55:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7930c00640 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af3092c640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7918014810 tx=0x7f791800d240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697d87a1bab49ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:22.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7912a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.848+0000 7f7912a00640 1 -- 172.21.15.114:0/30248786 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791801b060 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.849+0000 7f7912a00640 1 -- 172.21.15.114:0/30248786 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7918011970 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.849+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7920040070 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.849+0000 7f7912a00640 1 -- 172.21.15.114:0/30248786 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7918014e50 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.849+0000 7f793b04a740 10 client.25979 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.849+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af30654620 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.849+0000 7f7912a00640 1 -- 172.21.15.114:0/30248786 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7920040070 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f7912a00640 1 -- 172.21.15.114:0/30248786 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7918015040 con 0x55af305701b0 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f793b04a740 10 client.25979 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f793b04a740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f793b04a740 20 client.25979 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:22.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f793b04a740 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af3052ed80 0x55af30932420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f793b04a740 4 client.25979 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af3088c600 con 0x55af3052ed80 2024-09-21T06:55:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.850+0000 7f7931600640 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af3052ed80 0x55af30932420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.852+0000 7f7912a00640 10 client.25979 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.853+0000 7f7931600640 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af3052ed80 0x55af30932420 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2601bcffe0380aa3 server_cookie=4802d88e45a49382 in_seq=0 out_seq=0 2024-09-21T06:55:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.853+0000 7f7912a00640 1 -- 172.21.15.114:0/30248786 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af3088c600 con 0x55af3052ed80 2024-09-21T06:55:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.853+0000 7f7912a00640 10 client.25979 handle_command_reply: tid=0 2024-09-21T06:55:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 client.25979 shutdown 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 msgr2=0x55af3092c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af3092c640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7918014810 tx=0x7f791800d240 comp rx=0 tx=0).stop 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 shutdown_connections 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af3092c640 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af3052ed80 0x55af30932420 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af3054f4d0 0x55af309207f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 --2- 172.21.15.114:0/30248786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af309202b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 >> 172.21.15.114:0/30248786 conn(0x55af30564300 msgr2=0x55af30915c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.854+0000 7f7912a00640 0 client.25979 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.855+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 shutdown_connections 2024-09-21T06:55:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.855+0000 7f793b04a740 1 -- 172.21.15.114:0/30248786 wait complete. 2024-09-21T06:55:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.855+0000 7f793b04a740 20 client.25979 trim_cache size 0 max 16384 2024-09-21T06:55:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.893+0000 7f793b04a740 1 Processor -- start 2024-09-21T06:55:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.893+0000 7f793b04a740 1 -- start start 2024-09-21T06:55:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.894+0000 7f793b04a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af30911aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.894+0000 7f793b04a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 0x55af30916370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.894+0000 7f793b04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305701b0 0x55af3091d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.894+0000 7f793b04a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1bfd30 con 0x55af305472f0 2024-09-21T06:55:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.894+0000 7f793b04a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1c1930 con 0x55af3054f4d0 2024-09-21T06:55:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.894+0000 7f793b04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1c0400 con 0x55af305701b0 2024-09-21T06:55:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 0x55af30916370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 0x55af30916370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52756/0 (socket says 172.21.15.114:52756) 2024-09-21T06:55:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 -- 172.21.15.114:0/2534735007 learned_addr learned my addr 172.21.15.114:0/2534735007 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7930c00640 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af30911aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 -- 172.21.15.114:0/2534735007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305701b0 msgr2=0x55af3091d480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:55:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305701b0 0x55af3091d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 -- 172.21.15.114:0/2534735007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 msgr2=0x55af30911aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af30911aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 -- 172.21.15.114:0/2534735007 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af30917a40 con 0x55af3054f4d0 2024-09-21T06:55:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.895+0000 7f7931600640 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 0x55af30916370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f79200094e0 tx=0x7f792002a8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d22c5759751719 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.896+0000 7f791dc00640 1 -- 172.21.15.114:0/2534735007 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920031d80 con 0x55af3054f4d0 2024-09-21T06:55:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.896+0000 7f791dc00640 1 -- 172.21.15.114:0/2534735007 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7920040040 con 0x55af3054f4d0 2024-09-21T06:55:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.896+0000 7f791dc00640 1 -- 172.21.15.114:0/2534735007 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792002fde0 con 0x55af3054f4d0 2024-09-21T06:55:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.899+0000 7f793b04a740 1 -- 172.21.15.114:0/2534735007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 msgr2=0x55af30916370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.899+0000 7f793b04a740 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 0x55af30916370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f79200094e0 tx=0x7f792002a8d0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.900+0000 7f793b04a740 1 -- 172.21.15.114:0/2534735007 shutdown_connections 2024-09-21T06:55:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.900+0000 7f793b04a740 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305701b0 0x55af3091d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.900+0000 7f793b04a740 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af3054f4d0 0x55af30916370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.900+0000 7f793b04a740 1 --2- 172.21.15.114:0/2534735007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af305472f0 0x55af30911aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.900+0000 7f793b04a740 1 -- 172.21.15.114:0/2534735007 >> 172.21.15.114:0/2534735007 conn(0x55af30564300 msgr2=0x55af30915c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.901+0000 7f793b04a740 1 -- 172.21.15.114:0/2534735007 shutdown_connections 2024-09-21T06:55:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.901+0000 7f793b04a740 1 -- 172.21.15.114:0/2534735007 wait complete. 2024-09-21T06:55:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.902+0000 7f793b04a740 1 -- start start 2024-09-21T06:55:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.903+0000 7f793b04a740 1 Processor -- start 2024-09-21T06:55:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305472f0 0x55af30928fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af3054f4d0 0x55af30926510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af30926c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1c11a0 con 0x55af3054f4d0 2024-09-21T06:55:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1c0ad0 con 0x55af305701b0 2024-09-21T06:55:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1bdd10 con 0x55af305472f0 2024-09-21T06:55:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f793b04a740 10 client.0 _ll_register_callbacks cb 0x7f791800d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f7930c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af30926c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f7931600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af3054f4d0 0x55af30926510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f7930c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af30926c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52760/0 (socket says 172.21.15.114:52760) 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f7930c00640 1 -- 172.21.15.114:0/283584156 learned_addr learned my addr 172.21.15.114:0/283584156 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.904+0000 7f7930c00640 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305472f0 0x55af30928fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.905+0000 7f7930c00640 1 -- 172.21.15.114:0/283584156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305472f0 msgr2=0x55af30928fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.905+0000 7f7930c00640 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305472f0 0x55af30928fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.905+0000 7f7930c00640 1 -- 172.21.15.114:0/283584156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af3054f4d0 msgr2=0x55af30926510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.905+0000 7f7930c00640 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af3054f4d0 0x55af30926510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.905+0000 7f7930c00640 1 -- 172.21.15.114:0/283584156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3099bdf0 con 0x55af305701b0 2024-09-21T06:55:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.905+0000 7f7930c00640 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af30926c20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55af3092d640 tx=0x7f79180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=223f8f8f6cd652c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.906+0000 7f7912a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.906+0000 7f7912a00640 1 -- 172.21.15.114:0/283584156 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79180143a0 con 0x55af305701b0 2024-09-21T06:55:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.906+0000 7f7912a00640 1 -- 172.21.15.114:0/283584156 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7918016840 con 0x55af305701b0 2024-09-21T06:55:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.906+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7920040040 con 0x55af305701b0 2024-09-21T06:55:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.907+0000 7f7912a00640 1 -- 172.21.15.114:0/283584156 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79180059c0 con 0x55af305701b0 2024-09-21T06:55:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.907+0000 7f793b04a740 10 client.25985 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.907+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55af3099d020 con 0x55af305701b0 2024-09-21T06:55:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f7912a00640 1 -- 172.21.15.114:0/283584156 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7920040040 con 0x55af305701b0 2024-09-21T06:55:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f7912a00640 1 -- 172.21.15.114:0/283584156 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7918027430 con 0x55af305701b0 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f793b04a740 10 client.25985 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f793b04a740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f793b04a740 20 client.25985 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f793b04a740 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af305041c0 0x55af3099f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f793b04a740 4 client.25985 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.908+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af308880b0 con 0x55af305041c0 2024-09-21T06:55:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.909+0000 7f7931600640 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af305041c0 0x55af3099f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.911+0000 7f7931600640 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af305041c0 0x55af3099f880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=391edec2ceeaeb42 server_cookie=3943c9ca3ddbb213 in_seq=0 out_seq=0 2024-09-21T06:55:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.912+0000 7f7912a00640 10 client.25985 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.924+0000 7f7912a00640 1 -- 172.21.15.114:0/283584156 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af308880b0 con 0x55af305041c0 2024-09-21T06:55:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.924+0000 7f7912a00640 10 client.25985 handle_command_reply: tid=0 2024-09-21T06:55:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.924+0000 7f793b04a740 1 client.25985 shutdown 2024-09-21T06:55:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.924+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 msgr2=0x55af30926c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.924+0000 7f793b04a740 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af30926c20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55af3092d640 tx=0x7f79180046c0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 shutdown_connections 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af305701b0 0x55af30926c20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af3054f4d0 0x55af30926510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af305041c0 0x55af3099f880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 --2- 172.21.15.114:0/283584156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af305472f0 0x55af30928fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 >> 172.21.15.114:0/283584156 conn(0x55af30564300 msgr2=0x55af309220d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f7912a00640 0 client.25985 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 shutdown_connections 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 1 -- 172.21.15.114:0/283584156 wait complete. 2024-09-21T06:55:22.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.925+0000 7f793b04a740 20 client.25985 trim_cache size 0 max 16384 2024-09-21T06:55:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:55:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:55:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:55:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:55:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6900 msgr2=0x7f79140b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6900 0x7f79140b8e10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f792400c6a0 tx=0x7f7924005bc0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 msgr2=0x7f792c110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 0x7f792c110760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f792400e9b0 tx=0x7f792400ee70 comp rx=0 tx=0).stop 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 shutdown_connections 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f792c10bf70 0x7f792c110760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792c10b620 0x7f792c110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79140b6900 0x7f79140b8e10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 --2- 172.21.15.114:0/2880000801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792c105cf0 0x7f792c10fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.927+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 >> 172.21.15.114:0/2880000801 conn(0x7f792c1016f0 msgr2=0x7f792c10acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.928+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 shutdown_connections 2024-09-21T06:55:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:22.928+0000 7f7934000640 1 -- 172.21.15.114:0/2880000801 wait complete. 2024-09-21T06:55:23.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:22 smithi114 ceph-mon[30654]: pgmap v433: 129 pgs: 129 active+clean; 1021 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 718 op/s 2024-09-21T06:55:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:22 smithi183 ceph-mon[34219]: pgmap v433: 129 pgs: 129 active+clean; 1021 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 718 op/s 2024-09-21T06:55:23.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:22 smithi193 ceph-mon[34601]: pgmap v433: 129 pgs: 129 active+clean; 1021 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.3 MiB/s wr, 718 op/s 2024-09-21T06:55:25.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:24 smithi114 ceph-mon[30654]: pgmap v434: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.9 MiB/s wr, 632 op/s 2024-09-21T06:55:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:24 smithi183 ceph-mon[34219]: pgmap v434: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.9 MiB/s wr, 632 op/s 2024-09-21T06:55:25.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:24 smithi193 ceph-mon[34601]: pgmap v434: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.9 MiB/s wr, 632 op/s 2024-09-21T06:55:27.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:26 smithi114 ceph-mon[30654]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 578 op/s 2024-09-21T06:55:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:26 smithi183 ceph-mon[34219]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 578 op/s 2024-09-21T06:55:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:26 smithi193 ceph-mon[34601]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.8 MiB/s wr, 578 op/s 2024-09-21T06:55:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 50.0 s, 2214.9 tps, lat 45.446 ms stddev 51.728 2024-09-21T06:55:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:28 smithi114 ceph-mon[30654]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 757 op/s 2024-09-21T06:55:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:28 smithi183 ceph-mon[34219]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 757 op/s 2024-09-21T06:55:29.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:28 smithi193 ceph-mon[34601]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 MiB/s wr, 757 op/s 2024-09-21T06:55:30.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:29 smithi114 ceph-mon[30654]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-09-21T06:55:30.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:29 smithi183 ceph-mon[34219]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-09-21T06:55:30.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:29 smithi193 ceph-mon[34601]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-09-21T06:55:30.604 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:55:31.274 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:32.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.123+0000 7f72ef200640 1 -- 172.21.15.114:0/2448350911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 msgr2=0x7f72e8118950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.124+0000 7f72ef200640 1 --2- 172.21.15.114:0/2448350911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e8118950 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009ca0 tx=0x7f72e402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.124+0000 7f72ef200640 1 -- 172.21.15.114:0/2448350911 shutdown_connections 2024-09-21T06:55:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.124+0000 7f72ef200640 1 --2- 172.21.15.114:0/2448350911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e8118950 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.124+0000 7f72ef200640 1 --2- 172.21.15.114:0/2448350911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e810ad10 0x7f72e8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.124+0000 7f72ef200640 1 --2- 172.21.15.114:0/2448350911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72e810a3c0 0x7f72e810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.124+0000 7f72ef200640 1 -- 172.21.15.114:0/2448350911 >> 172.21.15.114:0/2448350911 conn(0x7f72e806ee50 msgr2=0x7f72e806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:32.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.125+0000 7f72ef200640 1 -- 172.21.15.114:0/2448350911 shutdown_connections 2024-09-21T06:55:32.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.127+0000 7f72ef200640 1 -- 172.21.15.114:0/2448350911 wait complete. 2024-09-21T06:55:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.128+0000 7f72ef200640 1 Processor -- start 2024-09-21T06:55:32.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.128+0000 7f72ef200640 1 -- start start 2024-09-21T06:55:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.129+0000 7f72ef200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e810a3c0 0x7f72e81a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.129+0000 7f72ef200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72e810ad10 0x7f72e81a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.129+0000 7f72ef200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e81aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.129+0000 7f72ef200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72e811e330 con 0x7f72e810ad10 2024-09-21T06:55:32.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.129+0000 7f72ef200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72e811e1b0 con 0x7f72e8116510 2024-09-21T06:55:32.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.129+0000 7f72ef200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72e811e4b0 con 0x7f72e810a3c0 2024-09-21T06:55:32.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ede00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e810a3c0 0x7f72e81a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e81aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e81aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39850/0 (socket says 172.21.15.114:39850) 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 -- 172.21.15.114:0/1472834819 learned_addr learned my addr 172.21.15.114:0/1472834819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 -- 172.21.15.114:0/1472834819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e810a3c0 msgr2=0x7f72e81a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e810a3c0 0x7f72e81a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 -- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72e810ad10 msgr2=0x7f72e81a5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.130+0000 7f72ee800640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72e810ad10 0x7f72e81a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.131+0000 7f72ee800640 1 -- 172.21.15.114:0/1472834819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e81aab50 con 0x7f72e8116510 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.132+0000 7f72ee800640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e81aa4e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005e30 tx=0x7f72e40317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:32.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.132+0000 7f72e2a00640 1 -- 172.21.15.114:0/1472834819 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003b70 con 0x7f72e8116510 2024-09-21T06:55:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.132+0000 7f72e2a00640 1 -- 172.21.15.114:0/1472834819 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e4003d10 con 0x7f72e8116510 2024-09-21T06:55:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.132+0000 7f72e2a00640 1 -- 172.21.15.114:0/1472834819 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e402fa00 con 0x7f72e8116510 2024-09-21T06:55:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.132+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e81aade0 con 0x7f72e8116510 2024-09-21T06:55:32.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.132+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81ab2d0 con 0x7f72e8116510 2024-09-21T06:55:32.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.141+0000 7f72e2a00640 1 -- 172.21.15.114:0/1472834819 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f72e4004580 con 0x7f72e8116510 2024-09-21T06:55:32.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.143+0000 7f72e2a00640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72d00b67b0 0x7f72d00b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.144+0000 7f72e2a00640 1 -- 172.21.15.114:0/1472834819 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f72e4111a80 con 0x7f72e8116510 2024-09-21T06:55:32.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.144+0000 7f72f75f3740 1 Processor -- start 2024-09-21T06:55:32.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.144+0000 7f72ede00640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72d00b67b0 0x7f72d00b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 -- start start 2024-09-21T06:55:32.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c882f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c883470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ee2c7f9f70 con 0x55ee2c4dc070 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee2c7e93a0 con 0x55ee2c4bb390 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.145+0000 7f72f75f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee2c7ed1b0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c883470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c883470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41758/0 (socket says 172.21.15.114:41758) 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 -- 172.21.15.114:0/2703048526 learned_addr learned my addr 172.21.15.114:0/2703048526 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ee800640 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 -- 172.21.15.114:0/2703048526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 msgr2=0x55ee2c8839b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8839b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 -- 172.21.15.114:0/2703048526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 msgr2=0x55ee2c882f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c882f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ed400640 1 -- 172.21.15.114:0/2703048526 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2c7b3c90 con 0x55ee2c4b31b0 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.146+0000 7f72ee800640 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8839b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.147+0000 7f72ed400640 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c883470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009b80 tx=0x7f72d800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f2139c06033499 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:32.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.147+0000 7f72d7000640 1 -- 172.21.15.114:0/2703048526 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004020 con 0x55ee2c4b31b0 2024-09-21T06:55:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.147+0000 7f72d7000640 1 -- 172.21.15.114:0/2703048526 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80041c0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.147+0000 7f72d7000640 1 -- 172.21.15.114:0/2703048526 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004020 con 0x55ee2c4b31b0 2024-09-21T06:55:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.149+0000 7f72ede00640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72d00b67b0 0x7f72d00b8cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00ba40 tx=0x7f72dc005cc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.150+0000 7f72f75f3740 1 -- 172.21.15.114:0/2703048526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 msgr2=0x55ee2c883470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.150+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c883470 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009b80 tx=0x7f72d800e720 comp rx=0 tx=0).stop 2024-09-21T06:55:32.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 -- 172.21.15.114:0/2703048526 shutdown_connections 2024-09-21T06:55:32.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8839b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c883470 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2703048526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c882f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 -- 172.21.15.114:0/2703048526 >> 172.21.15.114:0/2703048526 conn(0x55ee2c4d0460 msgr2=0x55ee2c880e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:32.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 -- 172.21.15.114:0/2703048526 shutdown_connections 2024-09-21T06:55:32.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.152+0000 7f72f75f3740 1 -- 172.21.15.114:0/2703048526 wait complete. 2024-09-21T06:55:32.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.153+0000 7f72f75f3740 1 -- start start 2024-09-21T06:55:32.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.153+0000 7f72f75f3740 1 Processor -- start 2024-09-21T06:55:32.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.154+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.154+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4bb390 0x55ee2c88ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.154+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4dc070 0x55ee2c89aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.154+0000 7f72f75f3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ee2c64a700 con 0x55ee2c4bb390 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.154+0000 7f72f75f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee2c784540 con 0x55ee2c4dc070 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.154+0000 7f72f75f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee2c891710 con 0x55ee2c4b31b0 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.155+0000 7f72f75f3740 10 client.0 _ll_register_callbacks cb 0x55ee2c794430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.155+0000 7f72ede00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.155+0000 7f72ede00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41762/0 (socket says 172.21.15.114:41762) 2024-09-21T06:55:32.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.155+0000 7f72ede00640 1 -- 172.21.15.114:0/2282688833 learned_addr learned my addr 172.21.15.114:0/2282688833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:32.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.156+0000 7f72ee800640 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4dc070 0x55ee2c89aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.156+0000 7f72ede00640 1 -- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4dc070 msgr2=0x55ee2c89aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.156+0000 7f72ede00640 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4dc070 0x55ee2c89aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.156+0000 7f72ede00640 1 -- 172.21.15.114:0/2282688833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4bb390 msgr2=0x55ee2c88ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.156+0000 7f72ede00640 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4bb390 0x55ee2c88ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.156+0000 7f72ede00640 1 -- 172.21.15.114:0/2282688833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2c547650 con 0x55ee2c4b31b0 2024-09-21T06:55:32.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.157+0000 7f72ede00640 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88e700 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00f9d0 tx=0x7f72dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412e73431c69e3d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:32.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.157+0000 7f72cb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:32.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.158+0000 7f72cb400640 1 -- 172.21.15.114:0/2282688833 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc020070 con 0x55ee2c4b31b0 2024-09-21T06:55:32.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.158+0000 7f72cb400640 1 -- 172.21.15.114:0/2282688833 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f72dc00ed40 con 0x55ee2c4b31b0 2024-09-21T06:55:32.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.158+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee2c786b50 con 0x55ee2c4b31b0 2024-09-21T06:55:32.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72cb400640 1 -- 172.21.15.114:0/2282688833 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72dc02a050 con 0x55ee2c4b31b0 2024-09-21T06:55:32.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72f75f3740 10 client.25984 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee2c7a1910 con 0x55ee2c4b31b0 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72cb400640 1 -- 172.21.15.114:0/2282688833 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee2c786b50 con 0x55ee2c4b31b0 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72cb400640 1 -- 172.21.15.114:0/2282688833 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f72dc0254d0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72f75f3740 10 client.25984 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.159+0000 7f72f75f3740 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.160+0000 7f72f75f3740 20 client.25984 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:32.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.160+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c49a9f0 0x55ee2c8a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.160+0000 7f72f75f3740 4 client.25984 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:32.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.160+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee2c7fbec0 con 0x55ee2c49a9f0 2024-09-21T06:55:32.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.160+0000 7f72ed400640 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c49a9f0 0x55ee2c8a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.161+0000 7f72cb400640 10 client.25984 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:32.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.162+0000 7f72ed400640 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c49a9f0 0x55ee2c8a09d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec96e6f41bd16359 server_cookie=23cc4eb665568aa0 in_seq=0 out_seq=0 2024-09-21T06:55:32.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.163+0000 7f72cb400640 1 -- 172.21.15.114:0/2282688833 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ee2c7fbec0 con 0x55ee2c49a9f0 2024-09-21T06:55:32.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.164+0000 7f72cb400640 10 client.25984 handle_command_reply: tid=0 2024-09-21T06:55:32.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.164+0000 7f72f75f3740 1 client.25984 shutdown 2024-09-21T06:55:32.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.164+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 msgr2=0x55ee2c88e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.164+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88e700 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00f9d0 tx=0x7f72dc0046c0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 shutdown_connections 2024-09-21T06:55:32.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4dc070 0x55ee2c89aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4bb390 0x55ee2c88ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c49a9f0 0x55ee2c8a09d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 --2- 172.21.15.114:0/2282688833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88e700 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 >> 172.21.15.114:0/2282688833 conn(0x55ee2c4d0460 msgr2=0x55ee2c886110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:32.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72cb400640 0 client.25984 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:32.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 shutdown_connections 2024-09-21T06:55:32.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.165+0000 7f72f75f3740 1 -- 172.21.15.114:0/2282688833 wait complete. 2024-09-21T06:55:32.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.166+0000 7f72f75f3740 20 client.25984 trim_cache size 0 max 16384 2024-09-21T06:55:32.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.197+0000 7f72f75f3740 1 Processor -- start 2024-09-21T06:55:32.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.197+0000 7f72f75f3740 1 -- start start 2024-09-21T06:55:32.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4dc070 0x55ee2c89a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72f75f3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b72a0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72f75f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b8ea0 con 0x55ee2c4bb390 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72f75f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b7970 con 0x55ee2c4dc070 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72ed400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72ed400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8905e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39884/0 (socket says 172.21.15.114:39884) 2024-09-21T06:55:32.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72ed400640 1 -- 172.21.15.114:0/1232475601 learned_addr learned my addr 172.21.15.114:0/1232475601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.199+0000 7f72ee800640 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4dc070 0x55ee2c89a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.200+0000 7f72ed400640 1 -- 172.21.15.114:0/1232475601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4dc070 msgr2=0x55ee2c89a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.200+0000 7f72ed400640 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4dc070 0x55ee2c89a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.200+0000 7f72ed400640 1 -- 172.21.15.114:0/1232475601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4b31b0 msgr2=0x55ee2c88d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.200+0000 7f72ed400640 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.200+0000 7f72ed400640 1 -- 172.21.15.114:0/1232475601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2c89d3d0 con 0x55ee2c4bb390 2024-09-21T06:55:32.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.200+0000 7f72ed400640 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8905e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72d800f010 tx=0x7f72d8009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5478d3113ef054ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.201+0000 7f72d6600640 1 -- 172.21.15.114:0/1232475601 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8009530 con 0x55ee2c4bb390 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.201+0000 7f72d6600640 1 -- 172.21.15.114:0/1232475601 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80096d0 con 0x55ee2c4bb390 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.201+0000 7f72d6600640 1 -- 172.21.15.114:0/1232475601 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80047a0 con 0x55ee2c4bb390 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.202+0000 7f72f75f3740 1 -- 172.21.15.114:0/1232475601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 msgr2=0x55ee2c8905e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.202+0000 7f72f75f3740 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8905e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f72d800f010 tx=0x7f72d8009d20 comp rx=0 tx=0).stop 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 -- 172.21.15.114:0/1232475601 shutdown_connections 2024-09-21T06:55:32.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4dc070 0x55ee2c89a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4bb390 0x55ee2c8905e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 --2- 172.21.15.114:0/1232475601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4b31b0 0x55ee2c88d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 -- 172.21.15.114:0/1232475601 >> 172.21.15.114:0/1232475601 conn(0x55ee2c4d0460 msgr2=0x55ee2c88eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 -- 172.21.15.114:0/1232475601 shutdown_connections 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 -- 172.21.15.114:0/1232475601 wait complete. 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.203+0000 7f72f75f3740 1 -- start start 2024-09-21T06:55:32.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 Processor -- start 2024-09-21T06:55:32.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4b31b0 0x55ee2c897270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4bb390 0x55ee2c891950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c891f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b8710 con 0x55ee2c4dc070 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b8040 con 0x55ee2c4b31b0 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.204+0000 7f72f75f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b52a0 con 0x55ee2c4bb390 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.205+0000 7f72f75f3740 10 client.0 _ll_register_callbacks cb 0x7f72dc00f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.205+0000 7f72ede00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4b31b0 0x55ee2c897270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.205+0000 7f72ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4bb390 0x55ee2c891950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.205+0000 7f72ed400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4bb390 0x55ee2c891950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41778/0 (socket says 172.21.15.114:41778) 2024-09-21T06:55:32.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.205+0000 7f72ed400640 1 -- 172.21.15.114:0/696442376 learned_addr learned my addr 172.21.15.114:0/696442376 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.206+0000 7f72ee800640 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c891f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.206+0000 7f72ede00640 1 -- 172.21.15.114:0/696442376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4bb390 msgr2=0x55ee2c891950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.206+0000 7f72ede00640 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4bb390 0x55ee2c891950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.206+0000 7f72ede00640 1 -- 172.21.15.114:0/696442376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 msgr2=0x55ee2c891f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.206+0000 7f72ede00640 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c891f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.206+0000 7f72ede00640 1 -- 172.21.15.114:0/696442376 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2c892710 con 0x55ee2c4b31b0 2024-09-21T06:55:32.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.207+0000 7f72ede00640 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4b31b0 0x55ee2c897270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e780 tx=0x7f72dc025670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485676b327bd398a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:32.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.207+0000 7f72cb400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:32.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.208+0000 7f72cb400640 1 -- 172.21.15.114:0/696442376 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc020070 con 0x55ee2c4b31b0 2024-09-21T06:55:32.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.208+0000 7f72cb400640 1 -- 172.21.15.114:0/696442376 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f72dc005040 con 0x55ee2c4b31b0 2024-09-21T06:55:32.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.208+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d80096d0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.208+0000 7f72cb400640 1 -- 172.21.15.114:0/696442376 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72dc03e050 con 0x55ee2c4b31b0 2024-09-21T06:55:32.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.208+0000 7f72f75f3740 10 client.26003 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:32.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.208+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee2c91d3f0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.209+0000 7f72cb400640 1 -- 172.21.15.114:0/696442376 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80096d0 con 0x55ee2c4b31b0 2024-09-21T06:55:32.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.209+0000 7f72cb400640 1 -- 172.21.15.114:0/696442376 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f72dc036070 con 0x55ee2c4b31b0 2024-09-21T06:55:32.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.209+0000 7f72f75f3740 10 client.26003 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:32.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.209+0000 7f72f75f3740 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:32.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.209+0000 7f72f75f3740 20 client.26003 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:32.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.209+0000 7f72f75f3740 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c46fda0 0x55ee2c91fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:32.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.210+0000 7f72f75f3740 4 client.26003 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:55:32.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.210+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee2c7f7970 con 0x55ee2c46fda0 2024-09-21T06:55:32.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.210+0000 7f72ed400640 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c46fda0 0x55ee2c91fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:32.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.211+0000 7f72ed400640 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c46fda0 0x55ee2c91fbd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9177673a265a2dd server_cookie=ba1e3b424512f4d3 in_seq=0 out_seq=0 2024-09-21T06:55:32.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.211+0000 7f72cb400640 10 client.26003 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:32.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.211+0000 7f72cb400640 1 -- 172.21.15.114:0/696442376 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55ee2c7f7970 con 0x55ee2c46fda0 2024-09-21T06:55:32.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.211+0000 7f72cb400640 10 client.26003 handle_command_reply: tid=0 2024-09-21T06:55:32.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.211+0000 7f72f75f3740 1 client.26003 shutdown 2024-09-21T06:55:32.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4b31b0 msgr2=0x55ee2c897270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4b31b0 0x55ee2c897270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00e780 tx=0x7f72dc025670 comp rx=0 tx=0).stop 2024-09-21T06:55:32.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 shutdown_connections 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ee2c4dc070 0x55ee2c891f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ee2c46fda0 0x55ee2c91fbd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee2c4bb390 0x55ee2c891950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 --2- 172.21.15.114:0/696442376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee2c4b31b0 0x55ee2c897270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72cb400640 0 client.26003 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.212+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 >> 172.21.15.114:0/696442376 conn(0x55ee2c4d0460 msgr2=0x55ee2c88eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.213+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 shutdown_connections 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.213+0000 7f72f75f3740 1 -- 172.21.15.114:0/696442376 wait complete. 2024-09-21T06:55:32.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.213+0000 7f72f75f3740 20 client.26003 trim_cache size 0 max 16384 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:55:32.216 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:32.217 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72d00b67b0 msgr2=0x7f72d00b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72d00b67b0 0x7f72d00b8cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00ba40 tx=0x7f72dc005cc0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 msgr2=0x7f72e81aa4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e81aa4e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005e30 tx=0x7f72e40317c0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 shutdown_connections 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72d00b67b0 0x7f72d00b8cc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72e8116510 0x7f72e81aa4e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72e810ad10 0x7f72e81a5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 --2- 172.21.15.114:0/1472834819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72e810a3c0 0x7f72e81a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:32.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.214+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 >> 172.21.15.114:0/1472834819 conn(0x7f72e806ee50 msgr2=0x7f72e8071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:32.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.215+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 shutdown_connections 2024-09-21T06:55:32.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:32.215+0000 7f72ef200640 1 -- 172.21.15.114:0/1472834819 wait complete. 2024-09-21T06:55:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 55.0 s, 1756.8 tps, lat 56.490 ms stddev 74.965 2024-09-21T06:55:32.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:32 smithi193 ceph-mon[34601]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-21T06:55:33.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:32 smithi114 ceph-mon[30654]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-21T06:55:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:32 smithi183 ceph-mon[34219]: pgmap v438: 129 pgs: 129 active+clean; 1.1 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-21T06:55:33.278 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-09-21T06:55:33.279 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:55:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:34 smithi193 ceph-mon[34601]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-09-21T06:55:35.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:34 smithi114 ceph-mon[30654]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-09-21T06:55:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:34 smithi183 ceph-mon[34219]: pgmap v439: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 10 MiB/s wr, 821 op/s 2024-09-21T06:55:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.234+0000 7f3113e00640 1 -- 172.21.15.114:0/3171144604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 msgr2=0x7f3114108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.234+0000 7f3113e00640 1 --2- 172.21.15.114:0/3171144604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f3114108600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3100009900 tx=0x7f310002a410 comp rx=0 tx=0).stop 2024-09-21T06:55:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.235+0000 7f3113e00640 1 -- 172.21.15.114:0/3171144604 shutdown_connections 2024-09-21T06:55:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.235+0000 7f3113e00640 1 --2- 172.21.15.114:0/3171144604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3114108b40 0x7f311410f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.236+0000 7f3113e00640 1 --2- 172.21.15.114:0/3171144604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f3114108600 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.236+0000 7f3113e00640 1 --2- 172.21.15.114:0/3171144604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f31141033d0 0x7f31141037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.236+0000 7f3113e00640 1 -- 172.21.15.114:0/3171144604 >> 172.21.15.114:0/3171144604 conn(0x7f3114078770 msgr2=0x7f3114078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.237+0000 7f3113e00640 1 -- 172.21.15.114:0/3171144604 shutdown_connections 2024-09-21T06:55:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.237+0000 7f3113e00640 1 -- 172.21.15.114:0/3171144604 wait complete. 2024-09-21T06:55:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.239+0000 7f3113e00640 1 Processor -- start 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.239+0000 7f3113e00640 1 -- start start 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3113e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 0x7f31141a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3113e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f31141a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3113e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3114108b40 0x7f31141a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3113e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111690 con 0x7f3114108b40 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3113e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111510 con 0x7f31141081f0 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3113e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3114111810 con 0x7f31141033d0 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3112000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f31141a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3112a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 0x7f31141a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3112a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 0x7f31141a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52702/0 (socket says 172.21.15.114:52702) 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.240+0000 7f3112a00640 1 -- 172.21.15.114:0/1404425417 learned_addr learned my addr 172.21.15.114:0/1404425417 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3113400640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3114108b40 0x7f31141a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3112a00640 1 -- 172.21.15.114:0/1404425417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 msgr2=0x7f31141a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3112a00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f31141a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3112a00640 1 -- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3114108b40 msgr2=0x7f31141a6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3112a00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3114108b40 0x7f31141a6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3112a00640 1 -- 172.21.15.114:0/1404425417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31141a6810 con 0x7f31141033d0 2024-09-21T06:55:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.241+0000 7f3112000640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f31141a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.243+0000 7f3112a00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 0x7f31141a08e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f310400c890 tx=0x7f310400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.243+0000 7f310b400640 1 -- 172.21.15.114:0/1404425417 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104004220 con 0x7f31141033d0 2024-09-21T06:55:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.243+0000 7f310b400640 1 -- 172.21.15.114:0/1404425417 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31040043c0 con 0x7f31141033d0 2024-09-21T06:55:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.243+0000 7f310b400640 1 -- 172.21.15.114:0/1404425417 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104004a00 con 0x7f31141033d0 2024-09-21T06:55:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.243+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141a6b00 con 0x7f31141033d0 2024-09-21T06:55:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.244+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311406cde0 con 0x7f31141033d0 2024-09-21T06:55:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.251+0000 7f3120834740 1 Processor -- start 2024-09-21T06:55:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 -- start start 2024-09-21T06:55:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47710de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 0x55c47710e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c4770836b0 con 0x55c476d484d0 2024-09-21T06:55:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c477072ae0 con 0x55c476d691b0 2024-09-21T06:55:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.254+0000 7f3120834740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c4770768f0 con 0x55c476d402f0 2024-09-21T06:55:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47710de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47710de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57634/0 (socket says 172.21.15.114:57634) 2024-09-21T06:55:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 -- 172.21.15.114:0/3983474759 learned_addr learned my addr 172.21.15.114:0/3983474759 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f310b400640 1 -- 172.21.15.114:0/1404425417 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f31040079b0 con 0x7f31141033d0 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 -- 172.21.15.114:0/3983474759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 msgr2=0x55c47710e350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710e350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 -- 172.21.15.114:0/3983474759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 msgr2=0x55c47710e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 0x55c47710e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.255+0000 7f3113400640 1 -- 172.21.15.114:0/3983474759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c47703f780 con 0x55c476d691b0 2024-09-21T06:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.256+0000 7f3113400640 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47710de10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f310c00a2d0 tx=0x7f310c00d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f32c7f24100ae51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.256+0000 7f30ff400640 1 -- 172.21.15.114:0/3983474759 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c01b070 con 0x55c476d691b0 2024-09-21T06:55:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.256+0000 7f30ff400640 1 -- 172.21.15.114:0/3983474759 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c00ae50 con 0x55c476d691b0 2024-09-21T06:55:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.256+0000 7f30ff400640 1 -- 172.21.15.114:0/3983474759 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c016680 con 0x55c476d691b0 2024-09-21T06:55:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.258+0000 7f310b400640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30f80b6730 0x7f30f80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.258+0000 7f310b400640 1 -- 172.21.15.114:0/1404425417 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f31040e7790 con 0x7f31141033d0 2024-09-21T06:55:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.259+0000 7f3112a00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30f80b6730 0x7f30f80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.261+0000 7f3120834740 1 -- 172.21.15.114:0/3983474759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 msgr2=0x55c47710de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.261+0000 7f3120834740 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47710de10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f310c00a2d0 tx=0x7f310c00d040 comp rx=0 tx=0).stop 2024-09-21T06:55:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.261+0000 7f3112a00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30f80b6730 0x7f30f80b8c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55c47710ca90 tx=0x7f31040a9110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 -- 172.21.15.114:0/3983474759 shutdown_connections 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 0x55c47710e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 --2- 172.21.15.114:0/3983474759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47710de10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 -- 172.21.15.114:0/3983474759 >> 172.21.15.114:0/3983474759 conn(0x55c476d5d300 msgr2=0x55c47710bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 -- 172.21.15.114:0/3983474759 shutdown_connections 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.262+0000 7f3120834740 1 -- 172.21.15.114:0/3983474759 wait complete. 2024-09-21T06:55:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.263+0000 7f3120834740 1 -- start start 2024-09-21T06:55:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.264+0000 7f3120834740 1 Processor -- start 2024-09-21T06:55:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.264+0000 7f3120834740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477119470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.265+0000 7f3120834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d484d0 0x55c4771199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.265+0000 7f3113400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477119470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3113400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477119470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42418/0 (socket says 172.21.15.114:42418) 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3113400640 1 -- 172.21.15.114:0/850183414 learned_addr learned my addr 172.21.15.114:0/850183414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d691b0 0x55c4771258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c476faaab0 con 0x55c476d402f0 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c477019570 con 0x55c476d484d0 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c47711c590 con 0x55c476d691b0 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3120834740 10 client.0 _ll_register_callbacks cb 0x55c477028100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.266+0000 7f3112000640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d484d0 0x55c4771199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3112000640 1 -- 172.21.15.114:0/850183414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d691b0 msgr2=0x55c4771258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3112000640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d691b0 0x55c4771258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3112000640 1 -- 172.21.15.114:0/850183414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 msgr2=0x55c477119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3112000640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477119470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3112000640 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c476dd4790 con 0x55c476d484d0 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3113400640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:55:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.267+0000 7f3112000640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d484d0 0x55c4771199b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3100009850 tx=0x7f3100005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57eb9b5a7a9b9871 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.268+0000 7f30f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.269+0000 7f30f3e00640 1 -- 172.21.15.114:0/850183414 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100035040 con 0x55c476d484d0 2024-09-21T06:55:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.269+0000 7f30f3e00640 1 -- 172.21.15.114:0/850183414 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3100004960 con 0x55c476d484d0 2024-09-21T06:55:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.269+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c00ae50 con 0x55c476d484d0 2024-09-21T06:55:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.269+0000 7f30f3e00640 1 -- 172.21.15.114:0/850183414 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3100004b60 con 0x55c476d484d0 2024-09-21T06:55:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.270+0000 7f3120834740 10 client.26015 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.270+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c476e4d620 con 0x55c476d484d0 2024-09-21T06:55:36.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.270+0000 7f30f3e00640 1 -- 172.21.15.114:0/850183414 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c00ae50 con 0x55c476d484d0 2024-09-21T06:55:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.271+0000 7f30f3e00640 1 -- 172.21.15.114:0/850183414 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f31000314a0 con 0x55c476d484d0 2024-09-21T06:55:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3120834740 10 client.26015 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3120834740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3120834740 20 client.26015 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476d27d80 0x55c47712b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3120834740 4 client.26015 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c477085600 con 0x55c476d27d80 2024-09-21T06:55:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.273+0000 7f3113400640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476d27d80 0x55c47712b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.274+0000 7f3113400640 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476d27d80 0x55c47712b690 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a3b70235f9915b server_cookie=af7adbc017f986b0 in_seq=0 out_seq=0 2024-09-21T06:55:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.275+0000 7f30f3e00640 10 client.26015 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.276+0000 7f30f3e00640 1 -- 172.21.15.114:0/850183414 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c477085600 con 0x55c476d27d80 2024-09-21T06:55:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.276+0000 7f30f3e00640 10 client.26015 handle_command_reply: tid=0 2024-09-21T06:55:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.276+0000 7f3120834740 1 client.26015 shutdown 2024-09-21T06:55:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d484d0 msgr2=0x55c4771199b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d484d0 0x55c4771199b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3100009850 tx=0x7f3100005270 comp rx=0 tx=0).stop 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 shutdown_connections 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476d27d80 0x55c47712b690 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d691b0 0x55c4771258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d484d0 0x55c4771199b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f30f3e00640 0 client.26015 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 --2- 172.21.15.114:0/850183414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477119470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.277+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 >> 172.21.15.114:0/850183414 conn(0x55c476d5d300 msgr2=0x55c47710ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.278+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 shutdown_connections 2024-09-21T06:55:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.278+0000 7f3120834740 1 -- 172.21.15.114:0/850183414 wait complete. 2024-09-21T06:55:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.278+0000 7f3120834740 20 client.26015 trim_cache size 0 max 16384 2024-09-21T06:55:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.314+0000 7f3120834740 1 Processor -- start 2024-09-21T06:55:36.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.314+0000 7f3120834740 1 -- start start 2024-09-21T06:55:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.315+0000 7f3120834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3120834740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 0x55c47711b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3120834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c477130e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3120834740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f31141bfbd0 con 0x55c476d484d0 2024-09-21T06:55:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3120834740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31141c17d0 con 0x55c476d691b0 2024-09-21T06:55:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3120834740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31141c02a0 con 0x55c476d402f0 2024-09-21T06:55:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3113400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c477130e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3113400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c477130e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57652/0 (socket says 172.21.15.114:57652) 2024-09-21T06:55:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3113400640 1 -- 172.21.15.114:0/2956026190 learned_addr learned my addr 172.21.15.114:0/2956026190 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.316+0000 7f3113400640 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.317+0000 7f3113400640 1 -- 172.21.15.114:0/2956026190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 msgr2=0x55c47710ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.317+0000 7f3113400640 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.317+0000 7f3113400640 1 -- 172.21.15.114:0/2956026190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 msgr2=0x55c47711b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.317+0000 7f3113400640 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 0x55c47711b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.317+0000 7f3113400640 1 -- 172.21.15.114:0/2956026190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c477110110 con 0x55c476d691b0 2024-09-21T06:55:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.318+0000 7f3113400640 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c477130e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f310c00bb90 tx=0x7f310c00be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b616608c168faf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.319+0000 7f30fea00640 1 -- 172.21.15.114:0/2956026190 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c01b040 con 0x55c476d691b0 2024-09-21T06:55:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.319+0000 7f30fea00640 1 -- 172.21.15.114:0/2956026190 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c01c040 con 0x55c476d691b0 2024-09-21T06:55:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.319+0000 7f30fea00640 1 -- 172.21.15.114:0/2956026190 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c005280 con 0x55c476d691b0 2024-09-21T06:55:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 -- 172.21.15.114:0/2956026190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 msgr2=0x55c477130e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c477130e50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f310c00bb90 tx=0x7f310c00be20 comp rx=0 tx=0).stop 2024-09-21T06:55:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 -- 172.21.15.114:0/2956026190 shutdown_connections 2024-09-21T06:55:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c477130e50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d484d0 0x55c47711b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 --2- 172.21.15.114:0/2956026190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d402f0 0x55c47710ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.321+0000 7f3120834740 1 -- 172.21.15.114:0/2956026190 >> 172.21.15.114:0/2956026190 conn(0x55c476d5d300 msgr2=0x55c47711a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.322+0000 7f3120834740 1 -- 172.21.15.114:0/2956026190 shutdown_connections 2024-09-21T06:55:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.322+0000 7f3120834740 1 -- 172.21.15.114:0/2956026190 wait complete. 2024-09-21T06:55:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.323+0000 7f3120834740 1 -- start start 2024-09-21T06:55:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.324+0000 7f3120834740 1 Processor -- start 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477121e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 0x55c47711f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47711f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f31141c1040 con 0x55c476d402f0 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31141c0970 con 0x55c476d691b0 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3114074f00 con 0x55c476d484d0 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3120834740 10 client.0 _ll_register_callbacks cb 0x7f3100009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3112000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 0x55c47711f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3112000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 0x55c47711f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52728/0 (socket says 172.21.15.114:52728) 2024-09-21T06:55:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.328+0000 7f3112000640 1 -- 172.21.15.114:0/2958860264 learned_addr learned my addr 172.21.15.114:0/2958860264 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.329+0000 7f3112000640 1 -- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 msgr2=0x55c47711f820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.329+0000 7f3112000640 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47711f820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.329+0000 7f3112000640 1 -- 172.21.15.114:0/2958860264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 msgr2=0x55c477121e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.329+0000 7f3112000640 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477121e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.329+0000 7f3112000640 1 -- 172.21.15.114:0/2958860264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47711fde0 con 0x55c476d484d0 2024-09-21T06:55:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.330+0000 7f3112000640 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 0x55c47711f2e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3100031870 tx=0x7f3100005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c9648f0eadf10a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.330+0000 7f30f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.332+0000 7f30f3e00640 1 -- 172.21.15.114:0/2958860264 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100035070 con 0x55c476d484d0 2024-09-21T06:55:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.332+0000 7f30f3e00640 1 -- 172.21.15.114:0/2958860264 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3100004760 con 0x55c476d484d0 2024-09-21T06:55:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.332+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c01c040 con 0x55c476d484d0 2024-09-21T06:55:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.333+0000 7f30f3e00640 1 -- 172.21.15.114:0/2958860264 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3100005df0 con 0x55c476d484d0 2024-09-21T06:55:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.333+0000 7f3120834740 10 client.26002 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.333+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c477195e70 con 0x55c476d484d0 2024-09-21T06:55:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.335+0000 7f30f3e00640 1 -- 172.21.15.114:0/2958860264 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c01c040 con 0x55c476d484d0 2024-09-21T06:55:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f30f3e00640 1 -- 172.21.15.114:0/2958860264 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f3100057450 con 0x55c476d484d0 2024-09-21T06:55:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f3120834740 10 client.26002 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f3120834740 10 client.26002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f3120834740 20 client.26002 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f3120834740 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476cfd1c0 0x55c477198770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f3120834740 4 client.26002 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:55:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.336+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4770810b0 con 0x55c476cfd1c0 2024-09-21T06:55:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.337+0000 7f3113400640 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476cfd1c0 0x55c477198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.337+0000 7f30f3e00640 10 client.26002 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.338+0000 7f3113400640 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476cfd1c0 0x55c477198770 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afa4e0dea4bd9d2c server_cookie=687fec91ab0a89fe in_seq=0 out_seq=0 2024-09-21T06:55:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.373+0000 7f30f3e00640 1 -- 172.21.15.114:0/2958860264 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4770810b0 con 0x55c476cfd1c0 2024-09-21T06:55:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.373+0000 7f30f3e00640 10 client.26002 handle_command_reply: tid=0 2024-09-21T06:55:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.373+0000 7f3120834740 1 client.26002 shutdown 2024-09-21T06:55:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 msgr2=0x55c47711f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 0x55c47711f2e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3100031870 tx=0x7f3100005970 comp rx=0 tx=0).stop 2024-09-21T06:55:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 shutdown_connections 2024-09-21T06:55:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c476d691b0 0x55c47711f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c476cfd1c0 0x55c477198770 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f30f3e00640 0 client.26002 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c476d484d0 0x55c47711f2e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 --2- 172.21.15.114:0/2958860264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c476d402f0 0x55c477121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 >> 172.21.15.114:0/2958860264 conn(0x55c476d5d300 msgr2=0x55c47710aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.374+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 shutdown_connections 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.375+0000 7f3120834740 1 -- 172.21.15.114:0/2958860264 wait complete. 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.375+0000 7f3120834740 20 client.26002 trim_cache size 0 max 16384 2024-09-21T06:55:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.376+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30f80b6730 msgr2=0x7f30f80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.376+0000 7f3113e00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30f80b6730 0x7f30f80b8c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55c47710ca90 tx=0x7f31040a9110 comp rx=0 tx=0).stop 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.376+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 msgr2=0x7f31141a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.376+0000 7f3113e00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 0x7f31141a08e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f310400c890 tx=0x7f310400cd50 comp rx=0 tx=0).stop 2024-09-21T06:55:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.378+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 shutdown_connections 2024-09-21T06:55:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.378+0000 7f3113e00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3114108b40 0x7f31141a6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.378+0000 7f3113e00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30f80b6730 0x7f30f80b8c40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.378+0000 7f3113e00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31141081f0 0x7f31141a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.378+0000 7f3113e00640 1 --2- 172.21.15.114:0/1404425417 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31141033d0 0x7f31141a08e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.378+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 >> 172.21.15.114:0/1404425417 conn(0x7f3114078770 msgr2=0x7f31141001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.379+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 shutdown_connections 2024-09-21T06:55:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:36.379+0000 7f3113e00640 1 -- 172.21.15.114:0/1404425417 wait complete. 2024-09-21T06:55:36.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:36 smithi193 ceph-mon[34601]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 817 op/s 2024-09-21T06:55:37.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:36 smithi114 ceph-mon[30654]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 817 op/s 2024-09-21T06:55:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:36 smithi183 ceph-mon[34219]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 817 op/s 2024-09-21T06:55:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 60.0 s, 1441.3 tps, lat 70.393 ms stddev 141.174 2024-09-21T06:55:38.440 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:55:38.440 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:55:38.660 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:38.924 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:38 smithi114 ceph-mon[30654]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-21T06:55:39.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:38 smithi183 ceph-mon[34219]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-21T06:55:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:38 smithi193 ceph-mon[34601]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 1.05k op/s 2024-09-21T06:55:39.488 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.523+0000 7fcbd0400640 1 -- 172.21.15.114:0/4231606536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 msgr2=0x7fcbc8107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:39.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.523+0000 7fcbd0400640 1 --2- 172.21.15.114:0/4231606536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc8107620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009930 tx=0x7fcbbc02a460 comp rx=0 tx=0).stop 2024-09-21T06:55:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.525+0000 7fcbd0400640 1 -- 172.21.15.114:0/4231606536 shutdown_connections 2024-09-21T06:55:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.525+0000 7fcbd0400640 1 --2- 172.21.15.114:0/4231606536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc8107be0 0x7fcbc81080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.525+0000 7fcbd0400640 1 --2- 172.21.15.114:0/4231606536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc8107620 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.525+0000 7fcbd0400640 1 --2- 172.21.15.114:0/4231606536 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbc81027a0 0x7fcbc8102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.525+0000 7fcbd0400640 1 -- 172.21.15.114:0/4231606536 >> 172.21.15.114:0/4231606536 conn(0x7fcbc8079730 msgr2=0x7fcbc8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.526+0000 7fcbd0400640 1 -- 172.21.15.114:0/4231606536 shutdown_connections 2024-09-21T06:55:39.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.526+0000 7fcbd0400640 1 -- 172.21.15.114:0/4231606536 wait complete. 2024-09-21T06:55:39.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.527+0000 7fcbd0400640 1 Processor -- start 2024-09-21T06:55:39.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.527+0000 7fcbd0400640 1 -- start start 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbd0400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbc81027a0 0x7fcbc81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbd0400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc81a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbd0400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc8107be0 0x7fcbc81a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbd0400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc811a1d0 con 0x7fcbc81027a0 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbd0400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc811a050 con 0x7fcbc81030f0 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbd0400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc811a350 con 0x7fcbc8107be0 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbcd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc81a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbcd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc81a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57678/0 (socket says 172.21.15.114:57678) 2024-09-21T06:55:39.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.529+0000 7fcbcd000640 1 -- 172.21.15.114:0/1509270071 learned_addr learned my addr 172.21.15.114:0/1509270071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcda00640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbc81027a0 0x7fcbc81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbce400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc8107be0 0x7fcbc81a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcd000640 1 -- 172.21.15.114:0/1509270071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc8107be0 msgr2=0x7fcbc81a6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcd000640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc8107be0 0x7fcbc81a6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcd000640 1 -- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbc81027a0 msgr2=0x7fcbc81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcd000640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbc81027a0 0x7fcbc81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcd000640 1 -- 172.21.15.114:0/1509270071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc81a6c50 con 0x7fcbc81030f0 2024-09-21T06:55:39.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.530+0000 7fcbcd000640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc81a1280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009900 tx=0x7fcbbc003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.531+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc03f070 con 0x7fcbc81030f0 2024-09-21T06:55:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.531+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbbc031d60 con 0x7fcbc81030f0 2024-09-21T06:55:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.531+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc035030 con 0x7fcbc81030f0 2024-09-21T06:55:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.531+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc81a6ee0 con 0x7fcbc81030f0 2024-09-21T06:55:39.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.531+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81ad6a0 con 0x7fcbc81030f0 2024-09-21T06:55:39.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.533+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc806b560 con 0x7fcbc81030f0 2024-09-21T06:55:39.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.539+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fcbbc042070 con 0x7fcbc81030f0 2024-09-21T06:55:39.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.543+0000 7fcbc6000640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb00b6700 0x7fcbb00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:39.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.543+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcbbc108710 con 0x7fcbc81030f0 2024-09-21T06:55:39.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.543+0000 7fcbcda00640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb00b6700 0x7fcbb00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:39.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.545+0000 7fcbcda00640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb00b6700 0x7fcbb00b8c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcbc806bc60 tx=0x7fcbb4008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:39.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.545+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcbbc0ce890 con 0x7fcbc81030f0 2024-09-21T06:55:39.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.969+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbc81076a0 con 0x7fcbc81030f0 2024-09-21T06:55:39.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.970+0000 7fcbc6000640 1 -- 172.21.15.114:0/1509270071 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fcbc81076a0 con 0x7fcbc81030f0 2024-09-21T06:55:39.975 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:55:39.976 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:55:39.976 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.974+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb00b6700 msgr2=0x7fcbb00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.974+0000 7fcbd0400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb00b6700 0x7fcbb00b8c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcbc806bc60 tx=0x7fcbb4008040 comp rx=0 tx=0).stop 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.974+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 msgr2=0x7fcbc81a1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.974+0000 7fcbd0400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc81a1280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009900 tx=0x7fcbbc003d70 comp rx=0 tx=0).stop 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 shutdown_connections 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbc8107be0 0x7fcbc81a6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb00b6700 0x7fcbb00b8c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbc81030f0 0x7fcbc81a1280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 --2- 172.21.15.114:0/1509270071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbc81027a0 0x7fcbc81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:39.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 >> 172.21.15.114:0/1509270071 conn(0x7fcbc8079730 msgr2=0x7fcbc81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:39.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 shutdown_connections 2024-09-21T06:55:39.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:39.975+0000 7fcbd0400640 1 -- 172.21.15.114:0/1509270071 wait complete. 2024-09-21T06:55:40.963 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:40 smithi193 ceph-mon[34601]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 956 op/s 2024-09-21T06:55:40.964 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:40 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1509270071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:41.069 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:40 smithi114 ceph-mon[30654]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 956 op/s 2024-09-21T06:55:41.069 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:40 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1509270071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:40 smithi183 ceph-mon[34219]: pgmap v442: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 956 op/s 2024-09-21T06:55:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:40 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1509270071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:42.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.017+0000 7fb468a00640 1 -- 172.21.15.114:0/628018716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb464116510 msgr2=0x7fb464118950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:42.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.017+0000 7fb468a00640 1 --2- 172.21.15.114:0/628018716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb464116510 0x7fb464118950 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009ca0 tx=0x7fb45c02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 -- 172.21.15.114:0/628018716 shutdown_connections 2024-09-21T06:55:42.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 --2- 172.21.15.114:0/628018716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb464116510 0x7fb464118950 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 --2- 172.21.15.114:0/628018716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 --2- 172.21.15.114:0/628018716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb46410a3c0 0x7fb46410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 -- 172.21.15.114:0/628018716 >> 172.21.15.114:0/628018716 conn(0x7fb46406ee50 msgr2=0x7fb46406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:42.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 -- 172.21.15.114:0/628018716 shutdown_connections 2024-09-21T06:55:42.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.022+0000 7fb468a00640 1 -- 172.21.15.114:0/628018716 wait complete. 2024-09-21T06:55:42.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.024+0000 7fb468a00640 1 Processor -- start 2024-09-21T06:55:42.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.026+0000 7fb468a00640 1 -- start start 2024-09-21T06:55:42.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.026+0000 7fb468a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb46410a3c0 0x7fb464114300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:42.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.026+0000 7fb468a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464114840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.026+0000 7fb468a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb464116510 0x7fb464114d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.027+0000 7fb45be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464114840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.027+0000 7fb45be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464114840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52760/0 (socket says 172.21.15.114:52760) 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.027+0000 7fb45be00640 1 -- 172.21.15.114:0/3584615269 learned_addr learned my addr 172.21.15.114:0/3584615269 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.027+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb46410d430 con 0x7fb46410a3c0 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.027+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb46410d2b0 con 0x7fb464116510 2024-09-21T06:55:42.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.027+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb46410d5b0 con 0x7fb46410ad10 2024-09-21T06:55:42.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.028+0000 7fb463200640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb464116510 0x7fb464114d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:42.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.028+0000 7fb45be00640 1 -- 172.21.15.114:0/3584615269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb464116510 msgr2=0x7fb464114d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:42.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.029+0000 7fb45be00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb464116510 0x7fb464114d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.029+0000 7fb45be00640 1 -- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb46410a3c0 msgr2=0x7fb464114300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:42.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.029+0000 7fb45be00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb46410a3c0 0x7fb464114300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.029+0000 7fb45be00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb464125cf0 con 0x7fb46410ad10 2024-09-21T06:55:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.029+0000 7fb463200640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb464116510 0x7fb464114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.029+0000 7fb45be00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464114840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d740 tx=0x7fb44c00dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:42.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.032+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c017070 con 0x7fb46410ad10 2024-09-21T06:55:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.032+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c00de50 con 0x7fb46410ad10 2024-09-21T06:55:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.032+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb464125fe0 con 0x7fb46410ad10 2024-09-21T06:55:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.032+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb464126570 con 0x7fb46410ad10 2024-09-21T06:55:42.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.032+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00f620 con 0x7fb46410ad10 2024-09-21T06:55:42.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.036+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb440005290 con 0x7fb46410ad10 2024-09-21T06:55:42.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.046+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb44c01a070 con 0x7fb46410ad10 2024-09-21T06:55:42.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.050+0000 7fb45b400640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb4480b6700 0x7fb4480b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:42.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.051+0000 7fb462800640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb4480b6700 0x7fb4480b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:42.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.051+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb44c0e6ef0 con 0x7fb46410ad10 2024-09-21T06:55:42.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.052+0000 7fb462800640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb4480b6700 0x7fb4480b8c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb4540058c0 tx=0x7fb454005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:42.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.055+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb44c0ad0d0 con 0x7fb46410ad10 2024-09-21T06:55:42.543 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:55:42.543 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:55:42.544 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:55:42.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.534+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb440005850 con 0x7fb46410ad10 2024-09-21T06:55:42.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.537+0000 7fb45b400640 1 -- 172.21.15.114:0/3584615269 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fb44c0b14b0 con 0x7fb46410ad10 2024-09-21T06:55:42.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb4480b6700 msgr2=0x7fb4480b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb4480b6700 0x7fb4480b8c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb4540058c0 tx=0x7fb454005850 comp rx=0 tx=0).stop 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 msgr2=0x7fb464114840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464114840 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d740 tx=0x7fb44c00dc00 comp rx=0 tx=0).stop 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 shutdown_connections 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb4480b6700 0x7fb4480b8c10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb464116510 0x7fb464114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb46410ad10 0x7fb464114840 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 --2- 172.21.15.114:0/3584615269 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb46410a3c0 0x7fb464114300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.551+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 >> 172.21.15.114:0/3584615269 conn(0x7fb46406ee50 msgr2=0x7fb464071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.553+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 shutdown_connections 2024-09-21T06:55:42.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:42.553+0000 7fb468a00640 1 -- 172.21.15.114:0/3584615269 wait complete. 2024-09-21T06:55:42.572 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0' 2024-09-21T06:55:42.573 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 137.0 --await-for 45.0 2024-09-21T06:55:42.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:42 smithi114 ceph-mon[30654]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 943 op/s 2024-09-21T06:55:42.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:42 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3584615269' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 65.0 s, 1058.2 tps, lat 70.887 ms stddev 168.520 2024-09-21T06:55:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:42 smithi183 ceph-mon[34219]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 943 op/s 2024-09-21T06:55:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:42 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3584615269' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:43.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:42 smithi193 ceph-mon[34601]: pgmap v443: 129 pgs: 129 active+clean; 1.1 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 943 op/s 2024-09-21T06:55:43.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:42 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3584615269' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:44.736 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:55:44.860 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:45.042 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:44 smithi114 ceph-mon[30654]: pgmap v444: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 1.60k op/s 2024-09-21T06:55:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:44 smithi183 ceph-mon[34219]: pgmap v444: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 1.60k op/s 2024-09-21T06:55:45.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:44 smithi193 ceph-mon[34601]: pgmap v444: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 1.60k op/s 2024-09-21T06:55:46.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.858+0000 7fe5aa600640 1 -- 172.21.15.114:0/3049206799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 msgr2=0x7fe5a4110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.858+0000 7fe5aa600640 1 --2- 172.21.15.114:0/3049206799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a4110320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ca0 tx=0x7fe59c02a8f0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.859+0000 7fe5aa600640 1 -- 172.21.15.114:0/3049206799 shutdown_connections 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.859+0000 7fe5aa600640 1 --2- 172.21.15.114:0/3049206799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a4110320 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.859+0000 7fe5aa600640 1 --2- 172.21.15.114:0/3049206799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 0x7fe5a410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.859+0000 7fe5aa600640 1 --2- 172.21.15.114:0/3049206799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.859+0000 7fe5aa600640 1 -- 172.21.15.114:0/3049206799 >> 172.21.15.114:0/3049206799 conn(0x7fe5a4079760 msgr2=0x7fe5a4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.860+0000 7fe5aa600640 1 -- 172.21.15.114:0/3049206799 shutdown_connections 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.860+0000 7fe5aa600640 1 -- 172.21.15.114:0/3049206799 wait complete. 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.862+0000 7fe5aa600640 1 Processor -- start 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.862+0000 7fe5aa600640 1 -- start start 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.863+0000 7fe5aa600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.863+0000 7fe5aa600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 0x7fe5a419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.863+0000 7fe59be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.863+0000 7fe59be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46590/0 (socket says 172.21.15.114:46590) 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.863+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 0x7fe5a419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.863+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 0x7fe5a419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33706/0 (socket says 172.21.15.114:33706) 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5aa600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34776/0 (socket says 172.21.15.114:34776) 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5a3400640 1 -- 172.21.15.114:0/615190238 learned_addr learned my addr 172.21.15.114:0/615190238 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4113b30 con 0x7fe5a4105320 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41139b0 con 0x7fe5a410de30 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.864+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4113cb0 con 0x7fe5a41049d0 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe5a3e00640 1 -- 172.21.15.114:0/615190238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 msgr2=0x7fe5a419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe5a3e00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe5a3e00640 1 -- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 msgr2=0x7fe5a419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe5a3e00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 0x7fe5a419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe5a3e00640 1 -- 172.21.15.114:0/615190238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a41a04d0 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe59be00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.865+0000 7fe5a3e00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a419fda0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ca0 tx=0x7fe59c031910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.866+0000 7fe5a0c00640 1 -- 172.21.15.114:0/615190238 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c03fce0 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.866+0000 7fe5a0c00640 1 -- 172.21.15.114:0/615190238 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c030cb0 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.866+0000 7fe5a0c00640 1 -- 172.21.15.114:0/615190238 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c04abd0 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.866+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a41a0760 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.866+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a41025f0 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.874+0000 7fe5a0c00640 1 -- 172.21.15.114:0/615190238 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe59c03a400 con 0x7fe5a410de30 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.875+0000 7fe5b129a740 1 Processor -- start 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.875+0000 7fe5b129a740 1 -- start start 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.876+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf47fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.876+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf442c980 0x559cf47fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.876+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf47fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.876+0000 7fe5b129a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559cf4772ca0 con 0x559cf442c980 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.876+0000 7fe5b129a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559cf47620d0 con 0x559cf4434b60 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.876+0000 7fe5b129a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559cf4765ee0 con 0x559cf4455840 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.877+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf47fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.877+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf47fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46592/0 (socket says 172.21.15.114:46592) 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.877+0000 7fe5a3400640 1 -- 172.21.15.114:0/2700384414 learned_addr learned my addr 172.21.15.114:0/2700384414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.877+0000 7fe5a3400640 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf47fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.878+0000 7fe5a0c00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5880b6730 0x7fe5880b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe59be00640 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf442c980 0x559cf47fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe5a3e00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5880b6730 0x7fe5880b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe5a0c00640 1 -- 172.21.15.114:0/615190238 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe59c10d620 con 0x7fe5a410de30 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe5a3400640 1 -- 172.21.15.114:0/2700384414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 msgr2=0x559cf47fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe5a3400640 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf47fddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe5a3400640 1 -- 172.21.15.114:0/2700384414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf442c980 msgr2=0x559cf47fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.879+0000 7fe5a3400640 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf442c980 0x559cf47fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.880+0000 7fe5a3400640 1 -- 172.21.15.114:0/2700384414 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf4714400 con 0x559cf4455840 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.880+0000 7fe5a3e00640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5880b6730 0x7fe5880b8c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009bf0 tx=0x7fe59c009480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.880+0000 7fe5a3400640 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf47fd350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe58c014900 tx=0x7fe58c00a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd1420d874b06cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.881+0000 7fe598c00640 1 -- 172.21.15.114:0/2700384414 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c005220 con 0x559cf4455840 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.881+0000 7fe598c00640 1 -- 172.21.15.114:0/2700384414 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe58c00acd0 con 0x559cf4455840 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.881+0000 7fe598c00640 1 -- 172.21.15.114:0/2700384414 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00da20 con 0x559cf4455840 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.882+0000 7fe5b129a740 1 -- 172.21.15.114:0/2700384414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 msgr2=0x559cf47fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.882+0000 7fe5b129a740 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf47fd350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe58c014900 tx=0x7fe58c00a6c0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.889+0000 7fe5b129a740 1 -- 172.21.15.114:0/2700384414 shutdown_connections 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.889+0000 7fe5b129a740 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf47fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.889+0000 7fe5b129a740 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf442c980 0x559cf47fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.889+0000 7fe5b129a740 1 --2- 172.21.15.114:0/2700384414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf47fd350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.890+0000 7fe5b129a740 1 -- 172.21.15.114:0/2700384414 >> 172.21.15.114:0/2700384414 conn(0x559cf4449990 msgr2=0x559cf47fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:46.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.890+0000 7fe5b129a740 1 -- 172.21.15.114:0/2700384414 shutdown_connections 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.890+0000 7fe5b129a740 1 -- 172.21.15.114:0/2700384414 wait complete. 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.891+0000 7fe5b129a740 1 -- start start 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.891+0000 7fe5b129a740 1 Processor -- start 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.892+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 0x559cf4822a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.892+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4823070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.892+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf4828090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.893+0000 7fe5b129a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559cf45c37a0 con 0x559cf4434b60 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.893+0000 7fe5b129a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559cf4707a10 con 0x559cf4455840 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.893+0000 7fe5b129a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559cf481cae0 con 0x559cf442c980 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.893+0000 7fe5b129a740 10 client.0 _ll_register_callbacks cb 0x559cf4717be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.894+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 0x559cf4822a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.894+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 0x559cf4822a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46600/0 (socket says 172.21.15.114:46600) 2024-09-21T06:55:46.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.894+0000 7fe5a3400640 1 -- 172.21.15.114:0/3607139776 learned_addr learned my addr 172.21.15.114:0/3607139776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.894+0000 7fe5a3400640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf4828090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.897+0000 7fe59be00640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4823070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.897+0000 7fe5a3400640 1 -- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 msgr2=0x559cf4828090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.897+0000 7fe5a3400640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf4828090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.897+0000 7fe5a3400640 1 -- 172.21.15.114:0/3607139776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 msgr2=0x559cf4823070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.897+0000 7fe5a3400640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4823070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.898+0000 7fe5a3400640 1 -- 172.21.15.114:0/3607139776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf44cfb90 con 0x559cf442c980 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.898+0000 7fe5a3400640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 0x559cf4822a90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe58c014510 tx=0x7fe58c0054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ef03f0f01ddd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.898+0000 7fe591600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.900+0000 7fe591600640 1 -- 172.21.15.114:0/3607139776 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00de00 con 0x559cf442c980 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.900+0000 7fe591600640 1 -- 172.21.15.114:0/3607139776 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe58c004020 con 0x559cf442c980 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.900+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe58c00acd0 con 0x559cf442c980 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.902+0000 7fe591600640 1 -- 172.21.15.114:0/3607139776 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe58c004220 con 0x559cf442c980 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.902+0000 7fe5b129a740 10 client.26014 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:46.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.902+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559cf4830fe0 con 0x559cf442c980 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.903+0000 7fe591600640 1 -- 172.21.15.114:0/3607139776 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe58c00acd0 con 0x559cf442c980 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.903+0000 7fe591600640 1 -- 172.21.15.114:0/3607139776 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe58c0035f0 con 0x559cf442c980 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe5b129a740 10 client.26014 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe5b129a740 10 client.26014 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe5b129a740 20 client.26014 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf44141c0 0x559cf4834340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe5b129a740 4 client.26014 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cf4774bf0 con 0x559cf44141c0 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.904+0000 7fe59be00640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf44141c0 0x559cf4834340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.905+0000 7fe591600640 10 client.26014 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.905+0000 7fe59be00640 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf44141c0 0x559cf4834340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0202268fae8c144 server_cookie=f424dedd70b79118 in_seq=0 out_seq=0 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.907+0000 7fe591600640 1 -- 172.21.15.114:0/3607139776 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cf4774bf0 con 0x559cf44141c0 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.908+0000 7fe591600640 10 client.26014 handle_command_reply: tid=0 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.908+0000 7fe5b129a740 1 client.26014 shutdown 2024-09-21T06:55:46.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.908+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 msgr2=0x559cf4822a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.908+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 0x559cf4822a90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe58c014510 tx=0x7fe58c0054d0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.909+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 shutdown_connections 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.909+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf4828090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.909+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4823070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.909+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf44141c0 0x559cf4834340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.909+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3607139776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf442c980 0x559cf4822a90 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe58c014510 tx=0x7fe58c0054d0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.909+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 >> 172.21.15.114:0/3607139776 conn(0x559cf4449990 msgr2=0x559cf48004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.910+0000 7fe591600640 0 client.26014 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.910+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 shutdown_connections 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.910+0000 7fe5b129a740 1 -- 172.21.15.114:0/3607139776 wait complete. 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.910+0000 7fe5b129a740 20 client.26014 trim_cache size 0 max 16384 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.949+0000 7fe5b129a740 1 Processor -- start 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.949+0000 7fe5b129a740 1 -- start start 2024-09-21T06:55:46.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.950+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4800b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.950+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf482ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.950+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf43e9570 0x559cf47faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.950+0000 7fe5b129a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe58000ff80 con 0x559cf4434b60 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.950+0000 7fe5b129a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe580011b80 con 0x559cf4455840 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.950+0000 7fe5b129a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe580010650 con 0x559cf43e9570 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.951+0000 7fe59be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf482ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.951+0000 7fe59be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf482ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34812/0 (socket says 172.21.15.114:34812) 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.951+0000 7fe59be00640 1 -- 172.21.15.114:0/3488756187 learned_addr learned my addr 172.21.15.114:0/3488756187 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe59be00640 1 -- 172.21.15.114:0/3488756187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf43e9570 msgr2=0x559cf47faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe5a3400640 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf43e9570 0x559cf47faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe59be00640 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf43e9570 0x559cf47faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe59be00640 1 -- 172.21.15.114:0/3488756187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 msgr2=0x559cf4800b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe59be00640 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4800b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe59be00640 1 -- 172.21.15.114:0/3488756187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf48001b0 con 0x559cf4455840 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.952+0000 7fe5a3400640 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf43e9570 0x559cf47faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.953+0000 7fe59be00640 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf482ee80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe594000c00 tx=0x7fe59400ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90e75f3c48c6d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.954+0000 7fe593e00640 1 -- 172.21.15.114:0/3488756187 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594021020 con 0x559cf4455840 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.955+0000 7fe593e00640 1 -- 172.21.15.114:0/3488756187 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe594018e30 con 0x559cf4455840 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.955+0000 7fe593e00640 1 -- 172.21.15.114:0/3488756187 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59401d410 con 0x559cf4455840 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.956+0000 7fe5b129a740 1 -- 172.21.15.114:0/3488756187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 msgr2=0x559cf482ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.956+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf482ee80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe594000c00 tx=0x7fe59400ed50 comp rx=0 tx=0).stop 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.956+0000 7fe5b129a740 1 -- 172.21.15.114:0/3488756187 shutdown_connections 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.956+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf43e9570 0x559cf47faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.957+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4455840 0x559cf482ee80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.957+0000 7fe5b129a740 1 --2- 172.21.15.114:0/3488756187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf4434b60 0x559cf4800b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.957+0000 7fe5b129a740 1 -- 172.21.15.114:0/3488756187 >> 172.21.15.114:0/3488756187 conn(0x559cf4449990 msgr2=0x559cf4800450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.957+0000 7fe5b129a740 1 -- 172.21.15.114:0/3488756187 shutdown_connections 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.957+0000 7fe5b129a740 1 -- 172.21.15.114:0/3488756187 wait complete. 2024-09-21T06:55:46.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.958+0000 7fe5b129a740 1 -- start start 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.958+0000 7fe5b129a740 1 Processor -- start 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.962+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf43e9570 0x559cf482e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf482ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe5b129a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf4828d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe5b129a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe5800113f0 con 0x559cf43e9570 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe5b129a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe580010d20 con 0x559cf4434b60 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe5b129a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe58000df60 con 0x559cf4455840 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe5b129a740 10 client.0 _ll_register_callbacks cb 0x7fe58c00aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe59be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf482ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe59be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf482ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34814/0 (socket says 172.21.15.114:34814) 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.963+0000 7fe59be00640 1 -- 172.21.15.114:0/4110860947 learned_addr learned my addr 172.21.15.114:0/4110860947 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.964+0000 7fe59be00640 1 -- 172.21.15.114:0/4110860947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 msgr2=0x559cf4828d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:46.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.964+0000 7fe59be00640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf4828d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.964+0000 7fe59be00640 1 -- 172.21.15.114:0/4110860947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf43e9570 msgr2=0x559cf482e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.966+0000 7fe5a3400640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf43e9570 0x559cf482e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.966+0000 7fe59be00640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf43e9570 0x559cf482e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.966+0000 7fe59be00640 1 -- 172.21.15.114:0/4110860947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf47f6c50 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.966+0000 7fe5a3400640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf43e9570 0x559cf482e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.967+0000 7fe59be00640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf482ecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe59400ffd0 tx=0x7fe594018d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e9223c79e901e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.967+0000 7fe591600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.967+0000 7fe591600640 1 -- 172.21.15.114:0/4110860947 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59402c070 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.967+0000 7fe591600640 1 -- 172.21.15.114:0/4110860947 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe594016af0 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.967+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe594018e30 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.968+0000 7fe591600640 1 -- 172.21.15.114:0/4110860947 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe594016cf0 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.968+0000 7fe5b129a740 10 client.26045 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.968+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559cf489f300 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.969+0000 7fe591600640 1 -- 172.21.15.114:0/4110860947 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe594018e30 con 0x559cf4434b60 2024-09-21T06:55:46.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.969+0000 7fe591600640 1 -- 172.21.15.114:0/4110860947 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe5940264a0 con 0x559cf4434b60 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.969+0000 7fe5b129a740 10 client.26045 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.969+0000 7fe5b129a740 10 client.26045 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.969+0000 7fe5b129a740 20 client.26045 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.969+0000 7fe5b129a740 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf489fa80 0x559cf48a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.970+0000 7fe5b129a740 4 client.26045 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}] 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.970+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.0, "await_for": 45.0}) -- 0x559cf47706a0 con 0x559cf489fa80 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.971+0000 7fe5a3400640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf489fa80 0x559cf48a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.972+0000 7fe5a3400640 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf489fa80 0x559cf48a1ea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd15f8b9e6a4e981 server_cookie=65c06ff9e2eebb3c in_seq=0 out_seq=0 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:45.972+0000 7fe591600640 10 client.26045 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.149+0000 7fe591600640 1 -- 172.21.15.114:0/4110860947 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559cf47706a0 con 0x559cf489fa80 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.149+0000 7fe591600640 10 client.26045 handle_command_reply: tid=0 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.149+0000 7fe5b129a740 1 client.26045 shutdown 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 msgr2=0x559cf482ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf482ecf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe59400ffd0 tx=0x7fe594018d10 comp rx=0 tx=0).stop 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 shutdown_connections 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559cf489fa80 0x559cf48a1ea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559cf4455840 0x559cf4828d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559cf4434b60 0x559cf482ecf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe591600640 0 client.26045 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 --2- 172.21.15.114:0/4110860947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559cf43e9570 0x559cf482e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 >> 172.21.15.114:0/4110860947 conn(0x559cf4449990 msgr2=0x559cf47fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 shutdown_connections 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.150+0000 7fe5b129a740 1 -- 172.21.15.114:0/4110860947 wait complete. 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:55:46.375 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 32, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 6, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "745fa756": { 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 6, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 137.0, 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:55:46.376 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5b129a740 20 client.26045 trim_cache size 0 max 16384 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5880b6730 msgr2=0x7fe5880b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5880b6730 0x7fe5880b8c40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009bf0 tx=0x7fe59c009480 comp rx=0 tx=0).stop 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 msgr2=0x7fe5a419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a419fda0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ca0 tx=0x7fe59c031910 comp rx=0 tx=0).stop 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 shutdown_connections 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe5880b6730 0x7fe5880b8c40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a410de30 0x7fe5a419fda0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe5a4105320 0x7fe5a419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 --2- 172.21.15.114:0/615190238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5a41049d0 0x7fe5a419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.151+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 >> 172.21.15.114:0/615190238 conn(0x7fe5a4079760 msgr2=0x7fe5a4107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.152+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 shutdown_connections 2024-09-21T06:55:46.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:46.152+0000 7fe5aa600640 1 -- 172.21.15.114:0/615190238 wait complete. 2024-09-21T06:55:46.968 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:46 smithi114 ceph-mon[30654]: pgmap v445: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-09-21T06:55:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:46 smithi183 ceph-mon[34219]: pgmap v445: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-09-21T06:55:47.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:46 smithi193 ceph-mon[34601]: pgmap v445: 129 pgs: 129 active+clean; 1.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 1.59k op/s 2024-09-21T06:55:47.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 70.0 s, 1855.6 tps, lat 47.620 ms stddev 166.757 2024-09-21T06:55:47.859 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '745fa756', quiesce took 14.6 seconds. Will release after: 2.4000000000000004 2024-09-21T06:55:47.930 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:48.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 -- 172.21.15.114:0/2481884985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd01094a0 msgr2=0x7f9bd01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2481884985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd01094a0 0x7f9bd01098b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009900 tx=0x7f9bbc02a410 comp rx=0 tx=0).stop 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 -- 172.21.15.114:0/2481884985 shutdown_connections 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2481884985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd0109df0 0x7f9bd0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2481884985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd01094a0 0x7f9bd01098b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2481884985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd0103b60 0x7f9bd0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 -- 172.21.15.114:0/2481884985 >> 172.21.15.114:0/2481884985 conn(0x7f9bd007b850 msgr2=0x7f9bd007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 -- 172.21.15.114:0/2481884985 shutdown_connections 2024-09-21T06:55:48.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.538+0000 7f9bd7600640 1 -- 172.21.15.114:0/2481884985 wait complete. 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 Processor -- start 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 -- start start 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 0x7f9bd019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd01094a0 0x7f9bd019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd0109df0 0x7f9bd019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0113ab0 con 0x7f9bd0103b60 2024-09-21T06:55:48.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0113930 con 0x7f9bd01094a0 2024-09-21T06:55:48.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd7600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0113c30 con 0x7f9bd0109df0 2024-09-21T06:55:48.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.540+0000 7f9bd4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 0x7f9bd019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 0x7f9bd019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33750/0 (socket says 172.21.15.114:33750) 2024-09-21T06:55:48.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 -- 172.21.15.114:0/2518468834 learned_addr learned my addr 172.21.15.114:0/2518468834 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:48.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd01094a0 0x7f9bd019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd5600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd0109df0 0x7f9bd019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 -- 172.21.15.114:0/2518468834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd0109df0 msgr2=0x7f9bd019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd0109df0 0x7f9bd019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 -- 172.21.15.114:0/2518468834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd01094a0 msgr2=0x7f9bd019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd01094a0 0x7f9bd019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 -- 172.21.15.114:0/2518468834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd01a04d0 con 0x7f9bd0103b60 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd01094a0 0x7f9bd019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.541+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 0x7f9bd019a560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e9f0 tx=0x7f9bc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.542+0000 7f9bcd600640 1 -- 172.21.15.114:0/2518468834 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000cd80 con 0x7f9bd0103b60 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.542+0000 7f9bcd600640 1 -- 172.21.15.114:0/2518468834 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc00108e0 con 0x7f9bd0103b60 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.542+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd01a07c0 con 0x7f9bd0103b60 2024-09-21T06:55:48.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.542+0000 7f9bcd600640 1 -- 172.21.15.114:0/2518468834 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0018980 con 0x7f9bd0103b60 2024-09-21T06:55:48.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.542+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01a6f20 con 0x7f9bd0103b60 2024-09-21T06:55:48.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.545+0000 7f9bde1cb740 1 Processor -- start 2024-09-21T06:55:48.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.545+0000 7f9bde1cb740 1 -- start start 2024-09-21T06:55:48.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bcd600640 1 -- 172.21.15.114:0/2518468834 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9bc0010430 con 0x7f9bd0103b60 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da851da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da852820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5566da7c8fb0 con 0x5566da4ab070 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566da7b83e0 con 0x5566da4821b0 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5566da7bc1f0 con 0x5566da48a390 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bd5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da851da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da852820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bd5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da851da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33762/0 (socket says 172.21.15.114:33762) 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bd5600640 1 -- 172.21.15.114:0/3941605637 learned_addr learned my addr 172.21.15.114:0/3941605637 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:48.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.546+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3941605637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 msgr2=0x5566da8522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3941605637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 msgr2=0x5566da851da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da851da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3941605637 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566da782cd0 con 0x5566da48a390 2024-09-21T06:55:48.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bd5600640 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da851da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:48.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da852820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02a930 tx=0x7f9bbc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6fa8ca9ba97a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:48.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bcd600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bb40b6680 0x7f9bb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bc5c00640 1 -- 172.21.15.114:0/3941605637 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc035040 con 0x5566da48a390 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bc5c00640 1 -- 172.21.15.114:0/3941605637 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bbc031720 con 0x5566da48a390 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.547+0000 7f9bc5c00640 1 -- 172.21.15.114:0/3941605637 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc004770 con 0x5566da48a390 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bcd600640 1 -- 172.21.15.114:0/2518468834 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bc00e7e20 con 0x7f9bd0103b60 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bd5600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bb40b6680 0x7f9bb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3941605637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 msgr2=0x5566da852820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bd5600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bb40b6680 0x7f9bb40b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009fd0 tx=0x7f9bc8009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da852820 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02a930 tx=0x7f9bbc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3941605637 shutdown_connections 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da852820 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3941605637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da851da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.548+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3941605637 >> 172.21.15.114:0/3941605637 conn(0x5566da49f460 msgr2=0x5566da84fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.549+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3941605637 shutdown_connections 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.549+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3941605637 wait complete. 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.549+0000 7f9bde1cb740 1 -- start start 2024-09-21T06:55:48.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.549+0000 7f9bde1cb740 1 Processor -- start 2024-09-21T06:55:48.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da861e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da86dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5566da619700 con 0x5566da4ab070 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566da753580 con 0x5566da4821b0 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5566da864850 con 0x5566da48a390 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bde1cb740 10 client.0 _ll_register_callbacks cb 0x5566da763470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bd5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da86dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da861e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bd5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da86dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33766/0 (socket says 172.21.15.114:33766) 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bd5600640 1 -- 172.21.15.114:0/2623063300 learned_addr learned my addr 172.21.15.114:0/2623063300 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.550+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bcfe00640 1 -- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 msgr2=0x5566da8618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bcfe00640 1 -- 172.21.15.114:0/2623063300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 msgr2=0x5566da86dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da86dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bcfe00640 1 -- 172.21.15.114:0/2623063300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566da516650 con 0x5566da48a390 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da861e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02a930 tx=0x7f9bbc005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf18abef7d20a6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bba000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bba000640 1 -- 172.21.15.114:0/2623063300 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc04c030 con 0x5566da48a390 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566da755b90 con 0x5566da48a390 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.551+0000 7f9bba000640 1 -- 172.21.15.114:0/2623063300 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bbc031520 con 0x5566da48a390 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bba000640 1 -- 172.21.15.114:0/2623063300 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9bbc059050 con 0x5566da48a390 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 10 client.26026 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566da770950 con 0x5566da48a390 2024-09-21T06:55:48.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bba000640 1 -- 172.21.15.114:0/2623063300 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5566da755b90 con 0x5566da48a390 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bba000640 1 -- 172.21.15.114:0/2623063300 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9bbc03c4d0 con 0x5566da48a390 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 10 client.26026 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 10 client.26026 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 20 client.26026 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da4699f0 0x5566da873740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 4 client.26026 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.552+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566da7caf00 con 0x5566da4699f0 2024-09-21T06:55:48.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.553+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da4699f0 0x5566da873740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.554+0000 7f9bba000640 10 client.26026 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.554+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da4699f0 0x5566da873740 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89fc83979a56e960 server_cookie=af6df8fdfc36b48f in_seq=0 out_seq=0 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.554+0000 7f9bba000640 1 -- 172.21.15.114:0/2623063300 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566da7caf00 con 0x5566da4699f0 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.554+0000 7f9bba000640 10 client.26026 handle_command_reply: tid=0 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.554+0000 7f9bde1cb740 1 client.26026 shutdown 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 msgr2=0x5566da861e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da861e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02a930 tx=0x7f9bbc005780 comp rx=0 tx=0).stop 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 shutdown_connections 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da4ab070 0x5566da86dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da48a390 0x5566da861e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da4699f0 0x5566da873740 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/2623063300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4821b0 0x5566da8618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 >> 172.21.15.114:0/2623063300 conn(0x5566da49f460 msgr2=0x5566da854f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bba000640 0 client.26026 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 shutdown_connections 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 1 -- 172.21.15.114:0/2623063300 wait complete. 2024-09-21T06:55:48.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.555+0000 7f9bde1cb740 20 client.26026 trim_cache size 0 max 16384 2024-09-21T06:55:48.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 Processor -- start 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 -- start start 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da84fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da86db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da85eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01aed00 con 0x5566da48a390 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b0900 con 0x5566da4ab070 2024-09-21T06:55:48.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01af3d0 con 0x5566da4821b0 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da86db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da86db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33782/0 (socket says 172.21.15.114:33782) 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.569+0000 7f9bcfe00640 1 -- 172.21.15.114:0/1289321849 learned_addr learned my addr 172.21.15.114:0/1289321849 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bd5600640 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da85eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da84fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bcfe00640 1 -- 172.21.15.114:0/1289321849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 msgr2=0x5566da84fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da84fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bcfe00640 1 -- 172.21.15.114:0/1289321849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 msgr2=0x5566da85eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da85eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bcfe00640 1 -- 172.21.15.114:0/1289321849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566da862740 con 0x5566da48a390 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da86db20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009610 tx=0x7f9bbc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c808c1164e93d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bc5200640 1 -- 172.21.15.114:0/1289321849 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc056070 con 0x5566da48a390 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.570+0000 7f9bc5200640 1 -- 172.21.15.114:0/1289321849 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bbc04d040 con 0x5566da48a390 2024-09-21T06:55:48.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bc5200640 1 -- 172.21.15.114:0/1289321849 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc04c070 con 0x5566da48a390 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 -- 172.21.15.114:0/1289321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 msgr2=0x5566da86db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da86db20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009610 tx=0x7f9bbc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 -- 172.21.15.114:0/1289321849 shutdown_connections 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da85eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da86db20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/1289321849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da84fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 -- 172.21.15.114:0/1289321849 >> 172.21.15.114:0/1289321849 conn(0x5566da49f460 msgr2=0x5566da84f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 -- 172.21.15.114:0/1289321849 shutdown_connections 2024-09-21T06:55:48.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.571+0000 7f9bde1cb740 1 -- 172.21.15.114:0/1289321849 wait complete. 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.572+0000 7f9bde1cb740 1 -- start start 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.572+0000 7f9bde1cb740 1 Processor -- start 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da874e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da875360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da8dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b0170 con 0x5566da48a390 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01afaa0 con 0x5566da4ab070 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01acce0 con 0x5566da4821b0 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bde1cb740 10 client.0 _ll_register_callbacks cb 0x7f9bbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da875360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da875360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33786/0 (socket says 172.21.15.114:33786) 2024-09-21T06:55:48.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3373026275 learned_addr learned my addr 172.21.15.114:0/3373026275 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da874e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bd5600640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da8dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3373026275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 msgr2=0x5566da874e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da874e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 msgr2=0x5566da8dcec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da8dcec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.573+0000 7f9bcfe00640 1 -- 172.21.15.114:0/3373026275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566da8dd400 con 0x5566da48a390 2024-09-21T06:55:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bcfe00640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da875360 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02a930 tx=0x7f9bbc005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9330dd470d5dae0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bba000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:55:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bba000640 1 -- 172.21.15.114:0/3373026275 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc04c070 con 0x5566da48a390 2024-09-21T06:55:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bba000640 1 -- 172.21.15.114:0/3373026275 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bbc03c640 con 0x5566da48a390 2024-09-21T06:55:48.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bbc04d040 con 0x5566da48a390 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bba000640 1 -- 172.21.15.114:0/3373026275 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9bbc06c050 con 0x5566da48a390 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bde1cb740 10 client.15981 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.574+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566da8de5f0 con 0x5566da48a390 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bba000640 1 -- 172.21.15.114:0/3373026275 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bbc04d040 con 0x5566da48a390 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bba000640 1 -- 172.21.15.114:0/3373026275 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9bbc004970 con 0x5566da48a390 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bde1cb740 10 client.15981 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bde1cb740 10 client.15981 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bde1cb740 20 client.15981 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da43eda0 0x5566da8e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bde1cb740 4 client.15981 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566da7c69b0 con 0x5566da43eda0 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.575+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da43eda0 0x5566da8e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bd4c00640 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da43eda0 0x5566da8e0d40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=919e2fbe0a2dbd5 server_cookie=dc81a18c93274c6f in_seq=0 out_seq=0 2024-09-21T06:55:48.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bba000640 10 client.15981 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bba000640 1 -- 172.21.15.114:0/3373026275 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566da7c69b0 con 0x5566da43eda0 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bba000640 10 client.15981 handle_command_reply: tid=0 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bde1cb740 1 client.15981 shutdown 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 msgr2=0x5566da875360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da875360 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc02a930 tx=0x7f9bbc005910 comp rx=0 tx=0).stop 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.576+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 shutdown_connections 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5566da43eda0 0x5566da8e0d40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566da4ab070 0x5566da8dcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5566da48a390 0x5566da875360 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 --2- 172.21.15.114:0/3373026275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5566da4821b0 0x5566da874e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bba000640 0 client.15981 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 >> 172.21.15.114:0/3373026275 conn(0x5566da49f460 msgr2=0x5566da852dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 shutdown_connections 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 1 -- 172.21.15.114:0/3373026275 wait complete. 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.577+0000 7f9bde1cb740 20 client.15981 trim_cache size 0 max 16384 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bb40b6680 msgr2=0x7f9bb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bb40b6680 0x7f9bb40b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009fd0 tx=0x7f9bc8009500 comp rx=0 tx=0).stop 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 msgr2=0x7f9bd019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 0x7f9bd019a560 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000e9f0 tx=0x7f9bc000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 shutdown_connections 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bd0109df0 0x7f9bd019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bb40b6680 0x7f9bb40b8b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bd01094a0 0x7f9bd019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 --2- 172.21.15.114:0/2518468834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bd0103b60 0x7f9bd019a560 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 >> 172.21.15.114:0/2518468834 conn(0x7f9bd007b850 msgr2=0x7f9bd0105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 shutdown_connections 2024-09-21T06:55:48.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:48.578+0000 7f9bd7600640 1 -- 172.21.15.114:0/2518468834 wait complete. 2024-09-21T06:55:49.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:48 smithi114 ceph-mon[30654]: pgmap v446: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2024-09-21T06:55:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:48 smithi183 ceph-mon[34219]: pgmap v446: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2024-09-21T06:55:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:48 smithi193 ceph-mon[34601]: pgmap v446: 129 pgs: 129 active+clean; 1.3 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 MiB/s wr, 1.81k op/s 2024-09-21T06:55:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 -- 172.21.15.114:0/4128187110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c076100 msgr2=0x7f853c10f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 --2- 172.21.15.114:0/4128187110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c076100 0x7f853c10f920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8528009900 tx=0x7f852802a410 comp rx=0 tx=0).stop 2024-09-21T06:55:49.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 -- 172.21.15.114:0/4128187110 shutdown_connections 2024-09-21T06:55:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 --2- 172.21.15.114:0/4128187110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853c10fe60 0x7f853c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 --2- 172.21.15.114:0/4128187110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c076100 0x7f853c10f920 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 --2- 172.21.15.114:0/4128187110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f853c0757b0 0x7f853c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.585+0000 7f8540c00640 1 -- 172.21.15.114:0/4128187110 >> 172.21.15.114:0/4128187110 conn(0x7f853c0ff330 msgr2=0x7f853c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.586+0000 7f8540c00640 1 -- 172.21.15.114:0/4128187110 shutdown_connections 2024-09-21T06:55:49.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.586+0000 7f8540c00640 1 -- 172.21.15.114:0/4128187110 wait complete. 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.586+0000 7f8540c00640 1 Processor -- start 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 -- start start 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f853c0757b0 0x7f853c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853c076100 0x7f853c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 0x7f853c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f853c115a90 con 0x7f853c10fe60 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f853c115910 con 0x7f853c076100 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f8540c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f853c115c10 con 0x7f853c0757b0 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f853be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 0x7f853c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f853be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 0x7f853c1a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33810/0 (socket says 172.21.15.114:33810) 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f853be00640 1 -- 172.21.15.114:0/2289469057 learned_addr learned my addr 172.21.15.114:0/2289469057 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f853b400640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f853c0757b0 0x7f853c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.587+0000 7f853aa00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853c076100 0x7f853c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f853be00640 1 -- 172.21.15.114:0/2289469057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f853c0757b0 msgr2=0x7f853c19c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f853be00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f853c0757b0 0x7f853c19c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f853be00640 1 -- 172.21.15.114:0/2289469057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853c076100 msgr2=0x7f853c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f853be00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853c076100 0x7f853c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f853be00640 1 -- 172.21.15.114:0/2289469057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c1a23d0 con 0x7f853c10fe60 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f853be00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 0x7f853c1a1d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f853400b990 tx=0x7f853400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f8533e00640 1 -- 172.21.15.114:0/2289469057 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534019070 con 0x7f853c10fe60 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1a26c0 con 0x7f853c10fe60 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.588+0000 7f8533e00640 1 -- 172.21.15.114:0/2289469057 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8534014440 con 0x7f853c10fe60 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.589+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c1a8ee0 con 0x7f853c10fe60 2024-09-21T06:55:49.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.589+0000 7f8533e00640 1 -- 172.21.15.114:0/2289469057 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853401d5b0 con 0x7f853c10fe60 2024-09-21T06:55:49.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.591+0000 7f854924c740 1 Processor -- start 2024-09-21T06:55:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.593+0000 7f8533e00640 1 -- 172.21.15.114:0/2289469057 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f853401c070 con 0x7f853c10fe60 2024-09-21T06:55:49.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 -- start start 2024-09-21T06:55:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bd201b0 0x55720c0c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55720c03a6b0 con 0x55720bcf72f0 2024-09-21T06:55:49.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55720c029ae0 con 0x55720bcff4d0 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f854924c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55720c02d8f0 con 0x55720bd201b0 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f853b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f853b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33812/0 (socket says 172.21.15.114:33812) 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f853b400640 1 -- 172.21.15.114:0/3218551132 learned_addr learned my addr 172.21.15.114:0/3218551132 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.594+0000 7f853aa00640 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bd201b0 0x55720c0c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853aa00640 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853b400640 1 -- 172.21.15.114:0/3218551132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bd201b0 msgr2=0x55720c0c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853b400640 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bd201b0 0x55720c0c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853b400640 1 -- 172.21.15.114:0/3218551132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 msgr2=0x55720c0c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853b400640 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853b400640 1 -- 172.21.15.114:0/3218551132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720bff6780 con 0x55720bcf72f0 2024-09-21T06:55:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f853b400640 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c5350 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f852c00ad00 tx=0x7f852c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df16194acb582600 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f8533e00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85200b6680 0x7f85200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f8527e00640 1 -- 172.21.15.114:0/3218551132 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c01d070 con 0x55720bcf72f0 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f8527e00640 1 -- 172.21.15.114:0/3218551132 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c018af0 con 0x55720bcf72f0 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f8527e00640 1 -- 172.21.15.114:0/3218551132 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c010b60 con 0x55720bcf72f0 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.595+0000 7f8533e00640 1 -- 172.21.15.114:0/2289469057 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f85340e6f20 con 0x7f853c10fe60 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.596+0000 7f853aa00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85200b6680 0x7f85200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.596+0000 7f854924c740 1 -- 172.21.15.114:0/3218551132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 msgr2=0x55720c0c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.596+0000 7f854924c740 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c5350 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f852c00ad00 tx=0x7f852c00f790 comp rx=0 tx=0).stop 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.596+0000 7f853aa00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85200b6680 0x7f85200b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8528005940 tx=0x7f85280058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 -- 172.21.15.114:0/3218551132 shutdown_connections 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c5350 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 --2- 172.21.15.114:0/3218551132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bd201b0 0x55720c0c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 -- 172.21.15.114:0/3218551132 >> 172.21.15.114:0/3218551132 conn(0x55720bd14300 msgr2=0x55720c0c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:49.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 -- 172.21.15.114:0/3218551132 shutdown_connections 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 -- 172.21.15.114:0/3218551132 wait complete. 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 -- start start 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.597+0000 7f854924c740 1 Processor -- start 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c0d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55720bf61ab0 con 0x55720bd201b0 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55720bfd0570 con 0x55720bcff4d0 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55720c0d3590 con 0x55720bcf72f0 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f854924c740 10 client.0 _ll_register_callbacks cb 0x55720bfdf100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f853b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f853be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c0d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f853b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46674/0 (socket says 172.21.15.114:46674) 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f853b400640 1 -- 172.21.15.114:0/3355264940 learned_addr learned my addr 172.21.15.114:0/3355264940 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.598+0000 7f853aa00640 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f853b400640 1 -- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 msgr2=0x55720c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f853b400640 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f853b400640 1 -- 172.21.15.114:0/3355264940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 msgr2=0x55720c0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f853b400640 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c0d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f853b400640 1 -- 172.21.15.114:0/3355264940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720bd8b790 con 0x55720bcf72f0 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f853b400640 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d0350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f852c000c00 tx=0x7f852c018670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38aae5c3d522f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:49.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f8524c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f8524c00640 1 -- 172.21.15.114:0/3355264940 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c01d040 con 0x55720bcf72f0 2024-09-21T06:55:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852c018af0 con 0x55720bcf72f0 2024-09-21T06:55:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.599+0000 7f8524c00640 1 -- 172.21.15.114:0/3355264940 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f852c0108e0 con 0x55720bcf72f0 2024-09-21T06:55:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f8524c00640 1 -- 172.21.15.114:0/3355264940 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f852c031050 con 0x55720bcf72f0 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 10 client.26041 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55720be04620 con 0x55720bcf72f0 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f8524c00640 1 -- 172.21.15.114:0/3355264940 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c018af0 con 0x55720bcf72f0 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f8524c00640 1 -- 172.21.15.114:0/3355264940 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f852c020030 con 0x55720bcf72f0 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 10 client.26041 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 10 client.26041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 20 client.26041 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcded80 0x55720c0e2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 4 client.26041 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.600+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55720c03c600 con 0x55720bcded80 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.601+0000 7f853aa00640 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcded80 0x55720c0e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.601+0000 7f853aa00640 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcded80 0x55720c0e2420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d93b04cbfe9d441 server_cookie=6562931dbfc84f44 in_seq=0 out_seq=0 2024-09-21T06:55:49.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.601+0000 7f8524c00640 10 client.26041 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f8524c00640 1 -- 172.21.15.114:0/3355264940 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55720c03c600 con 0x55720bcded80 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f8524c00640 10 client.26041 handle_command_reply: tid=0 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 client.26041 shutdown 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 msgr2=0x55720c0d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d0350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f852c000c00 tx=0x7f852c018670 comp rx=0 tx=0).stop 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 shutdown_connections 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c0d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcded80 0x55720c0e2420 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 --2- 172.21.15.114:0/3355264940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d0350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f8524c00640 0 client.26041 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.602+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 >> 172.21.15.114:0/3355264940 conn(0x55720bd14300 msgr2=0x55720c0c5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.603+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 shutdown_connections 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.603+0000 7f854924c740 1 -- 172.21.15.114:0/3355264940 wait complete. 2024-09-21T06:55:49.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.603+0000 7f854924c740 20 client.26041 trim_cache size 0 max 16384 2024-09-21T06:55:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 Processor -- start 2024-09-21T06:55:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 -- start start 2024-09-21T06:55:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcff4d0 0x55720c0c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bd201b0 0x55720c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b2e80 con 0x55720bcf72f0 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b4a80 con 0x55720bd201b0 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.618+0000 7f854924c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b3550 con 0x55720bcff4d0 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33840/0 (socket says 172.21.15.114:33840) 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 -- 172.21.15.114:0/1512513533 learned_addr learned my addr 172.21.15.114:0/1512513533 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853aa00640 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcff4d0 0x55720c0c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853be00640 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bd201b0 0x55720c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 -- 172.21.15.114:0/1512513533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcff4d0 msgr2=0x55720c0c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcff4d0 0x55720c0c1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 -- 172.21.15.114:0/1512513533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bd201b0 msgr2=0x55720c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bd201b0 0x55720c0cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 -- 172.21.15.114:0/1512513533 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55720c0d1120 con 0x55720bcf72f0 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.619+0000 7f853b400640 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c6800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f852c00f7d0 tx=0x7f852c00ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea03347348d7ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f8527400640 1 -- 172.21.15.114:0/1512513533 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c03a070 con 0x55720bcf72f0 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f8527400640 1 -- 172.21.15.114:0/1512513533 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c028070 con 0x55720bcf72f0 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f8527400640 1 -- 172.21.15.114:0/1512513533 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c01d040 con 0x55720bcf72f0 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 -- 172.21.15.114:0/1512513533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 msgr2=0x55720c0c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c6800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f852c00f7d0 tx=0x7f852c00ac00 comp rx=0 tx=0).stop 2024-09-21T06:55:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 -- 172.21.15.114:0/1512513533 shutdown_connections 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bd201b0 0x55720c0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcff4d0 0x55720c0c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 --2- 172.21.15.114:0/1512513533 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bcf72f0 0x55720c0c6800 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.620+0000 7f854924c740 1 -- 172.21.15.114:0/1512513533 >> 172.21.15.114:0/1512513533 conn(0x55720bd14300 msgr2=0x55720c0c1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.621+0000 7f854924c740 1 -- 172.21.15.114:0/1512513533 shutdown_connections 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.621+0000 7f854924c740 1 -- 172.21.15.114:0/1512513533 wait complete. 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.621+0000 7f854924c740 1 -- start start 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.621+0000 7f854924c740 1 Processor -- start 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b42f0 con 0x55720bd201b0 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b3c20 con 0x55720bcff4d0 2024-09-21T06:55:49.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b0e60 con 0x55720bcf72f0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f854924c740 10 client.0 _ll_register_callbacks cb 0x7f852c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c14abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33846/0 (socket says 172.21.15.114:33846) 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 -- 172.21.15.114:0/1918091788 learned_addr learned my addr 172.21.15.114:0/1918091788 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853b400640 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 -- 172.21.15.114:0/1918091788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 msgr2=0x55720c0d5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 -- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 msgr2=0x55720c0d5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.622+0000 7f853be00640 1 -- 172.21.15.114:0/1918091788 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55720c0d67e0 con 0x55720bd201b0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f853be00640 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c14abc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f85340a9d50 tx=0x7f85340ac310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56b4f547aa086eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:49.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f8524c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f8524c00640 1 -- 172.21.15.114:0/1918091788 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340b1510 con 0x55720bd201b0 2024-09-21T06:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f8524c00640 1 -- 172.21.15.114:0/1918091788 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f85340a8d50 con 0x55720bd201b0 2024-09-21T06:55:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852c028070 con 0x55720bd201b0 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f8524c00640 1 -- 172.21.15.114:0/1918091788 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f85340b63c0 con 0x55720bd201b0 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f854924c740 10 client.15999 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55720c14c120 con 0x55720bd201b0 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f8524c00640 1 -- 172.21.15.114:0/1918091788 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c028070 con 0x55720bd201b0 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f8524c00640 1 -- 172.21.15.114:0/1918091788 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f853400caa0 con 0x55720bd201b0 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f854924c740 10 client.15999 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.623+0000 7f854924c740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f854924c740 20 client.15999 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f854924c740 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcb41c0 0x55720c14e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f854924c740 4 client.15999 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:55:49.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55720c0380b0 con 0x55720bcb41c0 2024-09-21T06:55:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f853b400640 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcb41c0 0x55720c14e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f8524c00640 10 client.15999 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.624+0000 7f853b400640 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcb41c0 0x55720c14e9d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c17c2374756f6d server_cookie=63e3a04c5ad4e8ac in_seq=0 out_seq=0 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f8524c00640 1 -- 172.21.15.114:0/1918091788 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55720c0380b0 con 0x55720bcb41c0 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f8524c00640 10 client.15999 handle_command_reply: tid=0 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 client.15999 shutdown 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 msgr2=0x55720c14abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c14abc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f85340a9d50 tx=0x7f85340ac310 comp rx=0 tx=0).stop 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 shutdown_connections 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55720bd201b0 0x55720c14abc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55720bcff4d0 0x55720c0d5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55720bcb41c0 0x55720c14e9d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 --2- 172.21.15.114:0/1918091788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55720bcf72f0 0x55720c0d5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f8524c00640 0 client.15999 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 >> 172.21.15.114:0/1918091788 conn(0x55720bd14300 msgr2=0x55720c0c65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 shutdown_connections 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 1 -- 172.21.15.114:0/1918091788 wait complete. 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.641+0000 7f854924c740 20 client.15999 trim_cache size 0 max 16384 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:55:49.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:55:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85200b6680 msgr2=0x7f85200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85200b6680 0x7f85200b8b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8528005940 tx=0x7f85280058d0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 msgr2=0x7f853c1a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 0x7f853c1a1d60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f853400b990 tx=0x7f853400bd90 comp rx=0 tx=0).stop 2024-09-21T06:55:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 shutdown_connections 2024-09-21T06:55:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f853c10fe60 0x7f853c1a1d60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85200b6680 0x7f85200b8b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f853c076100 0x7f853c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 --2- 172.21.15.114:0/2289469057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f853c0757b0 0x7f853c19c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 >> 172.21.15.114:0/2289469057 conn(0x7f853c0ff330 msgr2=0x7f853c1088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 shutdown_connections 2024-09-21T06:55:49.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:55:49.642+0000 7f8540c00640 1 -- 172.21.15.114:0/2289469057 wait complete. 2024-09-21T06:55:50.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:55:50.260 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:55:50.419 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:50.878 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:50 smithi114 ceph-mon[30654]: pgmap v447: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2024-09-21T06:55:51.061 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:51.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.057+0000 7f79d6a00640 1 -- 172.21.15.114:0/1281118007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d01049b0 msgr2=0x7f79d0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.057+0000 7f79d6a00640 1 --2- 172.21.15.114:0/1281118007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d01049b0 0x7f79d0104dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009860 tx=0x7f79c002a340 comp rx=0 tx=0).stop 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.057+0000 7f79d6a00640 1 -- 172.21.15.114:0/1281118007 shutdown_connections 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.057+0000 7f79d6a00640 1 --2- 172.21.15.114:0/1281118007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d010de10 0x7f79d0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.057+0000 7f79d6a00640 1 --2- 172.21.15.114:0/1281118007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 0x7f79d010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.057+0000 7f79d6a00640 1 --2- 172.21.15.114:0/1281118007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d01049b0 0x7f79d0104dc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.058+0000 7f79d6a00640 1 -- 172.21.15.114:0/1281118007 >> 172.21.15.114:0/1281118007 conn(0x7f79d0079730 msgr2=0x7f79d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.058+0000 7f79d6a00640 1 -- 172.21.15.114:0/1281118007 shutdown_connections 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.058+0000 7f79d6a00640 1 -- 172.21.15.114:0/1281118007 wait complete. 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 Processor -- start 2024-09-21T06:55:51.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 -- start start 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 0x7f79d01026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 0x7f79d0102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d010de10 0x7f79d0103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0113a90 con 0x7f79d01049b0 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0113910 con 0x7f79d0105300 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.059+0000 7f79d6a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0113c10 con 0x7f79d010de10 2024-09-21T06:55:51.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 0x7f79d01026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cf400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 0x7f79d0102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 0x7f79d01026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33868/0 (socket says 172.21.15.114:33868) 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 -- 172.21.15.114:0/2277839560 learned_addr learned my addr 172.21.15.114:0/2277839560 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79d4a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d010de10 0x7f79d0103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 -- 172.21.15.114:0/2277839560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d010de10 msgr2=0x7f79d0103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d010de10 0x7f79d0103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 -- 172.21.15.114:0/2277839560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 msgr2=0x7f79d0102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 0x7f79d0102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cfe00640 1 -- 172.21.15.114:0/2277839560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d01a87a0 con 0x7f79d01049b0 2024-09-21T06:55:51.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.060+0000 7f79cf400640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 0x7f79d0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:51.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.061+0000 7f79cfe00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 0x7f79d01026f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f79c0000c00 tx=0x7f79c0032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:51.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.061+0000 7f79ccc00640 1 -- 172.21.15.114:0/2277839560 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0040020 con 0x7f79d01049b0 2024-09-21T06:55:51.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.061+0000 7f79ccc00640 1 -- 172.21.15.114:0/2277839560 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c0030d50 con 0x7f79d01049b0 2024-09-21T06:55:51.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.061+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d01a8a30 con 0x7f79d01049b0 2024-09-21T06:55:51.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.061+0000 7f79ccc00640 1 -- 172.21.15.114:0/2277839560 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0037040 con 0x7f79d01049b0 2024-09-21T06:55:51.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.061+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d01a8f10 con 0x7f79d01049b0 2024-09-21T06:55:51.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.064+0000 7f79dd692740 1 Processor -- start 2024-09-21T06:55:51.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.064+0000 7f79dd692740 1 -- start start 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79dd692740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cce0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79dd692740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946cce12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79dd692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946cce1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79dd692740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55946cc57fb0 con 0x55946c93a070 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79dd692740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55946cc473e0 con 0x55946c9111b0 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79dd692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55946cc4b1f0 con 0x55946c919390 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79d4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cce0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79d4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cce0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33878/0 (socket says 172.21.15.114:33878) 2024-09-21T06:55:51.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.065+0000 7f79d4a00640 1 -- 172.21.15.114:0/2955148350 learned_addr learned my addr 172.21.15.114:0/2955148350 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:51.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79d4a00640 1 -- 172.21.15.114:0/2955148350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 msgr2=0x55946cce1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79cf400640 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946cce1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79d4a00640 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946cce1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79d4a00640 1 -- 172.21.15.114:0/2955148350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 msgr2=0x55946cce12e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79d4a00640 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946cce12e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79d4a00640 1 -- 172.21.15.114:0/2955148350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946cc11cd0 con 0x55946c93a070 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79d4a00640 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cce0da0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009fd0 tx=0x7f79c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba02858c0f140a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79cf400640 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946cce1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79ccc00640 1 -- 172.21.15.114:0/2277839560 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f79c0043020 con 0x7f79d01049b0 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.066+0000 7f79c5200640 1 -- 172.21.15.114:0/2955148350 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8004d20 con 0x55946c93a070 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.067+0000 7f79c5200640 1 -- 172.21.15.114:0/2955148350 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c8004050 con 0x55946c93a070 2024-09-21T06:55:51.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.067+0000 7f79c5200640 1 -- 172.21.15.114:0/2955148350 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8005240 con 0x55946c93a070 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.067+0000 7f79dd692740 1 -- 172.21.15.114:0/2955148350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 msgr2=0x55946cce0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.067+0000 7f79dd692740 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cce0da0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009fd0 tx=0x7f79c80118d0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 -- 172.21.15.114:0/2955148350 shutdown_connections 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946cce1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946cce12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 --2- 172.21.15.114:0/2955148350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cce0da0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 -- 172.21.15.114:0/2955148350 >> 172.21.15.114:0/2955148350 conn(0x55946c92e460 msgr2=0x55946ccdec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 -- 172.21.15.114:0/2955148350 shutdown_connections 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79dd692740 1 -- 172.21.15.114:0/2955148350 wait complete. 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.068+0000 7f79ccc00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79b40b6590 0x7f79b40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.069+0000 7f79dd692740 1 -- start start 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.069+0000 7f79ccc00640 1 -- 172.21.15.114:0/2277839560 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f79c01073d0 con 0x7f79d01049b0 2024-09-21T06:55:51.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.069+0000 7f79d4a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79b40b6590 0x7f79b40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.069+0000 7f79dd692740 1 Processor -- start 2024-09-21T06:55:51.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79d4a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79b40b6590 0x7f79b40b8aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009de0 tx=0x7f79c8005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:51.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946ccf4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cd01070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55946caa8700 con 0x55946c93a070 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55946cbe2580 con 0x55946c9111b0 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55946ccf7cb0 con 0x55946c919390 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79dd692740 10 client.0 _ll_register_callbacks cb 0x55946cbf2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946ccf4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79cf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79cf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46734/0 (socket says 172.21.15.114:46734) 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79cf400640 1 -- 172.21.15.114:0/1866281087 learned_addr learned my addr 172.21.15.114:0/1866281087 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.070+0000 7f79d4a00640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cd01070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cfe00640 1 -- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 msgr2=0x55946ccf5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cfe00640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cfe00640 1 -- 172.21.15.114:0/1866281087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 msgr2=0x55946cd01070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cfe00640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cd01070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cfe00640 1 -- 172.21.15.114:0/1866281087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946c9a5650 con 0x55946c9111b0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cf400640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79cfe00640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946ccf4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f79c0104750 tx=0x7f79c00d67a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f74f3cdd09d843 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79b9600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79b9600640 1 -- 172.21.15.114:0/1866281087 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00eee60 con 0x55946c9111b0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55946cbe4b90 con 0x55946c9111b0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.071+0000 7f79b9600640 1 -- 172.21.15.114:0/1866281087 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f79c00c87a0 con 0x55946c9111b0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79b9600640 1 -- 172.21.15.114:0/1866281087 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79c00cc7a0 con 0x55946c9111b0 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 10 client.26087 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:51.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55946cbff950 con 0x55946c9111b0 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79b9600640 1 -- 172.21.15.114:0/1866281087 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c00cf020 con 0x55946c9111b0 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79b9600640 1 -- 172.21.15.114:0/1866281087 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f79c00cbb20 con 0x55946c9111b0 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 10 client.26087 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 10 client.26087 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 20 client.26087 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946c8f89f0 0x55946cd06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 4 client.26087 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.072+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946cc59f00 con 0x55946c8f89f0 2024-09-21T06:55:51.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.073+0000 7f79cf400640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946c8f89f0 0x55946cd06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.074+0000 7f79cf400640 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946c8f89f0 0x55946cd06d90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=683eb37bb7f9c75f server_cookie=90f00f75bd503a53 in_seq=0 out_seq=0 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.074+0000 7f79b9600640 10 client.26087 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79b9600640 1 -- 172.21.15.114:0/1866281087 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55946cc59f00 con 0x55946c8f89f0 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79b9600640 10 client.26087 handle_command_reply: tid=0 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 client.26087 shutdown 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 msgr2=0x55946ccf4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946ccf4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f79c0104750 tx=0x7f79c00d67a0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 shutdown_connections 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946c8f89f0 0x55946cd06d90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c93a070 0x55946cd01070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 --2- 172.21.15.114:0/1866281087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c9111b0 0x55946ccf4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 >> 172.21.15.114:0/1866281087 conn(0x55946c92e460 msgr2=0x55946ccdd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:51.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79b9600640 0 client.26087 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:55:51.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 shutdown_connections 2024-09-21T06:55:51.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.075+0000 7f79dd692740 1 -- 172.21.15.114:0/1866281087 wait complete. 2024-09-21T06:55:51.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.076+0000 7f79dd692740 20 client.26087 trim_cache size 0 max 16384 2024-09-21T06:55:51.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.089+0000 7f79dd692740 1 Processor -- start 2024-09-21T06:55:51.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.089+0000 7f79dd692740 1 -- start start 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79dd692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 0x55946ccdecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79dd692740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c919390 0x55946cd00f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79dd692740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79dd692740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79b000fdd0 con 0x55946c919390 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79dd692740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79b00119d0 con 0x55946c93a070 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79dd692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79b00104a0 con 0x55946c9111b0 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79cf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c919390 0x55946cd00f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 0x55946ccdecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79cf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c919390 0x55946cd00f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33888/0 (socket says 172.21.15.114:33888) 2024-09-21T06:55:51.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 0x55946ccdecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46748/0 (socket says 172.21.15.114:46748) 2024-09-21T06:55:51.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79cf400640 1 -- 172.21.15.114:0/3302660619 learned_addr learned my addr 172.21.15.114:0/3302660619 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:51.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.090+0000 7f79d4a00640 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79cfe00640 1 -- 172.21.15.114:0/3302660619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 msgr2=0x55946ccf1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79cfe00640 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79cfe00640 1 -- 172.21.15.114:0/3302660619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c919390 msgr2=0x55946cd00f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79cfe00640 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c919390 0x55946cd00f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79cfe00640 1 -- 172.21.15.114:0/3302660619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946ccdf610 con 0x55946c9111b0 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79cfe00640 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 0x55946ccdecc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f79c0035040 tx=0x7f79c003f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9bda3de1bafba11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79bbe00640 1 -- 172.21.15.114:0/3302660619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00eee60 con 0x55946c9111b0 2024-09-21T06:55:51.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79bbe00640 1 -- 172.21.15.114:0/3302660619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55946cbe4b90 con 0x55946c9111b0 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.091+0000 7f79bbe00640 1 -- 172.21.15.114:0/3302660619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00cc5c0 con 0x55946c9111b0 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 -- 172.21.15.114:0/3302660619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 msgr2=0x55946ccdecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 0x55946ccdecc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f79c0035040 tx=0x7f79c003f000 comp rx=0 tx=0).stop 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 -- 172.21.15.114:0/3302660619 shutdown_connections 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c919390 0x55946cd00f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 --2- 172.21.15.114:0/3302660619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c9111b0 0x55946ccdecc0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f79c0035040 tx=0x7f79c003f000 comp rx=0 tx=0).stop 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.092+0000 7f79dd692740 1 -- 172.21.15.114:0/3302660619 >> 172.21.15.114:0/3302660619 conn(0x55946c92e460 msgr2=0x55946ccde5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:51.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.093+0000 7f79dd692740 1 -- 172.21.15.114:0/3302660619 shutdown_connections 2024-09-21T06:55:51.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.093+0000 7f79dd692740 1 -- 172.21.15.114:0/3302660619 wait complete. 2024-09-21T06:55:51.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.093+0000 7f79dd692740 1 -- start start 2024-09-21T06:55:51.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.093+0000 7f79dd692740 1 Processor -- start 2024-09-21T06:55:51.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c8cdda0 0x55946cd6a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0011240 con 0x55946c8cdda0 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79b0010b70 con 0x55946c93a070 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79b000ddb0 con 0x55946c919390 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79dd692740 10 client.0 _ll_register_callbacks cb 0x7f79c00c92f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79d4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c8cdda0 0x55946cd6a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46760/0 (socket says 172.21.15.114:46760) 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79cfe00640 1 -- 172.21.15.114:0/3932269386 learned_addr learned my addr 172.21.15.114:0/3932269386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.094+0000 7f79cf400640 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79d4a00640 1 -- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 msgr2=0x55946ccf9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79d4a00640 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79d4a00640 1 -- 172.21.15.114:0/3932269386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 msgr2=0x55946ccf9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:51.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79d4a00640 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79d4a00640 1 -- 172.21.15.114:0/3932269386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946cd6adf0 con 0x55946c8cdda0 2024-09-21T06:55:51.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79d4a00640 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c8cdda0 0x55946cd6a780 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79c80048f0 tx=0x7f79c8004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaa5eee81d752c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:51.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.095+0000 7f79b9600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:55:51.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.096+0000 7f79b9600640 1 -- 172.21.15.114:0/3932269386 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8004e50 con 0x55946c8cdda0 2024-09-21T06:55:51.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.096+0000 7f79b9600640 1 -- 172.21.15.114:0/3932269386 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f79c8005720 con 0x55946c8cdda0 2024-09-21T06:55:51.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.096+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55946cbe4b90 con 0x55946c8cdda0 2024-09-21T06:55:51.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.096+0000 7f79b9600640 1 -- 172.21.15.114:0/3932269386 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79c8040050 con 0x55946c8cdda0 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.096+0000 7f79dd692740 10 client.16017 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.096+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55946cd72110 con 0x55946c8cdda0 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79b9600640 1 -- 172.21.15.114:0/3932269386 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55946cbe4b90 con 0x55946c8cdda0 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79b9600640 1 -- 172.21.15.114:0/3932269386 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f79c8025480 con 0x55946c8cdda0 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79dd692740 10 client.16017 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79dd692740 10 client.16017 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79dd692740 20 client.16017 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79dd692740 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946cd726c0 0x55946cd74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:51.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79dd692740 4 client.16017 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55946cc559b0 con 0x55946cd726c0 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.097+0000 7f79cfe00640 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946cd726c0 0x55946cd74ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79b9600640 10 client.16017 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79cfe00640 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946cd726c0 0x55946cd74ae0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63ac4fddfcaf1d1 server_cookie=f341468037f5ee4d in_seq=0 out_seq=0 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79b9600640 1 -- 172.21.15.114:0/3932269386 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55946cc559b0 con 0x55946cd726c0 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79b9600640 10 client.16017 handle_command_reply: tid=0 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79dd692740 1 client.16017 shutdown 2024-09-21T06:55:51.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c8cdda0 msgr2=0x55946cd6a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.098+0000 7f79dd692740 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c8cdda0 0x55946cd6a780 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79c80048f0 tx=0x7f79c8004950 comp rx=0 tx=0).stop 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 shutdown_connections 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55946cd726c0 0x55946cd74ae0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55946c8cdda0 0x55946cd6a780 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55946c93a070 0x55946ccf9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 --2- 172.21.15.114:0/3932269386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55946c919390 0x55946ccf9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 >> 172.21.15.114:0/3932269386 conn(0x55946c92e460 msgr2=0x55946cce1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:51.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79b9600640 0 client.16017 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:55:51.105 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:55:51.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 shutdown_connections 2024-09-21T06:55:51.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 1 -- 172.21.15.114:0/3932269386 wait complete. 2024-09-21T06:55:51.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.099+0000 7f79dd692740 20 client.16017 trim_cache size 0 max 16384 2024-09-21T06:55:51.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79b40b6590 msgr2=0x7f79b40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79b40b6590 0x7f79b40b8aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009de0 tx=0x7f79c8005fd0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 msgr2=0x7f79d01026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 0x7f79d01026f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f79c0000c00 tx=0x7f79c0032d40 comp rx=0 tx=0).stop 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 shutdown_connections 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d010de10 0x7f79d0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79b40b6590 0x7f79b40b8aa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79d0105300 0x7f79d0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 --2- 172.21.15.114:0/2277839560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f79d01049b0 0x7f79d01026f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 >> 172.21.15.114:0/2277839560 conn(0x7f79d0079730 msgr2=0x7f79d01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 shutdown_connections 2024-09-21T06:55:51.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:51.100+0000 7f79d6a00640 1 -- 172.21.15.114:0/2277839560 wait complete. 2024-09-21T06:55:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:50 smithi183 ceph-mon[34219]: pgmap v447: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2024-09-21T06:55:51.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:50 smithi193 ceph-mon[34601]: pgmap v447: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2024-09-21T06:55:51.988 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:55:52.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 -- 172.21.15.114:0/2193075110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd570105d80 msgr2=0x7fd570106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:52.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 --2- 172.21.15.114:0/2193075110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd570105d80 0x7fd570106190 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd560009900 tx=0x7fd56002a410 comp rx=0 tx=0).stop 2024-09-21T06:55:52.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 -- 172.21.15.114:0/2193075110 shutdown_connections 2024-09-21T06:55:52.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 --2- 172.21.15.114:0/2193075110 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd57010c010 0x7fd570112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 --2- 172.21.15.114:0/2193075110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010b6c0 0x7fd57010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 --2- 172.21.15.114:0/2193075110 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd570105d80 0x7fd570106190 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 -- 172.21.15.114:0/2193075110 >> 172.21.15.114:0/2193075110 conn(0x7fd570101810 msgr2=0x7fd570103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:52.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 -- 172.21.15.114:0/2193075110 shutdown_connections 2024-09-21T06:55:52.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.672+0000 7fd574e00640 1 -- 172.21.15.114:0/2193075110 wait complete. 2024-09-21T06:55:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.673+0000 7fd574e00640 1 Processor -- start 2024-09-21T06:55:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.673+0000 7fd574e00640 1 -- start start 2024-09-21T06:55:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.673+0000 7fd574e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570105d80 0x7fd57019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.673+0000 7fd574e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 0x7fd57019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd574e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010c010 0x7fd5701a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd574e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd570115d20 con 0x7fd57010b6c0 2024-09-21T06:55:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd574e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd570115ba0 con 0x7fd57010c010 2024-09-21T06:55:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd574e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd570115ea0 con 0x7fd570105d80 2024-09-21T06:55:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 0x7fd57019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 0x7fd57019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33930/0 (socket says 172.21.15.114:33930) 2024-09-21T06:55:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 -- 172.21.15.114:0/1278795818 learned_addr learned my addr 172.21.15.114:0/1278795818 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56de00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570105d80 0x7fd57019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56e800640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010c010 0x7fd5701a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 -- 172.21.15.114:0/1278795818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570105d80 msgr2=0x7fd57019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570105d80 0x7fd57019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 -- 172.21.15.114:0/1278795818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010c010 msgr2=0x7fd5701a1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010c010 0x7fd5701a1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56d400640 1 -- 172.21.15.114:0/1278795818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5701a2680 con 0x7fd57010b6c0 2024-09-21T06:55:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.674+0000 7fd56e800640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010c010 0x7fd5701a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.675+0000 7fd56d400640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 0x7fd57019ccb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00e9f0 tx=0x7fd55c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.675+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00ccc0 con 0x7fd57010b6c0 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.675+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c0108e0 con 0x7fd57010b6c0 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.675+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5701a2970 con 0x7fd57010b6c0 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.675+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c018860 con 0x7fd57010b6c0 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.675+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5701a90d0 con 0x7fd57010b6c0 2024-09-21T06:55:52.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.676+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd570106190 con 0x7fd57010b6c0 2024-09-21T06:55:52.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.680+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd55c010430 con 0x7fd57010b6c0 2024-09-21T06:55:52.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.682+0000 7fd566a00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5540b6680 0x7fd5540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:52.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.682+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd55c0e81c0 con 0x7fd57010b6c0 2024-09-21T06:55:52.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.682+0000 7fd56de00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5540b6680 0x7fd5540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:52.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.684+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd55c0b2780 con 0x7fd57010b6c0 2024-09-21T06:55:52.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.684+0000 7fd56de00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5540b6680 0x7fd5540b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd560005ab0 tx=0x7fd560005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:52.806 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:52.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:52 smithi114 ceph-mon[30654]: pgmap v448: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.44k op/s 2024-09-21T06:55:52.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 75.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:55:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.935+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd57010de10 con 0x7fd57010b6c0 2024-09-21T06:55:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.936+0000 7fd566a00640 1 -- 172.21.15.114:0/1278795818 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fd55c0b76c0 con 0x7fd57010b6c0 2024-09-21T06:55:52.939 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:55:52.939 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:55:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:55:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5540b6680 msgr2=0x7fd5540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5540b6680 0x7fd5540b8b90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd560005ab0 tx=0x7fd560005a20 comp rx=0 tx=0).stop 2024-09-21T06:55:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 msgr2=0x7fd57019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 0x7fd57019ccb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00e9f0 tx=0x7fd55c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 shutdown_connections 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5540b6680 0x7fd5540b8b90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57010c010 0x7fd5701a1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57010b6c0 0x7fd57019ccb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.938+0000 7fd574e00640 1 --2- 172.21.15.114:0/1278795818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570105d80 0x7fd57019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.939+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 >> 172.21.15.114:0/1278795818 conn(0x7fd570101810 msgr2=0x7fd570103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.939+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 shutdown_connections 2024-09-21T06:55:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:52.939+0000 7fd574e00640 1 -- 172.21.15.114:0/1278795818 wait complete. 2024-09-21T06:55:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:52 smithi183 ceph-mon[34219]: pgmap v448: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.44k op/s 2024-09-21T06:55:53.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:52 smithi193 ceph-mon[34601]: pgmap v448: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 1.44k op/s 2024-09-21T06:55:53.739 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:53 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1278795818' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:53.739 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:53 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:55:53.780 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 745fa756 --release --await' 2024-09-21T06:55:53.780 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 745fa756 --release --await 2024-09-21T06:55:54.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:53 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1278795818' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:54.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:53 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:55:54.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.122+0000 7f056fa00640 1 -- 172.21.15.114:0/3720059365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056810de30 msgr2=0x7f0568110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.122+0000 7f056fa00640 1 --2- 172.21.15.114:0/3720059365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056810de30 0x7f0568110320 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0560009ca0 tx=0x7f056002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 -- 172.21.15.114:0/3720059365 shutdown_connections 2024-09-21T06:55:54.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 --2- 172.21.15.114:0/3720059365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f056810de30 0x7f0568110320 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 --2- 172.21.15.114:0/3720059365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0568105320 0x7f056810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 --2- 172.21.15.114:0/3720059365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 0x7f0568104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 -- 172.21.15.114:0/3720059365 >> 172.21.15.114:0/3720059365 conn(0x7f0568079740 msgr2=0x7f0568079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:54.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 -- 172.21.15.114:0/3720059365 shutdown_connections 2024-09-21T06:55:54.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.123+0000 7f056fa00640 1 -- 172.21.15.114:0/3720059365 wait complete. 2024-09-21T06:55:54.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.124+0000 7f056fa00640 1 Processor -- start 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.124+0000 7f056fa00640 1 -- start start 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.124+0000 7f056fa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 0x7f056819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.124+0000 7f056fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 0x7f056819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.124+0000 7f056fa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056810de30 0x7f056819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f056fa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0568113ab0 con 0x7f056810de30 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f056fa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0568113930 con 0x7f0568105320 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f056fa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0568113c30 con 0x7f05681049d0 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 0x7f056819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 0x7f056819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35006/0 (socket says 172.21.15.114:35006) 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 -- 172.21.15.114:0/1370482362 learned_addr learned my addr 172.21.15.114:0/1370482362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:54.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f056da00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056810de30 0x7f056819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f056d000640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 0x7f056819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 -- 172.21.15.114:0/1370482362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 msgr2=0x7f056819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 0x7f056819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 -- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056810de30 msgr2=0x7f056819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056810de30 0x7f056819fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f0567e00640 1 -- 172.21.15.114:0/1370482362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05681a0470 con 0x7f0568105320 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.125+0000 7f056d000640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 0x7f056819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.126+0000 7f0567e00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 0x7f056819aaa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f055400e970 tx=0x7f055400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.126+0000 7f0565600640 1 -- 172.21.15.114:0/1370482362 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055400cd80 con 0x7f0568105320 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.126+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05681a0760 con 0x7f0568105320 2024-09-21T06:55:54.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.126+0000 7f0565600640 1 -- 172.21.15.114:0/1370482362 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05540035f0 con 0x7f0568105320 2024-09-21T06:55:54.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.126+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05681025f0 con 0x7f0568105320 2024-09-21T06:55:54.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.126+0000 7f0565600640 1 -- 172.21.15.114:0/1370482362 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554010690 con 0x7f0568105320 2024-09-21T06:55:54.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.129+0000 7f0576680740 1 Processor -- start 2024-09-21T06:55:54.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 -- start start 2024-09-21T06:55:54.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd02ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 0x562e0bd030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd03620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e0bc79fb0 con 0x562e0b93b390 2024-09-21T06:55:54.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e0bc693e0 con 0x562e0b95c070 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f0576680740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e0bc6d1f0 con 0x562e0b9331b0 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f056d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 0x562e0bd030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f056d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 0x562e0bd030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46792/0 (socket says 172.21.15.114:46792) 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f056d000640 1 -- 172.21.15.114:0/2775657288 learned_addr learned my addr 172.21.15.114:0/2775657288 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.130+0000 7f056da00640 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd02ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f056d000640 1 -- 172.21.15.114:0/2775657288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 msgr2=0x562e0bd02ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f056d000640 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd02ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f056d000640 1 -- 172.21.15.114:0/2775657288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 msgr2=0x562e0bd03620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:54.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f0565600640 1 -- 172.21.15.114:0/1370482362 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0554010830 con 0x7f0568105320 2024-09-21T06:55:54.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f056d000640 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd03620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f056d000640 1 -- 172.21.15.114:0/2775657288 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0bc33cd0 con 0x562e0b9331b0 2024-09-21T06:55:54.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.131+0000 7f056d000640 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 0x562e0bd030e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f055c009cf0 tx=0x7f055c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff3060d18590ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:54.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.132+0000 7f055a000640 1 -- 172.21.15.114:0/2775657288 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c012450 con 0x562e0b9331b0 2024-09-21T06:55:54.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.132+0000 7f055a000640 1 -- 172.21.15.114:0/2775657288 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055c012aa0 con 0x562e0b9331b0 2024-09-21T06:55:54.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.132+0000 7f055a000640 1 -- 172.21.15.114:0/2775657288 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c00fb90 con 0x562e0b9331b0 2024-09-21T06:55:54.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0565600640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05500b6780 0x7f05500b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0565600640 1 -- 172.21.15.114:0/1370482362 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f05540e7d70 con 0x7f0568105320 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f056da00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05500b6780 0x7f05500b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 -- 172.21.15.114:0/2775657288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 msgr2=0x562e0bd030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 0x562e0bd030e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f055c009cf0 tx=0x7f055c0117b0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 -- 172.21.15.114:0/2775657288 shutdown_connections 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd03620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b9331b0 0x562e0bd030e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 --2- 172.21.15.114:0/2775657288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd02ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 -- 172.21.15.114:0/2775657288 >> 172.21.15.114:0/2775657288 conn(0x562e0b950460 msgr2=0x562e0bd00a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:54.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.133+0000 7f0576680740 1 -- 172.21.15.114:0/2775657288 shutdown_connections 2024-09-21T06:55:54.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.134+0000 7f0576680740 1 -- 172.21.15.114:0/2775657288 wait complete. 2024-09-21T06:55:54.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.134+0000 7f056da00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05500b6780 0x7f05500b8c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f056002f500 tx=0x7f056002f310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:54.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.134+0000 7f0576680740 1 -- start start 2024-09-21T06:55:54.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.134+0000 7f0576680740 1 Processor -- start 2024-09-21T06:55:54.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b9331b0 0x562e0bd0e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b93b390 0x562e0bd0ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd1a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e0baca700 con 0x562e0b9331b0 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e0bc04580 con 0x562e0b95c070 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e0bd11380 con 0x562e0b93b390 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0576680740 10 client.0 _ll_register_callbacks cb 0x562e0bc14470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f056da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd1a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f056da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd1a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35030/0 (socket says 172.21.15.114:35030) 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f056da00640 1 -- 172.21.15.114:0/194505317 learned_addr learned my addr 172.21.15.114:0/194505317 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.135+0000 7f0567e00640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b93b390 0x562e0bd0ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056d000640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b9331b0 0x562e0bd0e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056da00640 1 -- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b93b390 msgr2=0x562e0bd0ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056da00640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b93b390 0x562e0bd0ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056da00640 1 -- 172.21.15.114:0/194505317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b9331b0 msgr2=0x562e0bd0e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056da00640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b9331b0 0x562e0bd0e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056da00640 1 -- 172.21.15.114:0/194505317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0b9c7650 con 0x562e0b95c070 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f0567e00640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b93b390 0x562e0bd0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f056da00640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd1a6a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0560009ca0 tx=0x7f056002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4074e6ac74031c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.136+0000 7f054ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f054ea00640 1 -- 172.21.15.114:0/194505317 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560003c40 con 0x562e0b95c070 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f054ea00640 1 -- 172.21.15.114:0/194505317 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0560003de0 con 0x562e0b95c070 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e0bc06b90 con 0x562e0b95c070 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f054ea00640 1 -- 172.21.15.114:0/194505317 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0560004040 con 0x562e0b95c070 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 10 client.26111 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562e0bc21950 con 0x562e0b95c070 2024-09-21T06:55:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f054ea00640 1 -- 172.21.15.114:0/194505317 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e0bc06b90 con 0x562e0b95c070 2024-09-21T06:55:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f054ea00640 1 -- 172.21.15.114:0/194505317 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0560051070 con 0x562e0b95c070 2024-09-21T06:55:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 10 client.26111 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 10 client.26111 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:55:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 20 client.26111 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.137+0000 7f0576680740 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b91a9f0 0x562e0bd20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.138+0000 7f0576680740 4 client.26111 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.138+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e0bc7bf00 con 0x562e0b91a9f0 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.138+0000 7f056d000640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b91a9f0 0x562e0bd20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.139+0000 7f056d000640 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b91a9f0 0x562e0bd20690 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4937a6fda48b4f7a server_cookie=bf1a4e043389ac04 in_seq=0 out_seq=0 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.139+0000 7f054ea00640 10 client.26111 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.139+0000 7f054ea00640 1 -- 172.21.15.114:0/194505317 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e0bc7bf00 con 0x562e0b91a9f0 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.139+0000 7f054ea00640 10 client.26111 handle_command_reply: tid=0 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.139+0000 7f0576680740 1 client.26111 shutdown 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 msgr2=0x562e0bd1a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd1a6a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0560009ca0 tx=0x7f056002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 shutdown_connections 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b91a9f0 0x562e0bd20690 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b95c070 0x562e0bd1a6a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b93b390 0x562e0bd0ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 --2- 172.21.15.114:0/194505317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b9331b0 0x562e0bd0e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f054ea00640 0 client.26111 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:55:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 >> 172.21.15.114:0/194505317 conn(0x562e0b950460 msgr2=0x562e0bd05d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 shutdown_connections 2024-09-21T06:55:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 1 -- 172.21.15.114:0/194505317 wait complete. 2024-09-21T06:55:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.140+0000 7f0576680740 20 client.26111 trim_cache size 0 max 16384 2024-09-21T06:55:54.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.154+0000 7f0576680740 1 Processor -- start 2024-09-21T06:55:54.157 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.154+0000 7f0576680740 1 -- start start 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0576680740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd10160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0576680740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd00040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0576680740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0576680740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b3070 con 0x562e0b93b390 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0576680740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b4c70 con 0x562e0b9331b0 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0576680740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b3740 con 0x562e0b95c070 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f056d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd10160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f056d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd10160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35046/0 (socket says 172.21.15.114:35046) 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f056d000640 1 -- 172.21.15.114:0/1018109439 learned_addr learned my addr 172.21.15.114:0/1018109439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f0567e00640 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd00040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.155+0000 7f056da00640 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f056d000640 1 -- 172.21.15.114:0/1018109439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 msgr2=0x562e0bd00580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f056d000640 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd00580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f056d000640 1 -- 172.21.15.114:0/1018109439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 msgr2=0x562e0bd00040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f056d000640 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd00040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f056d000640 1 -- 172.21.15.114:0/1018109439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0bcfacb0 con 0x562e0b9331b0 2024-09-21T06:55:54.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f056d000640 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd10160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f055c005860 tx=0x7f055c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3b522dec337490d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:54.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.156+0000 7f0559600640 1 -- 172.21.15.114:0/1018109439 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c031070 con 0x562e0b9331b0 2024-09-21T06:55:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.157+0000 7f0559600640 1 -- 172.21.15.114:0/1018109439 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055c005590 con 0x562e0b9331b0 2024-09-21T06:55:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.157+0000 7f0559600640 1 -- 172.21.15.114:0/1018109439 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c00fd80 con 0x562e0b9331b0 2024-09-21T06:55:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.157+0000 7f0576680740 1 -- 172.21.15.114:0/1018109439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 msgr2=0x562e0bd10160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.157+0000 7f0576680740 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd10160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f055c005860 tx=0x7f055c0058d0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 -- 172.21.15.114:0/1018109439 shutdown_connections 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd00580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd00040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 --2- 172.21.15.114:0/1018109439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd10160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 -- 172.21.15.114:0/1018109439 >> 172.21.15.114:0/1018109439 conn(0x562e0b950460 msgr2=0x562e0bd0fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 -- 172.21.15.114:0/1018109439 shutdown_connections 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 -- 172.21.15.114:0/1018109439 wait complete. 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.158+0000 7f0576680740 1 -- start start 2024-09-21T06:55:54.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 Processor -- start 2024-09-21T06:55:54.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:53 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1278795818' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:55:54.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:53 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:55:54.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd12f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd13500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b44e0 con 0x562e0b93b390 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b3e10 con 0x562e0b9331b0 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05681b1050 con 0x562e0b95c070 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0576680740 10 client.0 _ll_register_callbacks cb 0x7f0560031f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0567e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd13500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0567e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd13500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33980/0 (socket says 172.21.15.114:33980) 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f0567e00640 1 -- 172.21.15.114:0/2425252726 learned_addr learned my addr 172.21.15.114:0/2425252726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.159+0000 7f056d000640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd12f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0567e00640 1 -- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 msgr2=0x562e0bd84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0567e00640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0567e00640 1 -- 172.21.15.114:0/2425252726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 msgr2=0x562e0bd12f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0567e00640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd12f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0567e00640 1 -- 172.21.15.114:0/2425252726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0bd84700 con 0x562e0b93b390 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f056d000640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd12f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0567e00640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd13500 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f05540a9d70 tx=0x7f05540e56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523abe99d4854176 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f054ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f054ea00640 1 -- 172.21.15.114:0/2425252726 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540d2070 con 0x562e0b93b390 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f054ea00640 1 -- 172.21.15.114:0/2425252726 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0554010b20 con 0x562e0b93b390 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.160+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f055c005590 con 0x562e0b93b390 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f054ea00640 1 -- 172.21.15.114:0/2425252726 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f05540adef0 con 0x562e0b93b390 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 10 client.16038 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562e0bd8baa0 con 0x562e0b93b390 2024-09-21T06:55:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f054ea00640 1 -- 172.21.15.114:0/2425252726 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055c005590 con 0x562e0b93b390 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f054ea00640 1 -- 172.21.15.114:0/2425252726 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f05540ac4c0 con 0x562e0b93b390 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 10 client.16038 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 10 client.16038 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 20 client.16038 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b8efda0 0x562e0bd8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 4 client.16038 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.161+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e0bc779b0 con 0x562e0b8efda0 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.162+0000 7f056d000640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b8efda0 0x562e0bd8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.162+0000 7f054ea00640 10 client.16038 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.162+0000 7f056d000640 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b8efda0 0x562e0bd8e260 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83681759c00bdeb4 server_cookie=309467507bc4c3e6 in_seq=0 out_seq=0 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f054ea00640 1 -- 172.21.15.114:0/2425252726 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e0bc779b0 con 0x562e0b8efda0 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f054ea00640 10 client.16038 handle_command_reply: tid=0 2024-09-21T06:55:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 client.16038 shutdown 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 msgr2=0x562e0bd13500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd13500 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f05540a9d70 tx=0x7f05540e56b0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 shutdown_connections 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x562e0b8efda0 0x562e0bd8e260 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e0b95c070 0x562e0bd84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e0b93b390 0x562e0bd13500 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 --2- 172.21.15.114:0/2425252726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e0b9331b0 0x562e0bd12f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.163+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 >> 172.21.15.114:0/2425252726 conn(0x562e0b950460 msgr2=0x562e0bd03ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.164+0000 7f054ea00640 0 client.16038 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.164+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 shutdown_connections 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.164+0000 7f0576680740 1 -- 172.21.15.114:0/2425252726 wait complete. 2024-09-21T06:55:54.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.164+0000 7f0576680740 20 client.16038 trim_cache size 0 max 16384 2024-09-21T06:55:54.170 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:55:54.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05500b6780 msgr2=0x7f05500b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05500b6780 0x7f05500b8c90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f056002f500 tx=0x7f056002f310 comp rx=0 tx=0).stop 2024-09-21T06:55:54.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 msgr2=0x7f056819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:54.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 0x7f056819aaa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f055400e970 tx=0x7f055400ee30 comp rx=0 tx=0).stop 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 shutdown_connections 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f056810de30 0x7f056819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05500b6780 0x7f05500b8c90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0568105320 0x7f056819aaa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 --2- 172.21.15.114:0/1370482362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05681049d0 0x7f056819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 >> 172.21.15.114:0/1370482362 conn(0x7f0568079740 msgr2=0x7f0568106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 shutdown_connections 2024-09-21T06:55:54.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:54.165+0000 7f056fa00640 1 -- 172.21.15.114:0/1370482362 wait complete. 2024-09-21T06:55:54.179 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:55.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:54 smithi114 ceph-mon[30654]: pgmap v449: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-09-21T06:55:55.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:55:55.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:55:55.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:54 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:55:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:54 smithi183 ceph-mon[34219]: pgmap v449: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-09-21T06:55:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:55:55.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:55:55.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:54 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:55:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:54 smithi193 ceph-mon[34601]: pgmap v449: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-09-21T06:55:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:55:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:55:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:54 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' 2024-09-21T06:55:55.369 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:55:55.960 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:55 smithi114 ceph-mon[30654]: pgmap v450: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.3 MiB/s wr, 464 op/s 2024-09-21T06:55:56.024 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 -- 172.21.15.114:0/1802684779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 msgr2=0x7f8720112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 --2- 172.21.15.114:0/1802684779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f8720112510 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8718009ca0 tx=0x7f871802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 -- 172.21.15.114:0/1802684779 shutdown_connections 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 --2- 172.21.15.114:0/1802684779 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f8720112510 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 --2- 172.21.15.114:0/1802684779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8720076100 0x7f872010fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 --2- 172.21.15.114:0/1802684779 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87200757b0 0x7f8720075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 -- 172.21.15.114:0/1802684779 >> 172.21.15.114:0/1802684779 conn(0x7f87200ff5e0 msgr2=0x7f8720101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:56.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 -- 172.21.15.114:0/1802684779 shutdown_connections 2024-09-21T06:55:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.036+0000 7f8726c00640 1 -- 172.21.15.114:0/1802684779 wait complete. 2024-09-21T06:55:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.037+0000 7f8726c00640 1 Processor -- start 2024-09-21T06:55:56.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.037+0000 7f8726c00640 1 -- start start 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8726c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87200757b0 0x7f872019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8726c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8720076100 0x7f872019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8726c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f87201a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8726c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8720115cd0 con 0x7f87201100d0 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8726c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8720115b50 con 0x7f87200757b0 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8726c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8720115e50 con 0x7f8720076100 2024-09-21T06:55:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f8724c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f87201a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f871fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87200757b0 0x7f872019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f871fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87200757b0 0x7f872019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58310/0 (socket says 172.21.15.114:58310) 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f871fe00640 1 -- 172.21.15.114:0/4119512379 learned_addr learned my addr 172.21.15.114:0/4119512379 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.038+0000 7f871f400640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8720076100 0x7f872019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.039+0000 7f8724c00640 1 -- 172.21.15.114:0/4119512379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8720076100 msgr2=0x7f872019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.039+0000 7f8724c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8720076100 0x7f872019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.039+0000 7f8724c00640 1 -- 172.21.15.114:0/4119512379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87200757b0 msgr2=0x7f872019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.039+0000 7f8724c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87200757b0 0x7f872019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.039+0000 7f8724c00640 1 -- 172.21.15.114:0/4119512379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87201a2590 con 0x7f87201100d0 2024-09-21T06:55:56.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.039+0000 7f8724c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f87201a1e60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8718005e30 tx=0x7f87180317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.040+0000 7f871cc00640 1 -- 172.21.15.114:0/4119512379 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718003b70 con 0x7f87201100d0 2024-09-21T06:55:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.040+0000 7f871cc00640 1 -- 172.21.15.114:0/4119512379 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8718003d10 con 0x7f87201100d0 2024-09-21T06:55:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.040+0000 7f871cc00640 1 -- 172.21.15.114:0/4119512379 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871802f970 con 0x7f87201100d0 2024-09-21T06:55:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.040+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87201a2820 con 0x7f87201100d0 2024-09-21T06:55:56.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.040+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87201a8fe0 con 0x7f87201100d0 2024-09-21T06:55:56.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.044+0000 7f871cc00640 1 -- 172.21.15.114:0/4119512379 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f871802fb10 con 0x7f87201100d0 2024-09-21T06:55:56.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.044+0000 7f872d878740 1 Processor -- start 2024-09-21T06:55:56.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 -- start start 2024-09-21T06:55:56.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8cfa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8cfa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8cfac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569f8c71290 con 0x5569f8932840 2024-09-21T06:55:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569f8c606c0 con 0x5569f8953520 2024-09-21T06:55:56.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f872d878740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569f8c644d0 con 0x5569f892a660 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.045+0000 7f871cc00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87040b6680 0x7f87040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8cfa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871cc00640 1 -- 172.21.15.114:0/4119512379 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8718108ad0 con 0x7f87201100d0 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8cfa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58318/0 (socket says 172.21.15.114:58318) 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 -- 172.21.15.114:0/3356722278 learned_addr learned my addr 172.21.15.114:0/3356722278 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8cfac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871fe00640 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8cfa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f8724c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87040b6680 0x7f87040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 -- 172.21.15.114:0/3356722278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 msgr2=0x5569f8cfa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8cfa700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 -- 172.21.15.114:0/3356722278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 msgr2=0x5569f8cfac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8cfac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871f400640 1 -- 172.21.15.114:0/3356722278 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f8bed460 con 0x5569f8953520 2024-09-21T06:55:56.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.046+0000 7f871fe00640 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8cfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:55:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.047+0000 7f871f400640 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8cfa1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f870c0058c0 tx=0x7f870c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52bf63ca5b64d383 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.047+0000 7f8724c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87040b6680 0x7f87040b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8718105eb0 tx=0x7f87180093d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.047+0000 7f8715200640 1 -- 172.21.15.114:0/3356722278 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c004050 con 0x5569f8953520 2024-09-21T06:55:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.047+0000 7f8715200640 1 -- 172.21.15.114:0/3356722278 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f870c00fc70 con 0x5569f8953520 2024-09-21T06:55:56.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.047+0000 7f8715200640 1 -- 172.21.15.114:0/3356722278 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c003960 con 0x5569f8953520 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 -- 172.21.15.114:0/3356722278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 msgr2=0x5569f8cfa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8cfa1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f870c0058c0 tx=0x7f870c005850 comp rx=0 tx=0).stop 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 -- 172.21.15.114:0/3356722278 shutdown_connections 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8cfac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8cfa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 --2- 172.21.15.114:0/3356722278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8cfa1c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.048+0000 7f872d878740 1 -- 172.21.15.114:0/3356722278 >> 172.21.15.114:0/3356722278 conn(0x5569f8947910 msgr2=0x5569f8cf80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.049+0000 7f872d878740 1 -- 172.21.15.114:0/3356722278 shutdown_connections 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.049+0000 7f872d878740 1 -- 172.21.15.114:0/3356722278 wait complete. 2024-09-21T06:55:56.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.049+0000 7f872d878740 1 -- start start 2024-09-21T06:55:56.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.049+0000 7f872d878740 1 Processor -- start 2024-09-21T06:55:56.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 0x5569f8d05e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8932840 0x5569f8d063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8953520 0x5569f8d11cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569f8ac1bb0 con 0x5569f892a660 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569f8bfb6c0 con 0x5569f8932840 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569f8d089a0 con 0x5569f8953520 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f872d878740 10 client.0 _ll_register_callbacks cb 0x5569f8c16490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f871f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 0x5569f8d05e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f871f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 0x5569f8d05e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36748/0 (socket says 172.21.15.114:36748) 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f871f400640 1 -- 172.21.15.114:0/1737231352 learned_addr learned my addr 172.21.15.114:0/1737231352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.050+0000 7f871fe00640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8932840 0x5569f8d063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8953520 0x5569f8d11cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 -- 172.21.15.114:0/1737231352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8953520 msgr2=0x5569f8d11cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8953520 0x5569f8d11cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 -- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8932840 msgr2=0x5569f8d063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8932840 0x5569f8d063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 -- 172.21.15.114:0/1737231352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f89beb00 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f871f400640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 0x5569f8d05e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f870c004650 tx=0x7f870c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76b68f9dbc7c422 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.051+0000 7f8709600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.052+0000 7f8709600640 1 -- 172.21.15.114:0/1737231352 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c02d060 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.052+0000 7f8709600640 1 -- 172.21.15.114:0/1737231352 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f870c01f5f0 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.052+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f870c00fc70 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.052+0000 7f8709600640 1 -- 172.21.15.114:0/1737231352 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f870c023070 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.052+0000 7f872d878740 10 client.16050 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.052+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569f8a35960 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f8709600640 1 -- 172.21.15.114:0/1737231352 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f870c00fc70 con 0x5569f892a660 2024-09-21T06:55:56.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f8709600640 1 -- 172.21.15.114:0/1737231352 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f870c031820 con 0x5569f892a660 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f872d878740 10 client.16050 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f872d878740 10 client.16050 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f872d878740 20 client.16050 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f872d878740 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f8911ea0 0x5569f8d17c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f872d878740 4 client.16050 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569f8c731e0 con 0x5569f8911ea0 2024-09-21T06:55:56.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.053+0000 7f871fe00640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f8911ea0 0x5569f8d17c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.054+0000 7f8709600640 10 client.16050 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:56.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.054+0000 7f871fe00640 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f8911ea0 0x5569f8d17c60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66dc197bdcab41f3 server_cookie=6f3f2f14fd5d156d in_seq=0 out_seq=0 2024-09-21T06:55:56.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.055+0000 7f8709600640 1 -- 172.21.15.114:0/1737231352 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5569f8c731e0 con 0x5569f8911ea0 2024-09-21T06:55:56.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.055+0000 7f8709600640 10 client.16050 handle_command_reply: tid=0 2024-09-21T06:55:56.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.055+0000 7f872d878740 1 client.16050 shutdown 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.055+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 msgr2=0x5569f8d05e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.055+0000 7f872d878740 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 0x5569f8d05e10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f870c004650 tx=0x7f870c005fd0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 shutdown_connections 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f8911ea0 0x5569f8d17c60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8953520 0x5569f8d11cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8932840 0x5569f8d063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f8709600640 0 client.16050 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 --2- 172.21.15.114:0/1737231352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f892a660 0x5569f8d05e10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 >> 172.21.15.114:0/1737231352 conn(0x5569f8947910 msgr2=0x5569f8cfd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 shutdown_connections 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 1 -- 172.21.15.114:0/1737231352 wait complete. 2024-09-21T06:55:56.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.056+0000 7f872d878740 20 client.16050 trim_cache size 0 max 16384 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.075+0000 7f872d878740 1 Processor -- start 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.075+0000 7f872d878740 1 -- start start 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f872d878740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f892a660 0x5569f8cf8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f872d878740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 0x5569f8d11bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f872d878740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8953520 0x5569f8d12100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f872d878740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87201b3020 con 0x5569f8953520 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f872d878740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87201b4c20 con 0x5569f892a660 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f872d878740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87201b36f0 con 0x5569f8932840 2024-09-21T06:55:56.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f871f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f892a660 0x5569f8cf8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f871fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 0x5569f8d11bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f871fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 0x5569f8d11bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36482/0 (socket says 172.21.15.114:36482) 2024-09-21T06:55:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f871f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f892a660 0x5569f8cf8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58350/0 (socket says 172.21.15.114:58350) 2024-09-21T06:55:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f871fe00640 1 -- 172.21.15.114:0/2911864520 learned_addr learned my addr 172.21.15.114:0/2911864520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:56.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.076+0000 7f871f400640 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8953520 0x5569f8d12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.077+0000 7f871fe00640 1 -- 172.21.15.114:0/2911864520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f892a660 msgr2=0x5569f8cf8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.077+0000 7f871fe00640 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f892a660 0x5569f8cf8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.077+0000 7f871fe00640 1 -- 172.21.15.114:0/2911864520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8953520 msgr2=0x5569f8d12100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.077+0000 7f871fe00640 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8953520 0x5569f8d12100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.077+0000 7f871fe00640 1 -- 172.21.15.114:0/2911864520 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569f8d17fd0 con 0x5569f8932840 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.078+0000 7f871fe00640 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 0x5569f8d11bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x5569f8d181d0 tx=0x7f8710005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8122de202bc0e541 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.078+0000 7f870be00640 1 -- 172.21.15.114:0/2911864520 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871001f4b0 con 0x5569f8932840 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.078+0000 7f870be00640 1 -- 172.21.15.114:0/2911864520 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f871001fb00 con 0x5569f8932840 2024-09-21T06:55:56.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.078+0000 7f870be00640 1 -- 172.21.15.114:0/2911864520 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8710027c00 con 0x5569f8932840 2024-09-21T06:55:56.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 -- 172.21.15.114:0/2911864520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 msgr2=0x5569f8d11bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 0x5569f8d11bc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x5569f8d181d0 tx=0x7f8710005f90 comp rx=0 tx=0).stop 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 -- 172.21.15.114:0/2911864520 shutdown_connections 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8953520 0x5569f8d12100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f8932840 0x5569f8d11bc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 --2- 172.21.15.114:0/2911864520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f892a660 0x5569f8cf8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 -- 172.21.15.114:0/2911864520 >> 172.21.15.114:0/2911864520 conn(0x5569f8947910 msgr2=0x5569f8cf79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 -- 172.21.15.114:0/2911864520 shutdown_connections 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.079+0000 7f872d878740 1 -- 172.21.15.114:0/2911864520 wait complete. 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.080+0000 7f872d878740 1 -- start start 2024-09-21T06:55:56.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.080+0000 7f872d878740 1 Processor -- start 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8d0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8d0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8d08be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87201b4490 con 0x5569f8932840 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87201b3dc0 con 0x5569f8953520 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87201b1000 con 0x5569f892a660 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f872d878740 10 client.0 _ll_register_callbacks cb 0x7f870c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f871f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8d0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f871f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8d0dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36496/0 (socket says 172.21.15.114:36496) 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f871f400640 1 -- 172.21.15.114:0/3866468052 learned_addr learned my addr 172.21.15.114:0/3866468052 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f871fe00640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8d0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.081+0000 7f871f400640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8d08be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f871f400640 1 -- 172.21.15.114:0/3866468052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 msgr2=0x5569f8d0dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f871f400640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8d0dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f871f400640 1 -- 172.21.15.114:0/3866468052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 msgr2=0x5569f8d0e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f871f400640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8d0e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f871f400640 1 -- 172.21.15.114:0/3866468052 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569f8d09120 con 0x5569f8953520 2024-09-21T06:55:56.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f871f400640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8d08be0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f870c013010 tx=0x7f870c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62706385f76c03f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:56.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f8709600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:56.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f8709600640 1 -- 172.21.15.114:0/3866468052 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c042060 con 0x5569f8953520 2024-09-21T06:55:56.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f871001fb00 con 0x5569f8953520 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.082+0000 7f8709600640 1 -- 172.21.15.114:0/3866468052 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f870c011230 con 0x5569f8953520 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f8709600640 1 -- 172.21.15.114:0/3866468052 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f870c049050 con 0x5569f8953520 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 10 client.26135 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569f8d82c00 con 0x5569f8953520 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f8709600640 1 -- 172.21.15.114:0/3866468052 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f871001fb00 con 0x5569f8953520 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f8709600640 1 -- 172.21.15.114:0/3866468052 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f870c0318c0 con 0x5569f8953520 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 10 client.26135 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 10 client.26135 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 20 client.26135 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f88e7250 0x5569f8d85280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 4 client.26135 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "745fa756", "release": true, "await": true}] 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.083+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "745fa756", "release": true, "await": true}) -- 0x5569f8c6ec90 con 0x5569f88e7250 2024-09-21T06:55:56.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.084+0000 7f871fe00640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f88e7250 0x5569f8d85280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:56.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.084+0000 7f871fe00640 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f88e7250 0x5569f8d85280 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c21ae74be3b474 server_cookie=ce0e3ac5d833b316 in_seq=0 out_seq=0 2024-09-21T06:55:56.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.084+0000 7f8709600640 10 client.26135 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:56.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.108+0000 7f8709600640 1 -- 172.21.15.114:0/3866468052 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5569f8c6ec90 con 0x5569f88e7250 2024-09-21T06:55:56.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.108+0000 7f8709600640 10 client.26135 handle_command_reply: tid=0 2024-09-21T06:55:56.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.108+0000 7f872d878740 1 client.26135 shutdown 2024-09-21T06:55:56.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:55 smithi183 ceph-mon[34219]: pgmap v450: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.3 MiB/s wr, 464 op/s 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 msgr2=0x5569f8d08be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8d08be0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f870c013010 tx=0x7f870c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 shutdown_connections 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569f8953520 0x5569f8d08be0 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f870c013010 tx=0x7f870c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569f8932840 0x5569f8d0e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5569f88e7250 0x5569f8d85280 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f8709600640 0 client.26135 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 --2- 172.21.15.114:0/3866468052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5569f892a660 0x5569f8d0dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.111+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 >> 172.21.15.114:0/3866468052 conn(0x5569f8947910 msgr2=0x5569f8cf6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:56.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.114+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 shutdown_connections 2024-09-21T06:55:56.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.114+0000 7f872d878740 1 -- 172.21.15.114:0/3866468052 wait complete. 2024-09-21T06:55:56.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.115+0000 7f872d878740 20 client.26135 trim_cache size 0 max 16384 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 32, 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 8, 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "745fa756": { 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 8, 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:55:56.119 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 137.0, 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:56.120 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.116+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87040b6680 msgr2=0x7f87040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.116+0000 7f8726c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87040b6680 0x7f87040b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8718105eb0 tx=0x7f87180093d0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.116+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 msgr2=0x7f87201a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.116+0000 7f8726c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f87201a1e60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8718005e30 tx=0x7f87180317c0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 shutdown_connections 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87201100d0 0x7f87201a1e60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8720076100 0x7f872019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87040b6680 0x7f87040b8b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 --2- 172.21.15.114:0/4119512379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87200757b0 0x7f872019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 >> 172.21.15.114:0/4119512379 conn(0x7f87200ff5e0 msgr2=0x7f8720107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.117+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 shutdown_connections 2024-09-21T06:55:56.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:56.118+0000 7f8726c00640 1 -- 172.21.15.114:0/4119512379 wait complete. 2024-09-21T06:55:56.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:55 smithi193 ceph-mon[34601]: pgmap v450: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.3 MiB/s wr, 464 op/s 2024-09-21T06:55:57.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:55:57.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:55:57.058 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:56 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:55:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:55:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:55:57.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:56 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:55:57.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T06:55:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T06:55:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:56 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T06:55:57.203 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '745fa756', total seconds elapsed: 23.9 2024-09-21T06:55:57.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.053+0000 7f6bc5c00640 1 -- 172.21.15.114:0/3811086251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 msgr2=0x7f6bc010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.053+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/3811086251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 0x7f6bc010d850 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009900 tx=0x7f6bac02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.056+0000 7f6bc5c00640 1 -- 172.21.15.114:0/3811086251 shutdown_connections 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.056+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/3811086251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc010de10 0x7f6bc0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.056+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/3811086251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 0x7f6bc010d850 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.056+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/3811086251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bc01049b0 0x7f6bc0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.056+0000 7f6bc5c00640 1 -- 172.21.15.114:0/3811086251 >> 172.21.15.114:0/3811086251 conn(0x7f6bc0079730 msgr2=0x7f6bc0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.058+0000 7f6bc5c00640 1 -- 172.21.15.114:0/3811086251 shutdown_connections 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.058+0000 7f6bc5c00640 1 -- 172.21.15.114:0/3811086251 wait complete. 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.061+0000 7f6bc5c00640 1 Processor -- start 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.062+0000 7f6bc5c00640 1 -- start start 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.064+0000 7f6bc5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 0x7f6bc019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.065+0000 7f6bc5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 0x7f6bc019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.065+0000 7f6bc5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bc010de10 0x7f6bc019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.065+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 0x7f6bc019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bc5c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0113ac0 con 0x7f6bc010de10 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bc5c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0113940 con 0x7f6bc01049b0 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bc5c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0113c40 con 0x7f6bc0105300 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbe000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 0x7f6bc019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 0x7f6bc019a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58380/0 (socket says 172.21.15.114:58380) 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbea00640 1 -- 172.21.15.114:0/2337977547 learned_addr learned my addr 172.21.15.114:0/2337977547 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbf400640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bc010de10 0x7f6bc019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbea00640 1 -- 172.21.15.114:0/2337977547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 msgr2=0x7f6bc019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbea00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 0x7f6bc019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbea00640 1 -- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bc010de10 msgr2=0x7f6bc019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.066+0000 7f6bbea00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bc010de10 0x7f6bc019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.067+0000 7f6bbea00640 1 -- 172.21.15.114:0/2337977547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc01a0360 con 0x7f6bc01049b0 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.067+0000 7f6bbea00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 0x7f6bc019a4b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c970 tx=0x7f6bb000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:57.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.067+0000 7f6bb7400640 1 -- 172.21.15.114:0/2337977547 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0007c10 con 0x7f6bc01049b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.068+0000 7f6bb7400640 1 -- 172.21.15.114:0/2337977547 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb0007db0 con 0x7f6bc01049b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.068+0000 7f6bb7400640 1 -- 172.21.15.114:0/2337977547 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb000f810 con 0x7f6bc01049b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.068+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc01a0650 con 0x7f6bc01049b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.068+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc01a0b10 con 0x7f6bc01049b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.075+0000 7f6bcc804740 1 Processor -- start 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.075+0000 7f6bcc804740 1 -- start start 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f92070 0x562068338da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x5620683392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x562068339820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bcc804740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5620682affb0 con 0x562067f92070 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bcc804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56206829f3e0 con 0x562067f71390 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bcc804740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5620682a31f0 con 0x562067f691b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bb7400640 1 -- 172.21.15.114:0/2337977547 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6bb0018a50 con 0x7f6bc01049b0 2024-09-21T06:55:57.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.076+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x5620683392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x5620683392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36522/0 (socket says 172.21.15.114:36522) 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 -- 172.21.15.114:0/4289499619 learned_addr learned my addr 172.21.15.114:0/4289499619 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbe000640 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x562068339820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 -- 172.21.15.114:0/4289499619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 msgr2=0x562068339820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x562068339820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 -- 172.21.15.114:0/4289499619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f92070 msgr2=0x562068338da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f92070 0x562068338da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.077+0000 7f6bbea00640 1 -- 172.21.15.114:0/4289499619 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562068269cd0 con 0x562067f691b0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.078+0000 7f6bbea00640 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x5620683392e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00e53c0 tx=0x7f6bb00f0570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15eeb8a97f8075a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.078+0000 7f6babe00640 1 -- 172.21.15.114:0/4289499619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ea310 con 0x562067f691b0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.078+0000 7f6babe00640 1 -- 172.21.15.114:0/4289499619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb00eb500 con 0x562067f691b0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.078+0000 7f6babe00640 1 -- 172.21.15.114:0/4289499619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00e8790 con 0x562067f691b0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.080+0000 7f6bb7400640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ba40b68c0 0x7f6ba40b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.080+0000 7f6bb7400640 1 -- 172.21.15.114:0/2337977547 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6bb00e8100 con 0x7f6bc01049b0 2024-09-21T06:55:57.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.080+0000 7f6bbf400640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ba40b68c0 0x7f6ba40b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.083+0000 7f6bbf400640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ba40b68c0 0x7f6ba40b8dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8005c40 tx=0x7f6bb8005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.083+0000 7f6bcc804740 1 -- 172.21.15.114:0/4289499619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 msgr2=0x5620683392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.083+0000 7f6bcc804740 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x5620683392e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00e53c0 tx=0x7f6bb00f0570 comp rx=0 tx=0).stop 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.087+0000 7f6bcc804740 1 -- 172.21.15.114:0/4289499619 shutdown_connections 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.087+0000 7f6bcc804740 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x562068339820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.088+0000 7f6bcc804740 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x5620683392e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.088+0000 7f6bcc804740 1 --2- 172.21.15.114:0/4289499619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f92070 0x562068338da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.088+0000 7f6bcc804740 1 -- 172.21.15.114:0/4289499619 >> 172.21.15.114:0/4289499619 conn(0x562067f86460 msgr2=0x562068336c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.088+0000 7f6bcc804740 1 -- 172.21.15.114:0/4289499619 shutdown_connections 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.088+0000 7f6bcc804740 1 -- 172.21.15.114:0/4289499619 wait complete. 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.089+0000 7f6bcc804740 1 -- start start 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.089+0000 7f6bcc804740 1 Processor -- start 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x562068344640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068344c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f92070 0x5620683618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562068100700 con 0x562067f71390 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56206823a580 con 0x562067f92070 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562068358590 con 0x562067f691b0 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x562068344640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bcc804740 10 client.0 _ll_register_callbacks cb 0x56206824a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x562068344640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36530/0 (socket says 172.21.15.114:36530) 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.090+0000 7f6bbea00640 1 -- 172.21.15.114:0/3788404835 learned_addr learned my addr 172.21.15.114:0/3788404835 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbea00640 1 -- 172.21.15.114:0/3788404835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f92070 msgr2=0x5620683618b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbf400640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f92070 0x5620683618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbea00640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f92070 0x5620683618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbea00640 1 -- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 msgr2=0x562068344c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbea00640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068344c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbea00640 1 -- 172.21.15.114:0/3788404835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562067ffd650 con 0x562067f691b0 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbf400640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f92070 0x5620683618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:57.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6bbea00640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x562068344640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00adba0 tx=0x7f6bb000a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ed58a820fdd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.091+0000 7f6ba8c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.092+0000 7f6ba8c00640 1 -- 172.21.15.114:0/3788404835 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00aa260 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.092+0000 7f6ba8c00640 1 -- 172.21.15.114:0/3788404835 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6bb00a9e80 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.092+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56206823cb90 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.092+0000 7f6ba8c00640 1 -- 172.21.15.114:0/3788404835 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6bb00a9380 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.092+0000 7f6bcc804740 10 client.26113 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.092+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562068257950 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6ba8c00640 1 -- 172.21.15.114:0/3788404835 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56206823cb90 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6ba8c00640 1 -- 172.21.15.114:0/3788404835 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6bb0020070 con 0x562067f691b0 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6bcc804740 10 client.26113 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6bcc804740 10 client.26113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6bcc804740 20 client.26113 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6bcc804740 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f509f0 0x5620683675f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6bcc804740 4 client.26113 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.093+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620682b1f00 con 0x562067f509f0 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.095+0000 7f6bbe000640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f509f0 0x5620683675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.096+0000 7f6bbe000640 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f509f0 0x5620683675f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a118c981c04a365e server_cookie=c33d7655e3554df8 in_seq=0 out_seq=0 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.096+0000 7f6ba8c00640 10 client.26113 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.100+0000 7f6ba8c00640 1 -- 172.21.15.114:0/3788404835 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620682b1f00 con 0x562067f509f0 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.100+0000 7f6ba8c00640 10 client.26113 handle_command_reply: tid=0 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.100+0000 7f6bcc804740 1 client.26113 shutdown 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 msgr2=0x562068344640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x562068344640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00adba0 tx=0x7f6bb000a420 comp rx=0 tx=0).stop 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 shutdown_connections 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f509f0 0x5620683675f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f92070 0x5620683618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068344c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 --2- 172.21.15.114:0/3788404835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f691b0 0x562068344640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.101+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 >> 172.21.15.114:0/3788404835 conn(0x562067f86460 msgr2=0x56206833bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.102+0000 7f6ba8c00640 0 client.26113 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.102+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 shutdown_connections 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.102+0000 7f6bcc804740 1 -- 172.21.15.114:0/3788404835 wait complete. 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.103+0000 7f6bcc804740 20 client.26113 trim_cache size 0 max 16384 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.130+0000 7f6bcc804740 1 Processor -- start 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.130+0000 7f6bcc804740 1 -- start start 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 0x56206833c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068361820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068361d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bcc804740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01aec30 con 0x562067f71390 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bcc804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b0830 con 0x562067f691b0 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bcc804740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01af300 con 0x562067f92070 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 0x56206833c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 0x56206833c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58422/0 (socket says 172.21.15.114:58422) 2024-09-21T06:55:57.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.131+0000 7f6bbea00640 1 -- 172.21.15.114:0/1382417157 learned_addr learned my addr 172.21.15.114:0/1382417157 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbe000640 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068361820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbea00640 1 -- 172.21.15.114:0/1382417157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 msgr2=0x562068361d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbea00640 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068361d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbea00640 1 -- 172.21.15.114:0/1382417157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 msgr2=0x562068361820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbea00640 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068361820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbea00640 1 -- 172.21.15.114:0/1382417157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562067f50c20 con 0x562067f691b0 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbe000640 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068361820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bbea00640 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 0x56206833c5b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00ebf20 tx=0x7f6bb0018df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a1db97522a1657 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bab400640 1 -- 172.21.15.114:0/1382417157 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ad380 con 0x562067f691b0 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bab400640 1 -- 172.21.15.114:0/1382417157 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb00b7110 con 0x562067f691b0 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.132+0000 7f6bab400640 1 -- 172.21.15.114:0/1382417157 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb00ad560 con 0x562067f691b0 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.133+0000 7f6bcc804740 1 -- 172.21.15.114:0/1382417157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 msgr2=0x56206833c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 0x56206833c5b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00ebf20 tx=0x7f6bb0018df0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 -- 172.21.15.114:0/1382417157 shutdown_connections 2024-09-21T06:55:57.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068361d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f71390 0x562068361820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 --2- 172.21.15.114:0/1382417157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f691b0 0x56206833c5b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 -- 172.21.15.114:0/1382417157 >> 172.21.15.114:0/1382417157 conn(0x562067f86460 msgr2=0x56206833bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 -- 172.21.15.114:0/1382417157 shutdown_connections 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 -- 172.21.15.114:0/1382417157 wait complete. 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.134+0000 7f6bcc804740 1 -- start start 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 Processor -- start 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f691b0 0x5620683ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x5620683cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068368a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01b00a0 con 0x562067f691b0 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01af9d0 con 0x562067f71390 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc01acc10 con 0x562067f92070 2024-09-21T06:55:57.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.135+0000 7f6bcc804740 10 client.0 _ll_register_callbacks cb 0x7f6bb00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.136+0000 7f6bbe000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x5620683cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.136+0000 7f6bbe000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x5620683cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58430/0 (socket says 172.21.15.114:58430) 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.136+0000 7f6bbf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068368a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.136+0000 7f6bbf400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068368a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36544/0 (socket says 172.21.15.114:36544) 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.136+0000 7f6bbf400640 1 -- 172.21.15.114:0/2635679037 learned_addr learned my addr 172.21.15.114:0/2635679037 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6bbe000640 1 -- 172.21.15.114:0/2635679037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 msgr2=0x562068368a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6bbe000640 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068368a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6bbe000640 1 -- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f691b0 msgr2=0x5620683ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6bbe000640 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f691b0 0x5620683ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6bbe000640 1 -- 172.21.15.114:0/2635679037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562068368fa0 con 0x562067f71390 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6bbe000640 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x5620683cd310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009230 tx=0x7f6bac02ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad602bdf5442cc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6ba8c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.137+0000 7f6ba8c00640 1 -- 172.21.15.114:0/2635679037 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac035040 con 0x562067f71390 2024-09-21T06:55:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.138+0000 7f6ba8c00640 1 -- 172.21.15.114:0/2635679037 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6bac005450 con 0x562067f71390 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.139+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb00b7110 con 0x562067f71390 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.139+0000 7f6ba8c00640 1 -- 172.21.15.114:0/2635679037 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6bac005650 con 0x562067f71390 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.139+0000 7f6bcc804740 10 client.26150 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.139+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5620683e33e0 con 0x562067f71390 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6ba8c00640 1 -- 172.21.15.114:0/2635679037 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb00b7110 con 0x562067f71390 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6ba8c00640 1 -- 172.21.15.114:0/2635679037 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6bac0039d0 con 0x562067f71390 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6bcc804740 10 client.26150 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6bcc804740 10 client.26150 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6bcc804740 20 client.26150 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6bcc804740 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f25da0 0x5620683e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6bcc804740 4 client.26150 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.140+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5620682ad9b0 con 0x562067f25da0 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.141+0000 7f6bbea00640 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f25da0 0x5620683e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.145+0000 7f6bbea00640 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f25da0 0x5620683e5b80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=636de890fbc91b26 server_cookie=e8d5e2f940fcc7f1 in_seq=0 out_seq=0 2024-09-21T06:55:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.145+0000 7f6ba8c00640 10 client.26150 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.147+0000 7f6ba8c00640 1 -- 172.21.15.114:0/2635679037 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5620682ad9b0 con 0x562067f25da0 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.147+0000 7f6ba8c00640 10 client.26150 handle_command_reply: tid=0 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.147+0000 7f6bcc804740 1 client.26150 shutdown 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 msgr2=0x5620683cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x5620683cd310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009230 tx=0x7f6bac02ab40 comp rx=0 tx=0).stop 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 shutdown_connections 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x562067f25da0 0x5620683e5b80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562067f92070 0x562068368a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562067f71390 0x5620683cd310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 --2- 172.21.15.114:0/2635679037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562067f691b0 0x5620683ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 >> 172.21.15.114:0/2635679037 conn(0x562067f86460 msgr2=0x562068336c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6ba8c00640 0 client.26150 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:55:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 shutdown_connections 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 1 -- 172.21.15.114:0/2635679037 wait complete. 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.148+0000 7f6bcc804740 20 client.26150 trim_cache size 0 max 16384 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ba40b68c0 msgr2=0x7f6ba40b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ba40b68c0 0x7f6ba40b8dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8005c40 tx=0x7f6bb8005bd0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 msgr2=0x7f6bc019a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 0x7f6bc019a4b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6bb000c970 tx=0x7f6bb000ce30 comp rx=0 tx=0).stop 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 shutdown_connections 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6bc010de10 0x7f6bc019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bc0105300 0x7f6bc019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ba40b68c0 0x7f6ba40b8dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 --2- 172.21.15.114:0/2337977547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bc01049b0 0x7f6bc019a4b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 >> 172.21.15.114:0/2337977547 conn(0x7f6bc0079730 msgr2=0x7f6bc0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 shutdown_connections 2024-09-21T06:55:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:57.150+0000 7f6bc5c00640 1 -- 172.21.15.114:0/2337977547 wait complete. 2024-09-21T06:55:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 80.0 s, 733.6 tps, lat 322.323 ms stddev 1651.681 2024-09-21T06:55:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:55:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:55:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:55:58.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:55:57 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:55:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:55:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:55:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:55:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:55:57 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:55:58.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:55:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:55:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:55:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:55:57 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:55:58.181 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:55:58.864 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.713+0000 7f448ce00640 1 -- 172.21.15.114:0/3389053664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488077940 msgr2=0x7f4488077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.713+0000 7f448ce00640 1 --2- 172.21.15.114:0/3389053664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488077940 0x7f4488077d50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4480009be0 tx=0x7f448002a6c0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 -- 172.21.15.114:0/3389053664 shutdown_connections 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 --2- 172.21.15.114:0/3389053664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4488114770 0x7f4488116b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 --2- 172.21.15.114:0/3389053664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488078290 0x7f4488114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 --2- 172.21.15.114:0/3389053664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488077940 0x7f4488077d50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 -- 172.21.15.114:0/3389053664 >> 172.21.15.114:0/3389053664 conn(0x7f44881017c0 msgr2=0x7f4488103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:59.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 -- 172.21.15.114:0/3389053664 shutdown_connections 2024-09-21T06:55:59.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.714+0000 7f448ce00640 1 -- 172.21.15.114:0/3389053664 wait complete. 2024-09-21T06:55:59.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.716+0000 7f448ce00640 1 Processor -- start 2024-09-21T06:55:59.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.716+0000 7f448ce00640 1 -- start start 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f448ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488077940 0x7f44881a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f448ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4488078290 0x7f44881a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f448ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 0x7f44881a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f448ce00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f448811a100 con 0x7f4488077940 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f448ce00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4488119f80 con 0x7f4488114770 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f448ce00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f448811a280 con 0x7f4488078290 2024-09-21T06:55:59.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f4486800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 0x7f44881a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f4485400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4488078290 0x7f44881a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f4486800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 0x7f44881a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58446/0 (socket says 172.21.15.114:58446) 2024-09-21T06:55:59.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.717+0000 7f4486800640 1 -- 172.21.15.114:0/3030785212 learned_addr learned my addr 172.21.15.114:0/3030785212 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:59.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4486800640 1 -- 172.21.15.114:0/3030785212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4488078290 msgr2=0x7f44881a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4486800640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4488078290 0x7f44881a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4485e00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488077940 0x7f44881a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4486800640 1 -- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488077940 msgr2=0x7f44881a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4486800640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488077940 0x7f44881a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4486800640 1 -- 172.21.15.114:0/3030785212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44881a6a20 con 0x7f4488114770 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.718+0000 7f4485e00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488077940 0x7f44881a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.719+0000 7f4486800640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 0x7f44881a62f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f447800d750 tx=0x7f447800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.719+0000 7f447ea00640 1 -- 172.21.15.114:0/3030785212 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478017070 con 0x7f4488114770 2024-09-21T06:55:59.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.720+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44881a6d10 con 0x7f4488114770 2024-09-21T06:55:59.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.720+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44881ad470 con 0x7f4488114770 2024-09-21T06:55:59.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.720+0000 7f447ea00640 1 -- 172.21.15.114:0/3030785212 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44780054d0 con 0x7f4488114770 2024-09-21T06:55:59.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.720+0000 7f447ea00640 1 -- 172.21.15.114:0/3030785212 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478017070 con 0x7f4488114770 2024-09-21T06:55:59.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.726+0000 7f447ea00640 1 -- 172.21.15.114:0/3030785212 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f44780049c0 con 0x7f4488114770 2024-09-21T06:55:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.727+0000 7f4493a3a740 1 Processor -- start 2024-09-21T06:55:59.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.727+0000 7f4493a3a740 1 -- start start 2024-09-21T06:55:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.728+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9b9dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.728+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9b9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.728+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9b9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.728+0000 7f4493a3a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bdf9b14fb0 con 0x55bdf97ce1b0 2024-09-21T06:55:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.728+0000 7f4493a3a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdf9b043e0 con 0x55bdf97d6390 2024-09-21T06:55:59.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.728+0000 7f4493a3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bdf9b081f0 con 0x55bdf97f7070 2024-09-21T06:55:59.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.731+0000 7f447ea00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f446c0b6900 0x7f446c0b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.731+0000 7f4485e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9b9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.731+0000 7f4485e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9b9e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36868/0 (socket says 172.21.15.114:36868) 2024-09-21T06:55:59.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.731+0000 7f4485e00640 1 -- 172.21.15.114:0/2995459685 learned_addr learned my addr 172.21.15.114:0/2995459685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.731+0000 7f4486800640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f446c0b6900 0x7f446c0b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.732+0000 7f447ea00640 1 -- 172.21.15.114:0/3030785212 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f44780e7eb0 con 0x7f4488114770 2024-09-21T06:55:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.732+0000 7f4485400640 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9b9dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.732+0000 7f4485400640 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9b9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.733+0000 7f4485e00640 1 -- 172.21.15.114:0/2995459685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 msgr2=0x55bdf9b9dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.733+0000 7f4485e00640 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9b9dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.733+0000 7f4485e00640 1 -- 172.21.15.114:0/2995459685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 msgr2=0x55bdf9b9e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.733+0000 7f4486800640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f446c0b6900 0x7f446c0b8e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f44780aac00 tx=0x7f4478000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.733+0000 7f4485e00640 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9b9e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.733+0000 7f4485e00640 1 -- 172.21.15.114:0/2995459685 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf9acecd0 con 0x55bdf97ce1b0 2024-09-21T06:55:59.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.734+0000 7f4485e00640 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9b9e0e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4480034040 tx=0x7f4480030630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77653dc6d1e2733d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.734+0000 7f4473400640 1 -- 172.21.15.114:0/2995459685 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480041020 con 0x55bdf97ce1b0 2024-09-21T06:55:59.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.735+0000 7f4473400640 1 -- 172.21.15.114:0/2995459685 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4480037040 con 0x55bdf97ce1b0 2024-09-21T06:55:59.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.735+0000 7f4473400640 1 -- 172.21.15.114:0/2995459685 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480041020 con 0x55bdf97ce1b0 2024-09-21T06:55:59.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.737+0000 7f4493a3a740 1 -- 172.21.15.114:0/2995459685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 msgr2=0x55bdf9b9e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.737+0000 7f4493a3a740 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9b9e0e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4480034040 tx=0x7f4480030630 comp rx=0 tx=0).stop 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 -- 172.21.15.114:0/2995459685 shutdown_connections 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9b9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9b9e0e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 --2- 172.21.15.114:0/2995459685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9b9dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 -- 172.21.15.114:0/2995459685 >> 172.21.15.114:0/2995459685 conn(0x55bdf97eb460 msgr2=0x55bdf9b9ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 -- 172.21.15.114:0/2995459685 shutdown_connections 2024-09-21T06:55:59.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.738+0000 7f4493a3a740 1 -- 172.21.15.114:0/2995459685 wait complete. 2024-09-21T06:55:59.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.740+0000 7f4493a3a740 1 -- start start 2024-09-21T06:55:59.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.740+0000 7f4493a3a740 1 Processor -- start 2024-09-21T06:55:59.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9bb56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bdf9965700 con 0x55bdf97ce1b0 2024-09-21T06:55:59.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdf9a9f580 con 0x55bdf97d6390 2024-09-21T06:55:59.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bdf9bac320 con 0x55bdf97f7070 2024-09-21T06:55:59.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4493a3a740 10 client.0 _ll_register_callbacks cb 0x55bdf9aaf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:59.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4485e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4485e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58450/0 (socket says 172.21.15.114:58450) 2024-09-21T06:55:59.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.742+0000 7f4485e00640 1 -- 172.21.15.114:0/4070441651 learned_addr learned my addr 172.21.15.114:0/4070441651 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:59.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.743+0000 7f4485e00640 1 -- 172.21.15.114:0/4070441651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 msgr2=0x55bdf9bb56a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.743+0000 7f4485e00640 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9bb56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.743+0000 7f4485e00640 1 -- 172.21.15.114:0/4070441651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 msgr2=0x55bdf9ba96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.743+0000 7f4485400640 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.743+0000 7f4485e00640 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.743+0000 7f4485e00640 1 -- 172.21.15.114:0/4070441651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf9862650 con 0x55bdf97d6390 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.744+0000 7f4485e00640 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba9c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4480009be0 tx=0x7f4480030850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=949366bd60310f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.744+0000 7f4467e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.744+0000 7f4467e00640 1 -- 172.21.15.114:0/4070441651 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480041020 con 0x55bdf97d6390 2024-09-21T06:55:59.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.744+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bdf9aa1b90 con 0x55bdf97d6390 2024-09-21T06:55:59.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.744+0000 7f4467e00640 1 -- 172.21.15.114:0/4070441651 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4480030c40 con 0x55bdf97d6390 2024-09-21T06:55:59.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.745+0000 7f4467e00640 1 -- 172.21.15.114:0/4070441651 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4480059050 con 0x55bdf97d6390 2024-09-21T06:55:59.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.745+0000 7f4493a3a740 10 client.26162 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:59.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.745+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bdf9abc950 con 0x55bdf97d6390 2024-09-21T06:55:59.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.745+0000 7f4467e00640 1 -- 172.21.15.114:0/4070441651 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bdf9aa1b90 con 0x55bdf97d6390 2024-09-21T06:55:59.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.747+0000 7f4467e00640 1 -- 172.21.15.114:0/4070441651 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f448003d5a0 con 0x55bdf97d6390 2024-09-21T06:55:59.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4493a3a740 10 client.26162 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4493a3a740 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4493a3a740 20 client.26162 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4493a3a740 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf97b59f0 0x55bdf9bbb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4493a3a740 4 client.26162 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdf9b16f00 con 0x55bdf97b59f0 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.748+0000 7f4485400640 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf97b59f0 0x55bdf9bbb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.749+0000 7f4467e00640 10 client.26162 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:59.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.749+0000 7f4485400640 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf97b59f0 0x55bdf9bbb480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7deab3857fb6fe7e server_cookie=bc5f94bf5cc1cc45 in_seq=0 out_seq=0 2024-09-21T06:55:59.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.750+0000 7f4467e00640 1 -- 172.21.15.114:0/4070441651 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bdf9b16f00 con 0x55bdf97b59f0 2024-09-21T06:55:59.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.750+0000 7f4467e00640 10 client.26162 handle_command_reply: tid=0 2024-09-21T06:55:59.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.750+0000 7f4493a3a740 1 client.26162 shutdown 2024-09-21T06:55:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 msgr2=0x55bdf9ba9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba9c20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4480009be0 tx=0x7f4480030850 comp rx=0 tx=0).stop 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 shutdown_connections 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf97b59f0 0x55bdf9bbb480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97f7070 0x55bdf9bb56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba9c20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 --2- 172.21.15.114:0/4070441651 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 >> 172.21.15.114:0/4070441651 conn(0x55bdf97eb460 msgr2=0x55bdf9ba0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4467e00640 0 client.26162 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 shutdown_connections 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 1 -- 172.21.15.114:0/4070441651 wait complete. 2024-09-21T06:55:59.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.752+0000 7f4493a3a740 20 client.26162 trim_cache size 0 max 16384 2024-09-21T06:55:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.781+0000 7f4493a3a740 1 Processor -- start 2024-09-21T06:55:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.781+0000 7f4493a3a740 1 -- start start 2024-09-21T06:55:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9ba2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4493a3a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f446800fed0 con 0x55bdf97f7070 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4493a3a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4468011ad0 con 0x55bdf97d6390 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4493a3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f44680105a0 con 0x55bdf97ce1b0 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4485e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.782+0000 7f4485400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.783+0000 7f4485400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36594/0 (socket says 172.21.15.114:36594) 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.783+0000 7f4485400640 1 -- 172.21.15.114:0/1864615933 learned_addr learned my addr 172.21.15.114:0/1864615933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:59.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.783+0000 7f4485400640 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9ba2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.784+0000 7f4485400640 1 -- 172.21.15.114:0/1864615933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 msgr2=0x55bdf9ba82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.784+0000 7f4485400640 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.784+0000 7f4485400640 1 -- 172.21.15.114:0/1864615933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 msgr2=0x55bdf9ba2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.784+0000 7f4485400640 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9ba2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.784+0000 7f4485400640 1 -- 172.21.15.114:0/1864615933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdf9ba8d00 con 0x55bdf97ce1b0 2024-09-21T06:55:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.785+0000 7f4485400640 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba7d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4474009d30 tx=0x7f44740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e03a9cfecf5dcbf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:59.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.786+0000 7f4472a00640 1 -- 172.21.15.114:0/1864615933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474011020 con 0x55bdf97ce1b0 2024-09-21T06:55:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.786+0000 7f4472a00640 1 -- 172.21.15.114:0/1864615933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4474011690 con 0x55bdf97ce1b0 2024-09-21T06:55:59.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.786+0000 7f4472a00640 1 -- 172.21.15.114:0/1864615933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44740035f0 con 0x55bdf97ce1b0 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.787+0000 7f4493a3a740 1 -- 172.21.15.114:0/1864615933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 msgr2=0x55bdf9ba7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.787+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba7d60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4474009d30 tx=0x7f44740046c0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 -- 172.21.15.114:0/1864615933 shutdown_connections 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9ba2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9ba82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1864615933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9ba7d60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 -- 172.21.15.114:0/1864615933 >> 172.21.15.114:0/1864615933 conn(0x55bdf97eb460 msgr2=0x55bdf9ba4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:59.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 -- 172.21.15.114:0/1864615933 shutdown_connections 2024-09-21T06:55:59.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.789+0000 7f4493a3a740 1 -- 172.21.15.114:0/1864615933 wait complete. 2024-09-21T06:55:59.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.790+0000 7f4493a3a740 1 -- start start 2024-09-21T06:55:59.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.790+0000 7f4493a3a740 1 Processor -- start 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9bb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9bb1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9bac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4468011340 con 0x55bdf97f7070 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4468010c70 con 0x55bdf97d6390 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f446800deb0 con 0x55bdf97ce1b0 2024-09-21T06:55:59.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.791+0000 7f4493a3a740 10 client.0 _ll_register_callbacks cb 0x7f44800303a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:55:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.792+0000 7f4485400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9bb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.792+0000 7f4485e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9bb1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.793+0000 7f4485400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9bac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.793+0000 7f4485e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9bb1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58472/0 (socket says 172.21.15.114:58472) 2024-09-21T06:55:59.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.793+0000 7f4485e00640 1 -- 172.21.15.114:0/1053078003 learned_addr learned my addr 172.21.15.114:0/1053078003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:55:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4485e00640 1 -- 172.21.15.114:0/1053078003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 msgr2=0x55bdf9bb1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4485e00640 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9bb1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4485e00640 1 -- 172.21.15.114:0/1053078003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 msgr2=0x55bdf9bac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:55:59.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4485e00640 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9bac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4485e00640 1 -- 172.21.15.114:0/1053078003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdf9bb2090 con 0x55bdf97d6390 2024-09-21T06:55:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4485e00640 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9bb1b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f448002abe0 tx=0x7f4480009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db6d41ae564ea00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:55:59.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.796+0000 7f4467e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:55:59.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.797+0000 7f4467e00640 1 -- 172.21.15.114:0/1053078003 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480062060 con 0x55bdf97d6390 2024-09-21T06:55:59.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.797+0000 7f4467e00640 1 -- 172.21.15.114:0/1053078003 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f448003db90 con 0x55bdf97d6390 2024-09-21T06:55:59.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.798+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4474011690 con 0x55bdf97d6390 2024-09-21T06:55:59.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.798+0000 7f4467e00640 1 -- 172.21.15.114:0/1053078003 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4480041050 con 0x55bdf97d6390 2024-09-21T06:55:59.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.799+0000 7f4493a3a740 10 client.26168 fetch_fsmap learned FSMap version 32 2024-09-21T06:55:59.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.799+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bdf9c25ec0 con 0x55bdf97d6390 2024-09-21T06:55:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.799+0000 7f4467e00640 1 -- 172.21.15.114:0/1053078003 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4474011690 con 0x55bdf97d6390 2024-09-21T06:55:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4467e00640 1 -- 172.21.15.114:0/1053078003 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4480050740 con 0x55bdf97d6390 2024-09-21T06:55:59.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4493a3a740 10 client.26168 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:55:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4493a3a740 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:55:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4493a3a740 20 client.26168 populate_metadata read hostname 'smithi114' 2024-09-21T06:55:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf978ada0 0x55bdf9c286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:55:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4493a3a740 4 client.26168 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:55:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdf9b129b0 con 0x55bdf978ada0 2024-09-21T06:55:59.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.800+0000 7f4485400640 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf978ada0 0x55bdf9c286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:55:59.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.801+0000 7f4467e00640 10 client.26168 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:59.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.801+0000 7f4485400640 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf978ada0 0x55bdf9c286c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca77adcbc48abf7e server_cookie=db0ef24af851dbde in_seq=0 out_seq=0 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4467e00640 1 -- 172.21.15.114:0/1053078003 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdf9b129b0 con 0x55bdf978ada0 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4467e00640 10 client.26168 handle_command_reply: tid=0 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 client.26168 shutdown 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 msgr2=0x55bdf9bb1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9bb1b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f448002abe0 tx=0x7f4480009280 comp rx=0 tx=0).stop 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 shutdown_connections 2024-09-21T06:55:59.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bdf97f7070 0x55bdf9bac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdf97d6390 0x55bdf9bb1b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bdf978ada0 0x55bdf9c286c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 --2- 172.21.15.114:0/1053078003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdf97ce1b0 0x55bdf9bb1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 >> 172.21.15.114:0/1053078003 conn(0x55bdf97eb460 msgr2=0x55bdf9ba2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.802+0000 7f4467e00640 0 client.26168 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.803+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 shutdown_connections 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.803+0000 7f4493a3a740 1 -- 172.21.15.114:0/1053078003 wait complete. 2024-09-21T06:55:59.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.804+0000 7f4493a3a740 20 client.26168 trim_cache size 0 max 16384 2024-09-21T06:55:59.807 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:55:59.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f446c0b6900 msgr2=0x7f446c0b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f446c0b6900 0x7f446c0b8e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f44780aac00 tx=0x7f4478000bc0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 msgr2=0x7f44881a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 0x7f44881a62f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f447800d750 tx=0x7f447800dc10 comp rx=0 tx=0).stop 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 shutdown_connections 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f446c0b6900 0x7f446c0b8e10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4488114770 0x7f44881a62f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4488078290 0x7f44881a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 --2- 172.21.15.114:0/3030785212 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4488077940 0x7f44881a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:55:59.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.805+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 >> 172.21.15.114:0/3030785212 conn(0x7f44881017c0 msgr2=0x7f4488103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:55:59.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.807+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 shutdown_connections 2024-09-21T06:55:59.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:55:59.807+0000 7f448ce00640 1 -- 172.21.15.114:0/3030785212 wait complete. 2024-09-21T06:56:01.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:01 smithi114 ceph-mon[30654]: pgmap v451: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 360 op/s 2024-09-21T06:56:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:01 smithi114 ceph-mon[30654]: pgmap v452: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-21T06:56:01.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:01 smithi183 ceph-mon[34219]: pgmap v451: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 360 op/s 2024-09-21T06:56:01.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:01 smithi183 ceph-mon[34219]: pgmap v452: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-21T06:56:01.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:01 smithi193 ceph-mon[34601]: pgmap v451: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.3 MiB/s wr, 360 op/s 2024-09-21T06:56:01.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:01 smithi193 ceph-mon[34601]: pgmap v452: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-21T06:56:01.830 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:56:01.830 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:56:02.494 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:02.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:02 smithi114 ceph-mon[30654]: pgmap v453: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 242 op/s 2024-09-21T06:56:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:02 smithi183 ceph-mon[34219]: pgmap v453: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 242 op/s 2024-09-21T06:56:02.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:02 smithi193 ceph-mon[34601]: pgmap v453: 129 pgs: 129 active+clean; 1.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 242 op/s 2024-09-21T06:56:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 85.0 s, 1239.8 tps, lat 79.071 ms stddev 192.906 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 -- 172.21.15.114:0/3774233996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c10fdb0 msgr2=0x7f492c1122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 --2- 172.21.15.114:0/3774233996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c10fdb0 0x7f492c1122c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f491c009900 tx=0x7f491c02a410 comp rx=0 tx=0).stop 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 -- 172.21.15.114:0/3774233996 shutdown_connections 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 --2- 172.21.15.114:0/3774233996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c10fdb0 0x7f492c1122c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 --2- 172.21.15.114:0/3774233996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1022c0 0x7f492c10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 --2- 172.21.15.114:0/3774233996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f492c101970 0x7f492c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.938+0000 7f4931c00640 1 -- 172.21.15.114:0/3774233996 >> 172.21.15.114:0/3774233996 conn(0x7f492c078770 msgr2=0x7f492c078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.939+0000 7f4931c00640 1 -- 172.21.15.114:0/3774233996 shutdown_connections 2024-09-21T06:56:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.939+0000 7f4931c00640 1 -- 172.21.15.114:0/3774233996 wait complete. 2024-09-21T06:56:02.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.943+0000 7f4931c00640 1 Processor -- start 2024-09-21T06:56:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.946+0000 7f4931c00640 1 -- start start 2024-09-21T06:56:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.947+0000 7f4931c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c101970 0x7f492c10cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.947+0000 7f4931c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 0x7f492c10d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.947+0000 7f4931c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f492c10fdb0 0x7f492c10d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.947+0000 7f4931c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f492c115a30 con 0x7f492c10fdb0 2024-09-21T06:56:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.947+0000 7f4931c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1158b0 con 0x7f492c101970 2024-09-21T06:56:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.947+0000 7f4931c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f492c115bb0 con 0x7f492c1022c0 2024-09-21T06:56:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.948+0000 7f492b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 0x7f492c10d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.948+0000 7f492b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 0x7f492c10d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36624/0 (socket says 172.21.15.114:36624) 2024-09-21T06:56:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.948+0000 7f492b400640 1 -- 172.21.15.114:0/2835133227 learned_addr learned my addr 172.21.15.114:0/2835133227 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.948+0000 7f4931200640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f492c10fdb0 0x7f492c10d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.949+0000 7f492be00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c101970 0x7f492c10cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.950+0000 7f492b400640 1 -- 172.21.15.114:0/2835133227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c101970 msgr2=0x7f492c10cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.950+0000 7f492b400640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c101970 0x7f492c10cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.950+0000 7f492b400640 1 -- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f492c10fdb0 msgr2=0x7f492c10d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.950+0000 7f492b400640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f492c10fdb0 0x7f492c10d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.950+0000 7f492b400640 1 -- 172.21.15.114:0/2835133227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c1ac8a0 con 0x7f492c1022c0 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.950+0000 7f492b400640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 0x7f492c10d100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9f0 tx=0x7f491400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.951+0000 7f4928c00640 1 -- 172.21.15.114:0/2835133227 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491400ccc0 con 0x7f492c1022c0 2024-09-21T06:56:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.951+0000 7f4928c00640 1 -- 172.21.15.114:0/2835133227 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140108e0 con 0x7f492c1022c0 2024-09-21T06:56:02.953 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.951+0000 7f4928c00640 1 -- 172.21.15.114:0/2835133227 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914018860 con 0x7f492c1022c0 2024-09-21T06:56:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.951+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c1acb90 con 0x7f492c1022c0 2024-09-21T06:56:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.951+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c1ad090 con 0x7f492c1022c0 2024-09-21T06:56:02.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.960+0000 7f493a1c3740 1 Processor -- start 2024-09-21T06:56:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.961+0000 7f4928c00640 1 -- 172.21.15.114:0/2835133227 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4914010430 con 0x7f492c1022c0 2024-09-21T06:56:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.964+0000 7f4928c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f49100b67b0 0x7f49100b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.964+0000 7f4928c00640 1 -- 172.21.15.114:0/2835133227 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f49140e7d10 con 0x7f492c1022c0 2024-09-21T06:56:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.964+0000 7f4931200640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f49100b67b0 0x7f49100b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.965+0000 7f493a1c3740 1 -- start start 2024-09-21T06:56:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.965+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8e1d1b0 0x5604b91c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.965+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.965+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f493a1c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5604b91376b0 con 0x5604b8e1d1b0 2024-09-21T06:56:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f493a1c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604b9126ae0 con 0x5604b8df42f0 2024-09-21T06:56:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f493a1c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5604b912a8f0 con 0x5604b8dfc4d0 2024-09-21T06:56:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f4931200640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f49100b67b0 0x7f49100b8cc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f491c004770 tx=0x7f491c02a910 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58494/0 (socket says 172.21.15.114:58494) 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 -- 172.21.15.114:0/3895615322 learned_addr learned my addr 172.21.15.114:0/3895615322 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492be00640 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8e1d1b0 0x5604b91c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 -- 172.21.15.114:0/3895615322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 msgr2=0x5604b91c2890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91c2890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 -- 172.21.15.114:0/3895615322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8e1d1b0 msgr2=0x5604b91c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8e1d1b0 0x5604b91c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.966+0000 7f492b400640 1 -- 172.21.15.114:0/3895615322 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b90f3780 con 0x5604b8df42f0 2024-09-21T06:56:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.968+0000 7f492b400640 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91c2350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f491400c6a0 tx=0x7f49140096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4358e33073f4c933 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.968+0000 7f4921600640 1 -- 172.21.15.114:0/3895615322 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140b2270 con 0x5604b8df42f0 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.969+0000 7f4921600640 1 -- 172.21.15.114:0/3895615322 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140b71b0 con 0x5604b8df42f0 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.969+0000 7f4921600640 1 -- 172.21.15.114:0/3895615322 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140aed20 con 0x5604b8df42f0 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 -- 172.21.15.114:0/3895615322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 msgr2=0x5604b91c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91c2350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f491400c6a0 tx=0x7f49140096a0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 -- 172.21.15.114:0/3895615322 shutdown_connections 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91c2350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3895615322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8e1d1b0 0x5604b91c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.970+0000 7f493a1c3740 1 -- 172.21.15.114:0/3895615322 >> 172.21.15.114:0/3895615322 conn(0x5604b8e11300 msgr2=0x5604b91bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.971+0000 7f493a1c3740 1 -- 172.21.15.114:0/3895615322 shutdown_connections 2024-09-21T06:56:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.971+0000 7f493a1c3740 1 -- 172.21.15.114:0/3895615322 wait complete. 2024-09-21T06:56:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.972+0000 7f493a1c3740 1 -- start start 2024-09-21T06:56:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.972+0000 7f493a1c3740 1 Processor -- start 2024-09-21T06:56:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.976+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.976+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.976+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8e1d1b0 0x5604b91d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.976+0000 7f493a1c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5604b905eab0 con 0x5604b8dfc4d0 2024-09-21T06:56:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f492b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f4931200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8e1d1b0 0x5604b91d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f492be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f492b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36916/0 (socket says 172.21.15.114:36916) 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f492be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58504/0 (socket says 172.21.15.114:58504) 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f492b400640 1 -- 172.21.15.114:0/3232600323 learned_addr learned my addr 172.21.15.114:0/3232600323 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604b90cd570 con 0x5604b8df42f0 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5604b91d0590 con 0x5604b8e1d1b0 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.977+0000 7f493a1c3740 10 client.0 _ll_register_callbacks cb 0x5604b90dc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.978+0000 7f4931200640 1 -- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 msgr2=0x5604b91cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.978+0000 7f4931200640 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.978+0000 7f4931200640 1 -- 172.21.15.114:0/3232600323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 msgr2=0x5604b91cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.978+0000 7f4931200640 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.978+0000 7f4931200640 1 -- 172.21.15.114:0/3232600323 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b8e88790 con 0x5604b8e1d1b0 2024-09-21T06:56:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.978+0000 7f492b400640 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.980+0000 7f4931200640 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8e1d1b0 0x5604b91d9880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f491c009850 tx=0x7f491c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c10819eaf1a43d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.980+0000 7f491a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.980+0000 7f491a000640 1 -- 172.21.15.114:0/3232600323 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c04d020 con 0x5604b8e1d1b0 2024-09-21T06:56:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.980+0000 7f491a000640 1 -- 172.21.15.114:0/3232600323 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f491c003a00 con 0x5604b8e1d1b0 2024-09-21T06:56:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.982+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49140b71b0 con 0x5604b8e1d1b0 2024-09-21T06:56:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.982+0000 7f491a000640 1 -- 172.21.15.114:0/3232600323 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f491c035040 con 0x5604b8e1d1b0 2024-09-21T06:56:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.982+0000 7f493a1c3740 10 client.26140 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.982+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604b8f01620 con 0x5604b8e1d1b0 2024-09-21T06:56:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.983+0000 7f491a000640 1 -- 172.21.15.114:0/3232600323 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140b71b0 con 0x5604b8e1d1b0 2024-09-21T06:56:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.983+0000 7f491a000640 1 -- 172.21.15.114:0/3232600323 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f491c049c80 con 0x5604b8e1d1b0 2024-09-21T06:56:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.983+0000 7f493a1c3740 10 client.26140 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.984+0000 7f493a1c3740 10 client.26140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.984+0000 7f493a1c3740 20 client.26140 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.984+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8ddbd80 0x5604b91df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.984+0000 7f493a1c3740 4 client.26140 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.985+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b9139600 con 0x5604b8ddbd80 2024-09-21T06:56:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.985+0000 7f492be00640 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8ddbd80 0x5604b91df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.986+0000 7f492be00640 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8ddbd80 0x5604b91df6a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9060548107429ca server_cookie=172eb2ba98f572d7 in_seq=0 out_seq=0 2024-09-21T06:56:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.986+0000 7f491a000640 10 client.26140 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.988+0000 7f491a000640 1 -- 172.21.15.114:0/3232600323 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604b9139600 con 0x5604b8ddbd80 2024-09-21T06:56:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.988+0000 7f491a000640 10 client.26140 handle_command_reply: tid=0 2024-09-21T06:56:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.988+0000 7f493a1c3740 1 client.26140 shutdown 2024-09-21T06:56:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8e1d1b0 msgr2=0x5604b91d9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8e1d1b0 0x5604b91d9880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f491c009850 tx=0x7f491c009260 comp rx=0 tx=0).stop 2024-09-21T06:56:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 shutdown_connections 2024-09-21T06:56:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8ddbd80 0x5604b91df6a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8e1d1b0 0x5604b91d9880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 --2- 172.21.15.114:0/3232600323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8df42f0 0x5604b91cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 >> 172.21.15.114:0/3232600323 conn(0x5604b8e11300 msgr2=0x5604b91c2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.991+0000 7f491a000640 0 client.26140 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.992+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 shutdown_connections 2024-09-21T06:56:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.992+0000 7f493a1c3740 1 -- 172.21.15.114:0/3232600323 wait complete. 2024-09-21T06:56:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:02.992+0000 7f493a1c3740 20 client.26140 trim_cache size 0 max 16384 2024-09-21T06:56:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.031+0000 7f493a1c3740 1 Processor -- start 2024-09-21T06:56:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.032+0000 7f493a1c3740 1 -- start start 2024-09-21T06:56:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 0x5604b91bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b91ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f493a1c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1b7300 con 0x5604b8dfc4d0 2024-09-21T06:56:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f493a1c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1b8f00 con 0x5604b8e1d1b0 2024-09-21T06:56:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f493a1c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1b79d0 con 0x5604b8df42f0 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f492be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 0x5604b91bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f492be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 0x5604b91bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36644/0 (socket says 172.21.15.114:36644) 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f492be00640 1 -- 172.21.15.114:0/4041470380 learned_addr learned my addr 172.21.15.114:0/4041470380 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f492b400640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.033+0000 7f4931200640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b91ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f492b400640 1 -- 172.21.15.114:0/4041470380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 msgr2=0x5604b91bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f492b400640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 0x5604b91bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f492b400640 1 -- 172.21.15.114:0/4041470380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 msgr2=0x5604b91ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f492b400640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b91ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f492b400640 1 -- 172.21.15.114:0/4041470380 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b91ba000 con 0x5604b8dfc4d0 2024-09-21T06:56:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f492be00640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 0x5604b91bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.034+0000 7f4931200640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b91ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.035+0000 7f492b400640 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cf0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f491401f040 tx=0x7f49140b2170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa8e7f755887f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.035+0000 7f4920c00640 1 -- 172.21.15.114:0/4041470380 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140b2270 con 0x5604b8dfc4d0 2024-09-21T06:56:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.035+0000 7f4920c00640 1 -- 172.21.15.114:0/4041470380 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140aeac0 con 0x5604b8dfc4d0 2024-09-21T06:56:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.035+0000 7f4920c00640 1 -- 172.21.15.114:0/4041470380 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140b3400 con 0x5604b8dfc4d0 2024-09-21T06:56:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.037+0000 7f493a1c3740 1 -- 172.21.15.114:0/4041470380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 msgr2=0x5604b91cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.037+0000 7f493a1c3740 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cf0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f491401f040 tx=0x7f49140b2170 comp rx=0 tx=0).stop 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 -- 172.21.15.114:0/4041470380 shutdown_connections 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b91ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8dfc4d0 0x5604b91cf0b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 --2- 172.21.15.114:0/4041470380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8df42f0 0x5604b91bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 -- 172.21.15.114:0/4041470380 >> 172.21.15.114:0/4041470380 conn(0x5604b8e11300 msgr2=0x5604b91ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 -- 172.21.15.114:0/4041470380 shutdown_connections 2024-09-21T06:56:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.038+0000 7f493a1c3740 1 -- 172.21.15.114:0/4041470380 wait complete. 2024-09-21T06:56:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 -- start start 2024-09-21T06:56:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 Processor -- start 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 0x5604b91d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b9242000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1b8770 con 0x5604b8df42f0 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1b80a0 con 0x5604b8e1d1b0 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f492c1b52e0 con 0x5604b8dfc4d0 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.039+0000 7f493a1c3740 10 client.0 _ll_register_callbacks cb 0x7f491c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 0x5604b91d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 0x5604b91d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:36926/0 (socket says 172.21.15.114:36926) 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 -- 172.21.15.114:0/1190392085 learned_addr learned my addr 172.21.15.114:0/1190392085 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f4931200640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b9242000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492b400640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 -- 172.21.15.114:0/1190392085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 msgr2=0x5604b91d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 -- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 msgr2=0x5604b9242000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b9242000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.040+0000 7f492be00640 1 -- 172.21.15.114:0/1190392085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b9242640 con 0x5604b8df42f0 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.041+0000 7f492be00640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 0x5604b91d3070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4924005fd0 tx=0x7f492401f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145fe832778b4ca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.041+0000 7f491a000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.042+0000 7f491a000640 1 -- 172.21.15.114:0/1190392085 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492401e890 con 0x5604b8df42f0 2024-09-21T06:56:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.042+0000 7f491a000640 1 -- 172.21.15.114:0/1190392085 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f492401ea30 con 0x5604b8df42f0 2024-09-21T06:56:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.042+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49140aeac0 con 0x5604b8df42f0 2024-09-21T06:56:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.043+0000 7f491a000640 1 -- 172.21.15.114:0/1190392085 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f492401ec30 con 0x5604b8df42f0 2024-09-21T06:56:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.043+0000 7f493a1c3740 10 client.16086 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.043+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5604b92499e0 con 0x5604b8df42f0 2024-09-21T06:56:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.044+0000 7f491a000640 1 -- 172.21.15.114:0/1190392085 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140aeac0 con 0x5604b8df42f0 2024-09-21T06:56:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.044+0000 7f491a000640 1 -- 172.21.15.114:0/1190392085 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4924023400 con 0x5604b8df42f0 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f493a1c3740 10 client.16086 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f493a1c3740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f493a1c3740 20 client.16086 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f493a1c3740 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8db11c0 0x5604b924c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f493a1c3740 4 client.16086 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604b91350b0 con 0x5604b8db11c0 2024-09-21T06:56:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.045+0000 7f492b400640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8db11c0 0x5604b924c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.046+0000 7f492b400640 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8db11c0 0x5604b924c210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54fe8274ce81917 server_cookie=3bdc6e9f1a54594f in_seq=0 out_seq=0 2024-09-21T06:56:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.046+0000 7f491a000640 10 client.16086 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.059+0000 7f491a000640 1 -- 172.21.15.114:0/1190392085 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604b91350b0 con 0x5604b8db11c0 2024-09-21T06:56:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.059+0000 7f491a000640 10 client.16086 handle_command_reply: tid=0 2024-09-21T06:56:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.060+0000 7f493a1c3740 1 client.16086 shutdown 2024-09-21T06:56:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.060+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 msgr2=0x5604b91d3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.060+0000 7f493a1c3740 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 0x5604b91d3070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4924005fd0 tx=0x7f492401f780 comp rx=0 tx=0).stop 2024-09-21T06:56:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.060+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 shutdown_connections 2024-09-21T06:56:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.060+0000 7f493a1c3740 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604b8e1d1b0 0x5604b9242000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.060+0000 7f493a1c3740 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604b8db11c0 0x5604b924c210 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.061+0000 7f493a1c3740 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604b8dfc4d0 0x5604b91d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.061+0000 7f491a000640 0 client.16086 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.061+0000 7f493a1c3740 1 --2- 172.21.15.114:0/1190392085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604b8df42f0 0x5604b91d3070 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.061+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 >> 172.21.15.114:0/1190392085 conn(0x5604b8e11300 msgr2=0x5604b91beaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.062+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 shutdown_connections 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.062+0000 7f493a1c3740 1 -- 172.21.15.114:0/1190392085 wait complete. 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.062+0000 7f493a1c3740 20 client.16086 trim_cache size 0 max 16384 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:56:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f49100b67b0 msgr2=0x7f49100b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f49100b67b0 0x7f49100b8cc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f491c004770 tx=0x7f491c02a910 comp rx=0 tx=0).stop 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 msgr2=0x7f492c10d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 0x7f492c10d100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9f0 tx=0x7f491400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 shutdown_connections 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f492c10fdb0 0x7f492c10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1022c0 0x7f492c10d100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f49100b67b0 0x7f49100b8cc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 --2- 172.21.15.114:0/2835133227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c101970 0x7f492c10cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.065+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 >> 172.21.15.114:0/2835133227 conn(0x7f492c078770 msgr2=0x7f492c1118a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.066+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 shutdown_connections 2024-09-21T06:56:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:03.066+0000 7f4931c00640 1 -- 172.21.15.114:0/2835133227 wait complete. 2024-09-21T06:56:03.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.721+0000 7f0695200640 1 -- 172.21.15.114:0/810224957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 msgr2=0x7f069010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.721+0000 7f0695200640 1 --2- 172.21.15.114:0/810224957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f069010f330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0688009ca0 tx=0x7f068802a8d0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 -- 172.21.15.114:0/810224957 shutdown_connections 2024-09-21T06:56:03.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 --2- 172.21.15.114:0/810224957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f069010f330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 --2- 172.21.15.114:0/810224957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06901081b0 0x7f06901085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 --2- 172.21.15.114:0/810224957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0690102870 0x7f0690102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 -- 172.21.15.114:0/810224957 >> 172.21.15.114:0/810224957 conn(0x7f06900fe2c0 msgr2=0x7f0690100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 -- 172.21.15.114:0/810224957 shutdown_connections 2024-09-21T06:56:03.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.722+0000 7f0695200640 1 -- 172.21.15.114:0/810224957 wait complete. 2024-09-21T06:56:03.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.726+0000 7f0695200640 1 Processor -- start 2024-09-21T06:56:03.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.726+0000 7f0695200640 1 -- start start 2024-09-21T06:56:03.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f0695200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0690102870 0x7f069010cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f0695200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06901081b0 0x7f069010d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f0695200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f06901ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f0695200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0690111740 con 0x7f06901081b0 2024-09-21T06:56:03.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f0695200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06901115c0 con 0x7f0690108b00 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f0695200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06901118c0 con 0x7f0690102870 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f068fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f06901ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f068fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f06901ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58554/0 (socket says 172.21.15.114:58554) 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.727+0000 7f068fe00640 1 -- 172.21.15.114:0/1053375877 learned_addr learned my addr 172.21.15.114:0/1053375877 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.728+0000 7f068f400640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0690102870 0x7f069010cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.728+0000 7f068fe00640 1 -- 172.21.15.114:0/1053375877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0690102870 msgr2=0x7f069010cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.728+0000 7f068fe00640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0690102870 0x7f069010cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.728+0000 7f068fe00640 1 -- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06901081b0 msgr2=0x7f069010d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.728+0000 7f068fe00640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06901081b0 0x7f069010d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.728+0000 7f068fe00640 1 -- 172.21.15.114:0/1053375877 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06901ac9d0 con 0x7f0690108b00 2024-09-21T06:56:03.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.729+0000 7f068f400640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0690102870 0x7f069010cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:03.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.729+0000 7f068fe00640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f06901ac410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0688005e30 tx=0x7f0688003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.730+0000 7f0687e00640 1 -- 172.21.15.114:0/1053375877 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688004040 con 0x7f0690108b00 2024-09-21T06:56:03.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.730+0000 7f0687e00640 1 -- 172.21.15.114:0/1053375877 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0688004690 con 0x7f0690108b00 2024-09-21T06:56:03.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.730+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06901acc00 con 0x7f0690108b00 2024-09-21T06:56:03.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.731+0000 7f0687e00640 1 -- 172.21.15.114:0/1053375877 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688030c20 con 0x7f0690108b00 2024-09-21T06:56:03.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.733+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901ad140 con 0x7f0690108b00 2024-09-21T06:56:03.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.737+0000 7f0687e00640 1 -- 172.21.15.114:0/1053375877 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f06880041e0 con 0x7f0690108b00 2024-09-21T06:56:03.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.739+0000 7f069d452740 1 Processor -- start 2024-09-21T06:56:03.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.739+0000 7f069d452740 1 -- start start 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f069d452740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f069d452740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a1274e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f069d452740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a1274e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f069d452740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563a126c3e90 con 0x563a12385670 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f069d452740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a126b32c0 con 0x563a1237d490 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f069d452740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a126b70d0 con 0x563a123a6350 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a1274e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a1274e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58568/0 (socket says 172.21.15.114:58568) 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.740+0000 7f068f400640 1 -- 172.21.15.114:0/713517781 learned_addr learned my addr 172.21.15.114:0/713517781 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f068ea00640 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f068f400640 1 -- 172.21.15.114:0/713517781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 msgr2=0x563a1274df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f068f400640 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f068f400640 1 -- 172.21.15.114:0/713517781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 msgr2=0x563a1274e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f068f400640 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a1274e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f068f400640 1 -- 172.21.15.114:0/713517781 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a12411930 con 0x563a1237d490 2024-09-21T06:56:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.741+0000 7f0687e00640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06740b67d0 0x7f06740b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.742+0000 7f068ea00640 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.742+0000 7f068f400640 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a1274e440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f067c009cf0 tx=0x7f067c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f63e6cb31fd485d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.742+0000 7f068ea00640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06740b67d0 0x7f06740b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.743+0000 7f0684c00640 1 -- 172.21.15.114:0/713517781 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c012450 con 0x563a1237d490 2024-09-21T06:56:03.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.743+0000 7f0687e00640 1 -- 172.21.15.114:0/1053375877 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0688108750 con 0x7f0690108b00 2024-09-21T06:56:03.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.743+0000 7f0684c00640 1 -- 172.21.15.114:0/713517781 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f067c012aa0 con 0x563a1237d490 2024-09-21T06:56:03.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.743+0000 7f0684c00640 1 -- 172.21.15.114:0/713517781 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c00fb80 con 0x563a1237d490 2024-09-21T06:56:03.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.744+0000 7f069d452740 1 -- 172.21.15.114:0/713517781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 msgr2=0x563a1274e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.744+0000 7f069d452740 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a1274e440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f067c009cf0 tx=0x7f067c0117b0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.745+0000 7f068ea00640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06740b67d0 0x7f06740b8ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06901019c0 tx=0x7f0680009bf0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.745+0000 7f069d452740 1 -- 172.21.15.114:0/713517781 shutdown_connections 2024-09-21T06:56:03.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.745+0000 7f069d452740 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a1274e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.745+0000 7f069d452740 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a1274e440 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.745+0000 7f069d452740 1 --2- 172.21.15.114:0/713517781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.745+0000 7f069d452740 1 -- 172.21.15.114:0/713517781 >> 172.21.15.114:0/713517781 conn(0x563a1239a740 msgr2=0x563a1274bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.746+0000 7f069d452740 1 -- 172.21.15.114:0/713517781 shutdown_connections 2024-09-21T06:56:03.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.746+0000 7f069d452740 1 -- 172.21.15.114:0/713517781 wait complete. 2024-09-21T06:56:03.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.746+0000 7f069d452740 1 -- start start 2024-09-21T06:56:03.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.747+0000 7f069d452740 1 Processor -- start 2024-09-21T06:56:03.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a127599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a12385670 0x563a12759f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a123a6350 0x563a12765a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563a12514890 con 0x563a123a6350 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a1264e160 con 0x563a1237d490 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f068ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a12385670 0x563a12759f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a127599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f068ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a12385670 0x563a12759f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36694/0 (socket says 172.21.15.114:36694) 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f068ea00640 1 -- 172.21.15.114:0/1154431799 learned_addr learned my addr 172.21.15.114:0/1154431799 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a1275c6e0 con 0x563a12385670 2024-09-21T06:56:03.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f069d452740 10 client.0 _ll_register_callbacks cb 0x563a1266aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:03.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.748+0000 7f068fe00640 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a123a6350 0x563a12765a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f068f400640 1 -- 172.21.15.114:0/1154431799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a12385670 msgr2=0x563a12759f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f068f400640 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a12385670 0x563a12759f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f068f400640 1 -- 172.21.15.114:0/1154431799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a123a6350 msgr2=0x563a12765a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f068f400640 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a123a6350 0x563a12765a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f068f400640 1 -- 172.21.15.114:0/1154431799 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a12765f90 con 0x563a1237d490 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f068f400640 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a127599f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f067c01c040 tx=0x7f067c00f4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4858fb50f2546939 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f0679600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f0679600640 1 -- 172.21.15.114:0/1154431799 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c029020 con 0x563a1237d490 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f0679600640 1 -- 172.21.15.114:0/1154431799 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f067c004310 con 0x563a1237d490 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.749+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067c012aa0 con 0x563a1237d490 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.750+0000 7f0679600640 1 -- 172.21.15.114:0/1154431799 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f067c034050 con 0x563a1237d490 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.750+0000 7f069d452740 10 client.26198 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:03.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.750+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a127687e0 con 0x563a1237d490 2024-09-21T06:56:03.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.752+0000 7f0679600640 1 -- 172.21.15.114:0/1154431799 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f067c012aa0 con 0x563a1237d490 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.752+0000 7f0679600640 1 -- 172.21.15.114:0/1154431799 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f067c0035f0 con 0x563a1237d490 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.752+0000 7f069d452740 10 client.26198 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.753+0000 7f069d452740 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.753+0000 7f069d452740 20 client.26198 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.753+0000 7f069d452740 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a12364cd0 0x563a1276bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.753+0000 7f069d452740 4 client.26198 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:03.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.753+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a126c5de0 con 0x563a12364cd0 2024-09-21T06:56:03.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.753+0000 7f068ea00640 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a12364cd0 0x563a1276bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.754+0000 7f068ea00640 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a12364cd0 0x563a1276bc50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea2dca219faf5487 server_cookie=4a2454fe86001fb7 in_seq=0 out_seq=0 2024-09-21T06:56:03.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.754+0000 7f0679600640 10 client.26198 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:03.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.754+0000 7f0679600640 1 -- 172.21.15.114:0/1154431799 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a126c5de0 con 0x563a12364cd0 2024-09-21T06:56:03.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.754+0000 7f0679600640 10 client.26198 handle_command_reply: tid=0 2024-09-21T06:56:03.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.754+0000 7f069d452740 1 client.26198 shutdown 2024-09-21T06:56:03.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 msgr2=0x563a127599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a127599f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f067c01c040 tx=0x7f067c00f4b0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 shutdown_connections 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a123a6350 0x563a12765a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a12364cd0 0x563a1276bc50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a12385670 0x563a12759f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 --2- 172.21.15.114:0/1154431799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a127599f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f0679600640 0 client.26198 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.755+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 >> 172.21.15.114:0/1154431799 conn(0x563a1239a740 msgr2=0x563a12751090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.756+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 shutdown_connections 2024-09-21T06:56:03.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.756+0000 7f069d452740 1 -- 172.21.15.114:0/1154431799 wait complete. 2024-09-21T06:56:03.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.757+0000 7f069d452740 20 client.26198 trim_cache size 0 max 16384 2024-09-21T06:56:03.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 Processor -- start 2024-09-21T06:56:03.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 -- start start 2024-09-21T06:56:03.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12765950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12754020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06901b7260 con 0x563a12385670 2024-09-21T06:56:03.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06901b8e60 con 0x563a1237d490 2024-09-21T06:56:03.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.793+0000 7f069d452740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06901b7930 con 0x563a123a6350 2024-09-21T06:56:03.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.794+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12765950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.794+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12765950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58596/0 (socket says 172.21.15.114:58596) 2024-09-21T06:56:03.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.794+0000 7f068f400640 1 -- 172.21.15.114:0/217984646 learned_addr learned my addr 172.21.15.114:0/217984646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.794+0000 7f068fe00640 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.794+0000 7f068ea00640 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12754020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068fe00640 1 -- 172.21.15.114:0/217984646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 msgr2=0x563a12765950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068fe00640 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12765950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068fe00640 1 -- 172.21.15.114:0/217984646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 msgr2=0x563a12754020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068fe00640 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12754020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068fe00640 1 -- 172.21.15.114:0/217984646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a12758750 con 0x563a123a6350 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068ea00640 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12754020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:03.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f068fe00640 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274a390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0688105b30 tx=0x7f06880ca160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ceb2a5192a3458e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.795+0000 7f067be00640 1 -- 172.21.15.114:0/217984646 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880cda30 con 0x563a123a6350 2024-09-21T06:56:03.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.796+0000 7f067be00640 1 -- 172.21.15.114:0/217984646 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0688036030 con 0x563a123a6350 2024-09-21T06:56:03.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.796+0000 7f067be00640 1 -- 172.21.15.114:0/217984646 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880d7c50 con 0x563a123a6350 2024-09-21T06:56:03.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 -- 172.21.15.114:0/217984646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 msgr2=0x563a1274a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274a390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0688105b30 tx=0x7f06880ca160 comp rx=0 tx=0).stop 2024-09-21T06:56:03.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 -- 172.21.15.114:0/217984646 shutdown_connections 2024-09-21T06:56:03.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1274a390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12754020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 --2- 172.21.15.114:0/217984646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12765950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 -- 172.21.15.114:0/217984646 >> 172.21.15.114:0/217984646 conn(0x563a1239a740 msgr2=0x563a127549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.797+0000 7f069d452740 1 -- 172.21.15.114:0/217984646 shutdown_connections 2024-09-21T06:56:03.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.798+0000 7f069d452740 1 -- 172.21.15.114:0/217984646 wait complete. 2024-09-21T06:56:03.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.798+0000 7f069d452740 1 -- start start 2024-09-21T06:56:03.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.798+0000 7f069d452740 1 Processor -- start 2024-09-21T06:56:03.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.800+0000 7f069d452740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12761bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.800+0000 7f069d452740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12762100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.800+0000 7f069d452740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1275c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.801+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12761bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.801+0000 7f068f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12761bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58606/0 (socket says 172.21.15.114:58606) 2024-09-21T06:56:03.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.801+0000 7f068fe00640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1275c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.802+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06901b86d0 con 0x563a12385670 2024-09-21T06:56:03.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.802+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06901b8000 con 0x563a1237d490 2024-09-21T06:56:03.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.802+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06901b5240 con 0x563a123a6350 2024-09-21T06:56:03.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.802+0000 7f069d452740 10 client.0 _ll_register_callbacks cb 0x7f067c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:03.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.801+0000 7f068f400640 1 -- 172.21.15.114:0/2353482642 learned_addr learned my addr 172.21.15.114:0/2353482642 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:03.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068fe00640 1 -- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 msgr2=0x563a12761bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068fe00640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12761bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068fe00640 1 -- 172.21.15.114:0/2353482642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 msgr2=0x563a12762100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:03.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068ea00640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12762100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068fe00640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12762100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068fe00640 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a1276bfc0 con 0x563a123a6350 2024-09-21T06:56:03.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.803+0000 7f068ea00640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12762100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:03.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.804+0000 7f0679600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:03.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.804+0000 7f068fe00640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1275c970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0688004af0 tx=0x7f068802a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a90bbf89e08d902 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:03.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.805+0000 7f0679600640 1 -- 172.21.15.114:0/2353482642 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880b9020 con 0x563a123a6350 2024-09-21T06:56:03.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.805+0000 7f0679600640 1 -- 172.21.15.114:0/2353482642 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f06880d5270 con 0x563a123a6350 2024-09-21T06:56:03.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.805+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0688036030 con 0x563a123a6350 2024-09-21T06:56:03.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.806+0000 7f0679600640 1 -- 172.21.15.114:0/2353482642 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06881085a0 con 0x563a123a6350 2024-09-21T06:56:03.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.806+0000 7f069d452740 10 client.26158 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:03.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.806+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a127d6ae0 con 0x563a123a6350 2024-09-21T06:56:03.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.806+0000 7f0679600640 1 -- 172.21.15.114:0/2353482642 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0688036030 con 0x563a123a6350 2024-09-21T06:56:03.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.806+0000 7f0679600640 1 -- 172.21.15.114:0/2353482642 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0688108070 con 0x563a123a6350 2024-09-21T06:56:03.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f069d452740 10 client.26158 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:03.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f069d452740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f069d452740 20 client.26158 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f069d452740 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a1233a080 0x563a127d9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f069d452740 4 client.26158 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563a126c1890 con 0x563a1233a080 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.807+0000 7f068f400640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a1233a080 0x563a127d9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.809+0000 7f068f400640 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a1233a080 0x563a127d9250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685c40355f87a342 server_cookie=ed49873637757d24 in_seq=0 out_seq=0 2024-09-21T06:56:03.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.809+0000 7f0679600640 10 client.26158 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:03.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.810+0000 7f0679600640 1 -- 172.21.15.114:0/2353482642 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563a126c1890 con 0x563a1233a080 2024-09-21T06:56:03.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.810+0000 7f0679600640 10 client.26158 handle_command_reply: tid=0 2024-09-21T06:56:03.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.810+0000 7f069d452740 1 client.26158 shutdown 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 msgr2=0x563a1275c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1275c970 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0688004af0 tx=0x7f068802a8d0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 shutdown_connections 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563a1233a080 0x563a127d9250 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a123a6350 0x563a1275c970 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563a12385670 0x563a12762100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 --2- 172.21.15.114:0/2353482642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a1237d490 0x563a12761bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.811+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 >> 172.21.15.114:0/2353482642 conn(0x563a1239a740 msgr2=0x563a127549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.812+0000 7f0679600640 0 client.26158 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.813+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 shutdown_connections 2024-09-21T06:56:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.813+0000 7f069d452740 1 -- 172.21.15.114:0/2353482642 wait complete. 2024-09-21T06:56:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.813+0000 7f069d452740 20 client.26158 trim_cache size 0 max 16384 2024-09-21T06:56:03.815 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:56:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T06:56:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "4fe4b590-9a0b-4529-8c30-4c8a3215ba37", 2024-09-21T06:56:03.815 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T06:56:03.815 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.815+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06740b67d0 msgr2=0x7f06740b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.815+0000 7f0695200640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06740b67d0 0x7f06740b8ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f06901019c0 tx=0x7f0680009bf0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.815+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 msgr2=0x7f06901ac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.815+0000 7f0695200640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f06901ac410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0688005e30 tx=0x7f0688003ba0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 shutdown_connections 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06740b67d0 0x7f06740b8ce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0690108b00 0x7f06901ac410 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06901081b0 0x7f069010d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 --2- 172.21.15.114:0/1053375877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0690102870 0x7f069010cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 >> 172.21.15.114:0/1053375877 conn(0x7f06900fe2c0 msgr2=0x7f0690107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 shutdown_connections 2024-09-21T06:56:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:03.816+0000 7f0695200640 1 -- 172.21.15.114:0/1053375877 wait complete. 2024-09-21T06:56:04.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: pgmap v454: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.5 MiB/s wr, 999 op/s 2024-09-21T06:56:04.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:04.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: pgmap v454: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.5 MiB/s wr, 999 op/s 2024-09-21T06:56:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: pgmap v454: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.5 MiB/s wr, 999 op/s 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:05.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:05.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:56:06.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:05 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:56:06.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.090 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:06.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:05 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:56:06.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:05 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:56:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:56:07.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:06 smithi114 ceph-mon[30654]: pgmap v455: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-09-21T06:56:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:06 smithi183 ceph-mon[34219]: pgmap v455: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-09-21T06:56:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:06 smithi193 ceph-mon[34601]: pgmap v455: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 1.15k op/s 2024-09-21T06:56:07.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 -- 172.21.15.114:0/3889964351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 msgr2=0x7ff97010fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 --2- 172.21.15.114:0/3889964351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97010fc40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009900 tx=0x7ff95c02a480 comp rx=0 tx=0).stop 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 -- 172.21.15.114:0/3889964351 shutdown_connections 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 --2- 172.21.15.114:0/3889964351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff970110180 0x7ff9701125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 --2- 172.21.15.114:0/3889964351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97010fc40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 --2- 172.21.15.114:0/3889964351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9701025d0 0x7ff9701029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 -- 172.21.15.114:0/3889964351 >> 172.21.15.114:0/3889964351 conn(0x7ff970079730 msgr2=0x7ff970079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.591+0000 7ff976800640 1 -- 172.21.15.114:0/3889964351 shutdown_connections 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.592+0000 7ff976800640 1 -- 172.21.15.114:0/3889964351 wait complete. 2024-09-21T06:56:07.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.593+0000 7ff976800640 1 Processor -- start 2024-09-21T06:56:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.593+0000 7ff976800640 1 -- start start 2024-09-21T06:56:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff976800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9701025d0 0x7ff97019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff976800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff976800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970110180 0x7ff9701a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff976800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff970115ce0 con 0x7ff9701025d0 2024-09-21T06:56:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff976800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff970115b60 con 0x7ff970110180 2024-09-21T06:56:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff976800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff970115e60 con 0x7ff970102f20 2024-09-21T06:56:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37496/0 (socket says 172.21.15.114:37496) 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96ea00640 1 -- 172.21.15.114:0/1497406974 learned_addr learned my addr 172.21.15.114:0/1497406974 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96fe00640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970110180 0x7ff9701a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96f400640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9701025d0 0x7ff97019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96ea00640 1 -- 172.21.15.114:0/1497406974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970110180 msgr2=0x7ff9701a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96ea00640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970110180 0x7ff9701a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.594+0000 7ff96ea00640 1 -- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9701025d0 msgr2=0x7ff97019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff96ea00640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9701025d0 0x7ff97019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff96ea00640 1 -- 172.21.15.114:0/1497406974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9701a25b0 con 0x7ff970102f20 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff96fe00640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970110180 0x7ff9701a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff96f400640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9701025d0 0x7ff97019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:07.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff96ea00640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97019cc10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02aa70 tx=0x7ff95c003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:07.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff967e00640 1 -- 172.21.15.114:0/1497406974 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c03f070 con 0x7ff970102f20 2024-09-21T06:56:07.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff967e00640 1 -- 172.21.15.114:0/1497406974 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff95c031b50 con 0x7ff970102f20 2024-09-21T06:56:07.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff967e00640 1 -- 172.21.15.114:0/1497406974 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c035030 con 0x7ff970102f20 2024-09-21T06:56:07.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.595+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701a2840 con 0x7ff970102f20 2024-09-21T06:56:07.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.596+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97006cde0 con 0x7ff970102f20 2024-09-21T06:56:07.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.600+0000 7ff97d467740 1 Processor -- start 2024-09-21T06:56:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.601+0000 7ff97d467740 1 -- start start 2024-09-21T06:56:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.601+0000 7ff97d467740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f37042cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff97d467740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f37042d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff97d467740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37042d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f37042cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff97d467740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f3703a3f70 con 0x55f37005d1b0 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff97d467740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f3703933a0 con 0x55f370086070 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f37042cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60262/0 (socket says 172.21.15.114:60262) 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff97d467740 1 -- 172.21.15.114:0/3196388098 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f3703971b0 con 0x55f370065390 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff96fe00640 1 -- 172.21.15.114:0/3196388098 learned_addr learned my addr 172.21.15.114:0/3196388098 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.602+0000 7ff96f400640 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f37042d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96fe00640 1 -- 172.21.15.114:0/3196388098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 msgr2=0x55f37042d7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37042d7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96fe00640 1 -- 172.21.15.114:0/3196388098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 msgr2=0x55f37042d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:07.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f37042d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96fe00640 1 -- 172.21.15.114:0/3196388098 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37035dc90 con 0x55f370086070 2024-09-21T06:56:07.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96f400640 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f37042d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f37042cd60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c450 tx=0x7ff96800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eae68809de5a002 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff964c00640 1 -- 172.21.15.114:0/3196388098 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968007430 con 0x55f370086070 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff967e00640 1 -- 172.21.15.114:0/1497406974 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff95c042070 con 0x7ff970102f20 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff964c00640 1 -- 172.21.15.114:0/3196388098 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff968007a80 con 0x55f370086070 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.603+0000 7ff964c00640 1 -- 172.21.15.114:0/3196388098 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800aaa0 con 0x55f370086070 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 -- 172.21.15.114:0/3196388098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 msgr2=0x55f37042cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f37042cd60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c450 tx=0x7ff96800c910 comp rx=0 tx=0).stop 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 -- 172.21.15.114:0/3196388098 shutdown_connections 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37042d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f37042d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 --2- 172.21.15.114:0/3196388098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f37042cd60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.604+0000 7ff97d467740 1 -- 172.21.15.114:0/3196388098 >> 172.21.15.114:0/3196388098 conn(0x55f37007a460 msgr2=0x55f37042ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.605+0000 7ff97d467740 1 -- 172.21.15.114:0/3196388098 shutdown_connections 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.605+0000 7ff97d467740 1 -- 172.21.15.114:0/3196388098 wait complete. 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.605+0000 7ff967e00640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff9540b6620 0x7ff9540b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.605+0000 7ff967e00640 1 -- 172.21.15.114:0/1497406974 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff95c1083e0 con 0x7ff970102f20 2024-09-21T06:56:07.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.605+0000 7ff96f400640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff9540b6620 0x7ff9540b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 -- start start 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 Processor -- start 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370440d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff96f400640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff9540b6620 0x7ff9540b8b30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55f37042b9e0 tx=0x7ff960005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f370441240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f37044cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f3701f4700 con 0x55f370086070 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f37032e540 con 0x55f370065390 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.606+0000 7ff97d467740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f370443c70 con 0x55f37005d1b0 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.607+0000 7ff97d467740 10 client.0 _ll_register_callbacks cb 0x55f37033e430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:07.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.607+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f370441240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.607+0000 7ff96f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370440d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.607+0000 7ff96ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f370441240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60276/0 (socket says 172.21.15.114:60276) 2024-09-21T06:56:07.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.607+0000 7ff96ea00640 1 -- 172.21.15.114:0/3301818532 learned_addr learned my addr 172.21.15.114:0/3301818532 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:07.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.607+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f37044cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.608+0000 7ff96fe00640 1 -- 172.21.15.114:0/3301818532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 msgr2=0x55f370440d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:07.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.608+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370440d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.608+0000 7ff96fe00640 1 -- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 msgr2=0x55f370441240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:07.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.608+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f370441240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.608+0000 7ff96fe00640 1 -- 172.21.15.114:0/3301818532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3700f1650 con 0x55f370086070 2024-09-21T06:56:07.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.608+0000 7ff96ea00640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f370441240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:07.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.609+0000 7ff96fe00640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f37044cfd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff968009530 tx=0x7ff96800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2582f334da4ed473 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:07.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.609+0000 7ff959600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.609+0000 7ff959600640 1 -- 172.21.15.114:0/3301818532 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968009c60 con 0x55f370086070 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.609+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f370330b50 con 0x55f370086070 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.609+0000 7ff959600640 1 -- 172.21.15.114:0/3301818532 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff968012040 con 0x55f370086070 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.610+0000 7ff959600640 1 -- 172.21.15.114:0/3301818532 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff968016630 con 0x55f370086070 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.610+0000 7ff97d467740 10 client.16101 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.610+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f37034b910 con 0x55f370086070 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff959600640 1 -- 172.21.15.114:0/3301818532 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f370330b50 con 0x55f370086070 2024-09-21T06:56:07.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff959600640 1 -- 172.21.15.114:0/3301818532 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff96801e870 con 0x55f370086070 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff97d467740 10 client.16101 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff97d467740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff97d467740 20 client.16101 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff97d467740 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f3700449f0 0x55f370452c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff97d467740 4 client.16101 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3703a5ec0 con 0x55f3700449f0 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.611+0000 7ff96f400640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f3700449f0 0x55f370452c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.612+0000 7ff96f400640 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f3700449f0 0x55f370452c10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb5cb083176869 server_cookie=f903c062668bd589 in_seq=0 out_seq=0 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.612+0000 7ff959600640 10 client.16101 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff959600640 1 -- 172.21.15.114:0/3301818532 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f3703a5ec0 con 0x55f3700449f0 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff959600640 10 client.16101 handle_command_reply: tid=0 2024-09-21T06:56:07.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 client.16101 shutdown 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 msgr2=0x55f37044cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f37044cfd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff968009530 tx=0x7ff96800a620 comp rx=0 tx=0).stop 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 shutdown_connections 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f37044cfd0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f370441240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f3700449f0 0x55f370452c10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 --2- 172.21.15.114:0/3301818532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370440d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff959600640 0 client.16101 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 >> 172.21.15.114:0/3301818532 conn(0x55f37007a460 msgr2=0x55f370429210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:07.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 shutdown_connections 2024-09-21T06:56:07.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.613+0000 7ff97d467740 1 -- 172.21.15.114:0/3301818532 wait complete. 2024-09-21T06:56:07.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.614+0000 7ff97d467740 20 client.16101 trim_cache size 0 max 16384 2024-09-21T06:56:07.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.642+0000 7ff97d467740 1 Processor -- start 2024-09-21T06:56:07.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.642+0000 7ff97d467740 1 -- start start 2024-09-21T06:56:07.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.643+0000 7ff97d467740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f370441470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.643+0000 7ff97d467740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37043fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.643+0000 7ff97d467740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f370442930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.643+0000 7ff97d467740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700729b0 con 0x55f37005d1b0 2024-09-21T06:56:07.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.643+0000 7ff97d467740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700745b0 con 0x55f370086070 2024-09-21T06:56:07.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.643+0000 7ff97d467740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff970073080 con 0x55f370065390 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f370442930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f370442930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60280/0 (socket says 172.21.15.114:60280) 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 -- 172.21.15.114:0/2430183428 learned_addr learned my addr 172.21.15.114:0/2430183428 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96f400640 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f370441470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96ea00640 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37043fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 -- 172.21.15.114:0/2430183428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 msgr2=0x55f37043fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:07.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37043fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 -- 172.21.15.114:0/2430183428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 msgr2=0x55f370441470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:07.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f370441470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.644+0000 7ff96fe00640 1 -- 172.21.15.114:0/2430183428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f370424f30 con 0x55f370086070 2024-09-21T06:56:07.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.645+0000 7ff96fe00640 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f370442930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff968004b90 tx=0x7ff96800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33cccb30452d45f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:07.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.645+0000 7ff95be00640 1 -- 172.21.15.114:0/2430183428 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96800a470 con 0x55f370086070 2024-09-21T06:56:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.645+0000 7ff95be00640 1 -- 172.21.15.114:0/2430183428 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96801ee50 con 0x55f370086070 2024-09-21T06:56:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.645+0000 7ff95be00640 1 -- 172.21.15.114:0/2430183428 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968016a30 con 0x55f370086070 2024-09-21T06:56:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 -- 172.21.15.114:0/2430183428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 msgr2=0x55f370442930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f370442930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff968004b90 tx=0x7ff96800c3a0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 -- 172.21.15.114:0/2430183428 shutdown_connections 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370086070 0x55f370442930 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f370065390 0x55f37043fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 --2- 172.21.15.114:0/2430183428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f37005d1b0 0x55f370441470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.647+0000 7ff97d467740 1 -- 172.21.15.114:0/2430183428 >> 172.21.15.114:0/2430183428 conn(0x55f37007a460 msgr2=0x55f370442220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.648+0000 7ff97d467740 1 -- 172.21.15.114:0/2430183428 shutdown_connections 2024-09-21T06:56:07.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.648+0000 7ff97d467740 1 -- 172.21.15.114:0/2430183428 wait complete. 2024-09-21T06:56:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.649+0000 7ff97d467740 1 -- start start 2024-09-21T06:56:07.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.649+0000 7ff97d467740 1 Processor -- start 2024-09-21T06:56:07.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370444fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f3704454f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f370445a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff970073e20 con 0x55f370086070 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff970073750 con 0x55f370065390 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700709d0 con 0x55f37005d1b0 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff97d467740 10 client.0 _ll_register_callbacks cb 0x7ff96800ac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff96f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370444fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff96f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370444fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37520/0 (socket says 172.21.15.114:37520) 2024-09-21T06:56:07.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.650+0000 7ff96f400640 1 -- 172.21.15.114:0/1804217772 learned_addr learned my addr 172.21.15.114:0/1804217772 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:07.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96f400640 1 -- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 msgr2=0x55f3704454f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:56:07.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96f400640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f3704454f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96f400640 1 -- 172.21.15.114:0/1804217772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 msgr2=0x55f370445a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:56:07.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96fe00640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f370445a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96f400640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f370445a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96f400640 1 -- 172.21.15.114:0/1804217772 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3704bc5a0 con 0x55f37005d1b0 2024-09-21T06:56:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.651+0000 7ff96fe00640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f370445a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.652+0000 7ff96f400640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370444fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55f370453290 tx=0x7ff9600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a4a96590488fb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.652+0000 7ff959600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:07.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.652+0000 7ff959600640 1 -- 172.21.15.114:0/1804217772 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960028420 con 0x55f37005d1b0 2024-09-21T06:56:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.652+0000 7ff959600640 1 -- 172.21.15.114:0/1804217772 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff9600285c0 con 0x55f37005d1b0 2024-09-21T06:56:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.652+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff96801ee50 con 0x55f37005d1b0 2024-09-21T06:56:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.653+0000 7ff959600640 1 -- 172.21.15.114:0/1804217772 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff9600287c0 con 0x55f37005d1b0 2024-09-21T06:56:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.653+0000 7ff97d467740 10 client.26170 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.653+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3704bd8c0 con 0x55f37005d1b0 2024-09-21T06:56:07.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.653+0000 7ff959600640 1 -- 172.21.15.114:0/1804217772 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96801ee50 con 0x55f37005d1b0 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff959600640 1 -- 172.21.15.114:0/1804217772 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff960019040 con 0x55f37005d1b0 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff97d467740 10 client.26170 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff97d467740 10 client.26170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff97d467740 20 client.26170 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff97d467740 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f370019da0 0x55f3704c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff97d467740 4 client.26170 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.654+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3703a1970 con 0x55f370019da0 2024-09-21T06:56:07.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.655+0000 7ff96ea00640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f370019da0 0x55f3704c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.656+0000 7ff96ea00640 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f370019da0 0x55f3704c0120 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7708b081db11842 server_cookie=5b04c3753f67046d in_seq=0 out_seq=0 2024-09-21T06:56:07.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.656+0000 7ff959600640 10 client.26170 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.656+0000 7ff959600640 1 -- 172.21.15.114:0/1804217772 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55f3703a1970 con 0x55f370019da0 2024-09-21T06:56:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.656+0000 7ff959600640 10 client.26170 handle_command_reply: tid=0 2024-09-21T06:56:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.656+0000 7ff97d467740 1 client.26170 shutdown 2024-09-21T06:56:07.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 msgr2=0x55f370444fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370444fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55f370453290 tx=0x7ff9600046c0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 shutdown_connections 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f370086070 0x55f370445a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f370065390 0x55f3704454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f370019da0 0x55f3704c0120 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff959600640 0 client.26170 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:07.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 --2- 172.21.15.114:0/1804217772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f37005d1b0 0x55f370444fb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.657+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 >> 172.21.15.114:0/1804217772 conn(0x55f37007a460 msgr2=0x55f370441240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.658+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 shutdown_connections 2024-09-21T06:56:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.658+0000 7ff97d467740 1 -- 172.21.15.114:0/1804217772 wait complete. 2024-09-21T06:56:07.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.658+0000 7ff97d467740 20 client.26170 trim_cache size 0 max 16384 2024-09-21T06:56:07.660 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:56:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "4fe4b590-9a0b-4529-8c30-4c8a3215ba37", 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:56:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:56:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:56:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:56:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:56:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:56:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:56:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff9540b6620 msgr2=0x7ff9540b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff9540b6620 0x7ff9540b8b30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55f37042b9e0 tx=0x7ff960005f90 comp rx=0 tx=0).stop 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 msgr2=0x7ff97019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97019cc10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02aa70 tx=0x7ff95c003c80 comp rx=0 tx=0).stop 2024-09-21T06:56:07.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 shutdown_connections 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff9540b6620 0x7ff9540b8b30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970110180 0x7ff9701a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff970102f20 0x7ff97019cc10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 --2- 172.21.15.114:0/1497406974 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff9701025d0 0x7ff97019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.659+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 >> 172.21.15.114:0/1497406974 conn(0x7ff970079730 msgr2=0x7ff970109d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.660+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 shutdown_connections 2024-09-21T06:56:07.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:07.660+0000 7ff976800640 1 -- 172.21.15.114:0/1497406974 wait complete. 2024-09-21T06:56:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 90.0 s, 1534.4 tps, lat 65.787 ms stddev 83.786 2024-09-21T06:56:09.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:08 smithi114 ceph-mon[30654]: pgmap v456: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-09-21T06:56:09.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:08 smithi183 ceph-mon[34219]: pgmap v456: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-09-21T06:56:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:08 smithi193 ceph-mon[34601]: pgmap v456: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-09-21T06:56:10.963 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:10 smithi193 ceph-mon[34601]: pgmap v457: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-09-21T06:56:10.963 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:10 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:56:11.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:10 smithi114 ceph-mon[30654]: pgmap v457: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-09-21T06:56:11.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:10 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:56:11.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:10 smithi183 ceph-mon[34219]: pgmap v457: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 13 MiB/s wr, 1.49k op/s 2024-09-21T06:56:11.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:10 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:56:12.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:56:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:56:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:56:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:56:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:56:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:56:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:56:12.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:56:12.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:56:12.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:56:12.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:56:12.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:56:12.696 INFO:tasks.cephfs.filesystem:scrub status for tag:4fe4b590-9a0b-4529-8c30-4c8a3215ba37 - {'path': '/', 'tag': '4fe4b590-9a0b-4529-8c30-4c8a3215ba37', 'options': 'recursive,force'} 2024-09-21T06:56:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 95.0 s, 1842.2 tps, lat 54.866 ms stddev 69.001 2024-09-21T06:56:13.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:12 smithi114 ceph-mon[30654]: pgmap v458: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-21T06:56:13.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:12 smithi183 ceph-mon[34219]: pgmap v458: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-21T06:56:13.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:12 smithi193 ceph-mon[34601]: pgmap v458: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-21T06:56:15.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:14 smithi114 ceph-mon[30654]: pgmap v459: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-09-21T06:56:15.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:14 smithi183 ceph-mon[34219]: pgmap v459: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-09-21T06:56:15.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:14 smithi193 ceph-mon[34601]: pgmap v459: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 15 MiB/s wr, 1.52k op/s 2024-09-21T06:56:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.371+0000 7fc4cbc00640 1 -- 172.21.15.114:0/192266396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4103ad0 msgr2=0x7fc4c4103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.371+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/192266396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4103ad0 0x7fc4c4103ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009ca0 tx=0x7fc4bc02a8d0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.372+0000 7fc4cbc00640 1 -- 172.21.15.114:0/192266396 shutdown_connections 2024-09-21T06:56:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.372+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/192266396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4109d50 0x7fc4c4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.372+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/192266396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c4109400 0x7fc4c4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.372+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/192266396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4103ad0 0x7fc4c4103ee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.373+0000 7fc4cbc00640 1 -- 172.21.15.114:0/192266396 >> 172.21.15.114:0/192266396 conn(0x7fc4c40ff510 msgr2=0x7fc4c4101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.373+0000 7fc4cbc00640 1 -- 172.21.15.114:0/192266396 shutdown_connections 2024-09-21T06:56:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.373+0000 7fc4cbc00640 1 -- 172.21.15.114:0/192266396 wait complete. 2024-09-21T06:56:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.376+0000 7fc4cbc00640 1 Processor -- start 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.376+0000 7fc4cbc00640 1 -- start start 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4cbc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4103ad0 0x7fc4c4077130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4cbc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c4109400 0x7fc4c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4cbc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 0x7fc4c4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4cbc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4113aa0 con 0x7fc4c4109400 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4cbc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4113920 con 0x7fc4c4103ad0 2024-09-21T06:56:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4cbc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c4113c20 con 0x7fc4c4109d50 2024-09-21T06:56:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.377+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 0x7fc4c4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4103ad0 0x7fc4c4077130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c4109400 0x7fc4c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 0x7fc4c4075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:32884/0 (socket says 172.21.15.114:32884) 2024-09-21T06:56:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 -- 172.21.15.114:0/636535518 learned_addr learned my addr 172.21.15.114:0/636535518 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 -- 172.21.15.114:0/636535518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4103ad0 msgr2=0x7fc4c4077130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4103ad0 0x7fc4c4077130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 -- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c4109400 msgr2=0x7fc4c4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c4109400 0x7fc4c4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.378+0000 7fc4c9c00640 1 -- 172.21.15.114:0/636535518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4076470 con 0x7fc4c4109d50 2024-09-21T06:56:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.380+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 0x7fc4c4075c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c970 tx=0x7fc4b000cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.381+0000 7fc4c1600640 1 -- 172.21.15.114:0/636535518 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b000ae60 con 0x7fc4c4109d50 2024-09-21T06:56:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.382+0000 7fc4c1600640 1 -- 172.21.15.114:0/636535518 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b00054d0 con 0x7fc4c4109d50 2024-09-21T06:56:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.382+0000 7fc4c1600640 1 -- 172.21.15.114:0/636535518 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0004d60 con 0x7fc4c4109d50 2024-09-21T06:56:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.382+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c419e630 con 0x7fc4c4109d50 2024-09-21T06:56:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.382+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c419ebb0 con 0x7fc4c4109d50 2024-09-21T06:56:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.391+0000 7fc4c1600640 1 -- 172.21.15.114:0/636535518 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc4b0004f80 con 0x7fc4c4109d50 2024-09-21T06:56:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.392+0000 7fc4d2bd3740 1 Processor -- start 2024-09-21T06:56:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.393+0000 7fc4d2bd3740 1 -- start start 2024-09-21T06:56:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.393+0000 7fc4c1600640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4a80b6700 0x7fc4a80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cc151b0 0x56468cfb9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4c1600640 1 -- 172.21.15.114:0/636535518 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc4b00e7760 con 0x7fc4c4109d50 2024-09-21T06:56:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4c9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cc151b0 0x56468cfb9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:32896/0 (socket says 172.21.15.114:32896) 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.394+0000 7fc4c9200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cc151b0 0x56468cfb9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38884/0 (socket says 172.21.15.114:38884) 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56468cf2f6b0 con 0x56468cc151b0 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56468cf1eae0 con 0x56468cbec2f0 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56468cf228f0 con 0x56468cbf44d0 2024-09-21T06:56:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c9c00640 1 -- 172.21.15.114:0/3980150374 learned_addr learned my addr 172.21.15.114:0/3980150374 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4a80b6700 0x7fc4a80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c9c00640 1 -- 172.21.15.114:0/3980150374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 msgr2=0x56468cfba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c9c00640 1 -- 172.21.15.114:0/3980150374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cc151b0 msgr2=0x56468cfb9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cc151b0 0x56468cfb9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.395+0000 7fc4c9c00640 1 -- 172.21.15.114:0/3980150374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468ceeb780 con 0x56468cbf44d0 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.396+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfba6b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00aa450 tx=0x7fc4b00aca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a702e30cc123300 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.396+0000 7fc4ba000640 1 -- 172.21.15.114:0/3980150374 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001d030 con 0x56468cbf44d0 2024-09-21T06:56:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.397+0000 7fc4ba000640 1 -- 172.21.15.114:0/3980150374 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b0005680 con 0x56468cbf44d0 2024-09-21T06:56:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.397+0000 7fc4ba000640 1 -- 172.21.15.114:0/3980150374 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a95b0 con 0x56468cbf44d0 2024-09-21T06:56:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.397+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/3980150374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 msgr2=0x56468cfba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.397+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfba6b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00aa450 tx=0x7fc4b00aca10 comp rx=0 tx=0).stop 2024-09-21T06:56:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.398+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/3980150374 shutdown_connections 2024-09-21T06:56:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.398+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfba6b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.398+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.398+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/3980150374 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cc151b0 0x56468cfb9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.398+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/3980150374 >> 172.21.15.114:0/3980150374 conn(0x56468cc09300 msgr2=0x56468cfb7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.399+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/3980150374 shutdown_connections 2024-09-21T06:56:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.399+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4a80b6700 0x7fc4a80b8c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x56468cfb88b0 tx=0x7fc4b4006a30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.399+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/3980150374 wait complete. 2024-09-21T06:56:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.400+0000 7fc4d2bd3740 1 -- start start 2024-09-21T06:56:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.400+0000 7fc4d2bd3740 1 Processor -- start 2024-09-21T06:56:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbec2f0 0x56468cfc5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfc57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56468ce56ab0 con 0x56468cbf44d0 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56468cec5570 con 0x56468cc151b0 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56468cfc83b0 con 0x56468cbec2f0 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4d2bd3740 10 client.0 _ll_register_callbacks cb 0x56468ced4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59122/0 (socket says 172.21.15.114:59122) 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.403+0000 7fc4c9c00640 1 -- 172.21.15.114:0/2523075916 learned_addr learned my addr 172.21.15.114:0/2523075916 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4c9c00640 1 -- 172.21.15.114:0/2523075916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbec2f0 msgr2=0x56468cfc5260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:56:16.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbec2f0 0x56468cfc5260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4c9c00640 1 -- 172.21.15.114:0/2523075916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 msgr2=0x56468cfc57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfc57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4c9c00640 1 -- 172.21.15.114:0/2523075916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468cc80790 con 0x56468cc151b0 2024-09-21T06:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc5d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c940 tx=0x7fc4b005da40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81000bdbd0170fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.404+0000 7fc4aea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.405+0000 7fc4aea00640 1 -- 172.21.15.114:0/2523075916 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001d050 con 0x56468cc151b0 2024-09-21T06:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.405+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b0005680 con 0x56468cc151b0 2024-09-21T06:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.405+0000 7fc4aea00640 1 -- 172.21.15.114:0/2523075916 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc4b00d0b60 con 0x56468cc151b0 2024-09-21T06:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.405+0000 7fc4aea00640 1 -- 172.21.15.114:0/2523075916 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4b00b1cc0 con 0x56468cc151b0 2024-09-21T06:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.405+0000 7fc4d2bd3740 10 client.26228 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.405+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56468ccf9620 con 0x56468cc151b0 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.406+0000 7fc4aea00640 1 -- 172.21.15.114:0/2523075916 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b0005680 con 0x56468cc151b0 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.406+0000 7fc4aea00640 1 -- 172.21.15.114:0/2523075916 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc4b00b2e50 con 0x56468cc151b0 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.406+0000 7fc4d2bd3740 10 client.26228 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.406+0000 7fc4d2bd3740 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.407+0000 7fc4d2bd3740 20 client.26228 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.407+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cbd3d80 0x56468cfd71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.407+0000 7fc4d2bd3740 4 client.26228 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.407+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468cf31600 con 0x56468cbd3d80 2024-09-21T06:56:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.407+0000 7fc4c9200640 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cbd3d80 0x56468cfd71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.412+0000 7fc4aea00640 10 client.26228 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.412+0000 7fc4c9200640 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cbd3d80 0x56468cfd71e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e67bc5fff6246a6 server_cookie=e990a8968a1980b8 in_seq=0 out_seq=0 2024-09-21T06:56:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.412+0000 7fc4aea00640 1 -- 172.21.15.114:0/2523075916 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56468cf31600 con 0x56468cbd3d80 2024-09-21T06:56:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.412+0000 7fc4aea00640 10 client.26228 handle_command_reply: tid=0 2024-09-21T06:56:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.413+0000 7fc4d2bd3740 1 client.26228 shutdown 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.413+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 msgr2=0x56468cfc5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.413+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc5d20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c940 tx=0x7fc4b005da40 comp rx=0 tx=0).stop 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 shutdown_connections 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc5d20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfc57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cbd3d80 0x56468cfd71e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2523075916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbec2f0 0x56468cfc5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 >> 172.21.15.114:0/2523075916 conn(0x56468cc09300 msgr2=0x56468cfbac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4aea00640 0 client.26228 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.414+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 shutdown_connections 2024-09-21T06:56:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.415+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2523075916 wait complete. 2024-09-21T06:56:16.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.416+0000 7fc4d2bd3740 20 client.26228 trim_cache size 0 max 16384 2024-09-21T06:56:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.442+0000 7fc4d2bd3740 1 Processor -- start 2024-09-21T06:56:16.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 -- start start 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbec2f0 0x56468cfc6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfc2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41bfd80 con 0x56468cbec2f0 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41c1980 con 0x56468cc151b0 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.443+0000 7fc4d2bd3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41c0450 con 0x56468cbf44d0 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59132/0 (socket says 172.21.15.114:59132) 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 -- 172.21.15.114:0/354104226 learned_addr learned my addr 172.21.15.114:0/354104226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfc2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 -- 172.21.15.114:0/354104226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 msgr2=0x56468cfc2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9200640 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbec2f0 0x56468cfc6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfc2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 -- 172.21.15.114:0/354104226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbec2f0 msgr2=0x56468cfc6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbec2f0 0x56468cfc6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9c00640 1 -- 172.21.15.114:0/354104226 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468cfbaa10 con 0x56468cc151b0 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.444+0000 7fc4c9200640 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbec2f0 0x56468cfc6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:16.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.445+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc2970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00a9760 tx=0x7fc4b0007bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599e174c4da04423 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.445+0000 7fc4b9600640 1 -- 172.21.15.114:0/354104226 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b001d070 con 0x56468cc151b0 2024-09-21T06:56:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.445+0000 7fc4b9600640 1 -- 172.21.15.114:0/354104226 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b00ad8e0 con 0x56468cc151b0 2024-09-21T06:56:16.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.445+0000 7fc4b9600640 1 -- 172.21.15.114:0/354104226 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00e7330 con 0x56468cc151b0 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/354104226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 msgr2=0x56468cfc2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc2970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00a9760 tx=0x7fc4b0007bb0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/354104226 shutdown_connections 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cc151b0 0x56468cfc2970 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cbf44d0 0x56468cfc2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/354104226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbec2f0 0x56468cfc6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.447+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/354104226 >> 172.21.15.114:0/354104226 conn(0x56468cc09300 msgr2=0x56468cfc6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.448+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/354104226 shutdown_connections 2024-09-21T06:56:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.448+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/354104226 wait complete. 2024-09-21T06:56:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.448+0000 7fc4d2bd3740 1 -- start start 2024-09-21T06:56:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.448+0000 7fc4d2bd3740 1 Processor -- start 2024-09-21T06:56:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.449+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfd9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfda3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4c9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfd9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4c9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfd9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59146/0 (socket says 172.21.15.114:59146) 2024-09-21T06:56:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4d2bd3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cc151b0 0x56468d036900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfda3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfda3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38908/0 (socket says 172.21.15.114:38908) 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41c11f0 con 0x56468cbf44d0 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.450+0000 7fc4c9200640 1 -- 172.21.15.114:0/2826132171 learned_addr learned my addr 172.21.15.114:0/2826132171 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41c0b20 con 0x56468cbec2f0 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c41bde20 con 0x56468cc151b0 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4d2bd3740 10 client.0 _ll_register_callbacks cb 0x7fc4bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cc151b0 0x56468d036900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9200640 1 -- 172.21.15.114:0/2826132171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cc151b0 msgr2=0x56468d036900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9200640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cc151b0 0x56468d036900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9200640 1 -- 172.21.15.114:0/2826132171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 msgr2=0x56468cfda3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9200640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfda3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9200640 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468d036fa0 con 0x56468cbec2f0 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c9c00640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cc151b0 0x56468d036900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.451+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfda3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.452+0000 7fc4c9200640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfd9dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc02ae90 tx=0x7fc4bc031c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f441fdbbdfee5907 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.452+0000 7fc4aea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.452+0000 7fc4aea00640 1 -- 172.21.15.114:0/2826132171 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc052060 con 0x56468cbec2f0 2024-09-21T06:56:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.452+0000 7fc4aea00640 1 -- 172.21.15.114:0/2826132171 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc4bc031460 con 0x56468cbec2f0 2024-09-21T06:56:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.452+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b00ad8e0 con 0x56468cbec2f0 2024-09-21T06:56:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.452+0000 7fc4aea00640 1 -- 172.21.15.114:0/2826132171 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4bc031660 con 0x56468cbec2f0 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 10 client.26234 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56468d0415b0 con 0x56468cbec2f0 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4aea00640 1 -- 172.21.15.114:0/2826132171 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b00ad8e0 con 0x56468cbec2f0 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4aea00640 1 -- 172.21.15.114:0/2826132171 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc4bc004500 con 0x56468cbec2f0 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 10 client.26234 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 10 client.26234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 20 client.26234 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cba91c0 0x56468d043d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 4 client.26234 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.453+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56468cf2d0b0 con 0x56468cba91c0 2024-09-21T06:56:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.454+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cba91c0 0x56468d043d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.454+0000 7fc4c3e00640 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cba91c0 0x56468d043d70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94babfbe0e1c0b06 server_cookie=7648b502e5a69d71 in_seq=0 out_seq=0 2024-09-21T06:56:16.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.454+0000 7fc4aea00640 10 client.26234 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.472+0000 7fc4aea00640 1 -- 172.21.15.114:0/2826132171 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56468cf2d0b0 con 0x56468cba91c0 2024-09-21T06:56:16.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.472+0000 7fc4aea00640 10 client.26234 handle_command_reply: tid=0 2024-09-21T06:56:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.472+0000 7fc4d2bd3740 1 client.26234 shutdown 2024-09-21T06:56:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.472+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 msgr2=0x56468cfd9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfd9dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc02ae90 tx=0x7fc4bc031c60 comp rx=0 tx=0).stop 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 shutdown_connections 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56468cba91c0 0x56468d043d70 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56468cc151b0 0x56468d036900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56468cbf44d0 0x56468cfda3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 --2- 172.21.15.114:0/2826132171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468cbec2f0 0x56468cfd9dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 >> 172.21.15.114:0/2826132171 conn(0x56468cc09300 msgr2=0x56468cfb7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:16.475 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.473+0000 7fc4aea00640 0 client.26234 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.474+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 shutdown_connections 2024-09-21T06:56:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.474+0000 7fc4d2bd3740 1 -- 172.21.15.114:0/2826132171 wait complete. 2024-09-21T06:56:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.474+0000 7fc4d2bd3740 20 client.26234 trim_cache size 0 max 16384 2024-09-21T06:56:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:56:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:56:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:56:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:56:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4a80b6700 msgr2=0x7fc4a80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4a80b6700 0x7fc4a80b8c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x56468cfb88b0 tx=0x7fc4b4006a30 comp rx=0 tx=0).stop 2024-09-21T06:56:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 msgr2=0x7fc4c4075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 0x7fc4c4075c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c970 tx=0x7fc4b000cd70 comp rx=0 tx=0).stop 2024-09-21T06:56:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 shutdown_connections 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c4109d50 0x7fc4c4075c50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4c4109400 0x7fc4c4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4a80b6700 0x7fc4a80b8c10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 --2- 172.21.15.114:0/636535518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c4103ad0 0x7fc4c4077130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 >> 172.21.15.114:0/636535518 conn(0x7fc4c40ff510 msgr2=0x7fc4c4101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 shutdown_connections 2024-09-21T06:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:16.475+0000 7fc4cbc00640 1 -- 172.21.15.114:0/636535518 wait complete. 2024-09-21T06:56:17.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:16 smithi114 ceph-mon[30654]: pgmap v460: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.9 MiB/s wr, 787 op/s 2024-09-21T06:56:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:16 smithi183 ceph-mon[34219]: pgmap v460: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.9 MiB/s wr, 787 op/s 2024-09-21T06:56:17.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:16 smithi193 ceph-mon[34601]: pgmap v460: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.9 MiB/s wr, 787 op/s 2024-09-21T06:56:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 100.0 s, 2092.5 tps, lat 45.604 ms stddev 56.769 2024-09-21T06:56:19.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:18 smithi114 ceph-mon[30654]: pgmap v461: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.4 MiB/s wr, 848 op/s 2024-09-21T06:56:19.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:18 smithi183 ceph-mon[34219]: pgmap v461: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.4 MiB/s wr, 848 op/s 2024-09-21T06:56:19.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:18 smithi193 ceph-mon[34601]: pgmap v461: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 8.4 MiB/s wr, 848 op/s 2024-09-21T06:56:21.068 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:20 smithi114 ceph-mon[30654]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 640 op/s 2024-09-21T06:56:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:20 smithi183 ceph-mon[34219]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 640 op/s 2024-09-21T06:56:21.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:20 smithi193 ceph-mon[34601]: pgmap v462: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.0 MiB/s wr, 640 op/s 2024-09-21T06:56:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 105.0 s, 1471.9 tps, lat 70.595 ms stddev 95.753 2024-09-21T06:56:23.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:22 smithi114 ceph-mon[30654]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 558 op/s 2024-09-21T06:56:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:22 smithi183 ceph-mon[34219]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 558 op/s 2024-09-21T06:56:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:22 smithi193 ceph-mon[34601]: pgmap v463: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 MiB/s wr, 558 op/s 2024-09-21T06:56:25.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:24 smithi114 ceph-mon[30654]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 756 op/s 2024-09-21T06:56:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:24 smithi183 ceph-mon[34219]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 756 op/s 2024-09-21T06:56:25.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:24 smithi193 ceph-mon[34601]: pgmap v464: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 756 op/s 2024-09-21T06:56:27.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:26 smithi114 ceph-mon[30654]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 558 op/s 2024-09-21T06:56:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:26 smithi183 ceph-mon[34219]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 558 op/s 2024-09-21T06:56:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:26 smithi193 ceph-mon[34601]: pgmap v465: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.0 MiB/s wr, 558 op/s 2024-09-21T06:56:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 110.0 s, 1646.8 tps, lat 60.555 ms stddev 74.501 2024-09-21T06:56:28.774 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T06:56:28.775 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T06:56:28.775 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:56:29.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:28 smithi114 ceph-mon[30654]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 681 op/s 2024-09-21T06:56:29.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:28 smithi183 ceph-mon[34219]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 681 op/s 2024-09-21T06:56:29.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:28 smithi193 ceph-mon[34601]: pgmap v466: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 681 op/s 2024-09-21T06:56:29.443 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.802+0000 7f2db4c00640 1 -- 172.21.15.114:0/1851687830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db0077940 msgr2=0x7f2db0077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.802+0000 7f2db4c00640 1 --2- 172.21.15.114:0/1851687830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db0077940 0x7f2db0077d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009860 tx=0x7f2d9802a340 comp rx=0 tx=0).stop 2024-09-21T06:56:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.802+0000 7f2db4c00640 1 -- 172.21.15.114:0/1851687830 shutdown_connections 2024-09-21T06:56:29.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.802+0000 7f2db4c00640 1 --2- 172.21.15.114:0/1851687830 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db01146a0 0x7f2db0116ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.802+0000 7f2db4c00640 1 --2- 172.21.15.114:0/1851687830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db0078290 0x7f2db0114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.802+0000 7f2db4c00640 1 --2- 172.21.15.114:0/1851687830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db0077940 0x7f2db0077d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.803+0000 7f2db4c00640 1 -- 172.21.15.114:0/1851687830 >> 172.21.15.114:0/1851687830 conn(0x7f2db01016f0 msgr2=0x7f2db0103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.803+0000 7f2db4c00640 1 -- 172.21.15.114:0/1851687830 shutdown_connections 2024-09-21T06:56:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.803+0000 7f2db4c00640 1 -- 172.21.15.114:0/1851687830 wait complete. 2024-09-21T06:56:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 Processor -- start 2024-09-21T06:56:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 -- start start 2024-09-21T06:56:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db0077940 0x7f2db0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db0078290 0x7f2db006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 0x7f2db006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2db011a010 con 0x7f2db0077940 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0119e90 con 0x7f2db0078290 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.805+0000 7f2db4c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2db011a190 con 0x7f2db01146a0 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.807+0000 7f2dae800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 0x7f2db006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.807+0000 7f2dae800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 0x7f2db006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55852/0 (socket says 172.21.15.114:55852) 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.807+0000 7f2dae800640 1 -- 172.21.15.114:0/268340865 learned_addr learned my addr 172.21.15.114:0/268340865 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.807+0000 7f2dade00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db0077940 0x7f2db0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.809+0000 7f2dae800640 1 -- 172.21.15.114:0/268340865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db0078290 msgr2=0x7f2db006e230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:56:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.811+0000 7f2dad400640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db0078290 0x7f2db006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.811+0000 7f2dae800640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db0078290 0x7f2db006e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.811+0000 7f2dae800640 1 -- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db0077940 msgr2=0x7f2db0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.811+0000 7f2dae800640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db0077940 0x7f2db0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.811+0000 7f2dae800640 1 -- 172.21.15.114:0/268340865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db0073bd0 con 0x7f2db01146a0 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.812+0000 7f2dade00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db0077940 0x7f2db0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.812+0000 7f2dae800640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 0x7f2db006e8c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2da800eb10 tx=0x7f2da800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:29.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.813+0000 7f2da6a00640 1 -- 172.21.15.114:0/268340865 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8004b20 con 0x7f2db01146a0 2024-09-21T06:56:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.813+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db0073ec0 con 0x7f2db01146a0 2024-09-21T06:56:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.813+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db0074410 con 0x7f2db01146a0 2024-09-21T06:56:29.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.813+0000 7f2da6a00640 1 -- 172.21.15.114:0/268340865 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2da8004cc0 con 0x7f2db01146a0 2024-09-21T06:56:29.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.813+0000 7f2da6a00640 1 -- 172.21.15.114:0/268340865 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8010720 con 0x7f2db01146a0 2024-09-21T06:56:29.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.819+0000 7f2dbbc8f740 1 Processor -- start 2024-09-21T06:56:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.824+0000 7f2da6a00640 1 -- 172.21.15.114:0/268340865 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f2da8005020 con 0x7f2db01146a0 2024-09-21T06:56:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.824+0000 7f2dbbc8f740 1 -- start start 2024-09-21T06:56:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.825+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb251b0 0x56046cec9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.825+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046ceca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.825+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb044d0 0x56046ceca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.825+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56046ce3f6b0 con 0x56046cb044d0 2024-09-21T06:56:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.825+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56046ce2eae0 con 0x56046cb251b0 2024-09-21T06:56:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.825+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56046ce328f0 con 0x56046cafc2f0 2024-09-21T06:56:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.827+0000 7f2dad400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb251b0 0x56046cec9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.828+0000 7f2dad400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb251b0 0x56046cec9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36768/0 (socket says 172.21.15.114:36768) 2024-09-21T06:56:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.828+0000 7f2dad400640 1 -- 172.21.15.114:0/122547284 learned_addr learned my addr 172.21.15.114:0/122547284 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.828+0000 7f2dade00640 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046ceca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.828+0000 7f2dad400640 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb044d0 0x56046ceca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dade00640 1 -- 172.21.15.114:0/122547284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb251b0 msgr2=0x56046cec9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dade00640 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb251b0 0x56046cec9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dade00640 1 -- 172.21.15.114:0/122547284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb044d0 msgr2=0x56046ceca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dade00640 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb044d0 0x56046ceca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dade00640 1 -- 172.21.15.114:0/122547284 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046cdfb780 con 0x56046cafc2f0 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dad400640 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb044d0 0x56046ceca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.829+0000 7f2dade00640 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046ceca170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009780 tx=0x7f2d9802ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72fa77ace7a5dd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.830+0000 7f2da6a00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2d940b6780 0x7f2d940b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.830+0000 7f2da6a00640 1 -- 172.21.15.114:0/268340865 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2da80e7f50 con 0x7f2db01146a0 2024-09-21T06:56:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.830+0000 7f2dad400640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2d940b6780 0x7f2d940b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.830+0000 7f2d9f000640 1 -- 172.21.15.114:0/122547284 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98037040 con 0x56046cafc2f0 2024-09-21T06:56:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.830+0000 7f2d9f000640 1 -- 172.21.15.114:0/122547284 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d98042070 con 0x56046cafc2f0 2024-09-21T06:56:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.830+0000 7f2d9f000640 1 -- 172.21.15.114:0/122547284 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d980327f0 con 0x56046cafc2f0 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/122547284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 msgr2=0x56046ceca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046ceca170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009780 tx=0x7f2d9802ac80 comp rx=0 tx=0).stop 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dad400640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2d940b6780 0x7f2d940b8c90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005ec0 tx=0x7f2da0004730 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/122547284 shutdown_connections 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb044d0 0x56046ceca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046ceca170 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/122547284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb251b0 0x56046cec9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.831+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/122547284 >> 172.21.15.114:0/122547284 conn(0x56046cb19300 msgr2=0x56046cec7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.832+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/122547284 shutdown_connections 2024-09-21T06:56:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.832+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/122547284 wait complete. 2024-09-21T06:56:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.833+0000 7f2dbbc8f740 1 -- start start 2024-09-21T06:56:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.834+0000 7f2dbbc8f740 1 Processor -- start 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 0x56046ced5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046ced5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb251b0 0x56046ced5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56046cd66ab0 con 0x56046cafc2f0 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56046cdd5570 con 0x56046cb044d0 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56046ced8350 con 0x56046cb251b0 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.837+0000 7f2dade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 0x56046ced5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.837+0000 7f2dade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 0x56046ced5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46006/0 (socket says 172.21.15.114:46006) 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.837+0000 7f2dade00640 1 -- 172.21.15.114:0/976875332 learned_addr learned my addr 172.21.15.114:0/976875332 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.836+0000 7f2dbbc8f740 10 client.0 _ll_register_callbacks cb 0x56046cde4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.837+0000 7f2dae800640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb251b0 0x56046ced5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dae800640 1 -- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 msgr2=0x56046ced5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dad400640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046ced5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dae800640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046ced5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dae800640 1 -- 172.21.15.114:0/976875332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 msgr2=0x56046ced5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dae800640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 0x56046ced5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dae800640 1 -- 172.21.15.114:0/976875332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046cb90790 con 0x56046cb251b0 2024-09-21T06:56:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dade00640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 0x56046ced5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.838+0000 7f2dad400640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046ced5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.839+0000 7f2dae800640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb251b0 0x56046ced5c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2da80e52d0 tx=0x7f2da80a9930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c589046e3706507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.839+0000 7f2d93400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.840+0000 7f2d93400640 1 -- 172.21.15.114:0/976875332 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da80ad320 con 0x56046cb251b0 2024-09-21T06:56:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.840+0000 7f2d93400640 1 -- 172.21.15.114:0/976875332 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2da80a9530 con 0x56046cb251b0 2024-09-21T06:56:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.840+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d98042070 con 0x56046cb251b0 2024-09-21T06:56:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.841+0000 7f2d93400640 1 -- 172.21.15.114:0/976875332 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2da80ad140 con 0x56046cb251b0 2024-09-21T06:56:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.841+0000 7f2dbbc8f740 10 client.26194 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.841+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56046cc09620 con 0x56046cb251b0 2024-09-21T06:56:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.841+0000 7f2d93400640 1 -- 172.21.15.114:0/976875332 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d98042070 con 0x56046cb251b0 2024-09-21T06:56:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2d93400640 1 -- 172.21.15.114:0/976875332 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2da80c3b80 con 0x56046cb251b0 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2dbbc8f740 10 client.26194 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2dbbc8f740 10 client.26194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2dbbc8f740 20 client.26194 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cae3d80 0x56046cee7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2dbbc8f740 4 client.26194 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.842+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56046ce41600 con 0x56046cae3d80 2024-09-21T06:56:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.843+0000 7f2dade00640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cae3d80 0x56046cee7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.844+0000 7f2d93400640 10 client.26194 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.844+0000 7f2dade00640 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cae3d80 0x56046cee7460 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e2e686f9918767 server_cookie=6d8fe717cedbcd91 in_seq=0 out_seq=0 2024-09-21T06:56:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.845+0000 7f2d93400640 1 -- 172.21.15.114:0/976875332 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56046ce41600 con 0x56046cae3d80 2024-09-21T06:56:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.845+0000 7f2d93400640 10 client.26194 handle_command_reply: tid=0 2024-09-21T06:56:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.846+0000 7f2dbbc8f740 1 client.26194 shutdown 2024-09-21T06:56:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.846+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb251b0 msgr2=0x56046ced5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.846+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb251b0 0x56046ced5c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2da80e52d0 tx=0x7f2da80a9930 comp rx=0 tx=0).stop 2024-09-21T06:56:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 shutdown_connections 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cae3d80 0x56046cee7460 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb251b0 0x56046ced5c80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046ced5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/976875332 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cafc2f0 0x56046ced5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 >> 172.21.15.114:0/976875332 conn(0x56046cb19300 msgr2=0x56046cecac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.847+0000 7f2d93400640 0 client.26194 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.848+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 shutdown_connections 2024-09-21T06:56:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.848+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/976875332 wait complete. 2024-09-21T06:56:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.848+0000 7f2dbbc8f740 20 client.26194 trim_cache size 0 max 16384 2024-09-21T06:56:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.883+0000 7f2dbbc8f740 1 Processor -- start 2024-09-21T06:56:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.884+0000 7f2dbbc8f740 1 -- start start 2024-09-21T06:56:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.885+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 0x56046cecb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.885+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb044d0 0x56046ced56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.885+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 0x56046ced5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.885+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00fec0 con 0x56046cb251b0 2024-09-21T06:56:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.885+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c011ac0 con 0x56046cafc2f0 2024-09-21T06:56:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.885+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c010590 con 0x56046cb044d0 2024-09-21T06:56:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 0x56046cecb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 0x56046cecb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36790/0 (socket says 172.21.15.114:36790) 2024-09-21T06:56:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 -- 172.21.15.114:0/862784932 learned_addr learned my addr 172.21.15.114:0/862784932 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 -- 172.21.15.114:0/862784932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb044d0 msgr2=0x56046ced56e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:56:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb044d0 0x56046ced56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 -- 172.21.15.114:0/862784932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 msgr2=0x56046ced5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 0x56046ced5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.886+0000 7f2dade00640 1 -- 172.21.15.114:0/862784932 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046ced7b50 con 0x56046cafc2f0 2024-09-21T06:56:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.887+0000 7f2dade00640 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 0x56046cecb370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d980306b0 tx=0x7f2d9802ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5846804d980a61c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.890+0000 7f2d9e600640 1 -- 172.21.15.114:0/862784932 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98037040 con 0x56046cafc2f0 2024-09-21T06:56:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.890+0000 7f2d9e600640 1 -- 172.21.15.114:0/862784932 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d98042030 con 0x56046cafc2f0 2024-09-21T06:56:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.890+0000 7f2d9e600640 1 -- 172.21.15.114:0/862784932 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d980309e0 con 0x56046cafc2f0 2024-09-21T06:56:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.892+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/862784932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 msgr2=0x56046cecb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.893+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 0x56046cecb370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d980306b0 tx=0x7f2d9802ac80 comp rx=0 tx=0).stop 2024-09-21T06:56:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.894+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/862784932 shutdown_connections 2024-09-21T06:56:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.894+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 0x56046ced5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.894+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cb044d0 0x56046ced56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.894+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/862784932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cafc2f0 0x56046cecb370 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.894+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/862784932 >> 172.21.15.114:0/862784932 conn(0x56046cb19300 msgr2=0x56046cecac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.894+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/862784932 shutdown_connections 2024-09-21T06:56:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.895+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/862784932 wait complete. 2024-09-21T06:56:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.895+0000 7f2dbbc8f740 1 -- start start 2024-09-21T06:56:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.896+0000 7f2dbbc8f740 1 Processor -- start 2024-09-21T06:56:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.898+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046cedad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.900+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046cedb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.900+0000 7f2dbbc8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 0x56046cf4fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.900+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c011330 con 0x56046cb251b0 2024-09-21T06:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.900+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c010c60 con 0x56046cb044d0 2024-09-21T06:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.900+0000 7f2dbbc8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c00dea0 con 0x56046cafc2f0 2024-09-21T06:56:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.900+0000 7f2dbbc8f740 10 client.0 _ll_register_callbacks cb 0x7f2da8005af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.901+0000 7f2dade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046cedad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.901+0000 7f2dad400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046cedb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.901+0000 7f2dad400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046cedb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36806/0 (socket says 172.21.15.114:36806) 2024-09-21T06:56:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.901+0000 7f2dad400640 1 -- 172.21.15.114:0/1660594373 learned_addr learned my addr 172.21.15.114:0/1660594373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.902+0000 7f2dad400640 1 -- 172.21.15.114:0/1660594373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 msgr2=0x56046cedad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.902+0000 7f2dad400640 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046cedad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.902+0000 7f2dad400640 1 -- 172.21.15.114:0/1660594373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 msgr2=0x56046cf4fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.902+0000 7f2dad400640 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 0x56046cf4fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.902+0000 7f2dad400640 1 -- 172.21.15.114:0/1660594373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046cf50360 con 0x56046cb044d0 2024-09-21T06:56:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.903+0000 7f2dad400640 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046cedb240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2da0012bb0 tx=0x7f2da0009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd61ee55db8334c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.903+0000 7f2d93400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.903+0000 7f2d93400640 1 -- 172.21.15.114:0/1660594373 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da001ecc0 con 0x56046cb044d0 2024-09-21T06:56:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.903+0000 7f2d93400640 1 -- 172.21.15.114:0/1660594373 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2da001f420 con 0x56046cb044d0 2024-09-21T06:56:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.904+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d98042030 con 0x56046cb044d0 2024-09-21T06:56:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.905+0000 7f2d93400640 1 -- 172.21.15.114:0/1660594373 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2da0032050 con 0x56046cb044d0 2024-09-21T06:56:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.905+0000 7f2dbbc8f740 10 client.26246 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.905+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56046cf51660 con 0x56046cb044d0 2024-09-21T06:56:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.905+0000 7f2d93400640 1 -- 172.21.15.114:0/1660594373 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d98042030 con 0x56046cb044d0 2024-09-21T06:56:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2d93400640 1 -- 172.21.15.114:0/1660594373 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f2da001c410 con 0x56046cb044d0 2024-09-21T06:56:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dbbc8f740 10 client.26246 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dbbc8f740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dbbc8f740 20 client.26246 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cab91c0 0x56046cf53f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dbbc8f740 4 client.26246 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:56:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56046ce3d0b0 con 0x56046cab91c0 2024-09-21T06:56:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.907+0000 7f2dade00640 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cab91c0 0x56046cf53f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.908+0000 7f2dade00640 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cab91c0 0x56046cf53f60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cf8802c09aa825 server_cookie=84fb58760eee480f in_seq=0 out_seq=0 2024-09-21T06:56:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.909+0000 7f2d93400640 10 client.26246 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.987+0000 7f2d93400640 1 -- 172.21.15.114:0/1660594373 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56046ce3d0b0 con 0x56046cab91c0 2024-09-21T06:56:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.987+0000 7f2d93400640 10 client.26246 handle_command_reply: tid=0 2024-09-21T06:56:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 client.26246 shutdown 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 msgr2=0x56046cedb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046cedb240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2da0012bb0 tx=0x7f2da0009360 comp rx=0 tx=0).stop 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 shutdown_connections 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56046cb251b0 0x56046cf4fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56046cb044d0 0x56046cedb240 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2da0012bb0 tx=0x7f2da0009360 comp rx=0 tx=0).stop 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56046cab91c0 0x56046cf53f60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 --2- 172.21.15.114:0/1660594373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56046cafc2f0 0x56046cedad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 >> 172.21.15.114:0/1660594373 conn(0x56046cb19300 msgr2=0x56046cecac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.988+0000 7f2d93400640 0 client.26246 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.989+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 shutdown_connections 2024-09-21T06:56:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.990+0000 7f2dbbc8f740 1 -- 172.21.15.114:0/1660594373 wait complete. 2024-09-21T06:56:29.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.990+0000 7f2dbbc8f740 20 client.26246 trim_cache size 0 max 16384 2024-09-21T06:56:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:56:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:56:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:56:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:56:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.991+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2d940b6780 msgr2=0x7f2d940b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.991+0000 7f2db4c00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2d940b6780 0x7f2d940b8c90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2da0005ec0 tx=0x7f2da0004730 comp rx=0 tx=0).stop 2024-09-21T06:56:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.991+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 msgr2=0x7f2db006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.991+0000 7f2db4c00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 0x7f2db006e8c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2da800eb10 tx=0x7f2da800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 shutdown_connections 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2db01146a0 0x7f2db006e8c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2d940b6780 0x7f2d940b8c90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2db0078290 0x7f2db006e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 --2- 172.21.15.114:0/268340865 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2db0077940 0x7f2db0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 >> 172.21.15.114:0/268340865 conn(0x7f2db01016f0 msgr2=0x7f2db0116710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 shutdown_connections 2024-09-21T06:56:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:29.992+0000 7f2db4c00640 1 -- 172.21.15.114:0/268340865 wait complete. 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.241+0000 7ff4c5200640 1 -- 172.21.15.114:0/2921495043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c01092f0 msgr2=0x7ff4c0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.241+0000 7ff4c5200640 1 --2- 172.21.15.114:0/2921495043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c01092f0 0x7ff4c0109700 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009900 tx=0x7ff4b002a410 comp rx=0 tx=0).stop 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.242+0000 7ff4c5200640 1 -- 172.21.15.114:0/2921495043 shutdown_connections 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.242+0000 7ff4c5200640 1 --2- 172.21.15.114:0/2921495043 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4c0109c40 0x7ff4c0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.242+0000 7ff4c5200640 1 --2- 172.21.15.114:0/2921495043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c01092f0 0x7ff4c0109700 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.242+0000 7ff4c5200640 1 --2- 172.21.15.114:0/2921495043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 0x7ff4c0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.242+0000 7ff4c5200640 1 -- 172.21.15.114:0/2921495043 >> 172.21.15.114:0/2921495043 conn(0x7ff4c00ff400 msgr2=0x7ff4c0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.244+0000 7ff4c5200640 1 -- 172.21.15.114:0/2921495043 shutdown_connections 2024-09-21T06:56:30.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.244+0000 7ff4c5200640 1 -- 172.21.15.114:0/2921495043 wait complete. 2024-09-21T06:56:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.245+0000 7ff4c5200640 1 Processor -- start 2024-09-21T06:56:30.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.245+0000 7ff4c5200640 1 -- start start 2024-09-21T06:56:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.247+0000 7ff4c5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 0x7ff4c01a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.247+0000 7ff4c5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4c01092f0 0x7ff4c01a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:30.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.247+0000 7ff4c5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 0x7ff4c01a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.247+0000 7ff4c5200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c0111740 con 0x7ff4c01092f0 2024-09-21T06:56:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.247+0000 7ff4c5200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c01115c0 con 0x7ff4c01039b0 2024-09-21T06:56:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.247+0000 7ff4c5200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c01118c0 con 0x7ff4c0109c40 2024-09-21T06:56:30.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.248+0000 7ff4bfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 0x7ff4c01a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:30.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.248+0000 7ff4bfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 0x7ff4c01a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55890/0 (socket says 172.21.15.114:55890) 2024-09-21T06:56:30.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.248+0000 7ff4bfe00640 1 -- 172.21.15.114:0/1982401469 learned_addr learned my addr 172.21.15.114:0/1982401469 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:30.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.248+0000 7ff4bea00640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4c01092f0 0x7ff4c01a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:30.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.248+0000 7ff4bf400640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 0x7ff4c01a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.249+0000 7ff4bfe00640 1 -- 172.21.15.114:0/1982401469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 msgr2=0x7ff4c01a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.249+0000 7ff4bfe00640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 0x7ff4c01a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.249+0000 7ff4bfe00640 1 -- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4c01092f0 msgr2=0x7ff4c01a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.249+0000 7ff4bfe00640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4c01092f0 0x7ff4c01a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.249+0000 7ff4bfe00640 1 -- 172.21.15.114:0/1982401469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c01a6830 con 0x7ff4c0109c40 2024-09-21T06:56:30.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.249+0000 7ff4bf400640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 0x7ff4c01a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.250+0000 7ff4bfe00640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 0x7ff4c01a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800eb10 tx=0x7ff4b800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.250+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8004b20 con 0x7ff4c0109c40 2024-09-21T06:56:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.250+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4b8004cc0 con 0x7ff4c0109c40 2024-09-21T06:56:30.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.250+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8010670 con 0x7ff4c0109c40 2024-09-21T06:56:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.250+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c01a6b20 con 0x7ff4c0109c40 2024-09-21T06:56:30.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.250+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c01ad280 con 0x7ff4c0109c40 2024-09-21T06:56:30.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.252+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c0104c30 con 0x7ff4c0109c40 2024-09-21T06:56:30.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.256+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff4b8005020 con 0x7ff4c0109c40 2024-09-21T06:56:30.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.258+0000 7ff4b7e00640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4a40b6840 0x7ff4a40b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:30.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.258+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff4b80e8070 con 0x7ff4c0109c40 2024-09-21T06:56:30.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.259+0000 7ff4bf400640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4a40b6840 0x7ff4a40b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:30.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.261+0000 7ff4bf400640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4a40b6840 0x7ff4a40b8d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0095f0 tx=0x7ff4ac005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:30.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.263+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff4b80ae1f0 con 0x7ff4c0109c40 2024-09-21T06:56:30.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.710+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4c0103dc0 con 0x7ff4c0109c40 2024-09-21T06:56:30.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.711+0000 7ff4b7e00640 1 -- 172.21.15.114:0/1982401469 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7ff4c0103dc0 con 0x7ff4c0109c40 2024-09-21T06:56:30.717 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.716+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4a40b6840 msgr2=0x7ff4a40b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.716+0000 7ff4c5200640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4a40b6840 0x7ff4a40b8d50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0095f0 tx=0x7ff4ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.716+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 msgr2=0x7ff4c01a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.716+0000 7ff4c5200640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 0x7ff4c01a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800eb10 tx=0x7ff4b800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 shutdown_connections 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4c0109c40 0x7ff4c01a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff4c01092f0 0x7ff4c01a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff4a40b6840 0x7ff4a40b8d50 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0095f0 tx=0x7ff4ac005f90 comp rx=0 tx=0).stop 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 --2- 172.21.15.114:0/1982401469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c01039b0 0x7ff4c01a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 >> 172.21.15.114:0/1982401469 conn(0x7ff4c00ff400 msgr2=0x7ff4c0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 shutdown_connections 2024-09-21T06:56:30.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:30.717+0000 7ff4c5200640 1 -- 172.21.15.114:0/1982401469 wait complete. 2024-09-21T06:56:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:30 smithi114 ceph-mon[30654]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-09-21T06:56:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:30 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1982401469' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:30 smithi183 ceph-mon[34219]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-09-21T06:56:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:30 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1982401469' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:30 smithi193 ceph-mon[34601]: pgmap v467: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-09-21T06:56:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:30 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1982401469' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:31.704 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T06:56:31.704 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T06:56:32.374 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:32.805 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:32 smithi114 ceph-mon[30654]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-09-21T06:56:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 115.0 s, 1366.0 tps, lat 72.851 ms stddev 101.247 2024-09-21T06:56:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:32 smithi183 ceph-mon[34219]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-09-21T06:56:33.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:32 smithi193 ceph-mon[34601]: pgmap v468: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.1 MiB/s wr, 532 op/s 2024-09-21T06:56:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.251+0000 7f976b200640 1 -- 172.21.15.114:0/3653293095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c1047f0 msgr2=0x7f976c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.251+0000 7f976b200640 1 --2- 172.21.15.114:0/3653293095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c1047f0 0x7f976c104c00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f975c009900 tx=0x7f975c02a400 comp rx=0 tx=0).stop 2024-09-21T06:56:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 -- 172.21.15.114:0/3653293095 shutdown_connections 2024-09-21T06:56:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 --2- 172.21.15.114:0/3653293095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c10dc50 0x7f976c110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 --2- 172.21.15.114:0/3653293095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c105140 0x7f976c10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 --2- 172.21.15.114:0/3653293095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c1047f0 0x7f976c104c00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 -- 172.21.15.114:0/3653293095 >> 172.21.15.114:0/3653293095 conn(0x7f976c0fe330 msgr2=0x7f976c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:33.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 -- 172.21.15.114:0/3653293095 shutdown_connections 2024-09-21T06:56:33.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.252+0000 7f976b200640 1 -- 172.21.15.114:0/3653293095 wait complete. 2024-09-21T06:56:33.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.254+0000 7f976b200640 1 Processor -- start 2024-09-21T06:56:33.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.254+0000 7f976b200640 1 -- start start 2024-09-21T06:56:33.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c1047f0 0x7f976c1a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c105140 0x7f976c1a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c10dc50 0x7f976c1ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f976c113900 con 0x7f976c10dc50 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f976c113780 con 0x7f976c1047f0 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f976c113a80 con 0x7f976c105140 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f9769e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c1047f0 0x7f976c1a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f9769e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c1047f0 0x7f976c1a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36844/0 (socket says 172.21.15.114:36844) 2024-09-21T06:56:33.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f9763e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c105140 0x7f976c1a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f9769e00640 1 -- 172.21.15.114:0/114034102 learned_addr learned my addr 172.21.15.114:0/114034102 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.255+0000 7f976a800640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c10dc50 0x7f976c1ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.256+0000 7f9763e00640 1 -- 172.21.15.114:0/114034102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c1047f0 msgr2=0x7f976c1a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.256+0000 7f9763e00640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c1047f0 0x7f976c1a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.256+0000 7f9763e00640 1 -- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c10dc50 msgr2=0x7f976c1ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.256+0000 7f9763e00640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c10dc50 0x7f976c1ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.256+0000 7f9763e00640 1 -- 172.21.15.114:0/114034102 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c1a2360 con 0x7f976c105140 2024-09-21T06:56:33.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.256+0000 7f976a800640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c10dc50 0x7f976c1ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:33.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.258+0000 7f9763e00640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c105140 0x7f976c1a1b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f976c06a980 tx=0x7f975400db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:33.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.258+0000 7f9762a00640 1 -- 172.21.15.114:0/114034102 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754017070 con 0x7f976c105140 2024-09-21T06:56:33.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.258+0000 7f9762a00640 1 -- 172.21.15.114:0/114034102 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754009820 con 0x7f976c105140 2024-09-21T06:56:33.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.258+0000 7f9762a00640 1 -- 172.21.15.114:0/114034102 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754004b90 con 0x7f976c105140 2024-09-21T06:56:33.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.259+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c1acc00 con 0x7f976c105140 2024-09-21T06:56:33.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.259+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c1ad0f0 con 0x7f976c105140 2024-09-21T06:56:33.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.265+0000 7f9762a00640 1 -- 172.21.15.114:0/114034102 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9754005220 con 0x7f976c105140 2024-09-21T06:56:33.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.268+0000 7f9762a00640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97500b66f0 0x7f97500b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.268+0000 7f9762a00640 1 -- 172.21.15.114:0/114034102 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f97540e7180 con 0x7f976c105140 2024-09-21T06:56:33.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.269+0000 7f977761e740 1 Processor -- start 2024-09-21T06:56:33.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.269+0000 7f9769e00640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97500b66f0 0x7f97500b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.269+0000 7f977761e740 1 -- start start 2024-09-21T06:56:33.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.270+0000 7f977761e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.270+0000 7f977761e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.270+0000 7f977761e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.270+0000 7f977761e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a0f1048ca0 con 0x55a0f0d02980 2024-09-21T06:56:33.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.270+0000 7f977761e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0f10380d0 con 0x55a0f0d2b840 2024-09-21T06:56:33.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.270+0000 7f977761e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0f103bee0 con 0x55a0f0d0ab60 2024-09-21T06:56:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.271+0000 7f9763e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.271+0000 7f9769e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10d3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.271+0000 7f976a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.271+0000 7f976a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10d3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55932/0 (socket says 172.21.15.114:55932) 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.271+0000 7f976a800640 1 -- 172.21.15.114:0/204708851 learned_addr learned my addr 172.21.15.114:0/204708851 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9763e00640 1 -- 172.21.15.114:0/204708851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 msgr2=0x55a0f10d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9763e00640 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10d3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9763e00640 1 -- 172.21.15.114:0/204708851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 msgr2=0x55a0f10d3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9763e00640 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10d3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9763e00640 1 -- 172.21.15.114:0/204708851 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f0fea400 con 0x55a0f0d02980 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f976a800640 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:33.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9769e00640 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9769e00640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97500b66f0 0x7f97500b8c00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f975c005ec0 tx=0x7f975c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.272+0000 7f9763e00640 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10d3a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97540e4500 tx=0x7f97540ac710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9b41be7587db6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.273+0000 7f975b000640 1 -- 172.21.15.114:0/204708851 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97540b1020 con 0x55a0f0d02980 2024-09-21T06:56:33.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.273+0000 7f975b000640 1 -- 172.21.15.114:0/204708851 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97540ad300 con 0x55a0f0d02980 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f975b000640 1 -- 172.21.15.114:0/204708851 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754020070 con 0x55a0f0d02980 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 -- 172.21.15.114:0/204708851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 msgr2=0x55a0f10d3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10d3a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f97540e4500 tx=0x7f97540ac710 comp rx=0 tx=0).stop 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 -- 172.21.15.114:0/204708851 shutdown_connections 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10d3a50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 --2- 172.21.15.114:0/204708851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.274+0000 7f977761e740 1 -- 172.21.15.114:0/204708851 >> 172.21.15.114:0/204708851 conn(0x55a0f0d1f990 msgr2=0x55a0f10d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:33.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.275+0000 7f977761e740 1 -- 172.21.15.114:0/204708851 shutdown_connections 2024-09-21T06:56:33.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.275+0000 7f977761e740 1 -- 172.21.15.114:0/204708851 wait complete. 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.275+0000 7f977761e740 1 -- start start 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.276+0000 7f977761e740 1 Processor -- start 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0e997a0 con 0x55a0f0d02980 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0f0fdda10 con 0x55a0f0d2b840 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0f10e1d90 con 0x55a0f0d0ab60 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f977761e740 10 client.0 _ll_register_callbacks cb 0x55a0f0fedbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f9763e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f9763e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10df040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55940/0 (socket says 172.21.15.114:55940) 2024-09-21T06:56:33.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f9763e00640 1 -- 172.21.15.114:0/105686440 learned_addr learned my addr 172.21.15.114:0/105686440 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:33.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.277+0000 7f976a800640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.278+0000 7f9769e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.278+0000 7f9763e00640 1 -- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 msgr2=0x55a0f10df580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.278+0000 7f9763e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10df580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.278+0000 7f9763e00640 1 -- 172.21.15.114:0/105686440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 msgr2=0x55a0f10deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.278+0000 7f9763e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.278+0000 7f9763e00640 1 -- 172.21.15.114:0/105686440 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f0da5b90 con 0x55a0f0d0ab60 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.279+0000 7f9769e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.279+0000 7f9763e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10df040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f97540e44d0 tx=0x7f975400df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49483aac24e84b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:33.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.279+0000 7f974f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:33.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.279+0000 7f974f400640 1 -- 172.21.15.114:0/105686440 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754020070 con 0x55a0f0d0ab60 2024-09-21T06:56:33.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.279+0000 7f974f400640 1 -- 172.21.15.114:0/105686440 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f97540b11e0 con 0x55a0f0d0ab60 2024-09-21T06:56:33.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.280+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97540ad300 con 0x55a0f0d0ab60 2024-09-21T06:56:33.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.280+0000 7f974f400640 1 -- 172.21.15.114:0/105686440 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f97540b1040 con 0x55a0f0d0ab60 2024-09-21T06:56:33.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.280+0000 7f977761e740 10 client.26215 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:33.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.280+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a0f10fea30 con 0x55a0f0d0ab60 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f974f400640 1 -- 172.21.15.114:0/105686440 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97540ad300 con 0x55a0f0d0ab60 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f974f400640 1 -- 172.21.15.114:0/105686440 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f97540ab8d0 con 0x55a0f0d0ab60 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f977761e740 10 client.26215 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f977761e740 10 client.26215 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f977761e740 20 client.26215 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f977761e740 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cea1c0 0x55a0f1101d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f977761e740 4 client.26215 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0f104abf0 con 0x55a0f0cea1c0 2024-09-21T06:56:33.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.281+0000 7f9769e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cea1c0 0x55a0f1101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.282+0000 7f974f400640 10 client.26215 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:33.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.282+0000 7f9769e00640 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cea1c0 0x55a0f1101d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50197bd231bb9976 server_cookie=bd57644a7398b17 in_seq=0 out_seq=0 2024-09-21T06:56:33.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.283+0000 7f974f400640 1 -- 172.21.15.114:0/105686440 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0f104abf0 con 0x55a0f0cea1c0 2024-09-21T06:56:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.283+0000 7f974f400640 10 client.26215 handle_command_reply: tid=0 2024-09-21T06:56:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 client.26215 shutdown 2024-09-21T06:56:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 msgr2=0x55a0f10df040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10df040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f97540e44d0 tx=0x7f975400df90 comp rx=0 tx=0).stop 2024-09-21T06:56:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 shutdown_connections 2024-09-21T06:56:33.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f10df580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cea1c0 0x55a0f1101d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d0ab60 0x55a0f10df040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.284+0000 7f977761e740 1 --2- 172.21.15.114:0/105686440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d02980 0x55a0f10deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.285+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 >> 172.21.15.114:0/105686440 conn(0x55a0f0d1f990 msgr2=0x55a0f10d4540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:33.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.285+0000 7f974f400640 0 client.26215 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.285+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 shutdown_connections 2024-09-21T06:56:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.285+0000 7f977761e740 1 -- 172.21.15.114:0/105686440 wait complete. 2024-09-21T06:56:33.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.286+0000 7f977761e740 20 client.26215 trim_cache size 0 max 16384 2024-09-21T06:56:33.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.319+0000 7f977761e740 1 Processor -- start 2024-09-21T06:56:33.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 -- start start 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d02980 0x55a0f10e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f10ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d2b840 0x55a0f10df000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f974800ff20 con 0x55a0f0d0ab60 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9748011b20 con 0x55a0f0d02980 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.320+0000 7f977761e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97480105f0 con 0x55a0f0d2b840 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f976a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d2b840 0x55a0f10df000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f976a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d2b840 0x55a0f10df000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55946/0 (socket says 172.21.15.114:55946) 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f976a800640 1 -- 172.21.15.114:0/3848807652 learned_addr learned my addr 172.21.15.114:0/3848807652 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9763e00640 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f10ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9769e00640 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d02980 0x55a0f10e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9769e00640 1 -- 172.21.15.114:0/3848807652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d2b840 msgr2=0x55a0f10df000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9769e00640 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d2b840 0x55a0f10df000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9769e00640 1 -- 172.21.15.114:0/3848807652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 msgr2=0x55a0f10ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9769e00640 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f10ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.321+0000 7f9769e00640 1 -- 172.21.15.114:0/3848807652 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0f10fc6f0 con 0x55a0f0d02980 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.322+0000 7f9769e00640 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d02980 0x55a0f10e09e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f976c10d780 tx=0x7f975c040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a7c35e737a35c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.322+0000 7f975a600640 1 -- 172.21.15.114:0/3848807652 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c058070 con 0x55a0f0d02980 2024-09-21T06:56:33.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.322+0000 7f975a600640 1 -- 172.21.15.114:0/3848807652 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c031950 con 0x55a0f0d02980 2024-09-21T06:56:33.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 -- 172.21.15.114:0/3848807652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d02980 msgr2=0x55a0f10e09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d02980 0x55a0f10e09e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f976c10d780 tx=0x7f975c040000 comp rx=0 tx=0).stop 2024-09-21T06:56:33.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 -- 172.21.15.114:0/3848807652 shutdown_connections 2024-09-21T06:56:33.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d2b840 0x55a0f10df000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f10ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 --2- 172.21.15.114:0/3848807652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d02980 0x55a0f10e09e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 -- 172.21.15.114:0/3848807652 >> 172.21.15.114:0/3848807652 conn(0x55a0f0d1f990 msgr2=0x55a0f10ea060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:33.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.324+0000 7f977761e740 1 -- 172.21.15.114:0/3848807652 shutdown_connections 2024-09-21T06:56:33.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.325+0000 7f977761e740 1 -- 172.21.15.114:0/3848807652 wait complete. 2024-09-21T06:56:33.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.326+0000 7f977761e740 1 -- start start 2024-09-21T06:56:33.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.326+0000 7f977761e740 1 Processor -- start 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d02980 0x55a0f1104ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f1105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f1164c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9748011390 con 0x55a0f0d0ab60 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9748010cc0 con 0x55a0f0d2b840 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f974800dec0 con 0x55a0f0d02980 2024-09-21T06:56:33.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.327+0000 7f977761e740 10 client.0 _ll_register_callbacks cb 0x7f97540ac620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:33.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.328+0000 7f9763e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f1105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.328+0000 7f976a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f1164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.328+0000 7f976a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f1164c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36882/0 (socket says 172.21.15.114:36882) 2024-09-21T06:56:33.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.328+0000 7f976a800640 1 -- 172.21.15.114:0/1480375065 learned_addr learned my addr 172.21.15.114:0/1480375065 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:33.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.328+0000 7f9769e00640 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d02980 0x55a0f1104ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.329+0000 7f9769e00640 1 -- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 msgr2=0x55a0f1164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.329+0000 7f9769e00640 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f1164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.329+0000 7f9769e00640 1 -- 172.21.15.114:0/1480375065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 msgr2=0x55a0f1105480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:33.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.329+0000 7f9769e00640 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f1105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.329+0000 7f9769e00640 1 -- 172.21.15.114:0/1480375065 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0f1165270 con 0x55a0f0d02980 2024-09-21T06:56:33.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.330+0000 7f9769e00640 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d02980 0x55a0f1104ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f975c031440 tx=0x7f975c04b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a2981b60d86d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:33.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.330+0000 7f974f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:33.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.331+0000 7f974f400640 1 -- 172.21.15.114:0/1480375065 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c058070 con 0x55a0f0d02980 2024-09-21T06:56:33.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.331+0000 7f974f400640 1 -- 172.21.15.114:0/1480375065 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f975c031730 con 0x55a0f0d02980 2024-09-21T06:56:33.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.331+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c031950 con 0x55a0f0d02980 2024-09-21T06:56:33.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.331+0000 7f974f400640 1 -- 172.21.15.114:0/1480375065 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f975c031b00 con 0x55a0f0d02980 2024-09-21T06:56:33.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.332+0000 7f977761e740 10 client.26221 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:33.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.332+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a0f116c670 con 0x55a0f0d02980 2024-09-21T06:56:33.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f974f400640 1 -- 172.21.15.114:0/1480375065 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c031950 con 0x55a0f0d02980 2024-09-21T06:56:33.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f974f400640 1 -- 172.21.15.114:0/1480375065 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f975c03c710 con 0x55a0f0d02980 2024-09-21T06:56:33.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f977761e740 10 client.26221 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:33.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f977761e740 10 client.26221 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:56:33.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f977761e740 20 client.26221 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:33.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f977761e740 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cbf570 0x55a0f116eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:33.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f977761e740 4 client.26221 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T06:56:33.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.333+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x55a0f10466a0 con 0x55a0f0cbf570 2024-09-21T06:56:33.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.334+0000 7f9763e00640 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cbf570 0x55a0f116eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:33.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.334+0000 7f9763e00640 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cbf570 0x55a0f116eee0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f67f9b7a8d86d97 server_cookie=e7abb053d2df7268 in_seq=0 out_seq=0 2024-09-21T06:56:33.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.334+0000 7f974f400640 10 client.26221 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f974f400640 1 -- 172.21.15.114:0/1480375065 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a0f10466a0 con 0x55a0f0cbf570 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f974f400640 10 client.26221 handle_command_reply: tid=0 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 client.26221 shutdown 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d02980 msgr2=0x55a0f1104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d02980 0x55a0f1104ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f975c031440 tx=0x7f975c04b4f0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 shutdown_connections 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0f0d2b840 0x55a0f1164c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a0f0d0ab60 0x55a0f1105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a0f0cbf570 0x55a0f116eee0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 --2- 172.21.15.114:0/1480375065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0f0d02980 0x55a0f1104ea0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f974f400640 0 client.26221 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 >> 172.21.15.114:0/1480375065 conn(0x55a0f0d1f990 msgr2=0x55a0f10eaa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 shutdown_connections 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 1 -- 172.21.15.114:0/1480375065 wait complete. 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.519+0000 7f977761e740 20 client.26221 trim_cache size 0 max 16384 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 32, 2024-09-21T06:56:33.523 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 10, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "9fbabd86": { 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 10, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 147.8, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:56:33.524 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:33.525 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97500b66f0 msgr2=0x7f97500b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97500b66f0 0x7f97500b8c00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f975c005ec0 tx=0x7f975c005a20 comp rx=0 tx=0).stop 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c105140 msgr2=0x7f976c1a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c105140 0x7f976c1a1b20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f976c06a980 tx=0x7f975400db30 comp rx=0 tx=0).stop 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 shutdown_connections 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f976c10dc50 0x7f976c1ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f976c105140 0x7f976c1a1b20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97500b66f0 0x7f97500b8c00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 --2- 172.21.15.114:0/114034102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f976c1047f0 0x7f976c1a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.520+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 >> 172.21.15.114:0/114034102 conn(0x7f976c0fe330 msgr2=0x7f976c107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.521+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 shutdown_connections 2024-09-21T06:56:33.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:33.521+0000 7f976b200640 1 -- 172.21.15.114:0/114034102 wait complete. 2024-09-21T06:56:34.409 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9fbabd86', quiesce took 5.6 seconds. Will release after: 22.200000000000003 2024-09-21T06:56:35.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:34 smithi114 ceph-mon[30654]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 735 op/s 2024-09-21T06:56:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:34 smithi183 ceph-mon[34219]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 735 op/s 2024-09-21T06:56:35.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:34 smithi193 ceph-mon[34601]: pgmap v469: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.9 MiB/s wr, 735 op/s 2024-09-21T06:56:37.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:36 smithi114 ceph-mon[30654]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-09-21T06:56:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:36 smithi183 ceph-mon[34219]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-09-21T06:56:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:36 smithi193 ceph-mon[34601]: pgmap v470: 129 pgs: 129 active+clean; 1.5 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 546 op/s 2024-09-21T06:56:37.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 120.0 s, 135.1 tps, lat 83.113 ms stddev 90.517 2024-09-21T06:56:39.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:38 smithi114 ceph-mon[30654]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 594 op/s 2024-09-21T06:56:39.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:38 smithi183 ceph-mon[34219]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 594 op/s 2024-09-21T06:56:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:38 smithi193 ceph-mon[34601]: pgmap v471: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 594 op/s 2024-09-21T06:56:41.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:40 smithi114 ceph-mon[30654]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.0 MiB/s wr, 470 op/s 2024-09-21T06:56:41.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:40 smithi183 ceph-mon[34219]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.0 MiB/s wr, 470 op/s 2024-09-21T06:56:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:40 smithi193 ceph-mon[34601]: pgmap v472: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.0 MiB/s wr, 470 op/s 2024-09-21T06:56:42.697 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:56:42.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 125.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:56:43.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:42 smithi114 ceph-mon[30654]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 354 op/s 2024-09-21T06:56:43.087 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:42 smithi183 ceph-mon[34219]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 354 op/s 2024-09-21T06:56:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:42 smithi193 ceph-mon[34601]: pgmap v473: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.5 MiB/s wr, 354 op/s 2024-09-21T06:56:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 -- 172.21.15.114:0/2269979943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec103ad0 msgr2=0x7f06ec103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 --2- 172.21.15.114:0/2269979943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec103ad0 0x7f06ec103ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009be0 tx=0x7f06dc02a6c0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 -- 172.21.15.114:0/2269979943 shutdown_connections 2024-09-21T06:56:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 --2- 172.21.15.114:0/2269979943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec109d50 0x7f06ec110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 --2- 172.21.15.114:0/2269979943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06ec109400 0x7f06ec109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 --2- 172.21.15.114:0/2269979943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec103ad0 0x7f06ec103ee0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 -- 172.21.15.114:0/2269979943 >> 172.21.15.114:0/2269979943 conn(0x7f06ec0ff510 msgr2=0x7f06ec101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 -- 172.21.15.114:0/2269979943 shutdown_connections 2024-09-21T06:56:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.206+0000 7f06f2800640 1 -- 172.21.15.114:0/2269979943 wait complete. 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.207+0000 7f06f2800640 1 Processor -- start 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 -- start start 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 0x7f06ec19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06ec109400 0x7f06ec19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec109d50 0x7f06ec1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec113a10 con 0x7f06ec109d50 2024-09-21T06:56:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec113890 con 0x7f06ec109400 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06f2800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec113b90 con 0x7f06ec103ad0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06eb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 0x7f06ec19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec109d50 0x7f06ec1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06eb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 0x7f06ec19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53276/0 (socket says 172.21.15.114:53276) 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06eb400640 1 -- 172.21.15.114:0/1349271821 learned_addr learned my addr 172.21.15.114:0/1349271821 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.208+0000 7f06eaa00640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06ec109400 0x7f06ec19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06eb400640 1 -- 172.21.15.114:0/1349271821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06ec109400 msgr2=0x7f06ec19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06eb400640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06ec109400 0x7f06ec19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06eb400640 1 -- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec109d50 msgr2=0x7f06ec1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06eb400640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec109d50 0x7f06ec1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06eb400640 1 -- 172.21.15.114:0/1349271821 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec1a25c0 con 0x7f06ec103ad0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06eb400640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 0x7f06ec19c690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06dc02acb0 tx=0x7f06dc003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06e3e00640 1 -- 172.21.15.114:0/1349271821 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc041020 con 0x7f06ec103ad0 2024-09-21T06:56:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec1a2850 con 0x7f06ec103ad0 2024-09-21T06:56:43.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06e3e00640 1 -- 172.21.15.114:0/1349271821 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06dc0318f0 con 0x7f06ec103ad0 2024-09-21T06:56:43.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06e3e00640 1 -- 172.21.15.114:0/1349271821 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc036040 con 0x7f06ec103ad0 2024-09-21T06:56:43.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.209+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec1a9040 con 0x7f06ec103ad0 2024-09-21T06:56:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.213+0000 7f06f98bb740 1 Processor -- start 2024-09-21T06:56:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.213+0000 7f06f98bb740 1 -- start start 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199946c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995792f0 0x563199947170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06f98bb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5631998bc6b0 con 0x5631995792f0 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06f98bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5631998abae0 con 0x5631995814d0 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06f98bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631998af8f0 con 0x5631995a21b0 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199946c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199946c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53286/0 (socket says 172.21.15.114:53286) 2024-09-21T06:56:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 -- 172.21.15.114:0/3565693895 learned_addr learned my addr 172.21.15.114:0/3565693895 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06eaa00640 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 -- 172.21.15.114:0/3565693895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 msgr2=0x5631999476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 -- 172.21.15.114:0/3565693895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995792f0 msgr2=0x563199947170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995792f0 0x563199947170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06ebe00640 1 -- 172.21.15.114:0/3565693895 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563199878780 con 0x5631995a21b0 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.214+0000 7f06e3e00640 1 -- 172.21.15.114:0/1349271821 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f06dc044020 con 0x7f06ec103ad0 2024-09-21T06:56:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.215+0000 7f06ebe00640 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199946c30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009c70 tx=0x7f06e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601cd9c50bc86cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.215+0000 7f06e0c00640 1 -- 172.21.15.114:0/3565693895 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4004230 con 0x5631995a21b0 2024-09-21T06:56:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.215+0000 7f06e0c00640 1 -- 172.21.15.114:0/3565693895 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e40043d0 con 0x5631995a21b0 2024-09-21T06:56:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.215+0000 7f06e0c00640 1 -- 172.21.15.114:0/3565693895 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40197c0 con 0x5631995a21b0 2024-09-21T06:56:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 -- 172.21.15.114:0/3565693895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 msgr2=0x563199946c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199946c30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009c70 tx=0x7f06e4011810 comp rx=0 tx=0).stop 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 -- 172.21.15.114:0/3565693895 shutdown_connections 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995792f0 0x563199947170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 --2- 172.21.15.114:0/3565693895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199946c30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 -- 172.21.15.114:0/3565693895 >> 172.21.15.114:0/3565693895 conn(0x563199596300 msgr2=0x563199944b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 -- 172.21.15.114:0/3565693895 shutdown_connections 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 -- 172.21.15.114:0/3565693895 wait complete. 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06e3e00640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06d00b6590 0x7f06d00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06e3e00640 1 -- 172.21.15.114:0/1349271821 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f06dc1086a0 con 0x7f06ec103ad0 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06eaa00640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06d00b6590 0x7f06d00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.216+0000 7f06f98bb740 1 -- start start 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 Processor -- start 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06eaa00640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06d00b6590 0x7f06d00b8aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f06ec105bc0 tx=0x7f06d8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199952270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999527b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319995e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5631997e3ab0 con 0x5631995a21b0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563199852570 con 0x5631995814d0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5631999553b0 con 0x5631995792f0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06f98bb740 10 client.0 _ll_register_callbacks cb 0x563199861100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06eaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319995e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06eb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199952270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06eaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53718/0 (socket says 172.21.15.114:53718) 2024-09-21T06:56:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.217+0000 7f06eaa00640 1 -- 172.21.15.114:0/2771655837 learned_addr learned my addr 172.21.15.114:0/2771655837 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eaa00640 1 -- 172.21.15.114:0/2771655837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 msgr2=0x563199952270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eaa00640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199952270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eaa00640 1 -- 172.21.15.114:0/2771655837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 msgr2=0x56319995e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eaa00640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319995e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eaa00640 1 -- 172.21.15.114:0/2771655837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319960d790 con 0x5631995814d0 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eb400640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199952270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06ebe00640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319995e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06eaa00640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999527b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009fd0 tx=0x7f06d8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f280748454c546af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06d5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06d5600640 1 -- 172.21.15.114:0/2771655837 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d802e060 con 0x5631995814d0 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.218+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e40043d0 con 0x5631995814d0 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06d5600640 1 -- 172.21.15.114:0/2771655837 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f06d8018030 con 0x5631995814d0 2024-09-21T06:56:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06d5600640 1 -- 172.21.15.114:0/2771655837 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06d8035050 con 0x5631995814d0 2024-09-21T06:56:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 10 client.26279 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563199686620 con 0x5631995814d0 2024-09-21T06:56:43.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06d5600640 1 -- 172.21.15.114:0/2771655837 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06e40043d0 con 0x5631995814d0 2024-09-21T06:56:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06d5600640 1 -- 172.21.15.114:0/2771655837 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f06d801c3a0 con 0x5631995814d0 2024-09-21T06:56:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 10 client.26279 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 20 client.26279 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563199560d80 0x563199964490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 4 client.26279 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.219+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631998be600 con 0x563199560d80 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.220+0000 7f06eb400640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563199560d80 0x563199964490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.220+0000 7f06eb400640 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563199560d80 0x563199964490 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ef691925fe7f1a server_cookie=59b07bffbc793a99 in_seq=0 out_seq=0 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.220+0000 7f06d5600640 10 client.26279 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.221+0000 7f06d5600640 1 -- 172.21.15.114:0/2771655837 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631998be600 con 0x563199560d80 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.221+0000 7f06d5600640 10 client.26279 handle_command_reply: tid=0 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.221+0000 7f06f98bb740 1 client.26279 shutdown 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.221+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 msgr2=0x5631999527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.221+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999527b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009fd0 tx=0x7f06d8004730 comp rx=0 tx=0).stop 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 shutdown_connections 2024-09-21T06:56:43.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319995e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x5631999527b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563199560d80 0x563199964490 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2771655837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199952270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06d5600640 0 client.26279 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 >> 172.21.15.114:0/2771655837 conn(0x563199596300 msgr2=0x563199947c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 shutdown_connections 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 1 -- 172.21.15.114:0/2771655837 wait complete. 2024-09-21T06:56:43.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.222+0000 7f06f98bb740 20 client.26279 trim_cache size 0 max 16384 2024-09-21T06:56:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 Processor -- start 2024-09-21T06:56:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 -- start start 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199948730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x563199954150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319994f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b2fc0 con 0x5631995a21b0 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b4bc0 con 0x5631995814d0 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.237+0000 7f06f98bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b3690 con 0x5631995792f0 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319994f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x563199954150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319994f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46136/0 (socket says 172.21.15.114:46136) 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06ebe00640 1 -- 172.21.15.114:0/946808888 learned_addr learned my addr 172.21.15.114:0/946808888 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eb400640 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199948730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 -- 172.21.15.114:0/946808888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 msgr2=0x563199948730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199948730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 -- 172.21.15.114:0/946808888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 msgr2=0x56319994f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319994f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 -- 172.21.15.114:0/946808888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319995f380 con 0x5631995814d0 2024-09-21T06:56:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06ebe00640 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319994f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.238+0000 7f06eaa00640 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x563199954150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06d8013010 tx=0x7f06d801cf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfeda5cad68c43dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06d7e00640 1 -- 172.21.15.114:0/946808888 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d803e070 con 0x5631995814d0 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06d7e00640 1 -- 172.21.15.114:0/946808888 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d801ede0 con 0x5631995814d0 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06d7e00640 1 -- 172.21.15.114:0/946808888 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8029b30 con 0x5631995814d0 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 -- 172.21.15.114:0/946808888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 msgr2=0x563199954150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x563199954150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06d8013010 tx=0x7f06d801cf80 comp rx=0 tx=0).stop 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 -- 172.21.15.114:0/946808888 shutdown_connections 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995a21b0 0x56319994f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995814d0 0x563199954150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 --2- 172.21.15.114:0/946808888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995792f0 0x563199948730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.239+0000 7f06f98bb740 1 -- 172.21.15.114:0/946808888 >> 172.21.15.114:0/946808888 conn(0x563199596300 msgr2=0x56319995d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.240+0000 7f06f98bb740 1 -- 172.21.15.114:0/946808888 shutdown_connections 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.240+0000 7f06f98bb740 1 -- 172.21.15.114:0/946808888 wait complete. 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.240+0000 7f06f98bb740 1 -- start start 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.240+0000 7f06f98bb740 1 Processor -- start 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995792f0 0x56319995ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 0x563199958140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199958680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b4430 con 0x5631995814d0 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b3d60 con 0x5631995792f0 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec1b0fa0 con 0x5631995a21b0 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06f98bb740 10 client.0 _ll_register_callbacks cb 0x7f06d801e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 0x563199958140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995792f0 0x56319995ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 0x563199958140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46144/0 (socket says 172.21.15.114:46144) 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 -- 172.21.15.114:0/2831044678 learned_addr learned my addr 172.21.15.114:0/2831044678 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06ebe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199958680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 -- 172.21.15.114:0/2831044678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 msgr2=0x563199958680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199958680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 -- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995792f0 msgr2=0x56319995ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995792f0 0x56319995ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.241+0000 7f06eaa00640 1 -- 172.21.15.114:0/2831044678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563199958cc0 con 0x5631995814d0 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06eaa00640 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 0x563199958140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f06d8019040 tx=0x7f06d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c4a4a74119b649 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06d5600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06d5600640 1 -- 172.21.15.114:0/2831044678 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d803e050 con 0x5631995814d0 2024-09-21T06:56:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06d5600640 1 -- 172.21.15.114:0/2831044678 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f06d8005240 con 0x5631995814d0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d801ede0 con 0x5631995814d0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06d5600640 1 -- 172.21.15.114:0/2831044678 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f06d802e040 con 0x5631995814d0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06f98bb740 10 client.16152 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631999cec40 con 0x5631995814d0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.242+0000 7f06d5600640 1 -- 172.21.15.114:0/2831044678 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d801ede0 con 0x5631995814d0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06d5600640 1 -- 172.21.15.114:0/2831044678 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f06d8042420 con 0x5631995814d0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06f98bb740 10 client.16152 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06f98bb740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06f98bb740 20 client.16152 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5631995361c0 0x5631999d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06f98bb740 4 client.16152 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631998ba0b0 con 0x5631995361c0 2024-09-21T06:56:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06eb400640 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5631995361c0 0x5631999d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06eb400640 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5631995361c0 0x5631999d1310 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea1c1ccc0ee5dbe server_cookie=31d8622022c70ad in_seq=0 out_seq=0 2024-09-21T06:56:43.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.243+0000 7f06d5600640 10 client.16152 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06d5600640 1 -- 172.21.15.114:0/2831044678 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631998ba0b0 con 0x5631995361c0 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06d5600640 10 client.16152 handle_command_reply: tid=0 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 client.16152 shutdown 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 msgr2=0x563199958140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 0x563199958140 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f06d8019040 tx=0x7f06d8009260 comp rx=0 tx=0).stop 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 shutdown_connections 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5631995361c0 0x5631999d1310 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5631995a21b0 0x563199958680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06d5600640 0 client.16152 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5631995814d0 0x563199958140 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 --2- 172.21.15.114:0/2831044678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5631995792f0 0x56319995ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 >> 172.21.15.114:0/2831044678 conn(0x563199596300 msgr2=0x563199953e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 shutdown_connections 2024-09-21T06:56:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 1 -- 172.21.15.114:0/2831044678 wait complete. 2024-09-21T06:56:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.259+0000 7f06f98bb740 20 client.16152 trim_cache size 0 max 16384 2024-09-21T06:56:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:56:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:56:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:56:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06d00b6590 msgr2=0x7f06d00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06d00b6590 0x7f06d00b8aa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f06ec105bc0 tx=0x7f06d8005f90 comp rx=0 tx=0).stop 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 msgr2=0x7f06ec19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 0x7f06ec19c690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f06dc02acb0 tx=0x7f06dc003b80 comp rx=0 tx=0).stop 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 shutdown_connections 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f06ec109d50 0x7f06ec1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f06d00b6590 0x7f06d00b8aa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06ec109400 0x7f06ec19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 --2- 172.21.15.114:0/1349271821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06ec103ad0 0x7f06ec19c690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 >> 172.21.15.114:0/1349271821 conn(0x7f06ec0ff510 msgr2=0x7f06ec108b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 shutdown_connections 2024-09-21T06:56:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:43.260+0000 7f06f2800640 1 -- 172.21.15.114:0/1349271821 wait complete. 2024-09-21T06:56:43.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 -- 172.21.15.114:0/4196214942 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c105d80 msgr2=0x7f470c106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 --2- 172.21.15.114:0/4196214942 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c105d80 0x7f470c106190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009900 tx=0x7f46fc02a410 comp rx=0 tx=0).stop 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 -- 172.21.15.114:0/4196214942 shutdown_connections 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 --2- 172.21.15.114:0/4196214942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c10c010 0x7f470c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 --2- 172.21.15.114:0/4196214942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470c10b6c0 0x7f470c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 --2- 172.21.15.114:0/4196214942 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c105d80 0x7f470c106190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 -- 172.21.15.114:0/4196214942 >> 172.21.15.114:0/4196214942 conn(0x7f470c101810 msgr2=0x7f470c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 -- 172.21.15.114:0/4196214942 shutdown_connections 2024-09-21T06:56:43.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.699+0000 7f4713000640 1 -- 172.21.15.114:0/4196214942 wait complete. 2024-09-21T06:56:43.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.700+0000 7f4713000640 1 Processor -- start 2024-09-21T06:56:43.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.700+0000 7f4713000640 1 -- start start 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4713000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c105d80 0x7f470c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4713000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470c10b6c0 0x7f470c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4713000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 0x7f470c1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4713000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f470c115cd0 con 0x7f470c10c010 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4713000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f470c115b50 con 0x7f470c10b6c0 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4713000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f470c115e50 con 0x7f470c105d80 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4711000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 0x7f470c1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f470be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c105d80 0x7f470c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f470be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c105d80 0x7f470c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53332/0 (socket says 172.21.15.114:53332) 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f470b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470c10b6c0 0x7f470c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f4711000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 0x7f470c1a1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46158/0 (socket says 172.21.15.114:46158) 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.701+0000 7f470be00640 1 -- 172.21.15.114:0/1965898521 learned_addr learned my addr 172.21.15.114:0/1965898521 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4711000640 1 -- 172.21.15.114:0/1965898521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c105d80 msgr2=0x7f470c19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4711000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c105d80 0x7f470c19c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4711000640 1 -- 172.21.15.114:0/1965898521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470c10b6c0 msgr2=0x7f470c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4711000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470c10b6c0 0x7f470c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4711000640 1 -- 172.21.15.114:0/1965898521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c1a2650 con 0x7f470c10c010 2024-09-21T06:56:43.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4711000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 0x7f470c1a1f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4704009fd0 tx=0x7f470400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4708c00640 1 -- 172.21.15.114:0/1965898521 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470400ce20 con 0x7f470c10c010 2024-09-21T06:56:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c1a2940 con 0x7f470c10c010 2024-09-21T06:56:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4708c00640 1 -- 172.21.15.114:0/1965898521 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4704005020 con 0x7f470c10c010 2024-09-21T06:56:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4708c00640 1 -- 172.21.15.114:0/1965898521 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704010690 con 0x7f470c10c010 2024-09-21T06:56:43.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.702+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c1a90a0 con 0x7f470c10c010 2024-09-21T06:56:43.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 Processor -- start 2024-09-21T06:56:43.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4708c00640 1 -- 172.21.15.114:0/1965898521 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f47040054d0 con 0x7f470c10c010 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 -- start start 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b2764ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2764b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563b275c1f70 con 0x563b27283390 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b275b13a0 con 0x563b272a4070 2024-09-21T06:56:43.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.706+0000 7f4719bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563b275b51b0 con 0x563b2727b1b0 2024-09-21T06:56:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2764b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2764b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46168/0 (socket says 172.21.15.114:46168) 2024-09-21T06:56:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 -- 172.21.15.114:0/2323374414 learned_addr learned my addr 172.21.15.114:0/2323374414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b2764ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 -- 172.21.15.114:0/2323374414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 msgr2=0x563b2764b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 -- 172.21.15.114:0/2323374414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 msgr2=0x563b2764ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b2764ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470b400640 1 -- 172.21.15.114:0/2323374414 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b2757bc90 con 0x563b27283390 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.707+0000 7f470be00640 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f470b400640 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2764b7e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f46f800b6c0 tx=0x7f46f800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc11ef46cac7d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f4701600640 1 -- 172.21.15.114:0/2323374414 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8012d40 con 0x563b27283390 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f4708c00640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f46f00b6680 0x7f46f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f4701600640 1 -- 172.21.15.114:0/2323374414 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f8005020 con 0x563b27283390 2024-09-21T06:56:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f4701600640 1 -- 172.21.15.114:0/2323374414 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004a00 con 0x563b27283390 2024-09-21T06:56:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f4708c00640 1 -- 172.21.15.114:0/1965898521 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f47040e8140 con 0x7f470c10c010 2024-09-21T06:56:43.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.708+0000 7f470be00640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f46f00b6680 0x7f46f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 -- 172.21.15.114:0/2323374414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 msgr2=0x563b2764b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2764b7e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f46f800b6c0 tx=0x7f46f800b650 comp rx=0 tx=0).stop 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f470be00640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f46f00b6680 0x7f46f00b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f46fc031440 tx=0x7f46fc0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 -- 172.21.15.114:0/2323374414 shutdown_connections 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2764b7e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2323374414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b2764ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 -- 172.21.15.114:0/2323374414 >> 172.21.15.114:0/2323374414 conn(0x563b27298460 msgr2=0x563b27648c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 -- 172.21.15.114:0/2323374414 shutdown_connections 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.709+0000 7f4719bc9740 1 -- 172.21.15.114:0/2323374414 wait complete. 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.710+0000 7f4719bc9740 1 -- start start 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.710+0000 7f4719bc9740 1 Processor -- start 2024-09-21T06:56:43.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b27656580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b27656b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b27662830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563b27412700 con 0x563b27283390 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b2754c540 con 0x563b272a4070 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563b27659540 con 0x563b2727b1b0 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4719bc9740 10 client.0 _ll_register_callbacks cb 0x563b2755c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b27656b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b27656580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b27656b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46172/0 (socket says 172.21.15.114:46172) 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 -- 172.21.15.114:0/769890012 learned_addr learned my addr 172.21.15.114:0/769890012 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f4711000640 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b27662830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 -- 172.21.15.114:0/769890012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 msgr2=0x563b27656580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b27656580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 -- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 msgr2=0x563b27662830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b27662830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.711+0000 7f470b400640 1 -- 172.21.15.114:0/769890012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2730f650 con 0x563b27283390 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f470b400640 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b27656b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f46f8015040 tx=0x7f46f800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea37aa9c30a4a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f46f6000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f46f6000640 1 -- 172.21.15.114:0/769890012 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8005040 con 0x563b27283390 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f46f6000640 1 -- 172.21.15.114:0/769890012 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f46f80051e0 con 0x563b27283390 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b2754eb50 con 0x563b27283390 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f46f6000640 1 -- 172.21.15.114:0/769890012 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f46f80053e0 con 0x563b27283390 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f4719bc9740 10 client.16164 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.712+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b27569910 con 0x563b27283390 2024-09-21T06:56:43.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f46f6000640 1 -- 172.21.15.114:0/769890012 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563b2754eb50 con 0x563b27283390 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f46f6000640 1 -- 172.21.15.114:0/769890012 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f46f8004930 con 0x563b27283390 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f4719bc9740 10 client.16164 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f4719bc9740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f4719bc9740 20 client.16164 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f4719bc9740 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b272629f0 0x563b27668560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f4719bc9740 4 client.16164 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:43.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.713+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b275c3ec0 con 0x563b272629f0 2024-09-21T06:56:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.714+0000 7f470be00640 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b272629f0 0x563b27668560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.714+0000 7f470be00640 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b272629f0 0x563b27668560 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=632f0fa9409fafd5 server_cookie=cf29fe3c5f3fd7fa in_seq=0 out_seq=0 2024-09-21T06:56:43.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.714+0000 7f46f6000640 10 client.16164 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.715+0000 7f46f6000640 1 -- 172.21.15.114:0/769890012 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b275c3ec0 con 0x563b272629f0 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.715+0000 7f46f6000640 10 client.16164 handle_command_reply: tid=0 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.715+0000 7f4719bc9740 1 client.16164 shutdown 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 msgr2=0x563b27656b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b27656b20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f46f8015040 tx=0x7f46f800b410 comp rx=0 tx=0).stop 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 shutdown_connections 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b272a4070 0x563b27662830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b27656b20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b272629f0 0x563b27668560 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 --2- 172.21.15.114:0/769890012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b27656580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 >> 172.21.15.114:0/769890012 conn(0x563b27298460 msgr2=0x563b2764df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f46f6000640 0 client.16164 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 shutdown_connections 2024-09-21T06:56:43.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 1 -- 172.21.15.114:0/769890012 wait complete. 2024-09-21T06:56:43.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.716+0000 7f4719bc9740 20 client.16164 trim_cache size 0 max 16384 2024-09-21T06:56:43.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 Processor -- start 2024-09-21T06:56:43.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 -- start start 2024-09-21T06:56:43.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b27283390 0x563b27653770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 0x563b27653cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f470c1b2ff0 con 0x563b272a4070 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f470c1b4bf0 con 0x563b27283390 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.735+0000 7f4719bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f470c1b36c0 con 0x563b2727b1b0 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f470be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 0x563b27653cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f470be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:53364/0 (socket says 172.21.15.114:53364) 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 0x563b27653cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46186/0 (socket says 172.21.15.114:46186) 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f470be00640 1 -- 172.21.15.114:0/1753189922 learned_addr learned my addr 172.21.15.114:0/1753189922 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f470b400640 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b27283390 0x563b27653770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 -- 172.21.15.114:0/1753189922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 msgr2=0x563b2764e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 -- 172.21.15.114:0/1753189922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b27283390 msgr2=0x563b27653770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b27283390 0x563b27653770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 -- 172.21.15.114:0/1753189922 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b27649a40 con 0x563b272a4070 2024-09-21T06:56:43.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4711000640 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 0x563b27653cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f47040e54c0 tx=0x7f47040a9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea23152ba1fca02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.736+0000 7f4700c00640 1 -- 172.21.15.114:0/1753189922 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040b26a0 con 0x563b272a4070 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.737+0000 7f4700c00640 1 -- 172.21.15.114:0/1753189922 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47040b75e0 con 0x563b272a4070 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.737+0000 7f4700c00640 1 -- 172.21.15.114:0/1753189922 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47040b73b0 con 0x563b272a4070 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.737+0000 7f4719bc9740 1 -- 172.21.15.114:0/1753189922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 msgr2=0x563b27653cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.737+0000 7f4719bc9740 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 0x563b27653cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f47040e54c0 tx=0x7f47040a9330 comp rx=0 tx=0).stop 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.737+0000 7f4719bc9740 1 -- 172.21.15.114:0/1753189922 shutdown_connections 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.737+0000 7f4719bc9740 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b272a4070 0x563b27653cb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b27283390 0x563b27653770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 --2- 172.21.15.114:0/1753189922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b2727b1b0 0x563b2764e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 -- 172.21.15.114:0/1753189922 >> 172.21.15.114:0/1753189922 conn(0x563b27298460 msgr2=0x563b2764de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 -- 172.21.15.114:0/1753189922 shutdown_connections 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 -- 172.21.15.114:0/1753189922 wait complete. 2024-09-21T06:56:43.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 -- start start 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.738+0000 7f4719bc9740 1 Processor -- start 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b2727b1b0 0x563b2765f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2765a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b272a4070 0x563b2765aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f470c1b4460 con 0x563b27283390 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f470c1b3d90 con 0x563b2727b1b0 2024-09-21T06:56:43.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f470c1b0fd0 con 0x563b272a4070 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4719bc9740 10 client.0 _ll_register_callbacks cb 0x7f46f80126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f470b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2765a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f470b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2765a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46196/0 (socket says 172.21.15.114:46196) 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f470b400640 1 -- 172.21.15.114:0/2943522620 learned_addr learned my addr 172.21.15.114:0/2943522620 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f470be00640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b2727b1b0 0x563b2765f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.739+0000 7f4711000640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b272a4070 0x563b2765aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f470b400640 1 -- 172.21.15.114:0/2943522620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b272a4070 msgr2=0x563b2765aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f470b400640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b272a4070 0x563b2765aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f470b400640 1 -- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b2727b1b0 msgr2=0x563b2765f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f470b400640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b2727b1b0 0x563b2765f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f470b400640 1 -- 172.21.15.114:0/2943522620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2765b280 con 0x563b27283390 2024-09-21T06:56:43.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f470b400640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2765a4f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005d00 tx=0x7f46f800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edf733c049008c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f46f6000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f46f6000640 1 -- 172.21.15.114:0/2943522620 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004660 con 0x563b27283390 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47040b75e0 con 0x563b27283390 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.740+0000 7f46f6000640 1 -- 172.21.15.114:0/2943522620 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f46f8004800 con 0x563b27283390 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f46f6000640 1 -- 172.21.15.114:0/2943522620 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f46f8004a00 con 0x563b27283390 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 10 client.16170 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:43.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b276d2de0 con 0x563b27283390 2024-09-21T06:56:43.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f46f6000640 1 -- 172.21.15.114:0/2943522620 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47040b75e0 con 0x563b27283390 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f46f6000640 1 -- 172.21.15.114:0/2943522620 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f46f8003960 con 0x563b27283390 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 10 client.16170 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 20 client.16170 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b27237da0 0x563b276d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 4 client.16170 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.741+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b275bf970 con 0x563b27237da0 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.742+0000 7f470be00640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b27237da0 0x563b276d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.742+0000 7f470be00640 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b27237da0 0x563b276d55f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e785a82151af3c28 server_cookie=3ec7cbff63257a0d in_seq=0 out_seq=0 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.742+0000 7f46f6000640 10 client.16170 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f46f6000640 1 -- 172.21.15.114:0/2943522620 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x563b275bf970 con 0x563b27237da0 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f46f6000640 10 client.16170 handle_command_reply: tid=0 2024-09-21T06:56:43.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 client.16170 shutdown 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 msgr2=0x563b2765a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2765a4f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46f8005d00 tx=0x7f46f800bd30 comp rx=0 tx=0).stop 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 shutdown_connections 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563b27237da0 0x563b276d55f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b272a4070 0x563b2765aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563b27283390 0x563b2765a4f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 --2- 172.21.15.114:0/2943522620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b2727b1b0 0x563b2765f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 >> 172.21.15.114:0/2943522620 conn(0x563b27298460 msgr2=0x563b27657a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f46f6000640 0 client.16170 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 shutdown_connections 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 1 -- 172.21.15.114:0/2943522620 wait complete. 2024-09-21T06:56:43.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.743+0000 7f4719bc9740 20 client.16170 trim_cache size 0 max 16384 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:56:43.749 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:56:43.750 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f46f00b6680 msgr2=0x7f46f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f46f00b6680 0x7f46f00b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f46fc031440 tx=0x7f46fc0058d0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 msgr2=0x7f470c1a1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 0x7f470c1a1f20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4704009fd0 tx=0x7f470400ef10 comp rx=0 tx=0).stop 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 shutdown_connections 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f470c10c010 0x7f470c1a1f20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f46f00b6680 0x7f46f00b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470c10b6c0 0x7f470c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 --2- 172.21.15.114:0/1965898521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f470c105d80 0x7f470c19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 >> 172.21.15.114:0/1965898521 conn(0x7f470c101810 msgr2=0x7f470c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 shutdown_connections 2024-09-21T06:56:43.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:43.744+0000 7f4713000640 1 -- 172.21.15.114:0/1965898521 wait complete. 2024-09-21T06:56:44.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:56:44.626 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:56:44.818 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:44 smithi114 ceph-mon[30654]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-21T06:56:45.006 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:44 smithi183 ceph-mon[34219]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-21T06:56:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:44 smithi193 ceph-mon[34601]: pgmap v474: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-21T06:56:45.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.638+0000 7fbf62800640 1 -- 172.21.15.114:0/3144249206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c10de30 msgr2=0x7fbf5c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.638+0000 7fbf62800640 1 --2- 172.21.15.114:0/3144249206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c10de30 0x7fbf5c110320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf54009860 tx=0x7fbf5402a340 comp rx=0 tx=0).stop 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 -- 172.21.15.114:0/3144249206 shutdown_connections 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 --2- 172.21.15.114:0/3144249206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c10de30 0x7fbf5c110320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 --2- 172.21.15.114:0/3144249206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c105320 0x7fbf5c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 --2- 172.21.15.114:0/3144249206 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c1049d0 0x7fbf5c104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 -- 172.21.15.114:0/3144249206 >> 172.21.15.114:0/3144249206 conn(0x7fbf5c079740 msgr2=0x7fbf5c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 -- 172.21.15.114:0/3144249206 shutdown_connections 2024-09-21T06:56:45.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.639+0000 7fbf62800640 1 -- 172.21.15.114:0/3144249206 wait complete. 2024-09-21T06:56:45.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.640+0000 7fbf62800640 1 Processor -- start 2024-09-21T06:56:45.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.640+0000 7fbf62800640 1 -- start start 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.640+0000 7fbf62800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c1049d0 0x7fbf5c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf62800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 0x7fbf5c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf62800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c10de30 0x7fbf5c1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf62800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1033b0 con 0x7fbf5c10de30 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf62800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c103230 con 0x7fbf5c1049d0 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf62800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c103530 con 0x7fbf5c105320 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 0x7fbf5c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c10de30 0x7fbf5c1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf5b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c1049d0 0x7fbf5c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:45.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 0x7fbf5c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56102/0 (socket says 172.21.15.114:56102) 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf5b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c1049d0 0x7fbf5c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34906/0 (socket says 172.21.15.114:34906) 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 -- 172.21.15.114:0/106190627 learned_addr learned my addr 172.21.15.114:0/106190627 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf5be00640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c10de30 0x7fbf5c1a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41968/0 (socket says 172.21.15.114:41968) 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 -- 172.21.15.114:0/106190627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c1049d0 msgr2=0x7fbf5c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c1049d0 0x7fbf5c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 -- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c10de30 msgr2=0x7fbf5c1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c10de30 0x7fbf5c1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.641+0000 7fbf52a00640 1 -- 172.21.15.114:0/106190627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c1a2640 con 0x7fbf5c105320 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.642+0000 7fbf52a00640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 0x7fbf5c19cc70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.642+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400ccc0 con 0x7fbf5c105320 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.642+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c1a2930 con 0x7fbf5c105320 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.642+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf440054d0 con 0x7fbf5c105320 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.642+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44010430 con 0x7fbf5c105320 2024-09-21T06:56:45.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.642+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1a9090 con 0x7fbf5c105320 2024-09-21T06:56:45.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.643+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5c06c240 con 0x7fbf5c105320 2024-09-21T06:56:45.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.646+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fbf44005000 con 0x7fbf5c105320 2024-09-21T06:56:45.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.647+0000 7fbf58c00640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf400b6680 0x7fbf400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:45.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.647+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbf440e7fe0 con 0x7fbf5c105320 2024-09-21T06:56:45.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.648+0000 7fbf5b400640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf400b6680 0x7fbf400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:45.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.648+0000 7fbf5b400640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf400b6680 0x7fbf400b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0098d0 tx=0x7fbf4c010040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:45.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.649+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf44056710 con 0x7fbf5c105320 2024-09-21T06:56:45.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.884+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf5c19d800 con 0x7fbf5c105320 2024-09-21T06:56:45.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.885+0000 7fbf58c00640 1 -- 172.21.15.114:0/106190627 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fbf5c19d800 con 0x7fbf5c105320 2024-09-21T06:56:45.888 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:56:45.889 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:56:45.889 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:56:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.888+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf400b6680 msgr2=0x7fbf400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:45.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.888+0000 7fbf62800640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf400b6680 0x7fbf400b8b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0098d0 tx=0x7fbf4c010040 comp rx=0 tx=0).stop 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.888+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 msgr2=0x7fbf5c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 0x7fbf5c19cc70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ee80 comp rx=0 tx=0).stop 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 shutdown_connections 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf5c10de30 0x7fbf5c1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf5c105320 0x7fbf5c19cc70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf400b6680 0x7fbf400b8b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 --2- 172.21.15.114:0/106190627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c1049d0 0x7fbf5c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 >> 172.21.15.114:0/106190627 conn(0x7fbf5c079740 msgr2=0x7fbf5c10f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 shutdown_connections 2024-09-21T06:56:45.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:45.889+0000 7fbf62800640 1 -- 172.21.15.114:0/106190627 wait complete. 2024-09-21T06:56:46.790 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:56:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:46 smithi114 ceph-mon[30654]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-09-21T06:56:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:46 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/106190627' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:46 smithi183 ceph-mon[34219]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-09-21T06:56:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:46 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/106190627' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:47.161 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:47.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:46 smithi193 ceph-mon[34601]: pgmap v475: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-09-21T06:56:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:46 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/106190627' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:47.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.770+0000 7f0b9fe00640 1 -- 172.21.15.114:0/1486490070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b9810fff0 msgr2=0x7f0b98112500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.770+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/1486490070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b9810fff0 0x7f0b98112500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009ca0 tx=0x7f0b9002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 -- 172.21.15.114:0/1486490070 shutdown_connections 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/1486490070 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b9810fff0 0x7f0b98112500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/1486490070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b981024e0 0x7f0b9810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/1486490070 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b98101b00 0x7f0b98101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 -- 172.21.15.114:0/1486490070 >> 172.21.15.114:0/1486490070 conn(0x7f0b98077800 msgr2=0x7f0b98077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 -- 172.21.15.114:0/1486490070 shutdown_connections 2024-09-21T06:56:47.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.771+0000 7f0b9fe00640 1 -- 172.21.15.114:0/1486490070 wait complete. 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 Processor -- start 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 -- start start 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b98101b00 0x7f0b9819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b981024e0 0x7f0b9819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 0x7f0b981a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98115c70 con 0x7f0b9810fff0 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98115af0 con 0x7f0b98101b00 2024-09-21T06:56:47.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.772+0000 7f0b9fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b98115df0 con 0x7f0b981024e0 2024-09-21T06:56:47.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 0x7f0b981a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 0x7f0b981a1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41986/0 (socket says 172.21.15.114:41986) 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 -- 172.21.15.114:0/2802791265 learned_addr learned my addr 172.21.15.114:0/2802791265 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9d400640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b98101b00 0x7f0b9819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b981024e0 0x7f0b9819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 -- 172.21.15.114:0/2802791265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b981024e0 msgr2=0x7f0b9819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b981024e0 0x7f0b9819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 -- 172.21.15.114:0/2802791265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b98101b00 msgr2=0x7f0b9819c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b98101b00 0x7f0b9819c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 -- 172.21.15.114:0/2802791265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b981a2570 con 0x7f0b9810fff0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.773+0000 7f0b9de00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 0x7f0b981a1e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005cc0 tx=0x7f0b90030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.774+0000 7f0b96000640 1 -- 172.21.15.114:0/2802791265 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004040 con 0x7f0b9810fff0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.774+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b981a2800 con 0x7f0b9810fff0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.774+0000 7f0b96000640 1 -- 172.21.15.114:0/2802791265 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b90040070 con 0x7f0b9810fff0 2024-09-21T06:56:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.774+0000 7f0b96000640 1 -- 172.21.15.114:0/2802791265 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90045b10 con 0x7f0b9810fff0 2024-09-21T06:56:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.774+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b981a8fc0 con 0x7f0b9810fff0 2024-09-21T06:56:47.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.777+0000 7f0ba6a44740 1 Processor -- start 2024-09-21T06:56:47.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.777+0000 7f0b96000640 1 -- 172.21.15.114:0/2802791265 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0b90045cd0 con 0x7f0b9810fff0 2024-09-21T06:56:47.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 -- start start 2024-09-21T06:56:47.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 0x5563ba9dd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5563ba953fb0 con 0x5563ba615390 2024-09-21T06:56:47.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563ba9433e0 con 0x5563ba60d1b0 2024-09-21T06:56:47.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563ba9471f0 con 0x5563ba636070 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0b9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.778+0000 7f0b9d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 0x5563ba9dd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 0x5563ba9dd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34924/0 (socket says 172.21.15.114:34924) 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 -- 172.21.15.114:0/2865447934 learned_addr learned my addr 172.21.15.114:0/2865447934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 -- 172.21.15.114:0/2865447934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 msgr2=0x5563ba9dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 -- 172.21.15.114:0/2865447934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 msgr2=0x5563ba9dd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9dd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 -- 172.21.15.114:0/2865447934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ba90dcd0 con 0x5563ba60d1b0 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b96000640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b800b6680 0x7f0b800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9d400640 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 0x5563ba9dd0e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009cf0 tx=0x7f0b8c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6f532cedbe365b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b96000640 1 -- 172.21.15.114:0/2802791265 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b90108ae0 con 0x7f0b9810fff0 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.779+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b800b6680 0x7f0b800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.780+0000 7f0b8a600640 1 -- 172.21.15.114:0/2865447934 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c01e070 con 0x5563ba60d1b0 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.780+0000 7f0b8a600640 1 -- 172.21.15.114:0/2865447934 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8c019ae0 con 0x5563ba60d1b0 2024-09-21T06:56:47.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.780+0000 7f0b8a600640 1 -- 172.21.15.114:0/2865447934 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c00e910 con 0x5563ba60d1b0 2024-09-21T06:56:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.780+0000 7f0ba6a44740 1 -- 172.21.15.114:0/2865447934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 msgr2=0x5563ba9dd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 0x5563ba9dd0e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009cf0 tx=0x7f0b8c011810 comp rx=0 tx=0).stop 2024-09-21T06:56:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.780+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b800b6680 0x7f0b800b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400bfd0 tx=0x7f0b84004730 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 -- 172.21.15.114:0/2865447934 shutdown_connections 2024-09-21T06:56:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba60d1b0 0x5563ba9dd0e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/2865447934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 -- 172.21.15.114:0/2865447934 >> 172.21.15.114:0/2865447934 conn(0x5563ba62a460 msgr2=0x5563ba9daa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 -- 172.21.15.114:0/2865447934 shutdown_connections 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 -- 172.21.15.114:0/2865447934 wait complete. 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.781+0000 7f0ba6a44740 1 -- start start 2024-09-21T06:56:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 Processor -- start 2024-09-21T06:56:47.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9e8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9e8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563ba9f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5563ba7a4700 con 0x5563ba615390 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563ba8de580 con 0x5563ba636070 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563ba9eb380 con 0x5563ba60d1b0 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0ba6a44740 10 client.0 _ll_register_callbacks cb 0x5563ba8ee470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0b9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9e8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0b9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9e8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42006/0 (socket says 172.21.15.114:42006) 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0b9ca00640 1 -- 172.21.15.114:0/244652197 learned_addr learned my addr 172.21.15.114:0/244652197 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0b9de00640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563ba9f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.782+0000 7f0b9d400640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b9de00640 1 -- 172.21.15.114:0/244652197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 msgr2=0x5563ba9e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b9de00640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b9de00640 1 -- 172.21.15.114:0/244652197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 msgr2=0x5563ba9e8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b9de00640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9e8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b9de00640 1 -- 172.21.15.114:0/244652197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ba6a1650 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b9de00640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563ba9f46a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0b90105e60 tx=0x7f0b900d7e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4d7a935220a38d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b7ea00640 1 -- 172.21.15.114:0/244652197 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900cdcd0 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0b7ea00640 1 -- 172.21.15.114:0/244652197 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b900c9eb0 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.783+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5563ba8e0b90 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0b7ea00640 1 -- 172.21.15.114:0/244652197 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b900ce070 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 10 client.26321 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563ba8fb950 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0b7ea00640 1 -- 172.21.15.114:0/244652197 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5563ba8e0b90 con 0x5563ba636070 2024-09-21T06:56:47.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0b7ea00640 1 -- 172.21.15.114:0/244652197 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0b9004d030 con 0x5563ba636070 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 10 client.26321 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 10 client.26321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 20 client.26321 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5f49f0 0x5563ba9fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 4 client.26321 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.784+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ba955f00 con 0x5563ba5f49f0 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.785+0000 7f0b9d400640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5f49f0 0x5563ba9fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.785+0000 7f0b9d400640 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5f49f0 0x5563ba9fa690 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69d8d2290c396fab server_cookie=286bb108cab932b7 in_seq=0 out_seq=0 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.785+0000 7f0b7ea00640 10 client.26321 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.786+0000 7f0b7ea00640 1 -- 172.21.15.114:0/244652197 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563ba955f00 con 0x5563ba5f49f0 2024-09-21T06:56:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.786+0000 7f0b7ea00640 10 client.26321 handle_command_reply: tid=0 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.786+0000 7f0ba6a44740 1 client.26321 shutdown 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.786+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 msgr2=0x5563ba9f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.786+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563ba9f46a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0b90105e60 tx=0x7f0b900d7e60 comp rx=0 tx=0).stop 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 shutdown_connections 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5f49f0 0x5563ba9fa690 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563ba9f46a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0b7ea00640 0 client.26321 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9e8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/244652197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 >> 172.21.15.114:0/244652197 conn(0x5563ba62a460 msgr2=0x5563ba9dfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 shutdown_connections 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 1 -- 172.21.15.114:0/244652197 wait complete. 2024-09-21T06:56:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.787+0000 7f0ba6a44740 20 client.26321 trim_cache size 0 max 16384 2024-09-21T06:56:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.801+0000 7f0ba6a44740 1 Processor -- start 2024-09-21T06:56:47.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.801+0000 7f0ba6a44740 1 -- start start 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.801+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 0x5563ba9f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba615390 0x5563ba9e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b2ea0 con 0x5563ba60d1b0 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b4aa0 con 0x5563ba615390 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b3570 con 0x5563ba636070 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 0x5563ba9f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 0x5563ba9f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42008/0 (socket says 172.21.15.114:42008) 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9d400640 1 -- 172.21.15.114:0/1205668691 learned_addr learned my addr 172.21.15.114:0/1205668691 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9de00640 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba615390 0x5563ba9e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9de00640 1 -- 172.21.15.114:0/1205668691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba615390 msgr2=0x5563ba9e1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9de00640 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba615390 0x5563ba9e1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9de00640 1 -- 172.21.15.114:0/1205668691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 msgr2=0x5563ba9f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9de00640 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 0x5563ba9f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9de00640 1 -- 172.21.15.114:0/1205668691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ba9dfa00 con 0x5563ba636070 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.802+0000 7f0b9d400640 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 0x5563ba9f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.803+0000 7f0b9de00640 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9e21f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b900d7ef0 tx=0x7f0b9002a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da58de119756d7aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.803+0000 7f0b89c00640 1 -- 172.21.15.114:0/1205668691 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900cdcd0 con 0x5563ba636070 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.803+0000 7f0b89c00640 1 -- 172.21.15.114:0/1205668691 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b90036040 con 0x5563ba636070 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.803+0000 7f0b89c00640 1 -- 172.21.15.114:0/1205668691 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900ca400 con 0x5563ba636070 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 -- 172.21.15.114:0/1205668691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 msgr2=0x5563ba9e21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9e21f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0b900d7ef0 tx=0x7f0b9002a8b0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 -- 172.21.15.114:0/1205668691 shutdown_connections 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba636070 0x5563ba9e21f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba615390 0x5563ba9e1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/1205668691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba60d1b0 0x5563ba9f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 -- 172.21.15.114:0/1205668691 >> 172.21.15.114:0/1205668691 conn(0x5563ba62a460 msgr2=0x5563ba9e3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 -- 172.21.15.114:0/1205668691 shutdown_connections 2024-09-21T06:56:47.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 -- 172.21.15.114:0/1205668691 wait complete. 2024-09-21T06:56:47.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.804+0000 7f0ba6a44740 1 -- start start 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 Processor -- start 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9ed1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563baa64090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b4310 con 0x5563ba615390 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b3c40 con 0x5563ba636070 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b981b0e80 con 0x5563ba60d1b0 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0ba6a44740 10 client.0 _ll_register_callbacks cb 0x7f0b900caa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.805+0000 7f0b9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9ed1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9ed1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42018/0 (socket says 172.21.15.114:42018) 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9ca00640 1 -- 172.21.15.114:0/4129763661 learned_addr learned my addr 172.21.15.114:0/4129763661 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9de00640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563baa64090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 -- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 msgr2=0x5563baa64090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563baa64090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 -- 172.21.15.114:0/4129763661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 msgr2=0x5563ba9ed1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9ed1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 -- 172.21.15.114:0/4129763661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563baa645d0 con 0x5563ba60d1b0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b9d400640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9ecc80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00e5a0 tx=0x7f0b8c00e530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61c03aeb9ff09b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.806+0000 7f0b7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0b7ea00640 1 -- 172.21.15.114:0/4129763661 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c01e040 con 0x5563ba60d1b0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0b7ea00640 1 -- 172.21.15.114:0/4129763661 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b8c0051f0 con 0x5563ba60d1b0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b90036040 con 0x5563ba60d1b0 2024-09-21T06:56:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0b7ea00640 1 -- 172.21.15.114:0/4129763661 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c03c050 con 0x5563ba60d1b0 2024-09-21T06:56:47.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 10 client.26281 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:47.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5563baa657b0 con 0x5563ba60d1b0 2024-09-21T06:56:47.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0b7ea00640 1 -- 172.21.15.114:0/4129763661 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b90036040 con 0x5563ba60d1b0 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0b7ea00640 1 -- 172.21.15.114:0/4129763661 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0b8c00ec10 con 0x5563ba60d1b0 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 10 client.26281 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 10 client.26281 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 20 client.26281 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5c9da0 0x5563baa67de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 4 client.26281 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.807+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563ba9e90a0 con 0x5563ba5c9da0 2024-09-21T06:56:47.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.808+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5c9da0 0x5563baa67de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.808+0000 7f0b9ca00640 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5c9da0 0x5563baa67de0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3230df4d8466b0b server_cookie=5a90b85e3c2f0a50 in_seq=0 out_seq=0 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.808+0000 7f0b7ea00640 10 client.26281 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0b7ea00640 1 -- 172.21.15.114:0/4129763661 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563ba9e90a0 con 0x5563ba5c9da0 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0b7ea00640 10 client.26281 handle_command_reply: tid=0 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 client.26281 shutdown 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 msgr2=0x5563ba9ecc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9ecc80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00e5a0 tx=0x7f0b8c00e530 comp rx=0 tx=0).stop 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 shutdown_connections 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5563ba5c9da0 0x5563baa67de0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563ba636070 0x5563baa64090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563ba615390 0x5563ba9ed1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0b7ea00640 0 client.26281 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 --2- 172.21.15.114:0/4129763661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563ba60d1b0 0x5563ba9ecc80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 >> 172.21.15.114:0/4129763661 conn(0x5563ba62a460 msgr2=0x5563ba9e1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:47.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 shutdown_connections 2024-09-21T06:56:47.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 1 -- 172.21.15.114:0/4129763661 wait complete. 2024-09-21T06:56:47.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.809+0000 7f0ba6a44740 20 client.26281 trim_cache size 0 max 16384 2024-09-21T06:56:47.814 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b800b6680 msgr2=0x7f0b800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b800b6680 0x7f0b800b8b90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400bfd0 tx=0x7f0b84004730 comp rx=0 tx=0).stop 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 msgr2=0x7f0b981a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 0x7f0b981a1e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0b90005cc0 tx=0x7f0b90030d70 comp rx=0 tx=0).stop 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 shutdown_connections 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b9810fff0 0x7f0b981a1e40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b981024e0 0x7f0b9819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b800b6680 0x7f0b800b8b90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 --2- 172.21.15.114:0/2802791265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b98101b00 0x7f0b9819c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 >> 172.21.15.114:0/2802791265 conn(0x7f0b98077800 msgr2=0x7f0b980ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 shutdown_connections 2024-09-21T06:56:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:47.810+0000 7f0b9fe00640 1 -- 172.21.15.114:0/2802791265 wait complete. 2024-09-21T06:56:47.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 130.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:56:48.721 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:56:49.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:48 smithi114 ceph-mon[30654]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-21T06:56:49.092 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:48 smithi183 ceph-mon[34219]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-21T06:56:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:48 smithi193 ceph-mon[34601]: pgmap v476: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-09-21T06:56:49.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 -- 172.21.15.114:0/1517971285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca8110040 msgr2=0x7f9ca8112550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 --2- 172.21.15.114:0/1517971285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca8110040 0x7f9ca8112550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009ca0 tx=0x7f9ca002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 -- 172.21.15.114:0/1517971285 shutdown_connections 2024-09-21T06:56:49.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 --2- 172.21.15.114:0/1517971285 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca8110040 0x7f9ca8112550 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 --2- 172.21.15.114:0/1517971285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca806c0e0 0x7f9ca806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 --2- 172.21.15.114:0/1517971285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca81046a0 0x7f9ca806bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 -- 172.21.15.114:0/1517971285 >> 172.21.15.114:0/1517971285 conn(0x7f9ca80799a0 msgr2=0x7f9ca8079db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:49.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 -- 172.21.15.114:0/1517971285 shutdown_connections 2024-09-21T06:56:49.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.855+0000 7f9cada00640 1 -- 172.21.15.114:0/1517971285 wait complete. 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.856+0000 7f9cada00640 1 Processor -- start 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.856+0000 7f9cada00640 1 -- start start 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.856+0000 7f9cada00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 0x7f9ca819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9cada00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca81046a0 0x7f9ca819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9cada00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca8110040 0x7f9ca81a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9cada00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8115cc0 con 0x7f9ca806c0e0 2024-09-21T06:56:49.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9cada00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8115b40 con 0x7f9ca8110040 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9cada00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8115e40 con 0x7f9ca81046a0 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 0x7f9ca819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca8110040 0x7f9ca81a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca81046a0 0x7f9ca819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 0x7f9ca819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42040/0 (socket says 172.21.15.114:42040) 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca6a00640 1 -- 172.21.15.114:0/2854393158 learned_addr learned my addr 172.21.15.114:0/2854393158 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca7400640 1 -- 172.21.15.114:0/2854393158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca81046a0 msgr2=0x7f9ca819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca7400640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca81046a0 0x7f9ca819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca7400640 1 -- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 msgr2=0x7f9ca819c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca7400640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 0x7f9ca819c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca7400640 1 -- 172.21.15.114:0/2854393158 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca81a2580 con 0x7f9ca8110040 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.857+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 0x7f9ca819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:49.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.858+0000 7f9ca7400640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca8110040 0x7f9ca81a1e50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ca002ade0 tx=0x7f9ca00316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:49.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.858+0000 7f9c9fe00640 1 -- 172.21.15.114:0/2854393158 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0003b70 con 0x7f9ca8110040 2024-09-21T06:56:49.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.858+0000 7f9c9fe00640 1 -- 172.21.15.114:0/2854393158 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca0003d10 con 0x7f9ca8110040 2024-09-21T06:56:49.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.858+0000 7f9c9fe00640 1 -- 172.21.15.114:0/2854393158 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca002f9e0 con 0x7f9ca8110040 2024-09-21T06:56:49.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.858+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca81a2810 con 0x7f9ca8110040 2024-09-21T06:56:49.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.858+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca81a8fd0 con 0x7f9ca8110040 2024-09-21T06:56:49.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.861+0000 7f9cb4608740 1 Processor -- start 2024-09-21T06:56:49.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9c9fe00640 1 -- 172.21.15.114:0/2854393158 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9ca002fb80 con 0x7f9ca8110040 2024-09-21T06:56:49.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 -- start start 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56246a33cfb0 con 0x562469ff61b0 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56246a32c3e0 con 0x562469ffe390 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9cb4608740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56246a3301f0 con 0x56246a01f070 2024-09-21T06:56:49.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.862+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42050/0 (socket says 172.21.15.114:42050) 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3746247292 learned_addr learned my addr 172.21.15.114:0/3746247292 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9c9e000640 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3746247292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 msgr2=0x56246a3c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3746247292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 msgr2=0x56246a3c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3746247292 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56246a2f6cd0 con 0x562469ff61b0 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.863+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3c62e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9c980115c0 tx=0x7f9c98011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04dd421ea0fcbed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.864+0000 7f9c9fe00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c8c0b6700 0x7f9c8c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.864+0000 7f9c97e00640 1 -- 172.21.15.114:0/3746247292 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800f660 con 0x562469ff61b0 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.864+0000 7f9c9fe00640 1 -- 172.21.15.114:0/2854393158 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9ca0108af0 con 0x7f9ca8110040 2024-09-21T06:56:49.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.864+0000 7f9c97e00640 1 -- 172.21.15.114:0/3746247292 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9800fcb0 con 0x562469ff61b0 2024-09-21T06:56:49.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.864+0000 7f9c97e00640 1 -- 172.21.15.114:0/3746247292 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98019c70 con 0x562469ff61b0 2024-09-21T06:56:49.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.864+0000 7f9c9e000640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c8c0b6700 0x7f9c8c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 -- 172.21.15.114:0/3746247292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 msgr2=0x56246a3c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3c62e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9c980115c0 tx=0x7f9c98011a80 comp rx=0 tx=0).stop 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9c9e000640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c8c0b6700 0x7f9c8c0b8c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000bfd0 tx=0x7f9c9000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 -- 172.21.15.114:0/3746247292 shutdown_connections 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3c62e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3746247292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 -- 172.21.15.114:0/3746247292 >> 172.21.15.114:0/3746247292 conn(0x56246a013460 msgr2=0x56246a3c3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:49.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 -- 172.21.15.114:0/3746247292 shutdown_connections 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 -- 172.21.15.114:0/3746247292 wait complete. 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.865+0000 7f9cb4608740 1 -- start start 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 Processor -- start 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ffe390 0x56246a3d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56246a18d700 con 0x562469ffe390 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56246a2c7580 con 0x562469ff61b0 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56246a3d4580 con 0x56246a01f070 2024-09-21T06:56:49.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.866+0000 7f9cb4608740 10 client.0 _ll_register_callbacks cb 0x56246a2d7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ffe390 0x56246a3d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3dd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56202/0 (socket says 172.21.15.114:56202) 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ffe390 0x56246a3d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42056/0 (socket says 172.21.15.114:42056) 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 -- 172.21.15.114:0/1203232495 learned_addr learned my addr 172.21.15.114:0/1203232495 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 -- 172.21.15.114:0/1203232495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 msgr2=0x56246a3d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 -- 172.21.15.114:0/1203232495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ffe390 msgr2=0x56246a3d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ffe390 0x56246a3d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 -- 172.21.15.114:0/1203232495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56246a08a650 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9ca7400640 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3dd8a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00cab40 tx=0x7f9ca00ca540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3a27d3b5315eea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.867+0000 7f9c94c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9c94c00640 1 -- 172.21.15.114:0/1203232495 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00ce080 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56246a2c9b90 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9c94c00640 1 -- 172.21.15.114:0/1203232495 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9ca00d7d10 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9c94c00640 1 -- 172.21.15.114:0/1203232495 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ca00cec70 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9cb4608740 10 client.26293 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56246a2e4950 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9c94c00640 1 -- 172.21.15.114:0/1203232495 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56246a2c9b90 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.868+0000 7f9c94c00640 1 -- 172.21.15.114:0/1203232495 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9ca00c9ec0 con 0x56246a01f070 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9cb4608740 10 client.26293 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:49.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9cb4608740 10 client.26293 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9cb4608740 20 client.26293 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9cb4608740 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fdd9f0 0x56246a3e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9cb4608740 4 client.26293 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56246a33ef00 con 0x562469fdd9f0 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fdd9f0 0x56246a3e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fdd9f0 0x56246a3e3600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1c78d9d292664eda server_cookie=89cec33b23adf8f4 in_seq=0 out_seq=0 2024-09-21T06:56:49.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.869+0000 7f9c94c00640 10 client.26293 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.870+0000 7f9c94c00640 1 -- 172.21.15.114:0/1203232495 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56246a33ef00 con 0x562469fdd9f0 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.870+0000 7f9c94c00640 10 client.26293 handle_command_reply: tid=0 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 client.26293 shutdown 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 msgr2=0x56246a3dd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3dd8a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00cab40 tx=0x7f9ca00ca540 comp rx=0 tx=0).stop 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 shutdown_connections 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fdd9f0 0x56246a3e3600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3dd8a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ffe390 0x56246a3d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 --2- 172.21.15.114:0/1203232495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9c94c00640 0 client.26293 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:56:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 >> 172.21.15.114:0/1203232495 conn(0x56246a013460 msgr2=0x56246a3c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 shutdown_connections 2024-09-21T06:56:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 1 -- 172.21.15.114:0/1203232495 wait complete. 2024-09-21T06:56:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.871+0000 7f9cb4608740 20 client.26293 trim_cache size 0 max 16384 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 Processor -- start 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 -- start start 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3dd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81b2fd0 con 0x562469ff61b0 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81b4bd0 con 0x562469ffe390 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9cb4608740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81b36a0 con 0x56246a01f070 2024-09-21T06:56:49.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3d31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42066/0 (socket says 172.21.15.114:42066) 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3277869163 learned_addr learned my addr 172.21.15.114:0/3277869163 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.885+0000 7f9c9e000640 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3dd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca7400640 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3277869163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 msgr2=0x56246a3ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3ddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3277869163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 msgr2=0x56246a3dd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3dd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca6a00640 1 -- 172.21.15.114:0/3277869163 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56246a3c8090 con 0x562469ff61b0 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca7400640 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3d31b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009380 tx=0x7f9c98019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b193e3560220b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9c97400640 1 -- 172.21.15.114:0/3277869163 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98005130 con 0x562469ff61b0 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9c97400640 1 -- 172.21.15.114:0/3277869163 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c98019880 con 0x562469ff61b0 2024-09-21T06:56:49.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.886+0000 7f9c97400640 1 -- 172.21.15.114:0/3277869163 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800f970 con 0x562469ff61b0 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 -- 172.21.15.114:0/3277869163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 msgr2=0x56246a3d31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3d31b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009380 tx=0x7f9c98019770 comp rx=0 tx=0).stop 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 -- 172.21.15.114:0/3277869163 shutdown_connections 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56246a01f070 0x56246a3ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ffe390 0x56246a3dd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3277869163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562469ff61b0 0x56246a3d31b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 -- 172.21.15.114:0/3277869163 >> 172.21.15.114:0/3277869163 conn(0x56246a013460 msgr2=0x56246a3d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 -- 172.21.15.114:0/3277869163 shutdown_connections 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.887+0000 7f9cb4608740 1 -- 172.21.15.114:0/3277869163 wait complete. 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.888+0000 7f9cb4608740 1 -- start start 2024-09-21T06:56:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.888+0000 7f9cb4608740 1 Processor -- start 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562469ffe390 0x56246a3d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 0x56246a446fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81b4440 con 0x56246a01f070 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81b3d70 con 0x562469ff61b0 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca81b0fb0 con 0x562469ffe390 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9cb4608740 10 client.0 _ll_register_callbacks cb 0x7f9ca0106320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562469ffe390 0x56246a3d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 0x56246a446fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 0x56246a446fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42078/0 (socket says 172.21.15.114:42078) 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9c9e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562469ffe390 0x56246a3d6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56228/0 (socket says 172.21.15.114:56228) 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 -- 172.21.15.114:0/3768754176 learned_addr learned my addr 172.21.15.114:0/3768754176 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 -- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562469ffe390 msgr2=0x56246a3d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562469ffe390 0x56246a3d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 -- 172.21.15.114:0/3768754176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 msgr2=0x56246a3d60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.889+0000 7f9ca7400640 1 -- 172.21.15.114:0/3768754176 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56246a447650 con 0x56246a01f070 2024-09-21T06:56:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9ca7400640 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 0x56246a446fe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00cab40 tx=0x7f9ca00ca410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb5c68a2728078 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9c94c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9c94c00640 1 -- 172.21.15.114:0/3768754176 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00ce080 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c98019880 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9c94c00640 1 -- 172.21.15.114:0/3768754176 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9ca0004b40 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9c94c00640 1 -- 172.21.15.114:0/3768754176 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ca00d3050 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9cb4608740 10 client.16212 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56246a44e9a0 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.890+0000 7f9c94c00640 1 -- 172.21.15.114:0/3768754176 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c98019880 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9c94c00640 1 -- 172.21.15.114:0/3768754176 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9ca00f00d0 con 0x56246a01f070 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9cb4608740 10 client.16212 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9cb4608740 10 client.16212 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9cb4608740 20 client.16212 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fb2da0 0x56246a451000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9cb4608740 4 client.16212 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56246a33a9b0 con 0x562469fb2da0 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.891+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fb2da0 0x56246a451000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9ca6a00640 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fb2da0 0x56246a451000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b93e38e0e31a5f3e server_cookie=253fd01b6bf0e3f5 in_seq=0 out_seq=0 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9c94c00640 10 client.16212 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9c94c00640 1 -- 172.21.15.114:0/3768754176 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56246a33a9b0 con 0x562469fb2da0 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9c94c00640 10 client.16212 handle_command_reply: tid=0 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 client.16212 shutdown 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 msgr2=0x56246a446fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 0x56246a446fe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00cab40 tx=0x7f9ca00ca410 comp rx=0 tx=0).stop 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 shutdown_connections 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x562469fb2da0 0x56246a451000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56246a01f070 0x56246a446fe0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562469ffe390 0x56246a3d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 --2- 172.21.15.114:0/3768754176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562469ff61b0 0x56246a3d60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9c94c00640 0 client.16212 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:56:49.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.892+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 >> 172.21.15.114:0/3768754176 conn(0x56246a013460 msgr2=0x56246a3d2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:49.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 shutdown_connections 2024-09-21T06:56:49.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cb4608740 1 -- 172.21.15.114:0/3768754176 wait complete. 2024-09-21T06:56:49.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cb4608740 20 client.16212 trim_cache size 0 max 16384 2024-09-21T06:56:49.898 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c8c0b6700 msgr2=0x7f9c8c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cada00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c8c0b6700 0x7f9c8c0b8c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000bfd0 tx=0x7f9c9000b410 comp rx=0 tx=0).stop 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca8110040 msgr2=0x7f9ca81a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.893+0000 7f9cada00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca8110040 0x7f9ca81a1e50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9ca002ade0 tx=0x7f9ca00316f0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 shutdown_connections 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c8c0b6700 0x7f9c8c0b8c10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ca8110040 0x7f9ca81a1e50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ca81046a0 0x7f9ca819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 --2- 172.21.15.114:0/2854393158 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9ca806c0e0 0x7f9ca819c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 >> 172.21.15.114:0/2854393158 conn(0x7f9ca80799a0 msgr2=0x7f9ca81014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 shutdown_connections 2024-09-21T06:56:49.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:49.894+0000 7f9cada00640 1 -- 172.21.15.114:0/2854393158 wait complete. 2024-09-21T06:56:50.868 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:56:51.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:50 smithi114 ceph-mon[30654]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 816 KiB/s wr, 48 op/s 2024-09-21T06:56:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:50 smithi183 ceph-mon[34219]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 816 KiB/s wr, 48 op/s 2024-09-21T06:56:51.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:50 smithi193 ceph-mon[34601]: pgmap v477: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 816 KiB/s wr, 48 op/s 2024-09-21T06:56:51.237 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 -- 172.21.15.114:0/2108115142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd73810c020 msgr2=0x7fd738112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 --2- 172.21.15.114:0/2108115142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd73810c020 0x7fd738112850 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd73002a410 comp rx=0 tx=0).stop 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 -- 172.21.15.114:0/2108115142 shutdown_connections 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 --2- 172.21.15.114:0/2108115142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd73810c020 0x7fd738112850 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 --2- 172.21.15.114:0/2108115142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73810b6d0 0x7fd73810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 --2- 172.21.15.114:0/2108115142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd738105d90 0x7fd7381061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 -- 172.21.15.114:0/2108115142 >> 172.21.15.114:0/2108115142 conn(0x7fd7381017c0 msgr2=0x7fd738103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 -- 172.21.15.114:0/2108115142 shutdown_connections 2024-09-21T06:56:51.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.878+0000 7fd73e200640 1 -- 172.21.15.114:0/2108115142 wait complete. 2024-09-21T06:56:51.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.879+0000 7fd73e200640 1 Processor -- start 2024-09-21T06:56:51.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.879+0000 7fd73e200640 1 -- start start 2024-09-21T06:56:51.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd73e200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 0x7fd73819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd73e200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd73810b6d0 0x7fd73819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd73e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73810c020 0x7fd7381a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd73e200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd738115d30 con 0x7fd738105d90 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd73e200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd738115bb0 con 0x7fd73810b6d0 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd73e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd738115eb0 con 0x7fd73810c020 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 0x7fd73819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 0x7fd73819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42090/0 (socket says 172.21.15.114:42090) 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 -- 172.21.15.114:0/1157676235 learned_addr learned my addr 172.21.15.114:0/1157676235 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737e00640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73810c020 0x7fd7381a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd73810b6d0 0x7fd73819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 -- 172.21.15.114:0/1157676235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73810c020 msgr2=0x7fd7381a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73810c020 0x7fd7381a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 -- 172.21.15.114:0/1157676235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd73810b6d0 msgr2=0x7fd73819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd73810b6d0 0x7fd73819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.880+0000 7fd737400640 1 -- 172.21.15.114:0/1157676235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7381a2670 con 0x7fd738105d90 2024-09-21T06:56:51.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.881+0000 7fd737400640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 0x7fd73819c700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd72800e9f0 tx=0x7fd72800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:51.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.881+0000 7fd734c00640 1 -- 172.21.15.114:0/1157676235 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72800cd80 con 0x7fd738105d90 2024-09-21T06:56:51.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.881+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7381a2960 con 0x7fd738105d90 2024-09-21T06:56:51.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.881+0000 7fd734c00640 1 -- 172.21.15.114:0/1157676235 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7280108e0 con 0x7fd738105d90 2024-09-21T06:56:51.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.881+0000 7fd734c00640 1 -- 172.21.15.114:0/1157676235 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728018980 con 0x7fd738105d90 2024-09-21T06:56:51.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.881+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7381a90c0 con 0x7fd738105d90 2024-09-21T06:56:51.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.885+0000 7fd744e20740 1 Processor -- start 2024-09-21T06:56:51.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 -- start start 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a862da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4931b0 0x56537a8632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a863820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56537a7d9fb0 con 0x56537a4931b0 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56537a7c93e0 con 0x56537a49b390 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd744e20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56537a7cd1f0 con 0x56537a4bc070 2024-09-21T06:56:51.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd737e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a862da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd737e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a862da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56248/0 (socket says 172.21.15.114:56248) 2024-09-21T06:56:51.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.886+0000 7fd737e00640 1 -- 172.21.15.114:0/1680024760 learned_addr learned my addr 172.21.15.114:0/1680024760 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:51.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd734c00640 1 -- 172.21.15.114:0/1157676235 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd728010430 con 0x7fd738105d90 2024-09-21T06:56:51.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a863820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737e00640 1 -- 172.21.15.114:0/1680024760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 msgr2=0x56537a863820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737e00640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a863820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737e00640 1 -- 172.21.15.114:0/1680024760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4931b0 msgr2=0x56537a8632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737400640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4931b0 0x56537a8632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737e00640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4931b0 0x56537a8632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737e00640 1 -- 172.21.15.114:0/1680024760 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537a793cd0 con 0x56537a4bc070 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a863820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737400640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4931b0 0x56537a8632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.887+0000 7fd737e00640 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a862da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd730005f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4a282095f8739b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.888+0000 7fd727e00640 1 -- 172.21.15.114:0/1680024760 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730040070 con 0x56537a4bc070 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.888+0000 7fd727e00640 1 -- 172.21.15.114:0/1680024760 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7300044c0 con 0x56537a4bc070 2024-09-21T06:56:51.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.888+0000 7fd727e00640 1 -- 172.21.15.114:0/1680024760 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730035040 con 0x56537a4bc070 2024-09-21T06:56:51.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.888+0000 7fd734c00640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd71c0b6680 0x7fd71c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd734c00640 1 -- 172.21.15.114:0/1157676235 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd7280e7e20 con 0x7fd738105d90 2024-09-21T06:56:51.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd737400640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd71c0b6680 0x7fd71c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 -- 172.21.15.114:0/1680024760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 msgr2=0x56537a862da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a862da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd730009900 tx=0x7fd730005f30 comp rx=0 tx=0).stop 2024-09-21T06:56:51.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 -- 172.21.15.114:0/1680024760 shutdown_connections 2024-09-21T06:56:51.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a863820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4931b0 0x56537a8632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 --2- 172.21.15.114:0/1680024760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a862da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 -- 172.21.15.114:0/1680024760 >> 172.21.15.114:0/1680024760 conn(0x56537a4b0460 msgr2=0x56537a860c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:51.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 -- 172.21.15.114:0/1680024760 shutdown_connections 2024-09-21T06:56:51.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd744e20740 1 -- 172.21.15.114:0/1680024760 wait complete. 2024-09-21T06:56:51.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.889+0000 7fd737400640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd71c0b6680 0x7fd71c0b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56537a861a20 tx=0x7fd728000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:51.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 -- start start 2024-09-21T06:56:51.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 Processor -- start 2024-09-21T06:56:51.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a86e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 0x56537a86eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a87a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56537a62a700 con 0x56537a49b390 2024-09-21T06:56:51.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56537a764580 con 0x56537a4931b0 2024-09-21T06:56:51.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56537a871520 con 0x56537a4bc070 2024-09-21T06:56:51.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.890+0000 7fd744e20740 10 client.0 _ll_register_callbacks cb 0x56537a774470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:51.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 0x56537a86eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 0x56537a86eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42122/0 (socket says 172.21.15.114:42122) 2024-09-21T06:56:51.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 -- 172.21.15.114:0/1736492659 learned_addr learned my addr 172.21.15.114:0/1736492659 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd737e00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a87a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd737e00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a86e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 -- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 msgr2=0x56537a87a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a87a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 -- 172.21.15.114:0/1736492659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 msgr2=0x56537a86e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a86e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 -- 172.21.15.114:0/1736492659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537a527650 con 0x56537a49b390 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd737e00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a86e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 0x56537a86eb40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd720011970 tx=0x7fd720011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eb60ddb69eaee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.891+0000 7fd724c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd724c00640 1 -- 172.21.15.114:0/1736492659 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72001f060 con 0x56537a49b390 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd724c00640 1 -- 172.21.15.114:0/1736492659 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd72000fcd0 con 0x56537a49b390 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56537a766b90 con 0x56537a49b390 2024-09-21T06:56:51.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd724c00640 1 -- 172.21.15.114:0/1736492659 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd720004b20 con 0x56537a49b390 2024-09-21T06:56:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd744e20740 10 client.16221 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56537a781950 con 0x56537a49b390 2024-09-21T06:56:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.892+0000 7fd724c00640 1 -- 172.21.15.114:0/1736492659 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56537a766b90 con 0x56537a49b390 2024-09-21T06:56:51.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd724c00640 1 -- 172.21.15.114:0/1736492659 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd720004350 con 0x56537a49b390 2024-09-21T06:56:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd744e20740 10 client.16221 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd744e20740 10 client.16221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:56:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd744e20740 20 client.16221 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd744e20740 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a47a9f0 0x56537a880660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd744e20740 4 client.16221 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:51.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537a7dbf00 con 0x56537a47a9f0 2024-09-21T06:56:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.893+0000 7fd737e00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a47a9f0 0x56537a880660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.894+0000 7fd737e00640 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a47a9f0 0x56537a880660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f93525b23bb1367 server_cookie=732dd954c1790ff6 in_seq=0 out_seq=0 2024-09-21T06:56:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.894+0000 7fd724c00640 10 client.16221 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:56:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd724c00640 1 -- 172.21.15.114:0/1736492659 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56537a7dbf00 con 0x56537a47a9f0 2024-09-21T06:56:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd724c00640 10 client.16221 handle_command_reply: tid=0 2024-09-21T06:56:51.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 client.16221 shutdown 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 msgr2=0x56537a86eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 0x56537a86eb40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd720011970 tx=0x7fd720011e30 comp rx=0 tx=0).stop 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 shutdown_connections 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a47a9f0 0x56537a880660 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4bc070 0x56537a87a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a49b390 0x56537a86eb40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 --2- 172.21.15.114:0/1736492659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a86e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 >> 172.21.15.114:0/1736492659 conn(0x56537a4b0460 msgr2=0x56537a865f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd724c00640 0 client.16221 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:56:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 shutdown_connections 2024-09-21T06:56:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 1 -- 172.21.15.114:0/1736492659 wait complete. 2024-09-21T06:56:51.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.895+0000 7fd744e20740 20 client.16221 trim_cache size 0 max 16384 2024-09-21T06:56:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.909+0000 7fd744e20740 1 Processor -- start 2024-09-21T06:56:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.909+0000 7fd744e20740 1 -- start start 2024-09-21T06:56:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd744e20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4931b0 0x56537a86eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd744e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a865ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd744e20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd744e20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd7381b30a0 con 0x56537a4bc070 2024-09-21T06:56:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd744e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7381b4ca0 con 0x56537a49b390 2024-09-21T06:56:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd744e20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7381b3770 con 0x56537a4931b0 2024-09-21T06:56:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd737e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4931b0 0x56537a86eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd737e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a865ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a865ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35026/0 (socket says 172.21.15.114:35026) 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 -- 172.21.15.114:0/4279014398 learned_addr learned my addr 172.21.15.114:0/4279014398 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd737e00640 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8663e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42136/0 (socket says 172.21.15.114:42136) 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 -- 172.21.15.114:0/4279014398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4931b0 msgr2=0x56537a86eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4931b0 0x56537a86eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 -- 172.21.15.114:0/4279014398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 msgr2=0x56537a8663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.910+0000 7fd72ea00640 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8663e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.911+0000 7fd72ea00640 1 -- 172.21.15.114:0/4279014398 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537a865380 con 0x56537a49b390 2024-09-21T06:56:51.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.911+0000 7fd72ea00640 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a865ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd720005f40 tx=0x7fd7200095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df163e9ade9069a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:51.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.911+0000 7fd727400640 1 -- 172.21.15.114:0/4279014398 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72002e020 con 0x56537a49b390 2024-09-21T06:56:51.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.911+0000 7fd727400640 1 -- 172.21.15.114:0/4279014398 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd72000fd90 con 0x56537a49b390 2024-09-21T06:56:51.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.911+0000 7fd727400640 1 -- 172.21.15.114:0/4279014398 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72002ace0 con 0x56537a49b390 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 -- 172.21.15.114:0/4279014398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 msgr2=0x56537a865ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a865ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd720005f40 tx=0x7fd7200095f0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 -- 172.21.15.114:0/4279014398 shutdown_connections 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a49b390 0x56537a865ea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 --2- 172.21.15.114:0/4279014398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a4931b0 0x56537a86eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 -- 172.21.15.114:0/4279014398 >> 172.21.15.114:0/4279014398 conn(0x56537a4b0460 msgr2=0x56537a86e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 -- 172.21.15.114:0/4279014398 shutdown_connections 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.912+0000 7fd744e20740 1 -- 172.21.15.114:0/4279014398 wait complete. 2024-09-21T06:56:51.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 -- start start 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 Processor -- start 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a872750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a49b390 0x56537a872c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd7381b4510 con 0x56537a4bc070 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7381b3e40 con 0x56537a4931b0 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7381b1080 con 0x56537a49b390 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.913+0000 7fd744e20740 10 client.0 _ll_register_callbacks cb 0x7fd720011580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd737e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd737e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42140/0 (socket says 172.21.15.114:42140) 2024-09-21T06:56:51.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd737e00640 1 -- 172.21.15.114:0/1168008434 learned_addr learned my addr 172.21.15.114:0/1168008434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a49b390 0x56537a872c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd737e00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a872750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 -- 172.21.15.114:0/1168008434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 msgr2=0x56537a872750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a872750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 -- 172.21.15.114:0/1168008434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 msgr2=0x56537a8731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8731d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 -- 172.21.15.114:0/1168008434 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537a8ea070 con 0x56537a49b390 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd72ea00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a49b390 0x56537a872c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56537a7c5460 tx=0x7fd72000ff40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbe6b876211967cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.914+0000 7fd724c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd724c00640 1 -- 172.21.15.114:0/1168008434 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72002e040 con 0x56537a49b390 2024-09-21T06:56:51.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72000fd90 con 0x56537a49b390 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd724c00640 1 -- 172.21.15.114:0/1168008434 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd72002a480 con 0x56537a49b390 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd724c00640 1 -- 172.21.15.114:0/1168008434 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd72001f020 con 0x56537a49b390 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd744e20740 10 client.26317 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56537a8eb280 con 0x56537a49b390 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.915+0000 7fd724c00640 1 -- 172.21.15.114:0/1168008434 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd72000fd90 con 0x56537a49b390 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd724c00640 1 -- 172.21.15.114:0/1168008434 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd720010b50 con 0x56537a49b390 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd744e20740 10 client.26317 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd744e20740 10 client.26317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd744e20740 20 client.26317 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd744e20740 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a44fda0 0x56537a8ed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd744e20740 4 client.26317 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56537a7d79b0 con 0x56537a44fda0 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd737e00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a44fda0 0x56537a8ed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:51.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd737e00640 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a44fda0 0x56537a8ed960 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8246b8b458f7d134 server_cookie=34782f422aed2832 in_seq=0 out_seq=0 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.916+0000 7fd724c00640 10 client.26317 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd724c00640 1 -- 172.21.15.114:0/1168008434 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56537a7d79b0 con 0x56537a44fda0 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd724c00640 10 client.26317 handle_command_reply: tid=0 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 client.26317 shutdown 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a49b390 msgr2=0x56537a872c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a49b390 0x56537a872c90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56537a7c5460 tx=0x7fd72000ff40 comp rx=0 tx=0).stop 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 shutdown_connections 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x56537a44fda0 0x56537a8ed960 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56537a4bc070 0x56537a8731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56537a49b390 0x56537a872c90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 --2- 172.21.15.114:0/1168008434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56537a4931b0 0x56537a872750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd724c00640 0 client.26317 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 >> 172.21.15.114:0/1168008434 conn(0x56537a4b0460 msgr2=0x56537a865ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 shutdown_connections 2024-09-21T06:56:51.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 1 -- 172.21.15.114:0/1168008434 wait complete. 2024-09-21T06:56:51.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.917+0000 7fd744e20740 20 client.26317 trim_cache size 0 max 16384 2024-09-21T06:56:51.925 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:56:51.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd71c0b6680 msgr2=0x7fd71c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd71c0b6680 0x7fd71c0b8b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x56537a861a20 tx=0x7fd728000bc0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 msgr2=0x7fd73819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:51.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 0x7fd73819c700 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd72800e9f0 tx=0x7fd72800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 shutdown_connections 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd73810c020 0x7fd7381a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd71c0b6680 0x7fd71c0b8b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd73810b6d0 0x7fd73819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 --2- 172.21.15.114:0/1157676235 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd738105d90 0x7fd73819c700 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 >> 172.21.15.114:0/1157676235 conn(0x7fd7381017c0 msgr2=0x7fd73810aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 shutdown_connections 2024-09-21T06:56:51.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:51.918+0000 7fd73e200640 1 -- 172.21.15.114:0/1157676235 wait complete. 2024-09-21T06:56:52.792 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:56:52.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 135.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:56:53.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:52 smithi114 ceph-mon[30654]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 225 KiB/s wr, 24 op/s 2024-09-21T06:56:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:52 smithi183 ceph-mon[34219]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 225 KiB/s wr, 24 op/s 2024-09-21T06:56:53.160 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:53.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:52 smithi193 ceph-mon[34601]: pgmap v478: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 225 KiB/s wr, 24 op/s 2024-09-21T06:56:53.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 -- 172.21.15.114:0/2502766614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 msgr2=0x7fd6a4112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 --2- 172.21.15.114:0/2502766614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a4112840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009ca0 tx=0x7fd69c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 -- 172.21.15.114:0/2502766614 shutdown_connections 2024-09-21T06:56:53.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 --2- 172.21.15.114:0/2502766614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a4112840 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 --2- 172.21.15.114:0/2502766614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a410b6c0 0x7fd6a410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 --2- 172.21.15.114:0/2502766614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a4105d80 0x7fd6a4106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 -- 172.21.15.114:0/2502766614 >> 172.21.15.114:0/2502766614 conn(0x7fd6a4101810 msgr2=0x7fd6a4103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:53.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 -- 172.21.15.114:0/2502766614 shutdown_connections 2024-09-21T06:56:53.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.777+0000 7fd6a9800640 1 -- 172.21.15.114:0/2502766614 wait complete. 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.778+0000 7fd6a9800640 1 Processor -- start 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 -- start start 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4105d80 0x7fd6a419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a410b6c0 0x7fd6a419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a41a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4115cd0 con 0x7fd6a410c010 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4115b50 con 0x7fd6a410b6c0 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a9800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a4115e50 con 0x7fd6a4105d80 2024-09-21T06:56:53.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a41a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a41a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42160/0 (socket says 172.21.15.114:42160) 2024-09-21T06:56:53.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a3400640 1 -- 172.21.15.114:0/3089764726 learned_addr learned my addr 172.21.15.114:0/3089764726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:53.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd6a2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4105d80 0x7fd6a419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.779+0000 7fd69a000640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a410b6c0 0x7fd6a419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a2a00640 1 -- 172.21.15.114:0/3089764726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a410b6c0 msgr2=0x7fd6a419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a410b6c0 0x7fd6a419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a2a00640 1 -- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 msgr2=0x7fd6a41a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a41a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a2a00640 1 -- 172.21.15.114:0/3089764726 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a41a2690 con 0x7fd6a4105d80 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a3400640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a41a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.780+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4105d80 0x7fd6a419c720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e9f0 tx=0x7fd69400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.781+0000 7fd69be00640 1 -- 172.21.15.114:0/3089764726 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69400cd80 con 0x7fd6a4105d80 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.781+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a41a2980 con 0x7fd6a4105d80 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.781+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a41a90e0 con 0x7fd6a4105d80 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.781+0000 7fd69be00640 1 -- 172.21.15.114:0/3089764726 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6940035f0 con 0x7fd6a4105d80 2024-09-21T06:56:53.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.781+0000 7fd69be00640 1 -- 172.21.15.114:0/3089764726 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694018690 con 0x7fd6a4105d80 2024-09-21T06:56:53.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.784+0000 7fd6b0445740 1 Processor -- start 2024-09-21T06:56:53.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.784+0000 7fd6b0445740 1 -- start start 2024-09-21T06:56:53.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc3f070 0x56499cfe5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc1e390 0x56499cfe6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6b0445740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56499cf5cfb0 con 0x56499cc3f070 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6b0445740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56499cf4c3e0 con 0x56499cc1e390 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6b0445740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56499cf501f0 con 0x56499cc161b0 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69be00640 1 -- 172.21.15.114:0/3089764726 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd694004020 con 0x7fd6a4105d80 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc3f070 0x56499cfe5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6a3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc3f070 0x56499cfe5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42170/0 (socket says 172.21.15.114:42170) 2024-09-21T06:56:53.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6a3400640 1 -- 172.21.15.114:0/4283490734 learned_addr learned my addr 172.21.15.114:0/4283490734 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69a000640 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc1e390 0x56499cfe6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69a000640 1 -- 172.21.15.114:0/4283490734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 msgr2=0x56499cfe62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69a000640 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69a000640 1 -- 172.21.15.114:0/4283490734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc3f070 msgr2=0x56499cfe5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69a000640 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc3f070 0x56499cfe5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.785+0000 7fd69a000640 1 -- 172.21.15.114:0/4283490734 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499cf16cd0 con 0x56499cc1e390 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.786+0000 7fd69a000640 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc1e390 0x56499cfe6820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd6a406c810 tx=0x7fd68c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bc8bf12c1dc6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.786+0000 7fd693e00640 1 -- 172.21.15.114:0/4283490734 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c004050 con 0x56499cc1e390 2024-09-21T06:56:53.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.786+0000 7fd693e00640 1 -- 172.21.15.114:0/4283490734 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd68c0041f0 con 0x56499cc1e390 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.786+0000 7fd693e00640 1 -- 172.21.15.114:0/4283490734 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c019690 con 0x56499cc1e390 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.786+0000 7fd69be00640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6880b6700 0x7fd6880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd69be00640 1 -- 172.21.15.114:0/3089764726 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd6940e7c70 con 0x7fd6a4105d80 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6a3400640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6880b6700 0x7fd6880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 -- 172.21.15.114:0/4283490734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc1e390 msgr2=0x56499cfe6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc1e390 0x56499cfe6820 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd6a406c810 tx=0x7fd68c011810 comp rx=0 tx=0).stop 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 -- 172.21.15.114:0/4283490734 shutdown_connections 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc1e390 0x56499cfe6820 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 --2- 172.21.15.114:0/4283490734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc3f070 0x56499cfe5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 -- 172.21.15.114:0/4283490734 >> 172.21.15.114:0/4283490734 conn(0x56499cc33460 msgr2=0x56499cfe3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 -- 172.21.15.114:0/4283490734 shutdown_connections 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6b0445740 1 -- 172.21.15.114:0/4283490734 wait complete. 2024-09-21T06:56:53.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.787+0000 7fd6a3400640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6880b6700 0x7fd6880b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02f490 tx=0x7fd69c005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.788+0000 7fd6b0445740 1 -- start start 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.788+0000 7fd6b0445740 1 Processor -- start 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc161b0 0x56499cff1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 0x56499cff1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cffd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56499cdad700 con 0x56499cc161b0 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56499cee7580 con 0x56499cc3f070 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56499cff4580 con 0x56499cc1e390 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6b0445740 10 client.0 _ll_register_callbacks cb 0x56499cef7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:53.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc161b0 0x56499cff1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd69a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 0x56499cff1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc161b0 0x56499cff1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42184/0 (socket says 172.21.15.114:42184) 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd69a000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 0x56499cff1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56312/0 (socket says 172.21.15.114:56312) 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6a2a00640 1 -- 172.21.15.114:0/1920434752 learned_addr learned my addr 172.21.15.114:0/1920434752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.789+0000 7fd6a3400640 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cffd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd69a000640 1 -- 172.21.15.114:0/1920434752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 msgr2=0x56499cffd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd69a000640 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cffd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd69a000640 1 -- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc161b0 msgr2=0x56499cff1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd69a000640 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc161b0 0x56499cff1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd69a000640 1 -- 172.21.15.114:0/1920434752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499ccaa650 con 0x56499cc1e390 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd69a000640 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 0x56499cff1a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd68c003e70 tx=0x7fd68c005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b72788ea304765 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd690c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd690c00640 1 -- 172.21.15.114:0/1920434752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c028020 con 0x56499cc1e390 2024-09-21T06:56:53.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd690c00640 1 -- 172.21.15.114:0/1920434752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd68c0037f0 con 0x56499cc1e390 2024-09-21T06:56:53.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.790+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56499cee9b90 con 0x56499cc1e390 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.791+0000 7fd690c00640 1 -- 172.21.15.114:0/1920434752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd68c033050 con 0x56499cc1e390 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.791+0000 7fd6b0445740 10 client.26329 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.791+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56499cf04950 con 0x56499cc1e390 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.791+0000 7fd690c00640 1 -- 172.21.15.114:0/1920434752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56499cee9b90 con 0x56499cc1e390 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd690c00640 1 -- 172.21.15.114:0/1920434752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd68c0047e0 con 0x56499cc1e390 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6b0445740 10 client.26329 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6b0445740 10 client.26329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:56:53.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6b0445740 20 client.26329 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbfd9f0 0x56499d003620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6b0445740 4 client.26329 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499cf5ef00 con 0x56499cbfd9f0 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.792+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbfd9f0 0x56499d003620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.793+0000 7fd690c00640 10 client.26329 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.793+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbfd9f0 0x56499d003620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39d46f756c721715 server_cookie=f5ac6e27c03a1db3 in_seq=0 out_seq=0 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.793+0000 7fd690c00640 1 -- 172.21.15.114:0/1920434752 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56499cf5ef00 con 0x56499cbfd9f0 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.793+0000 7fd690c00640 10 client.26329 handle_command_reply: tid=0 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.793+0000 7fd6b0445740 1 client.26329 shutdown 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 msgr2=0x56499cff1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 0x56499cff1a60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd68c003e70 tx=0x7fd68c005d50 comp rx=0 tx=0).stop 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 shutdown_connections 2024-09-21T06:56:53.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbfd9f0 0x56499d003620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cffd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc1e390 0x56499cff1a60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd690c00640 0 client.26329 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1920434752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc161b0 0x56499cff1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 >> 172.21.15.114:0/1920434752 conn(0x56499cc33460 msgr2=0x56499cfe8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 shutdown_connections 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 1 -- 172.21.15.114:0/1920434752 wait complete. 2024-09-21T06:56:53.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.794+0000 7fd6b0445740 20 client.26329 trim_cache size 0 max 16384 2024-09-21T06:56:53.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.809+0000 7fd6b0445740 1 Processor -- start 2024-09-21T06:56:53.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.809+0000 7fd6b0445740 1 -- start start 2024-09-21T06:56:53.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.809+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.809+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cfee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.809+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cfeec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.809+0000 7fd6b0445740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41b30b0 con 0x56499cc1e390 2024-09-21T06:56:53.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6b0445740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41b4cb0 con 0x56499cc3f070 2024-09-21T06:56:53.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6b0445740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41b3780 con 0x56499cc161b0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56316/0 (socket says 172.21.15.114:56316) 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 -- 172.21.15.114:0/763024701 learned_addr learned my addr 172.21.15.114:0/763024701 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd69a000640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cfee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a3400640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cfeec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 -- 172.21.15.114:0/763024701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 msgr2=0x56499cfeec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cfeec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 -- 172.21.15.114:0/763024701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 msgr2=0x56499cfee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cfee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 -- 172.21.15.114:0/763024701 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56499cfe8090 con 0x56499cc161b0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd69a000640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cfee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a3400640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cfeec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.810+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe74e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd69400cfd0 tx=0x7fd694003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4528ebbdeeb4318 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.811+0000 7fd693400640 1 -- 172.21.15.114:0/763024701 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6940addf0 con 0x56499cc161b0 2024-09-21T06:56:53.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.811+0000 7fd693400640 1 -- 172.21.15.114:0/763024701 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6940b21d0 con 0x56499cc161b0 2024-09-21T06:56:53.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.811+0000 7fd693400640 1 -- 172.21.15.114:0/763024701 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd694004310 con 0x56499cc161b0 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 -- 172.21.15.114:0/763024701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 msgr2=0x56499cfe74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe74e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd69400cfd0 tx=0x7fd694003e10 comp rx=0 tx=0).stop 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 -- 172.21.15.114:0/763024701 shutdown_connections 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc3f070 0x56499cfeec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cfee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 --2- 172.21.15.114:0/763024701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc161b0 0x56499cfe74e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 -- 172.21.15.114:0/763024701 >> 172.21.15.114:0/763024701 conn(0x56499cc33460 msgr2=0x56499cfe6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 -- 172.21.15.114:0/763024701 shutdown_connections 2024-09-21T06:56:53.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 -- 172.21.15.114:0/763024701 wait complete. 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 -- start start 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.812+0000 7fd6b0445740 1 Processor -- start 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc161b0 0x56499cff9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cff53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc3f070 0x56499cff5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41b4520 con 0x56499cc1e390 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41b3e50 con 0x56499cc161b0 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a41b1090 con 0x56499cc3f070 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6b0445740 10 client.0 _ll_register_callbacks cb 0x7fd68c004430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd69a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cff53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd69a000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cff53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42194/0 (socket says 172.21.15.114:42194) 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd69a000640 1 -- 172.21.15.114:0/1139590661 learned_addr learned my addr 172.21.15.114:0/1139590661 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6a2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc161b0 0x56499cff9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.813+0000 7fd6a3400640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc3f070 0x56499cff5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd69a000640 1 -- 172.21.15.114:0/1139590661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc3f070 msgr2=0x56499cff5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd69a000640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc3f070 0x56499cff5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd69a000640 1 -- 172.21.15.114:0/1139590661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc161b0 msgr2=0x56499cff9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:53.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd69a000640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc161b0 0x56499cff9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd69a000640 1 -- 172.21.15.114:0/1139590661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56499cff6010 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc161b0 0x56499cff9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd69a000640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cff53e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd68c004c30 tx=0x7fd68c009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f835a61fb38835fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd690c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd690c00640 1 -- 172.21.15.114:0/1139590661 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c028040 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd690c00640 1 -- 172.21.15.114:0/1139590661 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd68c0048f0 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.814+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6940b21d0 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd690c00640 1 -- 172.21.15.114:0/1139590661 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd68c045050 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 10 client.16245 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56499d06dd10 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd690c00640 1 -- 172.21.15.114:0/1139590661 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6940b21d0 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd690c00640 1 -- 172.21.15.114:0/1139590661 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd68c005100 con 0x56499cc1e390 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 10 client.16245 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:56:53.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 20 client.16245 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbd2da0 0x56499d070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.815+0000 7fd6b0445740 4 client.16245 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.816+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56499cf5a9b0 con 0x56499cbd2da0 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.816+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbd2da0 0x56499d070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.816+0000 7fd6a2a00640 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbd2da0 0x56499d070460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=661872dc85d9ba35 server_cookie=e55851737d396346 in_seq=0 out_seq=0 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.816+0000 7fd690c00640 10 client.16245 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd690c00640 1 -- 172.21.15.114:0/1139590661 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56499cf5a9b0 con 0x56499cbd2da0 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd690c00640 10 client.16245 handle_command_reply: tid=0 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 client.16245 shutdown 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 msgr2=0x56499cff53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cff53e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd68c004c30 tx=0x7fd68c009fd0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 shutdown_connections 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56499cbd2da0 0x56499d070460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56499cc3f070 0x56499cff5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd690c00640 0 client.16245 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56499cc1e390 0x56499cff53e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 --2- 172.21.15.114:0/1139590661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56499cc161b0 0x56499cff9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 >> 172.21.15.114:0/1139590661 conn(0x56499cc33460 msgr2=0x56499cfefda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:53.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 shutdown_connections 2024-09-21T06:56:53.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 1 -- 172.21.15.114:0/1139590661 wait complete. 2024-09-21T06:56:53.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.817+0000 7fd6b0445740 20 client.16245 trim_cache size 0 max 16384 2024-09-21T06:56:53.822 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6880b6700 msgr2=0x7fd6880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6880b6700 0x7fd6880b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd69c02f490 tx=0x7fd69c005c50 comp rx=0 tx=0).stop 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4105d80 msgr2=0x7fd6a419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4105d80 0x7fd6a419c720 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e9f0 tx=0x7fd69400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 shutdown_connections 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd6a410c010 0x7fd6a41a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd6880b6700 0x7fd6880b8c10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd6a410b6c0 0x7fd6a419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 --2- 172.21.15.114:0/3089764726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6a4105d80 0x7fd6a419c720 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 >> 172.21.15.114:0/3089764726 conn(0x7fd6a4101810 msgr2=0x7fd6a410ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 shutdown_connections 2024-09-21T06:56:53.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:53.818+0000 7fd6a9800640 1 -- 172.21.15.114:0/3089764726 wait complete. 2024-09-21T06:56:54.746 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:56:55.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:54 smithi114 ceph-mon[30654]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 KiB/s wr, 27 op/s 2024-09-21T06:56:55.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:54 smithi183 ceph-mon[34219]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 KiB/s wr, 27 op/s 2024-09-21T06:56:55.157 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:54 smithi193 ceph-mon[34601]: pgmap v479: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 227 KiB/s wr, 27 op/s 2024-09-21T06:56:55.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 -- 172.21.15.114:0/1039955140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481039b0 msgr2=0x7fa548103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 --2- 172.21.15.114:0/1039955140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481039b0 0x7fa548103dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa538009900 tx=0x7fa53802a410 comp rx=0 tx=0).stop 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 -- 172.21.15.114:0/1039955140 shutdown_connections 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 --2- 172.21.15.114:0/1039955140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa548109c40 0x7fa548110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 --2- 172.21.15.114:0/1039955140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5481092f0 0x7fa548109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 --2- 172.21.15.114:0/1039955140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481039b0 0x7fa548103dc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 -- 172.21.15.114:0/1039955140 >> 172.21.15.114:0/1039955140 conn(0x7fa5480ff440 msgr2=0x7fa548101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 -- 172.21.15.114:0/1039955140 shutdown_connections 2024-09-21T06:56:55.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.795+0000 7fa54dc00640 1 -- 172.21.15.114:0/1039955140 wait complete. 2024-09-21T06:56:55.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.796+0000 7fa54dc00640 1 Processor -- start 2024-09-21T06:56:55.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 -- start start 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5481039b0 0x7fa54819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 0x7fa54819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa548109c40 0x7fa54819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa548113900 con 0x7fa5481092f0 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa548113780 con 0x7fa5481039b0 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54dc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa548113a80 con 0x7fa548109c40 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa547400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 0x7fa54819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa547400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 0x7fa54819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44304/0 (socket says 172.21.15.114:44304) 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa547400640 1 -- 172.21.15.114:0/2875144809 learned_addr learned my addr 172.21.15.114:0/2875144809 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa547e00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5481039b0 0x7fa54819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.797+0000 7fa54d200640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa548109c40 0x7fa54819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa547400640 1 -- 172.21.15.114:0/2875144809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa548109c40 msgr2=0x7fa54819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa547400640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa548109c40 0x7fa54819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa547400640 1 -- 172.21.15.114:0/2875144809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5481039b0 msgr2=0x7fa54819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa547400640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5481039b0 0x7fa54819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa547400640 1 -- 172.21.15.114:0/2875144809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5481a0280 con 0x7fa5481092f0 2024-09-21T06:56:55.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa547400640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 0x7fa54819a8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa53400e9f0 tx=0x7fa53400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:55.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa544c00640 1 -- 172.21.15.114:0/2875144809 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53400ccc0 con 0x7fa5481092f0 2024-09-21T06:56:55.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa544c00640 1 -- 172.21.15.114:0/2875144809 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5340108e0 con 0x7fa5481092f0 2024-09-21T06:56:55.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.798+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5481a0570 con 0x7fa5481092f0 2024-09-21T06:56:55.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.799+0000 7fa544c00640 1 -- 172.21.15.114:0/2875144809 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534018860 con 0x7fa5481092f0 2024-09-21T06:56:55.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.799+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5481a6cd0 con 0x7fa5481092f0 2024-09-21T06:56:55.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.803+0000 7fa555eb6740 1 Processor -- start 2024-09-21T06:56:55.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 -- start start 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa544c00640 1 -- 172.21.15.114:0/2875144809 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa534010430 con 0x7fa5481092f0 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f76f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 0x560197f774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197baf390 0x560197f779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560197eedfb0 con 0x560197bd0070 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560197edd3e0 con 0x560197baf390 2024-09-21T06:56:55.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa555eb6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560197ee11f0 con 0x560197ba71b0 2024-09-21T06:56:55.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa54d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f76f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa547e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 0x560197f774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa547400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197baf390 0x560197f779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa54d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f76f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44314/0 (socket says 172.21.15.114:44314) 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa547e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 0x560197f774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44048/0 (socket says 172.21.15.114:44048) 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.804+0000 7fa54d200640 1 -- 172.21.15.114:0/3676630806 learned_addr learned my addr 172.21.15.114:0/3676630806 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.805+0000 7fa547e00640 1 -- 172.21.15.114:0/3676630806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197baf390 msgr2=0x560197f779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.805+0000 7fa547e00640 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197baf390 0x560197f779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.805+0000 7fa547e00640 1 -- 172.21.15.114:0/3676630806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 msgr2=0x560197f76f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.805+0000 7fa547e00640 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f76f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.805+0000 7fa547e00640 1 -- 172.21.15.114:0/3676630806 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560197ea7cd0 con 0x560197ba71b0 2024-09-21T06:56:55.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.805+0000 7fa547e00640 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 0x560197f774b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa538009230 tx=0x7fa538032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01d475d83fecf14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:55.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.806+0000 7fa53d600640 1 -- 172.21.15.114:0/3676630806 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538040070 con 0x560197ba71b0 2024-09-21T06:56:55.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.806+0000 7fa53d600640 1 -- 172.21.15.114:0/3676630806 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa538004030 con 0x560197ba71b0 2024-09-21T06:56:55.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.806+0000 7fa53d600640 1 -- 172.21.15.114:0/3676630806 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538035040 con 0x560197ba71b0 2024-09-21T06:56:55.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.806+0000 7fa544c00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa52c0b6700 0x7fa52c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.806+0000 7fa544c00640 1 -- 172.21.15.114:0/2875144809 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa5340e7c90 con 0x7fa5481092f0 2024-09-21T06:56:55.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.806+0000 7fa54d200640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa52c0b6700 0x7fa52c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 -- 172.21.15.114:0/3676630806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 msgr2=0x560197f774b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 0x560197f774b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa538009230 tx=0x7fa538032b90 comp rx=0 tx=0).stop 2024-09-21T06:56:55.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa54d200640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa52c0b6700 0x7fa52c0b8c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa5400116b0 tx=0x7fa540009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:55.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 -- 172.21.15.114:0/3676630806 shutdown_connections 2024-09-21T06:56:55.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197baf390 0x560197f779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197ba71b0 0x560197f774b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3676630806 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f76f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 -- 172.21.15.114:0/3676630806 >> 172.21.15.114:0/3676630806 conn(0x560197bc4460 msgr2=0x560197f74e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 -- 172.21.15.114:0/3676630806 shutdown_connections 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.807+0000 7fa555eb6740 1 -- 172.21.15.114:0/3676630806 wait complete. 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 -- start start 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 Processor -- start 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197ba71b0 0x560197f86b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197bd0070 0x560197f92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560197d3e700 con 0x560197ba71b0 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560197e78580 con 0x560197bd0070 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.808+0000 7fa555eb6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560197f89a50 con 0x560197baf390 2024-09-21T06:56:55.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa555eb6740 10 client.0 _ll_register_callbacks cb 0x560197e88470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197ba71b0 0x560197f86b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa54d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197bd0070 0x560197f92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f87080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44058/0 (socket says 172.21.15.114:44058) 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa54d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197bd0070 0x560197f92d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54280/0 (socket says 172.21.15.114:54280) 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547400640 1 -- 172.21.15.114:0/3206190386 learned_addr learned my addr 172.21.15.114:0/3206190386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547e00640 1 -- 172.21.15.114:0/3206190386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 msgr2=0x560197f87080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547e00640 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f87080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547e00640 1 -- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197bd0070 msgr2=0x560197f92d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547e00640 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197bd0070 0x560197f92d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.809+0000 7fa547e00640 1 -- 172.21.15.114:0/3206190386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560197c3b650 con 0x560197ba71b0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.810+0000 7fa547e00640 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197ba71b0 0x560197f86b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x560197f75bf0 tx=0x7fa538009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59173990232cf741 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.810+0000 7fa532000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.811+0000 7fa532000640 1 -- 172.21.15.114:0/3206190386 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53804b020 con 0x560197ba71b0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.811+0000 7fa532000640 1 -- 172.21.15.114:0/3206190386 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa5380314a0 con 0x560197ba71b0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.811+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560197e7ab90 con 0x560197ba71b0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.811+0000 7fa532000640 1 -- 172.21.15.114:0/3206190386 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa538056050 con 0x560197ba71b0 2024-09-21T06:56:55.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.811+0000 7fa555eb6740 10 client.16257 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.811+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560197e95950 con 0x560197ba71b0 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa532000640 1 -- 172.21.15.114:0/3206190386 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560197e7ab90 con 0x560197ba71b0 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa532000640 1 -- 172.21.15.114:0/3206190386 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa538004a80 con 0x560197ba71b0 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa555eb6740 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa555eb6740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa555eb6740 20 client.16257 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b8e9f0 0x560197f98a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa555eb6740 4 client.16257 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560197eeff00 con 0x560197b8e9f0 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.812+0000 7fa547400640 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b8e9f0 0x560197f98a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.813+0000 7fa547400640 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b8e9f0 0x560197f98a00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84f05cb86cef8abd server_cookie=83d4fa9a97cfd2ee in_seq=0 out_seq=0 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.813+0000 7fa532000640 10 client.16257 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:55.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa532000640 1 -- 172.21.15.114:0/3206190386 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560197eeff00 con 0x560197b8e9f0 2024-09-21T06:56:55.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa532000640 10 client.16257 handle_command_reply: tid=0 2024-09-21T06:56:55.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 client.16257 shutdown 2024-09-21T06:56:55.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197ba71b0 msgr2=0x560197f86b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197ba71b0 0x560197f86b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x560197f75bf0 tx=0x7fa538009260 comp rx=0 tx=0).stop 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 shutdown_connections 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197bd0070 0x560197f92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b8e9f0 0x560197f98a00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f87080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 --2- 172.21.15.114:0/3206190386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197ba71b0 0x560197f86b40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 >> 172.21.15.114:0/3206190386 conn(0x560197bc4460 msgr2=0x560197f73600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa532000640 0 client.16257 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 shutdown_connections 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 1 -- 172.21.15.114:0/3206190386 wait complete. 2024-09-21T06:56:55.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.814+0000 7fa555eb6740 20 client.16257 trim_cache size 0 max 16384 2024-09-21T06:56:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.828+0000 7fa555eb6740 1 Processor -- start 2024-09-21T06:56:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.828+0000 7fa555eb6740 1 -- start start 2024-09-21T06:56:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f82320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197baf390 0x560197f74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 0x560197f74b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa555eb6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481aea00 con 0x560197baf390 2024-09-21T06:56:55.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa555eb6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481b0600 con 0x560197ba71b0 2024-09-21T06:56:55.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa555eb6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481af0d0 con 0x560197bd0070 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa547400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197baf390 0x560197f74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa54d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 0x560197f74b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa547e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f82320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa54d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 0x560197f74b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44074/0 (socket says 172.21.15.114:44074) 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.829+0000 7fa54d200640 1 -- 172.21.15.114:0/2379272628 learned_addr learned my addr 172.21.15.114:0/2379272628 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa54d200640 1 -- 172.21.15.114:0/2379272628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 msgr2=0x560197f82320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa54d200640 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f82320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa54d200640 1 -- 172.21.15.114:0/2379272628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197baf390 msgr2=0x560197f74610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa54d200640 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197baf390 0x560197f74610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa54d200640 1 -- 172.21.15.114:0/2379272628 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560197f98d70 con 0x560197bd0070 2024-09-21T06:56:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa54d200640 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 0x560197f74b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa54000f640 tx=0x7fa540019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2b1fe8d3cc3872 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa53cc00640 1 -- 172.21.15.114:0/2379272628 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540003a30 con 0x560197bd0070 2024-09-21T06:56:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa53cc00640 1 -- 172.21.15.114:0/2379272628 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa540003bd0 con 0x560197bd0070 2024-09-21T06:56:55.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.830+0000 7fa53cc00640 1 -- 172.21.15.114:0/2379272628 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540010940 con 0x560197bd0070 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 -- 172.21.15.114:0/2379272628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 msgr2=0x560197f74b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 0x560197f74b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa54000f640 tx=0x7fa540019000 comp rx=0 tx=0).stop 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 -- 172.21.15.114:0/2379272628 shutdown_connections 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197bd0070 0x560197f74b50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197baf390 0x560197f74610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 --2- 172.21.15.114:0/2379272628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f82320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 -- 172.21.15.114:0/2379272628 >> 172.21.15.114:0/2379272628 conn(0x560197bc4460 msgr2=0x560197f853c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:55.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 -- 172.21.15.114:0/2379272628 shutdown_connections 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.831+0000 7fa555eb6740 1 -- 172.21.15.114:0/2379272628 wait complete. 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 -- start start 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 Processor -- start 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f8ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f8b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481afe70 con 0x560197bd0070 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481af7a0 con 0x560197ba71b0 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa5481ac9e0 con 0x560197baf390 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.832+0000 7fa555eb6740 10 client.0 _ll_register_callbacks cb 0x7fa538009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:55.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa54d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f8b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f8b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f8ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f8b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44082/0 (socket says 172.21.15.114:44082) 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 -- 172.21.15.114:0/244190833 learned_addr learned my addr 172.21.15.114:0/244190833 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 -- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 msgr2=0x560197f8ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f8ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 -- 172.21.15.114:0/244190833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 msgr2=0x560197f8b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f8b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 -- 172.21.15.114:0/244190833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560198002270 con 0x560197baf390 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547e00640 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:55.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa547400640 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f8b2d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5340e5120 tx=0x7fa5340ea1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b0d8e53a1de2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.833+0000 7fa532000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa532000640 1 -- 172.21.15.114:0/244190833 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5340ad170 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa532000640 1 -- 172.21.15.114:0/244190833 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fa5340a9a90 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa540003bd0 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa532000640 1 -- 172.21.15.114:0/244190833 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5340acf90 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 10 client.26353 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5601980034f0 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa532000640 1 -- 172.21.15.114:0/244190833 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa540003bd0 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa532000640 1 -- 172.21.15.114:0/244190833 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa5340ec040 con 0x560197baf390 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 10 client.26353 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 10 client.26353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 20 client.26353 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:55.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b63da0 0x560198005d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:55.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 4 client.26353 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:56:55.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.834+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560197eeb9b0 con 0x560197b63da0 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.835+0000 7fa547e00640 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b63da0 0x560198005d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.835+0000 7fa547e00640 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b63da0 0x560198005d70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8004c8ed7461844 server_cookie=6279806623d4fcbd in_seq=0 out_seq=0 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.835+0000 7fa532000640 10 client.26353 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa532000640 1 -- 172.21.15.114:0/244190833 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560197eeb9b0 con 0x560197b63da0 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa532000640 10 client.26353 handle_command_reply: tid=0 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 client.26353 shutdown 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 msgr2=0x560197f8b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f8b2d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5340e5120 tx=0x7fa5340ea1a0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 shutdown_connections 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560197bd0070 0x560197f8b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560197b63da0 0x560198005d70 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560197baf390 0x560197f8b2d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 --2- 172.21.15.114:0/244190833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560197ba71b0 0x560197f8ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa532000640 0 client.26353 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 >> 172.21.15.114:0/244190833 conn(0x560197bc4460 msgr2=0x560197f7c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 shutdown_connections 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 1 -- 172.21.15.114:0/244190833 wait complete. 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.836+0000 7fa555eb6740 20 client.26353 trim_cache size 0 max 16384 2024-09-21T06:56:55.842 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa52c0b6700 msgr2=0x7fa52c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa52c0b6700 0x7fa52c0b8c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa5400116b0 tx=0x7fa540009500 comp rx=0 tx=0).stop 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 msgr2=0x7fa54819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 0x7fa54819a8b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa53400e9f0 tx=0x7fa53400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 shutdown_connections 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa548109c40 0x7fa54819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa5481092f0 0x7fa54819a8b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa52c0b6700 0x7fa52c0b8c10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 --2- 172.21.15.114:0/2875144809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5481039b0 0x7fa54819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:55.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 >> 172.21.15.114:0/2875144809 conn(0x7fa5480ff440 msgr2=0x7fa548101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:55.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 shutdown_connections 2024-09-21T06:56:55.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:55.837+0000 7fa54dc00640 1 -- 172.21.15.114:0/2875144809 wait complete. 2024-09-21T06:56:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.447+0000 7ff3e9400640 1 -- 172.21.15.114:0/353955121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e410dd90 msgr2=0x7ff3e4110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.447+0000 7ff3e9400640 1 --2- 172.21.15.114:0/353955121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e410dd90 0x7ff3e4110280 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009900 tx=0x7ff3dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 -- 172.21.15.114:0/353955121 shutdown_connections 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 --2- 172.21.15.114:0/353955121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e410dd90 0x7ff3e4110280 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 --2- 172.21.15.114:0/353955121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3e4105280 0x7ff3e410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 --2- 172.21.15.114:0/353955121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e4104930 0x7ff3e4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 -- 172.21.15.114:0/353955121 >> 172.21.15.114:0/353955121 conn(0x7ff3e40fe3c0 msgr2=0x7ff3e4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 -- 172.21.15.114:0/353955121 shutdown_connections 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.448+0000 7ff3e9400640 1 -- 172.21.15.114:0/353955121 wait complete. 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 Processor -- start 2024-09-21T06:56:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 -- start start 2024-09-21T06:56:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3e4104930 0x7ff3e419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 0x7ff3e419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e410dd90 0x7ff3e41a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e4113a10 con 0x7ff3e410dd90 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e4113890 con 0x7ff3e4104930 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.449+0000 7ff3e9400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e4113b90 con 0x7ff3e4105280 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3e3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e410dd90 0x7ff3e41a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 0x7ff3e419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 0x7ff3e419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44098/0 (socket says 172.21.15.114:44098) 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 -- 172.21.15.114:0/1529341305 learned_addr learned my addr 172.21.15.114:0/1529341305 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3e4104930 0x7ff3e419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3e3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e410dd90 0x7ff3e41a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44368/0 (socket says 172.21.15.114:44368) 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 -- 172.21.15.114:0/1529341305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3e4104930 msgr2=0x7ff3e419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3e4104930 0x7ff3e419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 -- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e410dd90 msgr2=0x7ff3e41a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e410dd90 0x7ff3e41a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 -- 172.21.15.114:0/1529341305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e41a2500 con 0x7ff3e4105280 2024-09-21T06:56:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.450+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 0x7ff3e419cb70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00e9c0 tx=0x7ff3cc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.451+0000 7ff3dbe00640 1 -- 172.21.15.114:0/1529341305 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00ccc0 con 0x7ff3e4105280 2024-09-21T06:56:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.451+0000 7ff3dbe00640 1 -- 172.21.15.114:0/1529341305 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc0054d0 con 0x7ff3e4105280 2024-09-21T06:56:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.451+0000 7ff3dbe00640 1 -- 172.21.15.114:0/1529341305 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc010430 con 0x7ff3e4105280 2024-09-21T06:56:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.451+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e41a27f0 con 0x7ff3e4105280 2024-09-21T06:56:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.451+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e41a8f80 con 0x7ff3e4105280 2024-09-21T06:56:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 Processor -- start 2024-09-21T06:56:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 -- start start 2024-09-21T06:56:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5eec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ef6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c0ab5646b0 con 0x55c0ab2212f0 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0ab553ae0 con 0x55c0ab24a1b0 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.454+0000 7ff3f0400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0ab5578f0 con 0x55c0ab2294d0 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44382/0 (socket says 172.21.15.114:44382) 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 -- 172.21.15.114:0/672481266 learned_addr learned my addr 172.21.15.114:0/672481266 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e3200640 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 -- 172.21.15.114:0/672481266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 msgr2=0x55c0ab5ef6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ef6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 -- 172.21.15.114:0/672481266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 msgr2=0x55c0ab5eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 -- 172.21.15.114:0/672481266 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ab520780 con 0x55c0ab2212f0 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3e2800640 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5ef170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40114f0 tx=0x7ff3d40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61971ede8a7c742a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.455+0000 7ff3dbe00640 1 -- 172.21.15.114:0/1529341305 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff3cc005020 con 0x7ff3e4105280 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.456+0000 7ff3d3400640 1 -- 172.21.15.114:0/672481266 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d401e070 con 0x55c0ab2212f0 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.456+0000 7ff3d3400640 1 -- 172.21.15.114:0/672481266 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d400fad0 con 0x55c0ab2212f0 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.456+0000 7ff3d3400640 1 -- 172.21.15.114:0/672481266 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4019ac0 con 0x55c0ab2212f0 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.456+0000 7ff3f0400740 1 -- 172.21.15.114:0/672481266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 msgr2=0x55c0ab5ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.456+0000 7ff3f0400740 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5ef170 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40114f0 tx=0x7ff3d40119b0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 -- 172.21.15.114:0/672481266 shutdown_connections 2024-09-21T06:56:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ef6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5ef170 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 --2- 172.21.15.114:0/672481266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 -- 172.21.15.114:0/672481266 >> 172.21.15.114:0/672481266 conn(0x55c0ab23e300 msgr2=0x55c0ab5ecb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 -- 172.21.15.114:0/672481266 shutdown_connections 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 -- 172.21.15.114:0/672481266 wait complete. 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3dbe00640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3c80b6610 0x7ff3c80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 -- start start 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3dbe00640 1 -- 172.21.15.114:0/1529341305 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff3cc0e7d70 con 0x7ff3e4105280 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3e3200640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3c80b6610 0x7ff3c80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.457+0000 7ff3f0400740 1 Processor -- start 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5fa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab6066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c0ab48bab0 con 0x55c0ab2212f0 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0ab4fa570 con 0x55c0ab24a1b0 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0ab5fd3b0 con 0x55c0ab2294d0 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3e3200640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3c80b6610 0x7ff3c80b8b20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc006010 tx=0x7ff3dc0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3f0400740 10 client.0 _ll_register_callbacks cb 0x55c0ab509100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44396/0 (socket says 172.21.15.114:44396) 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3e2800640 1 -- 172.21.15.114:0/2383708197 learned_addr learned my addr 172.21.15.114:0/2383708197 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:56.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5fa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.458+0000 7ff3e3200640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab6066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e2800640 1 -- 172.21.15.114:0/2383708197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 msgr2=0x55c0ab5fa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e2800640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5fa7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e2800640 1 -- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 msgr2=0x55c0ab6066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e2800640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab6066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e2800640 1 -- 172.21.15.114:0/2383708197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ab2b5790 con 0x55c0ab2212f0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e3200640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab6066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3e2800640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fa2a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009370 tx=0x7ff3d400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8bb4f673fdd741 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2383708197 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d401e070 con 0x55c0ab2212f0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2383708197 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff3d4004c50 con 0x55c0ab2212f0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.459+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3d400fad0 con 0x55c0ab2212f0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2383708197 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3d4031050 con 0x55c0ab2212f0 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 10 client.16275 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:56.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0ab32e620 con 0x55c0ab2212f0 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2383708197 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d400fad0 con 0x55c0ab2212f0 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2383708197 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff3d402c030 con 0x55c0ab2212f0 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 10 client.16275 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 20 client.16275 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab208d80 0x55c0ab60c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 4 client.16275 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.460+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0ab566600 con 0x55c0ab208d80 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.461+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab208d80 0x55c0ab60c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.461+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab208d80 0x55c0ab60c2c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad58e2b497c1ab9 server_cookie=e05ffd8c7a81a6c1 in_seq=0 out_seq=0 2024-09-21T06:56:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.461+0000 7ff3c7e00640 10 client.16275 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2383708197 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0ab566600 con 0x55c0ab208d80 2024-09-21T06:56:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3c7e00640 10 client.16275 handle_command_reply: tid=0 2024-09-21T06:56:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 client.16275 shutdown 2024-09-21T06:56:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 msgr2=0x55c0ab5fa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fa2a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009370 tx=0x7ff3d400f8d0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 shutdown_connections 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab24a1b0 0x55c0ab6066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab208d80 0x55c0ab60c2c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2294d0 0x55c0ab5fa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2383708197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fa2a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3c7e00640 0 client.16275 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 >> 172.21.15.114:0/2383708197 conn(0x55c0ab23e300 msgr2=0x55c0ab5efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 shutdown_connections 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 1 -- 172.21.15.114:0/2383708197 wait complete. 2024-09-21T06:56:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.462+0000 7ff3f0400740 20 client.16275 trim_cache size 0 max 16384 2024-09-21T06:56:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.479+0000 7ff3f0400740 1 Processor -- start 2024-09-21T06:56:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.479+0000 7ff3f0400740 1 -- start start 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.479+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 0x55c0ab5f0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.479+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2294d0 0x55c0ab6065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.479+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3f0400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e41b3010 con 0x55c0ab24a1b0 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3f0400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e41b4c10 con 0x55c0ab2294d0 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3f0400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e41b36e0 con 0x55c0ab2212f0 2024-09-21T06:56:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 0x55c0ab5f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 0x55c0ab5f0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44118/0 (socket says 172.21.15.114:44118) 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2294d0 0x55c0ab6065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 -- 172.21.15.114:0/3510388930 learned_addr learned my addr 172.21.15.114:0/3510388930 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e3200640 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 -- 172.21.15.114:0/3510388930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2294d0 msgr2=0x55c0ab6065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2294d0 0x55c0ab6065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 -- 172.21.15.114:0/3510388930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab24a1b0 msgr2=0x55c0ab5ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 -- 172.21.15.114:0/3510388930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ab5f87a0 con 0x55c0ab2212f0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2294d0 0x55c0ab6065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e3200640 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.480+0000 7ff3e2800640 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 0x55c0ab5f0370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4005440 tx=0x7ff3d4005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd1f75b84459469c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.481+0000 7ff3d2a00640 1 -- 172.21.15.114:0/3510388930 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d401e040 con 0x55c0ab2212f0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.481+0000 7ff3d2a00640 1 -- 172.21.15.114:0/3510388930 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d4005040 con 0x55c0ab2212f0 2024-09-21T06:56:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.481+0000 7ff3d2a00640 1 -- 172.21.15.114:0/3510388930 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40054e0 con 0x55c0ab2212f0 2024-09-21T06:56:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 -- 172.21.15.114:0/3510388930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 msgr2=0x55c0ab5f0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 0x55c0ab5f0370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4005440 tx=0x7ff3d4005cf0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 -- 172.21.15.114:0/3510388930 shutdown_connections 2024-09-21T06:56:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab24a1b0 0x55c0ab5ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2294d0 0x55c0ab6065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 --2- 172.21.15.114:0/3510388930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab2212f0 0x55c0ab5f0370 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 -- 172.21.15.114:0/3510388930 >> 172.21.15.114:0/3510388930 conn(0x55c0ab23e300 msgr2=0x55c0ab5efc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 -- 172.21.15.114:0/3510388930 shutdown_connections 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 -- 172.21.15.114:0/3510388930 wait complete. 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 -- start start 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.482+0000 7ff3f0400740 1 Processor -- start 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab24a1b0 0x55c0ab674bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e41b4480 con 0x55c0ab2294d0 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e41b3db0 con 0x55c0ab2212f0 2024-09-21T06:56:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e41b0ff0 con 0x55c0ab24a1b0 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3f0400740 10 client.0 _ll_register_callbacks cb 0x7ff3d401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3e2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54344/0 (socket says 172.21.15.114:54344) 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ff4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44408/0 (socket says 172.21.15.114:44408) 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3e2800640 1 -- 172.21.15.114:0/2562572241 learned_addr learned my addr 172.21.15.114:0/2562572241 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.483+0000 7ff3e3200640 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab24a1b0 0x55c0ab674bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3e2800640 1 -- 172.21.15.114:0/2562572241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab24a1b0 msgr2=0x55c0ab674bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3e2800640 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab24a1b0 0x55c0ab674bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3e2800640 1 -- 172.21.15.114:0/2562572241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2294d0 msgr2=0x55c0ab5ff4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:56:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3e2800640 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ff4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3e2800640 1 -- 172.21.15.114:0/2562572241 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ab675130 con 0x55c0ab2212f0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3e2800640 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009370 tx=0x7ff3d4009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb8384df21185d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2562572241 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4041060 con 0x55c0ab2212f0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c0ab5620b0 con 0x55c0ab2212f0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2562572241 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff3d40054c0 con 0x55c0ab2212f0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2562572241 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3d4048050 con 0x55c0ab2212f0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3f0400740 10 client.26417 fetch_fsmap learned FSMap version 32 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.484+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0ab676290 con 0x55c0ab2212f0 2024-09-21T06:56:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2562572241 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c0ab5620b0 con 0x55c0ab2212f0 2024-09-21T06:56:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2562572241 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff3d402c070 con 0x55c0ab2212f0 2024-09-21T06:56:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3f0400740 10 client.26417 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:56:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3f0400740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:56:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3f0400740 20 client.26417 populate_metadata read hostname 'smithi114' 2024-09-21T06:56:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab1de1c0 0x55c0ab678ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3f0400740 4 client.26417 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:56:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0ab55f4e0 con 0x55c0ab1de1c0 2024-09-21T06:56:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.485+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab1de1c0 0x55c0ab678ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.486+0000 7ff3d9e00640 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab1de1c0 0x55c0ab678ae0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3323c95e1c371d server_cookie=ec4828b6b9b4ecc1 in_seq=0 out_seq=0 2024-09-21T06:56:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.486+0000 7ff3c7e00640 10 client.26417 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.498+0000 7ff3c7e00640 1 -- 172.21.15.114:0/2562572241 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0ab55f4e0 con 0x55c0ab1de1c0 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.498+0000 7ff3c7e00640 10 client.26417 handle_command_reply: tid=0 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.498+0000 7ff3f0400740 1 client.26417 shutdown 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 msgr2=0x55c0ab5fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fef60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009370 tx=0x7ff3d4009f20 comp rx=0 tx=0).stop 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 shutdown_connections 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0ab1de1c0 0x55c0ab678ae0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0ab24a1b0 0x55c0ab674bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0ab2294d0 0x55c0ab5ff4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 --2- 172.21.15.114:0/2562572241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0ab2212f0 0x55c0ab5fef60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3c7e00640 0 client.26417 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 >> 172.21.15.114:0/2562572241 conn(0x55c0ab23e300 msgr2=0x55c0ab5fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 shutdown_connections 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 1 -- 172.21.15.114:0/2562572241 wait complete. 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.499+0000 7ff3f0400740 20 client.26417 trim_cache size 0 max 16384 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:56:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:56:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3c80b6610 msgr2=0x7ff3c80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3c80b6610 0x7ff3c80b8b20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc006010 tx=0x7ff3dc0058d0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 msgr2=0x7ff3e419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:56.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 0x7ff3e419cb70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00e9c0 tx=0x7ff3cc00ee80 comp rx=0 tx=0).stop 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 shutdown_connections 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff3e410dd90 0x7ff3e41a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3e4105280 0x7ff3e419cb70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff3c80b6610 0x7ff3c80b8b20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 --2- 172.21.15.114:0/1529341305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3e4104930 0x7ff3e419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 >> 172.21.15.114:0/1529341305 conn(0x7ff3e40fe3c0 msgr2=0x7ff3e410cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 shutdown_connections 2024-09-21T06:56:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:56:56.500+0000 7ff3e9400640 1 -- 172.21.15.114:0/1529341305 wait complete. 2024-09-21T06:56:56.610 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:56:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:56:57.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:56 smithi114 ceph-mon[30654]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-21T06:56:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:56 smithi183 ceph-mon[34219]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-21T06:56:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:56 smithi193 ceph-mon[34601]: pgmap v480: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-21T06:56:57.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 -- 172.21.15.114:0/1965354539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec0757b0 msgr2=0x7f30ec075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:57.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 --2- 172.21.15.114:0/1965354539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec0757b0 0x7f30ec075bc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009900 tx=0x7f30dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 -- 172.21.15.114:0/1965354539 shutdown_connections 2024-09-21T06:56:57.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 --2- 172.21.15.114:0/1965354539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec10ff10 0x7f30ec112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 --2- 172.21.15.114:0/1965354539 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec076100 0x7f30ec10f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 --2- 172.21.15.114:0/1965354539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec0757b0 0x7f30ec075bc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 -- 172.21.15.114:0/1965354539 >> 172.21.15.114:0/1965354539 conn(0x7f30ec0ff440 msgr2=0x7f30ec101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 -- 172.21.15.114:0/1965354539 shutdown_connections 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.584+0000 7f30f0e00640 1 -- 172.21.15.114:0/1965354539 wait complete. 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.585+0000 7f30f0e00640 1 Processor -- start 2024-09-21T06:56:57.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 -- start start 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 0x7f30ec19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec076100 0x7f30ec19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec10ff10 0x7f30ec1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec115ac0 con 0x7f30ec10ff10 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec115940 con 0x7f30ec076100 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30f0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec115c40 con 0x7f30ec0757b0 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.586+0000 7f30eb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 0x7f30ec19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 0x7f30ec19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44146/0 (socket says 172.21.15.114:44146) 2024-09-21T06:56:57.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 -- 172.21.15.114:0/2485401695 learned_addr learned my addr 172.21.15.114:0/2485401695 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:56:57.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eaa00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec076100 0x7f30ec19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:57.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30ebe00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec10ff10 0x7f30ec1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:57.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 -- 172.21.15.114:0/2485401695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec076100 msgr2=0x7f30ec19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec076100 0x7f30ec19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 -- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec10ff10 msgr2=0x7f30ec1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec10ff10 0x7f30ec1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eb400640 1 -- 172.21.15.114:0/2485401695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec1a2420 con 0x7f30ec0757b0 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30eaa00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec076100 0x7f30ec19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.587+0000 7f30ebe00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec10ff10 0x7f30ec1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.588+0000 7f30eb400640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 0x7f30ec19c4b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005940 tx=0x7f30dc0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.588+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc035070 con 0x7f30ec0757b0 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.588+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec1a26b0 con 0x7f30ec0757b0 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.588+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30dc0044b0 con 0x7f30ec0757b0 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.588+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec103560 con 0x7f30ec0757b0 2024-09-21T06:56:57.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.588+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc049630 con 0x7f30ec0757b0 2024-09-21T06:56:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.590+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec076cc0 con 0x7f30ec0757b0 2024-09-21T06:56:57.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.591+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f30dc0497d0 con 0x7f30ec0757b0 2024-09-21T06:56:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.593+0000 7f30e3e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30d00b6700 0x7f30d00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:56:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.593+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f30dc1107b0 con 0x7f30ec0757b0 2024-09-21T06:56:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.593+0000 7f30eaa00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30d00b6700 0x7f30d00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:56:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.594+0000 7f30eaa00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30d00b6700 0x7f30d00b8c10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f30ec1123a0 tx=0x7f30d8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:56:57.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.597+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f30dc0d6930 con 0x7f30ec0757b0 2024-09-21T06:56:57.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:56:57.728 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:56:57.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 140.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:56:57.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.940+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30ec103810 con 0x7f30ec0757b0 2024-09-21T06:56:57.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.946+0000 7f30e3e00640 1 -- 172.21.15.114:0/2485401695 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7f30dc0dad10 con 0x7f30ec0757b0 2024-09-21T06:56:57.949 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:56:57.954 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:56:57.955 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:56:57.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.953+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30d00b6700 msgr2=0x7f30d00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:57.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.953+0000 7f30f0e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30d00b6700 0x7f30d00b8c10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f30ec1123a0 tx=0x7f30d8005f90 comp rx=0 tx=0).stop 2024-09-21T06:56:57.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.953+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 msgr2=0x7f30ec19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:56:57.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.953+0000 7f30f0e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 0x7f30ec19c4b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005940 tx=0x7f30dc0040c0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.954+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 shutdown_connections 2024-09-21T06:56:57.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.954+0000 7f30f0e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30ec10ff10 0x7f30ec1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.954+0000 7f30f0e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30d00b6700 0x7f30d00b8c10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.954+0000 7f30f0e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30ec076100 0x7f30ec19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.954+0000 7f30f0e00640 1 --2- 172.21.15.114:0/2485401695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30ec0757b0 0x7f30ec19c4b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:56:57.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.954+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 >> 172.21.15.114:0/2485401695 conn(0x7f30ec0ff440 msgr2=0x7f30ec107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:56:57.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.955+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 shutdown_connections 2024-09-21T06:56:57.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:56:57.955+0000 7f30f0e00640 1 -- 172.21.15.114:0/2485401695 wait complete. 2024-09-21T06:56:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:56:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:56:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:56:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:57 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:56:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:56:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:56:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:56:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:57 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:56:58.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:56:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:56:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:56:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:57 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:56:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:58 smithi183 ceph-mon[34219]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-21T06:56:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:56:58 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2485401695' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:59.137 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:58 smithi114 ceph-mon[30654]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-21T06:56:59.137 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:56:58 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2485401695' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:58 smithi193 ceph-mon[34601]: pgmap v481: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-09-21T06:56:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:56:58 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2485401695' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:56:59.182 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 9fbabd86 --release --await' 2024-09-21T06:56:59.182 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 9fbabd86 --release --await 2024-09-21T06:56:59.209 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:01.008 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:00 smithi114 ceph-mon[30654]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-09-21T06:57:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:00 smithi183 ceph-mon[34219]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-09-21T06:57:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:00 smithi193 ceph-mon[34601]: pgmap v482: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-09-21T06:57:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 -- 172.21.15.114:0/2557237660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44109e00 msgr2=0x7f0d44110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 --2- 172.21.15.114:0/2557237660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44109e00 0x7f0d44110630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009ca0 tx=0x7f0d3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 -- 172.21.15.114:0/2557237660 shutdown_connections 2024-09-21T06:57:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 --2- 172.21.15.114:0/2557237660 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44109e00 0x7f0d44110630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 --2- 172.21.15.114:0/2557237660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d441094b0 0x7f0d441098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 --2- 172.21.15.114:0/2557237660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d44103b70 0x7f0d44103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 -- 172.21.15.114:0/2557237660 >> 172.21.15.114:0/2557237660 conn(0x7f0d440ff5e0 msgr2=0x7f0d44101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 -- 172.21.15.114:0/2557237660 shutdown_connections 2024-09-21T06:57:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.191+0000 7f0d4a400640 1 -- 172.21.15.114:0/2557237660 wait complete. 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.192+0000 7f0d4a400640 1 Processor -- start 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.192+0000 7f0d4a400640 1 -- start start 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d4a400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 0x7f0d44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d4a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d441094b0 0x7f0d44075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d4a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d44109e00 0x7f0d44076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d4a400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44113ac0 con 0x7f0d44103b70 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d4a400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44113940 con 0x7f0d441094b0 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d4a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44113c40 con 0x7f0d44109e00 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d43400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 0x7f0d44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d43400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 0x7f0d44075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44428/0 (socket says 172.21.15.114:44428) 2024-09-21T06:57:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d43400640 1 -- 172.21.15.114:0/3702982462 learned_addr learned my addr 172.21.15.114:0/3702982462 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d42a00640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d441094b0 0x7f0d44075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.193+0000 7f0d43e00640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d44109e00 0x7f0d44076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d43400640 1 -- 172.21.15.114:0/3702982462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d44109e00 msgr2=0x7f0d44076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d43400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d44109e00 0x7f0d44076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d43400640 1 -- 172.21.15.114:0/3702982462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d441094b0 msgr2=0x7f0d44075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d43400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d441094b0 0x7f0d44075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d43400640 1 -- 172.21.15.114:0/3702982462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d441a8730 con 0x7f0d44103b70 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d42a00640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d441094b0 0x7f0d44075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.194+0000 7f0d43400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 0x7f0d44075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400ea60 tx=0x7f0d3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.195+0000 7f0d3be00640 1 -- 172.21.15.114:0/3702982462 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3400cde0 con 0x7f0d44103b70 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.195+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d441a8a20 con 0x7f0d44103b70 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.195+0000 7f0d3be00640 1 -- 172.21.15.114:0/3702982462 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340035f0 con 0x7f0d44103b70 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.195+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a8fb0 con 0x7f0d44103b70 2024-09-21T06:57:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.195+0000 7f0d3be00640 1 -- 172.21.15.114:0/3702982462 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34010690 con 0x7f0d44103b70 2024-09-21T06:57:01.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.199+0000 7f0d3be00640 1 -- 172.21.15.114:0/3702982462 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0d340037a0 con 0x7f0d44103b70 2024-09-21T06:57:01.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.199+0000 7f0d511d3740 1 Processor -- start 2024-09-21T06:57:01.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 -- start start 2024-09-21T06:57:01.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 0x5640475af250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564047524e90 con 0x564047207350 2024-09-21T06:57:01.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5640475142c0 con 0x5640471e6670 2024-09-21T06:57:01.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d511d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5640475180d0 con 0x5640471de490 2024-09-21T06:57:01.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d43400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 0x5640475af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.200+0000 7f0d43400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 0x5640475af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44166/0 (socket says 172.21.15.114:44166) 2024-09-21T06:57:01.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 -- 172.21.15.114:0/3668467371 learned_addr learned my addr 172.21.15.114:0/3668467371 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d3be00640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0d280b6680 0x7f0d280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43e00640 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d42a00640 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d3be00640 1 -- 172.21.15.114:0/3702982462 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0d340e7d40 con 0x7f0d44103b70 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 -- 172.21.15.114:0/3668467371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 msgr2=0x5640475af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 -- 172.21.15.114:0/3668467371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 msgr2=0x5640475aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 -- 172.21.15.114:0/3668467371 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564047272930 con 0x5640471de490 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43e00640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0d280b6680 0x7f0d280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.201+0000 7f0d43400640 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 0x5640475af250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ad520 tx=0x7f0d34003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6cefc96a34d0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.202+0000 7f0d38c00640 1 -- 172.21.15.114:0/3668467371 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d340ad330 con 0x5640471de490 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.202+0000 7f0d38c00640 1 -- 172.21.15.114:0/3668467371 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340adf20 con 0x5640471de490 2024-09-21T06:57:01.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.202+0000 7f0d38c00640 1 -- 172.21.15.114:0/3668467371 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d34004260 con 0x5640471de490 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 -- 172.21.15.114:0/3668467371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 msgr2=0x5640475af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 0x5640475af250 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ad520 tx=0x7f0d34003e90 comp rx=0 tx=0).stop 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d43e00640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0d280b6680 0x7f0d280b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009a60 tx=0x7f0d3c02a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 -- 172.21.15.114:0/3668467371 shutdown_connections 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471de490 0x5640475af250 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3668467371 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.203+0000 7f0d511d3740 1 -- 172.21.15.114:0/3668467371 >> 172.21.15.114:0/3668467371 conn(0x5640471fb740 msgr2=0x5640475acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:01.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.204+0000 7f0d511d3740 1 -- 172.21.15.114:0/3668467371 shutdown_connections 2024-09-21T06:57:01.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.204+0000 7f0d511d3740 1 -- 172.21.15.114:0/3668467371 wait complete. 2024-09-21T06:57:01.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.204+0000 7f0d511d3740 1 -- start start 2024-09-21T06:57:01.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.204+0000 7f0d511d3740 1 Processor -- start 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471de490 0x5640475ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475c6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564047375890 con 0x564047207350 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5640474af160 con 0x5640471de490 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5640475bd430 con 0x5640471e6670 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d511d3740 10 client.0 _ll_register_callbacks cb 0x5640474cba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44446/0 (socket says 172.21.15.114:44446) 2024-09-21T06:57:01.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d43e00640 1 -- 172.21.15.114:0/2679945745 learned_addr learned my addr 172.21.15.114:0/2679945745 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:01.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.205+0000 7f0d43400640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471de490 0x5640475ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d42a00640 1 -- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471de490 msgr2=0x5640475ba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d42a00640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471de490 0x5640475ba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d42a00640 1 -- 172.21.15.114:0/2679945745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 msgr2=0x5640475c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d42a00640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d42a00640 1 -- 172.21.15.114:0/2679945745 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640475c6d40 con 0x5640471e6670 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d43e00640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d42a00640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475bae80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d4406bab0 tx=0x7f0d3000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94cd83d0adde7198 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.206+0000 7f0d2d600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.207+0000 7f0d2d600640 1 -- 172.21.15.114:0/2679945745 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30012110 con 0x5640471e6670 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.207+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d340adf20 con 0x5640471e6670 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.207+0000 7f0d2d600640 1 -- 172.21.15.114:0/2679945745 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0d30004050 con 0x5640471e6670 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.207+0000 7f0d2d600640 1 -- 172.21.15.114:0/2679945745 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0d300122b0 con 0x5640471e6670 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.207+0000 7f0d511d3740 10 client.26386 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.207+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5640475d1f10 con 0x5640471e6670 2024-09-21T06:57:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d2d600640 1 -- 172.21.15.114:0/2679945745 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d340adf20 con 0x5640471e6670 2024-09-21T06:57:01.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d2d600640 1 -- 172.21.15.114:0/2679945745 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0d30010430 con 0x5640471e6670 2024-09-21T06:57:01.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d511d3740 10 client.26386 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:01.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d511d3740 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:01.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d511d3740 20 client.26386 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:01.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d511d3740 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640471c5cd0 0x5640475d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d511d3740 4 client.26386 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564047526de0 con 0x5640471c5cd0 2024-09-21T06:57:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.208+0000 7f0d43400640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640471c5cd0 0x5640475d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.209+0000 7f0d43400640 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640471c5cd0 0x5640475d5500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13ef161d1ba63dd server_cookie=22444b085a236ec2 in_seq=0 out_seq=0 2024-09-21T06:57:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.209+0000 7f0d2d600640 10 client.26386 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d2d600640 1 -- 172.21.15.114:0/2679945745 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564047526de0 con 0x5640471c5cd0 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d2d600640 10 client.26386 handle_command_reply: tid=0 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 client.26386 shutdown 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 msgr2=0x5640475bae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475bae80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d4406bab0 tx=0x7f0d3000f790 comp rx=0 tx=0).stop 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 shutdown_connections 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564047207350 0x5640475c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640471c5cd0 0x5640475d5500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475bae80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 --2- 172.21.15.114:0/2679945745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471de490 0x5640475ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d2d600640 0 client.26386 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:01.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.210+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 >> 172.21.15.114:0/2679945745 conn(0x5640471fb740 msgr2=0x5640475ab280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.211+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 shutdown_connections 2024-09-21T06:57:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.211+0000 7f0d511d3740 1 -- 172.21.15.114:0/2679945745 wait complete. 2024-09-21T06:57:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.211+0000 7f0d511d3740 20 client.26386 trim_cache size 0 max 16384 2024-09-21T06:57:01.235 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 Processor -- start 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 -- start start 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564047207350 0x5640475b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441aeba0 con 0x5640471de490 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441b07a0 con 0x5640471e6670 2024-09-21T06:57:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.237+0000 7f0d511d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441af270 con 0x564047207350 2024-09-21T06:57:01.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d43400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564047207350 0x5640475b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564047207350 0x5640475b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44176/0 (socket says 172.21.15.114:44176) 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475ab7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54420/0 (socket says 172.21.15.114:54420) 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d43e00640 1 -- 172.21.15.114:0/3349817085 learned_addr learned my addr 172.21.15.114:0/3349817085 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d43400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44462/0 (socket says 172.21.15.114:44462) 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 -- 172.21.15.114:0/3349817085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564047207350 msgr2=0x5640475b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564047207350 0x5640475b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 -- 172.21.15.114:0/3349817085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 msgr2=0x5640475ab280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475ab280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 -- 172.21.15.114:0/3349817085 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640475bb7e0 con 0x5640471e6670 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.238+0000 7f0d42a00640 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475ab7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000ffd0 tx=0x7f0d30010b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a61d3164f726c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.239+0000 7f0d2fe00640 1 -- 172.21.15.114:0/3349817085 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30000e60 con 0x5640471e6670 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.239+0000 7f0d2fe00640 1 -- 172.21.15.114:0/3349817085 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30005590 con 0x5640471e6670 2024-09-21T06:57:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.239+0000 7f0d2fe00640 1 -- 172.21.15.114:0/3349817085 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30003a50 con 0x5640471e6670 2024-09-21T06:57:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 -- 172.21.15.114:0/3349817085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 msgr2=0x5640475ab7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475ab7c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000ffd0 tx=0x7f0d30010b40 comp rx=0 tx=0).stop 2024-09-21T06:57:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 -- 172.21.15.114:0/3349817085 shutdown_connections 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564047207350 0x5640475b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640471e6670 0x5640475ab7c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 --2- 172.21.15.114:0/3349817085 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475ab280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 -- 172.21.15.114:0/3349817085 >> 172.21.15.114:0/3349817085 conn(0x5640471fb740 msgr2=0x5640475b8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 -- 172.21.15.114:0/3349817085 shutdown_connections 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 -- 172.21.15.114:0/3349817085 wait complete. 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.240+0000 7f0d511d3740 1 -- start start 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 Processor -- start 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 0x5640475bd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441b0010 con 0x5640471de490 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441af940 con 0x564047207350 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441acb80 con 0x5640471e6670 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d511d3740 10 client.0 _ll_register_callbacks cb 0x7f0d30009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 0x5640475bd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d42a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d43e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 0x5640475bd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54436/0 (socket says 172.21.15.114:54436) 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.241+0000 7f0d43e00640 1 -- 172.21.15.114:0/1157571091 learned_addr learned my addr 172.21.15.114:0/1157571091 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43400640 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43e00640 1 -- 172.21.15.114:0/1157571091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 msgr2=0x5640475c2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43e00640 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475c2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43e00640 1 -- 172.21.15.114:0/1157571091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 msgr2=0x5640475c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43e00640 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43e00640 1 -- 172.21.15.114:0/1157571091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640475c32a0 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d43e00640 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 0x5640475bd670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c003ba0 tx=0x7f0d3c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b634203957b3a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d2d600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.242+0000 7f0d2d600640 1 -- 172.21.15.114:0/1157571091 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c058060 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d2d600640 1 -- 172.21.15.114:0/1157571091 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0d3c004450 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30005590 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d2d600640 1 -- 172.21.15.114:0/1157571091 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0d3c05f050 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d511d3740 10 client.26441 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564047640360 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d2d600640 1 -- 172.21.15.114:0/1157571091 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30005590 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.243+0000 7f0d2d600640 1 -- 172.21.15.114:0/1157571091 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0d3c0305c0 con 0x564047207350 2024-09-21T06:57:01.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d511d3740 10 client.26441 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d511d3740 10 client.26441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d511d3740 20 client.26441 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d511d3740 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56404719b080 0x564047642c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d511d3740 4 client.26441 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564047522890 con 0x56404719b080 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d43400640 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56404719b080 0x564047642c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.245+0000 7f0d2d600640 10 client.26441 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:01.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.244+0000 7f0d43400640 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56404719b080 0x564047642c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599251cdc4297206 server_cookie=ba10ad0c5161f59 in_seq=0 out_seq=0 2024-09-21T06:57:01.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.246+0000 7f0d2d600640 1 -- 172.21.15.114:0/1157571091 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564047522890 con 0x56404719b080 2024-09-21T06:57:01.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.246+0000 7f0d2d600640 10 client.26441 handle_command_reply: tid=0 2024-09-21T06:57:01.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.246+0000 7f0d511d3740 1 client.26441 shutdown 2024-09-21T06:57:01.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 msgr2=0x5640475bd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 0x5640475bd670 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c003ba0 tx=0x7f0d3c0095c0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 shutdown_connections 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564047207350 0x5640475bd670 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56404719b080 0x564047642c40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640471e6670 0x5640475c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 --2- 172.21.15.114:0/1157571091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640471de490 0x5640475c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 >> 172.21.15.114:0/1157571091 conn(0x5640471fb740 msgr2=0x5640475b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d2d600640 0 client.26441 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 shutdown_connections 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 1 -- 172.21.15.114:0/1157571091 wait complete. 2024-09-21T06:57:01.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.247+0000 7f0d511d3740 20 client.26441 trim_cache size 0 max 16384 2024-09-21T06:57:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:01.251 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T06:57:01.251 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b", 2024-09-21T06:57:01.251 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T06:57:01.251 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0d280b6680 msgr2=0x7f0d280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0d280b6680 0x7f0d280b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009a60 tx=0x7f0d3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 msgr2=0x7f0d44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 0x7f0d44075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400ea60 tx=0x7f0d3400ef20 comp rx=0 tx=0).stop 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 shutdown_connections 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d44109e00 0x7f0d44076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0d280b6680 0x7f0d280b8b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d441094b0 0x7f0d44075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 --2- 172.21.15.114:0/3702982462 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0d44103b70 0x7f0d44075710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.248+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 >> 172.21.15.114:0/3702982462 conn(0x7f0d440ff5e0 msgr2=0x7f0d44108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.249+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 shutdown_connections 2024-09-21T06:57:01.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:01.249+0000 7f0d4a400640 1 -- 172.21.15.114:0/3702982462 wait complete. 2024-09-21T06:57:01.614 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: 2024-09-21T06:57:01.266+0000 7fba801ed640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...4,head] /volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal/ auth{1=1} v1387 ap=3 f(v5 m2024-09-21T06:56:27.972108+0000 32=31+1) n(v8 rc2024-09-21T06:56:33.342986+0000 b520093696 33=31+2) old_inodes=2 (inest sync->mix g=1) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1727 nref=3)) caps={25444=pAsLsXsFs/pAsLsXsFsxcral@202} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x56315727cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000007:[<0x10000000006/pg_wal v570>,<0x10000000005/data v104>,<0x100000001fb/postgres v17>,<0x100000001fa/17dcb9ad-b76e-4c95-912f-73d77a25c644 v35>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v38>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x10000000007:[<0x10000000006/pg_wal v1387>,<0x10000000005/data v276>,<0x100000001fb/postgres v141>,<0x100000001fa/17dcb9ad-b76e-4c95-912f-73d77a25c644 v163>,<0x10000000001/sv_1 v186>,<0x10000000000/qa v162>,<0x1/volumes v144>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:56:08.116556+0000 31=30+1)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:56:33.342986+0000 b520093696 33=31+2)","memory_value.dirstat":"f(v5 m2024-09-21T06:56:27.972108+0000 32=31+1)","memory_value.rstat":"n(v8 rc2024-09-21T06:56:33.342986+0000 b520093696 33=31+2)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-21T06:57:01.614 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: 2024-09-21T06:57:01.277+0000 7fba801ed640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004a7 [...4,head] /volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/base/13494/ auth{2=1,3=1,4=1} fragtree_t(*^3) v4318 ap=3 f(v1 m2024-09-21T06:55:27.420474+0000 310=310+0) n(v11 rc2024-09-21T06:56:33.359986+0000 b527733295 311=310+1) old_inodes=2 (inest sync->mix g=2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1742 nref=3)) caps={25444=pAsLsXsFs/pAsLsXsFs@390} mcw={2=AsLsXsFs,3=AsLsXsFs,4=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=8 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x563157ed6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004a7:[<0x10000000014/13494 v3232>,<0x10000000006/base v548>,<0x10000000005/data v104>,<0x100000001fb/postgres v17>,<0x100000001fa/17dcb9ad-b76e-4c95-912f-73d77a25c644 v35>,<0x10000000001/sv_1 v57>,<0x10000000000/qa v38>,<0x1/volumes v39>]//[]","memoryvalue":"(2)0x100000004a7:[<0x10000000014/13494 v4318>,<0x10000000006/base v1391>,<0x10000000005/data v276>,<0x100000001fb/postgres v141>,<0x100000001fa/17dcb9ad-b76e-4c95-912f-73d77a25c644 v163>,<0x10000000001/sv_1 v186>,<0x10000000000/qa v162>,<0x1/volumes v144>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-21T06:54:38.049590+0000 309=309+0)","ondisk_value.rstat":"n(v0 rc2024-09-21T06:56:33.359986+0000 b527733295 311=310+1)","memory_value.dirstat":"f(v1 m2024-09-21T06:55:27.420474+0000 310=310+0)","memory_value.rstat":"n(v11 rc2024-09-21T06:56:33.359986+0000 b527733295 311=310+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-21T06:57:02.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 -- 172.21.15.114:0/95224599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 msgr2=0x7fb2c810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 --2- 172.21.15.114:0/95224599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c810bae0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009900 tx=0x7fb2b402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 -- 172.21.15.114:0/95224599 shutdown_connections 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 --2- 172.21.15.114:0/95224599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c810c020 0x7fb2c8112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 --2- 172.21.15.114:0/95224599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c810bae0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 --2- 172.21.15.114:0/95224599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 0x7fb2c81061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.095+0000 7fb2ce200640 1 -- 172.21.15.114:0/95224599 >> 172.21.15.114:0/95224599 conn(0x7fb2c8101860 msgr2=0x7fb2c8103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.096+0000 7fb2ce200640 1 -- 172.21.15.114:0/95224599 shutdown_connections 2024-09-21T06:57:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.096+0000 7fb2ce200640 1 -- 172.21.15.114:0/95224599 wait complete. 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.099 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.097+0000 7fb2ce200640 1 Processor -- start 2024-09-21T06:57:02.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.097+0000 7fb2ce200640 1 -- start start 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2ce200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 0x7fb2c819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2ce200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2ce200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c810c020 0x7fb2c81a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2ce200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8115ce0 con 0x7fb2c810b6d0 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2ce200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8115b60 con 0x7fb2c8105d90 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2ce200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c8115e60 con 0x7fb2c810c020 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44474/0 (socket says 172.21.15.114:44474) 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.098+0000 7fb2c6a00640 1 -- 172.21.15.114:0/2260618501 learned_addr learned my addr 172.21.15.114:0/2260618501 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c7400640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 0x7fb2c819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c810c020 0x7fb2c81a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c6a00640 1 -- 172.21.15.114:0/2260618501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c810c020 msgr2=0x7fb2c81a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c810c020 0x7fb2c81a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c6a00640 1 -- 172.21.15.114:0/2260618501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 msgr2=0x7fb2c819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 0x7fb2c819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c6a00640 1 -- 172.21.15.114:0/2260618501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c81a2640 con 0x7fb2c810b6d0 2024-09-21T06:57:02.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c7400640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 0x7fb2c819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:02.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c819cc70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402a910 tx=0x7fb2b4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:02.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.099+0000 7fb2bfe00640 1 -- 172.21.15.114:0/2260618501 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b403f030 con 0x7fb2c810b6d0 2024-09-21T06:57:02.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.100+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c81a28d0 con 0x7fb2c810b6d0 2024-09-21T06:57:02.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.100+0000 7fb2bfe00640 1 -- 172.21.15.114:0/2260618501 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b4004490 con 0x7fb2c810b6d0 2024-09-21T06:57:02.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.100+0000 7fb2bfe00640 1 -- 172.21.15.114:0/2260618501 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4035070 con 0x7fb2c810b6d0 2024-09-21T06:57:02.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.100+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c81a9090 con 0x7fb2c810b6d0 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.103+0000 7fb2d4dfa740 1 Processor -- start 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.103+0000 7fb2d4dfa740 1 -- start start 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.103+0000 7fb2bfe00640 1 -- 172.21.15.114:0/2260618501 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb2b4042030 con 0x7fb2c810b6d0 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c74700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c74c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f430beb290 con 0x55f4308ac840 2024-09-21T06:57:02.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f430bda6c0 con 0x55f4308cd520 2024-09-21T06:57:02.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f430bde4d0 con 0x55f4308a4660 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c74c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c74700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c74700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44210/0 (socket says 172.21.15.114:44210) 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 -- 172.21.15.114:0/653978438 learned_addr learned my addr 172.21.15.114:0/653978438 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54464/0 (socket says 172.21.15.114:54464) 2024-09-21T06:57:02.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 -- 172.21.15.114:0/653978438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 msgr2=0x55f430c741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 -- 172.21.15.114:0/653978438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 msgr2=0x55f430c74c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c74c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.104+0000 7fb2c7400640 1 -- 172.21.15.114:0/653978438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f430b67460 con 0x55f4308a4660 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2c7400640 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c74700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009d50 tx=0x7fb2b80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9500c26f75b9a71d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2b3e00640 1 -- 172.21.15.114:0/653978438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8012650 con 0x55f4308a4660 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2b3e00640 1 -- 172.21.15.114:0/653978438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b8009e40 con 0x55f4308a4660 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2b3e00640 1 -- 172.21.15.114:0/653978438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b800fb90 con 0x55f4308a4660 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2bfe00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2ac0b6680 0x7fb2ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2bfe00640 1 -- 172.21.15.114:0/2260618501 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb2b4107e90 con 0x7fb2c810b6d0 2024-09-21T06:57:02.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.105+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2ac0b6680 0x7fb2ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/653978438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 msgr2=0x55f430c74700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2ac0b6680 0x7fb2ac0b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005fd0 tx=0x7fb2c0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c74700 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009d50 tx=0x7fb2b80117b0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/653978438 shutdown_connections 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c74c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c74700 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/653978438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/653978438 >> 172.21.15.114:0/653978438 conn(0x55f4308c1910 msgr2=0x55f430c720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.106+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/653978438 shutdown_connections 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.107+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/653978438 wait complete. 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.107+0000 7fb2d4dfa740 1 -- start start 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.107+0000 7fb2d4dfa740 1 Processor -- start 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 0x55f430c7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308ac840 0x55f430c801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308cd520 0x55f430c8bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f430a3bbb0 con 0x55f4308cd520 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f430b756c0 con 0x55f4308a4660 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f430c829a0 con 0x55f4308ac840 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2d4dfa740 10 client.0 _ll_register_callbacks cb 0x55f430b90490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 0x55f430c7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2c7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 0x55f430c7fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54470/0 (socket says 172.21.15.114:54470) 2024-09-21T06:57:02.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2c7400640 1 -- 172.21.15.114:0/1284172121 learned_addr learned my addr 172.21.15.114:0/1284172121 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:02.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308ac840 0x55f430c801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.108+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308cd520 0x55f430c8bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c6a00640 1 -- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 msgr2=0x55f430c7fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 0x55f430c7fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c6a00640 1 -- 172.21.15.114:0/1284172121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308cd520 msgr2=0x55f430c8bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308cd520 0x55f430c8bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c6a00640 1 -- 172.21.15.114:0/1284172121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f430938b00 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c7400640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 0x55f430c7fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308ac840 0x55f430c801b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4105270 tx=0x7fb2b402a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec90e75688501c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2b0c00640 1 -- 172.21.15.114:0/1284172121 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4049070 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2b0c00640 1 -- 172.21.15.114:0/1284172121 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb2b410bd90 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f430bed1e0 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2b0c00640 1 -- 172.21.15.114:0/1284172121 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb2b40c9fc0 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2d4dfa740 10 client.26404 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.109+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4309af960 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2b0c00640 1 -- 172.21.15.114:0/1284172121 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f430bed1e0 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2b0c00640 1 -- 172.21.15.114:0/1284172121 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb2b40c92c0 con 0x55f4308ac840 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2d4dfa740 10 client.26404 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2d4dfa740 10 client.26404 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:57:02.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2d4dfa740 20 client.26404 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f43088bea0 0x55f430c91dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2d4dfa740 4 client.26404 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f430be8c90 con 0x55f43088bea0 2024-09-21T06:57:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.110+0000 7fb2c7400640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f43088bea0 0x55f430c91dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.111+0000 7fb2c7400640 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f43088bea0 0x55f430c91dc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117ffd180d82c48a server_cookie=32f70f362833bf2f in_seq=0 out_seq=0 2024-09-21T06:57:02.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.111+0000 7fb2b0c00640 10 client.26404 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:02.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2b0c00640 1 -- 172.21.15.114:0/1284172121 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f430be8c90 con 0x55f43088bea0 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2b0c00640 10 client.26404 handle_command_reply: tid=0 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 client.26404 shutdown 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308ac840 msgr2=0x55f430c801b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308ac840 0x55f430c801b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4105270 tx=0x7fb2b402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 shutdown_connections 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308cd520 0x55f430c8bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f43088bea0 0x55f430c91dc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308ac840 0x55f430c801b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/1284172121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308a4660 0x55f430c7fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2b0c00640 0 client.26404 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 >> 172.21.15.114:0/1284172121 conn(0x55f4308c1910 msgr2=0x55f430c77350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 shutdown_connections 2024-09-21T06:57:02.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/1284172121 wait complete. 2024-09-21T06:57:02.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.112+0000 7fb2d4dfa740 20 client.26404 trim_cache size 0 max 16384 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-09-21T06:57:02.119 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.120 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.120 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 Processor -- start 2024-09-21T06:57:02.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 -- start start 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308a4660 0x55f430c71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308ac840 0x55f430c8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 0x55f430c8c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81b3030 con 0x55f4308a4660 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81b4c30 con 0x55f4308ac840 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.130+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81b3700 con 0x55f4308cd520 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 0x55f430c8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 0x55f430c8c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44224/0 (socket says 172.21.15.114:44224) 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 -- 172.21.15.114:0/948406723 learned_addr learned my addr 172.21.15.114:0/948406723 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308ac840 0x55f430c8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7400640 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308a4660 0x55f430c71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 -- 172.21.15.114:0/948406723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308ac840 msgr2=0x55f430c8bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308ac840 0x55f430c8bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 -- 172.21.15.114:0/948406723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308a4660 msgr2=0x55f430c71010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308a4660 0x55f430c71010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7e00640 1 -- 172.21.15.114:0/948406723 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f430c724b0 con 0x55f4308cd520 2024-09-21T06:57:02.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.131+0000 7fb2c7400640 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308a4660 0x55f430c71010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:02.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.132+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 0x55f430c8c100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0003fb0 tx=0x7fb2c001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2bfe7dd3f3cffc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.132+0000 7fb2b3400640 1 -- 172.21.15.114:0/948406723 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c002d070 con 0x55f4308cd520 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.132+0000 7fb2b3400640 1 -- 172.21.15.114:0/948406723 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c000fb10 con 0x55f4308cd520 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.132+0000 7fb2b3400640 1 -- 172.21.15.114:0/948406723 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c001dc20 con 0x55f4308cd520 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/948406723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 msgr2=0x55f430c8c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 0x55f430c8c100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0003fb0 tx=0x7fb2c001a000 comp rx=0 tx=0).stop 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/948406723 shutdown_connections 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308cd520 0x55f430c8c100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308ac840 0x55f430c8bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/948406723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308a4660 0x55f430c71010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/948406723 >> 172.21.15.114:0/948406723 conn(0x55f4308c1910 msgr2=0x55f430c719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/948406723 shutdown_connections 2024-09-21T06:57:02.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.133+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/948406723 wait complete. 2024-09-21T06:57:02.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.134+0000 7fb2d4dfa740 1 -- start start 2024-09-21T06:57:02.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.134+0000 7fb2d4dfa740 1 Processor -- start 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c87c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c881d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c82be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81b44a0 con 0x55f4308ac840 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81b3dd0 con 0x55f4308cd520 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c81b1010 con 0x55f4308a4660 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2d4dfa740 10 client.0 _ll_register_callbacks cb 0x7fb2b4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c881d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c881d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44502/0 (socket says 172.21.15.114:44502) 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3755401962 learned_addr learned my addr 172.21.15.114:0/3755401962 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c7400640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c87c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c7e00640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c82be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3755401962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 msgr2=0x55f430c87c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c87c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 msgr2=0x55f430c82be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c82be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3755401962 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f430c83120 con 0x55f4308ac840 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.135+0000 7fb2c7400640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2c6a00640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c881d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40c96d0 tx=0x7fb2b40cce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918a032eb18817c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2b0c00640 1 -- 172.21.15.114:0/3755401962 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4049070 con 0x55f4308ac840 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2b0c00640 1 -- 172.21.15.114:0/3755401962 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb2b40d35e0 con 0x55f4308ac840 2024-09-21T06:57:02.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c000fb10 con 0x55f4308ac840 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2b0c00640 1 -- 172.21.15.114:0/3755401962 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb2b40cd0e0 con 0x55f4308ac840 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2d4dfa740 10 client.16323 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f430cfcf20 con 0x55f4308ac840 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.136+0000 7fb2b0c00640 1 -- 172.21.15.114:0/3755401962 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c000fb10 con 0x55f4308ac840 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2b0c00640 1 -- 172.21.15.114:0/3755401962 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb2b40f2020 con 0x55f4308ac840 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2d4dfa740 10 client.16323 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2d4dfa740 10 client.16323 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2d4dfa740 20 client.16323 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f430861250 0x55f430cff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2d4dfa740 4 client.16323 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9fbabd86", "release": true, "await": true}] 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9fbabd86", "release": true, "await": true}) -- 0x55f430be60c0 con 0x55f430861250 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2c7400640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f430861250 0x55f430cff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2c7400640 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f430861250 0x55f430cff5d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ec8765e7aaae81 server_cookie=9878470d9f231bfd in_seq=0 out_seq=0 2024-09-21T06:57:02.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.137+0000 7fb2b0c00640 10 client.16323 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: Scrub error on inode 0x100000004a7 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/base/13494) see mds.b log and `damage ls` output for details 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:02.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.281+0000 7fb2b0c00640 1 -- 172.21.15.114:0/3755401962 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f430be60c0 con 0x55f430861250 2024-09-21T06:57:02.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.281+0000 7fb2b0c00640 10 client.16323 handle_command_reply: tid=0 2024-09-21T06:57:02.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.281+0000 7fb2d4dfa740 1 client.16323 shutdown 2024-09-21T06:57:02.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 msgr2=0x55f430c881d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c881d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40c96d0 tx=0x7fb2b40cce00 comp rx=0 tx=0).stop 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 shutdown_connections 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f4308cd520 0x55f430c82be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f4308ac840 0x55f430c881d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f430861250 0x55f430cff5d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 --2- 172.21.15.114:0/3755401962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4308a4660 0x55f430c87c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 >> 172.21.15.114:0/3755401962 conn(0x55f4308c1910 msgr2=0x55f430c71910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.282+0000 7fb2b0c00640 0 client.16323 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.283+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 shutdown_connections 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.283+0000 7fb2d4dfa740 1 -- 172.21.15.114:0/3755401962 wait complete. 2024-09-21T06:57:02.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.283+0000 7fb2d4dfa740 20 client.16323 trim_cache size 0 max 16384 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 32, 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 12, 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "9fbabd86": { 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 12, 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:57:02.286 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 147.8, 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:02.287 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.284+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2ac0b6680 msgr2=0x7fb2ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.284+0000 7fb2ce200640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2ac0b6680 0x7fb2ac0b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005fd0 tx=0x7fb2c0009500 comp rx=0 tx=0).stop 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.284+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 msgr2=0x7fb2c819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.284+0000 7fb2ce200640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c819cc70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb2b402a910 tx=0x7fb2b4031a20 comp rx=0 tx=0).stop 2024-09-21T06:57:02.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 shutdown_connections 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2c810c020 0x7fb2c81a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2c810b6d0 0x7fb2c819cc70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2ac0b6680 0x7fb2ac0b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 --2- 172.21.15.114:0/2260618501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2c8105d90 0x7fb2c819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 >> 172.21.15.114:0/2260618501 conn(0x7fb2c8101860 msgr2=0x7fb2c810ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 shutdown_connections 2024-09-21T06:57:02.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:02.285+0000 7fb2ce200640 1 -- 172.21.15.114:0/2260618501 wait complete. 2024-09-21T06:57:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 145.0 s, 78.9 tps, lat 7464.910 ms stddev 12528.857 2024-09-21T06:57:03.050 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:57:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:57:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:57:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:57:03.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:02 smithi183 ceph-mon[34219]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.157 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:02 smithi114 ceph-mon[30654]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-21T06:57:03.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:57:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:57:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:57:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:57:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:02 smithi193 ceph-mon[34601]: pgmap v483: 129 pgs: 129 active+clean; 1.5 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-21T06:57:03.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9fbabd86', total seconds elapsed: 35.0 2024-09-21T06:57:03.771 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.8 2024-09-21T06:57:03.772 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:57:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:04.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 -- 172.21.15.114:0/1294775686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7268102f00 msgr2=0x7f7268107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 --2- 172.21.15.114:0/1294775686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7268102f00 0x7f7268107430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7250009930 tx=0x7f725002a460 comp rx=0 tx=0).stop 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 -- 172.21.15.114:0/1294775686 shutdown_connections 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 --2- 172.21.15.114:0/1294775686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72681079f0 0x7f7268107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 --2- 172.21.15.114:0/1294775686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7268102f00 0x7f7268107430 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 --2- 172.21.15.114:0/1294775686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 0x7f72681029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.849+0000 7f7267e00640 1 -- 172.21.15.114:0/1294775686 >> 172.21.15.114:0/1294775686 conn(0x7f72680fe2c0 msgr2=0x7f7268100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.850+0000 7f7267e00640 1 -- 172.21.15.114:0/1294775686 shutdown_connections 2024-09-21T06:57:04.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.850+0000 7f7267e00640 1 -- 172.21.15.114:0/1294775686 wait complete. 2024-09-21T06:57:04.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.851+0000 7f7267e00640 1 Processor -- start 2024-09-21T06:57:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.853+0000 7f7267e00640 1 -- start start 2024-09-21T06:57:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.854+0000 7f7267e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 0x7f72681a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.854+0000 7f7267e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 0x7f72681a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.854+0000 7f7267e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72681079f0 0x7f72681a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.854+0000 7f7266a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 0x7f72681a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.855+0000 7f7266a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 0x7f72681a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47070/0 (socket says 172.21.15.114:47070) 2024-09-21T06:57:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.856+0000 7f725e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 0x7f72681a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.857+0000 7f725e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 0x7f72681a0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45152/0 (socket says 172.21.15.114:45152) 2024-09-21T06:57:04.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.857+0000 7f7267400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72681079f0 0x7f72681a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.857+0000 7f7267400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72681079f0 0x7f72681a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34570/0 (socket says 172.21.15.114:34570) 2024-09-21T06:57:04.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.857+0000 7f7266a00640 1 -- 172.21.15.114:0/800350378 learned_addr learned my addr 172.21.15.114:0/800350378 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:04.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.858+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7268119fc0 con 0x7f72681025b0 2024-09-21T06:57:04.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.858+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7268119e40 con 0x7f72681079f0 2024-09-21T06:57:04.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.859+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f726811a140 con 0x7f7268102f00 2024-09-21T06:57:04.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.859+0000 7f725e000640 1 -- 172.21.15.114:0/800350378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72681079f0 msgr2=0x7f72681a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.859+0000 7f725e000640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72681079f0 0x7f72681a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.859+0000 7f725e000640 1 -- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 msgr2=0x7f72681a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.859+0000 7f725e000640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 0x7f72681a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.862+0000 7f725e000640 1 -- 172.21.15.114:0/800350378 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72681a68a0 con 0x7f7268102f00 2024-09-21T06:57:04.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.862+0000 7f725e000640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 0x7f72681a0ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7250009230 tx=0x7f7250003b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:04.866 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:04.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.865+0000 7f725fe00640 1 -- 172.21.15.114:0/800350378 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725003f070 con 0x7f7268102f00 2024-09-21T06:57:04.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.865+0000 7f725fe00640 1 -- 172.21.15.114:0/800350378 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7250031b50 con 0x7f7268102f00 2024-09-21T06:57:04.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.865+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72681a6b30 con 0x7f7268102f00 2024-09-21T06:57:04.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.865+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72681ad2f0 con 0x7f7268102f00 2024-09-21T06:57:04.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.865+0000 7f725fe00640 1 -- 172.21.15.114:0/800350378 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250035030 con 0x7f7268102f00 2024-09-21T06:57:04.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.871+0000 7f725fe00640 1 -- 172.21.15.114:0/800350378 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7250042070 con 0x7f7268102f00 2024-09-21T06:57:04.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.872+0000 7f727464d740 1 Processor -- start 2024-09-21T06:57:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.875+0000 7f727464d740 1 -- start start 2024-09-21T06:57:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.876+0000 7f727464d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5e2070 0x562e8a988d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.877+0000 7f725fe00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f724c0b6730 0x7f724c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.877+0000 7f725fe00640 1 -- 172.21.15.114:0/800350378 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7250107fb0 con 0x7f7268102f00 2024-09-21T06:57:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.877+0000 7f727464d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a989270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.877+0000 7f7266a00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f724c0b6730 0x7f724c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.877+0000 7f727464d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5c1390 0x562e8a9897b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.877+0000 7f725e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a989270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.878+0000 7f725e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a989270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34574/0 (socket says 172.21.15.114:34574) 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.878+0000 7f7267400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5e2070 0x562e8a988d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.878+0000 7f7266a00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f724c0b6730 0x7f724c0b8c40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7258009c90 tx=0x7f7258005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.878+0000 7f7267400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5e2070 0x562e8a988d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47076/0 (socket says 172.21.15.114:47076) 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.879+0000 7f727464d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e8a8fff70 con 0x562e8a5e2070 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.879+0000 7f727464d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e8a8ef3a0 con 0x562e8a5b91b0 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.879+0000 7f727464d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e8a8f31b0 con 0x562e8a5c1390 2024-09-21T06:57:04.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.879+0000 7f725e000640 1 -- 172.21.15.114:0/131472686 learned_addr learned my addr 172.21.15.114:0/131472686 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:04.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.880+0000 7f725e000640 1 -- 172.21.15.114:0/131472686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5c1390 msgr2=0x562e8a9897b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.880+0000 7f725e000640 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5c1390 0x562e8a9897b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.880+0000 7f725e000640 1 -- 172.21.15.114:0/131472686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5e2070 msgr2=0x562e8a988d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.880+0000 7f725e000640 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5e2070 0x562e8a988d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.880+0000 7f725e000640 1 -- 172.21.15.114:0/131472686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8a8b9c90 con 0x562e8a5b91b0 2024-09-21T06:57:04.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.881+0000 7f725e000640 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a989270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72500cdad0 tx=0x7f7250031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406acc78c5820d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:04.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.881+0000 7f7257e00640 1 -- 172.21.15.114:0/131472686 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250049070 con 0x562e8a5b91b0 2024-09-21T06:57:04.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.881+0000 7f7257e00640 1 -- 172.21.15.114:0/131472686 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72500cd650 con 0x562e8a5b91b0 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.881+0000 7f7257e00640 1 -- 172.21.15.114:0/131472686 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725003bb30 con 0x562e8a5b91b0 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.883+0000 7f727464d740 1 -- 172.21.15.114:0/131472686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 msgr2=0x562e8a989270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.883+0000 7f727464d740 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a989270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72500cdad0 tx=0x7f7250031400 comp rx=0 tx=0).stop 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.884+0000 7f727464d740 1 -- 172.21.15.114:0/131472686 shutdown_connections 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.884+0000 7f727464d740 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5c1390 0x562e8a9897b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.884+0000 7f727464d740 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a989270 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.884+0000 7f727464d740 1 --2- 172.21.15.114:0/131472686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5e2070 0x562e8a988d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.884+0000 7f727464d740 1 -- 172.21.15.114:0/131472686 >> 172.21.15.114:0/131472686 conn(0x562e8a5d6460 msgr2=0x562e8a986c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:04.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.885+0000 7f727464d740 1 -- 172.21.15.114:0/131472686 shutdown_connections 2024-09-21T06:57:04.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.885+0000 7f727464d740 1 -- 172.21.15.114:0/131472686 wait complete. 2024-09-21T06:57:04.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.886+0000 7f727464d740 1 -- start start 2024-09-21T06:57:04.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.886+0000 7f727464d740 1 Processor -- start 2024-09-21T06:57:04.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a994960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.889 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5c1390 0x562e8a994f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9a0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e8a750700 con 0x562e8a5c1390 2024-09-21T06:57:04.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e8a88a540 con 0x562e8a5b91b0 2024-09-21T06:57:04.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e8a9972d0 con 0x562e8a5e2070 2024-09-21T06:57:04.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.887+0000 7f727464d740 10 client.0 _ll_register_callbacks cb 0x562e8a89a430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:04.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.888+0000 7f7266a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a994960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.888+0000 7f7266a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a994960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34578/0 (socket says 172.21.15.114:34578) 2024-09-21T06:57:04.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.889+0000 7f7266a00640 1 -- 172.21.15.114:0/4290802184 learned_addr learned my addr 172.21.15.114:0/4290802184 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:04.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.889+0000 7f7267400640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.889+0000 7f7266a00640 1 -- 172.21.15.114:0/4290802184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 msgr2=0x562e8a9a0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7266a00640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9a0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7266a00640 1 -- 172.21.15.114:0/4290802184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5c1390 msgr2=0x562e8a994f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7266a00640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5c1390 0x562e8a994f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7266a00640 1 -- 172.21.15.114:0/4290802184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8a64d650 con 0x562e8a5b91b0 2024-09-21T06:57:04.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7267400640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:04.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7266a00640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a994960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f725801ca10 tx=0x7f7258004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e6845fe8633532 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:04.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.890+0000 7f7254c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:04.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.891+0000 7f7254c00640 1 -- 172.21.15.114:0/4290802184 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800eab0 con 0x562e8a5b91b0 2024-09-21T06:57:04.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.891+0000 7f7254c00640 1 -- 172.21.15.114:0/4290802184 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f725800ec50 con 0x562e8a5b91b0 2024-09-21T06:57:04.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.891+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e8a88cb50 con 0x562e8a5b91b0 2024-09-21T06:57:04.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.892+0000 7f7254c00640 1 -- 172.21.15.114:0/4290802184 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f725800ee50 con 0x562e8a5b91b0 2024-09-21T06:57:04.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.892+0000 7f727464d740 10 client.26471 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:04.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.892+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562e8a8a7910 con 0x562e8a5b91b0 2024-09-21T06:57:04.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.893+0000 7f7254c00640 1 -- 172.21.15.114:0/4290802184 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725800ee50 con 0x562e8a5b91b0 2024-09-21T06:57:04.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.894+0000 7f7254c00640 1 -- 172.21.15.114:0/4290802184 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f72580274e0 con 0x562e8a5b91b0 2024-09-21T06:57:04.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.894+0000 7f727464d740 10 client.26471 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:04.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.895+0000 7f727464d740 10 client.26471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:04.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.896+0000 7f727464d740 20 client.26471 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:04.898 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:04 smithi114 ceph-mon[30654]: pgmap v484: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.9 MiB/s wr, 419 op/s 2024-09-21T06:57:04.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.896+0000 7f727464d740 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a5a09f0 0x562e8a9b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.896+0000 7f727464d740 4 client.26471 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.897+0000 7f725e000640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a5a09f0 0x562e8a9b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.898+0000 7f7254c00640 10 client.26471 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.898+0000 7f725e000640 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a5a09f0 0x562e8a9b7710 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2ba99190651ace server_cookie=9e6b7ef84d0e4b60 in_seq=0 out_seq=0 2024-09-21T06:57:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.904+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e8a901ec0 con 0x562e8a5a09f0 2024-09-21T06:57:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.905+0000 7f7254c00640 1 -- 172.21.15.114:0/4290802184 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e8a901ec0 con 0x562e8a5a09f0 2024-09-21T06:57:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.906+0000 7f7254c00640 10 client.26471 handle_command_reply: tid=0 2024-09-21T06:57:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.906+0000 7f727464d740 1 client.26471 shutdown 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 msgr2=0x562e8a994960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a994960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f725801ca10 tx=0x7f7258004730 comp rx=0 tx=0).stop 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 shutdown_connections 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a5a09f0 0x562e8a9b7710 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9a0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5c1390 0x562e8a994f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 --2- 172.21.15.114:0/4290802184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5b91b0 0x562e8a994960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.908+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 >> 172.21.15.114:0/4290802184 conn(0x562e8a5d6460 msgr2=0x562e8a98bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.909+0000 7f7254c00640 0 client.26471 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.911+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 shutdown_connections 2024-09-21T06:57:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.914+0000 7f727464d740 1 -- 172.21.15.114:0/4290802184 wait complete. 2024-09-21T06:57:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.914+0000 7f727464d740 20 client.26471 trim_cache size 0 max 16384 2024-09-21T06:57:04.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 Processor -- start 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 -- start start 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a98de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a98e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72681b0e90 con 0x562e8a5b91b0 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72681aebc0 con 0x562e8a5c1390 2024-09-21T06:57:04.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.953+0000 7f727464d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72681b07c0 con 0x562e8a5e2070 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.954+0000 7f7267400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.954+0000 7f7267400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45192/0 (socket says 172.21.15.114:45192) 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.954+0000 7f7267400640 1 -- 172.21.15.114:0/3637946734 learned_addr learned my addr 172.21.15.114:0/3637946734 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.954+0000 7f725e000640 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a98e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.954+0000 7f7267400640 1 -- 172.21.15.114:0/3637946734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 msgr2=0x562e8a98e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.955+0000 7f7267400640 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a98e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.955+0000 7f7267400640 1 -- 172.21.15.114:0/3637946734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 msgr2=0x562e8a98de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.955+0000 7f7267400640 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a98de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.955+0000 7f7267400640 1 -- 172.21.15.114:0/3637946734 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e8a98b020 con 0x562e8a5e2070 2024-09-21T06:57:04.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.955+0000 7f725e000640 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a98e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:04.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.957+0000 7f7267400640 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9861d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f726000a700 tx=0x7f726000e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94faa6525392afb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:04.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.957+0000 7f7257400640 1 -- 172.21.15.114:0/3637946734 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260012a00 con 0x562e8a5e2070 2024-09-21T06:57:04.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.957+0000 7f7257400640 1 -- 172.21.15.114:0/3637946734 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7260012ba0 con 0x562e8a5e2070 2024-09-21T06:57:04.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.957+0000 7f7257400640 1 -- 172.21.15.114:0/3637946734 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726001e400 con 0x562e8a5e2070 2024-09-21T06:57:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.959+0000 7f727464d740 1 -- 172.21.15.114:0/3637946734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 msgr2=0x562e8a9861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.959+0000 7f727464d740 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9861d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f726000a700 tx=0x7f726000e040 comp rx=0 tx=0).stop 2024-09-21T06:57:04.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 -- 172.21.15.114:0/3637946734 shutdown_connections 2024-09-21T06:57:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a9861d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a98e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 --2- 172.21.15.114:0/3637946734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a98de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 -- 172.21.15.114:0/3637946734 >> 172.21.15.114:0/3637946734 conn(0x562e8a5d6460 msgr2=0x562e8a98f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:04.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 -- 172.21.15.114:0/3637946734 shutdown_connections 2024-09-21T06:57:04.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.960+0000 7f727464d740 1 -- 172.21.15.114:0/3637946734 wait complete. 2024-09-21T06:57:04.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.961+0000 7f727464d740 1 -- start start 2024-09-21T06:57:04.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.961+0000 7f727464d740 1 Processor -- start 2024-09-21T06:57:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.963+0000 7f727464d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a998590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.963+0000 7f727464d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a998ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f727464d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a999010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f727464d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72681af960 con 0x562e8a5b91b0 2024-09-21T06:57:04.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f7267400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a999010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f7267400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a999010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45196/0 (socket says 172.21.15.114:45196) 2024-09-21T06:57:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f7267400640 1 -- 172.21.15.114:0/59262535 learned_addr learned my addr 172.21.15.114:0/59262535 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f727464d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72681af290 con 0x562e8a5c1390 2024-09-21T06:57:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.964+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72681b0030 con 0x562e8a5e2070 2024-09-21T06:57:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f7266a00640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a998590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f725e000640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a998ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f7267400640 1 -- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 msgr2=0x562e8a998ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f7267400640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a998ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f7267400640 1 -- 172.21.15.114:0/59262535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 msgr2=0x562e8a998590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:04.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f7267400640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a998590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.965+0000 7f7267400640 1 -- 172.21.15.114:0/59262535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e8aa21240 con 0x562e8a5e2070 2024-09-21T06:57:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.966+0000 7f727464d740 10 client.0 _ll_register_callbacks cb 0x7f725801c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.966+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7260012ba0 con 0x562e8a5e2070 2024-09-21T06:57:04.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.969+0000 7f7254c00640 10 client.26422 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:04.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.969+0000 7f7267400640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a999010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7260012f80 tx=0x7f7260012b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4419fdf3fc7890fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.969+0000 7f7254c00640 1 -- 172.21.15.114:0/59262535 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7260016620 con 0x562e8a5e2070 2024-09-21T06:57:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.970+0000 7f7254c00640 1 -- 172.21.15.114:0/59262535 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f7260016420 con 0x562e8a5e2070 2024-09-21T06:57:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.970+0000 7f7254c00640 1 -- 172.21.15.114:0/59262535 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7260016620 con 0x562e8a5e2070 2024-09-21T06:57:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.970+0000 7f727464d740 10 client.26422 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:04.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.970+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=32}) -- 0x562e8aa222c0 con 0x562e8a5e2070 2024-09-21T06:57:04.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.971+0000 7f7254c00640 1 -- 172.21.15.114:0/59262535 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f7260031420 con 0x562e8a5e2070 2024-09-21T06:57:04.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f727464d740 10 client.26422 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:04.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f727464d740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:04.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f727464d740 20 client.26422 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:04.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f727464d740 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a575da0 0x562e8aa24930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:04.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f727464d740 4 client.26422 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:57:04.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e8a8fd970 con 0x562e8a575da0 2024-09-21T06:57:04.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.973+0000 7f7266a00640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a575da0 0x562e8aa24930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:04.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.974+0000 7f7266a00640 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a575da0 0x562e8aa24930 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfaf8fdadf91486 server_cookie=dcff43ad016e8a2f in_seq=0 out_seq=0 2024-09-21T06:57:04.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.974+0000 7f7254c00640 10 client.26422 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:04.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.975+0000 7f7254c00640 1 -- 172.21.15.114:0/59262535 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x562e8a8fd970 con 0x562e8a575da0 2024-09-21T06:57:04.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.975+0000 7f7254c00640 10 client.26422 handle_command_reply: tid=0 2024-09-21T06:57:04.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.975+0000 7f727464d740 1 client.26422 shutdown 2024-09-21T06:57:04.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.975+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 msgr2=0x562e8a999010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a999010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7260012f80 tx=0x7f7260012b10 comp rx=0 tx=0).stop 2024-09-21T06:57:04.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 shutdown_connections 2024-09-21T06:57:04.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e8a575da0 0x562e8aa24930 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e8a5e2070 0x562e8a999010 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e8a5c1390 0x562e8a998ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 --2- 172.21.15.114:0/59262535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e8a5b91b0 0x562e8a998590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.976+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 >> 172.21.15.114:0/59262535 conn(0x562e8a5d6460 msgr2=0x562e8a98f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:04.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.977+0000 7f7254c00640 0 client.26422 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:04.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.977+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 shutdown_connections 2024-09-21T06:57:04.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.977+0000 7f727464d740 1 -- 172.21.15.114:0/59262535 wait complete. 2024-09-21T06:57:04.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.978+0000 7f727464d740 20 client.26422 trim_cache size 0 max 16384 2024-09-21T06:57:04.980 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b", 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:04.981 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:57:04.982 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:04.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:04.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:04.984 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:04.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.981+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f724c0b6730 msgr2=0x7f724c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.981+0000 7f7267e00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f724c0b6730 0x7f724c0b8c40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7258009c90 tx=0x7f7258005f90 comp rx=0 tx=0).stop 2024-09-21T06:57:04.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.981+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 msgr2=0x7f72681a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:04.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.981+0000 7f7267e00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 0x7f72681a0ed0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7250009230 tx=0x7f7250003b00 comp rx=0 tx=0).stop 2024-09-21T06:57:04.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.982+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 shutdown_connections 2024-09-21T06:57:04.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.982+0000 7f7267e00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f724c0b6730 0x7f724c0b8c40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.982+0000 7f7267e00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72681079f0 0x7f72681a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.982+0000 7f7267e00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7268102f00 0x7f72681a0ed0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.982+0000 7f7267e00640 1 --2- 172.21.15.114:0/800350378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72681025b0 0x7f72681a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:04.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.982+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 >> 172.21.15.114:0/800350378 conn(0x7f72680fe2c0 msgr2=0x7f7268105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.987+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 shutdown_connections 2024-09-21T06:57:04.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:04.987+0000 7f7267e00640 1 -- 172.21.15.114:0/800350378 wait complete. 2024-09-21T06:57:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:04 smithi183 ceph-mon[34219]: pgmap v484: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.9 MiB/s wr, 419 op/s 2024-09-21T06:57:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:04 smithi193 ceph-mon[34601]: pgmap v484: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 3.9 MiB/s wr, 419 op/s 2024-09-21T06:57:06.313 INFO:tasks.cephfs.filesystem:scrub status for tag:de17ea7f-f4c5-41e1-92ba-3fa9d189f02b - {'path': '/', 'tag': 'de17ea7f-f4c5-41e1-92ba-3fa9d189f02b', 'options': 'recursive,force'} 2024-09-21T06:57:06.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 -- 172.21.15.114:0/3142706544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07411cba0 msgr2=0x7fe07411efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 --2- 172.21.15.114:0/3142706544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07411cba0 0x7fe07411efc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe070009ca0 tx=0x7fe07002a8d0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 -- 172.21.15.114:0/3142706544 shutdown_connections 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 --2- 172.21.15.114:0/3142706544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe07411cba0 0x7fe07411efc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 --2- 172.21.15.114:0/3142706544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 0x7fe07410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 --2- 172.21.15.114:0/3142706544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe074111cd0 0x7fe07410d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.060+0000 7fe07b200640 1 -- 172.21.15.114:0/3142706544 >> 172.21.15.114:0/3142706544 conn(0x7fe07406e9f0 msgr2=0x7fe07406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.061+0000 7fe07b200640 1 -- 172.21.15.114:0/3142706544 shutdown_connections 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.061+0000 7fe07b200640 1 -- 172.21.15.114:0/3142706544 wait complete. 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.062+0000 7fe07b200640 1 Processor -- start 2024-09-21T06:57:06.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 -- start start 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 0x7fe0741a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe074111cd0 0x7fe0741a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe07411cba0 0x7fe0741ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe074122720 con 0x7fe07410dbf0 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0741225a0 con 0x7fe074111cd0 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.063+0000 7fe07b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0741228a0 con 0x7fe07411cba0 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe07a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe07411cba0 0x7fe0741ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe07a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe07411cba0 0x7fe0741ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45218/0 (socket says 172.21.15.114:45218) 2024-09-21T06:57:06.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe07a800640 1 -- 172.21.15.114:0/879350609 learned_addr learned my addr 172.21.15.114:0/879350609 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079400640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe074111cd0 0x7fe0741a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079e00640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 0x7fe0741a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079400640 1 -- 172.21.15.114:0/879350609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe07411cba0 msgr2=0x7fe0741ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079400640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe07411cba0 0x7fe0741ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079400640 1 -- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 msgr2=0x7fe0741a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079400640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 0x7fe0741a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079400640 1 -- 172.21.15.114:0/879350609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0741af040 con 0x7fe074111cd0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.064+0000 7fe079e00640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 0x7fe0741a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.065+0000 7fe079400640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe074111cd0 0x7fe0741a9670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe07406c910 tx=0x7fe06400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.065+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064015020 con 0x7fe074111cd0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.065+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe064011ce0 con 0x7fe074111cd0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.065+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064010d00 con 0x7fe074111cd0 2024-09-21T06:57:06.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.065+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0741af330 con 0x7fe074111cd0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.066+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0741b5a90 con 0x7fe074111cd0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.068+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07410e000 con 0x7fe074111cd0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.073+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe064018020 con 0x7fe074111cd0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.075+0000 7fe06ea00640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe05c0b6800 0x7fe05c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.075+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe0640e70c0 con 0x7fe074111cd0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.075+0000 7fe079e00640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe05c0b6800 0x7fe05c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.077+0000 7fe079e00640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe05c0b6800 0x7fe05c0b8d10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe074110950 tx=0x7fe068005d30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:06.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.079+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe06401f020 con 0x7fe074111cd0 2024-09-21T06:57:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.537+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0741b5e30 con 0x7fe074111cd0 2024-09-21T06:57:06.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.538+0000 7fe06ea00640 1 -- 172.21.15.114:0/879350609 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11996 (secure 0 0 0) 0x7fe0640ad240 con 0x7fe074111cd0 2024-09-21T06:57:06.543 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:57:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 32 2024-09-21T06:57:06.544 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":32,"btime":"2024-09-21T06:54:55:674607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:54:55.609542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T06:57:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.542+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe05c0b6800 msgr2=0x7fe05c0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.542+0000 7fe07b200640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe05c0b6800 0x7fe05c0b8d10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe074110950 tx=0x7fe068005d30 comp rx=0 tx=0).stop 2024-09-21T06:57:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.542+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe074111cd0 msgr2=0x7fe0741a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.542+0000 7fe07b200640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe074111cd0 0x7fe0741a9670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe07406c910 tx=0x7fe06400b850 comp rx=0 tx=0).stop 2024-09-21T06:57:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.543+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 shutdown_connections 2024-09-21T06:57:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.543+0000 7fe07b200640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe07411cba0 0x7fe0741ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.543+0000 7fe07b200640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe05c0b6800 0x7fe05c0b8d10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.543+0000 7fe07b200640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe074111cd0 0x7fe0741a9670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.543+0000 7fe07b200640 1 --2- 172.21.15.114:0/879350609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe07410dbf0 0x7fe0741a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.543+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 >> 172.21.15.114:0/879350609 conn(0x7fe07406e9f0 msgr2=0x7fe074110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.544+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 shutdown_connections 2024-09-21T06:57:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:06.544+0000 7fe07b200640 1 -- 172.21.15.114:0/879350609 wait complete. 2024-09-21T06:57:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:06 smithi183 ceph-mon[34219]: pgmap v485: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.3 MiB/s wr, 465 op/s 2024-09-21T06:57:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:06 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/879350609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:06 smithi193 ceph-mon[34601]: pgmap v485: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.3 MiB/s wr, 465 op/s 2024-09-21T06:57:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:06 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/879350609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:07.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:06 smithi114 ceph-mon[30654]: pgmap v485: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 4.3 MiB/s wr, 465 op/s 2024-09-21T06:57:07.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:06 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/879350609' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:07.502 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0' 2024-09-21T06:57:07.502 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 147.8 --await-for 45.0 2024-09-21T06:57:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 150.0 s, 332.0 tps, lat 302.781 ms stddev 726.553 2024-09-21T06:57:08.142 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 -- 172.21.15.114:0/2600778215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 msgr2=0x7f72cc103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 --2- 172.21.15.114:0/2600778215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 0x7f72cc103fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009900 tx=0x7f72bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 -- 172.21.15.114:0/2600778215 shutdown_connections 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 --2- 172.21.15.114:0/2600778215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc109e20 0x7f72cc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 --2- 172.21.15.114:0/2600778215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 --2- 172.21.15.114:0/2600778215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 0x7f72cc103fa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:08.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.982+0000 7f72d0a00640 1 -- 172.21.15.114:0/2600778215 >> 172.21.15.114:0/2600778215 conn(0x7f72cc0ff5c0 msgr2=0x7f72cc101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:08.985 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:08 smithi114 ceph-mon[30654]: pgmap v486: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-09-21T06:57:08.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.983+0000 7f72d0a00640 1 -- 172.21.15.114:0/2600778215 shutdown_connections 2024-09-21T06:57:08.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.983+0000 7f72d0a00640 1 -- 172.21.15.114:0/2600778215 wait complete. 2024-09-21T06:57:08.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.985+0000 7f72d0a00640 1 Processor -- start 2024-09-21T06:57:08.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.986+0000 7f72d0a00640 1 -- start start 2024-09-21T06:57:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.986+0000 7f72d0a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 0x7f72cc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.986+0000 7f72d0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:08.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72d0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc109e20 0x7f72cc1a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72d0a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc113b30 con 0x7f72cc1094d0 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72d0a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1139b0 con 0x7f72cc103b90 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72d0a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc113cb0 con 0x7f72cc109e20 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 0x7f72cc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72c9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72ca800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc109e20 0x7f72cc1a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72c9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47132/0 (socket says 172.21.15.114:47132) 2024-09-21T06:57:08.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.987+0000 7f72c9400640 1 -- 172.21.15.114:0/1270639791 learned_addr learned my addr 172.21.15.114:0/1270639791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:08.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.988+0000 7f72c9400640 1 -- 172.21.15.114:0/1270639791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc109e20 msgr2=0x7f72cc1a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:08.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.988+0000 7f72c9400640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc109e20 0x7f72cc1a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:08.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.988+0000 7f72c9400640 1 -- 172.21.15.114:0/1270639791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 msgr2=0x7f72cc19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:08.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.988+0000 7f72c9400640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 0x7f72cc19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:08.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.988+0000 7f72c9400640 1 -- 172.21.15.114:0/1270639791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc1a2630 con 0x7f72cc1094d0 2024-09-21T06:57:08.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.989+0000 7f72c9400640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc19cc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f72b800d950 tx=0x7f72b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:08.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.990+0000 7f72c2a00640 1 -- 172.21.15.114:0/1270639791 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8017070 con 0x7f72cc1094d0 2024-09-21T06:57:08.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.990+0000 7f72c2a00640 1 -- 172.21.15.114:0/1270639791 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8009c30 con 0x7f72cc1094d0 2024-09-21T06:57:08.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.990+0000 7f72c2a00640 1 -- 172.21.15.114:0/1270639791 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80047a0 con 0x7f72cc1094d0 2024-09-21T06:57:08.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.991+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc1a2920 con 0x7f72cc1094d0 2024-09-21T06:57:08.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.991+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc1a9080 con 0x7f72cc1094d0 2024-09-21T06:57:08.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.996+0000 7f72d766f740 1 Processor -- start 2024-09-21T06:57:08.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.997+0000 7f72c2a00640 1 -- 172.21.15.114:0/1270639791 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f72b8004020 con 0x7f72cc1094d0 2024-09-21T06:57:09.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.998+0000 7f72d766f740 1 -- start start 2024-09-21T06:57:09.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72d766f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4e13840 0x560cb51bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72d766f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 0x560cb51bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72ca800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4e13840 0x560cb51bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72d766f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4df2b60 0x560cb51bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72ca800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4e13840 0x560cb51bb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34648/0 (socket says 172.21.15.114:34648) 2024-09-21T06:57:09.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72d766f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560cb5130ca0 con 0x560cb4df2b60 2024-09-21T06:57:09.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 0x560cb51bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:08.999+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 0x560cb51bb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45244/0 (socket says 172.21.15.114:45244) 2024-09-21T06:57:09.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72c9e00640 1 -- 172.21.15.114:0/4207328895 learned_addr learned my addr 172.21.15.114:0/4207328895 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560cb51200d0 con 0x560cb4e13840 2024-09-21T06:57:09.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560cb5123ee0 con 0x560cb4dea980 2024-09-21T06:57:09.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72c9e00640 1 -- 172.21.15.114:0/4207328895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4e13840 msgr2=0x560cb51bb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72c9e00640 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4e13840 0x560cb51bb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72c9e00640 1 -- 172.21.15.114:0/4207328895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4df2b60 msgr2=0x560cb51bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72c9e00640 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4df2b60 0x560cb51bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.000+0000 7f72c9e00640 1 -- 172.21.15.114:0/4207328895 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb50d2400 con 0x560cb4dea980 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.001+0000 7f72c9e00640 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 0x560cb51bb890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009230 tx=0x7f72bc004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4aa6f6d304ca8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.001+0000 7f72b7000640 1 -- 172.21.15.114:0/4207328895 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc035040 con 0x560cb4dea980 2024-09-21T06:57:09.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.001+0000 7f72b7000640 1 -- 172.21.15.114:0/4207328895 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72bc0054f0 con 0x560cb4dea980 2024-09-21T06:57:09.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.001+0000 7f72b7000640 1 -- 172.21.15.114:0/4207328895 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc003850 con 0x560cb4dea980 2024-09-21T06:57:09.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.002+0000 7f72c2a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6780 0x7f72b00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.002+0000 7f72c2a00640 1 -- 172.21.15.114:0/1270639791 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f72b80e76f0 con 0x7f72cc1094d0 2024-09-21T06:57:09.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72ca800640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6780 0x7f72b00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 msgr2=0x560cb51bb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 0x560cb51bb890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009230 tx=0x7f72bc004170 comp rx=0 tx=0).stop 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 shutdown_connections 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4df2b60 0x560cb51bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4dea980 0x560cb51bb890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 --2- 172.21.15.114:0/4207328895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4e13840 0x560cb51bb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 >> 172.21.15.114:0/4207328895 conn(0x560cb4e07990 msgr2=0x560cb51b9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 shutdown_connections 2024-09-21T06:57:09.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.003+0000 7f72d766f740 1 -- 172.21.15.114:0/4207328895 wait complete. 2024-09-21T06:57:09.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.004+0000 7f72d766f740 1 -- start start 2024-09-21T06:57:09.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.005+0000 7f72d766f740 1 Processor -- start 2024-09-21T06:57:09.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.005+0000 7f72ca800640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6780 0x7f72b00b8c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72c4005fd0 tx=0x7f72c4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560cb4f817a0 con 0x560cb4dea980 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560cb50c5a10 con 0x560cb4df2b60 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560cb51c9ad0 con 0x560cb4e13840 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72d766f740 10 client.0 _ll_register_callbacks cb 0x560cb50d5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47148/0 (socket says 172.21.15.114:47148) 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72c9e00640 1 -- 172.21.15.114:0/2667401349 learned_addr learned my addr 172.21.15.114:0/2667401349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.006+0000 7f72ca800640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.007+0000 7f72c9400640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.007+0000 7f72c9e00640 1 -- 172.21.15.114:0/2667401349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 msgr2=0x560cb51d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.007+0000 7f72c9e00640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.007+0000 7f72c9e00640 1 -- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 msgr2=0x560cb51c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.007+0000 7f72c9e00640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.007+0000 7f72c9e00640 1 -- 172.21.15.114:0/2667401349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb4e8db90 con 0x560cb4dea980 2024-09-21T06:57:09.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.008+0000 7f72c9e00640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51c6a40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f72bc031440 tx=0x7f72bc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a24491f3fed0a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.009+0000 7f72ab400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:09.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.009+0000 7f72ab400640 1 -- 172.21.15.114:0/2667401349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc035040 con 0x560cb4dea980 2024-09-21T06:57:09.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.009+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72bc0054f0 con 0x560cb4dea980 2024-09-21T06:57:09.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.009+0000 7f72ab400640 1 -- 172.21.15.114:0/2667401349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f72bc004dc0 con 0x560cb4dea980 2024-09-21T06:57:09.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.010+0000 7f72ab400640 1 -- 172.21.15.114:0/2667401349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72bc009520 con 0x560cb4dea980 2024-09-21T06:57:09.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.010+0000 7f72d766f740 10 client.16344 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:09.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.010+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560cb51d5890 con 0x560cb4dea980 2024-09-21T06:57:09.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72ab400640 1 -- 172.21.15.114:0/2667401349 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72bc0054f0 con 0x560cb4dea980 2024-09-21T06:57:09.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72ab400640 1 -- 172.21.15.114:0/2667401349 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f72bc055030 con 0x560cb4dea980 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72d766f740 10 client.16344 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72d766f740 10 client.16344 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72d766f740 20 client.16344 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72d766f740 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4dd21c0 0x560cb51d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72d766f740 4 client.16344 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cb5132bf0 con 0x560cb4dd21c0 2024-09-21T06:57:09.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.011+0000 7f72c9400640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4dd21c0 0x560cb51d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.013+0000 7f72ab400640 10 client.16344 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.013+0000 7f72c9400640 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4dd21c0 0x560cb51d8bf0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702d449c17336467 server_cookie=65d410e573f40389 in_seq=0 out_seq=0 2024-09-21T06:57:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.014+0000 7f72ab400640 1 -- 172.21.15.114:0/2667401349 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560cb5132bf0 con 0x560cb4dd21c0 2024-09-21T06:57:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.014+0000 7f72ab400640 10 client.16344 handle_command_reply: tid=0 2024-09-21T06:57:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 client.16344 shutdown 2024-09-21T06:57:09.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 msgr2=0x560cb51c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51c6a40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f72bc031440 tx=0x7f72bc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 shutdown_connections 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4dd21c0 0x560cb51d8bf0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 --2- 172.21.15.114:0/2667401349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51c6a40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72ab400640 0 client.16344 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.015+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 >> 172.21.15.114:0/2667401349 conn(0x560cb4e07990 msgr2=0x560cb51b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.016+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 shutdown_connections 2024-09-21T06:57:09.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.018+0000 7f72d766f740 1 -- 172.21.15.114:0/2667401349 wait complete. 2024-09-21T06:57:09.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.018+0000 7f72d766f740 20 client.16344 trim_cache size 0 max 16384 2024-09-21T06:57:09.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.056+0000 7f72d766f740 1 Processor -- start 2024-09-21T06:57:09.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.056+0000 7f72d766f740 1 -- start start 2024-09-21T06:57:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72d766f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4dea980 0x560cb51c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72d766f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 0x560cb51c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72d766f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4e13840 0x560cb51c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72d766f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b3090 con 0x560cb4e13840 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72d766f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b4c90 con 0x560cb4dea980 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72d766f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b3760 con 0x560cb4df2b60 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72c9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 0x560cb51c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72c9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 0x560cb51c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45266/0 (socket says 172.21.15.114:45266) 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.057+0000 7f72c9400640 1 -- 172.21.15.114:0/1529416347 learned_addr learned my addr 172.21.15.114:0/1529416347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.058+0000 7f72c9400640 1 -- 172.21.15.114:0/1529416347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4dea980 msgr2=0x560cb51c8800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.058+0000 7f72c9400640 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4dea980 0x560cb51c8800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.058+0000 7f72c9400640 1 -- 172.21.15.114:0/1529416347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4e13840 msgr2=0x560cb51c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.058+0000 7f72c9400640 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4e13840 0x560cb51c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.058+0000 7f72c9400640 1 -- 172.21.15.114:0/1529416347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb51b4c50 con 0x560cb4df2b60 2024-09-21T06:57:09.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.058+0000 7f72c9400640 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 0x560cb51c3c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f72b8005fd0 tx=0x7f72b8094040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a137db0c2f4a709 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.059+0000 7f72b6600640 1 -- 172.21.15.114:0/1529416347 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8098070 con 0x560cb4df2b60 2024-09-21T06:57:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.059+0000 7f72b6600640 1 -- 172.21.15.114:0/1529416347 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b80d0040 con 0x560cb4df2b60 2024-09-21T06:57:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.059+0000 7f72b6600640 1 -- 172.21.15.114:0/1529416347 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80b6b90 con 0x560cb4df2b60 2024-09-21T06:57:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.060+0000 7f72d766f740 1 -- 172.21.15.114:0/1529416347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 msgr2=0x560cb51c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.060+0000 7f72d766f740 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 0x560cb51c3c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f72b8005fd0 tx=0x7f72b8094040 comp rx=0 tx=0).stop 2024-09-21T06:57:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 -- 172.21.15.114:0/1529416347 shutdown_connections 2024-09-21T06:57:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4e13840 0x560cb51c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4df2b60 0x560cb51c3c30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 --2- 172.21.15.114:0/1529416347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4dea980 0x560cb51c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 -- 172.21.15.114:0/1529416347 >> 172.21.15.114:0/1529416347 conn(0x560cb4e07990 msgr2=0x560cb51bcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 -- 172.21.15.114:0/1529416347 shutdown_connections 2024-09-21T06:57:09.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.061+0000 7f72d766f740 1 -- 172.21.15.114:0/1529416347 wait complete. 2024-09-21T06:57:09.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.062+0000 7f72d766f740 1 -- start start 2024-09-21T06:57:09.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.062+0000 7f72d766f740 1 Processor -- start 2024-09-21T06:57:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.063+0000 7f72d766f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.063+0000 7f72d766f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.063+0000 7f72d766f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.063+0000 7f72c9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72c9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51d9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47162/0 (socket says 172.21.15.114:47162) 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72ca800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72ca800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51da8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45270/0 (socket says 172.21.15.114:45270) 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72c9e00640 1 -- 172.21.15.114:0/239774727 learned_addr learned my addr 172.21.15.114:0/239774727 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b4500 con 0x560cb4dea980 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b3e30 con 0x560cb4df2b60 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.064+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1b1070 con 0x560cb4e13840 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.065+0000 7f72ca800640 1 -- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 msgr2=0x560cb51da360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.065+0000 7f72d766f740 10 client.0 _ll_register_callbacks cb 0x7f72c4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.065+0000 7f72ca800640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51da360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.065+0000 7f72ca800640 1 -- 172.21.15.114:0/239774727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 msgr2=0x560cb51d9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.065+0000 7f72ca800640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51d9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.065+0000 7f72ca800640 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb5253530 con 0x560cb4e13840 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.066+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b80d0040 con 0x560cb4e13840 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.066+0000 7f72c9e00640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.066+0000 7f72c9400640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51da360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.066+0000 7f72ca800640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51da8a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72c4004250 tx=0x7f72c4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f96f5bac161c90d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.066+0000 7f72ab400640 10 client.26443 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:09.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.068+0000 7f72ab400640 1 -- 172.21.15.114:0/239774727 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c402e060 con 0x560cb4e13840 2024-09-21T06:57:09.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.068+0000 7f72ab400640 1 -- 172.21.15.114:0/239774727 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f72c400f980 con 0x560cb4e13840 2024-09-21T06:57:09.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.068+0000 7f72ab400640 1 -- 172.21.15.114:0/239774727 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f72c400fbe0 con 0x560cb4e13840 2024-09-21T06:57:09.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.068+0000 7f72d766f740 10 client.26443 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:09.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.068+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=32}) -- 0x560cb5254720 con 0x560cb4e13840 2024-09-21T06:57:09.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.069+0000 7f72ab400640 1 -- 172.21.15.114:0/239774727 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f72c401e430 con 0x560cb4e13840 2024-09-21T06:57:09.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.069+0000 7f72d766f740 10 client.26443 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:09.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.069+0000 7f72d766f740 10 client.26443 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:57:09.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.069+0000 7f72d766f740 20 client.26443 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:09.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.070+0000 7f72d766f740 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4da7570 0x560cb5256f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.070+0000 7f72d766f740 4 client.26443 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}] 2024-09-21T06:57:09.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.070+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.8, "await_for": 45.0}) -- 0x560cb512e6a0 con 0x560cb4da7570 2024-09-21T06:57:09.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.070+0000 7f72c9e00640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4da7570 0x560cb5256f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.071+0000 7f72c9e00640 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4da7570 0x560cb5256f00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf694bbc3659dde server_cookie=6ee76004f5060a73 in_seq=0 out_seq=0 2024-09-21T06:57:09.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.071+0000 7f72ab400640 10 client.26443 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:08 smithi193 ceph-mon[34601]: pgmap v486: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-09-21T06:57:09.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.219+0000 7f72ab400640 1 -- 172.21.15.114:0/239774727 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560cb512e6a0 con 0x560cb4da7570 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.219+0000 7f72ab400640 10 client.26443 handle_command_reply: tid=0 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.219+0000 7f72d766f740 1 client.26443 shutdown 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.219+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 msgr2=0x560cb51da8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.219+0000 7f72d766f740 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51da8a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f72c4004250 tx=0x7f72c4004ab0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 shutdown_connections 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560cb4da7570 0x560cb5256f00 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560cb4e13840 0x560cb51da8a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560cb4df2b60 0x560cb51da360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 --2- 172.21.15.114:0/239774727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560cb4dea980 0x560cb51d9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72ab400640 0 client.26443 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 >> 172.21.15.114:0/239774727 conn(0x560cb4e07990 msgr2=0x560cb51bc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 shutdown_connections 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 1 -- 172.21.15.114:0/239774727 wait complete. 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.220+0000 7f72d766f740 20 client.26443 trim_cache size 0 max 16384 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 32, 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 14, 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:57:09.223 INFO:teuthology.orchestra.run.smithi114.stdout: "ee3cc198": { 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 14, 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 147.8, 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:57:09.224 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:09.225 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:09.225 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6780 msgr2=0x7f72b00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6780 0x7f72b00b8c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72c4005fd0 tx=0x7f72c4009500 comp rx=0 tx=0).stop 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 msgr2=0x7f72cc19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc19cc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f72b800d950 tx=0x7f72b800de10 comp rx=0 tx=0).stop 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 shutdown_connections 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc109e20 0x7f72cc1a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72cc1094d0 0x7f72cc19cc60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6780 0x7f72b00b8c90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 --2- 172.21.15.114:0/1270639791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc103b90 0x7f72cc19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 >> 172.21.15.114:0/1270639791 conn(0x7f72cc0ff5c0 msgr2=0x7f72cc108a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 shutdown_connections 2024-09-21T06:57:09.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:09.221+0000 7f72d0a00640 1 -- 172.21.15.114:0/1270639791 wait complete. 2024-09-21T06:57:09.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:08 smithi183 ceph-mon[34219]: pgmap v486: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 13 MiB/s wr, 1.15k op/s 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 -- 172.21.15.114:0/500031491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410b600 msgr2=0x7f6cc410ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 --2- 172.21.15.114:0/500031491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410b600 0x7f6cc410ba10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009900 tx=0x7f6cb002a410 comp rx=0 tx=0).stop 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 -- 172.21.15.114:0/500031491 shutdown_connections 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 --2- 172.21.15.114:0/500031491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc410bf50 0x7f6cc4112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 --2- 172.21.15.114:0/500031491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410b600 0x7f6cc410ba10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 --2- 172.21.15.114:0/500031491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cc4105cd0 0x7f6cc41060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 -- 172.21.15.114:0/500031491 >> 172.21.15.114:0/500031491 conn(0x7f6cc4101710 msgr2=0x7f6cc4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 -- 172.21.15.114:0/500031491 shutdown_connections 2024-09-21T06:57:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.687+0000 7f6ccae00640 1 -- 172.21.15.114:0/500031491 wait complete. 2024-09-21T06:57:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.688+0000 7f6ccae00640 1 Processor -- start 2024-09-21T06:57:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.688+0000 7f6ccae00640 1 -- start start 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6ccae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 0x7f6cc419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6ccae00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cc410b600 0x7f6cc419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6ccae00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410bf50 0x7f6cc41a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6ccae00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc4115c10 con 0x7f6cc410bf50 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6ccae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc4115a90 con 0x7f6cc4105cd0 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6ccae00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc4115d90 con 0x7f6cc410b600 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 0x7f6cc419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 0x7f6cc419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34704/0 (socket says 172.21.15.114:34704) 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 -- 172.21.15.114:0/2762186559 learned_addr learned my addr 172.21.15.114:0/2762186559 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3400640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cc410b600 0x7f6cc419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410bf50 0x7f6cc41a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 -- 172.21.15.114:0/2762186559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cc410b600 msgr2=0x7f6cc419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cc410b600 0x7f6cc419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 -- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410bf50 msgr2=0x7f6cc41a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410bf50 0x7f6cc41a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.689+0000 7f6cc3e00640 1 -- 172.21.15.114:0/2762186559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc41a2640 con 0x7f6cc4105cd0 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.690+0000 7f6cc3e00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 0x7f6cc419c710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e9f0 tx=0x7f6cb400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.690+0000 7f6cc0c00640 1 -- 172.21.15.114:0/2762186559 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400ccc0 con 0x7f6cc4105cd0 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.690+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc41a2930 con 0x7f6cc4105cd0 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.690+0000 7f6cc0c00640 1 -- 172.21.15.114:0/2762186559 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb400ce60 con 0x7f6cc4105cd0 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.690+0000 7f6cc0c00640 1 -- 172.21.15.114:0/2762186559 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4010430 con 0x7f6cc4105cd0 2024-09-21T06:57:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.690+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc41a90c0 con 0x7f6cc4105cd0 2024-09-21T06:57:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 Processor -- start 2024-09-21T06:57:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 -- start start 2024-09-21T06:57:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 0x557706432c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cc0c00640 1 -- 172.21.15.114:0/2762186559 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6cb40046c0 con 0x7f6cc4105cd0 2024-09-21T06:57:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577060652f0 0x557706433170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770606d4d0 0x5577064336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5577063a86b0 con 0x5577060652f0 2024-09-21T06:57:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557706397ae0 con 0x55770606d4d0 2024-09-21T06:57:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.693+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55770639b8f0 con 0x55770608e1b0 2024-09-21T06:57:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 0x557706432c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577060652f0 0x557706433170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770606d4d0 0x5577064336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 0x557706432c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45286/0 (socket says 172.21.15.114:45286) 2024-09-21T06:57:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 -- 172.21.15.114:0/1687684426 learned_addr learned my addr 172.21.15.114:0/1687684426 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 -- 172.21.15.114:0/1687684426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770606d4d0 msgr2=0x5577064336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770606d4d0 0x5577064336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 -- 172.21.15.114:0/1687684426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577060652f0 msgr2=0x557706433170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577060652f0 0x557706433170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.694+0000 7f6cc8e00640 1 -- 172.21.15.114:0/1687684426 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557706364780 con 0x55770608e1b0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 0x557706432c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc011470 tx=0x7f6cbc011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c175c2f4aed0cf1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cb9600640 1 -- 172.21.15.114:0/1687684426 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc01e070 con 0x55770608e1b0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cb9600640 1 -- 172.21.15.114:0/1687684426 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc003c30 con 0x55770608e1b0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cb9600640 1 -- 172.21.15.114:0/1687684426 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00f920 con 0x55770608e1b0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cc0c00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ca80b6680 0x7f6ca80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cc0c00640 1 -- 172.21.15.114:0/2762186559 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6cb40e7f00 con 0x7f6cc4105cd0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.695+0000 7f6cc3e00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ca80b6680 0x7f6ca80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 -- 172.21.15.114:0/1687684426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 msgr2=0x557706432c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 0x557706432c30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc011470 tx=0x7f6cbc011930 comp rx=0 tx=0).stop 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 -- 172.21.15.114:0/1687684426 shutdown_connections 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55770606d4d0 0x5577064336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5577060652f0 0x557706433170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/1687684426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770608e1b0 0x557706432c30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 -- 172.21.15.114:0/1687684426 >> 172.21.15.114:0/1687684426 conn(0x557706082300 msgr2=0x557706430b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 -- 172.21.15.114:0/1687684426 shutdown_connections 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cd1e61740 1 -- 172.21.15.114:0/1687684426 wait complete. 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.696+0000 7f6cc3e00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ca80b6680 0x7f6ca80b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5577064318b0 tx=0x7f6cb40eb9b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 -- start start 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 Processor -- start 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x55770643e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5577062cfab0 con 0x55770608e1b0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55770633e570 con 0x5577060652f0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5577064413b0 con 0x55770606d4d0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cd1e61740 10 client.0 _ll_register_callbacks cb 0x55770634d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.697+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x55770643e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x55770643e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45290/0 (socket says 172.21.15.114:45290) 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 -- 172.21.15.114:0/3539913170 learned_addr learned my addr 172.21.15.114:0/3539913170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 -- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 msgr2=0x55770643e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 -- 172.21.15.114:0/3539913170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 msgr2=0x55770643ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 -- 172.21.15.114:0/3539913170 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577060f9790 con 0x55770606d4d0 2024-09-21T06:57:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cc3400640 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x55770643e7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009900 tx=0x7f6cb0032e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe223022f487a9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.698+0000 7f6cae000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cae000640 1 -- 172.21.15.114:0/3539913170 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0035040 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cbc003c30 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cae000640 1 -- 172.21.15.114:0/3539913170 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6cb0031760 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cae000640 1 -- 172.21.15.114:0/3539913170 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6cb004f050 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 10 client.26455 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557706172620 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cae000640 1 -- 172.21.15.114:0/3539913170 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc003c30 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cae000640 1 -- 172.21.15.114:0/3539913170 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6cb00054b0 con 0x55770606d4d0 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 10 client.26455 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 10 client.26455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 20 client.26455 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55770604cd80 0x5577064504e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 4 client.26455 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.699+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577063aa600 con 0x55770604cd80 2024-09-21T06:57:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.700+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55770604cd80 0x5577064504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.700+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55770604cd80 0x5577064504e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2571411ec7101805 server_cookie=bfce9cea2c082d39 in_seq=0 out_seq=0 2024-09-21T06:57:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.700+0000 7f6cae000640 10 client.26455 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cae000640 1 -- 172.21.15.114:0/3539913170 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577063aa600 con 0x55770604cd80 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cae000640 10 client.26455 handle_command_reply: tid=0 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 client.26455 shutdown 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 msgr2=0x55770643e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x55770643e7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009900 tx=0x7f6cb0032e90 comp rx=0 tx=0).stop 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 shutdown_connections 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55770604cd80 0x5577064504e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x55770643e7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3539913170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cae000640 0 client.26455 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 >> 172.21.15.114:0/3539913170 conn(0x557706082300 msgr2=0x557706433c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 shutdown_connections 2024-09-21T06:57:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.701+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3539913170 wait complete. 2024-09-21T06:57:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.702+0000 7f6cd1e61740 20 client.26455 trim_cache size 0 max 16384 2024-09-21T06:57:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.716+0000 7f6cd1e61740 1 Processor -- start 2024-09-21T06:57:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 -- start start 2024-09-21T06:57:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x557706434370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41b3010 con 0x55770608e1b0 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41b4c10 con 0x5577060652f0 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41b36e0 con 0x55770606d4d0 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x557706434370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x557706434370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45298/0 (socket says 172.21.15.114:45298) 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cc3400640 1 -- 172.21.15.114:0/3970870763 learned_addr learned my addr 172.21.15.114:0/3970870763 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.717+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47212/0 (socket says 172.21.15.114:47212) 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cc8e00640 1 -- 172.21.15.114:0/3970870763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 msgr2=0x557706434370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x557706434370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cc8e00640 1 -- 172.21.15.114:0/3970870763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 msgr2=0x55770643e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cc8e00640 1 -- 172.21.15.114:0/3970870763 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770644cef0 con 0x55770608e1b0 2024-09-21T06:57:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643b430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc01c040 tx=0x7f6cbc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ee8e51b3ca7979 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cb8c00640 1 -- 172.21.15.114:0/3970870763 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc01e040 con 0x55770608e1b0 2024-09-21T06:57:09.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cb8c00640 1 -- 172.21.15.114:0/3970870763 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc003bd0 con 0x55770608e1b0 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.718+0000 7f6cb8c00640 1 -- 172.21.15.114:0/3970870763 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00f970 con 0x55770608e1b0 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3970870763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 msgr2=0x55770643b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643b430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc01c040 tx=0x7f6cbc009500 comp rx=0 tx=0).stop 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3970870763 shutdown_connections 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x55770643b430 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x557706434370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/3970870763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x55770643e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3970870763 >> 172.21.15.114:0/3970870763 conn(0x557706082300 msgr2=0x557706433c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3970870763 shutdown_connections 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.719+0000 7f6cd1e61740 1 -- 172.21.15.114:0/3970870763 wait complete. 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 -- start start 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 Processor -- start 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x557706446f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x5577064443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x557706444a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41b4480 con 0x55770608e1b0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41b3db0 con 0x5577060652f0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc41b0ff0 con 0x55770606d4d0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.720+0000 7f6cd1e61740 10 client.0 _ll_register_callbacks cb 0x7f6cb0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc8e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x557706444a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x5577064443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x5577064443d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45304/0 (socket says 172.21.15.114:45304) 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 -- 172.21.15.114:0/2012992324 learned_addr learned my addr 172.21.15.114:0/2012992324 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x557706446f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 -- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 msgr2=0x557706446f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x557706446f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 -- 172.21.15.114:0/2012992324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 msgr2=0x557706444a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x557706444a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 -- 172.21.15.114:0/2012992324 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577064451e0 con 0x55770606d4d0 2024-09-21T06:57:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x557706446f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cc3400640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x5577064443d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0004800 tx=0x7f6cb0004c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7655974f812d2d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.721+0000 7f6cae000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cae000640 1 -- 172.21.15.114:0/2012992324 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0058060 con 0x55770606d4d0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cbc003bd0 con 0x55770606d4d0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cae000640 1 -- 172.21.15.114:0/2012992324 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6cb0031d20 con 0x55770606d4d0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cae000640 1 -- 172.21.15.114:0/2012992324 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6cb005f050 con 0x55770606d4d0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cd1e61740 10 client.26461 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5577064baf00 con 0x55770606d4d0 2024-09-21T06:57:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cae000640 1 -- 172.21.15.114:0/2012992324 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cbc003bd0 con 0x55770606d4d0 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cae000640 1 -- 172.21.15.114:0/2012992324 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6cb0005240 con 0x55770606d4d0 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cd1e61740 10 client.26461 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.722+0000 7f6cd1e61740 10 client.26461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cd1e61740 20 client.26461 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5577060221c0 0x5577064bd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cd1e61740 4 client.26461 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577063a60b0 con 0x5577060221c0 2024-09-21T06:57:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5577060221c0 0x5577064bd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cc8e00640 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5577060221c0 0x5577064bd750 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89390803d9d05b3 server_cookie=56fef6431db8d833 in_seq=0 out_seq=0 2024-09-21T06:57:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.723+0000 7f6cae000640 10 client.26461 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cae000640 1 -- 172.21.15.114:0/2012992324 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577063a60b0 con 0x5577060221c0 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cae000640 10 client.26461 handle_command_reply: tid=0 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 client.26461 shutdown 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 msgr2=0x5577064443d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x5577064443d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0004800 tx=0x7f6cb0004c20 comp rx=0 tx=0).stop 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 shutdown_connections 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55770608e1b0 0x557706444a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5577060221c0 0x5577064bd750 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55770606d4d0 0x5577064443d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 --2- 172.21.15.114:0/2012992324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5577060652f0 0x557706446f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cae000640 0 client.26461 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 >> 172.21.15.114:0/2012992324 conn(0x557706082300 msgr2=0x55770642f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 shutdown_connections 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 1 -- 172.21.15.114:0/2012992324 wait complete. 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.732+0000 7f6cd1e61740 20 client.26461 trim_cache size 0 max 16384 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:57:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ca80b6680 msgr2=0x7f6ca80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ca80b6680 0x7f6ca80b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x5577064318b0 tx=0x7f6cb40eb9b0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 msgr2=0x7f6cc419c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 0x7f6cc419c710 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400e9f0 tx=0x7f6cb400edf0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 shutdown_connections 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6cc410bf50 0x7f6cc41a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cc410b600 0x7f6cc419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6ca80b6680 0x7f6ca80b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 --2- 172.21.15.114:0/2762186559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cc4105cd0 0x7f6cc419c710 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 >> 172.21.15.114:0/2762186559 conn(0x7f6cc4101710 msgr2=0x7f6cc4103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 shutdown_connections 2024-09-21T06:57:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:09.733+0000 7f6ccae00640 1 -- 172.21.15.114:0/2762186559 wait complete. 2024-09-21T06:57:11.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:10 smithi193 ceph-mon[34601]: pgmap v487: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2024-09-21T06:57:11.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:10 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:57:11.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:10 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:57:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:10 smithi183 ceph-mon[34219]: pgmap v487: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2024-09-21T06:57:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:10 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:57:11.167 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:10 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:57:11.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:10 smithi114 ceph-mon[30654]: pgmap v487: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 17 MiB/s wr, 1.41k op/s 2024-09-21T06:57:11.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:10 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:57:11.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:10 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:57:12.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:57:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:57:12.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:57:12.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:57:12.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:57:12.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:57:12.167 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee3cc198', quiesce took 8.4 seconds. Will release after: 19.4 2024-09-21T06:57:12.367 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:57:12.367 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:57:12.367 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:57:12.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 155.0 s, 327.8 tps, lat 77.943 ms stddev 91.908 2024-09-21T06:57:13.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:12 smithi193 ceph-mon[34601]: pgmap v488: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-09-21T06:57:13.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:12 smithi114 ceph-mon[30654]: pgmap v488: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-09-21T06:57:13.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:12 smithi183 ceph-mon[34219]: pgmap v488: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2024-09-21T06:57:15.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:14 smithi193 ceph-mon[34601]: pgmap v489: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2024-09-21T06:57:15.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:14 smithi114 ceph-mon[30654]: pgmap v489: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2024-09-21T06:57:15.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:14 smithi183 ceph-mon[34219]: pgmap v489: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 16 MiB/s wr, 1.36k op/s 2024-09-21T06:57:17.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:16 smithi193 ceph-mon[34601]: pgmap v490: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-09-21T06:57:17.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:16 smithi114 ceph-mon[30654]: pgmap v490: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-09-21T06:57:17.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:16 smithi183 ceph-mon[34219]: pgmap v490: 129 pgs: 129 active+clean; 1.7 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2024-09-21T06:57:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 160.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:57:19.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:18 smithi193 ceph-mon[34601]: pgmap v491: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-09-21T06:57:19.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:18 smithi114 ceph-mon[30654]: pgmap v491: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-09-21T06:57:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:18 smithi183 ceph-mon[34219]: pgmap v491: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2024-09-21T06:57:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:20 smithi193 ceph-mon[34601]: pgmap v492: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 525 op/s 2024-09-21T06:57:21.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:20 smithi114 ceph-mon[30654]: pgmap v492: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 525 op/s 2024-09-21T06:57:21.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:20 smithi183 ceph-mon[34219]: pgmap v492: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 525 op/s 2024-09-21T06:57:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 165.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 -- 172.21.15.114:0/3866130261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 msgr2=0x7f5cc810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3866130261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc810ba20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009900 tx=0x7f5cb002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 -- 172.21.15.114:0/3866130261 shutdown_connections 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3866130261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc810bf60 0x7f5cc8112790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3866130261 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc810ba20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3866130261 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc8105ce0 0x7f5cc81060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 -- 172.21.15.114:0/3866130261 >> 172.21.15.114:0/3866130261 conn(0x7f5cc81016e0 msgr2=0x7f5cc8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 -- 172.21.15.114:0/3866130261 shutdown_connections 2024-09-21T06:57:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.925+0000 7f5ccde00640 1 -- 172.21.15.114:0/3866130261 wait complete. 2024-09-21T06:57:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.926+0000 7f5ccde00640 1 Processor -- start 2024-09-21T06:57:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.926+0000 7f5ccde00640 1 -- start start 2024-09-21T06:57:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5ccde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc8105ce0 0x7f5cc819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5ccde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5ccde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc810bf60 0x7f5cc81a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5ccde00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8115c20 con 0x7f5cc810b610 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5ccde00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8115aa0 con 0x7f5cc810bf60 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5ccde00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8115da0 con 0x7f5cc8105ce0 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53526/0 (socket says 172.21.15.114:53526) 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 -- 172.21.15.114:0/3215834876 learned_addr learned my addr 172.21.15.114:0/3215834876 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cc7400640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc810bf60 0x7f5cc81a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc8105ce0 0x7f5cc819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 -- 172.21.15.114:0/3215834876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc8105ce0 msgr2=0x7f5cc819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc8105ce0 0x7f5cc819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 -- 172.21.15.114:0/3215834876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc810bf60 msgr2=0x7f5cc81a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc810bf60 0x7f5cc81a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.927+0000 7f5cbe000640 1 -- 172.21.15.114:0/3215834876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc81a2630 con 0x7f5cc810b610 2024-09-21T06:57:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.928+0000 7f5cbe000640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc819cca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5cb002a910 tx=0x7f5cb0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.928+0000 7f5ccca00640 1 -- 172.21.15.114:0/3215834876 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb003f030 con 0x7f5cc810b610 2024-09-21T06:57:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.928+0000 7f5ccca00640 1 -- 172.21.15.114:0/3215834876 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb0004490 con 0x7f5cc810b610 2024-09-21T06:57:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.928+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc81a28c0 con 0x7f5cc810b610 2024-09-21T06:57:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.928+0000 7f5ccca00640 1 -- 172.21.15.114:0/3215834876 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0035070 con 0x7f5cc810b610 2024-09-21T06:57:22.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.928+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81a90b0 con 0x7f5cc810b610 2024-09-21T06:57:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.931+0000 7f5cd4e7f740 1 Processor -- start 2024-09-21T06:57:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 -- start start 2024-09-21T06:57:22.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5ccca00640 1 -- 172.21.15.114:0/3215834876 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5cb0042030 con 0x7f5cc810b610 2024-09-21T06:57:22.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563f019a86b0 con 0x563f0166d4d0 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563f01997ae0 con 0x563f016652f0 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563f0199b8f0 con 0x563f0168e1b0 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a336b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53540/0 (socket says 172.21.15.114:53540) 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cbe000640 1 -- 172.21.15.114:0/1071501160 learned_addr learned my addr 172.21.15.114:0/1071501160 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.932+0000 7f5cc7400640 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cc6a00640 1 -- 172.21.15.114:0/1071501160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 msgr2=0x563f01a32c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a32c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cc6a00640 1 -- 172.21.15.114:0/1071501160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 msgr2=0x563f01a336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cc6a00640 1 -- 172.21.15.114:0/1071501160 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f01964780 con 0x563f016652f0 2024-09-21T06:57:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cbe000640 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a33170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009c30 tx=0x7f5cb8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bce0a0a78b2a4b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cb7e00640 1 -- 172.21.15.114:0/1071501160 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8004230 con 0x563f016652f0 2024-09-21T06:57:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cb7e00640 1 -- 172.21.15.114:0/1071501160 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb80043d0 con 0x563f016652f0 2024-09-21T06:57:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.933+0000 7f5cb7e00640 1 -- 172.21.15.114:0/1071501160 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8004d00 con 0x563f016652f0 2024-09-21T06:57:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.934+0000 7f5ccca00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5cac0b6680 0x7f5cac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.934+0000 7f5cc7400640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5cac0b6680 0x7f5cac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.934+0000 7f5ccca00640 1 -- 172.21.15.114:0/3215834876 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5cb0107db0 con 0x7f5cc810b610 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.934+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1071501160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 msgr2=0x563f01a33170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.934+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a33170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009c30 tx=0x7f5cb8011810 comp rx=0 tx=0).stop 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1071501160 shutdown_connections 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a33170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1071501160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a32c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1071501160 >> 172.21.15.114:0/1071501160 conn(0x563f01682300 msgr2=0x563f01a30b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cc7400640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5cac0b6680 0x7f5cac0b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0005fd0 tx=0x7f5cc0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1071501160 shutdown_connections 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1071501160 wait complete. 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.935+0000 7f5cd4e7f740 1 -- start start 2024-09-21T06:57:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 Processor -- start 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a3e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0166d4d0 0x563f01a3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563f018cfab0 con 0x563f0168e1b0 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563f0193e570 con 0x563f016652f0 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563f01a413b0 con 0x563f0166d4d0 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cd4e7f740 10 client.0 _ll_register_callbacks cb 0x563f0194d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cc7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a3e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cc7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a3ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53554/0 (socket says 172.21.15.114:53554) 2024-09-21T06:57:22.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a3e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37932/0 (socket says 172.21.15.114:37932) 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.936+0000 7f5cc7400640 1 -- 172.21.15.114:0/4224821639 learned_addr learned my addr 172.21.15.114:0/4224821639 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cbe000640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0166d4d0 0x563f01a3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc7400640 1 -- 172.21.15.114:0/4224821639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0166d4d0 msgr2=0x563f01a3e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc7400640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0166d4d0 0x563f01a3e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc7400640 1 -- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 msgr2=0x563f01a3e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc7400640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a3e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc7400640 1 -- 172.21.15.114:0/4224821639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f016f9790 con 0x563f0168e1b0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a3e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cc7400640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a3ec90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00099c0 tx=0x7f5cc0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be76111f790b0aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cb4c00640 1 -- 172.21.15.114:0/4224821639 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0004b20 con 0x563f0168e1b0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cb4c00640 1 -- 172.21.15.114:0/4224821639 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5cc0004cc0 con 0x563f0168e1b0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.937+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb80043d0 con 0x563f0168e1b0 2024-09-21T06:57:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cb4c00640 1 -- 172.21.15.114:0/4224821639 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5cc000f7d0 con 0x563f0168e1b0 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 10 client.16377 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f01772620 con 0x563f0168e1b0 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cb4c00640 1 -- 172.21.15.114:0/4224821639 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb80043d0 con 0x563f0168e1b0 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cb4c00640 1 -- 172.21.15.114:0/4224821639 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f5cc0026070 con 0x563f0168e1b0 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 10 client.16377 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 20 client.16377 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f0164cd80 0x563f01a505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 4 client.16377 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.938+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f019aa600 con 0x563f0164cd80 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.939+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f0164cd80 0x563f01a505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.939+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f0164cd80 0x563f01a505a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb2a5f89c9270fa9 server_cookie=b8a42b29e18dc2ba in_seq=0 out_seq=0 2024-09-21T06:57:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.939+0000 7f5cb4c00640 10 client.16377 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cb4c00640 1 -- 172.21.15.114:0/4224821639 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f019aa600 con 0x563f0164cd80 2024-09-21T06:57:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cb4c00640 10 client.16377 handle_command_reply: tid=0 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cd4e7f740 1 client.16377 shutdown 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 msgr2=0x563f01a3ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a3ec90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00099c0 tx=0x7f5cc0004ab0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 shutdown_connections 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a3ec90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.940+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f0164cd80 0x563f01a505a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0166d4d0 0x563f01a3e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/4224821639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a3e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cb4c00640 0 client.16377 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 >> 172.21.15.114:0/4224821639 conn(0x563f01682300 msgr2=0x563f01a33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 shutdown_connections 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/4224821639 wait complete. 2024-09-21T06:57:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.941+0000 7f5cd4e7f740 20 client.16377 trim_cache size 0 max 16384 2024-09-21T06:57:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 Processor -- start 2024-09-21T06:57:22.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 -- start start 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a2faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a34370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a55d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b30c0 con 0x563f0166d4d0 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b4cc0 con 0x563f016652f0 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b3790 con 0x563f0168e1b0 2024-09-21T06:57:22.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a34370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a34370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53566/0 (socket says 172.21.15.114:53566) 2024-09-21T06:57:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cbe000640 1 -- 172.21.15.114:0/1044380131 learned_addr learned my addr 172.21.15.114:0/1044380131 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a2faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.956+0000 7f5cc7400640 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a55d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cbe000640 1 -- 172.21.15.114:0/1044380131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 msgr2=0x563f01a55d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cbe000640 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a55d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cbe000640 1 -- 172.21.15.114:0/1044380131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 msgr2=0x563f01a2faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cbe000640 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a2faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cbe000640 1 -- 172.21.15.114:0/1044380131 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f01a2bfa0 con 0x563f0166d4d0 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cc7400640 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a55d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cbe000640 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a34370 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0105240 tx=0x7f5cb010b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ff9fdf72b650ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cb7400640 1 -- 172.21.15.114:0/1044380131 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0049040 con 0x563f0166d4d0 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cb7400640 1 -- 172.21.15.114:0/1044380131 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb00cd450 con 0x563f0166d4d0 2024-09-21T06:57:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.957+0000 7f5cb7400640 1 -- 172.21.15.114:0/1044380131 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00cd0b0 con 0x563f0166d4d0 2024-09-21T06:57:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1044380131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 msgr2=0x563f01a34370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a34370 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0105240 tx=0x7f5cb010b900 comp rx=0 tx=0).stop 2024-09-21T06:57:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1044380131 shutdown_connections 2024-09-21T06:57:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f0168e1b0 0x563f01a55d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0166d4d0 0x563f01a34370 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/1044380131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f016652f0 0x563f01a2faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1044380131 >> 172.21.15.114:0/1044380131 conn(0x563f01682300 msgr2=0x563f01a33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1044380131 shutdown_connections 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/1044380131 wait complete. 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.958+0000 7f5cd4e7f740 1 -- start start 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 Processor -- start 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 0x563f01a446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f0166d4d0 0x563f01a44c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a45150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b4530 con 0x563f0168e1b0 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b3e60 con 0x563f0166d4d0 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81b10a0 con 0x563f016652f0 2024-09-21T06:57:22.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cd4e7f740 10 client.0 _ll_register_callbacks cb 0x7f5cc0009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 0x563f01a446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cbe000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f0166d4d0 0x563f01a44c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cc7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a45150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cc6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 0x563f01a446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43988/0 (socket says 172.21.15.114:43988) 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.959+0000 7f5cc6a00640 1 -- 172.21.15.114:0/3865194622 learned_addr learned my addr 172.21.15.114:0/3865194622 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cc6a00640 1 -- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f0166d4d0 msgr2=0x563f01a44c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f0166d4d0 0x563f01a44c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cc6a00640 1 -- 172.21.15.114:0/3865194622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 msgr2=0x563f01a45150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a45150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cc6a00640 1 -- 172.21.15.114:0/3865194622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f01aba020 con 0x563f016652f0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cc6a00640 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 0x563f01a446d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009710 tx=0x7f5cb80046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18768271b70265b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cb4c00640 1 -- 172.21.15.114:0/3865194622 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8031070 con 0x563f016652f0 2024-09-21T06:57:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb00cd450 con 0x563f016652f0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.960+0000 7f5cb4c00640 1 -- 172.21.15.114:0/3865194622 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5cb8004450 con 0x563f016652f0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cb4c00640 1 -- 172.21.15.114:0/3865194622 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5cb803d050 con 0x563f016652f0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 10 client.26479 fetch_fsmap learned FSMap version 32 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f01abb2b0 con 0x563f016652f0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cb4c00640 1 -- 172.21.15.114:0/3865194622 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cb00cd450 con 0x563f016652f0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cb4c00640 1 -- 172.21.15.114:0/3865194622 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f5cb8024040 con 0x563f016652f0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 10 client.26479 fetch_fsmap finished waiting for FSMap version 32 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 10 client.26479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 20 client.26479 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f016221c0 0x563f01abd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 4 client.26479 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f019a60b0 con 0x563f016221c0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.961+0000 7f5cbe000640 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f016221c0 0x563f01abd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:22.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.962+0000 7f5cb4c00640 10 client.26479 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.962+0000 7f5cbe000640 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f016221c0 0x563f01abd960 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46aaa3c5bf5434e server_cookie=79919669240f3bf1 in_seq=0 out_seq=0 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cb4c00640 1 -- 172.21.15.114:0/3865194622 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f019a60b0 con 0x563f016221c0 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cb4c00640 10 client.26479 handle_command_reply: tid=0 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 client.26479 shutdown 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 msgr2=0x563f01a446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 0x563f01a446d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009710 tx=0x7f5cb80046e0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 shutdown_connections 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563f0168e1b0 0x563f01a45150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f0166d4d0 0x563f01a44c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563f016221c0 0x563f01abd960 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 --2- 172.21.15.114:0/3865194622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563f016652f0 0x563f01a446d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cb4c00640 0 client.26479 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 >> 172.21.15.114:0/3865194622 conn(0x563f01682300 msgr2=0x563f01a2f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 shutdown_connections 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 1 -- 172.21.15.114:0/3865194622 wait complete. 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.972+0000 7f5cd4e7f740 20 client.26479 trim_cache size 0 max 16384 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:57:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5cac0b6680 msgr2=0x7f5cac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5cac0b6680 0x7f5cac0b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0005fd0 tx=0x7f5cc0009500 comp rx=0 tx=0).stop 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 msgr2=0x7f5cc819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc819cca0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5cb002a910 tx=0x7f5cb0031a20 comp rx=0 tx=0).stop 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 shutdown_connections 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5cac0b6680 0x7f5cac0b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cc810bf60 0x7f5cc81a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5cc810b610 0x7f5cc819cca0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 --2- 172.21.15.114:0/3215834876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5cc8105ce0 0x7f5cc819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 >> 172.21.15.114:0/3215834876 conn(0x7f5cc81016e0 msgr2=0x7f5cc8108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 shutdown_connections 2024-09-21T06:57:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:22.973+0000 7f5ccde00640 1 -- 172.21.15.114:0/3215834876 wait complete. 2024-09-21T06:57:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:22 smithi193 ceph-mon[34601]: pgmap v493: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-09-21T06:57:23.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:22 smithi114 ceph-mon[30654]: pgmap v493: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-09-21T06:57:23.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:22 smithi183 ceph-mon[34219]: pgmap v493: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-09-21T06:57:25.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:24 smithi193 ceph-mon[34601]: pgmap v494: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 270 op/s 2024-09-21T06:57:25.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:24 smithi114 ceph-mon[30654]: pgmap v494: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 270 op/s 2024-09-21T06:57:25.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:24 smithi183 ceph-mon[34219]: pgmap v494: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.5 MiB/s wr, 270 op/s 2024-09-21T06:57:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:26 smithi193 ceph-mon[34601]: pgmap v495: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-21T06:57:27.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:26 smithi114 ceph-mon[30654]: pgmap v495: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-21T06:57:27.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:26 smithi183 ceph-mon[34219]: pgmap v495: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-21T06:57:27.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 170.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:57:29.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:28 smithi114 ceph-mon[30654]: pgmap v496: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-21T06:57:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:28 smithi183 ceph-mon[34219]: pgmap v496: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-21T06:57:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:28 smithi193 ceph-mon[34601]: pgmap v496: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-09-21T06:57:30.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:30 smithi193 ceph-mon[34601]: pgmap v497: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 914 KiB/s wr, 82 op/s 2024-09-21T06:57:31.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:30 smithi114 ceph-mon[30654]: pgmap v497: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 914 KiB/s wr, 82 op/s 2024-09-21T06:57:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:30 smithi183 ceph-mon[34219]: pgmap v497: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 914 KiB/s wr, 82 op/s 2024-09-21T06:57:31.568 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:57:31.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:31 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:31.952 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:32.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:31 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:32.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:31 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 -- 172.21.15.114:0/4090074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 msgr2=0x7f4f4010fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 --2- 172.21.15.114:0/4090074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4010fc10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009900 tx=0x7f4f2802a410 comp rx=0 tx=0).stop 2024-09-21T06:57:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 -- 172.21.15.114:0/4090074 shutdown_connections 2024-09-21T06:57:32.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 --2- 172.21.15.114:0/4090074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f40110150 0x7f4f40112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 --2- 172.21.15.114:0/4090074 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4010fc10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 --2- 172.21.15.114:0/4090074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 0x7f4f40075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 -- 172.21.15.114:0/4090074 >> 172.21.15.114:0/4090074 conn(0x7f4f400ff600 msgr2=0x7f4f40101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 -- 172.21.15.114:0/4090074 shutdown_connections 2024-09-21T06:57:32.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.597+0000 7f4f47400640 1 -- 172.21.15.114:0/4090074 wait complete. 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.598+0000 7f4f47400640 1 Processor -- start 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 -- start start 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 0x7f4f4019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f40110150 0x7f4f401a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40115d10 con 0x7f4f400760a0 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40115b90 con 0x7f4f40110150 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f47400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f40115e90 con 0x7f4f40075750 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f37e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f37e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49934/0 (socket says 172.21.15.114:49934) 2024-09-21T06:57:32.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f44a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 0x7f4f4019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:32.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.599+0000 7f4f37e00640 1 -- 172.21.15.114:0/3502301467 learned_addr learned my addr 172.21.15.114:0/3502301467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:32.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f45400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f40110150 0x7f4f401a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:32.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f44a00640 1 -- 172.21.15.114:0/3502301467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f40110150 msgr2=0x7f4f401a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:32.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f44a00640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f40110150 0x7f4f401a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f44a00640 1 -- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 msgr2=0x7f4f4019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f44a00640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f44a00640 1 -- 172.21.15.114:0/3502301467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f401a2730 con 0x7f4f40075750 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f44a00640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 0x7f4f4019c7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000e9f0 tx=0x7f4f3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000ccc0 con 0x7f4f40075750 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30005020 con 0x7f4f40075750 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30004a00 con 0x7f4f40075750 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.600+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f401a2a20 con 0x7f4f40075750 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.601+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f401a9180 con 0x7f4f40075750 2024-09-21T06:57:32.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.602+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40076c60 con 0x7f4f40075750 2024-09-21T06:57:32.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.604+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4f30004ba0 con 0x7f4f40075750 2024-09-21T06:57:32.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.606+0000 7f4f3e000640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f240b6680 0x7f4f240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:32.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.606+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4f300e7b20 con 0x7f4f40075750 2024-09-21T06:57:32.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.606+0000 7f4f37e00640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f240b6680 0x7f4f240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:32.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.607+0000 7f4f37e00640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f240b6680 0x7f4f240b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4f28006010 tx=0x7f4f28005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:32.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.607+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4f300add30 con 0x7f4f40075750 2024-09-21T06:57:32.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:32 smithi114 ceph-mon[30654]: pgmap v498: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 488 KiB/s wr, 28 op/s 2024-09-21T06:57:32.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 175.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:57:32.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.838+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f40075b60 con 0x7f4f40075750 2024-09-21T06:57:32.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.839+0000 7f4f3e000640 1 -- 172.21.15.114:0/3502301467 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11994 (secure 0 0 0) 0x7f4f300b2110 con 0x7f4f40075750 2024-09-21T06:57:32.841 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:57:32.841 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":33,"btime":"2024-09-21T06:57:30:630006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:57:30.612026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:57:32.841 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 33 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f240b6680 msgr2=0x7f4f240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f240b6680 0x7f4f240b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4f28006010 tx=0x7f4f28005a20 comp rx=0 tx=0).stop 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 msgr2=0x7f4f4019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 0x7f4f4019c7c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000e9f0 tx=0x7f4f3000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 shutdown_connections 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f240b6680 0x7f4f240b8b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f40110150 0x7f4f401a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f400760a0 0x7f4f4019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 --2- 172.21.15.114:0/3502301467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f40075750 0x7f4f4019c7c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:32.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 >> 172.21.15.114:0/3502301467 conn(0x7f4f400ff600 msgr2=0x7f4f40109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:32.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 shutdown_connections 2024-09-21T06:57:32.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:32.841+0000 7f4f47400640 1 -- 172.21.15.114:0/3502301467 wait complete. 2024-09-21T06:57:32.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:32 smithi193 ceph-mon[34601]: pgmap v498: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 488 KiB/s wr, 28 op/s 2024-09-21T06:57:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:32 smithi183 ceph-mon[34219]: pgmap v498: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 488 KiB/s wr, 28 op/s 2024-09-21T06:57:33.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:33 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3502301467' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:34.016 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:33 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3502301467' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:34.055 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id ee3cc198 --release --await' 2024-09-21T06:57:34.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id ee3cc198 --release --await 2024-09-21T06:57:34.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:33 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3502301467' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:34.439 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:34.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:34 smithi193 ceph-mon[34601]: pgmap v499: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 191 KiB/s wr, 31 op/s 2024-09-21T06:57:34.912 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:34 smithi114 ceph-mon[30654]: pgmap v499: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 191 KiB/s wr, 31 op/s 2024-09-21T06:57:35.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 -- 172.21.15.114:0/2970138827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8114570 msgr2=0x7f43f8116990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 --2- 172.21.15.114:0/2970138827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8114570 0x7f43f8116990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009ca0 tx=0x7f43f002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 -- 172.21.15.114:0/2970138827 shutdown_connections 2024-09-21T06:57:35.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 --2- 172.21.15.114:0/2970138827 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8114570 0x7f43f8116990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 --2- 172.21.15.114:0/2970138827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f8078290 0x7f43f8114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 --2- 172.21.15.114:0/2970138827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43f8077940 0x7f43f8077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 -- 172.21.15.114:0/2970138827 >> 172.21.15.114:0/2970138827 conn(0x7f43f8101680 msgr2=0x7f43f8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:35.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 -- 172.21.15.114:0/2970138827 shutdown_connections 2024-09-21T06:57:35.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.098+0000 7f43fee00640 1 -- 172.21.15.114:0/2970138827 wait complete. 2024-09-21T06:57:35.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.099+0000 7f43fee00640 1 Processor -- start 2024-09-21T06:57:35.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.099+0000 7f43fee00640 1 -- start start 2024-09-21T06:57:35.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 0x7f43f81a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43f8078290 0x7f43f81a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f8114570 0x7f43f81a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8119f40 con 0x7f43f8077940 2024-09-21T06:57:35.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43f8119dc0 con 0x7f43f8078290 2024-09-21T06:57:35.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43f811a0c0 con 0x7f43f8114570 2024-09-21T06:57:35.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 0x7f43f81a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 0x7f43f81a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41070/0 (socket says 172.21.15.114:41070) 2024-09-21T06:57:35.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.100+0000 7f43fda00640 1 -- 172.21.15.114:0/3877670688 learned_addr learned my addr 172.21.15.114:0/3877670688 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:35.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fd000640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43f8078290 0x7f43f81a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fe400640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f8114570 0x7f43f81a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fda00640 1 -- 172.21.15.114:0/3877670688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f8114570 msgr2=0x7f43f81a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fda00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f8114570 0x7f43f81a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fda00640 1 -- 172.21.15.114:0/3877670688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43f8078290 msgr2=0x7f43f81a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fda00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43f8078290 0x7f43f81a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fda00640 1 -- 172.21.15.114:0/3877670688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f81a6820 con 0x7f43f8077940 2024-09-21T06:57:35.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.101+0000 7f43fda00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 0x7f43f81a08b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00ea60 tx=0x7f43ec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:35.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.102+0000 7f43f6000640 1 -- 172.21.15.114:0/3877670688 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00cde0 con 0x7f43f8077940 2024-09-21T06:57:35.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.102+0000 7f43f6000640 1 -- 172.21.15.114:0/3877670688 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43ec0035f0 con 0x7f43f8077940 2024-09-21T06:57:35.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.102+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f81a6b10 con 0x7f43f8077940 2024-09-21T06:57:35.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.102+0000 7f43f6000640 1 -- 172.21.15.114:0/3877670688 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec010690 con 0x7f43f8077940 2024-09-21T06:57:35.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.102+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f81ad270 con 0x7f43f8077940 2024-09-21T06:57:35.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.105+0000 7f44070bd740 1 Processor -- start 2024-09-21T06:57:35.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.105+0000 7f44070bd740 1 -- start start 2024-09-21T06:57:35.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f44070bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d85e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43f6000640 1 -- 172.21.15.114:0/3877670688 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f43ec0037a0 con 0x7f43f8077940 2024-09-21T06:57:35.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f44070bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d85e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f44070bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d85edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f44070bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556e3d7d5290 con 0x556e3d48e660 2024-09-21T06:57:35.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f44070bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556e3d7c46c0 con 0x556e3d4b7520 2024-09-21T06:57:35.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f44070bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e3d7c84d0 con 0x556e3d496840 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d85e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d85e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37334/0 (socket says 172.21.15.114:37334) 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 -- 172.21.15.114:0/3473514357 learned_addr learned my addr 172.21.15.114:0/3473514357 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fda00640 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d85e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 -- 172.21.15.114:0/3473514357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 msgr2=0x556e3d85edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d85edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 -- 172.21.15.114:0/3473514357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 msgr2=0x556e3d85e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d85e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.106+0000 7f43fe400640 1 -- 172.21.15.114:0/3473514357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e3d751460 con 0x556e3d4b7520 2024-09-21T06:57:35.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.107+0000 7f43fda00640 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d85e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:35.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.107+0000 7f43fe400640 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d85e340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009ca0 tx=0x7f43f0009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83543e1b922458b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:35.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.107+0000 7f43eaa00640 1 -- 172.21.15.114:0/3473514357 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031cf0 con 0x556e3d4b7520 2024-09-21T06:57:35.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.107+0000 7f43eaa00640 1 -- 172.21.15.114:0/3473514357 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f0036030 con 0x556e3d4b7520 2024-09-21T06:57:35.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.107+0000 7f43eaa00640 1 -- 172.21.15.114:0/3473514357 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0004280 con 0x556e3d4b7520 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.108+0000 7f43f6000640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43e00b6700 0x7f43e00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.108+0000 7f43f6000640 1 -- 172.21.15.114:0/3877670688 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f43ec0e7d40 con 0x7f43f8077940 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.108+0000 7f43fda00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43e00b6700 0x7f43e00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.108+0000 7f44070bd740 1 -- 172.21.15.114:0/3473514357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 msgr2=0x556e3d85e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.108+0000 7f44070bd740 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d85e340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009ca0 tx=0x7f43f0009430 comp rx=0 tx=0).stop 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 -- 172.21.15.114:0/3473514357 shutdown_connections 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d85edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d85e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 --2- 172.21.15.114:0/3473514357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d85e340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 -- 172.21.15.114:0/3473514357 >> 172.21.15.114:0/3473514357 conn(0x556e3d4ab910 msgr2=0x556e3d85c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 -- 172.21.15.114:0/3473514357 shutdown_connections 2024-09-21T06:57:35.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.109+0000 7f44070bd740 1 -- 172.21.15.114:0/3473514357 wait complete. 2024-09-21T06:57:35.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.110+0000 7f44070bd740 1 -- start start 2024-09-21T06:57:35.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.110+0000 7f44070bd740 1 Processor -- start 2024-09-21T06:57:35.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.110+0000 7f43fda00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43e00b6700 0x7f43e00b8c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0e51d0 tx=0x7f43ec003e90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d869f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d496840 0x556e3d86a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d875c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556e3d625bb0 con 0x556e3d496840 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556e3d75f6c0 con 0x556e3d4b7520 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556e3d86c940 con 0x556e3d48e660 2024-09-21T06:57:35.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f44070bd740 10 client.0 _ll_register_callbacks cb 0x556e3d77a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:35.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f43fe400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d869f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f43fe400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d869f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34956/0 (socket says 172.21.15.114:34956) 2024-09-21T06:57:35.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.111+0000 7f43fe400640 1 -- 172.21.15.114:0/674321569 learned_addr learned my addr 172.21.15.114:0/674321569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:35.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d875c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fd000640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d496840 0x556e3d86a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:34 smithi183 ceph-mon[34219]: pgmap v499: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 191 KiB/s wr, 31 op/s 2024-09-21T06:57:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 -- 172.21.15.114:0/674321569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 msgr2=0x556e3d869f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d869f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 -- 172.21.15.114:0/674321569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d496840 msgr2=0x556e3d86a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d496840 0x556e3d86a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 -- 172.21.15.114:0/674321569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e3d522b00 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43fe400640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d875c30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003c80 tx=0x7f43f0005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51170f433d7b4dca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.112+0000 7f43df400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.113+0000 7f43df400640 1 -- 172.21.15.114:0/674321569 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0059060 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.113+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f0036030 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.113+0000 7f43df400640 1 -- 172.21.15.114:0/674321569 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f43f0031450 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.113+0000 7f43df400640 1 -- 172.21.15.114:0/674321569 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43f0060050 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.113+0000 7f44070bd740 10 client.26546 fetch_fsmap learned FSMap version 33 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.113+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556e3d599960 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f43df400640 1 -- 172.21.15.114:0/674321569 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f0036030 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f43df400640 1 -- 172.21.15.114:0/674321569 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f43f0043070 con 0x556e3d4b7520 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f44070bd740 10 client.26546 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f44070bd740 10 client.26546 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f44070bd740 20 client.26546 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f44070bd740 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d475ea0 0x556e3d87ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f44070bd740 4 client.26546 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e3d7d71e0 con 0x556e3d475ea0 2024-09-21T06:57:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.114+0000 7f43fd000640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d475ea0 0x556e3d87ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.115+0000 7f43fd000640 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d475ea0 0x556e3d87ba00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5edb0ee3842fed7 server_cookie=49e626c4a25f14b3 in_seq=0 out_seq=0 2024-09-21T06:57:35.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.115+0000 7f43df400640 10 client.26546 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f43df400640 1 -- 172.21.15.114:0/674321569 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e3d7d71e0 con 0x556e3d475ea0 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f43df400640 10 client.26546 handle_command_reply: tid=0 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 client.26546 shutdown 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 msgr2=0x556e3d875c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d875c30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43f0003c80 tx=0x7f43f0005df0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 shutdown_connections 2024-09-21T06:57:35.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d875c30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d496840 0x556e3d86a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d475ea0 0x556e3d87ba00 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 --2- 172.21.15.114:0/674321569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d869f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 >> 172.21.15.114:0/674321569 conn(0x556e3d4ab910 msgr2=0x556e3d861520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.116+0000 7f43df400640 0 client.26546 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.117+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 shutdown_connections 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.117+0000 7f44070bd740 1 -- 172.21.15.114:0/674321569 wait complete. 2024-09-21T06:57:35.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.117+0000 7f44070bd740 20 client.26546 trim_cache size 0 max 16384 2024-09-21T06:57:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.134+0000 7f44070bd740 1 Processor -- start 2024-09-21T06:57:35.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.134+0000 7f44070bd740 1 -- start start 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f44070bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d875b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f44070bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d496840 0x556e3d876070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f44070bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d4b7520 0x556e3d85bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f44070bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b71e0 con 0x556e3d4b7520 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f44070bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b8db0 con 0x556e3d496840 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f44070bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b78b0 con 0x556e3d48e660 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d875b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d875b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34970/0 (socket says 172.21.15.114:34970) 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 -- 172.21.15.114:0/2401032920 learned_addr learned my addr 172.21.15.114:0/2401032920 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d4b7520 0x556e3d85bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fd000640 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d496840 0x556e3d876070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 -- 172.21.15.114:0/2401032920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d496840 msgr2=0x556e3d876070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d496840 0x556e3d876070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 -- 172.21.15.114:0/2401032920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d4b7520 msgr2=0x556e3d85bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.135+0000 7f43fe400640 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d4b7520 0x556e3d85bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.136+0000 7f43fe400640 1 -- 172.21.15.114:0/2401032920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e3d865600 con 0x556e3d48e660 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.136+0000 7f43fe400640 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d875b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43f0002230 tx=0x7f43f0033040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935479ba50b80214 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.136+0000 7f43ea000640 1 -- 172.21.15.114:0/2401032920 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031af0 con 0x556e3d48e660 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.136+0000 7f43ea000640 1 -- 172.21.15.114:0/2401032920 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f0036070 con 0x556e3d48e660 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.136+0000 7f43ea000640 1 -- 172.21.15.114:0/2401032920 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f0031450 con 0x556e3d48e660 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 -- 172.21.15.114:0/2401032920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 msgr2=0x556e3d875b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d875b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f43f0002230 tx=0x7f43f0033040 comp rx=0 tx=0).stop 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 -- 172.21.15.114:0/2401032920 shutdown_connections 2024-09-21T06:57:35.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d4b7520 0x556e3d85bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d496840 0x556e3d876070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 --2- 172.21.15.114:0/2401032920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d48e660 0x556e3d875b30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 -- 172.21.15.114:0/2401032920 >> 172.21.15.114:0/2401032920 conn(0x556e3d4ab910 msgr2=0x556e3d8685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.137+0000 7f44070bd740 1 -- 172.21.15.114:0/2401032920 shutdown_connections 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.138+0000 7f44070bd740 1 -- 172.21.15.114:0/2401032920 wait complete. 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.138+0000 7f44070bd740 1 -- start start 2024-09-21T06:57:35.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.138+0000 7f44070bd740 1 Processor -- start 2024-09-21T06:57:35.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d871ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d872180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d86cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b8650 con 0x556e3d48e660 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b7f80 con 0x556e3d4b7520 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f43f81b51c0 con 0x556e3d496840 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f44070bd740 10 client.0 _ll_register_callbacks cb 0x7f43f0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f43fe400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d871ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.139+0000 7f43fe400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d871ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41104/0 (socket says 172.21.15.114:41104) 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fe400640 1 -- 172.21.15.114:0/1157508574 learned_addr learned my addr 172.21.15.114:0/1157508574 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d872180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fe400640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d86cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 -- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 msgr2=0x556e3d86cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d86cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 -- 172.21.15.114:0/1157508574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 msgr2=0x556e3d871ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d871ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 -- 172.21.15.114:0/1157508574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e3d87bd70 con 0x556e3d496840 2024-09-21T06:57:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43fd000640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d872180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x556e3d87bf70 tx=0x7f43e4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad7a58a1c0bb0251 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.140+0000 7f43df400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f43df400640 1 -- 172.21.15.114:0/1157508574 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4028060 con 0x556e3d496840 2024-09-21T06:57:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43f0036070 con 0x556e3d496840 2024-09-21T06:57:35.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f43df400640 1 -- 172.21.15.114:0/1157508574 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f43e401d620 con 0x556e3d496840 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f43df400640 1 -- 172.21.15.114:0/1157508574 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43e402f050 con 0x556e3d496840 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f44070bd740 10 client.26500 fetch_fsmap learned FSMap version 33 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556e3d8e68b0 con 0x556e3d496840 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.141+0000 7f43df400640 1 -- 172.21.15.114:0/1157508574 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f0036070 con 0x556e3d496840 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f43df400640 1 -- 172.21.15.114:0/1157508574 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f43e400f930 con 0x556e3d496840 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f44070bd740 10 client.26500 fetch_fsmap finished waiting for FSMap version 33 2024-09-21T06:57:35.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f44070bd740 10 client.26500 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f44070bd740 20 client.26500 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f44070bd740 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d44b250 0x556e3d8e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f44070bd740 4 client.26500 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee3cc198", "release": true, "await": true}] 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee3cc198", "release": true, "await": true}) -- 0x556e3d7d2c90 con 0x556e3d44b250 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.142+0000 7f43fe400640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d44b250 0x556e3d8e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.143+0000 7f43fe400640 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d44b250 0x556e3d8e9050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e4694b13e82bdd server_cookie=65e0642ac6206487 in_seq=0 out_seq=0 2024-09-21T06:57:35.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.143+0000 7f43df400640 10 client.26500 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:35.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.164+0000 7f43df400640 1 -- 172.21.15.114:0/1157508574 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556e3d7d2c90 con 0x556e3d44b250 2024-09-21T06:57:35.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.164+0000 7f43df400640 10 client.26500 handle_command_reply: tid=0 2024-09-21T06:57:35.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.164+0000 7f44070bd740 1 client.26500 shutdown 2024-09-21T06:57:35.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.167+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 msgr2=0x556e3d872180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.167+0000 7f44070bd740 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d872180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x556e3d87bf70 tx=0x7f43e4010800 comp rx=0 tx=0).stop 2024-09-21T06:57:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.169+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 shutdown_connections 2024-09-21T06:57:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.170+0000 7f44070bd740 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556e3d4b7520 0x556e3d86cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.170+0000 7f44070bd740 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556e3d44b250 0x556e3d8e9050 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.170+0000 7f44070bd740 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556e3d496840 0x556e3d872180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.170+0000 7f44070bd740 1 --2- 172.21.15.114:0/1157508574 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556e3d48e660 0x556e3d871ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.170+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 >> 172.21.15.114:0/1157508574 conn(0x556e3d4ab910 msgr2=0x556e3d85bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.170+0000 7f43df400640 0 client.26500 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:35.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.171+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 shutdown_connections 2024-09-21T06:57:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.171+0000 7f44070bd740 1 -- 172.21.15.114:0/1157508574 wait complete. 2024-09-21T06:57:35.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.171+0000 7f44070bd740 20 client.26500 trim_cache size 0 max 16384 2024-09-21T06:57:35.173 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:35.173 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 33, 2024-09-21T06:57:35.173 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:57:35.173 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 16, 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "ee3cc198": { 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 16, 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 147.8, 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:57:35.174 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:35.175 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43e00b6700 msgr2=0x7f43e00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43e00b6700 0x7f43e00b8c10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0e51d0 tx=0x7f43ec003e90 comp rx=0 tx=0).stop 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 msgr2=0x7f43f81a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 0x7f43f81a08b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00ea60 tx=0x7f43ec00ef20 comp rx=0 tx=0).stop 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 shutdown_connections 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43f8114570 0x7f43f81a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f43e00b6700 0x7f43e00b8c10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43f8078290 0x7f43f81a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 --2- 172.21.15.114:0/3877670688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f43f8077940 0x7f43f81a08b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.173+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 >> 172.21.15.114:0/3877670688 conn(0x7f43f8101680 msgr2=0x7f43f8116550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:35.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.174+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 shutdown_connections 2024-09-21T06:57:35.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:35.174+0000 7f43fee00640 1 -- 172.21.15.114:0/3877670688 wait complete. 2024-09-21T06:57:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.285+0000 7f66e8a00640 1 -- 172.21.15.114:0/762904238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108120 msgr2=0x7f66e4108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.285+0000 7f66e8a00640 1 --2- 172.21.15.114:0/762904238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108120 0x7f66e4108530 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f66d000d960 tx=0x7f66d002a0a0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 -- 172.21.15.114:0/762904238 shutdown_connections 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 --2- 172.21.15.114:0/762904238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108a70 0x7f66e410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 --2- 172.21.15.114:0/762904238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108120 0x7f66e4108530 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 --2- 172.21.15.114:0/762904238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 0x7f66e4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 -- 172.21.15.114:0/762904238 >> 172.21.15.114:0/762904238 conn(0x7f66e40fe230 msgr2=0x7f66e4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 -- 172.21.15.114:0/762904238 shutdown_connections 2024-09-21T06:57:36.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.286+0000 7f66e8a00640 1 -- 172.21.15.114:0/762904238 wait complete. 2024-09-21T06:57:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.288+0000 7f66e8a00640 1 Processor -- start 2024-09-21T06:57:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.288+0000 7f66e8a00640 1 -- start start 2024-09-21T06:57:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.289+0000 7f66e8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 0x7f66e419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108120 0x7f66e4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108a70 0x7f66e41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e8a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66e41115d0 con 0x7f66e4108a70 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111450 con 0x7f66e41027f0 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e8a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111750 con 0x7f66e4108120 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108120 0x7f66e4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 0x7f66e419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 0x7f66e419ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37368/0 (socket says 172.21.15.114:37368) 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e3400640 1 -- 172.21.15.114:0/3774199111 learned_addr learned my addr 172.21.15.114:0/3774199111 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.290+0000 7f66e3e00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108a70 0x7f66e41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e2a00640 1 -- 172.21.15.114:0/3774199111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 msgr2=0x7f66e419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e2a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 0x7f66e419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e2a00640 1 -- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108a70 msgr2=0x7f66e41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e2a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108a70 0x7f66e41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e2a00640 1 -- 172.21.15.114:0/3774199111 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e419d810 con 0x7f66e4108120 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e3e00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108a70 0x7f66e41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.291+0000 7f66e2a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108120 0x7f66e4197e70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f66d000d880 tx=0x7f66d0004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.292+0000 7f66dbe00640 1 -- 172.21.15.114:0/3774199111 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d003f020 con 0x7f66e4108120 2024-09-21T06:57:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.292+0000 7f66dbe00640 1 -- 172.21.15.114:0/3774199111 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d0032440 con 0x7f66e4108120 2024-09-21T06:57:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.292+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e419daa0 con 0x7f66e4108120 2024-09-21T06:57:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.292+0000 7f66dbe00640 1 -- 172.21.15.114:0/3774199111 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0037040 con 0x7f66e4108120 2024-09-21T06:57:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.292+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e419dfb0 con 0x7f66e4108120 2024-09-21T06:57:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.300+0000 7f66f0fc0740 1 Processor -- start 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.301+0000 7f66f0fc0740 1 -- start start 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.302+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.302+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.302+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.302+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5560ba32b6b0 con 0x5560ba0111b0 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.302+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5560ba31aae0 con 0x5560b9fe82f0 2024-09-21T06:57:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.302+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560ba31e8f0 con 0x5560b9ff04d0 2024-09-21T06:57:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.303+0000 7f66dbe00640 1 -- 172.21.15.114:0/3774199111 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f66d0042020 con 0x7f66e4108120 2024-09-21T06:57:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.303+0000 7f66e2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.303+0000 7f66e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.303+0000 7f66e2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35010/0 (socket says 172.21.15.114:35010) 2024-09-21T06:57:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.303+0000 7f66e2a00640 1 -- 172.21.15.114:0/1323946566 learned_addr learned my addr 172.21.15.114:0/1323946566 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:36.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66e3400640 1 -- 172.21.15.114:0/1323946566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 msgr2=0x5560ba3b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66e3400640 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66e3400640 1 -- 172.21.15.114:0/1323946566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 msgr2=0x5560ba3b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66e3400640 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66e3400640 1 -- 172.21.15.114:0/1323946566 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ba2e7780 con 0x5560b9fe82f0 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66e3400640 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3b6350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f66d400f3e0 tx=0x7f66d4009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541d5efcd81a4903 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.304+0000 7f66cfe00640 1 -- 172.21.15.114:0/1323946566 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d401e070 con 0x5560b9fe82f0 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.305+0000 7f66cfe00640 1 -- 172.21.15.114:0/1323946566 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d4009c30 con 0x5560b9fe82f0 2024-09-21T06:57:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.305+0000 7f66cfe00640 1 -- 172.21.15.114:0/1323946566 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40175d0 con 0x5560b9fe82f0 2024-09-21T06:57:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.307+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1323946566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 msgr2=0x5560ba3b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.307+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3b6350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f66d400f3e0 tx=0x7f66d4009460 comp rx=0 tx=0).stop 2024-09-21T06:57:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.308+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1323946566 shutdown_connections 2024-09-21T06:57:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.308+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.308+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3b6350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.308+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1323946566 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.308+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1323946566 >> 172.21.15.114:0/1323946566 conn(0x5560ba005300 msgr2=0x5560ba3b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:36.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.309+0000 7f66dbe00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66c80b67b0 0x7f66c80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.309+0000 7f66dbe00640 1 -- 172.21.15.114:0/3774199111 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f66d0107430 con 0x7f66e4108120 2024-09-21T06:57:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.309+0000 7f66e3400640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66c80b67b0 0x7f66c80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.309+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1323946566 shutdown_connections 2024-09-21T06:57:36.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.310+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1323946566 wait complete. 2024-09-21T06:57:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.311+0000 7f66f0fc0740 1 -- start start 2024-09-21T06:57:36.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.311+0000 7f66f0fc0740 1 Processor -- start 2024-09-21T06:57:36.314 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:57:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.313+0000 7f66e3400640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66c80b67b0 0x7f66c80b8cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f66d4000c00 tx=0x7f66d4019d60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:36.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3c1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66e3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37372/0 (socket says 172.21.15.114:37372) 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66e3400640 1 -- 172.21.15.114:0/542335705 learned_addr learned my addr 172.21.15.114:0/542335705 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5560ba252ab0 con 0x5560ba0111b0 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5560ba2c1570 con 0x5560b9fe82f0 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.314+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5560ba3d55a0 con 0x5560b9ff04d0 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e2a00640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66f0fc0740 10 client.0 _ll_register_callbacks cb 0x5560ba2d0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e3e00640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e2a00640 1 -- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 msgr2=0x5560ba3c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e2a00640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e2a00640 1 -- 172.21.15.114:0/542335705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 msgr2=0x5560ba3c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e2a00640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3c1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.315+0000 7f66e2a00640 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ba07c790 con 0x5560b9ff04d0 2024-09-21T06:57:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.317+0000 7f66e2a00640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3c1940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f66d00ca120 tx=0x7f66d00d66a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b922103bf24c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.317+0000 7f66ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.318+0000 7f66ccc00640 1 -- 172.21.15.114:0/542335705 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00c9500 con 0x5560b9ff04d0 2024-09-21T06:57:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.318+0000 7f66ccc00640 1 -- 172.21.15.114:0/542335705 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f66d00cc110 con 0x5560b9ff04d0 2024-09-21T06:57:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.319+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d4009c30 con 0x5560b9ff04d0 2024-09-21T06:57:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.320+0000 7f66ccc00640 1 -- 172.21.15.114:0/542335705 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f66d00cc620 con 0x5560b9ff04d0 2024-09-21T06:57:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.320+0000 7f66f0fc0740 10 client.26512 fetch_fsmap learned FSMap version 34 2024-09-21T06:57:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.320+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560ba0f5620 con 0x5560b9ff04d0 2024-09-21T06:57:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66ccc00640 1 -- 172.21.15.114:0/542335705 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d4009c30 con 0x5560b9ff04d0 2024-09-21T06:57:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66ccc00640 1 -- 172.21.15.114:0/542335705 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f66d003a030 con 0x5560b9ff04d0 2024-09-21T06:57:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66f0fc0740 10 client.26512 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T06:57:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66f0fc0740 10 client.26512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66f0fc0740 20 client.26512 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fcfd80 0x5560ba3e4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66f0fc0740 4 client.26512 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.322+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560ba32d600 con 0x5560b9fcfd80 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.323+0000 7f66e3400640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fcfd80 0x5560ba3e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.323+0000 7f66e3400640 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fcfd80 0x5560ba3e4650 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab076ee03e09503 server_cookie=a7a8a1a3c1781ced in_seq=0 out_seq=0 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.323+0000 7f66ccc00640 10 client.26512 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.324+0000 7f66ccc00640 1 -- 172.21.15.114:0/542335705 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560ba32d600 con 0x5560b9fcfd80 2024-09-21T06:57:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.324+0000 7f66ccc00640 10 client.26512 handle_command_reply: tid=0 2024-09-21T06:57:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.324+0000 7f66f0fc0740 1 client.26512 shutdown 2024-09-21T06:57:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.326+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 msgr2=0x5560ba3c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.326+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3c1940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f66d00ca120 tx=0x7f66d00d66a0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 shutdown_connections 2024-09-21T06:57:36.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fcfd80 0x5560ba3e4650 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9ff04d0 0x5560ba3c1940 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/542335705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9fe82f0 0x5560ba3c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 >> 172.21.15.114:0/542335705 conn(0x5560ba005300 msgr2=0x5560ba3b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.327+0000 7f66ccc00640 0 client.26512 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.328+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 shutdown_connections 2024-09-21T06:57:36.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.329+0000 7f66f0fc0740 1 -- 172.21.15.114:0/542335705 wait complete. 2024-09-21T06:57:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.330+0000 7f66f0fc0740 20 client.26512 trim_cache size 0 max 16384 2024-09-21T06:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.364+0000 7f66f0fc0740 1 Processor -- start 2024-09-21T06:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.364+0000 7f66f0fc0740 1 -- start start 2024-09-21T06:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.365+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.365+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.365+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3beb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.365+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66c400fe70 con 0x5560ba0111b0 2024-09-21T06:57:36.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.365+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66c40119e0 con 0x5560b9ff04d0 2024-09-21T06:57:36.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.365+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66c4010540 con 0x5560b9fe82f0 2024-09-21T06:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.367+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3beb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.367+0000 7f66e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3beb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41142/0 (socket says 172.21.15.114:41142) 2024-09-21T06:57:36.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.367+0000 7f66e3e00640 1 -- 172.21.15.114:0/1295329611 learned_addr learned my addr 172.21.15.114:0/1295329611 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.368+0000 7f66e2a00640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.368+0000 7f66e3400640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.368+0000 7f66e2a00640 1 -- 172.21.15.114:0/1295329611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 msgr2=0x5560ba3b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e2a00640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e2a00640 1 -- 172.21.15.114:0/1295329611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 msgr2=0x5560ba3beb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e2a00640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3beb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e2a00640 1 -- 172.21.15.114:0/1295329611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ba3bf920 con 0x5560b9ff04d0 2024-09-21T06:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e3e00640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e3400640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.369+0000 7f66e2a00640 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3be610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f66d00d19e0 tx=0x7f66d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6840f27b614786 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.370+0000 7f66cf400640 1 -- 172.21.15.114:0/1295329611 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00cc800 con 0x5560b9ff04d0 2024-09-21T06:57:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.370+0000 7f66cf400640 1 -- 172.21.15.114:0/1295329611 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d0005d20 con 0x5560b9ff04d0 2024-09-21T06:57:36.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.370+0000 7f66cf400640 1 -- 172.21.15.114:0/1295329611 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00cc800 con 0x5560b9ff04d0 2024-09-21T06:57:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1295329611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 msgr2=0x5560ba3be610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3be610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f66d00d19e0 tx=0x7f66d0005f90 comp rx=0 tx=0).stop 2024-09-21T06:57:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1295329611 shutdown_connections 2024-09-21T06:57:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3beb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3be610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/1295329611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1295329611 >> 172.21.15.114:0/1295329611 conn(0x5560ba005300 msgr2=0x5560ba3b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1295329611 shutdown_connections 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.371+0000 7f66f0fc0740 1 -- 172.21.15.114:0/1295329611 wait complete. 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.372+0000 7f66f0fc0740 1 -- start start 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 Processor -- start 2024-09-21T06:57:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f66c40112e0 con 0x5560ba0111b0 2024-09-21T06:57:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66c4010c10 con 0x5560b9ff04d0 2024-09-21T06:57:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f66c400de50 con 0x5560b9fe82f0 2024-09-21T06:57:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.373+0000 7f66f0fc0740 10 client.0 _ll_register_callbacks cb 0x7f66d00c9350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:36.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee3cc198', total seconds elapsed: 32.6 2024-09-21T06:57:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.374+0000 7f66e2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.374+0000 7f66e2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3d8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37390/0 (socket says 172.21.15.114:37390) 2024-09-21T06:57:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.374+0000 7f66e2a00640 1 -- 172.21.15.114:0/527320774 learned_addr learned my addr 172.21.15.114:0/527320774 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.374+0000 7f66e3400640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.374+0000 7f66e3e00640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.375+0000 7f66e3400640 1 -- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 msgr2=0x5560ba3d8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.375+0000 7f66e3400640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3d8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.375+0000 7f66e3400640 1 -- 172.21.15.114:0/527320774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 msgr2=0x5560ba3d89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.375+0000 7f66e3400640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3d89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.375+0000 7f66e3400640 1 -- 172.21.15.114:0/527320774 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ba3d90b0 con 0x5560b9fe82f0 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.376+0000 7f66e3400640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3dafc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5560ba3e4dc0 tx=0x7f66d401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68cb395e491b62e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.376+0000 7f66ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.377+0000 7f66ccc00640 1 -- 172.21.15.114:0/527320774 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d401e040 con 0x5560b9fe82f0 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.377+0000 7f66ccc00640 1 -- 172.21.15.114:0/527320774 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f66d4005180 con 0x5560b9fe82f0 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.377+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5560ba3290b0 con 0x5560b9fe82f0 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.378+0000 7f66ccc00640 1 -- 172.21.15.114:0/527320774 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f66d4005680 con 0x5560b9fe82f0 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.378+0000 7f66f0fc0740 10 client.26518 fetch_fsmap learned FSMap version 34 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.378+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560ba44ef10 con 0x5560b9fe82f0 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.378+0000 7f66ccc00640 1 -- 172.21.15.114:0/527320774 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5560ba3290b0 con 0x5560b9fe82f0 2024-09-21T06:57:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.378+0000 7f66ccc00640 1 -- 172.21.15.114:0/527320774 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f66d4024540 con 0x5560b9fe82f0 2024-09-21T06:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.379+0000 7f66f0fc0740 10 client.26518 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T06:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.379+0000 7f66f0fc0740 10 client.26518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.379+0000 7f66f0fc0740 20 client.26518 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.379+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fa51c0 0x5560ba451460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.379+0000 7f66f0fc0740 4 client.26518 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:57:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.379+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560ba3264e0 con 0x5560b9fa51c0 2024-09-21T06:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.380+0000 7f66e2a00640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fa51c0 0x5560ba451460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.381+0000 7f66e2a00640 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fa51c0 0x5560ba451460 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c69970456cd160cb server_cookie=5d1bfd953cdc33d2 in_seq=0 out_seq=0 2024-09-21T06:57:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.381+0000 7f66ccc00640 10 client.26518 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66ccc00640 1 -- 172.21.15.114:0/527320774 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560ba3264e0 con 0x5560b9fa51c0 2024-09-21T06:57:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66ccc00640 10 client.26518 handle_command_reply: tid=0 2024-09-21T06:57:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 client.26518 shutdown 2024-09-21T06:57:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 msgr2=0x5560ba3dafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3dafc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5560ba3e4dc0 tx=0x7f66d401f000 comp rx=0 tx=0).stop 2024-09-21T06:57:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 shutdown_connections 2024-09-21T06:57:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5560ba0111b0 0x5560ba3d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5560b9ff04d0 0x5560ba3d8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5560b9fa51c0 0x5560ba451460 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 --2- 172.21.15.114:0/527320774 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5560b9fe82f0 0x5560ba3dafc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.399+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 >> 172.21.15.114:0/527320774 conn(0x5560ba005300 msgr2=0x5560ba3b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.400+0000 7f66ccc00640 0 client.26518 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.400+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 shutdown_connections 2024-09-21T06:57:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.400+0000 7f66f0fc0740 1 -- 172.21.15.114:0/527320774 wait complete. 2024-09-21T06:57:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.401+0000 7f66f0fc0740 20 client.26518 trim_cache size 0 max 16384 2024-09-21T06:57:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:57:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:57:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:57:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:57:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66c80b67b0 msgr2=0x7f66c80b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66c80b67b0 0x7f66c80b8cc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f66d4000c00 tx=0x7f66d4019d60 comp rx=0 tx=0).stop 2024-09-21T06:57:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108120 msgr2=0x7f66e4197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108120 0x7f66e4197e70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f66d000d880 tx=0x7f66d0004530 comp rx=0 tx=0).stop 2024-09-21T06:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 shutdown_connections 2024-09-21T06:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f66e4108a70 0x7f66e41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66e4108120 0x7f66e4197e70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f66c80b67b0 0x7f66c80b8cc0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f66d4000c00 tx=0x7f66d4019d60 comp rx=0 tx=0).stop 2024-09-21T06:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 --2- 172.21.15.114:0/3774199111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66e41027f0 0x7f66e419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.404+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 >> 172.21.15.114:0/3774199111 conn(0x7f66e40fe230 msgr2=0x7f66e4100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.408+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 shutdown_connections 2024-09-21T06:57:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:36.408+0000 7f66e8a00640 1 -- 172.21.15.114:0/3774199111 wait complete. 2024-09-21T06:57:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:37.069 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:36 smithi114 ceph-mon[30654]: pgmap v500: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-21T06:57:37.069 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:36 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:36 smithi183 ceph-mon[34219]: pgmap v500: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-21T06:57:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:36 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:36 smithi193 ceph-mon[34601]: pgmap v500: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-09-21T06:57:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:36 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 180.0 s, 638.1 tps, lat 901.781 ms stddev 4574.665 2024-09-21T06:57:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 -- 172.21.15.114:0/4036550217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 msgr2=0x7fb7f81061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4036550217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 0x7fb7f81061c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009ca0 tx=0x7fb7ec02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 -- 172.21.15.114:0/4036550217 shutdown_connections 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4036550217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 0x7fb7f8112870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4036550217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 0x7fb7f810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4036550217 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 0x7fb7f81061c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.881+0000 7fb7ff800640 1 -- 172.21.15.114:0/4036550217 >> 172.21.15.114:0/4036550217 conn(0x7fb7f81017c0 msgr2=0x7fb7f8103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.882+0000 7fb7ff800640 1 -- 172.21.15.114:0/4036550217 shutdown_connections 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.882+0000 7fb7ff800640 1 -- 172.21.15.114:0/4036550217 wait complete. 2024-09-21T06:57:37.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.883+0000 7fb7ff800640 1 Processor -- start 2024-09-21T06:57:37.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 -- start start 2024-09-21T06:57:37.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 0x7fb7f819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 0x7fb7f819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 0x7fb7f81a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8115d50 con 0x7fb7f8105db0 2024-09-21T06:57:37.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8115bd0 con 0x7fb7f810b6f0 2024-09-21T06:57:37.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.884+0000 7fb7ff800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8115ed0 con 0x7fb7f810c040 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7fd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 0x7fb7f81a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 0x7fb7f819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7fd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 0x7fb7f81a1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35054/0 (socket says 172.21.15.114:35054) 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7fd800640 1 -- 172.21.15.114:0/4259488794 learned_addr learned my addr 172.21.15.114:0/4259488794 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7fce00640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 0x7fb7f819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 -- 172.21.15.114:0/4259488794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 msgr2=0x7fb7f81a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 0x7fb7f81a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 -- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 msgr2=0x7fb7f819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 0x7fb7f819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 -- 172.21.15.114:0/4259488794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f81a2690 con 0x7fb7f810b6f0 2024-09-21T06:57:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.885+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 0x7fb7f819ccc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb7f806bb30 tx=0x7fb7e400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.886+0000 7fb7f5600640 1 -- 172.21.15.114:0/4259488794 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400cd00 con 0x7fb7f810b6f0 2024-09-21T06:57:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.886+0000 7fb7f5600640 1 -- 172.21.15.114:0/4259488794 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e40035f0 con 0x7fb7f810b6f0 2024-09-21T06:57:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.886+0000 7fb7f5600640 1 -- 172.21.15.114:0/4259488794 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4010690 con 0x7fb7f810b6f0 2024-09-21T06:57:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.886+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f81a2980 con 0x7fb7f810b6f0 2024-09-21T06:57:37.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.886+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a90e0 con 0x7fb7f810b6f0 2024-09-21T06:57:37.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb805ffd740 1 Processor -- start 2024-09-21T06:57:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb7f5600640 1 -- 172.21.15.114:0/4259488794 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb7e40037a0 con 0x7fb7f810b6f0 2024-09-21T06:57:37.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb805ffd740 1 -- start start 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ee81b0 0x5568362b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 0x5568362b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.893+0000 7fb805ffd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55683622ef70 con 0x556835f11070 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb805ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55683621e3a0 con 0x556835ee81b0 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb805ffd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5568362221b0 con 0x556835ef0390 2024-09-21T06:57:37.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 0x5568362b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 0x5568362b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35070/0 (socket says 172.21.15.114:35070) 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 -- 172.21.15.114:0/3504153870 learned_addr learned my addr 172.21.15.114:0/3504153870 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7fd800640 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7fce00640 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ee81b0 0x5568362b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 -- 172.21.15.114:0/3504153870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ee81b0 msgr2=0x5568362b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ee81b0 0x5568362b82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 -- 172.21.15.114:0/3504153870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 msgr2=0x5568362b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.894+0000 7fb7f7e00640 1 -- 172.21.15.114:0/3504153870 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568361e8c90 con 0x556835ef0390 2024-09-21T06:57:37.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.895+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 0x5568362b87e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40ebce0 tx=0x7fb7e40ebdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634d62eece00528b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:37.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.895+0000 7fb7ea000640 1 -- 172.21.15.114:0/3504153870 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40ee470 con 0x556835ef0390 2024-09-21T06:57:37.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.895+0000 7fb7ea000640 1 -- 172.21.15.114:0/3504153870 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e40eeac0 con 0x556835ef0390 2024-09-21T06:57:37.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.895+0000 7fb7ea000640 1 -- 172.21.15.114:0/3504153870 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40f88b0 con 0x556835ef0390 2024-09-21T06:57:37.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.896+0000 7fb805ffd740 1 -- 172.21.15.114:0/3504153870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 msgr2=0x5568362b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.896+0000 7fb805ffd740 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 0x5568362b87e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40ebce0 tx=0x7fb7e40ebdd0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.897+0000 7fb805ffd740 1 -- 172.21.15.114:0/3504153870 shutdown_connections 2024-09-21T06:57:37.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.897+0000 7fb805ffd740 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ef0390 0x5568362b87e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.897+0000 7fb805ffd740 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ee81b0 0x5568362b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.897+0000 7fb805ffd740 1 --2- 172.21.15.114:0/3504153870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.897+0000 7fb805ffd740 1 -- 172.21.15.114:0/3504153870 >> 172.21.15.114:0/3504153870 conn(0x556835f05460 msgr2=0x5568362b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:37.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.897+0000 7fb805ffd740 1 -- 172.21.15.114:0/3504153870 shutdown_connections 2024-09-21T06:57:37.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.898+0000 7fb805ffd740 1 -- 172.21.15.114:0/3504153870 wait complete. 2024-09-21T06:57:37.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.898+0000 7fb805ffd740 1 -- start start 2024-09-21T06:57:37.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.900+0000 7fb805ffd740 1 Processor -- start 2024-09-21T06:57:37.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 0x5568362c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55683607f700 con 0x556835ef0390 2024-09-21T06:57:37.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5568361b9540 con 0x556835f11070 2024-09-21T06:57:37.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5568362be4b0 con 0x556835ee81b0 2024-09-21T06:57:37.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.901+0000 7fb805ffd740 10 client.0 _ll_register_callbacks cb 0x5568361c9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:37.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.902+0000 7fb7fce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.902+0000 7fb7fce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41196/0 (socket says 172.21.15.114:41196) 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.902+0000 7fb7fce00640 1 -- 172.21.15.114:0/4117756955 learned_addr learned my addr 172.21.15.114:0/4117756955 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.902+0000 7fb7f5600640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6640 0x7fb7e00b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.902+0000 7fb7f5600640 1 -- 172.21.15.114:0/4259488794 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb7e40e7d00 con 0x7fb7f810b6f0 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fce00640 1 -- 172.21.15.114:0/4117756955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 msgr2=0x5568362c3d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fce00640 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fce00640 1 -- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 msgr2=0x5568362c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fce00640 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 0x5568362c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fce00640 1 -- 172.21.15.114:0/4117756955 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556835f7c650 con 0x556835ef0390 2024-09-21T06:57:37.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fd800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6640 0x7fb7e00b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.903+0000 7fb7fce00640 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c42f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f80 tx=0x7fb7ec030f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b268f51b4977d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb7dea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb7dea00640 1 -- 172.21.15.114:0/4117756955 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec030c10 con 0x556835ef0390 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568361bbb50 con 0x556835ef0390 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb7dea00640 1 -- 172.21.15.114:0/4117756955 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb7ec003970 con 0x556835ef0390 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb7dea00640 1 -- 172.21.15.114:0/4117756955 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec04e050 con 0x556835ef0390 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb805ffd740 10 client.16431 fetch_fsmap learned FSMap version 34 2024-09-21T06:57:37.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.904+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5568361d6910 con 0x556835ef0390 2024-09-21T06:57:37.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.905+0000 7fb7dea00640 1 -- 172.21.15.114:0/4117756955 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5568361bbb50 con 0x556835ef0390 2024-09-21T06:57:37.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb7dea00640 1 -- 172.21.15.114:0/4117756955 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fb7ec045480 con 0x556835ef0390 2024-09-21T06:57:37.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb805ffd740 10 client.16431 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T06:57:37.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb805ffd740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb805ffd740 20 client.16431 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ecf9f0 0x5568362e6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb805ffd740 4 client.16431 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556836230ec0 con 0x556835ecf9f0 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb7fd800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6640 0x7fb7e00b8b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5568362ca3c0 tx=0x7fb7f0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.906+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ecf9f0 0x5568362e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.907+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ecf9f0 0x5568362e6710 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506253378d02cd98 server_cookie=84189c5747200930 in_seq=0 out_seq=0 2024-09-21T06:57:37.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.907+0000 7fb7dea00640 10 client.16431 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:37.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.908+0000 7fb7dea00640 1 -- 172.21.15.114:0/4117756955 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556836230ec0 con 0x556835ecf9f0 2024-09-21T06:57:37.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.908+0000 7fb7dea00640 10 client.16431 handle_command_reply: tid=0 2024-09-21T06:57:37.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.908+0000 7fb805ffd740 1 client.16431 shutdown 2024-09-21T06:57:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 msgr2=0x5568362c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c42f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f80 tx=0x7fb7ec030f00 comp rx=0 tx=0).stop 2024-09-21T06:57:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 shutdown_connections 2024-09-21T06:57:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 0x5568362c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c42f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ecf9f0 0x5568362e6710 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.909+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4117756955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.910+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 >> 172.21.15.114:0/4117756955 conn(0x556835f05460 msgr2=0x5568362baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:37.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.910+0000 7fb7dea00640 0 client.16431 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:37.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.912+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 shutdown_connections 2024-09-21T06:57:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.912+0000 7fb805ffd740 1 -- 172.21.15.114:0/4117756955 wait complete. 2024-09-21T06:57:37.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.912+0000 7fb805ffd740 20 client.16431 trim_cache size 0 max 16384 2024-09-21T06:57:37.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.946+0000 7fb805ffd740 1 Processor -- start 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.946+0000 7fb805ffd740 1 -- start start 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.946+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.946+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 0x5568362c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb805ffd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81aec20 con 0x556835ef0390 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb805ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b0790 con 0x556835f11070 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb805ffd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81af2f0 con 0x556835ee81b0 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35096/0 (socket says 172.21.15.114:35096) 2024-09-21T06:57:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 -- 172.21.15.114:0/841717813 learned_addr learned my addr 172.21.15.114:0/841717813 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 -- 172.21.15.114:0/841717813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 msgr2=0x5568362c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 0x5568362c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 -- 172.21.15.114:0/841717813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 msgr2=0x5568362c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.947+0000 7fb7fce00640 1 -- 172.21.15.114:0/841717813 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568362b8b90 con 0x556835ee81b0 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.948+0000 7fb7fce00640 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec006010 tx=0x7fb7ec0456c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a8f268c4e1d2223 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.948+0000 7fb7e9600640 1 -- 172.21.15.114:0/841717813 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec045cc0 con 0x556835ee81b0 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.948+0000 7fb7e9600640 1 -- 172.21.15.114:0/841717813 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec036050 con 0x556835ee81b0 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.949+0000 7fb7e9600640 1 -- 172.21.15.114:0/841717813 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec004290 con 0x556835ee81b0 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 -- 172.21.15.114:0/841717813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 msgr2=0x5568362c3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec006010 tx=0x7fb7ec0456c0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 -- 172.21.15.114:0/841717813 shutdown_connections 2024-09-21T06:57:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835f11070 0x5568362c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835ef0390 0x5568362c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 --2- 172.21.15.114:0/841717813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x5568362c3d30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 -- 172.21.15.114:0/841717813 >> 172.21.15.114:0/841717813 conn(0x556835f05460 msgr2=0x5568362c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.950+0000 7fb805ffd740 1 -- 172.21.15.114:0/841717813 shutdown_connections 2024-09-21T06:57:37.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.951+0000 7fb805ffd740 1 -- 172.21.15.114:0/841717813 wait complete. 2024-09-21T06:57:37.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.951+0000 7fb805ffd740 1 -- start start 2024-09-21T06:57:37.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.952+0000 7fb805ffd740 1 Processor -- start 2024-09-21T06:57:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.954+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x55683634c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.954+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ef0390 0x5568362f7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.954+0000 7fb805ffd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.954+0000 7fb7fce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x55683634c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb7fce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x55683634c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35106/0 (socket says 172.21.15.114:35106) 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb805ffd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b0090 con 0x556835f11070 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb805ffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81af9c0 con 0x556835ef0390 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb805ffd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81acc20 con 0x556835ee81b0 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb7fce00640 1 -- 172.21.15.114:0/4198510708 learned_addr learned my addr 172.21.15.114:0/4198510708 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb805ffd740 10 client.0 _ll_register_callbacks cb 0x7fb7ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.955+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ef0390 0x5568362f7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.956+0000 7fb7f7e00640 1 -- 172.21.15.114:0/4198510708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 msgr2=0x55683634c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.956+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x55683634c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.956+0000 7fb7f7e00640 1 -- 172.21.15.114:0/4198510708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 msgr2=0x5568362f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.956+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.956+0000 7fb7f7e00640 1 -- 172.21.15.114:0/4198510708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568362f8a50 con 0x556835ef0390 2024-09-21T06:57:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.957+0000 7fb7dea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.958+0000 7fb7f7e00640 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ef0390 0x5568362f7e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40a9d50 tx=0x7fb7e4003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07cfea1fa71430d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:37.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.958+0000 7fb7dea00640 1 -- 172.21.15.114:0/4198510708 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40a9dd0 con 0x556835ef0390 2024-09-21T06:57:37.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.959+0000 7fb7dea00640 1 -- 172.21.15.114:0/4198510708 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb7e40b6d10 con 0x556835ef0390 2024-09-21T06:57:37.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.959+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec036050 con 0x556835ef0390 2024-09-21T06:57:37.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.960+0000 7fb7dea00640 1 -- 172.21.15.114:0/4198510708 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7e40acef0 con 0x556835ef0390 2024-09-21T06:57:37.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.960+0000 7fb805ffd740 10 client.26579 fetch_fsmap learned FSMap version 34 2024-09-21T06:57:37.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.960+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556836362070 con 0x556835ef0390 2024-09-21T06:57:37.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb7dea00640 1 -- 172.21.15.114:0/4198510708 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e40d3020 con 0x556835ef0390 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb7dea00640 1 -- 172.21.15.114:0/4198510708 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fb7e40f7070 con 0x556835ef0390 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb805ffd740 10 client.26579 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb805ffd740 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb805ffd740 20 client.26579 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ea4da0 0x5568363642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb805ffd740 4 client.26579 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.961+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55683622c970 con 0x556835ea4da0 2024-09-21T06:57:37.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.962+0000 7fb7fce00640 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ea4da0 0x5568363642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:37.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.964+0000 7fb7fce00640 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ea4da0 0x5568363642a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30bbb84625904ee server_cookie=9a6ad986d0355ff6 in_seq=0 out_seq=0 2024-09-21T06:57:37.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.964+0000 7fb7dea00640 10 client.26579 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:37.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.964+0000 7fb7dea00640 1 -- 172.21.15.114:0/4198510708 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x55683622c970 con 0x556835ea4da0 2024-09-21T06:57:37.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.964+0000 7fb7dea00640 10 client.26579 handle_command_reply: tid=0 2024-09-21T06:57:37.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.964+0000 7fb805ffd740 1 client.26579 shutdown 2024-09-21T06:57:37.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.965+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ef0390 msgr2=0x5568362f7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.965+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ef0390 0x5568362f7e20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40a9d50 tx=0x7fb7e4003e00 comp rx=0 tx=0).stop 2024-09-21T06:57:37.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 shutdown_connections 2024-09-21T06:57:37.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556835f11070 0x5568362f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556835ef0390 0x5568362f7e20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556835ea4da0 0x5568363642a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 --2- 172.21.15.114:0/4198510708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556835ee81b0 0x55683634c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 >> 172.21.15.114:0/4198510708 conn(0x556835f05460 msgr2=0x5568362c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:37.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb7dea00640 0 client.26579 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:37.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 shutdown_connections 2024-09-21T06:57:37.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.966+0000 7fb805ffd740 1 -- 172.21.15.114:0/4198510708 wait complete. 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:37.970 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:37.971 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:57:37.972 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:37.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.967+0000 7fb805ffd740 20 client.26579 trim_cache size 0 max 16384 2024-09-21T06:57:37.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.968+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6640 msgr2=0x7fb7e00b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.968+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6640 0x7fb7e00b8b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5568362ca3c0 tx=0x7fb7f0005f90 comp rx=0 tx=0).stop 2024-09-21T06:57:37.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 msgr2=0x7fb7f819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:37.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 0x7fb7f819ccc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb7f806bb30 tx=0x7fb7e400ee50 comp rx=0 tx=0).stop 2024-09-21T06:57:37.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 shutdown_connections 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f810c040 0x7fb7f81a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6640 0x7fb7e00b8b50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810b6f0 0x7fb7f819ccc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 --2- 172.21.15.114:0/4259488794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f8105db0 0x7fb7f819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 >> 172.21.15.114:0/4259488794 conn(0x7fb7f81017c0 msgr2=0x7fb7f8104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.969+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 shutdown_connections 2024-09-21T06:57:37.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:37.970+0000 7fb7ff800640 1 -- 172.21.15.114:0/4259488794 wait complete. 2024-09-21T06:57:39.041 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:38 smithi114 ceph-mon[30654]: pgmap v501: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 409 KiB/s wr, 66 op/s 2024-09-21T06:57:39.109 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:57:39.109 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:57:39.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:38 smithi183 ceph-mon[34219]: pgmap v501: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 409 KiB/s wr, 66 op/s 2024-09-21T06:57:39.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:38 smithi193 ceph-mon[34601]: pgmap v501: 129 pgs: 129 active+clean; 1.7 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 409 KiB/s wr, 66 op/s 2024-09-21T06:57:39.786 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:40.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 -- 172.21.15.114:0/2087450150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd411cc30 msgr2=0x7facd411f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:40.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 --2- 172.21.15.114:0/2087450150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd411cc30 0x7facd411f140 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faccc009ca0 tx=0x7faccc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:57:40.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 -- 172.21.15.114:0/2087450150 shutdown_connections 2024-09-21T06:57:40.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 --2- 172.21.15.114:0/2087450150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd411cc30 0x7facd411f140 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:40.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 --2- 172.21.15.114:0/2087450150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd410a9e0 0x7facd410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:40.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 --2- 172.21.15.114:0/2087450150 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd4111f40 0x7facd410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:40.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.629+0000 7facd9800640 1 -- 172.21.15.114:0/2087450150 >> 172.21.15.114:0/2087450150 conn(0x7facd4070ec0 msgr2=0x7facd40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:40.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.630+0000 7facd9800640 1 -- 172.21.15.114:0/2087450150 shutdown_connections 2024-09-21T06:57:40.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.630+0000 7facd9800640 1 -- 172.21.15.114:0/2087450150 wait complete. 2024-09-21T06:57:40.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.633+0000 7facd9800640 1 Processor -- start 2024-09-21T06:57:40.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 -- start start 2024-09-21T06:57:40.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 0x7facd41a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:40.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd4111f40 0x7facd41a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:40.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd411cc30 0x7facd41aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:40.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7facd4122960 con 0x7facd410a9e0 2024-09-21T06:57:40.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7facd41227e0 con 0x7facd4111f40 2024-09-21T06:57:40.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.634+0000 7facd9800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7facd4122ae0 con 0x7facd411cc30 2024-09-21T06:57:40.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.635+0000 7facd2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 0x7facd41a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:40.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.635+0000 7facd2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 0x7facd41a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41240/0 (socket says 172.21.15.114:41240) 2024-09-21T06:57:40.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.635+0000 7facd2a00640 1 -- 172.21.15.114:0/1863120706 learned_addr learned my addr 172.21.15.114:0/1863120706 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:40.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.635+0000 7facd2000640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd4111f40 0x7facd41a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:40.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.636+0000 7facd2000640 1 -- 172.21.15.114:0/1863120706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd411cc30 msgr2=0x7facd41aeb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:57:40.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.636+0000 7facd2000640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd411cc30 0x7facd41aeb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:40.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.636+0000 7facd2000640 1 -- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 msgr2=0x7facd41a9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:40.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.636+0000 7facd2000640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 0x7facd41a9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:40.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.639+0000 7facd2000640 1 -- 172.21.15.114:0/1863120706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41af2a0 con 0x7facd4111f40 2024-09-21T06:57:40.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.639+0000 7facd2a00640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 0x7facd41a9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:40.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.640+0000 7facd2000640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd4111f40 0x7facd41a98d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facd406c6e0 tx=0x7facc400ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:40.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.640+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc400cc90 con 0x7facd4111f40 2024-09-21T06:57:40.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.641+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc400ce30 con 0x7facd4111f40 2024-09-21T06:57:40.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.641+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41af530 con 0x7facd4111f40 2024-09-21T06:57:40.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.641+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd40755f0 con 0x7facd4111f40 2024-09-21T06:57:40.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.641+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc403a860 con 0x7facd4111f40 2024-09-21T06:57:40.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.643+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb40052c0 con 0x7facd4111f40 2024-09-21T06:57:40.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.646+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7facc4041020 con 0x7facd4111f40 2024-09-21T06:57:40.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.647+0000 7faccb400640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7facb80b6850 0x7facb80b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:40.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.648+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7facc4109d80 con 0x7facd4111f40 2024-09-21T06:57:40.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.649+0000 7facd2a00640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7facb80b6850 0x7facb80b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:40.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.650+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7facc40cff80 con 0x7facd4111f40 2024-09-21T06:57:40.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:40.652+0000 7facd2a00640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7facb80b6850 0x7facb80b8d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7facc0009e10 tx=0x7facc00091c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:40.963 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:40 smithi193 ceph-mon[34601]: pgmap v502: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-21T06:57:41.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:40 smithi114 ceph-mon[30654]: pgmap v502: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-21T06:57:41.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.090+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facb40055b0 con 0x7facd4111f40 2024-09-21T06:57:41.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.091+0000 7faccb400640 1 -- 172.21.15.114:0/1863120706 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11989 (secure 0 0 0) 0x7facc40d4360 con 0x7facd4111f40 2024-09-21T06:57:41.093 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:57:41.093 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":35,"btime":"2024-09-21T06:57:40:680215+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:57:40.612768+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:57:41.094 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 35 2024-09-21T06:57:41.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.094+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7facb80b6850 msgr2=0x7facb80b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:41.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.094+0000 7facd9800640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7facb80b6850 0x7facb80b8d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7facc0009e10 tx=0x7facc00091c0 comp rx=0 tx=0).stop 2024-09-21T06:57:41.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd4111f40 msgr2=0x7facd41a98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:41.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd4111f40 0x7facd41a98d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7facd406c6e0 tx=0x7facc400ed90 comp rx=0 tx=0).stop 2024-09-21T06:57:41.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 shutdown_connections 2024-09-21T06:57:41.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7facd411cc30 0x7facd41aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:41.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7facb80b6850 0x7facb80b8d60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:41.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd4111f40 0x7facd41a98d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:41.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 --2- 172.21.15.114:0/1863120706 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7facd410a9e0 0x7facd41a9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:41.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 >> 172.21.15.114:0/1863120706 conn(0x7facd4070ec0 msgr2=0x7facd411e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:41.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 shutdown_connections 2024-09-21T06:57:41.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:41.095+0000 7facd9800640 1 -- 172.21.15.114:0/1863120706 wait complete. 2024-09-21T06:57:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:40 smithi183 ceph-mon[34219]: pgmap v502: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-21T06:57:41.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:41 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:41.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1863120706' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:42.015 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:41 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:42.015 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1863120706' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:42.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:41 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:57:42.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1863120706' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:57:42.417 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:57:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 185.0 s, 1552.2 tps, lat 64.487 ms stddev 81.475 2024-09-21T06:57:43.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:42 smithi114 ceph-mon[30654]: pgmap v503: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-21T06:57:43.085 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:42 smithi183 ceph-mon[34219]: pgmap v503: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-21T06:57:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:42 smithi193 ceph-mon[34601]: pgmap v503: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-21T06:57:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 -- 172.21.15.114:0/2355945113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 msgr2=0x7f51e4110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 --2- 172.21.15.114:0/2355945113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e4110110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009ca0 tx=0x7f51dc02a8d0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 -- 172.21.15.114:0/2355945113 shutdown_connections 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 --2- 172.21.15.114:0/2355945113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e4110110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 --2- 172.21.15.114:0/2355945113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4105110 0x7f51e410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 --2- 172.21.15.114:0/2355945113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e41047c0 0x7f51e4104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.316+0000 7f51eaa00640 1 -- 172.21.15.114:0/2355945113 >> 172.21.15.114:0/2355945113 conn(0x7f51e40fe2c0 msgr2=0x7f51e4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.317+0000 7f51eaa00640 1 -- 172.21.15.114:0/2355945113 shutdown_connections 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.317+0000 7f51eaa00640 1 -- 172.21.15.114:0/2355945113 wait complete. 2024-09-21T06:57:44.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.318+0000 7f51eaa00640 1 Processor -- start 2024-09-21T06:57:44.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 -- start start 2024-09-21T06:57:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e41047c0 0x7f51e410abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4105110 0x7f51e410b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e410b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4103fb0 con 0x7f51e41047c0 2024-09-21T06:57:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4103e30 con 0x7f51e4105110 2024-09-21T06:57:44.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.319+0000 7f51eaa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4104130 con 0x7f51e410dc20 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e410b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e410b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35142/0 (socket says 172.21.15.114:35142) 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 -- 172.21.15.114:0/4065061373 learned_addr learned my addr 172.21.15.114:0/4065061373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51e9600640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e41047c0 0x7f51e410abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 -- 172.21.15.114:0/4065061373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4105110 msgr2=0x7f51e410b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4105110 0x7f51e410b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 -- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e41047c0 msgr2=0x7f51e410abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e41047c0 0x7f51e410abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.320+0000 7f51ea000640 1 -- 172.21.15.114:0/4065061373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e4116250 con 0x7f51e410dc20 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.321+0000 7f51ea000640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e410b650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51dc031aa0 tx=0x7f51dc003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.321+0000 7f51e2000640 1 -- 172.21.15.114:0/4065061373 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc003c90 con 0x7f51e410dc20 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.322+0000 7f51e2000640 1 -- 172.21.15.114:0/4065061373 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51dc003e30 con 0x7f51e410dc20 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.322+0000 7f51e2000640 1 -- 172.21.15.114:0/4065061373 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc02fc50 con 0x7f51e410dc20 2024-09-21T06:57:44.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.322+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e41164e0 con 0x7f51e410dc20 2024-09-21T06:57:44.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.322+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e4116a50 con 0x7f51e410dc20 2024-09-21T06:57:44.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.327+0000 7f51f2df2740 1 Processor -- start 2024-09-21T06:57:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.329+0000 7f51e2000640 1 -- 172.21.15.114:0/4065061373 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f51dc004040 con 0x7f51e410dc20 2024-09-21T06:57:44.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.329+0000 7f51f2df2740 1 -- start start 2024-09-21T06:57:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.331+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6aedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.331+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e671e1b0 0x5641e6aee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.331+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e6726390 0x5641e6aee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.331+0000 7f51f2df2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5641e6a64fb0 con 0x5641e6747070 2024-09-21T06:57:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.331+0000 7f51f2df2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641e6a543e0 con 0x5641e671e1b0 2024-09-21T06:57:44.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.331+0000 7f51f2df2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641e6a581f0 con 0x5641e6726390 2024-09-21T06:57:44.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.332+0000 7f51e9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e671e1b0 0x5641e6aee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.332+0000 7f51e8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6aedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6aedd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41264/0 (socket says 172.21.15.114:41264) 2024-09-21T06:57:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e8c00640 1 -- 172.21.15.114:0/4220852743 learned_addr learned my addr 172.21.15.114:0/4220852743 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:44.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e8c00640 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e6726390 0x5641e6aee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e9600640 1 -- 172.21.15.114:0/4220852743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e6726390 msgr2=0x5641e6aee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e9600640 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e6726390 0x5641e6aee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e9600640 1 -- 172.21.15.114:0/4220852743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 msgr2=0x5641e6aedd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e9600640 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6aedd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.334+0000 7f51e9600640 1 -- 172.21.15.114:0/4220852743 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e6a1ecd0 con 0x5641e671e1b0 2024-09-21T06:57:44.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.335+0000 7f51e9600640 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e671e1b0 0x5641e6aee270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f51d800cb50 tx=0x7f51d800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed57cb6a087bd03b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:44.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.336+0000 7f51d6600640 1 -- 172.21.15.114:0/4220852743 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004220 con 0x5641e671e1b0 2024-09-21T06:57:44.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.336+0000 7f51d6600640 1 -- 172.21.15.114:0/4220852743 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51d800ce50 con 0x5641e671e1b0 2024-09-21T06:57:44.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.338+0000 7f51f2df2740 1 -- 172.21.15.114:0/4220852743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e671e1b0 msgr2=0x5641e6aee270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.338+0000 7f51d6600640 1 -- 172.21.15.114:0/4220852743 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004220 con 0x5641e671e1b0 2024-09-21T06:57:44.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.338+0000 7f51f2df2740 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e671e1b0 0x5641e6aee270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f51d800cb50 tx=0x7f51d800e720 comp rx=0 tx=0).stop 2024-09-21T06:57:44.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.341+0000 7f51f2df2740 1 -- 172.21.15.114:0/4220852743 shutdown_connections 2024-09-21T06:57:44.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.341+0000 7f51f2df2740 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e6726390 0x5641e6aee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.341+0000 7f51f2df2740 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e671e1b0 0x5641e6aee270 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.341+0000 7f51f2df2740 1 --2- 172.21.15.114:0/4220852743 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6aedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.341+0000 7f51f2df2740 1 -- 172.21.15.114:0/4220852743 >> 172.21.15.114:0/4220852743 conn(0x5641e673b460 msgr2=0x5641e6aebbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:44.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.341+0000 7f51f2df2740 1 -- 172.21.15.114:0/4220852743 shutdown_connections 2024-09-21T06:57:44.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.342+0000 7f51f2df2740 1 -- 172.21.15.114:0/4220852743 wait complete. 2024-09-21T06:57:44.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.342+0000 7f51e2000640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51cc0b6600 0x7f51cc0b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.342+0000 7f51e2000640 1 -- 172.21.15.114:0/4065061373 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f51dc108ff0 con 0x7f51e410dc20 2024-09-21T06:57:44.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.343+0000 7f51f2df2740 1 -- start start 2024-09-21T06:57:44.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.343+0000 7f51f2df2740 1 Processor -- start 2024-09-21T06:57:44.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51e9600640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51cc0b6600 0x7f51cc0b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6af9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6af9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6b058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51f2df2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5641e68b5700 con 0x5641e6747070 2024-09-21T06:57:44.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51f2df2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641e69ef580 con 0x5641e6726390 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.344+0000 7f51f2df2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5641e6afc560 con 0x5641e671e1b0 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51f2df2740 10 client.0 _ll_register_callbacks cb 0x5641e69ff470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51e9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6af9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51ea000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6b058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51ea000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6b058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41272/0 (socket says 172.21.15.114:41272) 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51ea000640 1 -- 172.21.15.114:0/3843745531 learned_addr learned my addr 172.21.15.114:0/3843745531 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51e9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6af9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35146/0 (socket says 172.21.15.114:35146) 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51e8c00640 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6af9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51e8c00640 1 -- 172.21.15.114:0/3843745531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 msgr2=0x5641e6af9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.345+0000 7f51e9600640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51cc0b6600 0x7f51cc0b8b10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5641e6aec9b0 tx=0x7f51d8003ea0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.346+0000 7f51e8c00640 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6af9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.346+0000 7f51e8c00640 1 -- 172.21.15.114:0/3843745531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 msgr2=0x5641e6b058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.346+0000 7f51e8c00640 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6b058a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.346+0000 7f51e8c00640 1 -- 172.21.15.114:0/3843745531 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e67b2650 con 0x5641e6726390 2024-09-21T06:57:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.346+0000 7f51e8c00640 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6af9db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51d0006fd0 tx=0x7f51d000d6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a7f211bf6da29e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.346+0000 7f51caa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.347+0000 7f51caa00640 1 -- 172.21.15.114:0/3843745531 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d000f030 con 0x5641e6726390 2024-09-21T06:57:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.347+0000 7f51caa00640 1 -- 172.21.15.114:0/3843745531 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f51d00145c0 con 0x5641e6726390 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.347+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641e69f1b90 con 0x5641e6726390 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.348+0000 7f51caa00640 1 -- 172.21.15.114:0/3843745531 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f51d00147c0 con 0x5641e6726390 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.348+0000 7f51f2df2740 10 client.26591 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.348+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5641e6a0c950 con 0x5641e6726390 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.348+0000 7f51caa00640 1 -- 172.21.15.114:0/3843745531 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641e69f1b90 con 0x5641e6726390 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51caa00640 1 -- 172.21.15.114:0/3843745531 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f51d0015ad0 con 0x5641e6726390 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51f2df2740 10 client.26591 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51f2df2740 10 client.26591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51f2df2740 20 client.26591 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e67059f0 0x5641e6b1ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51f2df2740 4 client.26591 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641e6a66f00 con 0x5641e67059f0 2024-09-21T06:57:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.349+0000 7f51e9600640 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e67059f0 0x5641e6b1ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.350+0000 7f51caa00640 10 client.26591 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:57:44.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.351+0000 7f51e9600640 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e67059f0 0x5641e6b1ce80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=641ff2674a4875b8 server_cookie=1f71f4886716820 in_seq=0 out_seq=0 2024-09-21T06:57:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.352+0000 7f51caa00640 1 -- 172.21.15.114:0/3843745531 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641e6a66f00 con 0x5641e67059f0 2024-09-21T06:57:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.352+0000 7f51caa00640 10 client.26591 handle_command_reply: tid=0 2024-09-21T06:57:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.352+0000 7f51f2df2740 1 client.26591 shutdown 2024-09-21T06:57:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.353+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 msgr2=0x5641e6af9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.353+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6af9db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51d0006fd0 tx=0x7f51d000d6f0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 shutdown_connections 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6b058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e67059f0 0x5641e6b1ce80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51caa00640 0 client.26591 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6af9db0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3843745531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6af9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 >> 172.21.15.114:0/3843745531 conn(0x5641e673b460 msgr2=0x5641e6af0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.354+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 shutdown_connections 2024-09-21T06:57:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.355+0000 7f51f2df2740 1 -- 172.21.15.114:0/3843745531 wait complete. 2024-09-21T06:57:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.355+0000 7f51f2df2740 20 client.26591 trim_cache size 0 max 16384 2024-09-21T06:57:44.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.383+0000 7f51f2df2740 1 Processor -- start 2024-09-21T06:57:44.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 -- start start 2024-09-21T06:57:44.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6aebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6af6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51e406f130 con 0x5641e6747070 2024-09-21T06:57:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4070c70 con 0x5641e6726390 2024-09-21T06:57:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.384+0000 7f51f2df2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51e406f800 con 0x5641e671e1b0 2024-09-21T06:57:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.385+0000 7f51e9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.385+0000 7f51e9600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b057a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35148/0 (socket says 172.21.15.114:35148) 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.385+0000 7f51e9600640 1 -- 172.21.15.114:0/2882205451 learned_addr learned my addr 172.21.15.114:0/2882205451 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.385+0000 7f51e8c00640 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6aebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.386+0000 7f51e9600640 1 -- 172.21.15.114:0/2882205451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 msgr2=0x5641e6aebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.386+0000 7f51e9600640 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6aebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.386+0000 7f51e9600640 1 -- 172.21.15.114:0/2882205451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 msgr2=0x5641e6af6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.386+0000 7f51e9600640 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6af6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.386+0000 7f51e9600640 1 -- 172.21.15.114:0/2882205451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641e6af0b90 con 0x5641e671e1b0 2024-09-21T06:57:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.387+0000 7f51e9600640 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b057a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51d8003e70 tx=0x7f51d8029000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc89311eb3e806b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.388+0000 7f51d5c00640 1 -- 172.21.15.114:0/2882205451 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d803e070 con 0x5641e671e1b0 2024-09-21T06:57:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.388+0000 7f51d5c00640 1 -- 172.21.15.114:0/2882205451 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641e6a66f00 con 0x5641e671e1b0 2024-09-21T06:57:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.389+0000 7f51d5c00640 1 -- 172.21.15.114:0/2882205451 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8003ac0 con 0x5641e671e1b0 2024-09-21T06:57:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.390+0000 7f51f2df2740 1 -- 172.21.15.114:0/2882205451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 msgr2=0x5641e6b057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.390+0000 7f51f2df2740 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b057a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51d8003e70 tx=0x7f51d8029000 comp rx=0 tx=0).stop 2024-09-21T06:57:44.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.391+0000 7f51f2df2740 1 -- 172.21.15.114:0/2882205451 shutdown_connections 2024-09-21T06:57:44.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.391+0000 7f51f2df2740 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6747070 0x5641e6af6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.391+0000 7f51f2df2740 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6726390 0x5641e6aebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.391+0000 7f51f2df2740 1 --2- 172.21.15.114:0/2882205451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b057a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.391+0000 7f51f2df2740 1 -- 172.21.15.114:0/2882205451 >> 172.21.15.114:0/2882205451 conn(0x5641e673b460 msgr2=0x5641e6aeb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:44.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.391+0000 7f51f2df2740 1 -- 172.21.15.114:0/2882205451 shutdown_connections 2024-09-21T06:57:44.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.392+0000 7f51f2df2740 1 -- 172.21.15.114:0/2882205451 wait complete. 2024-09-21T06:57:44.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.393+0000 7f51f2df2740 1 -- start start 2024-09-21T06:57:44.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.393+0000 7f51f2df2740 1 Processor -- start 2024-09-21T06:57:44.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b01810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6726390 0x5641e6b01d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 0x5641e6afc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f51e40705a0 con 0x5641e6726390 2024-09-21T06:57:44.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51e406fed0 con 0x5641e6747070 2024-09-21T06:57:44.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f51e410bed0 con 0x5641e671e1b0 2024-09-21T06:57:44.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.394+0000 7f51f2df2740 10 client.0 _ll_register_callbacks cb 0x7f51d0009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:44.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.395+0000 7f51e8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6726390 0x5641e6b01d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.395+0000 7f51e8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6726390 0x5641e6b01d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41294/0 (socket says 172.21.15.114:41294) 2024-09-21T06:57:44.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.395+0000 7f51ea000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 0x5641e6afc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.395+0000 7f51ea000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 0x5641e6afc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37488/0 (socket says 172.21.15.114:37488) 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.395+0000 7f51e8c00640 1 -- 172.21.15.114:0/3670269910 learned_addr learned my addr 172.21.15.114:0/3670269910 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.395+0000 7f51e9600640 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b01810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51ea000640 1 -- 172.21.15.114:0/3670269910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 msgr2=0x5641e6b01810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51ea000640 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b01810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51ea000640 1 -- 172.21.15.114:0/3670269910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6726390 msgr2=0x5641e6b01d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51ea000640 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6726390 0x5641e6b01d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51ea000640 1 -- 172.21.15.114:0/3670269910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641e6afcd00 con 0x5641e6747070 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51ea000640 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 0x5641e6afc7c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0cb0a0 tx=0x7f51dc0ce4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d558fff4392b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.396+0000 7f51caa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.397+0000 7f51caa00640 1 -- 172.21.15.114:0/3670269910 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0cf1d0 con 0x5641e6747070 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.397+0000 7f51caa00640 1 -- 172.21.15.114:0/3670269910 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f51dc0ca050 con 0x5641e6747070 2024-09-21T06:57:44.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.397+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5641e6a66f00 con 0x5641e6747070 2024-09-21T06:57:44.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.397+0000 7f51caa00640 1 -- 172.21.15.114:0/3670269910 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f51dc0d35b0 con 0x5641e6747070 2024-09-21T06:57:44.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.397+0000 7f51f2df2740 10 client.26597 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:44.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.398+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5641e6b87dd0 con 0x5641e6747070 2024-09-21T06:57:44.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.398+0000 7f51caa00640 1 -- 172.21.15.114:0/3670269910 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5641e6a66f00 con 0x5641e6747070 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.398+0000 7f51caa00640 1 -- 172.21.15.114:0/3670269910 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f51dc0cad40 con 0x5641e6747070 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.398+0000 7f51f2df2740 10 client.26597 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.398+0000 7f51f2df2740 10 client.26597 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.398+0000 7f51f2df2740 20 client.26597 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.399+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e66dada0 0x5641e6b8a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.399+0000 7f51f2df2740 4 client.26597 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.399+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641e6a629b0 con 0x5641e66dada0 2024-09-21T06:57:44.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.399+0000 7f51e9600640 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e66dada0 0x5641e6b8a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:44.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.401+0000 7f51e9600640 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e66dada0 0x5641e6b8a4e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4d87de27eff605b0 server_cookie=7229ba4a5a65b662 in_seq=0 out_seq=0 2024-09-21T06:57:44.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.401+0000 7f51caa00640 10 client.26597 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:57:44.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51caa00640 1 -- 172.21.15.114:0/3670269910 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641e6a629b0 con 0x5641e66dada0 2024-09-21T06:57:44.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51caa00640 10 client.26597 handle_command_reply: tid=0 2024-09-21T06:57:44.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51f2df2740 1 client.26597 shutdown 2024-09-21T06:57:44.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 msgr2=0x5641e6afc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 0x5641e6afc7c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0cb0a0 tx=0x7f51dc0ce4e0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 shutdown_connections 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.402+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5641e66dada0 0x5641e6b8a4e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.403+0000 7f51caa00640 0 client.26597 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.403+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641e6747070 0x5641e6afc7c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.403+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5641e6726390 0x5641e6b01d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.403+0000 7f51f2df2740 1 --2- 172.21.15.114:0/3670269910 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5641e671e1b0 0x5641e6b01810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.403+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 >> 172.21.15.114:0/3670269910 conn(0x5641e673b460 msgr2=0x5641e6aeed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.404+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 shutdown_connections 2024-09-21T06:57:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.404+0000 7f51f2df2740 1 -- 172.21.15.114:0/3670269910 wait complete. 2024-09-21T06:57:44.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.404+0000 7f51f2df2740 20 client.26597 trim_cache size 0 max 16384 2024-09-21T06:57:44.407 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:57:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.405+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51cc0b6600 msgr2=0x7f51cc0b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.405+0000 7f51eaa00640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51cc0b6600 0x7f51cc0b8b10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5641e6aec9b0 tx=0x7f51d8003ea0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.405+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 msgr2=0x7f51e410b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.405+0000 7f51eaa00640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e410b650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f51dc031aa0 tx=0x7f51dc003970 comp rx=0 tx=0).stop 2024-09-21T06:57:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 shutdown_connections 2024-09-21T06:57:44.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f51e410dc20 0x7f51e410b650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51cc0b6600 0x7f51cc0b8b10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e4105110 0x7f51e410b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 --2- 172.21.15.114:0/4065061373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f51e41047c0 0x7f51e410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 >> 172.21.15.114:0/4065061373 conn(0x7f51e40fe2c0 msgr2=0x7f51e4113f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 shutdown_connections 2024-09-21T06:57:44.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:44.406+0000 7f51eaa00640 1 -- 172.21.15.114:0/4065061373 wait complete. 2024-09-21T06:57:45.053 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:44 smithi114 ceph-mon[30654]: pgmap v504: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-09-21T06:57:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:44 smithi183 ceph-mon[34219]: pgmap v504: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-09-21T06:57:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:44 smithi193 ceph-mon[34601]: pgmap v504: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 2.9 MiB/s wr, 342 op/s 2024-09-21T06:57:45.399 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:57:46.061 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:46.768 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:46 smithi114 ceph-mon[30654]: pgmap v505: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.0 MiB/s wr, 362 op/s 2024-09-21T06:57:46.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 -- 172.21.15.114:0/4261814979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6000109c40 msgr2=0x7f6000110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:46.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 --2- 172.21.15.114:0/4261814979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6000109c40 0x7f6000110470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009860 tx=0x7f5fec02a340 comp rx=0 tx=0).stop 2024-09-21T06:57:46.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 -- 172.21.15.114:0/4261814979 shutdown_connections 2024-09-21T06:57:46.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 --2- 172.21.15.114:0/4261814979 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6000109c40 0x7f6000110470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 --2- 172.21.15.114:0/4261814979 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f60001092f0 0x7f6000109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 --2- 172.21.15.114:0/4261814979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 0x7f6000103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 -- 172.21.15.114:0/4261814979 >> 172.21.15.114:0/4261814979 conn(0x7f60000ff440 msgr2=0x7f6000101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 -- 172.21.15.114:0/4261814979 shutdown_connections 2024-09-21T06:57:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.939+0000 7f6005200640 1 -- 172.21.15.114:0/4261814979 wait complete. 2024-09-21T06:57:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.941+0000 7f6005200640 1 Processor -- start 2024-09-21T06:57:46.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.941+0000 7f6005200640 1 -- start start 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f6005200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 0x7f600010e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f6005200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60001092f0 0x7f600010a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f6005200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6000109c40 0x7f600010ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f6005200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6000113900 con 0x7f6000109c40 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f6005200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6000113780 con 0x7f60001092f0 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f6005200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6000113a80 con 0x7f60001039b0 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f5fff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 0x7f600010e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f5fff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 0x7f600010e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33392/0 (socket says 172.21.15.114:33392) 2024-09-21T06:57:46.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f5fff400640 1 -- 172.21.15.114:0/2835736018 learned_addr learned my addr 172.21.15.114:0/2835736018 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:46.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.942+0000 7f5ffea00640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60001092f0 0x7f600010a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.943+0000 7f5fffe00640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6000109c40 0x7f600010ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.943+0000 7f5ffea00640 1 -- 172.21.15.114:0/2835736018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 msgr2=0x7f600010e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:46.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.943+0000 7f5ffea00640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 0x7f600010e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.943+0000 7f5ffea00640 1 -- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6000109c40 msgr2=0x7f600010ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.943+0000 7f5ffea00640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6000109c40 0x7f600010ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.943+0000 7f5ffea00640 1 -- 172.21.15.114:0/2835736018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60001aca40 con 0x7f60001092f0 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.944+0000 7f5ffea00640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60001092f0 0x7f600010a440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f600006b990 tx=0x7f5ff800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.944+0000 7f5ff7e00640 1 -- 172.21.15.114:0/2835736018 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8010430 con 0x7f60001092f0 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.944+0000 7f5ff7e00640 1 -- 172.21.15.114:0/2835736018 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff8010a80 con 0x7f60001092f0 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.944+0000 7f5ff7e00640 1 -- 172.21.15.114:0/2835736018 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8018b20 con 0x7f60001092f0 2024-09-21T06:57:46.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.944+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60001acc70 con 0x7f60001092f0 2024-09-21T06:57:46.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.945+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60001ad160 con 0x7f60001092f0 2024-09-21T06:57:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.950+0000 7f600d605740 1 Processor -- start 2024-09-21T06:57:46.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.950+0000 7f600d605740 1 -- start start 2024-09-21T06:57:46.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f600d605740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 0x56127c3b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f600d605740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfe51b0 0x56127c3b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f600d605740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f600d605740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56127c32bfb0 con 0x56127bfe51b0 2024-09-21T06:57:46.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f600d605740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56127c31b3e0 con 0x56127c00e070 2024-09-21T06:57:46.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f600d605740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56127c31f1f0 con 0x56127bfed390 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f5fffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 0x56127c3b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f5fffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 0x56127c3b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49560/0 (socket says 172.21.15.114:49560) 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.951+0000 7f5fffe00640 1 -- 172.21.15.114:0/3504937906 learned_addr learned my addr 172.21.15.114:0/3504937906 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.952+0000 7f5fffe00640 1 -- 172.21.15.114:0/3504937906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 msgr2=0x56127c3b59f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.952+0000 7f5fffe00640 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b59f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.952+0000 7f5fffe00640 1 -- 172.21.15.114:0/3504937906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfe51b0 msgr2=0x56127c3b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:46.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.952+0000 7f5fffe00640 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfe51b0 0x56127c3b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.952+0000 7f5fffe00640 1 -- 172.21.15.114:0/3504937906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56127c2e5cd0 con 0x56127c00e070 2024-09-21T06:57:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.952+0000 7f5fffe00640 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 0x56127c3b4f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009830 tx=0x7f5fec005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab4b218af5ea346 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.953+0000 7f5ff4c00640 1 -- 172.21.15.114:0/3504937906 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec041030 con 0x56127c00e070 2024-09-21T06:57:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.953+0000 7f5ff4c00640 1 -- 172.21.15.114:0/3504937906 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fec030c90 con 0x56127c00e070 2024-09-21T06:57:46.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.953+0000 7f5ff4c00640 1 -- 172.21.15.114:0/3504937906 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037040 con 0x56127c00e070 2024-09-21T06:57:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.954+0000 7f5ff7e00640 1 -- 172.21.15.114:0/2835736018 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5ff801f020 con 0x7f60001092f0 2024-09-21T06:57:46.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 -- 172.21.15.114:0/3504937906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 msgr2=0x56127c3b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:46.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 0x56127c3b4f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009830 tx=0x7f5fec005e60 comp rx=0 tx=0).stop 2024-09-21T06:57:46.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 -- 172.21.15.114:0/3504937906 shutdown_connections 2024-09-21T06:57:46.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfe51b0 0x56127c3b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 --2- 172.21.15.114:0/3504937906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127c00e070 0x56127c3b4f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.955+0000 7f600d605740 1 -- 172.21.15.114:0/3504937906 >> 172.21.15.114:0/3504937906 conn(0x56127c002460 msgr2=0x56127c3b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.956+0000 7f5ff7e00640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5fe40b6780 0x7f5fe40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.956+0000 7f5fff400640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5fe40b6780 0x7f5fe40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.956+0000 7f600d605740 1 -- 172.21.15.114:0/3504937906 shutdown_connections 2024-09-21T06:57:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.956+0000 7f600d605740 1 -- 172.21.15.114:0/3504937906 wait complete. 2024-09-21T06:57:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.956+0000 7f5ff7e00640 1 -- 172.21.15.114:0/2835736018 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5ff80e8130 con 0x7f60001092f0 2024-09-21T06:57:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.957+0000 7f5fff400640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5fe40b6780 0x7f5fe40b8c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00096c0 tx=0x7f5ff00046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.957+0000 7f600d605740 1 -- start start 2024-09-21T06:57:46.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.957+0000 7f600d605740 1 Processor -- start 2024-09-21T06:57:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56127c17c700 con 0x56127c00e070 2024-09-21T06:57:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56127c2b6580 con 0x56127bfe51b0 2024-09-21T06:57:46.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56127c3c36f0 con 0x56127bfed390 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.958+0000 7f600d605740 10 client.0 _ll_register_callbacks cb 0x56127c2c6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49570/0 (socket says 172.21.15.114:49570) 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fff400640 1 -- 172.21.15.114:0/1485784902 learned_addr learned my addr 172.21.15.114:0/1485784902 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fffe00640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 -- 172.21.15.114:0/1485784902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 msgr2=0x56127c3c07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 -- 172.21.15.114:0/1485784902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 msgr2=0x56127c3cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:46.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 -- 172.21.15.114:0/1485784902 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127c079650 con 0x56127bfed390 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3c0d60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80e54b0 tx=0x7f5ff80e5a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55e51a731101ebe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fe9600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fe9600640 1 -- 172.21.15.114:0/1485784902 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80a9320 con 0x56127bfed390 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.959+0000 7f5fe9600640 1 -- 172.21.15.114:0/1485784902 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5ff80cfdc0 con 0x56127bfed390 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56127c2b8b90 con 0x56127bfed390 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f5fe9600640 1 -- 172.21.15.114:0/1485784902 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ff80ad6c0 con 0x56127bfed390 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 10 client.26563 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:46.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56127c2d3950 con 0x56127bfed390 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f5fe9600640 1 -- 172.21.15.114:0/1485784902 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56127c2b8b90 con 0x56127bfed390 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f5fe9600640 1 -- 172.21.15.114:0/1485784902 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5ff8096020 con 0x56127bfed390 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 10 client.26563 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 10 client.26563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 20 client.26563 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfcc9f0 0x56127c3d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 4 client.26563 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.960+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56127c32df00 con 0x56127bfcc9f0 2024-09-21T06:57:46.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.962+0000 7f5fff400640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfcc9f0 0x56127c3d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.962+0000 7f5fff400640 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfcc9f0 0x56127c3d27f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de2f231358969288 server_cookie=36f5dc5fb2db269 in_seq=0 out_seq=0 2024-09-21T06:57:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.962+0000 7f5fe9600640 10 client.26563 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:57:46.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.963+0000 7f5fe9600640 1 -- 172.21.15.114:0/1485784902 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56127c32df00 con 0x56127bfcc9f0 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.963+0000 7f5fe9600640 10 client.26563 handle_command_reply: tid=0 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 client.26563 shutdown 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 msgr2=0x56127c3c0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3c0d60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80e54b0 tx=0x7f5ff80e5a30 comp rx=0 tx=0).stop 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 shutdown_connections 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfcc9f0 0x56127c3d27f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3c0d60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.964+0000 7f600d605740 1 --2- 172.21.15.114:0/1485784902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.965+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 >> 172.21.15.114:0/1485784902 conn(0x56127c002460 msgr2=0x56127c3b8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.965+0000 7f5fe9600640 0 client.26563 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:57:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.965+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 shutdown_connections 2024-09-21T06:57:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.965+0000 7f600d605740 1 -- 172.21.15.114:0/1485784902 wait complete. 2024-09-21T06:57:46.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.965+0000 7f600d605740 20 client.26563 trim_cache size 0 max 16384 2024-09-21T06:57:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.989+0000 7f600d605740 1 Processor -- start 2024-09-21T06:57:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.989+0000 7f600d605740 1 -- start start 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f600d605740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3bc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f600d605740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f600d605740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f600d605740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60001b72a0 con 0x56127c00e070 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f600d605740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60001b8de0 con 0x56127bfe51b0 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f600d605740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60001b7970 con 0x56127bfed390 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f5fffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.990+0000 7f5fffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3c1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51412/0 (socket says 172.21.15.114:51412) 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fffe00640 1 -- 172.21.15.114:0/1646714028 learned_addr learned my addr 172.21.15.114:0/1646714028 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fff400640 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3bc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fffe00640 1 -- 172.21.15.114:0/1646714028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 msgr2=0x56127c3b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fffe00640 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fffe00640 1 -- 172.21.15.114:0/1646714028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 msgr2=0x56127c3bc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fffe00640 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3bc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.991+0000 7f5fffe00640 1 -- 172.21.15.114:0/1646714028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56127c3c2bb0 con 0x56127c00e070 2024-09-21T06:57:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.992+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.992+0000 7f5fffe00640 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3c1d00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5fec005850 tx=0x7f5fec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b946cff9cae7d781 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.993+0000 7f5febe00640 1 -- 172.21.15.114:0/1646714028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037070 con 0x56127c00e070 2024-09-21T06:57:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.993+0000 7f5febe00640 1 -- 172.21.15.114:0/1646714028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fec030450 con 0x56127c00e070 2024-09-21T06:57:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.993+0000 7f5febe00640 1 -- 172.21.15.114:0/1646714028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037070 con 0x56127c00e070 2024-09-21T06:57:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.994+0000 7f600d605740 1 -- 172.21.15.114:0/1646714028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 msgr2=0x56127c3c1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.994+0000 7f600d605740 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3c1d00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5fec005850 tx=0x7f5fec005880 comp rx=0 tx=0).stop 2024-09-21T06:57:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 -- 172.21.15.114:0/1646714028 shutdown_connections 2024-09-21T06:57:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127c00e070 0x56127c3c1d00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127bfed390 0x56127c3b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 --2- 172.21.15.114:0/1646714028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3bc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 -- 172.21.15.114:0/1646714028 >> 172.21.15.114:0/1646714028 conn(0x56127c002460 msgr2=0x56127c3b8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 -- 172.21.15.114:0/1646714028 shutdown_connections 2024-09-21T06:57:46.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.995+0000 7f600d605740 1 -- 172.21.15.114:0/1646714028 wait complete. 2024-09-21T06:57:46.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.996+0000 7f600d605740 1 -- start start 2024-09-21T06:57:46.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.996+0000 7f600d605740 1 Processor -- start 2024-09-21T06:57:46.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfed390 0x56127c3c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:46.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127c00e070 0x56127c3bca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:47.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60001b8710 con 0x56127bfed390 2024-09-21T06:57:47.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60001b8040 con 0x56127bfe51b0 2024-09-21T06:57:47.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60001b5280 con 0x56127c00e070 2024-09-21T06:57:47.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f600d605740 10 client.0 _ll_register_callbacks cb 0x7f5ff80aa2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:47.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f5ffea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfed390 0x56127c3c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:47.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.997+0000 7f5fff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5fffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127c00e070 0x56127c3bca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5fff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49588/0 (socket says 172.21.15.114:49588) 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5fff400640 1 -- 172.21.15.114:0/1103628179 learned_addr learned my addr 172.21.15.114:0/1103628179 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5ffea00640 1 -- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127c00e070 msgr2=0x56127c3bca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127c00e070 0x56127c3bca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5ffea00640 1 -- 172.21.15.114:0/1103628179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 msgr2=0x56127c3c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.998+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.999+0000 7f5ffea00640 1 -- 172.21.15.114:0/1103628179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127c3c10d0 con 0x56127bfed390 2024-09-21T06:57:47.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.999+0000 7f5ffea00640 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfed390 0x56127c3c91b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80adb40 tx=0x7f5ff8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f04e6d1598a4b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:47.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.999+0000 7f5fe9600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:47.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.999+0000 7f5fe9600640 1 -- 172.21.15.114:0/1103628179 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80a9320 con 0x56127bfed390 2024-09-21T06:57:47.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:46.999+0000 7f5fe9600640 1 -- 172.21.15.114:0/1103628179 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5ff80a9500 con 0x56127bfed390 2024-09-21T06:57:47.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.000+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fec030450 con 0x56127bfed390 2024-09-21T06:57:47.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.000+0000 7f5fe9600640 1 -- 172.21.15.114:0/1103628179 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ff80ad500 con 0x56127bfed390 2024-09-21T06:57:47.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.000+0000 7f600d605740 10 client.16458 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:47.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.000+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56127c3d2b60 con 0x56127bfed390 2024-09-21T06:57:47.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f5fe9600640 1 -- 172.21.15.114:0/1103628179 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff80ae2b0 con 0x56127bfed390 2024-09-21T06:57:47.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f5fe9600640 1 -- 172.21.15.114:0/1103628179 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f5ff8096070 con 0x56127bfed390 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f600d605740 10 client.16458 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f600d605740 10 client.16458 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f600d605740 20 client.16458 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f600d605740 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfa1da0 0x56127c451170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f600d605740 4 client.16458 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.001+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5fec030450 con 0x56127bfa1da0 2024-09-21T06:57:47.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.002+0000 7f5fff400640 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfa1da0 0x56127c451170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:47.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.002+0000 7f5fe9600640 10 client.16458 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:57:47.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.003+0000 7f5fff400640 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfa1da0 0x56127c451170 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6feee381197a8159 server_cookie=37e5cf714195c7f2 in_seq=0 out_seq=0 2024-09-21T06:57:47.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.003+0000 7f5fe9600640 1 -- 172.21.15.114:0/1103628179 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5fec030450 con 0x56127bfa1da0 2024-09-21T06:57:47.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.003+0000 7f5fe9600640 10 client.16458 handle_command_reply: tid=0 2024-09-21T06:57:47.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.003+0000 7f600d605740 1 client.16458 shutdown 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfed390 msgr2=0x56127c3c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfed390 0x56127c3c91b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80adb40 tx=0x7f5ff8010800 comp rx=0 tx=0).stop 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 shutdown_connections 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56127bfa1da0 0x56127c451170 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56127c00e070 0x56127c3bca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56127bfed390 0x56127c3c91b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 --2- 172.21.15.114:0/1103628179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56127bfe51b0 0x56127c3c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 >> 172.21.15.114:0/1103628179 conn(0x56127c002460 msgr2=0x56127c3b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f5fe9600640 0 client.16458 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:57:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.005+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 shutdown_connections 2024-09-21T06:57:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.006+0000 7f600d605740 1 -- 172.21.15.114:0/1103628179 wait complete. 2024-09-21T06:57:47.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.006+0000 7f600d605740 20 client.16458 trim_cache size 0 max 16384 2024-09-21T06:57:47.011 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:57:47.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5fe40b6780 msgr2=0x7f5fe40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5fe40b6780 0x7f5fe40b8c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00096c0 tx=0x7f5ff00046c0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60001092f0 msgr2=0x7f600010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60001092f0 0x7f600010a440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f600006b990 tx=0x7f5ff800ef90 comp rx=0 tx=0).stop 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 shutdown_connections 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6000109c40 0x7f600010ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5fe40b6780 0x7f5fe40b8c90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60001092f0 0x7f600010a440 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 --2- 172.21.15.114:0/2835736018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60001039b0 0x7f600010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 >> 172.21.15.114:0/2835736018 conn(0x7f60000ff440 msgr2=0x7f6000108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 shutdown_connections 2024-09-21T06:57:47.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:47.008+0000 7f6005200640 1 -- 172.21.15.114:0/2835736018 wait complete. 2024-09-21T06:57:47.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:46 smithi183 ceph-mon[34219]: pgmap v505: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.0 MiB/s wr, 362 op/s 2024-09-21T06:57:47.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:46 smithi193 ceph-mon[34601]: pgmap v505: 129 pgs: 129 active+clean; 1.8 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.0 MiB/s wr, 362 op/s 2024-09-21T06:57:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 190.0 s, 1717.2 tps, lat 58.505 ms stddev 71.666 2024-09-21T06:57:48.353 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:57:49.018 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:48 smithi114 ceph-mon[30654]: pgmap v506: 129 pgs: 129 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.9 MiB/s wr, 557 op/s 2024-09-21T06:57:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:48 smithi183 ceph-mon[34219]: pgmap v506: 129 pgs: 129 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.9 MiB/s wr, 557 op/s 2024-09-21T06:57:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:48 smithi193 ceph-mon[34601]: pgmap v506: 129 pgs: 129 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 4.9 MiB/s wr, 557 op/s 2024-09-21T06:57:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.736+0000 7f4832200640 1 -- 172.21.15.114:0/1687021131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 msgr2=0x7f482c102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.736+0000 7f4832200640 1 --2- 172.21.15.114:0/1687021131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c102940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4820009900 tx=0x7f482002a410 comp rx=0 tx=0).stop 2024-09-21T06:57:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 -- 172.21.15.114:0/1687021131 shutdown_connections 2024-09-21T06:57:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 --2- 172.21.15.114:0/1687021131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f482c1079f0 0x7f482c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 --2- 172.21.15.114:0/1687021131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 0x7f482c1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 --2- 172.21.15.114:0/1687021131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c102940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 -- 172.21.15.114:0/1687021131 >> 172.21.15.114:0/1687021131 conn(0x7f482c0fe230 msgr2=0x7f482c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 -- 172.21.15.114:0/1687021131 shutdown_connections 2024-09-21T06:57:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.737+0000 7f4832200640 1 -- 172.21.15.114:0/1687021131 wait complete. 2024-09-21T06:57:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.739+0000 7f4832200640 1 Processor -- start 2024-09-21T06:57:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.739+0000 7f4832200640 1 -- start start 2024-09-21T06:57:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.740+0000 7f4832200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f4832200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 0x7f482c1a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f4832200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f482c1079f0 0x7f482c1a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f4832200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f482c119e80 con 0x7f482c1079f0 2024-09-21T06:57:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f4832200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f482c119d00 con 0x7f482c102530 2024-09-21T06:57:49.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f4832200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f482c11a000 con 0x7f482c102e80 2024-09-21T06:57:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f482be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 0x7f482c1a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f482be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 0x7f482c1a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33438/0 (socket says 172.21.15.114:33438) 2024-09-21T06:57:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f482be00640 1 -- 172.21.15.114:0/1466798200 learned_addr learned my addr 172.21.15.114:0/1466798200 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.741+0000 7f4830e00640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.742+0000 7f4830e00640 1 -- 172.21.15.114:0/1466798200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 msgr2=0x7f482c1a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.742+0000 7f4830e00640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 0x7f482c1a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.742+0000 7f4830e00640 1 -- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f482c1079f0 msgr2=0x7f482c1a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.742+0000 7f4830e00640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f482c1079f0 0x7f482c1a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.742+0000 7f4830e00640 1 -- 172.21.15.114:0/1466798200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c1a67d0 con 0x7f482c102530 2024-09-21T06:57:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.744+0000 7f4830e00640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c1a08a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f482002aa00 tx=0x7f4820003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.746+0000 7f4829600640 1 -- 172.21.15.114:0/1466798200 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482003f070 con 0x7f482c102530 2024-09-21T06:57:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.746+0000 7f4829600640 1 -- 172.21.15.114:0/1466798200 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820031b50 con 0x7f482c102530 2024-09-21T06:57:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.746+0000 7f4829600640 1 -- 172.21.15.114:0/1466798200 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820035030 con 0x7f482c102530 2024-09-21T06:57:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.746+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c1a6a60 con 0x7f482c102530 2024-09-21T06:57:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.747+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c1a6ed0 con 0x7f482c102530 2024-09-21T06:57:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.750+0000 7f4829600640 1 -- 172.21.15.114:0/1466798200 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4820042070 con 0x7f482c102530 2024-09-21T06:57:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.751+0000 7f483a898740 1 Processor -- start 2024-09-21T06:57:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.752+0000 7f483a898740 1 -- start start 2024-09-21T06:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.752+0000 7f4829600640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48140b6680 0x7f48140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f4829600640 1 -- 172.21.15.114:0/1466798200 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4820107c30 con 0x7f482c102530 2024-09-21T06:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f483a898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eac3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f483a898740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eac4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f483a898740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eac4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f483a898740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d82ea396b0 con 0x55d82e6f62f0 2024-09-21T06:57:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f483a898740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d82ea28ae0 con 0x55d82e6fe4d0 2024-09-21T06:57:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f483a898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d82ea2c8f0 con 0x55d82e71f1b0 2024-09-21T06:57:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f4831800640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48140b6680 0x7f48140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f4830e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eac3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f482be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eac4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f4830e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eac3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33450/0 (socket says 172.21.15.114:33450) 2024-09-21T06:57:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f4830e00640 1 -- 172.21.15.114:0/2116025299 learned_addr learned my addr 172.21.15.114:0/2116025299 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.753+0000 7f4831800640 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eac4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4830e00640 1 -- 172.21.15.114:0/2116025299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 msgr2=0x55d82eac4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4830e00640 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eac4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4830e00640 1 -- 172.21.15.114:0/2116025299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 msgr2=0x55d82eac4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4830e00640 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eac4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4830e00640 1 -- 172.21.15.114:0/2116025299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82e9f5780 con 0x55d82e71f1b0 2024-09-21T06:57:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4830e00640 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eac3e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f48200cd410 tx=0x7f48200cd5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25b750563519e4d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f4831800640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48140b6680 0x7f48140b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4824009810 tx=0x7f4824009780 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.754+0000 7f481e000640 1 -- 172.21.15.114:0/2116025299 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820049040 con 0x55d82e71f1b0 2024-09-21T06:57:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.755+0000 7f481e000640 1 -- 172.21.15.114:0/2116025299 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820031d00 con 0x55d82e71f1b0 2024-09-21T06:57:49.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.755+0000 7f481e000640 1 -- 172.21.15.114:0/2116025299 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48200d6f00 con 0x55d82e71f1b0 2024-09-21T06:57:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.757+0000 7f483a898740 1 -- 172.21.15.114:0/2116025299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 msgr2=0x55d82eac3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.757+0000 7f483a898740 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eac3e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f48200cd410 tx=0x7f48200cd5b0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.759+0000 7f483a898740 1 -- 172.21.15.114:0/2116025299 shutdown_connections 2024-09-21T06:57:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.759+0000 7f483a898740 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eac4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.759+0000 7f483a898740 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eac4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.759+0000 7f483a898740 1 --2- 172.21.15.114:0/2116025299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eac3e10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.760+0000 7f483a898740 1 -- 172.21.15.114:0/2116025299 >> 172.21.15.114:0/2116025299 conn(0x55d82e713300 msgr2=0x55d82eac1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.763 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.760+0000 7f483a898740 1 -- 172.21.15.114:0/2116025299 shutdown_connections 2024-09-21T06:57:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.760+0000 7f483a898740 1 -- 172.21.15.114:0/2116025299 wait complete. 2024-09-21T06:57:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.761+0000 7f483a898740 1 -- start start 2024-09-21T06:57:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.761+0000 7f483a898740 1 Processor -- start 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eacf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6fe4d0 0x55d82eacf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 0x55d82eadb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d82e960ab0 con 0x55d82e6f62f0 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d82e9cf570 con 0x55d82e71f1b0 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d82ead2350 con 0x55d82e6fe4d0 2024-09-21T06:57:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.762+0000 7f483a898740 10 client.0 _ll_register_callbacks cb 0x55d82e9de100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.763+0000 7f4830e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eacf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.763+0000 7f4831800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 0x55d82eadb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.763+0000 7f4831800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 0x55d82eadb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49624/0 (socket says 172.21.15.114:49624) 2024-09-21T06:57:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.763+0000 7f4831800640 1 -- 172.21.15.114:0/671144218 learned_addr learned my addr 172.21.15.114:0/671144218 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.763+0000 7f482be00640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6fe4d0 0x55d82eacf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4831800640 1 -- 172.21.15.114:0/671144218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6fe4d0 msgr2=0x55d82eacf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4831800640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6fe4d0 0x55d82eacf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4831800640 1 -- 172.21.15.114:0/671144218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 msgr2=0x55d82eacf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4831800640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eacf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4831800640 1 -- 172.21.15.114:0/671144218 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82e78a790 con 0x55d82e71f1b0 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4830e00640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eacf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4831800640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 0x55d82eadb530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4824013fd0 tx=0x7f4824009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d6168ed56dc4a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.764+0000 7f4812a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.765+0000 7f4812a00640 1 -- 172.21.15.114:0/671144218 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824022070 con 0x55d82e71f1b0 2024-09-21T06:57:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.765+0000 7f4812a00640 1 -- 172.21.15.114:0/671144218 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4824004390 con 0x55d82e71f1b0 2024-09-21T06:57:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.765+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4820031d00 con 0x55d82e71f1b0 2024-09-21T06:57:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.765+0000 7f4812a00640 1 -- 172.21.15.114:0/671144218 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4824030050 con 0x55d82e71f1b0 2024-09-21T06:57:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.765+0000 7f483a898740 10 client.26624 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.765+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d82e803620 con 0x55d82e71f1b0 2024-09-21T06:57:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f4812a00640 1 -- 172.21.15.114:0/671144218 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820031d00 con 0x55d82e71f1b0 2024-09-21T06:57:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f4812a00640 1 -- 172.21.15.114:0/671144218 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f482400f040 con 0x55d82e71f1b0 2024-09-21T06:57:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f483a898740 10 client.26624 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:49.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f483a898740 10 client.26624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f483a898740 20 client.26624 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f483a898740 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6ddd80 0x55d82eae12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f483a898740 4 client.26624 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.766+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d82ea3b600 con 0x55d82e6ddd80 2024-09-21T06:57:49.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.767+0000 7f4830e00640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6ddd80 0x55d82eae12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.767+0000 7f4830e00640 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6ddd80 0x55d82eae12b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fecf8983db6aa74 server_cookie=d56aacd0c74852d1 in_seq=0 out_seq=0 2024-09-21T06:57:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.767+0000 7f4812a00640 10 client.26624 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.768+0000 7f4812a00640 1 -- 172.21.15.114:0/671144218 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d82ea3b600 con 0x55d82e6ddd80 2024-09-21T06:57:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.768+0000 7f4812a00640 10 client.26624 handle_command_reply: tid=0 2024-09-21T06:57:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.768+0000 7f483a898740 1 client.26624 shutdown 2024-09-21T06:57:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.768+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 msgr2=0x55d82eadb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.768+0000 7f483a898740 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 0x55d82eadb530 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4824013fd0 tx=0x7f4824009500 comp rx=0 tx=0).stop 2024-09-21T06:57:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.769+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 shutdown_connections 2024-09-21T06:57:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.769+0000 7f483a898740 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e71f1b0 0x55d82eadb530 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.769+0000 7f483a898740 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6ddd80 0x55d82eae12b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.769+0000 7f483a898740 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6fe4d0 0x55d82eacf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.769+0000 7f483a898740 1 --2- 172.21.15.114:0/671144218 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6f62f0 0x55d82eacf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.770+0000 7f4812a00640 0 client.26624 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:49.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.769+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 >> 172.21.15.114:0/671144218 conn(0x55d82e713300 msgr2=0x55d82eac4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.770+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 shutdown_connections 2024-09-21T06:57:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.771+0000 7f483a898740 1 -- 172.21.15.114:0/671144218 wait complete. 2024-09-21T06:57:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.771+0000 7f483a898740 20 client.26624 trim_cache size 0 max 16384 2024-09-21T06:57:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.795+0000 7f483a898740 1 Processor -- start 2024-09-21T06:57:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.795+0000 7f483a898740 1 -- start start 2024-09-21T06:57:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f483a898740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6f62f0 0x55d82eac5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f483a898740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eadb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f483a898740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e71f1b0 0x55d82eacc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f483a898740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1aead0 con 0x55d82e71f1b0 2024-09-21T06:57:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f483a898740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1b0610 con 0x55d82e6fe4d0 2024-09-21T06:57:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f483a898740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1af1a0 con 0x55d82e6f62f0 2024-09-21T06:57:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.796+0000 7f4830e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6f62f0 0x55d82eac5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.797+0000 7f4830e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6f62f0 0x55d82eac5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33462/0 (socket says 172.21.15.114:33462) 2024-09-21T06:57:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.797+0000 7f482be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eadb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.797+0000 7f482be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eadb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49626/0 (socket says 172.21.15.114:49626) 2024-09-21T06:57:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.797+0000 7f482be00640 1 -- 172.21.15.114:0/2750444918 learned_addr learned my addr 172.21.15.114:0/2750444918 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.797+0000 7f4831800640 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e71f1b0 0x55d82eacc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.798+0000 7f482be00640 1 -- 172.21.15.114:0/2750444918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6f62f0 msgr2=0x55d82eac5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.798+0000 7f482be00640 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6f62f0 0x55d82eac5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.799+0000 7f482be00640 1 -- 172.21.15.114:0/2750444918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e71f1b0 msgr2=0x55d82eacc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.799+0000 7f482be00640 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e71f1b0 0x55d82eacc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.799+0000 7f482be00640 1 -- 172.21.15.114:0/2750444918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82eac6110 con 0x55d82e6fe4d0 2024-09-21T06:57:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.799+0000 7f4831800640 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e71f1b0 0x55d82eacc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:57:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.801+0000 7f482be00640 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eadb4b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4818009f50 tx=0x7f4818005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15317a9c7239839 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.804+0000 7f481d600640 1 -- 172.21.15.114:0/2750444918 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48180230d0 con 0x55d82e6fe4d0 2024-09-21T06:57:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.804+0000 7f481d600640 1 -- 172.21.15.114:0/2750444918 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4818023270 con 0x55d82e6fe4d0 2024-09-21T06:57:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.804+0000 7f481d600640 1 -- 172.21.15.114:0/2750444918 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818021440 con 0x55d82e6fe4d0 2024-09-21T06:57:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.805+0000 7f483a898740 1 -- 172.21.15.114:0/2750444918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 msgr2=0x55d82eadb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.805+0000 7f483a898740 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eadb4b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4818009f50 tx=0x7f4818005f90 comp rx=0 tx=0).stop 2024-09-21T06:57:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 -- 172.21.15.114:0/2750444918 shutdown_connections 2024-09-21T06:57:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e71f1b0 0x55d82eacc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6fe4d0 0x55d82eadb4b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 --2- 172.21.15.114:0/2750444918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e6f62f0 0x55d82eac5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 -- 172.21.15.114:0/2750444918 >> 172.21.15.114:0/2750444918 conn(0x55d82e713300 msgr2=0x55d82eac4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 -- 172.21.15.114:0/2750444918 shutdown_connections 2024-09-21T06:57:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.806+0000 7f483a898740 1 -- 172.21.15.114:0/2750444918 wait complete. 2024-09-21T06:57:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.807+0000 7f483a898740 1 -- start start 2024-09-21T06:57:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.807+0000 7f483a898740 1 Processor -- start 2024-09-21T06:57:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.810+0000 7f483a898740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6f62f0 0x55d82ead4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.810+0000 7f483a898740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6fe4d0 0x55d82ead5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.810+0000 7f4830e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6f62f0 0x55d82ead4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f4830e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6f62f0 0x55d82ead4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49640/0 (socket says 172.21.15.114:49640) 2024-09-21T06:57:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f4830e00640 1 -- 172.21.15.114:0/2800589412 learned_addr learned my addr 172.21.15.114:0/2800589412 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eb49cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1aff40 con 0x55d82e6fe4d0 2024-09-21T06:57:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1af870 con 0x55d82e6f62f0 2024-09-21T06:57:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f482c1a19c0 con 0x55d82e71f1b0 2024-09-21T06:57:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.811+0000 7f483a898740 10 client.0 _ll_register_callbacks cb 0x7f4824003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.812+0000 7f482be00640 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6fe4d0 0x55d82ead5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.812+0000 7f482be00640 1 -- 172.21.15.114:0/2800589412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 msgr2=0x55d82eb49cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:57:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.812+0000 7f482be00640 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eb49cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.812+0000 7f482be00640 1 -- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6f62f0 msgr2=0x55d82ead4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.812+0000 7f482be00640 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6f62f0 0x55d82ead4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.812+0000 7f482be00640 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82eb4a350 con 0x55d82e6fe4d0 2024-09-21T06:57:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.813+0000 7f482be00640 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6fe4d0 0x55d82ead5360 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4818023fd0 tx=0x7f48180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fc58177006cdbb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.813+0000 7f4812a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.815+0000 7f4812a00640 1 -- 172.21.15.114:0/2800589412 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481803b070 con 0x55d82e6fe4d0 2024-09-21T06:57:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.815+0000 7f4812a00640 1 -- 172.21.15.114:0/2800589412 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4818005230 con 0x55d82e6fe4d0 2024-09-21T06:57:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.816+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4818023270 con 0x55d82e6fe4d0 2024-09-21T06:57:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.816+0000 7f4812a00640 1 -- 172.21.15.114:0/2800589412 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f48180230d0 con 0x55d82e6fe4d0 2024-09-21T06:57:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.817+0000 7f483a898740 10 client.16473 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.817+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d82eb4b5c0 con 0x55d82e6fe4d0 2024-09-21T06:57:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.818+0000 7f4812a00640 1 -- 172.21.15.114:0/2800589412 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4818023270 con 0x55d82e6fe4d0 2024-09-21T06:57:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f4812a00640 1 -- 172.21.15.114:0/2800589412 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f4818003b30 con 0x55d82e6fe4d0 2024-09-21T06:57:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f483a898740 10 client.16473 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f483a898740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f483a898740 20 client.16473 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6b31c0 0x55d82eb4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f483a898740 4 client.16473 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:57:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.819+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d82ea370b0 con 0x55d82e6b31c0 2024-09-21T06:57:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.820+0000 7f4830e00640 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6b31c0 0x55d82eb4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.822+0000 7f4812a00640 10 client.16473 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.822+0000 7f4830e00640 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6b31c0 0x55d82eb4dc30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cedd1ac0f0b4749 server_cookie=1c889bba273062e in_seq=0 out_seq=0 2024-09-21T06:57:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.872+0000 7f04a3200640 1 -- 172.21.15.114:0/83950667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 msgr2=0x7f04a41123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.872+0000 7f04a3200640 1 --2- 172.21.15.114:0/83950667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41123b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0494009900 tx=0x7f049402a4d0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.875+0000 7f04a3200640 1 -- 172.21.15.114:0/83950667 shutdown_connections 2024-09-21T06:57:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.875+0000 7f04a3200640 1 --2- 172.21.15.114:0/83950667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41123b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.875+0000 7f04a3200640 1 --2- 172.21.15.114:0/83950667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 0x7f04a410fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.875+0000 7f04a3200640 1 --2- 172.21.15.114:0/83950667 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04a41023e0 0x7f04a41027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.875+0000 7f04a3200640 1 -- 172.21.15.114:0/83950667 >> 172.21.15.114:0/83950667 conn(0x7f04a40fe2c0 msgr2=0x7f04a4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.876+0000 7f04a3200640 1 -- 172.21.15.114:0/83950667 shutdown_connections 2024-09-21T06:57:49.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.876+0000 7f04a3200640 1 -- 172.21.15.114:0/83950667 wait complete. 2024-09-21T06:57:49.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.878+0000 7f04a3200640 1 Processor -- start 2024-09-21T06:57:49.886 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.882+0000 7f04a3200640 1 -- start start 2024-09-21T06:57:49.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04a41023e0 0x7f04a41b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 0x7f04a41ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a3200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f04a406da90 con 0x7f04a41023e0 2024-09-21T06:57:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04a406d910 con 0x7f04a4102d30 2024-09-21T06:57:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04a406dc10 con 0x7f04a410ff90 2024-09-21T06:57:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 0x7f04a41ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04a41023e0 0x7f04a41b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33498/0 (socket says 172.21.15.114:33498) 2024-09-21T06:57:49.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.890+0000 7f04a2800640 1 -- 172.21.15.114:0/19319482 learned_addr learned my addr 172.21.15.114:0/19319482 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.891+0000 7f04a1400640 1 -- 172.21.15.114:0/19319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 msgr2=0x7f04a41ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.891+0000 7f04a1400640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.891+0000 7f04a1400640 1 -- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04a41023e0 msgr2=0x7f04a41b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.891+0000 7f04a1400640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04a41023e0 0x7f04a41b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.891+0000 7f04a1400640 1 -- 172.21.15.114:0/19319482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a41b19f0 con 0x7f04a4102d30 2024-09-21T06:57:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.892+0000 7f04a1400640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 0x7f04a41ac050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f049000d6d0 tx=0x7f049000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.893+0000 7f049aa00640 1 -- 172.21.15.114:0/19319482 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490017070 con 0x7f04a4102d30 2024-09-21T06:57:49.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.893+0000 7f049aa00640 1 -- 172.21.15.114:0/19319482 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0490009820 con 0x7f04a4102d30 2024-09-21T06:57:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.893+0000 7f049aa00640 1 -- 172.21.15.114:0/19319482 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490004c40 con 0x7f04a4102d30 2024-09-21T06:57:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.894+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a41b1ce0 con 0x7f04a4102d30 2024-09-21T06:57:49.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.894+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a41b2250 con 0x7f04a4102d30 2024-09-21T06:57:49.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.899+0000 7f04afcda740 1 Processor -- start 2024-09-21T06:57:49.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.899+0000 7f04afcda740 1 -- start start 2024-09-21T06:57:49.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.900+0000 7f04afcda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc0093d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.900+0000 7f04a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc0093d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.900+0000 7f04a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc0093d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51490/0 (socket says 172.21.15.114:51490) 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04a2800640 1 -- 172.21.15.114:0/2181221998 learned_addr learned my addr 172.21.15.114:0/2181221998 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04afcda740 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0094270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04afcda740 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc00947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558fc000afb0 con 0x558fbfced070 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558fbfffa3e0 con 0x558fbfccc390 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558fbfffe1f0 con 0x558fbfcc41b0 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.901+0000 7f04a1e00640 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0094270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.902+0000 7f04a2800640 1 -- 172.21.15.114:0/2181221998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 msgr2=0x558fc0094270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.902+0000 7f04a2800640 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0094270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.902+0000 7f04a2800640 1 -- 172.21.15.114:0/2181221998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 msgr2=0x558fc00947b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.902+0000 7f04a2800640 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc00947b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.902+0000 7f04a2800640 1 -- 172.21.15.114:0/2181221998 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbffc4cd0 con 0x558fbfced070 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.904+0000 7f04a2800640 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc0093d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f04940043b0 tx=0x7f04940043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ce6c1afb00cc9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.905+0000 7f048f400640 1 -- 172.21.15.114:0/2181221998 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494035040 con 0x558fbfced070 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.905+0000 7f048f400640 1 -- 172.21.15.114:0/2181221998 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0494004740 con 0x558fbfced070 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.905+0000 7f048f400640 1 -- 172.21.15.114:0/2181221998 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494003850 con 0x558fbfced070 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.907+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 msgr2=0x558fc0093d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.907+0000 7f04afcda740 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc0093d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f04940043b0 tx=0x7f04940043e0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.908+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 shutdown_connections 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.908+0000 7f04afcda740 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc00947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.908+0000 7f04afcda740 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0094270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.908+0000 7f04afcda740 1 --2- 172.21.15.114:0/2181221998 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc0093d30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.908+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 >> 172.21.15.114:0/2181221998 conn(0x558fbfce1460 msgr2=0x558fc0091bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.911+0000 7f049aa00640 1 -- 172.21.15.114:0/19319482 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0490005220 con 0x7f04a4102d30 2024-09-21T06:57:49.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.912+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 shutdown_connections 2024-09-21T06:57:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.912+0000 7f04afcda740 1 -- 172.21.15.114:0/2181221998 wait complete. 2024-09-21T06:57:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.914+0000 7f049aa00640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04880b66e0 0x7f04880b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.915+0000 7f049aa00640 1 -- 172.21.15.114:0/19319482 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f04900e7190 con 0x7f04a4102d30 2024-09-21T06:57:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.915+0000 7f04afcda740 1 -- start start 2024-09-21T06:57:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.915+0000 7f04a2800640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04880b66e0 0x7f04880b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.915+0000 7f04afcda740 1 Processor -- start 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc009f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc009ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558fbfe5b700 con 0x558fbfced070 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558fbff95580 con 0x558fbfccc390 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558fc00a24b0 con 0x558fbfcc41b0 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04afcda740 10 client.0 _ll_register_callbacks cb 0x558fbffa5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04a1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc009ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.916+0000 7f04a1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc009ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49652/0 (socket says 172.21.15.114:49652) 2024-09-21T06:57:49.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 -- 172.21.15.114:0/1284991464 learned_addr learned my addr 172.21.15.114:0/1284991464 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a2800640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04880b66e0 0x7f04880b8bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0494004f80 tx=0x7f0494031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 -- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 msgr2=0x558fc009f9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc009f9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 -- 172.21.15.114:0/1284991464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 msgr2=0x558fc00ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 -- 172.21.15.114:0/1284991464 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbfd58650 con 0x558fbfccc390 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f04a1400640 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc009ffd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f04900acca0 tx=0x7f04900ac950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821e60baa849b66c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.917+0000 7f0483e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.918+0000 7f0483e00640 1 -- 172.21.15.114:0/1284991464 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490020050 con 0x558fbfccc390 2024-09-21T06:57:49.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.918+0000 7f0483e00640 1 -- 172.21.15.114:0/1284991464 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f04900b4d70 con 0x558fbfccc390 2024-09-21T06:57:49.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.920+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fbff97b90 con 0x558fbfccc390 2024-09-21T06:57:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.920+0000 7f4812a00640 1 -- 172.21.15.114:0/2800589412 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d82ea370b0 con 0x55d82e6b31c0 2024-09-21T06:57:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.920+0000 7f4812a00640 10 client.16473 handle_command_reply: tid=0 2024-09-21T06:57:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.920+0000 7f483a898740 1 client.16473 shutdown 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6fe4d0 msgr2=0x55d82ead5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6fe4d0 0x55d82ead5360 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4818023fd0 tx=0x7f48180048f0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 shutdown_connections 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d82e6b31c0 0x55d82eb4dc30 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d82e71f1b0 0x55d82eb49cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d82e6fe4d0 0x55d82ead5360 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 --2- 172.21.15.114:0/2800589412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d82e6f62f0 0x55d82ead4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f4812a00640 0 client.16473 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 >> 172.21.15.114:0/2800589412 conn(0x55d82e713300 msgr2=0x55d82eac4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f0483e00640 1 -- 172.21.15.114:0/1284991464 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f04900b6730 con 0x558fbfccc390 2024-09-21T06:57:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 shutdown_connections 2024-09-21T06:57:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.922+0000 7f483a898740 1 -- 172.21.15.114:0/2800589412 wait complete. 2024-09-21T06:57:49.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.921+0000 7f04afcda740 10 client.26639 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.922+0000 7f483a898740 20 client.16473 trim_cache size 0 max 16384 2024-09-21T06:57:49.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.922+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558fbffb2950 con 0x558fbfccc390 2024-09-21T06:57:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:57:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:57:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:57:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:57:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.922+0000 7f0483e00640 1 -- 172.21.15.114:0/1284991464 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558fbff97b90 con 0x558fbfccc390 2024-09-21T06:57:49.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f0483e00640 1 -- 172.21.15.114:0/1284991464 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f04900ab9e0 con 0x558fbfccc390 2024-09-21T06:57:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48140b6680 msgr2=0x7f48140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48140b6680 0x7f48140b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4824009810 tx=0x7f4824009780 comp rx=0 tx=0).stop 2024-09-21T06:57:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 msgr2=0x7f482c1a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c1a08a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f482002aa00 tx=0x7f4820003b90 comp rx=0 tx=0).stop 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 shutdown_connections 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f482c1079f0 0x7f482c1a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f482c102e80 0x7f482c1a0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48140b6680 0x7f48140b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 --2- 172.21.15.114:0/1466798200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f482c102530 0x7f482c1a08a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 >> 172.21.15.114:0/1466798200 conn(0x7f482c0fe230 msgr2=0x7f482c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.924+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 shutdown_connections 2024-09-21T06:57:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:57:49.924+0000 7f4832200640 1 -- 172.21.15.114:0/1466798200 wait complete. 2024-09-21T06:57:49.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04afcda740 10 client.26639 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04afcda740 10 client.26639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04afcda740 20 client.26639 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04afcda740 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfcab9f0 0x558fc00b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04afcda740 4 client.26639 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc000cf00 con 0x558fbfcab9f0 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.923+0000 7f04a1e00640 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfcab9f0 0x558fc00b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.925+0000 7f0483e00640 10 client.26639 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:57:49.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.927+0000 7f04a1e00640 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfcab9f0 0x558fc00b1a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b54cbed0add6e293 server_cookie=4a2b81e9340f1b07 in_seq=0 out_seq=0 2024-09-21T06:57:49.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f0483e00640 1 -- 172.21.15.114:0/1284991464 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558fc000cf00 con 0x558fbfcab9f0 2024-09-21T06:57:49.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f0483e00640 10 client.26639 handle_command_reply: tid=0 2024-09-21T06:57:49.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 client.26639 shutdown 2024-09-21T06:57:49.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 msgr2=0x558fc009ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc009ffd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f04900acca0 tx=0x7f04900ac950 comp rx=0 tx=0).stop 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 shutdown_connections 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfcab9f0 0x558fc00b1a50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfccc390 0x558fc009ffd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 --2- 172.21.15.114:0/1284991464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc009f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f0483e00640 0 client.26639 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 >> 172.21.15.114:0/1284991464 conn(0x558fbfce1460 msgr2=0x558fc0096ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 shutdown_connections 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.928+0000 7f04afcda740 1 -- 172.21.15.114:0/1284991464 wait complete. 2024-09-21T06:57:49.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.929+0000 7f04afcda740 20 client.26639 trim_cache size 0 max 16384 2024-09-21T06:57:49.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.951+0000 7f04afcda740 1 Processor -- start 2024-09-21T06:57:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.951+0000 7f04afcda740 1 -- start start 2024-09-21T06:57:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.951+0000 7f04afcda740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0095470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.951+0000 7f04afcda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfccc390 0x558fc0098fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04afcda740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfced070 0x558fc00994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04afcda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f048400fef0 con 0x558fbfccc390 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04afcda740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0484011a30 con 0x558fbfced070 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04afcda740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04840105c0 con 0x558fbfcc41b0 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0095470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04a1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0095470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33530/0 (socket says 172.21.15.114:33530) 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04a1e00640 1 -- 172.21.15.114:0/4111910590 learned_addr learned my addr 172.21.15.114:0/4111910590 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.952+0000 7f04a1400640 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfccc390 0x558fc0098fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a2800640 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfced070 0x558fc00994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a1400640 1 -- 172.21.15.114:0/4111910590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 msgr2=0x558fc0095470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a1400640 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0095470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a1400640 1 -- 172.21.15.114:0/4111910590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfced070 msgr2=0x558fc00994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a1400640 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfced070 0x558fc00994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a1400640 1 -- 172.21.15.114:0/4111910590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc00929e0 con 0x558fbfccc390 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a2800640 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfced070 0x558fc00994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f04a1400640 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfccc390 0x558fc0098fb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0490009610 tx=0x7f0490000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0efd6bc60438ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f048ea00640 1 -- 172.21.15.114:0/4111910590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0490020070 con 0x558fbfccc390 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f048ea00640 1 -- 172.21.15.114:0/4111910590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04900b1020 con 0x558fbfccc390 2024-09-21T06:57:49.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.953+0000 7f048ea00640 1 -- 172.21.15.114:0/4111910590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049005c070 con 0x558fbfccc390 2024-09-21T06:57:49.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 -- 172.21.15.114:0/4111910590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfccc390 msgr2=0x558fc0098fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfccc390 0x558fc0098fb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0490009610 tx=0x7f0490000bc0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 -- 172.21.15.114:0/4111910590 shutdown_connections 2024-09-21T06:57:49.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfced070 0x558fc00994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfccc390 0x558fc0098fb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 --2- 172.21.15.114:0/4111910590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfcc41b0 0x558fc0095470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.955+0000 7f04afcda740 1 -- 172.21.15.114:0/4111910590 >> 172.21.15.114:0/4111910590 conn(0x558fbfce1460 msgr2=0x558fc0094d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.956+0000 7f04afcda740 1 -- 172.21.15.114:0/4111910590 shutdown_connections 2024-09-21T06:57:49.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.956+0000 7f04afcda740 1 -- 172.21.15.114:0/4111910590 wait complete. 2024-09-21T06:57:49.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.957+0000 7f04afcda740 1 -- start start 2024-09-21T06:57:49.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.957+0000 7f04afcda740 1 Processor -- start 2024-09-21T06:57:49.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.959+0000 7f04afcda740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfcc41b0 0x558fc00a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.959+0000 7f04afcda740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 0x558fc00a2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.959+0000 7f04afcda740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.960+0000 7f04a1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 0x558fc00a2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.960+0000 7f04a1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 0x558fc00a2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33532/0 (socket says 172.21.15.114:33532) 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04afcda740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0484011360 con 0x558fbfced070 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04afcda740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0484010c90 con 0x558fbfcc41b0 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04afcda740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f048400ded0 con 0x558fbfccc390 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04afcda740 10 client.0 _ll_register_callbacks cb 0x7f04900ac4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04a2800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51520/0 (socket says 172.21.15.114:51520) 2024-09-21T06:57:49.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04a2800640 1 -- 172.21.15.114:0/4042553430 learned_addr learned my addr 172.21.15.114:0/4042553430 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:49.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.961+0000 7f04a1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfcc41b0 0x558fc00a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.963+0000 7f04a1400640 1 -- 172.21.15.114:0/4042553430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfcc41b0 msgr2=0x558fc00a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.963+0000 7f04a1400640 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfcc41b0 0x558fc00a7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.963+0000 7f04a1400640 1 -- 172.21.15.114:0/4042553430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 msgr2=0x558fc00a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.963+0000 7f04a1400640 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.963+0000 7f04a1400640 1 -- 172.21.15.114:0/4042553430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc00a3330 con 0x558fbfccc390 2024-09-21T06:57:49.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.964+0000 7f04a1400640 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 0x558fc00a2750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x558fc0099b50 tx=0x7f04900a9190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee609bb3579da30a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:49.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.964+0000 7f0483e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:49.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.964+0000 7f0483e00640 1 -- 172.21.15.114:0/4042553430 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04900c6070 con 0x558fbfccc390 2024-09-21T06:57:49.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.964+0000 7f0483e00640 1 -- 172.21.15.114:0/4042553430 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f04900ac170 con 0x558fbfccc390 2024-09-21T06:57:49.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.964+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04900b1020 con 0x558fbfccc390 2024-09-21T06:57:49.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.965+0000 7f0483e00640 1 -- 172.21.15.114:0/4042553430 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0490020050 con 0x558fbfccc390 2024-09-21T06:57:49.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.965+0000 7f04afcda740 10 client.26596 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:49.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.965+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558fc011d1d0 con 0x558fbfccc390 2024-09-21T06:57:49.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.965+0000 7f0483e00640 1 -- 172.21.15.114:0/4042553430 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04900b1020 con 0x558fbfccc390 2024-09-21T06:57:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.966+0000 7f0483e00640 1 -- 172.21.15.114:0/4042553430 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f04900b11d0 con 0x558fbfccc390 2024-09-21T06:57:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.966+0000 7f04afcda740 10 client.26596 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.966+0000 7f04afcda740 10 client.26596 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:57:49.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.967+0000 7f04afcda740 20 client.26596 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.967+0000 7f04afcda740 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfc80da0 0x558fc011f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.967+0000 7f04afcda740 4 client.26596 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:57:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.967+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fc00089b0 con 0x558fbfc80da0 2024-09-21T06:57:49.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.967+0000 7f04a1e00640 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfc80da0 0x558fc011f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:49.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.968+0000 7f0483e00640 10 client.26596 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:57:49.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.968+0000 7f04a1e00640 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfc80da0 0x558fc011f6b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7b528d22e450283 server_cookie=113313ff1bff9801 in_seq=0 out_seq=0 2024-09-21T06:57:49.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.969+0000 7f0483e00640 1 -- 172.21.15.114:0/4042553430 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fc00089b0 con 0x558fbfc80da0 2024-09-21T06:57:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.969+0000 7f0483e00640 10 client.26596 handle_command_reply: tid=0 2024-09-21T06:57:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.969+0000 7f04afcda740 1 client.26596 shutdown 2024-09-21T06:57:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.969+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 msgr2=0x558fc00a2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.969+0000 7f04afcda740 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 0x558fc00a2750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x558fc0099b50 tx=0x7f04900a9190 comp rx=0 tx=0).stop 2024-09-21T06:57:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 shutdown_connections 2024-09-21T06:57:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x558fbfc80da0 0x558fc011f6b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558fbfced070 0x558fc00a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558fbfccc390 0x558fc00a2750 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 --2- 172.21.15.114:0/4042553430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558fbfcc41b0 0x558fc00a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 >> 172.21.15.114:0/4042553430 conn(0x558fbfce1460 msgr2=0x558fc00aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f0483e00640 0 client.26596 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:57:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 shutdown_connections 2024-09-21T06:57:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 1 -- 172.21.15.114:0/4042553430 wait complete. 2024-09-21T06:57:49.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.970+0000 7f04afcda740 20 client.26596 trim_cache size 0 max 16384 2024-09-21T06:57:49.973 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.971+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04880b66e0 msgr2=0x7f04880b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.971+0000 7f04a3200640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04880b66e0 0x7f04880b8bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0494004f80 tx=0x7f0494031400 comp rx=0 tx=0).stop 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.971+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 msgr2=0x7f04a41ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.971+0000 7f04a3200640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 0x7f04a41ac050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f049000d6d0 tx=0x7f049000db90 comp rx=0 tx=0).stop 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 shutdown_connections 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04a410ff90 0x7f04a41ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04880b66e0 0x7f04880b8bf0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04a4102d30 0x7f04a41ac050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 --2- 172.21.15.114:0/19319482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04a41023e0 0x7f04a41b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 >> 172.21.15.114:0/19319482 conn(0x7f04a40fe2c0 msgr2=0x7f04a4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 shutdown_connections 2024-09-21T06:57:49.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:49.972+0000 7f04a3200640 1 -- 172.21.15.114:0/19319482 wait complete. 2024-09-21T06:57:50.949 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:57:51.027 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:50 smithi114 ceph-mon[30654]: pgmap v507: 129 pgs: 129 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 564 op/s 2024-09-21T06:57:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:50 smithi183 ceph-mon[34219]: pgmap v507: 129 pgs: 129 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 564 op/s 2024-09-21T06:57:51.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:50 smithi193 ceph-mon[34601]: pgmap v507: 129 pgs: 129 active+clean; 1.8 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 564 op/s 2024-09-21T06:57:51.595 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 -- 172.21.15.114:0/3428790428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 msgr2=0x7ff23010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 --2- 172.21.15.114:0/3428790428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 0x7ff23010d8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009900 tx=0x7ff21c02a440 comp rx=0 tx=0).stop 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 -- 172.21.15.114:0/3428790428 shutdown_connections 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 --2- 172.21.15.114:0/3428790428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 0x7ff230110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 --2- 172.21.15.114:0/3428790428 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 0x7ff23010d8f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 --2- 172.21.15.114:0/3428790428 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 0x7ff230104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.638+0000 7ff235600640 1 -- 172.21.15.114:0/3428790428 >> 172.21.15.114:0/3428790428 conn(0x7ff230079730 msgr2=0x7ff230079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:52.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.639+0000 7ff235600640 1 -- 172.21.15.114:0/3428790428 shutdown_connections 2024-09-21T06:57:52.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.639+0000 7ff235600640 1 -- 172.21.15.114:0/3428790428 wait complete. 2024-09-21T06:57:52.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.644+0000 7ff235600640 1 Processor -- start 2024-09-21T06:57:52.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.644+0000 7ff235600640 1 -- start start 2024-09-21T06:57:52.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.646+0000 7ff235600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 0x7ff230102670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.646+0000 7ff235600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 0x7ff230102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.646+0000 7ff235600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 0x7ff2301030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff22e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 0x7ff230102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 0x7ff230102670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 0x7ff230102670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51544/0 (socket says 172.21.15.114:51544) 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff22ea00640 1 -- 172.21.15.114:0/2192413336 learned_addr learned my addr 172.21.15.114:0/2192413336 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff235600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff230113ba0 con 0x7ff230104880 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff22f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 0x7ff2301030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff230113a20 con 0x7ff23010de30 2024-09-21T06:57:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.647+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff230113d20 con 0x7ff2301052b0 2024-09-21T06:57:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.648+0000 7ff22f400640 1 -- 172.21.15.114:0/2192413336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 msgr2=0x7ff230102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.648+0000 7ff22f400640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 0x7ff230102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.649+0000 7ff22f400640 1 -- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 msgr2=0x7ff230102670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.649+0000 7ff22f400640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 0x7ff230102670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.649+0000 7ff22f400640 1 -- 172.21.15.114:0/2192413336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff230074140 con 0x7ff23010de30 2024-09-21T06:57:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.650+0000 7ff22f400640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 0x7ff2301030f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff22800be50 tx=0x7ff22800bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.651+0000 7ff227400640 1 -- 172.21.15.114:0/2192413336 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228019070 con 0x7ff23010de30 2024-09-21T06:57:52.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.651+0000 7ff227400640 1 -- 172.21.15.114:0/2192413336 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2280148f0 con 0x7ff23010de30 2024-09-21T06:57:52.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.651+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff230074430 con 0x7ff23010de30 2024-09-21T06:57:52.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.651+0000 7ff227400640 1 -- 172.21.15.114:0/2192413336 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff228012860 con 0x7ff23010de30 2024-09-21T06:57:52.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.651+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff230074a10 con 0x7ff23010de30 2024-09-21T06:57:52.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.656+0000 7ff227400640 1 -- 172.21.15.114:0/2192413336 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff22801c070 con 0x7ff23010de30 2024-09-21T06:57:52.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.659+0000 7ff23c3fb740 1 Processor -- start 2024-09-21T06:57:52.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.659+0000 7ff227400640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2140b67b0 0x7ff2140b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.660+0000 7ff22e000640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2140b67b0 0x7ff2140b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.661+0000 7ff23c3fb740 1 -- start start 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.661+0000 7ff227400640 1 -- 172.21.15.114:0/2192413336 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff2280e7110 con 0x7ff23010de30 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc7bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 0x557e8cc7c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557e8cbf2fb0 con 0x557e8c8b4390 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557e8cbe23e0 con 0x557e8c8d5070 2024-09-21T06:57:52.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557e8cbe61f0 con 0x557e8c8ac1b0 2024-09-21T06:57:52.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc7bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc7bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49716/0 (socket says 172.21.15.114:49716) 2024-09-21T06:57:52.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.662+0000 7ff22ea00640 1 -- 172.21.15.114:0/2176177505 learned_addr learned my addr 172.21.15.114:0/2176177505 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:52.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.663+0000 7ff22ea00640 1 -- 172.21.15.114:0/2176177505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 msgr2=0x557e8cc7c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:52.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.663+0000 7ff22ea00640 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.663+0000 7ff22ea00640 1 -- 172.21.15.114:0/2176177505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 msgr2=0x557e8cc7c820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:52.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.663+0000 7ff22ea00640 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 0x557e8cc7c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.664+0000 7ff22ea00640 1 -- 172.21.15.114:0/2176177505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8cbaccd0 con 0x557e8c8d5070 2024-09-21T06:57:52.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.665+0000 7ff22e000640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2140b67b0 0x7ff2140b8cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02aa30 tx=0x7ff21c005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:52.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.665+0000 7ff22ea00640 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc7bda0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff220009ae0 tx=0x7ff22000e1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67225fea673410c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:52.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.666+0000 7ff21b400640 1 -- 172.21.15.114:0/2176177505 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220017070 con 0x557e8c8d5070 2024-09-21T06:57:52.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.666+0000 7ff21b400640 1 -- 172.21.15.114:0/2176177505 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22000ebb0 con 0x557e8c8d5070 2024-09-21T06:57:52.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.666+0000 7ff21b400640 1 -- 172.21.15.114:0/2176177505 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220012630 con 0x557e8c8d5070 2024-09-21T06:57:52.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.667+0000 7ff23c3fb740 1 -- 172.21.15.114:0/2176177505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 msgr2=0x557e8cc7bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.667+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc7bda0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff220009ae0 tx=0x7ff22000e1d0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.668+0000 7ff23c3fb740 1 -- 172.21.15.114:0/2176177505 shutdown_connections 2024-09-21T06:57:52.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.668+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 0x557e8cc7c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.668+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.669+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/2176177505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc7bda0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.669+0000 7ff23c3fb740 1 -- 172.21.15.114:0/2176177505 >> 172.21.15.114:0/2176177505 conn(0x557e8c8c9460 msgr2=0x557e8cc79c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:52.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.669+0000 7ff23c3fb740 1 -- 172.21.15.114:0/2176177505 shutdown_connections 2024-09-21T06:57:52.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.670+0000 7ff23c3fb740 1 -- 172.21.15.114:0/2176177505 wait complete. 2024-09-21T06:57:52.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.671+0000 7ff23c3fb740 1 -- start start 2024-09-21T06:57:52.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.671+0000 7ff23c3fb740 1 Processor -- start 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.675+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.675+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 0x557e8cc87b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.675+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.675+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557e8ca43700 con 0x557e8c8b4390 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.675+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557e8cb7d580 con 0x557e8c8d5070 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.675+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557e8cc8a580 con 0x557e8c8ac1b0 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff23c3fb740 10 client.0 _ll_register_callbacks cb 0x557e8cb8d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc93870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49718/0 (socket says 172.21.15.114:49718) 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 -- 172.21.15.114:0/3333522902 learned_addr learned my addr 172.21.15.114:0/3333522902 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 -- 172.21.15.114:0/3333522902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 msgr2=0x557e8cc875b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc875b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 -- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 msgr2=0x557e8cc87b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 0x557e8cc87b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.676+0000 7ff22f400640 1 -- 172.21.15.114:0/3333522902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8c940650 con 0x557e8c8d5070 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff22f400640 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc93870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2280ad8d0 tx=0x7ff2280ad340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9ad1289dc8e042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff20fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff20fe00640 1 -- 172.21.15.114:0/3333522902 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2280e4910 con 0x557e8c8d5070 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff20fe00640 1 -- 172.21.15.114:0/3333522902 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff2280ace30 con 0x557e8c8d5070 2024-09-21T06:57:52.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e8cb7fb90 con 0x557e8c8d5070 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff20fe00640 1 -- 172.21.15.114:0/3333522902 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff2280ad6e0 con 0x557e8c8d5070 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff23c3fb740 10 client.26651 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.677+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557e8cb9a950 con 0x557e8c8d5070 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff20fe00640 1 -- 172.21.15.114:0/3333522902 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557e8cb7fb90 con 0x557e8c8d5070 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff20fe00640 1 -- 172.21.15.114:0/3333522902 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff2280b2140 con 0x557e8c8d5070 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff23c3fb740 10 client.26651 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff23c3fb740 10 client.26651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff23c3fb740 20 client.26651 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c8939f0 0x557e8ccaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff23c3fb740 4 client.26651 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:52.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.678+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e8cbf4f00 con 0x557e8c8939f0 2024-09-21T06:57:52.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.680+0000 7ff22ea00640 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c8939f0 0x557e8ccaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.681+0000 7ff22ea00640 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c8939f0 0x557e8ccaa760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee4ca6981b6bb682 server_cookie=cf6fde51351d1bae in_seq=0 out_seq=0 2024-09-21T06:57:52.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.682+0000 7ff20fe00640 10 client.26651 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:57:52.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.683+0000 7ff20fe00640 1 -- 172.21.15.114:0/3333522902 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557e8cbf4f00 con 0x557e8c8939f0 2024-09-21T06:57:52.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.683+0000 7ff20fe00640 10 client.26651 handle_command_reply: tid=0 2024-09-21T06:57:52.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.683+0000 7ff23c3fb740 1 client.26651 shutdown 2024-09-21T06:57:52.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.684+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 msgr2=0x557e8cc93870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.684+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc93870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2280ad8d0 tx=0x7ff2280ad340 comp rx=0 tx=0).stop 2024-09-21T06:57:52.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 shutdown_connections 2024-09-21T06:57:52.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c8939f0 0x557e8ccaa760 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8d5070 0x557e8cc93870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8b4390 0x557e8cc87b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3333522902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 >> 172.21.15.114:0/3333522902 conn(0x557e8c8c9460 msgr2=0x557e8cc7ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff20fe00640 0 client.26651 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 shutdown_connections 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.688+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3333522902 wait complete. 2024-09-21T06:57:52.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.689+0000 7ff23c3fb740 20 client.26651 trim_cache size 0 max 16384 2024-09-21T06:57:52.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 Processor -- start 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 -- start start 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8b4390 0x557e8cc85e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301b7820 con 0x557e8c8d5070 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301b9360 con 0x557e8c8ac1b0 2024-09-21T06:57:52.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.717+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301b7ef0 con 0x557e8c8b4390 2024-09-21T06:57:52.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.718+0000 7ff22f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.718+0000 7ff22f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc847a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:51578/0 (socket says 172.21.15.114:51578) 2024-09-21T06:57:52.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.718+0000 7ff22f400640 1 -- 172.21.15.114:0/307983972 learned_addr learned my addr 172.21.15.114:0/307983972 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.719+0000 7ff22ea00640 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.720+0000 7ff22ea00640 1 -- 172.21.15.114:0/307983972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8b4390 msgr2=0x557e8cc85e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:57:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.720+0000 7ff22ea00640 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8b4390 0x557e8cc85e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.720+0000 7ff22ea00640 1 -- 172.21.15.114:0/307983972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 msgr2=0x557e8cc847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.720+0000 7ff22ea00640 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc847a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.720+0000 7ff22ea00640 1 -- 172.21.15.114:0/307983972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8cc87ea0 con 0x557e8c8ac1b0 2024-09-21T06:57:52.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.721+0000 7ff22ea00640 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7f860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff220013de0 tx=0x7ff2200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2d16506377fae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.722+0000 7ff21aa00640 1 -- 172.21.15.114:0/307983972 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22002a070 con 0x557e8c8ac1b0 2024-09-21T06:57:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.722+0000 7ff21aa00640 1 -- 172.21.15.114:0/307983972 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff220022030 con 0x557e8c8ac1b0 2024-09-21T06:57:52.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.722+0000 7ff21aa00640 1 -- 172.21.15.114:0/307983972 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220017040 con 0x557e8c8ac1b0 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.723+0000 7ff23c3fb740 1 -- 172.21.15.114:0/307983972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8ac1b0 msgr2=0x557e8cc7f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.723+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7f860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff220013de0 tx=0x7ff2200046c0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.724+0000 7ff23c3fb740 1 -- 172.21.15.114:0/307983972 shutdown_connections 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.724+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.724+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8b4390 0x557e8cc85e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.724+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/307983972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8ac1b0 0x557e8cc7f860 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.724+0000 7ff23c3fb740 1 -- 172.21.15.114:0/307983972 >> 172.21.15.114:0/307983972 conn(0x557e8c8c9460 msgr2=0x557e8cc82760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.725+0000 7ff23c3fb740 1 -- 172.21.15.114:0/307983972 shutdown_connections 2024-09-21T06:57:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.726+0000 7ff23c3fb740 1 -- 172.21.15.114:0/307983972 wait complete. 2024-09-21T06:57:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.727+0000 7ff23c3fb740 1 -- start start 2024-09-21T06:57:52.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.727+0000 7ff23c3fb740 1 Processor -- start 2024-09-21T06:57:52.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.728+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc8b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc8b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8b4390 0x557e8cc8bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff23c3fb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc8c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301b8c90 con 0x557e8c8d5070 2024-09-21T06:57:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2301b85c0 con 0x557e8c8b4390 2024-09-21T06:57:52.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff23c3fb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff230103970 con 0x557e8c8ac1b0 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff23c3fb740 10 client.0 _ll_register_callbacks cb 0x7ff2280ad9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff22ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc8b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33588/0 (socket says 172.21.15.114:33588) 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff22ea00640 1 -- 172.21.15.114:0/3373904920 learned_addr learned my addr 172.21.15.114:0/3373904920 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff22e000640 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8b4390 0x557e8cc8bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff22ea00640 1 -- 172.21.15.114:0/3373904920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8b4390 msgr2=0x557e8cc8bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.729+0000 7ff22ea00640 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8b4390 0x557e8cc8bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.730+0000 7ff22ea00640 1 -- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 msgr2=0x557e8cc8c220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.730+0000 7ff22ea00640 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc8c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.730+0000 7ff22ea00640 1 -- 172.21.15.114:0/3373904920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8cd14160 con 0x557e8c8ac1b0 2024-09-21T06:57:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.730+0000 7ff22ea00640 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc8b7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff2200091d0 tx=0x7ff220006df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e7b91080afa6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.730+0000 7ff20fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:52.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.731+0000 7ff20fe00640 1 -- 172.21.15.114:0/3373904920 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22002a030 con 0x557e8c8ac1b0 2024-09-21T06:57:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.731+0000 7ff20fe00640 1 -- 172.21.15.114:0/3373904920 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff220013470 con 0x557e8c8ac1b0 2024-09-21T06:57:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.731+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff220022030 con 0x557e8c8ac1b0 2024-09-21T06:57:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.731+0000 7ff20fe00640 1 -- 172.21.15.114:0/3373904920 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff220017040 con 0x557e8c8ac1b0 2024-09-21T06:57:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.731+0000 7ff23c3fb740 10 client.26605 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.731+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557e8cd153e0 con 0x557e8c8ac1b0 2024-09-21T06:57:52.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff20fe00640 1 -- 172.21.15.114:0/3373904920 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff220022030 con 0x557e8c8ac1b0 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff20fe00640 1 -- 172.21.15.114:0/3373904920 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff220030410 con 0x557e8c8ac1b0 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff23c3fb740 10 client.26605 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff23c3fb740 10 client.26605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff23c3fb740 20 client.26605 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c868da0 0x557e8cd17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff23c3fb740 4 client.26605 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.732+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e8cbf09b0 con 0x557e8c868da0 2024-09-21T06:57:52.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.733+0000 7ff22e000640 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c868da0 0x557e8cd17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.734+0000 7ff20fe00640 10 client.26605 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.734+0000 7ff22e000640 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c868da0 0x557e8cd17820 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc67499d108a5299 server_cookie=63b861ff2d737f3e in_seq=0 out_seq=0 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.735+0000 7ff20fe00640 1 -- 172.21.15.114:0/3373904920 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e8cbf09b0 con 0x557e8c868da0 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.735+0000 7ff20fe00640 10 client.26605 handle_command_reply: tid=0 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.735+0000 7ff23c3fb740 1 client.26605 shutdown 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.735+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 msgr2=0x557e8cc8b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.735+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc8b7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff2200091d0 tx=0x7ff220006df0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 shutdown_connections 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x557e8c868da0 0x557e8cd17820 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557e8c8d5070 0x557e8cc8c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e8c8b4390 0x557e8cc8bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 --2- 172.21.15.114:0/3373904920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557e8c8ac1b0 0x557e8cc8b7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 >> 172.21.15.114:0/3373904920 conn(0x557e8c8c9460 msgr2=0x557e8cc82760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff20fe00640 0 client.26605 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 shutdown_connections 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.736+0000 7ff23c3fb740 1 -- 172.21.15.114:0/3373904920 wait complete. 2024-09-21T06:57:52.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.737+0000 7ff23c3fb740 20 client.26605 trim_cache size 0 max 16384 2024-09-21T06:57:52.740 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:57:52.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.738+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2140b67b0 msgr2=0x7ff2140b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.738+0000 7ff235600640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2140b67b0 0x7ff2140b8cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff21c02aa30 tx=0x7ff21c005fd0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.738+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 msgr2=0x7ff2301030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:52.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.738+0000 7ff235600640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 0x7ff2301030f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff22800be50 tx=0x7ff22800bf40 comp rx=0 tx=0).stop 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 shutdown_connections 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2140b67b0 0x7ff2140b8cc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff23010de30 0x7ff2301030f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2301052b0 0x7ff230102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 --2- 172.21.15.114:0/2192413336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff230104880 0x7ff230102670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 >> 172.21.15.114:0/2192413336 conn(0x7ff230079730 msgr2=0x7ff230100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 shutdown_connections 2024-09-21T06:57:52.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:52.739+0000 7ff235600640 1 -- 172.21.15.114:0/2192413336 wait complete. 2024-09-21T06:57:52.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 195.0 s, 1397.2 tps, lat 70.767 ms stddev 92.110 2024-09-21T06:57:53.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:52 smithi114 ceph-mon[30654]: pgmap v508: 129 pgs: 129 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-09-21T06:57:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:52 smithi183 ceph-mon[34219]: pgmap v508: 129 pgs: 129 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-09-21T06:57:53.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:52 smithi193 ceph-mon[34601]: pgmap v508: 129 pgs: 129 active+clean; 1.8 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.4 MiB/s wr, 511 op/s 2024-09-21T06:57:53.773 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:57:54.399 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:54.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:54 smithi114 ceph-mon[30654]: pgmap v509: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.7 MiB/s wr, 724 op/s 2024-09-21T06:57:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:54 smithi183 ceph-mon[34219]: pgmap v509: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.7 MiB/s wr, 724 op/s 2024-09-21T06:57:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:54 smithi193 ceph-mon[34601]: pgmap v509: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.7 MiB/s wr, 724 op/s 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.354+0000 7f286c800640 1 -- 172.21.15.114:0/2709618526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 msgr2=0x7f28640757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.354+0000 7f286c800640 1 --2- 172.21.15.114:0/2709618526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28640757b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2858009900 tx=0x7f285802a410 comp rx=0 tx=0).stop 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 -- 172.21.15.114:0/2709618526 shutdown_connections 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 --2- 172.21.15.114:0/2709618526 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864114440 0x7f2864116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 --2- 172.21.15.114:0/2709618526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2864075dd0 0x7f28640761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 --2- 172.21.15.114:0/2709618526 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28640757b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 -- 172.21.15.114:0/2709618526 >> 172.21.15.114:0/2709618526 conn(0x7f28640ff5e0 msgr2=0x7f2864101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 -- 172.21.15.114:0/2709618526 shutdown_connections 2024-09-21T06:57:55.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.355+0000 7f286c800640 1 -- 172.21.15.114:0/2709618526 wait complete. 2024-09-21T06:57:55.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.356+0000 7f286c800640 1 Processor -- start 2024-09-21T06:57:55.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.356+0000 7f286c800640 1 -- start start 2024-09-21T06:57:55.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f286c800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864075dd0 0x7f28641a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f286c800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28641a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f286c800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2864114440 0x7f28641a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f286c800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f286411a150 con 0x7f2864077380 2024-09-21T06:57:55.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f286c800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2864119fd0 con 0x7f2864114440 2024-09-21T06:57:55.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f286c800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f286411a2d0 con 0x7f2864075dd0 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f2869400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28641a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.357+0000 7f2869e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864075dd0 0x7f28641a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864075dd0 0x7f28641a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52358/0 (socket says 172.21.15.114:52358) 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869e00640 1 -- 172.21.15.114:0/160253322 learned_addr learned my addr 172.21.15.114:0/160253322 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28641a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44406/0 (socket says 172.21.15.114:44406) 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f286a800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2864114440 0x7f28641a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869400640 1 -- 172.21.15.114:0/160253322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864075dd0 msgr2=0x7f28641a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869400640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864075dd0 0x7f28641a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869400640 1 -- 172.21.15.114:0/160253322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2864114440 msgr2=0x7f28641a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869400640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2864114440 0x7f28641a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.358+0000 7f2869400640 1 -- 172.21.15.114:0/160253322 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28641a6af0 con 0x7f2864077380 2024-09-21T06:57:55.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.359+0000 7f2869400640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28641a1120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f285000e9c0 tx=0x7f285000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:55.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.359+0000 7f285ea00640 1 -- 172.21.15.114:0/160253322 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285000cd80 con 0x7f2864077380 2024-09-21T06:57:55.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.359+0000 7f285ea00640 1 -- 172.21.15.114:0/160253322 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28500035f0 con 0x7f2864077380 2024-09-21T06:57:55.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.359+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28641a6de0 con 0x7f2864077380 2024-09-21T06:57:55.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.359+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28641ad540 con 0x7f2864077380 2024-09-21T06:57:55.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.360+0000 7f285ea00640 1 -- 172.21.15.114:0/160253322 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850005020 con 0x7f2864077380 2024-09-21T06:57:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.367+0000 7f287348e740 1 Processor -- start 2024-09-21T06:57:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.367+0000 7f285ea00640 1 -- 172.21.15.114:0/160253322 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f28500037a0 con 0x7f2864077380 2024-09-21T06:57:55.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.368+0000 7f287348e740 1 -- start start 2024-09-21T06:57:55.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.369+0000 7f287348e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d104eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.369+0000 7f287348e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.369+0000 7f287348e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c87390 0x55d1d104f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.369+0000 7f287348e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d1d0fc5fb0 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.369+0000 7f287348e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1d0fb53e0 con 0x55d1d0ca8070 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.369+0000 7f287348e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d1d0fb91f0 con 0x55d1d0c87390 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.370+0000 7f286a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d104eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d104eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54412/0 (socket says 172.21.15.114:54412) 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 -- 172.21.15.114:0/3188010163 learned_addr learned my addr 172.21.15.114:0/3188010163 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 -- 172.21.15.114:0/3188010163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c87390 msgr2=0x55d1d104f820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c87390 0x55d1d104f820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 -- 172.21.15.114:0/3188010163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 msgr2=0x55d1d104f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:55.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.371+0000 7f286a800640 1 -- 172.21.15.114:0/3188010163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d0f7fcd0 con 0x55d1d0ca8070 2024-09-21T06:57:55.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.374+0000 7f285ea00640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f284c0b6700 0x7f284c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.375+0000 7f286a800640 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d104eda0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f286000a4d0 tx=0x7f2860009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa606ad7b6b158b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:55.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.375+0000 7f2869e00640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f284c0b6700 0x7f284c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.375+0000 7f285ea00640 1 -- 172.21.15.114:0/160253322 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f28500e7d70 con 0x7f2864077380 2024-09-21T06:57:55.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.375+0000 7f2857000640 1 -- 172.21.15.114:0/3188010163 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860029020 con 0x55d1d0ca8070 2024-09-21T06:57:55.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.375+0000 7f2857000640 1 -- 172.21.15.114:0/3188010163 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2860025cd0 con 0x55d1d0ca8070 2024-09-21T06:57:55.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.375+0000 7f2857000640 1 -- 172.21.15.114:0/3188010163 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860029020 con 0x55d1d0ca8070 2024-09-21T06:57:55.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 -- 172.21.15.114:0/3188010163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 msgr2=0x55d1d104eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d104eda0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f286000a4d0 tx=0x7f2860009650 comp rx=0 tx=0).stop 2024-09-21T06:57:55.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 -- 172.21.15.114:0/3188010163 shutdown_connections 2024-09-21T06:57:55.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c87390 0x55d1d104f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 --2- 172.21.15.114:0/3188010163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d104eda0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.377+0000 7f287348e740 1 -- 172.21.15.114:0/3188010163 >> 172.21.15.114:0/3188010163 conn(0x55d1d0c9c460 msgr2=0x55d1d104cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:55.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.378+0000 7f287348e740 1 -- 172.21.15.114:0/3188010163 shutdown_connections 2024-09-21T06:57:55.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.378+0000 7f287348e740 1 -- 172.21.15.114:0/3188010163 wait complete. 2024-09-21T06:57:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.381+0000 7f2869e00640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f284c0b6700 0x7f284c0b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d1d104da20 tx=0x7f2858005d90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.381+0000 7f287348e740 1 -- start start 2024-09-21T06:57:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.382+0000 7f287348e740 1 Processor -- start 2024-09-21T06:57:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.382+0000 7f287348e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.383+0000 7f287348e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c87390 0x55d1d105adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.383+0000 7f2869e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.383+0000 7f2869e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52374/0 (socket says 172.21.15.114:52374) 2024-09-21T06:57:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.383+0000 7f2869e00640 1 -- 172.21.15.114:0/984842270 learned_addr learned my addr 172.21.15.114:0/984842270 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.383+0000 7f2869400640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c87390 0x55d1d105adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.384+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d1066870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.384+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d1d0e16700 con 0x55d1d0c87390 2024-09-21T06:57:55.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.384+0000 7f286a800640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d1066870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.384+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1d0f50580 con 0x55d1d0ca8070 2024-09-21T06:57:55.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.385+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d1d105d580 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.386+0000 7f2869400640 1 -- 172.21.15.114:0/984842270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c7f1b0 msgr2=0x55d1d105a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:55.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.386+0000 7f2869400640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.386+0000 7f2869400640 1 -- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 msgr2=0x55d1d1066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:55.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.386+0000 7f2869400640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d1066870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.386+0000 7f286a800640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d1066870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:57:55.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.386+0000 7f2869400640 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d0d13650 con 0x55d1d0c87390 2024-09-21T06:57:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.387+0000 7f2869400640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c87390 0x55d1d105adc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f28500e50f0 tx=0x7f28500a9d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f28fe347befc0e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:55.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.387+0000 7f284b400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:57:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.387+0000 7f287348e740 10 client.0 _ll_register_callbacks cb 0x55d1d0f60470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.388+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1d0f52b90 con 0x55d1d0c87390 2024-09-21T06:57:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.388+0000 7f284b400640 1 -- 172.21.15.114:0/984842270 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500d2070 con 0x55d1d0c87390 2024-09-21T06:57:55.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.389+0000 7f284b400640 1 -- 172.21.15.114:0/984842270 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f28500aeb20 con 0x55d1d0c87390 2024-09-21T06:57:55.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.389+0000 7f284b400640 1 -- 172.21.15.114:0/984842270 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f28500e5580 con 0x55d1d0c87390 2024-09-21T06:57:55.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.389+0000 7f287348e740 10 client.16500 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:55.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.389+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({fsmap=35}) -- 0x55d1d0f6d950 con 0x55d1d0c87390 2024-09-21T06:57:55.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.390+0000 7f284b400640 1 -- 172.21.15.114:0/984842270 <== mon.0 v2:172.21.15.114:3300/0 4 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f28500a9140 con 0x55d1d0c87390 2024-09-21T06:57:55.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.390+0000 7f287348e740 10 client.16500 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:55.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.390+0000 7f287348e740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:55.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.390+0000 7f287348e740 20 client.16500 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:55.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.390+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d0c669f0 0x55d1d106c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.391+0000 7f287348e740 4 client.16500 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:55.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.391+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1d0f52b90 con 0x55d1d0c669f0 2024-09-21T06:57:55.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.391+0000 7f2869e00640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d0c669f0 0x55d1d106c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.392+0000 7f2869e00640 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d0c669f0 0x55d1d106c430 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca99168f66cf5449 server_cookie=364406c4ff217ee8 in_seq=0 out_seq=0 2024-09-21T06:57:55.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.392+0000 7f284b400640 10 client.16500 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:55.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.394+0000 7f284b400640 1 -- 172.21.15.114:0/984842270 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1d0f52b90 con 0x55d1d0c669f0 2024-09-21T06:57:55.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.394+0000 7f284b400640 10 client.16500 handle_command_reply: tid=0 2024-09-21T06:57:55.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.394+0000 7f287348e740 1 client.16500 shutdown 2024-09-21T06:57:55.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c87390 msgr2=0x55d1d105adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c87390 0x55d1d105adc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f28500e50f0 tx=0x7f28500a9d10 comp rx=0 tx=0).stop 2024-09-21T06:57:55.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 shutdown_connections 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0ca8070 0x55d1d1066870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c87390 0x55d1d105adc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d0c669f0 0x55d1d106c430 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 --2- 172.21.15.114:0/984842270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 >> 172.21.15.114:0/984842270 conn(0x55d1d0c9c460 msgr2=0x55d1d1051f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.395+0000 7f284b400640 0 client.16500 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.396+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 shutdown_connections 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.396+0000 7f287348e740 1 -- 172.21.15.114:0/984842270 wait complete. 2024-09-21T06:57:55.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.396+0000 7f287348e740 20 client.16500 trim_cache size 0 max 16384 2024-09-21T06:57:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.423+0000 7f287348e740 1 Processor -- start 2024-09-21T06:57:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.423+0000 7f287348e740 1 -- start start 2024-09-21T06:57:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f287348e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f287348e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d1050490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f287348e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c3bda0 0x55d1d1066770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f287348e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28641aee10 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f287348e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28641b0950 con 0x55d1d0c3bda0 2024-09-21T06:57:55.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f287348e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f28641af4e0 con 0x55d1d0ca8070 2024-09-21T06:57:55.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d1050490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d1050490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52388/0 (socket says 172.21.15.114:52388) 2024-09-21T06:57:55.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869400640 1 -- 172.21.15.114:0/3466446749 learned_addr learned my addr 172.21.15.114:0/3466446749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869e00640 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869e00640 1 -- 172.21.15.114:0/3466446749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 msgr2=0x55d1d1050490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869e00640 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d1050490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869e00640 1 -- 172.21.15.114:0/3466446749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c3bda0 msgr2=0x55d1d1066770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.424+0000 7f2869e00640 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c3bda0 0x55d1d1066770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.425+0000 7f2869e00640 1 -- 172.21.15.114:0/3466446749 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1d10512e0 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.425+0000 7f2869e00640 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104ce60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55d1d106cba0 tx=0x7f2858040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ab6f6a47920d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.425+0000 7f2856600640 1 -- 172.21.15.114:0/3466446749 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858035040 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.425+0000 7f2856600640 1 -- 172.21.15.114:0/3466446749 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2858057070 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.425+0000 7f2856600640 1 -- 172.21.15.114:0/3466446749 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858035040 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.426+0000 7f287348e740 1 -- 172.21.15.114:0/3466446749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 msgr2=0x55d1d104ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.426+0000 7f287348e740 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104ce60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55d1d106cba0 tx=0x7f2858040000 comp rx=0 tx=0).stop 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 -- 172.21.15.114:0/3466446749 shutdown_connections 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c3bda0 0x55d1d1066770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d1050490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 --2- 172.21.15.114:0/3466446749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c7f1b0 0x55d1d104ce60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 -- 172.21.15.114:0/3466446749 >> 172.21.15.114:0/3466446749 conn(0x55d1d0c9c460 msgr2=0x55d1d104fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 -- 172.21.15.114:0/3466446749 shutdown_connections 2024-09-21T06:57:55.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.427+0000 7f287348e740 1 -- 172.21.15.114:0/3466446749 wait complete. 2024-09-21T06:57:55.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.428+0000 7f287348e740 1 -- start start 2024-09-21T06:57:55.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.429+0000 7f287348e740 1 Processor -- start 2024-09-21T06:57:55.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c3bda0 0x55d1d1062f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d105f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f28641b0280 con 0x55d1d0c3bda0 2024-09-21T06:57:55.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28641afbb0 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2864074bb0 con 0x55d1d0ca8070 2024-09-21T06:57:55.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f287348e740 10 client.0 _ll_register_callbacks cb 0x7f28500ad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f2869e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c3bda0 0x55d1d1062f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.430+0000 7f2869400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54432/0 (socket says 172.21.15.114:54432) 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 -- 172.21.15.114:0/678584737 learned_addr learned my addr 172.21.15.114:0/678584737 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f286a800640 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d105f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 -- 172.21.15.114:0/678584737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 msgr2=0x55d1d105f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d105f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 -- 172.21.15.114:0/678584737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c3bda0 msgr2=0x55d1d1062f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c3bda0 0x55d1d1062f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 -- 172.21.15.114:0/678584737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1d10600b0 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f2869400640 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105f430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f28500ad0d0 tx=0x7f28500e7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f67623cfb2e67c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:55.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.431+0000 7f284b400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:55.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.432+0000 7f284b400640 1 -- 172.21.15.114:0/678584737 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28500b22d0 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.432+0000 7f284b400640 1 -- 172.21.15.114:0/678584737 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f28500ac4c0 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.432+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2858057070 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.432+0000 7f284b400640 1 -- 172.21.15.114:0/678584737 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f28500b7210 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.432+0000 7f287348e740 10 client.26672 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.432+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d1d10e8d40 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f284b400640 1 -- 172.21.15.114:0/678584737 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2858057070 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f284b400640 1 -- 172.21.15.114:0/678584737 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f28500cf140 con 0x55d1d0c7f1b0 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f287348e740 10 client.26672 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f287348e740 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:55.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f287348e740 20 client.26672 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:55.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f287348e740 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d10e9460 0x55d1d10eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:55.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f287348e740 4 client.26672 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:57:55.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1d0fc7f00 con 0x55d1d10e9460 2024-09-21T06:57:55.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.433+0000 7f2869e00640 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d10e9460 0x55d1d10eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.435+0000 7f284b400640 10 client.26672 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.435+0000 7f2869e00640 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d10e9460 0x55d1d10eb880 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ccb70f7ad9e5159 server_cookie=74c1a97059b14c07 in_seq=0 out_seq=0 2024-09-21T06:57:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.435+0000 7f284b400640 1 -- 172.21.15.114:0/678584737 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1d0fc7f00 con 0x55d1d10e9460 2024-09-21T06:57:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.435+0000 7f284b400640 10 client.26672 handle_command_reply: tid=0 2024-09-21T06:57:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.435+0000 7f287348e740 1 client.26672 shutdown 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 msgr2=0x55d1d105f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105f430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f28500ad0d0 tx=0x7f28500e7000 comp rx=0 tx=0).stop 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 shutdown_connections 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d1d10e9460 0x55d1d10eb880 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d1d0ca8070 0x55d1d105f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1d0c7f1b0 0x55d1d105f430 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 --2- 172.21.15.114:0/678584737 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d1d0c3bda0 0x55d1d1062f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 >> 172.21.15.114:0/678584737 conn(0x55d1d0c9c460 msgr2=0x55d1d1050740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f284b400640 0 client.26672 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 shutdown_connections 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.436+0000 7f287348e740 1 -- 172.21.15.114:0/678584737 wait complete. 2024-09-21T06:57:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.437+0000 7f287348e740 20 client.26672 trim_cache size 0 max 16384 2024-09-21T06:57:55.440 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:57:55.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f284c0b6700 msgr2=0x7f284c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f284c0b6700 0x7f284c0b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55d1d104da20 tx=0x7f2858005d90 comp rx=0 tx=0).stop 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 msgr2=0x7f28641a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28641a1120 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f285000e9c0 tx=0x7f285000ee80 comp rx=0 tx=0).stop 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 shutdown_connections 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f284c0b6700 0x7f284c0b8c10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2864114440 0x7f28641a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2864077380 0x7f28641a1120 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 --2- 172.21.15.114:0/160253322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2864075dd0 0x7f28641a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.438+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 >> 172.21.15.114:0/160253322 conn(0x7f28640ff5e0 msgr2=0x7f2864110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.439+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 shutdown_connections 2024-09-21T06:57:55.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:55.439+0000 7f286c800640 1 -- 172.21.15.114:0/160253322 wait complete. 2024-09-21T06:57:56.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:56 smithi114 ceph-mon[30654]: pgmap v510: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-09-21T06:57:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:56 smithi183 ceph-mon[34219]: pgmap v510: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-09-21T06:57:57.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:56 smithi193 ceph-mon[34601]: pgmap v510: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.2 MiB/s wr, 551 op/s 2024-09-21T06:57:57.594 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:57:57.594 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:57:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 200.0 s, 1079.5 tps, lat 91.536 ms stddev 103.721 2024-09-21T06:57:58.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:57 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:57:58.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:57 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:57:58.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:57 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:57:58.304 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:57:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:58 smithi114 ceph-mon[30654]: pgmap v511: 129 pgs: 129 active+clean; 1.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 667 op/s 2024-09-21T06:57:59.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:58 smithi183 ceph-mon[34219]: pgmap v511: 129 pgs: 129 active+clean; 1.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 667 op/s 2024-09-21T06:57:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:58 smithi193 ceph-mon[34601]: pgmap v511: 129 pgs: 129 active+clean; 1.8 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.1 MiB/s wr, 667 op/s 2024-09-21T06:57:59.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.362+0000 7febc8200640 1 -- 172.21.15.114:0/3042818929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc0075dd0 msgr2=0x7febc00761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.362+0000 7febc8200640 1 --2- 172.21.15.114:0/3042818929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc0075dd0 0x7febc00761e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7febac009860 tx=0x7febac02a360 comp rx=0 tx=0).stop 2024-09-21T06:57:59.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.363+0000 7febc8200640 1 -- 172.21.15.114:0/3042818929 shutdown_connections 2024-09-21T06:57:59.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.363+0000 7febc8200640 1 --2- 172.21.15.114:0/3042818929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0114440 0x7febc0116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.363+0000 7febc8200640 1 --2- 172.21.15.114:0/3042818929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc0075dd0 0x7febc00761e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.363+0000 7febc8200640 1 --2- 172.21.15.114:0/3042818929 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febc0077380 0x7febc00757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.363+0000 7febc8200640 1 -- 172.21.15.114:0/3042818929 >> 172.21.15.114:0/3042818929 conn(0x7febc00ff5e0 msgr2=0x7febc0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:59.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.364+0000 7febc8200640 1 -- 172.21.15.114:0/3042818929 shutdown_connections 2024-09-21T06:57:59.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.364+0000 7febc8200640 1 -- 172.21.15.114:0/3042818929 wait complete. 2024-09-21T06:57:59.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.365+0000 7febc8200640 1 Processor -- start 2024-09-21T06:57:59.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.365+0000 7febc8200640 1 -- start start 2024-09-21T06:57:59.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc8200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 0x7febc01a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc8200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febc0114440 0x7febc01a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc8200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc01a6410 0x7febc01a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc8200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febc011a0c0 con 0x7febc0114440 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc8200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febc0119f40 con 0x7febc0077380 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc8200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febc011a240 con 0x7febc01a6410 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 0x7febc01a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 0x7febc01a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54448/0 (socket says 172.21.15.114:54448) 2024-09-21T06:57:59.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc5800640 1 -- 172.21.15.114:0/142494314 learned_addr learned my addr 172.21.15.114:0/142494314 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:59.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.366+0000 7febc6200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc01a6410 0x7febc01a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc5800640 1 -- 172.21.15.114:0/142494314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc01a6410 msgr2=0x7febc01a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc5800640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc01a6410 0x7febc01a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc5800640 1 -- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febc0114440 msgr2=0x7febc01a1230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc5800640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febc0114440 0x7febc01a1230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc5800640 1 -- 172.21.15.114:0/142494314 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc01a6ff0 con 0x7febc0077380 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc5800640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 0x7febc01a0c90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febb400b4a0 tx=0x7febb400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febbe000640 1 -- 172.21.15.114:0/142494314 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4014070 con 0x7febc0077380 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc01ad590 con 0x7febc0077380 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febbe000640 1 -- 172.21.15.114:0/142494314 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb4010ac0 con 0x7febc0077380 2024-09-21T06:57:59.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.367+0000 7febbe000640 1 -- 172.21.15.114:0/142494314 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4017c30 con 0x7febc0077380 2024-09-21T06:57:59.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.368+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc01ad9e0 con 0x7febc0077380 2024-09-21T06:57:59.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.373+0000 7febbe000640 1 -- 172.21.15.114:0/142494314 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7febb4017070 con 0x7febc0077380 2024-09-21T06:57:59.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.373+0000 7febceec9740 1 Processor -- start 2024-09-21T06:57:59.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.374+0000 7febceec9740 1 -- start start 2024-09-21T06:57:59.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.374+0000 7febceec9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c945d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.374+0000 7febceec9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c946250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.374+0000 7febceec9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c946790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.374+0000 7febceec9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55837c8bbe90 con 0x55837c575490 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.374+0000 7febceec9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55837c8ab2c0 con 0x55837c57d670 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febceec9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55837c8af0d0 con 0x55837c59e350 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c945d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c945d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52420/0 (socket says 172.21.15.114:52420) 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 -- 172.21.15.114:0/3770864151 learned_addr learned my addr 172.21.15.114:0/3770864151 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc4e00640 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c946790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 -- 172.21.15.114:0/3770864151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 msgr2=0x55837c946790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c946790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 -- 172.21.15.114:0/3770864151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 msgr2=0x55837c946250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c946250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc6200640 1 -- 172.21.15.114:0/3770864151 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837c609930 con 0x55837c59e350 2024-09-21T06:57:59.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.375+0000 7febc4e00640 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c946790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febbe000640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba80b6780 0x7feba80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febc6200640 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c945d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7febb8009f60 tx=0x7febb8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=866d439bc53072e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:59.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febb2a00640 1 -- 172.21.15.114:0/3770864151 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb801d070 con 0x55837c59e350 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febbe000640 1 -- 172.21.15.114:0/142494314 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7febb40e70d0 con 0x7febc0077380 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febb2a00640 1 -- 172.21.15.114:0/3770864151 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb800daa0 con 0x55837c59e350 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febb2a00640 1 -- 172.21.15.114:0/3770864151 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8018c00 con 0x55837c59e350 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.376+0000 7febc5800640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba80b6780 0x7feba80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 -- 172.21.15.114:0/3770864151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 msgr2=0x55837c945d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c945d10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7febb8009f60 tx=0x7febb8009f90 comp rx=0 tx=0).stop 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 -- 172.21.15.114:0/3770864151 shutdown_connections 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c946790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c946250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 --2- 172.21.15.114:0/3770864151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c945d10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.377+0000 7febceec9740 1 -- 172.21.15.114:0/3770864151 >> 172.21.15.114:0/3770864151 conn(0x55837c592740 msgr2=0x55837c943c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.378+0000 7febceec9740 1 -- 172.21.15.114:0/3770864151 shutdown_connections 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.378+0000 7febc5800640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba80b6780 0x7feba80b8c90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55837c944990 tx=0x7febb4010f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:59.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.378+0000 7febceec9740 1 -- 172.21.15.114:0/3770864151 wait complete. 2024-09-21T06:57:59.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.378+0000 7febceec9740 1 -- start start 2024-09-21T06:57:59.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 Processor -- start 2024-09-21T06:57:59.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c9518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c951df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c96e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55837c70c890 con 0x55837c575490 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55837c846160 con 0x55837c57d670 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55837c965500 con 0x55837c59e350 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.379+0000 7febceec9740 10 client.0 _ll_register_callbacks cb 0x55837c862a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c951df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c951df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54474/0 (socket says 172.21.15.114:54474) 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 -- 172.21.15.114:0/1557382736 learned_addr learned my addr 172.21.15.114:0/1557382736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:59.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc6200640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c9518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc6200640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c96e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 -- 172.21.15.114:0/1557382736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 msgr2=0x55837c96e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c96e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 -- 172.21.15.114:0/1557382736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 msgr2=0x55837c9518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c9518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.380+0000 7febc4e00640 1 -- 172.21.15.114:0/1557382736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837c96ed60 con 0x55837c57d670 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.381+0000 7febc6200640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c96e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.381+0000 7febc4e00640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c951df0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febac035040 tx=0x7febac004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8328cbf1ca7c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.381+0000 7feba7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.381+0000 7feba7400640 1 -- 172.21.15.114:0/1557382736 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac040020 con 0x55837c57d670 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.381+0000 7feba7400640 1 -- 172.21.15.114:0/1557382736 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7febac005040 con 0x55837c57d670 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.381+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb800daa0 con 0x55837c57d670 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.382+0000 7feba7400640 1 -- 172.21.15.114:0/1557382736 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7febac04b050 con 0x55837c57d670 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.382+0000 7febceec9740 10 client.26681 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:59.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.382+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55837c971600 con 0x55837c57d670 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.382+0000 7feba7400640 1 -- 172.21.15.114:0/1557382736 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb800daa0 con 0x55837c57d670 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7feba7400640 1 -- 172.21.15.114:0/1557382736 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7febac0035f0 con 0x55837c57d670 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7febceec9740 10 client.26681 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7febceec9740 10 client.26681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7febceec9740 20 client.26681 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7febceec9740 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c55ccd0 0x55837c974b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7febceec9740 4 client.26681 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:57:59.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.383+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837c8bdde0 con 0x55837c55ccd0 2024-09-21T06:57:59.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.384+0000 7febc6200640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c55ccd0 0x55837c974b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.386+0000 7feba7400640 10 client.26681 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:59.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.386+0000 7febc6200640 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c55ccd0 0x55837c974b90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7848fe389f326b1b server_cookie=ef6fb014dd676ccf in_seq=0 out_seq=0 2024-09-21T06:57:59.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.387+0000 7feba7400640 1 -- 172.21.15.114:0/1557382736 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55837c8bdde0 con 0x55837c55ccd0 2024-09-21T06:57:59.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.387+0000 7feba7400640 10 client.26681 handle_command_reply: tid=0 2024-09-21T06:57:59.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.387+0000 7febceec9740 1 client.26681 shutdown 2024-09-21T06:57:59.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.387+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 msgr2=0x55837c951df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.387+0000 7febceec9740 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c951df0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febac035040 tx=0x7febac004270 comp rx=0 tx=0).stop 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 shutdown_connections 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7febceec9740 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c55ccd0 0x55837c974b90 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7febceec9740 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c59e350 0x55837c96e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7febceec9740 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c57d670 0x55837c951df0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7febceec9740 1 --2- 172.21.15.114:0/1557382736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c575490 0x55837c9518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 >> 172.21.15.114:0/1557382736 conn(0x55837c592740 msgr2=0x55837c948ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:59.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.388+0000 7feba7400640 0 client.26681 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.389+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 shutdown_connections 2024-09-21T06:57:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.390+0000 7febceec9740 1 -- 172.21.15.114:0/1557382736 wait complete. 2024-09-21T06:57:59.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.390+0000 7febceec9740 20 client.26681 trim_cache size 0 max 16384 2024-09-21T06:57:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.430+0000 7febceec9740 1 Processor -- start 2024-09-21T06:57:59.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.430+0000 7febceec9740 1 -- start start 2024-09-21T06:57:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.431+0000 7febceec9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c575490 0x55837c96e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.431+0000 7febceec9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c57d670 0x55837c943c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.431+0000 7febceec9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 0x55837c950f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.431+0000 7febceec9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febc01af2d0 con 0x55837c57d670 2024-09-21T06:57:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.431+0000 7febceec9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b0e10 con 0x55837c59e350 2024-09-21T06:57:59.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.431+0000 7febceec9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febc01af9a0 con 0x55837c575490 2024-09-21T06:57:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.432+0000 7febc6200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 0x55837c950f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.432+0000 7febc6200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 0x55837c950f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54476/0 (socket says 172.21.15.114:54476) 2024-09-21T06:57:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.432+0000 7febc6200640 1 -- 172.21.15.114:0/3232073990 learned_addr learned my addr 172.21.15.114:0/3232073990 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:59.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.433+0000 7febc4e00640 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c57d670 0x55837c943c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.435+0000 7febc6200640 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c575490 0x55837c96e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.435+0000 7febc6200640 1 -- 172.21.15.114:0/3232073990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c575490 msgr2=0x55837c96e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.435+0000 7febc6200640 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c575490 0x55837c96e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.435+0000 7febc6200640 1 -- 172.21.15.114:0/3232073990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c57d670 msgr2=0x55837c943c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.435+0000 7febc6200640 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c57d670 0x55837c943c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.435+0000 7febc6200640 1 -- 172.21.15.114:0/3232073990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837c9449c0 con 0x55837c59e350 2024-09-21T06:57:59.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.438+0000 7febc6200640 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 0x55837c950f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7febb800ffd0 tx=0x7febb80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d087496d81baa24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:59.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.439+0000 7febb2000640 1 -- 172.21.15.114:0/3232073990 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb801d040 con 0x55837c59e350 2024-09-21T06:57:59.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.439+0000 7febb2000640 1 -- 172.21.15.114:0/3232073990 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb801e040 con 0x55837c59e350 2024-09-21T06:57:59.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.439+0000 7febb2000640 1 -- 172.21.15.114:0/3232073990 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8018b30 con 0x55837c59e350 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 -- 172.21.15.114:0/3232073990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 msgr2=0x55837c950f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 0x55837c950f40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7febb800ffd0 tx=0x7febb80095c0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 -- 172.21.15.114:0/3232073990 shutdown_connections 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c59e350 0x55837c950f40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c57d670 0x55837c943c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 --2- 172.21.15.114:0/3232073990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c575490 0x55837c96e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 -- 172.21.15.114:0/3232073990 >> 172.21.15.114:0/3232073990 conn(0x55837c592740 msgr2=0x55837c943540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:59.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 -- 172.21.15.114:0/3232073990 shutdown_connections 2024-09-21T06:57:59.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.440+0000 7febceec9740 1 -- 172.21.15.114:0/3232073990 wait complete. 2024-09-21T06:57:59.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.441+0000 7febceec9740 1 -- start start 2024-09-21T06:57:59.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.441+0000 7febceec9740 1 Processor -- start 2024-09-21T06:57:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c575490 0x55837c96aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 0x55837c979f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c59e350 0x55837c97a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b0740 con 0x55837c59e350 2024-09-21T06:57:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febc01b0070 con 0x55837c575490 2024-09-21T06:57:59.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7febc0074bc0 con 0x55837c57d670 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.442+0000 7febceec9740 10 client.0 _ll_register_callbacks cb 0x7febac035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.443+0000 7febc4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 0x55837c979f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.443+0000 7febc4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 0x55837c979f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52452/0 (socket says 172.21.15.114:52452) 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.443+0000 7febc4e00640 1 -- 172.21.15.114:0/511223021 learned_addr learned my addr 172.21.15.114:0/511223021 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.443+0000 7febc6200640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c575490 0x55837c96aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.443+0000 7febc6200640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c59e350 0x55837c97a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.444+0000 7febc4e00640 1 -- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c575490 msgr2=0x55837c96aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.444+0000 7febc4e00640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c575490 0x55837c96aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.444+0000 7febc4e00640 1 -- 172.21.15.114:0/511223021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c59e350 msgr2=0x55837c97a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:57:59.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.444+0000 7febc4e00640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c59e350 0x55837c97a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.445+0000 7febc4e00640 1 -- 172.21.15.114:0/511223021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837c97ab60 con 0x55837c57d670 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.445+0000 7febc6200640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c59e350 0x55837c97a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.446+0000 7febc4e00640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 0x55837c979f10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7febac0095f0 tx=0x7febac003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48653838b90b4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.446+0000 7feba7400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.446+0000 7feba7400640 1 -- 172.21.15.114:0/511223021 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac040030 con 0x55837c57d670 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.446+0000 7feba7400640 1 -- 172.21.15.114:0/511223021 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7febac005040 con 0x55837c57d670 2024-09-21T06:57:59.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.447+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb801e040 con 0x55837c57d670 2024-09-21T06:57:59.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.448+0000 7feba7400640 1 -- 172.21.15.114:0/511223021 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7febac056050 con 0x55837c57d670 2024-09-21T06:57:59.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.449+0000 7febceec9740 10 client.26632 fetch_fsmap learned FSMap version 35 2024-09-21T06:57:59.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.449+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55837c9df520 con 0x55837c57d670 2024-09-21T06:57:59.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7feba7400640 1 -- 172.21.15.114:0/511223021 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb801e040 con 0x55837c57d670 2024-09-21T06:57:59.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7feba7400640 1 -- 172.21.15.114:0/511223021 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7febac004b20 con 0x55837c57d670 2024-09-21T06:57:59.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7febceec9740 10 client.26632 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:57:59.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7febceec9740 10 client.26632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:57:59.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7febceec9740 20 client.26632 populate_metadata read hostname 'smithi114' 2024-09-21T06:57:59.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7febceec9740 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c532080 0x55837c9e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:57:59.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7febceec9740 4 client.26632 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:57:59.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.451+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55837c8b9890 con 0x55837c532080 2024-09-21T06:57:59.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.453+0000 7febc6200640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c532080 0x55837c9e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:57:59.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.455+0000 7feba7400640 10 client.26632 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:59.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.455+0000 7febc6200640 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c532080 0x55837c9e1c50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c750c998f486ee2a server_cookie=94b36b294bb59a92 in_seq=0 out_seq=0 2024-09-21T06:57:59.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.458+0000 7feba7400640 1 -- 172.21.15.114:0/511223021 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55837c8b9890 con 0x55837c532080 2024-09-21T06:57:59.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.458+0000 7feba7400640 10 client.26632 handle_command_reply: tid=0 2024-09-21T06:57:59.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.458+0000 7febceec9740 1 client.26632 shutdown 2024-09-21T06:57:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 msgr2=0x55837c979f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 0x55837c979f10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7febac0095f0 tx=0x7febac003880 comp rx=0 tx=0).stop 2024-09-21T06:57:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 shutdown_connections 2024-09-21T06:57:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55837c59e350 0x55837c97a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55837c532080 0x55837c9e1c50 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55837c57d670 0x55837c979f10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 --2- 172.21.15.114:0/511223021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837c575490 0x55837c96aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 >> 172.21.15.114:0/511223021 conn(0x55837c592740 msgr2=0x55837c946d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:59.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.459+0000 7feba7400640 0 client.26632 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:57:59.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.460+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 shutdown_connections 2024-09-21T06:57:59.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.460+0000 7febceec9740 1 -- 172.21.15.114:0/511223021 wait complete. 2024-09-21T06:57:59.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.463+0000 7febceec9740 20 client.26632 trim_cache size 0 max 16384 2024-09-21T06:57:59.465 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:57:59.465 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T06:57:59.465 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "01b042dd-9ca7-4bf9-833e-c06fc008677f", 2024-09-21T06:57:59.465 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T06:57:59.465 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:57:59.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba80b6780 msgr2=0x7feba80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba80b6780 0x7feba80b8c90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55837c944990 tx=0x7febb4010f90 comp rx=0 tx=0).stop 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 msgr2=0x7febc01a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 0x7febc01a0c90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febb400b4a0 tx=0x7febb400b960 comp rx=0 tx=0).stop 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 shutdown_connections 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febc01a6410 0x7febc01a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7febc0114440 0x7febc01a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:57:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feba80b6780 0x7feba80b8c90 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55837c944990 tx=0x7febb4010f90 comp rx=0 tx=0).stop 2024-09-21T06:57:59.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 --2- 172.21.15.114:0/142494314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febc0077380 0x7febc01a0c90 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febb400b4a0 tx=0x7febb400b960 comp rx=0 tx=0).stop 2024-09-21T06:57:59.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.464+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 >> 172.21.15.114:0/142494314 conn(0x7febc00ff5e0 msgr2=0x7febc0110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:57:59.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.466+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 shutdown_connections 2024-09-21T06:57:59.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:57:59.466+0000 7febc8200640 1 -- 172.21.15.114:0/142494314 wait complete. 2024-09-21T06:58:00.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:58:00.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:58:00.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:59 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:58:00.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:59 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T06:58:00.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:59 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:57:59 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:58:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:58:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:58:00.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:59 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:58:00.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:59 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T06:58:00.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:59 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:57:59 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:58:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:58:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:58:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:59 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:58:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:59 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T06:58:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:59 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:57:59 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:58:00.603 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: pgmap v512: 129 pgs: 129 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 561 op/s 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:00.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:00 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.068 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:58:01.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: pgmap v512: 129 pgs: 129 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 561 op/s 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:00 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: pgmap v512: 129 pgs: 129 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.1 MiB/s wr, 561 op/s 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:00 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.805+0000 7f0b19a00640 1 -- 172.21.15.114:0/4230831113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b14109c40 msgr2=0x7f0b14110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.805+0000 7f0b19a00640 1 --2- 172.21.15.114:0/4230831113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b14109c40 0x7f0b14110470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009860 tx=0x7f0b0002a340 comp rx=0 tx=0).stop 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.806+0000 7f0b19a00640 1 -- 172.21.15.114:0/4230831113 shutdown_connections 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.806+0000 7f0b19a00640 1 --2- 172.21.15.114:0/4230831113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b14109c40 0x7f0b14110470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.806+0000 7f0b19a00640 1 --2- 172.21.15.114:0/4230831113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b141092f0 0x7f0b14109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.806+0000 7f0b19a00640 1 --2- 172.21.15.114:0/4230831113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 0x7f0b14103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.806+0000 7f0b19a00640 1 -- 172.21.15.114:0/4230831113 >> 172.21.15.114:0/4230831113 conn(0x7f0b140ff400 msgr2=0x7f0b14101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.807+0000 7f0b19a00640 1 -- 172.21.15.114:0/4230831113 shutdown_connections 2024-09-21T06:58:01.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.807+0000 7f0b19a00640 1 -- 172.21.15.114:0/4230831113 wait complete. 2024-09-21T06:58:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.809+0000 7f0b19a00640 1 Processor -- start 2024-09-21T06:58:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.809+0000 7f0b19a00640 1 -- start start 2024-09-21T06:58:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.810+0000 7f0b19a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 0x7f0b1419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.810+0000 7f0b19a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b141092f0 0x7f0b1419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.810+0000 7f0b19a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 0x7f0b1419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.810+0000 7f0b19a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14113900 con 0x7f0b141039b0 2024-09-21T06:58:01.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b19000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 0x7f0b1419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b19000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 0x7f0b1419fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54510/0 (socket says 172.21.15.114:54510) 2024-09-21T06:58:01.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b19000640 1 -- 172.21.15.114:0/1368497482 learned_addr learned my addr 172.21.15.114:0/1368497482 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:01.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14113780 con 0x7f0b14109c40 2024-09-21T06:58:01.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14113a80 con 0x7f0b141092f0 2024-09-21T06:58:01.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b13e00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 0x7f0b1419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.811+0000 7f0b13400640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b141092f0 0x7f0b1419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b19000640 1 -- 172.21.15.114:0/1368497482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b141092f0 msgr2=0x7f0b1419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b19000640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b141092f0 0x7f0b1419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b19000640 1 -- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 msgr2=0x7f0b1419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b19000640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 0x7f0b1419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b19000640 1 -- 172.21.15.114:0/1368497482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b141a01e0 con 0x7f0b14109c40 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b13e00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 0x7f0b1419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.812+0000 7f0b19000640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 0x7f0b1419fab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0b0002a930 tx=0x7f0b00032df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.813+0000 7f0b10c00640 1 -- 172.21.15.114:0/1368497482 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00037040 con 0x7f0b14109c40 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.813+0000 7f0b10c00640 1 -- 172.21.15.114:0/1368497482 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b00004050 con 0x7f0b14109c40 2024-09-21T06:58:01.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.813+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b141a0470 con 0x7f0b14109c40 2024-09-21T06:58:01.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.813+0000 7f0b10c00640 1 -- 172.21.15.114:0/1368497482 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00005280 con 0x7f0b14109c40 2024-09-21T06:58:01.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.813+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b140755f0 con 0x7f0b14109c40 2024-09-21T06:58:01.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.817+0000 7f0b21e0f740 1 Processor -- start 2024-09-21T06:58:01.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.820+0000 7f0b10c00640 1 -- 172.21.15.114:0/1368497482 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0b00043030 con 0x7f0b14109c40 2024-09-21T06:58:01.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.820+0000 7f0b21e0f740 1 -- start start 2024-09-21T06:58:01.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.821+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e582cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.821+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e583230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.821+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e583770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.821+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f3e4f9f70 con 0x561f3e1dc070 2024-09-21T06:58:01.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.821+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561f3e4e93a0 con 0x561f3e1bb390 2024-09-21T06:58:01.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.821+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561f3e4ed1b0 con 0x561f3e1b31b0 2024-09-21T06:58:01.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e583230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e583230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52476/0 (socket says 172.21.15.114:52476) 2024-09-21T06:58:01.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 -- 172.21.15.114:0/1605241259 learned_addr learned my addr 172.21.15.114:0/1605241259 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:01.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13400640 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e583770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13400640 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e582cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 -- 172.21.15.114:0/1605241259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 msgr2=0x561f3e583770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e583770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 -- 172.21.15.114:0/1605241259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 msgr2=0x561f3e582cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e582cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13e00640 1 -- 172.21.15.114:0/1605241259 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f3e4b3c90 con 0x561f3e1b31b0 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.823+0000 7f0b13400640 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e582cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:01.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.824+0000 7f0b13e00640 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e583230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009f20 tx=0x7f0b0c00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610fa4cc8fdbf46b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:01.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.824+0000 7f0b10c00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0af80b6700 0x7f0af80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.824+0000 7f0b09600640 1 -- 172.21.15.114:0/1605241259 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c003b70 con 0x561f3e1b31b0 2024-09-21T06:58:01.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.825+0000 7f0b09600640 1 -- 172.21.15.114:0/1605241259 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0c003d10 con 0x561f3e1b31b0 2024-09-21T06:58:01.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.825+0000 7f0b09600640 1 -- 172.21.15.114:0/1605241259 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c011920 con 0x561f3e1b31b0 2024-09-21T06:58:01.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.825+0000 7f0b10c00640 1 -- 172.21.15.114:0/1368497482 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b00106fb0 con 0x7f0b14109c40 2024-09-21T06:58:01.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.827+0000 7f0b13400640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0af80b6700 0x7f0af80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.827+0000 7f0b21e0f740 1 -- 172.21.15.114:0/1605241259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 msgr2=0x561f3e583230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.827+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e583230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009f20 tx=0x7f0b0c00f7f0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b13400640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0af80b6700 0x7f0af80b8c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009510 tx=0x7f0b04009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:01.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 -- 172.21.15.114:0/1605241259 shutdown_connections 2024-09-21T06:58:01.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e583770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e583230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/1605241259 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e582cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 -- 172.21.15.114:0/1605241259 >> 172.21.15.114:0/1605241259 conn(0x561f3e1d0460 msgr2=0x561f3e580b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 -- 172.21.15.114:0/1605241259 shutdown_connections 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.828+0000 7f0b21e0f740 1 -- 172.21.15.114:0/1605241259 wait complete. 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.829+0000 7f0b21e0f740 1 -- start start 2024-09-21T06:58:01.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.829+0000 7f0b21e0f740 1 Processor -- start 2024-09-21T06:58:01.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e58e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e58ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e5a3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561f3e34a700 con 0x561f3e1dc070 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561f3e484540 con 0x561f3e1bb390 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561f3e599ce0 con 0x561f3e1b31b0 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e58e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b13400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e58ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e58e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52492/0 (socket says 172.21.15.114:52492) 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b13e00640 1 -- 172.21.15.114:0/2594357300 learned_addr learned my addr 172.21.15.114:0/2594357300 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.830+0000 7f0b21e0f740 10 client.0 _ll_register_callbacks cb 0x561f3e494430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b19000640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e5a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b13e00640 1 -- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 msgr2=0x561f3e58ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b13e00640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e58ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b13e00640 1 -- 172.21.15.114:0/2594357300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 msgr2=0x561f3e5a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b13e00640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e5a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b13e00640 1 -- 172.21.15.114:0/2594357300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f3e247650 con 0x561f3e1b31b0 2024-09-21T06:58:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.831+0000 7f0b19000640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e5a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.832+0000 7f0b13e00640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e58e8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0093e0 tx=0x7f0b0c011890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35ffca7380c283a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.832+0000 7f0afe000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:01.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.832+0000 7f0afe000640 1 -- 172.21.15.114:0/2594357300 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c009af0 con 0x561f3e1b31b0 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.832+0000 7f0afe000640 1 -- 172.21.15.114:0/2594357300 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b0c011cc0 con 0x561f3e1b31b0 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.832+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f3e486b50 con 0x561f3e1b31b0 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.833+0000 7f0afe000640 1 -- 172.21.15.114:0/2594357300 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0b0c004a40 con 0x561f3e1b31b0 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.833+0000 7f0b21e0f740 10 client.26644 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.833+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561f3e4a1910 con 0x561f3e1b31b0 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.833+0000 7f0afe000640 1 -- 172.21.15.114:0/2594357300 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561f3e486b50 con 0x561f3e1b31b0 2024-09-21T06:58:01.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.833+0000 7f0afe000640 1 -- 172.21.15.114:0/2594357300 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f0b0c02f470 con 0x561f3e1b31b0 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b21e0f740 10 client.26644 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b21e0f740 10 client.26644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b21e0f740 20 client.26644 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e19a9f0 0x561f3e5a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b21e0f740 4 client.26644 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f3e4fbec0 con 0x561f3e19a9f0 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0b13400640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e19a9f0 0x561f3e5a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.834+0000 7f0afe000640 10 client.26644 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:01.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.835+0000 7f0b13400640 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e19a9f0 0x561f3e5a8f10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455eac8bffaad239 server_cookie=859f06d38078883e in_seq=0 out_seq=0 2024-09-21T06:58:01.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0afe000640 1 -- 172.21.15.114:0/2594357300 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f3e4fbec0 con 0x561f3e19a9f0 2024-09-21T06:58:01.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0afe000640 10 client.26644 handle_command_reply: tid=0 2024-09-21T06:58:01.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 client.26644 shutdown 2024-09-21T06:58:01.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 msgr2=0x561f3e58e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e58e8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c0093e0 tx=0x7f0b0c011890 comp rx=0 tx=0).stop 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 shutdown_connections 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1dc070 0x561f3e5a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e58ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e19a9f0 0x561f3e5a8f10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2594357300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e58e8f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0afe000640 0 client.26644 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.836+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 >> 172.21.15.114:0/2594357300 conn(0x561f3e1d0460 msgr2=0x561f3e585ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:01.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.837+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 shutdown_connections 2024-09-21T06:58:01.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.837+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2594357300 wait complete. 2024-09-21T06:58:01.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.837+0000 7f0b21e0f740 20 client.26644 trim_cache size 0 max 16384 2024-09-21T06:58:01.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 Processor -- start 2024-09-21T06:58:01.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 -- start start 2024-09-21T06:58:01.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e5898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1bb390 0x561f3e5801d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1dc070 0x561f3e580710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141aeac0 con 0x561f3e1bb390 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141b0600 con 0x561f3e1dc070 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.862+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141af190 con 0x561f3e1b31b0 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e5898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e5898a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52500/0 (socket says 172.21.15.114:52500) 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b13e00640 1 -- 172.21.15.114:0/2242672661 learned_addr learned my addr 172.21.15.114:0/2242672661 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:01.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b19000640 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1dc070 0x561f3e580710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b13400640 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1bb390 0x561f3e5801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b19000640 1 -- 172.21.15.114:0/2242672661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 msgr2=0x561f3e5898a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b19000640 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e5898a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b19000640 1 -- 172.21.15.114:0/2242672661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1bb390 msgr2=0x561f3e5801d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b19000640 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1bb390 0x561f3e5801d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.863+0000 7f0b19000640 1 -- 172.21.15.114:0/2242672661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f3e584fe0 con 0x561f3e1dc070 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.864+0000 7f0b19000640 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1dc070 0x561f3e580710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0b00104390 tx=0x7f0b000c8a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275661fc931e7c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.864+0000 7f0b08c00640 1 -- 172.21.15.114:0/2242672661 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000c8290 con 0x561f3e1dc070 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.864+0000 7f0b08c00640 1 -- 172.21.15.114:0/2242672661 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000cde50 con 0x561f3e1dc070 2024-09-21T06:58:01.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.864+0000 7f0b08c00640 1 -- 172.21.15.114:0/2242672661 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000d64b0 con 0x561f3e1dc070 2024-09-21T06:58:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2242672661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1dc070 msgr2=0x561f3e580710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1dc070 0x561f3e580710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0b00104390 tx=0x7f0b000c8a80 comp rx=0 tx=0).stop 2024-09-21T06:58:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2242672661 shutdown_connections 2024-09-21T06:58:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1dc070 0x561f3e580710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1bb390 0x561f3e5801d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/2242672661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1b31b0 0x561f3e5898a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2242672661 >> 172.21.15.114:0/2242672661 conn(0x561f3e1d0460 msgr2=0x561f3e58cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2242672661 shutdown_connections 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 -- 172.21.15.114:0/2242672661 wait complete. 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.866+0000 7f0b21e0f740 1 -- start start 2024-09-21T06:58:01.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 Processor -- start 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 0x561f3e59f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e59bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1dc070 0x561f3e59c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141aff30 con 0x561f3e1b31b0 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141af860 con 0x561f3e1bb390 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b141acaa0 con 0x561f3e1dc070 2024-09-21T06:58:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.867+0000 7f0b21e0f740 10 client.0 _ll_register_callbacks cb 0x7f0b0c009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 0x561f3e59f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 0x561f3e59f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44562/0 (socket says 172.21.15.114:44562) 2024-09-21T06:58:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 -- 172.21.15.114:0/30973204 learned_addr learned my addr 172.21.15.114:0/30973204 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13400640 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e59bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 -- 172.21.15.114:0/30973204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1dc070 msgr2=0x561f3e59c0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:58:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1dc070 0x561f3e59c0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 -- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 msgr2=0x561f3e59bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e59bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.868+0000 7f0b13e00640 1 -- 172.21.15.114:0/30973204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f3e59c800 con 0x561f3e1b31b0 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0b13e00640 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 0x561f3e59f6c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c005c50 tx=0x7f0b0c0115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd660ab9f720f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0afe000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0afe000640 1 -- 172.21.15.114:0/30973204 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0093c0 con 0x561f3e1b31b0 2024-09-21T06:58:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0afe000640 1 -- 172.21.15.114:0/30973204 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f0b0c02fc70 con 0x561f3e1b31b0 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b000cde50 con 0x561f3e1b31b0 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0afe000640 1 -- 172.21.15.114:0/30973204 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0b0c038450 con 0x561f3e1b31b0 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0b21e0f740 10 client.16524 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.869+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561f3e614640 con 0x561f3e1b31b0 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0afe000640 1 -- 172.21.15.114:0/30973204 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000cde50 con 0x561f3e1b31b0 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0afe000640 1 -- 172.21.15.114:0/30973204 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f0b0c0425c0 con 0x561f3e1b31b0 2024-09-21T06:58:01.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0b21e0f740 10 client.16524 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0b21e0f740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0b21e0f740 20 client.16524 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e16fda0 0x561f3e616d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0b21e0f740 4 client.16524 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:58:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.870+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f3e4f7970 con 0x561f3e16fda0 2024-09-21T06:58:01.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.871+0000 7f0b13400640 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e16fda0 0x561f3e616d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.871+0000 7f0afe000640 10 client.16524 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.871+0000 7f0b13400640 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e16fda0 0x561f3e616d40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f71c38c8df3b14b6 server_cookie=50d432cbf357732 in_seq=0 out_seq=0 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0afe000640 1 -- 172.21.15.114:0/30973204 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x561f3e4f7970 con 0x561f3e16fda0 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0afe000640 10 client.16524 handle_command_reply: tid=0 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 client.16524 shutdown 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 msgr2=0x561f3e59f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 0x561f3e59f6c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c005c50 tx=0x7f0b0c0115f0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 shutdown_connections 2024-09-21T06:58:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561f3e16fda0 0x561f3e616d40 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561f3e1dc070 0x561f3e59c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f3e1bb390 0x561f3e59bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 --2- 172.21.15.114:0/30973204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561f3e1b31b0 0x561f3e59f6c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 >> 172.21.15.114:0/30973204 conn(0x561f3e1d0460 msgr2=0x561f3e587cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0afe000640 0 client.16524 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 shutdown_connections 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 1 -- 172.21.15.114:0/30973204 wait complete. 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.872+0000 7f0b21e0f740 20 client.16524 trim_cache size 0 max 16384 2024-09-21T06:58:01.876 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "01b042dd-9ca7-4bf9-833e-c06fc008677f", 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:01.877 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:01.878 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:58:01.879 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0af80b6700 msgr2=0x7f0af80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0af80b6700 0x7f0af80b8c10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009510 tx=0x7f0b04009260 comp rx=0 tx=0).stop 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 msgr2=0x7f0b1419fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 0x7f0b1419fab0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0b0002a930 tx=0x7f0b00032df0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 shutdown_connections 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0af80b6700 0x7f0af80b8c10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b14109c40 0x7f0b1419fab0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b141092f0 0x7f0b1419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 --2- 172.21.15.114:0/1368497482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b141039b0 0x7f0b1419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 >> 172.21.15.114:0/1368497482 conn(0x7f0b140ff400 msgr2=0x7f0b14101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.873+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 shutdown_connections 2024-09-21T06:58:01.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:01.874+0000 7f0b19a00640 1 -- 172.21.15.114:0/1368497482 wait complete. 2024-09-21T06:58:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 205.0 s, 699.8 tps, lat 121.881 ms stddev 161.015 2024-09-21T06:58:03.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:02 smithi114 ceph-mon[30654]: pgmap v513: 129 pgs: 129 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.8 MiB/s wr, 516 op/s 2024-09-21T06:58:03.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:02 smithi183 ceph-mon[34219]: pgmap v513: 129 pgs: 129 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.8 MiB/s wr, 516 op/s 2024-09-21T06:58:03.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:02 smithi193 ceph-mon[34601]: pgmap v513: 129 pgs: 129 active+clean; 1.8 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.8 MiB/s wr, 516 op/s 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 -- 172.21.15.114:0/1004287949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc105b10 msgr2=0x7f8efc105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 --2- 172.21.15.114:0/1004287949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc105b10 0x7f8efc105f20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009900 tx=0x7f8eec02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 -- 172.21.15.114:0/1004287949 shutdown_connections 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 --2- 172.21.15.114:0/1004287949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8efc10bd90 0x7f8efc1125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 --2- 172.21.15.114:0/1004287949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc10b440 0x7f8efc10b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 --2- 172.21.15.114:0/1004287949 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc105b10 0x7f8efc105f20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 -- 172.21.15.114:0/1004287949 >> 172.21.15.114:0/1004287949 conn(0x7f8efc101550 msgr2=0x7f8efc1039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 -- 172.21.15.114:0/1004287949 shutdown_connections 2024-09-21T06:58:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.176+0000 7f8efb200640 1 -- 172.21.15.114:0/1004287949 wait complete. 2024-09-21T06:58:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.177+0000 7f8efb200640 1 Processor -- start 2024-09-21T06:58:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.177+0000 7f8efb200640 1 -- start start 2024-09-21T06:58:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 0x7f8efc19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efb200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8efc10b440 0x7f8efc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efb200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc10bd90 0x7f8efc1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efb200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc115aa0 con 0x7f8efc10bd90 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc115920 con 0x7f8efc105b10 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efb200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc115c20 con 0x7f8efc10b440 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 0x7f8efc19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 0x7f8efc19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54542/0 (socket says 172.21.15.114:54542) 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 -- 172.21.15.114:0/1920527563 learned_addr learned my addr 172.21.15.114:0/1920527563 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8efc10b440 0x7f8efc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8efa800640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc10bd90 0x7f8efc1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 -- 172.21.15.114:0/1920527563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8efc10b440 msgr2=0x7f8efc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8efc10b440 0x7f8efc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 -- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc10bd90 msgr2=0x7f8efc1a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.178+0000 7f8ef9e00640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc10bd90 0x7f8efc1a1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8ef9e00640 1 -- 172.21.15.114:0/1920527563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc1a2450 con 0x7f8efc105b10 2024-09-21T06:58:03.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8efa800640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc10bd90 0x7f8efc1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8ef9e00640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 0x7f8efc19c520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005ab0 tx=0x7f8eec003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8ef2a00640 1 -- 172.21.15.114:0/1920527563 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec03f070 con 0x7f8efc105b10 2024-09-21T06:58:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc1a26e0 con 0x7f8efc105b10 2024-09-21T06:58:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8ef2a00640 1 -- 172.21.15.114:0/1920527563 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eec031b60 con 0x7f8efc105b10 2024-09-21T06:58:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8ef2a00640 1 -- 172.21.15.114:0/1920527563 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eec035030 con 0x7f8efc105b10 2024-09-21T06:58:03.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.179+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc1a8ed0 con 0x7f8efc105b10 2024-09-21T06:58:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.182+0000 7f8f07c8f740 1 Processor -- start 2024-09-21T06:58:03.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.182+0000 7f8f07c8f740 1 -- start start 2024-09-21T06:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5611847576b0 con 0x56118443d1b0 2024-09-21T06:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561184746ae0 con 0x5611844142f0 2024-09-21T06:58:03.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56118474a8f0 con 0x56118441c4d0 2024-09-21T06:58:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847e1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44602/0 (socket says 172.21.15.114:44602) 2024-09-21T06:58:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 -- 172.21.15.114:0/4239012975 learned_addr learned my addr 172.21.15.114:0/4239012975 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8ef9400640 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8ef2a00640 1 -- 172.21.15.114:0/1920527563 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8eec042070 con 0x7f8efc105b10 2024-09-21T06:58:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 -- 172.21.15.114:0/4239012975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 msgr2=0x5611847e2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847e2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 -- 172.21.15.114:0/4239012975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 msgr2=0x5611847e2350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:58:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e2350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8efa800640 1 -- 172.21.15.114:0/4239012975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561184713780 con 0x56118443d1b0 2024-09-21T06:58:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.183+0000 7f8ef9400640 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847e2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.184+0000 7f8efa800640 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847e1e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4011790 tx=0x7f8ef4011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadff5601f5122de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.184+0000 7f8ee7000640 1 -- 172.21.15.114:0/4239012975 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4003c30 con 0x56118443d1b0 2024-09-21T06:58:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.184+0000 7f8ee7000640 1 -- 172.21.15.114:0/4239012975 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef400fe50 con 0x56118443d1b0 2024-09-21T06:58:03.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.184+0000 7f8ee7000640 1 -- 172.21.15.114:0/4239012975 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40193e0 con 0x56118443d1b0 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8ef2a00640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ee00b6680 0x7f8ee00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4239012975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 msgr2=0x5611847e1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847e1e10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4011790 tx=0x7f8ef4011c50 comp rx=0 tx=0).stop 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8ef2a00640 1 -- 172.21.15.114:0/1920527563 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8eec108220 con 0x7f8efc105b10 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4239012975 shutdown_connections 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847e2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4239012975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847e1e10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4239012975 >> 172.21.15.114:0/4239012975 conn(0x561184431300 msgr2=0x5611847dfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8ef9e00640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ee00b6680 0x7f8ee00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.191 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4239012975 shutdown_connections 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.185+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4239012975 wait complete. 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8ef9e00640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ee00b6680 0x7f8ee00b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5611847e0a90 tx=0x7f8eec02a3f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 -- start start 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 Processor -- start 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5611844142f0 0x5611847ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56118443d1b0 0x5611847edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56118467eab0 con 0x5611844142f0 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.186+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5611846ed570 con 0x56118443d1b0 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5611847f0590 con 0x56118441c4d0 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8f07c8f740 10 client.0 _ll_register_callbacks cb 0x5611846fc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8efa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5611844142f0 0x5611847ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8efa800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5611844142f0 0x5611847ed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44606/0 (socket says 172.21.15.114:44606) 2024-09-21T06:58:03.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8efa800640 1 -- 172.21.15.114:0/1399910030 learned_addr learned my addr 172.21.15.114:0/1399910030 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8ef9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8efa800640 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56118443d1b0 0x5611847edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8ef9400640 1 -- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56118443d1b0 msgr2=0x5611847edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56118443d1b0 0x5611847edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8ef9400640 1 -- 172.21.15.114:0/1399910030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5611844142f0 msgr2=0x5611847ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5611844142f0 0x5611847ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.187+0000 7f8ef9400640 1 -- 172.21.15.114:0/1399910030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611844a8790 con 0x56118441c4d0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847ed970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8efc06c620 tx=0x7f8ee8012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a54b77a3674307 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8edb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8edb400640 1 -- 172.21.15.114:0/1399910030 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800fa30 con 0x56118441c4d0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8edb400640 1 -- 172.21.15.114:0/1399910030 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8ee800fbd0 con 0x56118441c4d0 2024-09-21T06:58:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef400fe50 con 0x56118441c4d0 2024-09-21T06:58:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8edb400640 1 -- 172.21.15.114:0/1399910030 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8ee8024050 con 0x56118441c4d0 2024-09-21T06:58:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8f07c8f740 10 client.26659 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.188+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561184521620 con 0x56118441c4d0 2024-09-21T06:58:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8edb400640 1 -- 172.21.15.114:0/1399910030 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ef400fe50 con 0x56118441c4d0 2024-09-21T06:58:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8edb400640 1 -- 172.21.15.114:0/1399910030 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8ee8003ae0 con 0x56118441c4d0 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8f07c8f740 10 client.26659 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8f07c8f740 10 client.26659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8f07c8f740 20 client.26659 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843fbd80 0x5611847ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8f07c8f740 4 client.26659 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561184759600 con 0x5611843fbd80 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.189+0000 7f8efa800640 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843fbd80 0x5611847ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.190+0000 7f8edb400640 10 client.26659 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.190+0000 7f8efa800640 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843fbd80 0x5611847ff6e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d292a0ebe6beb0b4 server_cookie=de0f49d50336cad6 in_seq=0 out_seq=0 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8edb400640 1 -- 172.21.15.114:0/1399910030 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561184759600 con 0x5611843fbd80 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8edb400640 10 client.26659 handle_command_reply: tid=0 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 client.26659 shutdown 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 msgr2=0x5611847ed970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847ed970 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8efc06c620 tx=0x7f8ee8012b60 comp rx=0 tx=0).stop 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 shutdown_connections 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56118443d1b0 0x5611847edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843fbd80 0x5611847ff6e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847ed970 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8edb400640 0 client.26659 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1399910030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5611844142f0 0x5611847ed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 >> 172.21.15.114:0/1399910030 conn(0x561184431300 msgr2=0x5611847e2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 shutdown_connections 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1399910030 wait complete. 2024-09-21T06:58:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.191+0000 7f8f07c8f740 20 client.26659 trim_cache size 0 max 16384 2024-09-21T06:58:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 Processor -- start 2024-09-21T06:58:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 -- start start 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 0x5611847ea600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118443d1b0 0x5611847eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1b2e10 con 0x56118441c4d0 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1b4950 con 0x5611844142f0 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.207+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1b34e0 con 0x56118443d1b0 2024-09-21T06:58:03.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 0x5611847ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 0x5611847ea600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44612/0 (socket says 172.21.15.114:44612) 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 -- 172.21.15.114:0/4140625075 learned_addr learned my addr 172.21.15.114:0/4140625075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8efa800640 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8efa800640 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118443d1b0 0x5611847eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 -- 172.21.15.114:0/4140625075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118443d1b0 msgr2=0x5611847eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118443d1b0 0x5611847eab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 -- 172.21.15.114:0/4140625075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 msgr2=0x5611847e3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 -- 172.21.15.114:0/4140625075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611847f9f40 con 0x56118441c4d0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8efa800640 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118443d1b0 0x5611847eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ef9400640 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 0x5611847ea600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8003f80 tx=0x7f8ee8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91135c7cd81aed6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ee6600640 1 -- 172.21.15.114:0/4140625075 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800fa30 con 0x56118441c4d0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ee6600640 1 -- 172.21.15.114:0/4140625075 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee8010e20 con 0x56118441c4d0 2024-09-21T06:58:03.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.208+0000 7f8ee6600640 1 -- 172.21.15.114:0/4140625075 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80055c0 con 0x56118441c4d0 2024-09-21T06:58:03.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4140625075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 msgr2=0x5611847ea600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 0x5611847ea600 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8003f80 tx=0x7f8ee8010830 comp rx=0 tx=0).stop 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4140625075 shutdown_connections 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118443d1b0 0x5611847eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118441c4d0 0x5611847ea600 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/4140625075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847e3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4140625075 >> 172.21.15.114:0/4140625075 conn(0x561184431300 msgr2=0x5611847e2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4140625075 shutdown_connections 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.209+0000 7f8f07c8f740 1 -- 172.21.15.114:0/4140625075 wait complete. 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 -- start start 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 Processor -- start 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1b4280 con 0x56118443d1b0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1b3bb0 con 0x5611844142f0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc1b0df0 con 0x56118441c4d0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.210+0000 7f8f07c8f740 10 client.0 _ll_register_callbacks cb 0x7f8ee8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54582/0 (socket says 172.21.15.114:54582) 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 -- 172.21.15.114:0/1754375511 learned_addr learned my addr 172.21.15.114:0/1754375511 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 -- 172.21.15.114:0/1754375511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 msgr2=0x5611847f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 -- 172.21.15.114:0/1754375511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 msgr2=0x5611847f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 -- 172.21.15.114:0/1754375511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611847f4010 con 0x5611844142f0 2024-09-21T06:58:03.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8efa800640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847f5f20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009560 tx=0x7f8ef4011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c83e84a1e015f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8edb400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.211+0000 7f8edb400640 1 -- 172.21.15.114:0/1754375511 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef401f040 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee8010e20 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8edb400640 1 -- 172.21.15.114:0/1754375511 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f8ef4003d80 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8edb400640 1 -- 172.21.15.114:0/1754375511 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8ef4004a00 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 10 client.26717 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561184869fc0 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8edb400640 1 -- 172.21.15.114:0/1754375511 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ee8010e20 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8edb400640 1 -- 172.21.15.114:0/1754375511 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f8ef4025040 con 0x5611844142f0 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 10 client.26717 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 10 client.26717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 20 client.26717 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843d11c0 0x56118486c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 4 client.26717 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:58:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.212+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611847550b0 con 0x5611843d11c0 2024-09-21T06:58:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.213+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843d11c0 0x56118486c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.213+0000 7f8edb400640 10 client.26717 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:03.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.213+0000 7f8ef9400640 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843d11c0 0x56118486c5d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06622c241535c73 server_cookie=f296b61ebec64cef in_seq=0 out_seq=0 2024-09-21T06:58:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.607+0000 7f8edb400640 1 -- 172.21.15.114:0/1754375511 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611847550b0 con 0x5611843d11c0 2024-09-21T06:58:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.607+0000 7f8edb400640 10 client.26717 handle_command_reply: tid=0 2024-09-21T06:58:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.607+0000 7f8f07c8f740 1 client.26717 shutdown 2024-09-21T06:58:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.609+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 msgr2=0x5611847f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.609+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847f5f20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009560 tx=0x7f8ef4011c50 comp rx=0 tx=0).stop 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.610+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 shutdown_connections 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.610+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56118443d1b0 0x5611847f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.610+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5611843d11c0 0x56118486c5d0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.610+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56118441c4d0 0x5611847f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.610+0000 7f8f07c8f740 1 --2- 172.21.15.114:0/1754375511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5611844142f0 0x5611847f5f20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.610+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 >> 172.21.15.114:0/1754375511 conn(0x561184431300 msgr2=0x5611847e2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.611+0000 7f8edb400640 0 client.26717 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.611+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 shutdown_connections 2024-09-21T06:58:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.611+0000 7f8f07c8f740 1 -- 172.21.15.114:0/1754375511 wait complete. 2024-09-21T06:58:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.611+0000 7f8f07c8f740 20 client.26717 trim_cache size 0 max 16384 2024-09-21T06:58:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:58:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:58:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:58:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:58:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.613+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ee00b6680 msgr2=0x7f8ee00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.613+0000 7f8efb200640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ee00b6680 0x7f8ee00b8b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x5611847e0a90 tx=0x7f8eec02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 msgr2=0x7f8efc19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 0x7f8efc19c520 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8eec005ab0 tx=0x7f8eec003c10 comp rx=0 tx=0).stop 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 shutdown_connections 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8efc10bd90 0x7f8efc1a1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8efc10b440 0x7f8efc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8ee00b6680 0x7f8ee00b8b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 --2- 172.21.15.114:0/1920527563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8efc105b10 0x7f8efc19c520 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.614+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 >> 172.21.15.114:0/1920527563 conn(0x7f8efc101550 msgr2=0x7f8efc103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.615+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 shutdown_connections 2024-09-21T06:58:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:03.615+0000 7f8efb200640 1 -- 172.21.15.114:0/1920527563 wait complete. 2024-09-21T06:58:05.069 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:04 smithi114 ceph-mon[30654]: pgmap v514: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.7 MiB/s wr, 784 op/s 2024-09-21T06:58:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:04 smithi183 ceph-mon[34219]: pgmap v514: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.7 MiB/s wr, 784 op/s 2024-09-21T06:58:05.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:04 smithi193 ceph-mon[34601]: pgmap v514: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.7 MiB/s wr, 784 op/s 2024-09-21T06:58:06.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:05 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:58:06.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:05 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:58:06.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:05 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:58:06.415 INFO:tasks.cephfs.filesystem:scrub status for tag:01b042dd-9ca7-4bf9-833e-c06fc008677f - {'path': '/', 'tag': '01b042dd-9ca7-4bf9-833e-c06fc008677f', 'options': 'recursive,force'} 2024-09-21T06:58:07.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:06 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:58:07.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:06 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:58:07.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:06 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:58:07.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:06 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:58:07.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:06 smithi114 ceph-mon[30654]: pgmap v515: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.1 MiB/s wr, 869 op/s 2024-09-21T06:58:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:06 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:58:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:06 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:58:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:06 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:58:07.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:06 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:58:07.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:06 smithi183 ceph-mon[34219]: pgmap v515: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.1 MiB/s wr, 869 op/s 2024-09-21T06:58:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:06 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:58:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:06 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:58:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:06 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:58:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:06 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:58:07.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:06 smithi193 ceph-mon[34601]: pgmap v515: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.1 MiB/s wr, 869 op/s 2024-09-21T06:58:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 210.0 s, 939.0 tps, lat 124.691 ms stddev 344.513 2024-09-21T06:58:09.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:08 smithi183 ceph-mon[34219]: pgmap v516: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-21T06:58:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:08 smithi193 ceph-mon[34601]: pgmap v516: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-21T06:58:09.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:08 smithi114 ceph-mon[30654]: pgmap v516: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 13 MiB/s wr, 1.44k op/s 2024-09-21T06:58:11.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:10 smithi183 ceph-mon[34219]: pgmap v517: 129 pgs: 129 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T06:58:11.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:10 smithi193 ceph-mon[34601]: pgmap v517: 129 pgs: 129 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T06:58:11.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:10 smithi114 ceph-mon[30654]: pgmap v517: 129 pgs: 129 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 14 MiB/s wr, 1.53k op/s 2024-09-21T06:58:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 215.0 s, 2291.8 tps, lat 43.199 ms stddev 51.859 2024-09-21T06:58:13.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:12 smithi183 ceph-mon[34219]: pgmap v518: 129 pgs: 129 active+clean; 2.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-09-21T06:58:13.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:12 smithi193 ceph-mon[34601]: pgmap v518: 129 pgs: 129 active+clean; 2.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-09-21T06:58:13.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:12 smithi114 ceph-mon[30654]: pgmap v518: 129 pgs: 129 active+clean; 2.0 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-09-21T06:58:15.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:14 smithi193 ceph-mon[34601]: pgmap v519: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 1.71k op/s 2024-09-21T06:58:15.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:14 smithi114 ceph-mon[30654]: pgmap v519: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 1.71k op/s 2024-09-21T06:58:15.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:14 smithi183 ceph-mon[34219]: pgmap v519: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 16 MiB/s wr, 1.71k op/s 2024-09-21T06:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.921+0000 7fdfdb800640 1 -- 172.21.15.114:0/1844868341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4103040 msgr2=0x7fdfd4107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.921+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1844868341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4103040 0x7fdfd4107560 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009900 tx=0x7fdfc002a410 comp rx=0 tx=0).stop 2024-09-21T06:58:16.924 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.922+0000 7fdfdb800640 1 -- 172.21.15.114:0/1844868341 shutdown_connections 2024-09-21T06:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.922+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1844868341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfd4107bb0 0x7fdfd4108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.922+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1844868341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4103040 0x7fdfd4107560 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.922+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1844868341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd41026f0 0x7fdfd4102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.922+0000 7fdfdb800640 1 -- 172.21.15.114:0/1844868341 >> 172.21.15.114:0/1844868341 conn(0x7fdfd407a740 msgr2=0x7fdfd407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.923+0000 7fdfdb800640 1 -- 172.21.15.114:0/1844868341 shutdown_connections 2024-09-21T06:58:16.925 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.923+0000 7fdfdb800640 1 -- 172.21.15.114:0/1844868341 wait complete. 2024-09-21T06:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.925+0000 7fdfdb800640 1 Processor -- start 2024-09-21T06:58:16.926 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.925+0000 7fdfdb800640 1 -- start start 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfdb800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfd41026f0 0x7fdfd41a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfdb800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd4103040 0x7fdfd41a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfdb800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 0x7fdfd41a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfdb800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd411a030 con 0x7fdfd41026f0 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfdb800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4119eb0 con 0x7fdfd4103040 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfdb800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd411a1b0 con 0x7fdfd4107bb0 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 0x7fdfd41a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 0x7fdfd41a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50498/0 (socket says 172.21.15.114:50498) 2024-09-21T06:58:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.926+0000 7fdfd9800640 1 -- 172.21.15.114:0/1449743658 learned_addr learned my addr 172.21.15.114:0/1449743658 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.927+0000 7fdfd9800640 1 -- 172.21.15.114:0/1449743658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd4103040 msgr2=0x7fdfd41a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.927+0000 7fdfd9800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd4103040 0x7fdfd41a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.927+0000 7fdfd9800640 1 -- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfd41026f0 msgr2=0x7fdfd41a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.927+0000 7fdfd9800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfd41026f0 0x7fdfd41a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.927+0000 7fdfd9800640 1 -- 172.21.15.114:0/1449743658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41a69a0 con 0x7fdfd4107bb0 2024-09-21T06:58:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.928+0000 7fdfd9800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 0x7fdfd41a62a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00eb40 tx=0x7fdfcc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:16.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.929+0000 7fdfd1600640 1 -- 172.21.15.114:0/1449743658 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc004b20 con 0x7fdfd4107bb0 2024-09-21T06:58:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.929+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41a6c90 con 0x7fdfd4107bb0 2024-09-21T06:58:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.929+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41ad420 con 0x7fdfd4107bb0 2024-09-21T06:58:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.930+0000 7fdfd1600640 1 -- 172.21.15.114:0/1449743658 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc004cc0 con 0x7fdfd4107bb0 2024-09-21T06:58:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.930+0000 7fdfd1600640 1 -- 172.21.15.114:0/1449743658 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0107b0 con 0x7fdfd4107bb0 2024-09-21T06:58:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.941+0000 7fdfe27e0740 1 Processor -- start 2024-09-21T06:58:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.941+0000 7fdfd1600640 1 -- 172.21.15.114:0/1449743658 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fdfcc005000 con 0x7fdfd4107bb0 2024-09-21T06:58:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.944+0000 7fdfd1600640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfbc0b6700 0x7fdfbc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.944+0000 7fdfe27e0740 1 -- start start 2024-09-21T06:58:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.945+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed4041b0 0x5640ed7a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.945+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed3db2f0 0x5640ed7a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.945+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.945+0000 7fdfd1600640 1 -- 172.21.15.114:0/1449743658 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfcc0a9d30 con 0x7fdfd4107bb0 2024-09-21T06:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5640ed71e6b0 con 0x5640ed3db2f0 2024-09-21T06:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5640ed70dae0 con 0x5640ed4041b0 2024-09-21T06:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5640ed7118f0 con 0x5640ed3e34d0 2024-09-21T06:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50510/0 (socket says 172.21.15.114:50510) 2024-09-21T06:58:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 -- 172.21.15.114:0/3417676096 learned_addr learned my addr 172.21.15.114:0/3417676096 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 -- 172.21.15.114:0/3417676096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed4041b0 msgr2=0x5640ed7a8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed4041b0 0x5640ed7a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 -- 172.21.15.114:0/3417676096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed3db2f0 msgr2=0x5640ed7a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed3db2f0 0x5640ed7a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.946+0000 7fdfd9800640 1 -- 172.21.15.114:0/3417676096 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ed6da780 con 0x5640ed3e34d0 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.947+0000 7fdfd9800640 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a96b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0aa960 tx=0x7fdfcc0b58b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4380fe2805593b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:16.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.947+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfbc0b6700 0x7fdfbc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.947+0000 7fdfc6000640 1 -- 172.21.15.114:0/3417676096 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0aa2b0 con 0x5640ed3e34d0 2024-09-21T06:58:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.947+0000 7fdfc6000640 1 -- 172.21.15.114:0/3417676096 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc0ae1c0 con 0x5640ed3e34d0 2024-09-21T06:58:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3417676096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 msgr2=0x5640ed7a96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfc6000640 1 -- 172.21.15.114:0/3417676096 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0aa2b0 con 0x5640ed3e34d0 2024-09-21T06:58:16.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a96b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0aa960 tx=0x7fdfcc0b58b0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfbc0b6700 0x7fdfbc0b8c10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdfc002a930 tx=0x7fdfc0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3417676096 shutdown_connections 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a96b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed3db2f0 0x5640ed7a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3417676096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed4041b0 0x5640ed7a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3417676096 >> 172.21.15.114:0/3417676096 conn(0x5640ed3f8300 msgr2=0x5640ed7a6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3417676096 shutdown_connections 2024-09-21T06:58:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.949+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3417676096 wait complete. 2024-09-21T06:58:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.952+0000 7fdfe27e0740 1 -- start start 2024-09-21T06:58:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.953+0000 7fdfe27e0740 1 Processor -- start 2024-09-21T06:58:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.953+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 0x5640ed7b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3e34d0 0x5640ed7b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfd8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 0x5640ed7b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfd8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 0x5640ed7b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50514/0 (socket says 172.21.15.114:50514) 2024-09-21T06:58:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5640ed645ab0 con 0x5640ed4041b0 2024-09-21T06:58:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5640ed6b4570 con 0x5640ed3e34d0 2024-09-21T06:58:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.954+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5640ed7b7310 con 0x5640ed3db2f0 2024-09-21T06:58:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfe27e0740 10 client.0 _ll_register_callbacks cb 0x5640ed6c3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3503621147 learned_addr learned my addr 172.21.15.114:0/3503621147 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3e34d0 0x5640ed7b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd9800640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3e34d0 msgr2=0x5640ed7b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3e34d0 0x5640ed7b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3503621147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 msgr2=0x5640ed7b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3503621147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ed46f790 con 0x5640ed3db2f0 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.955+0000 7fdfd9800640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.956+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3e34d0 0x5640ed7b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.956+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 0x5640ed7b41c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e600 tx=0x7fdfc800eac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7170b76d78a5075 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.956+0000 7fdfbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.956+0000 7fdfbaa00640 1 -- 172.21.15.114:0/3503621147 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8005220 con 0x5640ed3db2f0 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.956+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc0ae1c0 con 0x5640ed3db2f0 2024-09-21T06:58:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.956+0000 7fdfbaa00640 1 -- 172.21.15.114:0/3503621147 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfc80053c0 con 0x5640ed3db2f0 2024-09-21T06:58:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.960+0000 7fdfbaa00640 1 -- 172.21.15.114:0/3503621147 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdfc80055c0 con 0x5640ed3db2f0 2024-09-21T06:58:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.960+0000 7fdfe27e0740 10 client.26677 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.960+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5640ed4e8620 con 0x5640ed3db2f0 2024-09-21T06:58:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.960+0000 7fdfbaa00640 1 -- 172.21.15.114:0/3503621147 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc0ae1c0 con 0x5640ed3db2f0 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfbaa00640 1 -- 172.21.15.114:0/3503621147 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fdfc8010c90 con 0x5640ed3db2f0 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfe27e0740 10 client.26677 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfe27e0740 10 client.26677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfe27e0740 20 client.26677 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed3c2d80 0x5640ed7c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfe27e0740 4 client.26677 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640ed720600 con 0x5640ed3c2d80 2024-09-21T06:58:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.961+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed3c2d80 0x5640ed7c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.962+0000 7fdfbaa00640 10 client.26677 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.963+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed3c2d80 0x5640ed7c63f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7476016b8cd82b47 server_cookie=ff858e93b494cb05 in_seq=0 out_seq=0 2024-09-21T06:58:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.964+0000 7fdfbaa00640 1 -- 172.21.15.114:0/3503621147 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5640ed720600 con 0x5640ed3c2d80 2024-09-21T06:58:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.964+0000 7fdfbaa00640 10 client.26677 handle_command_reply: tid=0 2024-09-21T06:58:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.964+0000 7fdfe27e0740 1 client.26677 shutdown 2024-09-21T06:58:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 msgr2=0x5640ed7b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 0x5640ed7b41c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800e600 tx=0x7fdfc800eac0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 shutdown_connections 2024-09-21T06:58:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3e34d0 0x5640ed7b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed3c2d80 0x5640ed7c63f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3503621147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3db2f0 0x5640ed7b41c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 >> 172.21.15.114:0/3503621147 conn(0x5640ed3f8300 msgr2=0x5640ed7a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.966+0000 7fdfbaa00640 0 client.26677 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.967+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 shutdown_connections 2024-09-21T06:58:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.967+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3503621147 wait complete. 2024-09-21T06:58:16.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.967+0000 7fdfe27e0740 20 client.26677 trim_cache size 0 max 16384 2024-09-21T06:58:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 Processor -- start 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 -- start start 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb400fec0 con 0x5640ed4041b0 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4011a00 con 0x5640ed3981c0 2024-09-21T06:58:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.997+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4010590 con 0x5640ed3e34d0 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50518/0 (socket says 172.21.15.114:50518) 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3925050200 learned_addr learned my addr 172.21.15.114:0/3925050200 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3925050200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 msgr2=0x5640ed7b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3925050200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 msgr2=0x5640ed7b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.998+0000 7fdfd8e00640 1 -- 172.21.15.114:0/3925050200 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ed7c2e80 con 0x5640ed3e34d0 2024-09-21T06:58:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.999+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.999+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a57f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009250 tx=0x7fdfc8009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e5f67ef99ff790 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.999+0000 7fdfc5600640 1 -- 172.21.15.114:0/3925050200 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8004240 con 0x5640ed3e34d0 2024-09-21T06:58:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.999+0000 7fdfc5600640 1 -- 172.21.15.114:0/3925050200 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc80043e0 con 0x5640ed3e34d0 2024-09-21T06:58:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:16.999+0000 7fdfc5600640 1 -- 172.21.15.114:0/3925050200 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8003a60 con 0x5640ed3e34d0 2024-09-21T06:58:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.001+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3925050200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 msgr2=0x5640ed7a57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.001+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a57f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009250 tx=0x7fdfc8009f60 comp rx=0 tx=0).stop 2024-09-21T06:58:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.002+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3925050200 shutdown_connections 2024-09-21T06:58:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.002+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.002+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed7b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.002+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/3925050200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed7a57f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.002+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3925050200 >> 172.21.15.114:0/3925050200 conn(0x5640ed3f8300 msgr2=0x5640ed7a50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.003+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3925050200 shutdown_connections 2024-09-21T06:58:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.003+0000 7fdfe27e0740 1 -- 172.21.15.114:0/3925050200 wait complete. 2024-09-21T06:58:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.004+0000 7fdfe27e0740 1 -- start start 2024-09-21T06:58:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.005+0000 7fdfe27e0740 1 Processor -- start 2024-09-21T06:58:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed829cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed82a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4011330 con 0x5640ed4041b0 2024-09-21T06:58:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4010c60 con 0x5640ed3981c0 2024-09-21T06:58:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb400dea0 con 0x5640ed3e34d0 2024-09-21T06:58:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfe27e0740 10 client.0 _ll_register_callbacks cb 0x7fdfc8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfd8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.006+0000 7fdfd8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7bcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60894/0 (socket says 172.21.15.114:60894) 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd8e00640 1 -- 172.21.15.114:0/4156273665 learned_addr learned my addr 172.21.15.114:0/4156273665 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed829cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd8e00640 1 -- 172.21.15.114:0/4156273665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 msgr2=0x5640ed829cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed829cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd8e00640 1 -- 172.21.15.114:0/4156273665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 msgr2=0x5640ed82a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed82a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.007+0000 7fdfd8e00640 1 -- 172.21.15.114:0/4156273665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ed7bbc10 con 0x5640ed3981c0 2024-09-21T06:58:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.008+0000 7fdfd8e00640 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7bcfc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800eb00 tx=0x7fdfc8005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b678a2047c2d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.008+0000 7fdfbaa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.008+0000 7fdfbaa00640 1 -- 172.21.15.114:0/4156273665 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8018c00 con 0x5640ed3981c0 2024-09-21T06:58:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.008+0000 7fdfbaa00640 1 -- 172.21.15.114:0/4156273665 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fdfc8018da0 con 0x5640ed3981c0 2024-09-21T06:58:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.009+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc80043e0 con 0x5640ed3981c0 2024-09-21T06:58:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.009+0000 7fdfbaa00640 1 -- 172.21.15.114:0/4156273665 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdfc8039800 con 0x5640ed3981c0 2024-09-21T06:58:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.009+0000 7fdfe27e0740 10 client.26726 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.009+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5640ed841f10 con 0x5640ed3981c0 2024-09-21T06:58:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.010+0000 7fdfbaa00640 1 -- 172.21.15.114:0/4156273665 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc80043e0 con 0x5640ed3981c0 2024-09-21T06:58:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.010+0000 7fdfbaa00640 1 -- 172.21.15.114:0/4156273665 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fdfc8027ae0 con 0x5640ed3981c0 2024-09-21T06:58:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfe27e0740 10 client.26726 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfe27e0740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfe27e0740 20 client.26726 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed842570 0x5640ed8449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfe27e0740 4 client.26726 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:58:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640ed71c0b0 con 0x5640ed842570 2024-09-21T06:58:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.011+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed842570 0x5640ed8449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.012+0000 7fdfd3e00640 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed842570 0x5640ed8449b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d97f64c90f86255 server_cookie=3f6a3260ac7af763 in_seq=0 out_seq=0 2024-09-21T06:58:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.012+0000 7fdfbaa00640 10 client.26726 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfbaa00640 1 -- 172.21.15.114:0/4156273665 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640ed71c0b0 con 0x5640ed842570 2024-09-21T06:58:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfbaa00640 10 client.26726 handle_command_reply: tid=0 2024-09-21T06:58:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfe27e0740 1 client.26726 shutdown 2024-09-21T06:58:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 msgr2=0x5640ed7bcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7bcfc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdfc800eb00 tx=0x7fdfc8005b90 comp rx=0 tx=0).stop 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 shutdown_connections 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5640ed4041b0 0x5640ed82a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.026+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5640ed842570 0x5640ed8449b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5640ed3e34d0 0x5640ed829cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfe27e0740 1 --2- 172.21.15.114:0/4156273665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5640ed3981c0 0x5640ed7bcfc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfbaa00640 0 client.26726 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 >> 172.21.15.114:0/4156273665 conn(0x5640ed3f8300 msgr2=0x5640ed7a6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 shutdown_connections 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfe27e0740 1 -- 172.21.15.114:0/4156273665 wait complete. 2024-09-21T06:58:17.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.027+0000 7fdfe27e0740 20 client.26726 trim_cache size 0 max 16384 2024-09-21T06:58:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:58:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:58:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:58:17.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfbc0b6700 msgr2=0x7fdfbc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfbc0b6700 0x7fdfbc0b8c10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdfc002a930 tx=0x7fdfc0000bc0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 msgr2=0x7fdfd41a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 0x7fdfd41a62a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00eb40 tx=0x7fdfcc00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 shutdown_connections 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfd4107bb0 0x7fdfd41a62a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdfbc0b6700 0x7fdfbc0b8c10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfd4103040 0x7fdfd41a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 --2- 172.21.15.114:0/1449743658 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdfd41026f0 0x7fdfd41a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 >> 172.21.15.114:0/1449743658 conn(0x7fdfd407a740 msgr2=0x7fdfd4105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 shutdown_connections 2024-09-21T06:58:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:17.028+0000 7fdfdb800640 1 -- 172.21.15.114:0/1449743658 wait complete. 2024-09-21T06:58:17.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:16 smithi193 ceph-mon[34601]: pgmap v520: 129 pgs: 129 active+clean; 2.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-09-21T06:58:17.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:16 smithi114 ceph-mon[30654]: pgmap v520: 129 pgs: 129 active+clean; 2.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-09-21T06:58:17.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:16 smithi183 ceph-mon[34219]: pgmap v520: 129 pgs: 129 active+clean; 2.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 15 MiB/s wr, 1.51k op/s 2024-09-21T06:58:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 220.0 s, 1321.4 tps, lat 74.973 ms stddev 85.694 2024-09-21T06:58:19.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:18 smithi193 ceph-mon[34601]: pgmap v521: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-09-21T06:58:19.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:18 smithi114 ceph-mon[30654]: pgmap v521: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-09-21T06:58:19.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:18 smithi183 ceph-mon[34219]: pgmap v521: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 14 MiB/s wr, 1.36k op/s 2024-09-21T06:58:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:20 smithi193 ceph-mon[34601]: pgmap v522: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 867 op/s 2024-09-21T06:58:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:20 smithi114 ceph-mon[30654]: pgmap v522: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 867 op/s 2024-09-21T06:58:21.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:20 smithi183 ceph-mon[34219]: pgmap v522: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.9 MiB/s wr, 867 op/s 2024-09-21T06:58:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 225.0 s, 1532.8 tps, lat 65.503 ms stddev 87.487 2024-09-21T06:58:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:22 smithi193 ceph-mon[34601]: pgmap v523: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 648 op/s 2024-09-21T06:58:23.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:22 smithi114 ceph-mon[30654]: pgmap v523: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 648 op/s 2024-09-21T06:58:23.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:22 smithi183 ceph-mon[34219]: pgmap v523: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.8 MiB/s wr, 648 op/s 2024-09-21T06:58:25.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:24 smithi193 ceph-mon[34601]: pgmap v524: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 804 op/s 2024-09-21T06:58:25.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:24 smithi114 ceph-mon[30654]: pgmap v524: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 804 op/s 2024-09-21T06:58:25.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:24 smithi183 ceph-mon[34219]: pgmap v524: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.3 MiB/s wr, 804 op/s 2024-09-21T06:58:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:26 smithi114 ceph-mon[30654]: pgmap v525: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 600 op/s 2024-09-21T06:58:27.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:26 smithi183 ceph-mon[34219]: pgmap v525: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 600 op/s 2024-09-21T06:58:27.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:26 smithi193 ceph-mon[34601]: pgmap v525: 129 pgs: 129 active+clean; 2.1 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.5 MiB/s wr, 600 op/s 2024-09-21T06:58:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 230.0 s, 1302.8 tps, lat 77.212 ms stddev 99.334 2024-09-21T06:58:29.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:28 smithi114 ceph-mon[30654]: pgmap v526: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 657 op/s 2024-09-21T06:58:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:28 smithi183 ceph-mon[34219]: pgmap v526: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 657 op/s 2024-09-21T06:58:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:28 smithi193 ceph-mon[34601]: pgmap v526: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.2 MiB/s wr, 657 op/s 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.326+0000 7fc4b5200640 1 -- 172.21.15.114:0/3091529395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 msgr2=0x7fc4b010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.326+0000 7fc4b5200640 1 --2- 172.21.15.114:0/3091529395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b010d810 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009860 tx=0x7fc49c02a340 comp rx=0 tx=0).stop 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.327+0000 7fc4b5200640 1 -- 172.21.15.114:0/3091529395 shutdown_connections 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.327+0000 7fc4b5200640 1 --2- 172.21.15.114:0/3091529395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4b010dd50 0x7fc4b0110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.327+0000 7fc4b5200640 1 --2- 172.21.15.114:0/3091529395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b010d810 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.327+0000 7fc4b5200640 1 --2- 172.21.15.114:0/3091529395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 0x7fc4b0104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.327+0000 7fc4b5200640 1 -- 172.21.15.114:0/3091529395 >> 172.21.15.114:0/3091529395 conn(0x7fc4b007a740 msgr2=0x7fc4b007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.328+0000 7fc4b5200640 1 -- 172.21.15.114:0/3091529395 shutdown_connections 2024-09-21T06:58:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.328+0000 7fc4b5200640 1 -- 172.21.15.114:0/3091529395 wait complete. 2024-09-21T06:58:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.330+0000 7fc4b5200640 1 Processor -- start 2024-09-21T06:58:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.330+0000 7fc4b5200640 1 -- start start 2024-09-21T06:58:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4b5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 0x7fc4b019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4b5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4b5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4b010dd50 0x7fc4b01a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4b5200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0113990 con 0x7fc4b010dd50 2024-09-21T06:58:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4b5200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0113810 con 0x7fc4b01047a0 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4b5200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b0113b10 con 0x7fc4b01051d0 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 0x7fc4b019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ae800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4b010dd50 0x7fc4b01a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33378/0 (socket says 172.21.15.114:33378) 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 -- 172.21.15.114:0/537539451 learned_addr learned my addr 172.21.15.114:0/537539451 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 -- 172.21.15.114:0/537539451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 msgr2=0x7fc4b019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 0x7fc4b019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 -- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4b010dd50 msgr2=0x7fc4b01a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4b010dd50 0x7fc4b01a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ad400640 1 -- 172.21.15.114:0/537539451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b01a2660 con 0x7fc4b01051d0 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.331+0000 7fc4ade00640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 0x7fc4b019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.332+0000 7fc4ad400640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b019ccd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc49c035040 tx=0x7fc49c032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.332+0000 7fc4a6a00640 1 -- 172.21.15.114:0/537539451 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c037040 con 0x7fc4b01051d0 2024-09-21T06:58:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.332+0000 7fc4a6a00640 1 -- 172.21.15.114:0/537539451 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc49c030c10 con 0x7fc4b01051d0 2024-09-21T06:58:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.332+0000 7fc4a6a00640 1 -- 172.21.15.114:0/537539451 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0035f0 con 0x7fc4b01051d0 2024-09-21T06:58:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.333+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b01a28f0 con 0x7fc4b01051d0 2024-09-21T06:58:30.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.333+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b01a2d30 con 0x7fc4b01051d0 2024-09-21T06:58:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.338+0000 7fc4bc3ef740 1 Processor -- start 2024-09-21T06:58:30.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.338+0000 7fc4bc3ef740 1 -- start start 2024-09-21T06:58:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.338+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881fd9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fda170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555881f4f6b0 con 0x555881c144d0 2024-09-21T06:58:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555881f3eae0 con 0x555881c351b0 2024-09-21T06:58:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555881f428f0 con 0x555881c0c2f0 2024-09-21T06:58:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fda170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fda170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33390/0 (socket says 172.21.15.114:33390) 2024-09-21T06:58:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ade00640 1 -- 172.21.15.114:0/2630312695 learned_addr learned my addr 172.21.15.114:0/2630312695 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ae800640 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881fd9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4a6a00640 1 -- 172.21.15.114:0/537539451 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc49c043020 con 0x7fc4b01051d0 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ae800640 1 -- 172.21.15.114:0/2630312695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 msgr2=0x555881fda170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ae800640 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fda170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.339+0000 7fc4ae800640 1 -- 172.21.15.114:0/2630312695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 msgr2=0x555881fda6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.340+0000 7fc4ad400640 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.340+0000 7fc4ae800640 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fda6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.340+0000 7fc4ae800640 1 -- 172.21.15.114:0/2630312695 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555881f0b780 con 0x555881c351b0 2024-09-21T06:58:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.340+0000 7fc4ad400640 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fda6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.340+0000 7fc4ae800640 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881fd9c30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009cf0 tx=0x7fc4a80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de68ceed6c32cdc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.341+0000 7fc49b400640 1 -- 172.21.15.114:0/2630312695 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8022070 con 0x555881c351b0 2024-09-21T06:58:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.341+0000 7fc49b400640 1 -- 172.21.15.114:0/2630312695 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a8003e20 con 0x555881c351b0 2024-09-21T06:58:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.341+0000 7fc49b400640 1 -- 172.21.15.114:0/2630312695 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80046e0 con 0x555881c351b0 2024-09-21T06:58:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4a6a00640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4940b6700 0x7fc4940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4a6a00640 1 -- 172.21.15.114:0/537539451 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc49c106b20 con 0x7fc4b01051d0 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/2630312695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 msgr2=0x555881fd9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881fd9c30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009cf0 tx=0x7fc4a80095c0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4ade00640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4940b6700 0x7fc4940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/2630312695 shutdown_connections 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fda6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fda170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/2630312695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881fd9c30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/2630312695 >> 172.21.15.114:0/2630312695 conn(0x555881c29300 msgr2=0x555881fd7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.342+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/2630312695 shutdown_connections 2024-09-21T06:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.343+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/2630312695 wait complete. 2024-09-21T06:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.343+0000 7fc4ade00640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4940b6700 0x7fc4940b8c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x555881fd88b0 tx=0x7fc4a0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.343+0000 7fc4bc3ef740 1 -- start start 2024-09-21T06:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.344+0000 7fc4bc3ef740 1 Processor -- start 2024-09-21T06:58:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fe5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fe57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881ff1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555881e76ab0 con 0x555881c144d0 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555881ee5570 con 0x555881c351b0 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555881fe83b0 con 0x555881c0c2f0 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fe5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fe5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33394/0 (socket says 172.21.15.114:33394) 2024-09-21T06:58:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4ade00640 1 -- 172.21.15.114:0/3970062101 learned_addr learned my addr 172.21.15.114:0/3970062101 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4ad400640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fe57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4ade00640 1 -- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 msgr2=0x555881ff1750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:58:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4ade00640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881ff1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.345+0000 7fc4bc3ef740 10 client.0 _ll_register_callbacks cb 0x555881ef4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc4ade00640 1 -- 172.21.15.114:0/3970062101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 msgr2=0x555881fe57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc4ade00640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fe57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc4ade00640 1 -- 172.21.15.114:0/3970062101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555881ca0790 con 0x555881c0c2f0 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc4ad400640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fe57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a8003e20 con 0x555881c0c2f0 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc4ade00640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fe5270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000f780 tx=0x7fc4a0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0cb149c8643d3de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.346+0000 7fc48fe00640 10 client.26695 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.347+0000 7fc48fe00640 1 -- 172.21.15.114:0/3970062101 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0011950 con 0x555881c0c2f0 2024-09-21T06:58:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.347+0000 7fc48fe00640 1 -- 172.21.15.114:0/3970062101 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc4a0011af0 con 0x555881c0c2f0 2024-09-21T06:58:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.347+0000 7fc48fe00640 1 -- 172.21.15.114:0/3970062101 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0011cf0 con 0x555881c0c2f0 2024-09-21T06:58:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.347+0000 7fc4bc3ef740 10 client.26695 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.347+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=35}) -- 0x555881d19620 con 0x555881c0c2f0 2024-09-21T06:58:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc48fe00640 1 -- 172.21.15.114:0/3970062101 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc4a0027070 con 0x555881c0c2f0 2024-09-21T06:58:30.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4bc3ef740 10 client.26695 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4bc3ef740 10 client.26695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4bc3ef740 20 client.26695 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bf3d80 0x555881ff7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4bc3ef740 4 client.26695 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555881f51600 con 0x555881bf3d80 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.348+0000 7fc4ad400640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bf3d80 0x555881ff7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.349+0000 7fc4ad400640 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bf3d80 0x555881ff7390 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194c5c84636ae767 server_cookie=ed7325168f490ef in_seq=0 out_seq=0 2024-09-21T06:58:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.349+0000 7fc48fe00640 10 client.26695 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.349+0000 7fc48fe00640 1 -- 172.21.15.114:0/3970062101 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555881f51600 con 0x555881bf3d80 2024-09-21T06:58:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc48fe00640 10 client.26695 handle_command_reply: tid=0 2024-09-21T06:58:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 client.26695 shutdown 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 msgr2=0x555881fe5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fe5270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000f780 tx=0x7fc4a0004730 comp rx=0 tx=0).stop 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 shutdown_connections 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x555881ff1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fe57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bf3d80 0x555881ff7390 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/3970062101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c0c2f0 0x555881fe5270 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 >> 172.21.15.114:0/3970062101 conn(0x555881c29300 msgr2=0x555881fdac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.350+0000 7fc48fe00640 0 client.26695 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.351+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 shutdown_connections 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.351+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/3970062101 wait complete. 2024-09-21T06:58:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.351+0000 7fc4bc3ef740 20 client.26695 trim_cache size 0 max 16384 2024-09-21T06:58:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.378+0000 7fc4bc3ef740 1 Processor -- start 2024-09-21T06:58:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.379+0000 7fc4bc3ef740 1 -- start start 2024-09-21T06:58:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.379+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 0x555881ff1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.379+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fdb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.379+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c351b0 0x555881fe2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.380+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01aec80 con 0x555881c144d0 2024-09-21T06:58:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.380+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01b07c0 con 0x555881c0c2f0 2024-09-21T06:58:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.380+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01af350 con 0x555881c351b0 2024-09-21T06:58:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.380+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 0x555881ff1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.380+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 0x555881ff1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34782/0 (socket says 172.21.15.114:34782) 2024-09-21T06:58:30.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.380+0000 7fc4ade00640 1 -- 172.21.15.114:0/1917985690 learned_addr learned my addr 172.21.15.114:0/1917985690 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.381+0000 7fc4ade00640 1 -- 172.21.15.114:0/1917985690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c351b0 msgr2=0x555881fe2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.381+0000 7fc4ade00640 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c351b0 0x555881fe2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.381+0000 7fc4ade00640 1 -- 172.21.15.114:0/1917985690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 msgr2=0x555881fdb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.381+0000 7fc4ade00640 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fdb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.381+0000 7fc4ade00640 1 -- 172.21.15.114:0/1917985690 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555881fdca70 con 0x555881c0c2f0 2024-09-21T06:58:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.385+0000 7fc4ade00640 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 0x555881ff1650 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009dd0 tx=0x7fc4a0023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6bf64234394ec00 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.386+0000 7fc49aa00640 1 -- 172.21.15.114:0/1917985690 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0038070 con 0x555881c0c2f0 2024-09-21T06:58:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.386+0000 7fc49aa00640 1 -- 172.21.15.114:0/1917985690 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a8003e20 con 0x555881c0c2f0 2024-09-21T06:58:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.386+0000 7fc49aa00640 1 -- 172.21.15.114:0/1917985690 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0029770 con 0x555881c0c2f0 2024-09-21T06:58:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.388+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1917985690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 msgr2=0x555881ff1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.389+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 0x555881ff1650 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009dd0 tx=0x7fc4a0023000 comp rx=0 tx=0).stop 2024-09-21T06:58:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.390+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1917985690 shutdown_connections 2024-09-21T06:58:30.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.390+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c351b0 0x555881fe2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.390+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c144d0 0x555881fdb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.390+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1917985690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c0c2f0 0x555881ff1650 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.390+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1917985690 >> 172.21.15.114:0/1917985690 conn(0x555881c29300 msgr2=0x555881fdac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.392+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1917985690 shutdown_connections 2024-09-21T06:58:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.392+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1917985690 wait complete. 2024-09-21T06:58:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.392+0000 7fc4bc3ef740 1 -- start start 2024-09-21T06:58:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.393+0000 7fc4bc3ef740 1 Processor -- start 2024-09-21T06:58:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 0x555881feaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c144d0 0x555881feb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x55588205fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01b00f0 con 0x555881c0c2f0 2024-09-21T06:58:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01afa20 con 0x555881c351b0 2024-09-21T06:58:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b01acc60 con 0x555881c144d0 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.394+0000 7fc4bc3ef740 10 client.0 _ll_register_callbacks cb 0x7fc4a0012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 0x555881feaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 0x555881feaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56812/0 (socket says 172.21.15.114:56812) 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 -- 172.21.15.114:0/1459083310 learned_addr learned my addr 172.21.15.114:0/1459083310 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ae800640 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x55588205fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 -- 172.21.15.114:0/1459083310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c144d0 msgr2=0x555881feb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c144d0 0x555881feb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 -- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 msgr2=0x55588205fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x55588205fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.395+0000 7fc4ade00640 1 -- 172.21.15.114:0/1459083310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558820603d0 con 0x555881c0c2f0 2024-09-21T06:58:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.396+0000 7fc48fe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:58:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.396+0000 7fc4ade00640 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 0x555881feaf10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0011f30 tx=0x7fc4a00339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbce3ea2f3aee5a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.397+0000 7fc48fe00640 1 -- 172.21.15.114:0/1459083310 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0038070 con 0x555881c0c2f0 2024-09-21T06:58:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.397+0000 7fc48fe00640 1 -- 172.21.15.114:0/1459083310 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc4a003d660 con 0x555881c0c2f0 2024-09-21T06:58:30.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.398+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a8003e20 con 0x555881c0c2f0 2024-09-21T06:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.398+0000 7fc48fe00640 1 -- 172.21.15.114:0/1459083310 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0033470 con 0x555881c0c2f0 2024-09-21T06:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.398+0000 7fc4bc3ef740 10 client.16554 fetch_fsmap learned FSMap version 35 2024-09-21T06:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.398+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555882061640 con 0x555881c0c2f0 2024-09-21T06:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.399+0000 7fc48fe00640 1 -- 172.21.15.114:0/1459083310 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a0033470 con 0x555881c0c2f0 2024-09-21T06:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.399+0000 7fc48fe00640 1 -- 172.21.15.114:0/1459083310 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc4a0027040 con 0x555881c0c2f0 2024-09-21T06:58:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.400+0000 7fc4bc3ef740 10 client.16554 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T06:58:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.400+0000 7fc4bc3ef740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.400+0000 7fc4bc3ef740 20 client.16554 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.400+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bc91c0 0x555882063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.400+0000 7fc4bc3ef740 4 client.16554 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:58:30.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.400+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555881f4d0b0 con 0x555881bc91c0 2024-09-21T06:58:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.402+0000 7fc4ad400640 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bc91c0 0x555882063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.402+0000 7fc4ad400640 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bc91c0 0x555882063cf0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16145b75ccb02a69 server_cookie=ee0d70ecb0af702e in_seq=0 out_seq=0 2024-09-21T06:58:30.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.402+0000 7fc48fe00640 10 client.16554 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:30.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.457+0000 7fc48fe00640 1 -- 172.21.15.114:0/1459083310 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555881f4d0b0 con 0x555881bc91c0 2024-09-21T06:58:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.457+0000 7fc48fe00640 10 client.16554 handle_command_reply: tid=0 2024-09-21T06:58:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.457+0000 7fc4bc3ef740 1 client.16554 shutdown 2024-09-21T06:58:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.457+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 msgr2=0x555881feaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 0x555881feaf10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0011f30 tx=0x7fc4a00339a0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 shutdown_connections 2024-09-21T06:58:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555881c351b0 0x55588205fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555881bc91c0 0x555882063cf0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555881c144d0 0x555881feb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 --2- 172.21.15.114:0/1459083310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555881c0c2f0 0x555881feaf10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.458+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 >> 172.21.15.114:0/1459083310 conn(0x555881c29300 msgr2=0x555881fe6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.459+0000 7fc48fe00640 0 client.16554 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.459+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 shutdown_connections 2024-09-21T06:58:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.459+0000 7fc4bc3ef740 1 -- 172.21.15.114:0/1459083310 wait complete. 2024-09-21T06:58:30.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.459+0000 7fc4bc3ef740 20 client.16554 trim_cache size 0 max 16384 2024-09-21T06:58:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:58:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:58:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:58:30.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:58:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.461+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4940b6700 msgr2=0x7fc4940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.461+0000 7fc4b5200640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4940b6700 0x7fc4940b8c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x555881fd88b0 tx=0x7fc4a0005f90 comp rx=0 tx=0).stop 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.461+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 msgr2=0x7fc4b019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.461+0000 7fc4b5200640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b019ccd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc49c035040 tx=0x7fc49c032d50 comp rx=0 tx=0).stop 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 shutdown_connections 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc4b010dd50 0x7fc4b01a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b01051d0 0x7fc4b019ccd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc4940b6700 0x7fc4940b8c10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 --2- 172.21.15.114:0/537539451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4b01047a0 0x7fc4b019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 >> 172.21.15.114:0/537539451 conn(0x7fc4b007a740 msgr2=0x7fc4b0109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 shutdown_connections 2024-09-21T06:58:30.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:30.462+0000 7fc4b5200640 1 -- 172.21.15.114:0/537539451 wait complete. 2024-09-21T06:58:31.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:30 smithi114 ceph-mon[30654]: pgmap v527: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 620 op/s 2024-09-21T06:58:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:30 smithi183 ceph-mon[34219]: pgmap v527: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 620 op/s 2024-09-21T06:58:31.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:30 smithi193 ceph-mon[34601]: pgmap v527: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 620 op/s 2024-09-21T06:58:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 235.0 s, 911.4 tps, lat 108.843 ms stddev 127.626 2024-09-21T06:58:33.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:32 smithi114 ceph-mon[30654]: pgmap v528: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 535 op/s 2024-09-21T06:58:33.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:32 smithi183 ceph-mon[34219]: pgmap v528: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 535 op/s 2024-09-21T06:58:33.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:32 smithi193 ceph-mon[34601]: pgmap v528: 129 pgs: 129 active+clean; 2.1 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.1 MiB/s wr, 535 op/s 2024-09-21T06:58:34.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:33 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:34.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:33 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:34.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:33 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:35.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:34 smithi114 ceph-mon[30654]: pgmap v529: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 649 op/s 2024-09-21T06:58:35.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:34 smithi183 ceph-mon[34219]: pgmap v529: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 649 op/s 2024-09-21T06:58:35.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:34 smithi193 ceph-mon[34601]: pgmap v529: 129 pgs: 129 active+clean; 2.1 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 649 op/s 2024-09-21T06:58:36.416 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:58:37.079 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:37.249 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:36 smithi114 ceph-mon[30654]: pgmap v530: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-09-21T06:58:37.249 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:36 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:36 smithi183 ceph-mon[34219]: pgmap v530: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-09-21T06:58:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:36 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:36 smithi193 ceph-mon[34601]: pgmap v530: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.2 MiB/s wr, 567 op/s 2024-09-21T06:58:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:36 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 240.0 s, 1020.4 tps, lat 85.120 ms stddev 97.801 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 -- 172.21.15.114:0/3897840467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410c020 msgr2=0x7ff7b4112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 --2- 172.21.15.114:0/3897840467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410c020 0x7ff7b4112850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009ca0 tx=0x7ff7ac02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 -- 172.21.15.114:0/3897840467 shutdown_connections 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 --2- 172.21.15.114:0/3897840467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410c020 0x7ff7b4112850 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 --2- 172.21.15.114:0/3897840467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b410b6d0 0x7ff7b410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 --2- 172.21.15.114:0/3897840467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b4105d90 0x7ff7b41061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 -- 172.21.15.114:0/3897840467 >> 172.21.15.114:0/3897840467 conn(0x7ff7b41017c0 msgr2=0x7ff7b4103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 -- 172.21.15.114:0/3897840467 shutdown_connections 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.237+0000 7ff7b9800640 1 -- 172.21.15.114:0/3897840467 wait complete. 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.238+0000 7ff7b9800640 1 Processor -- start 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.238+0000 7ff7b9800640 1 -- start start 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b9800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b4105d90 0x7ff7b419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b9800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 0x7ff7b419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b410c020 0x7ff7b41a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b9800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4115ce0 con 0x7ff7b4105d90 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b9800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4115b60 con 0x7ff7b410c020 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b9800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4115e60 con 0x7ff7b410b6d0 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 0x7ff7b419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b4105d90 0x7ff7b419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 0x7ff7b419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40106/0 (socket says 172.21.15.114:40106) 2024-09-21T06:58:38.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7aa000640 1 -- 172.21.15.114:0/1389700911 learned_addr learned my addr 172.21.15.114:0/1389700911 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:38.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.239+0000 7ff7b3400640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b410c020 0x7ff7b41a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7aa000640 1 -- 172.21.15.114:0/1389700911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b410c020 msgr2=0x7ff7b41a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7aa000640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b410c020 0x7ff7b41a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7aa000640 1 -- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b4105d90 msgr2=0x7ff7b419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7aa000640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b4105d90 0x7ff7b419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7aa000640 1 -- 172.21.15.114:0/1389700911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b41a25c0 con 0x7ff7b410b6d0 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7b3400640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b410c020 0x7ff7b41a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b4105d90 0x7ff7b419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7aa000640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 0x7ff7b419cbf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7abe00640 1 -- 172.21.15.114:0/1389700911 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c00ccc0 con 0x7ff7b410b6d0 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7abe00640 1 -- 172.21.15.114:0/1389700911 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c0108e0 con 0x7ff7b410b6d0 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b41a28b0 con 0x7ff7b410b6d0 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7abe00640 1 -- 172.21.15.114:0/1389700911 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c018860 con 0x7ff7b410b6d0 2024-09-21T06:58:38.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.240+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b41a9010 con 0x7ff7b410b6d0 2024-09-21T06:58:38.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.243+0000 7ff7c0488740 1 Processor -- start 2024-09-21T06:58:38.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.243+0000 7ff7c0488740 1 -- start start 2024-09-21T06:58:38.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e18bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e18c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7abe00640 1 -- 172.21.15.114:0/1389700911 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff79c010430 con 0x7ff7b410b6d0 2024-09-21T06:58:38.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e18c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7c0488740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556c5e102f70 con 0x556c5dde5070 2024-09-21T06:58:38.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7c0488740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c5e0f23a0 con 0x556c5ddc4390 2024-09-21T06:58:38.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7c0488740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556c5e0f61b0 con 0x556c5ddbc1b0 2024-09-21T06:58:38.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e18c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e18c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45416/0 (socket says 172.21.15.114:45416) 2024-09-21T06:58:38.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7aa000640 1 -- 172.21.15.114:0/1705923107 learned_addr learned my addr 172.21.15.114:0/1705923107 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:38.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e18c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.244+0000 7ff7b3400640 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e18bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.245+0000 7ff7b2a00640 1 -- 172.21.15.114:0/1705923107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 msgr2=0x556c5e18c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.245+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e18c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.245+0000 7ff7b2a00640 1 -- 172.21.15.114:0/1705923107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 msgr2=0x556c5e18bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.245+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e18bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.245+0000 7ff7b2a00640 1 -- 172.21.15.114:0/1705923107 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5e0bcc90 con 0x556c5ddbc1b0 2024-09-21T06:58:38.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.245+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e18c2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009fd0 tx=0x7ff7a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fa0262dd7fc10d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:38.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.246+0000 7ff7a3e00640 1 -- 172.21.15.114:0/1705923107 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4004030 con 0x556c5ddbc1b0 2024-09-21T06:58:38.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.246+0000 7ff7a3e00640 1 -- 172.21.15.114:0/1705923107 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a40054d0 con 0x556c5ddbc1b0 2024-09-21T06:58:38.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.246+0000 7ff7a3e00640 1 -- 172.21.15.114:0/1705923107 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4004c40 con 0x556c5ddbc1b0 2024-09-21T06:58:38.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.246+0000 7ff7abe00640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7980b6700 0x7ff7980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.246+0000 7ff7abe00640 1 -- 172.21.15.114:0/1389700911 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff79c0e7da0 con 0x7ff7b410b6d0 2024-09-21T06:58:38.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.246+0000 7ff7b3400640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7980b6700 0x7ff7980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 -- 172.21.15.114:0/1705923107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 msgr2=0x556c5e18c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e18c2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009fd0 tx=0x7ff7a40118d0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 -- 172.21.15.114:0/1705923107 shutdown_connections 2024-09-21T06:58:38.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e18c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e18c2a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 --2- 172.21.15.114:0/1705923107 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e18bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 -- 172.21.15.114:0/1705923107 >> 172.21.15.114:0/1705923107 conn(0x556c5ddd9460 msgr2=0x556c5e189c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 -- 172.21.15.114:0/1705923107 shutdown_connections 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7b3400640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7980b6700 0x7ff7980b8c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0094e0 tx=0x7ff7ac005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.247+0000 7ff7c0488740 1 -- 172.21.15.114:0/1705923107 wait complete. 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.248+0000 7ff7c0488740 1 -- start start 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.248+0000 7ff7c0488740 1 Processor -- start 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddbc1b0 0x556c5e197530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e197a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 0x556c5e1a3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556c5df53700 con 0x556c5ddbc1b0 2024-09-21T06:58:38.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c5e08d540 con 0x556c5ddc4390 2024-09-21T06:58:38.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556c5e19a540 con 0x556c5dde5070 2024-09-21T06:58:38.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7c0488740 10 client.0 _ll_register_callbacks cb 0x556c5e09d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:38.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7b3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 0x556c5e1a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7b3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 0x556c5e1a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40128/0 (socket says 172.21.15.114:40128) 2024-09-21T06:58:38.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7b3400640 1 -- 172.21.15.114:0/796835000 learned_addr learned my addr 172.21.15.114:0/796835000 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:38.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.249+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddbc1b0 0x556c5e197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7aa000640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b3400640 1 -- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 msgr2=0x556c5e197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b3400640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b3400640 1 -- 172.21.15.114:0/796835000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddbc1b0 msgr2=0x556c5e197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b3400640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddbc1b0 0x556c5e197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b3400640 1 -- 172.21.15.114:0/796835000 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5de50650 con 0x556c5dde5070 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddbc1b0 0x556c5e197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:38.264 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7aa000640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7b3400640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 0x556c5e1a3830 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac035000 tx=0x7ff7ac02a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d48af032892c7cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7a0c00640 1 -- 172.21.15.114:0/796835000 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac030c80 con 0x556c5dde5070 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7a0c00640 1 -- 172.21.15.114:0/796835000 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff7ac003b80 con 0x556c5dde5070 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.250+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c5e08fb50 con 0x556c5dde5070 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7a0c00640 1 -- 172.21.15.114:0/796835000 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff7ac003d80 con 0x556c5dde5070 2024-09-21T06:58:38.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 10 client.26707 fetch_fsmap learned FSMap version 37 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556c5e0aa910 con 0x556c5dde5070 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7a0c00640 1 -- 172.21.15.114:0/796835000 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556c5e08fb50 con 0x556c5dde5070 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7a0c00640 1 -- 172.21.15.114:0/796835000 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7ff7ac05e020 con 0x556c5dde5070 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 10 client.26707 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 10 client.26707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 20 client.26707 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dda39f0 0x556c5e1a9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 4 client.26707 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.251+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c5e104ec0 con 0x556c5dda39f0 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.252+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dda39f0 0x556c5e1a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.252+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dda39f0 0x556c5e1a9820 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6542d667790aa088 server_cookie=422502838efd4ef5 in_seq=0 out_seq=0 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.252+0000 7ff7a0c00640 10 client.26707 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7a0c00640 1 -- 172.21.15.114:0/796835000 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c5e104ec0 con 0x556c5dda39f0 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7a0c00640 10 client.26707 handle_command_reply: tid=0 2024-09-21T06:58:38.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 client.26707 shutdown 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 msgr2=0x556c5e1a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 0x556c5e1a3830 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac035000 tx=0x7ff7ac02a8b0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 shutdown_connections 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dda39f0 0x556c5e1a9820 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5dde5070 0x556c5e1a3830 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddc4390 0x556c5e197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7a0c00640 0 client.26707 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:38.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 --2- 172.21.15.114:0/796835000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddbc1b0 0x556c5e197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.253+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 >> 172.21.15.114:0/796835000 conn(0x556c5ddd9460 msgr2=0x556c5e18ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:38.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.254+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 shutdown_connections 2024-09-21T06:58:38.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.254+0000 7ff7c0488740 1 -- 172.21.15.114:0/796835000 wait complete. 2024-09-21T06:58:38.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.254+0000 7ff7c0488740 20 client.26707 trim_cache size 0 max 16384 2024-09-21T06:58:38.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.272+0000 7ff7c0488740 1 Processor -- start 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.272+0000 7ff7c0488740 1 -- start start 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e196420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddc4390 0x556c5e18f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5dde5070 0x556c5e194720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7c0488740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b30a0 con 0x556c5ddc4390 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7c0488740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b4c40 con 0x556c5dde5070 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7c0488740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b3770 con 0x556c5ddbc1b0 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddc4390 0x556c5e18f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7b3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5dde5070 0x556c5e194720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddc4390 0x556c5e18f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38252/0 (socket says 172.21.15.114:38252) 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7aa000640 1 -- 172.21.15.114:0/3277160930 learned_addr learned my addr 172.21.15.114:0/3277160930 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7b2a00640 1 -- 172.21.15.114:0/3277160930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5dde5070 msgr2=0x556c5e194720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5dde5070 0x556c5e194720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7b2a00640 1 -- 172.21.15.114:0/3277160930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddc4390 msgr2=0x556c5e18f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.273+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddc4390 0x556c5e18f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.274+0000 7ff7b2a00640 1 -- 172.21.15.114:0/3277160930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c5e18e550 con 0x556c5ddbc1b0 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.274+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e196420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff7a401b040 tx=0x7ff7a4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb75d1e7fa7b328a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.274+0000 7ff7a3400640 1 -- 172.21.15.114:0/3277160930 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400fb20 con 0x556c5ddbc1b0 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.274+0000 7ff7a3400640 1 -- 172.21.15.114:0/3277160930 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a40054f0 con 0x556c5ddbc1b0 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.274+0000 7ff7a3400640 1 -- 172.21.15.114:0/3277160930 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4003980 con 0x556c5ddbc1b0 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 -- 172.21.15.114:0/3277160930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 msgr2=0x556c5e196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e196420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff7a401b040 tx=0x7ff7a4009310 comp rx=0 tx=0).stop 2024-09-21T06:58:38.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 -- 172.21.15.114:0/3277160930 shutdown_connections 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5dde5070 0x556c5e194720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5ddc4390 0x556c5e18f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 --2- 172.21.15.114:0/3277160930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddbc1b0 0x556c5e196420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 -- 172.21.15.114:0/3277160930 >> 172.21.15.114:0/3277160930 conn(0x556c5ddd9460 msgr2=0x556c5e18ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 -- 172.21.15.114:0/3277160930 shutdown_connections 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.275+0000 7ff7c0488740 1 -- 172.21.15.114:0/3277160930 wait complete. 2024-09-21T06:58:38.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 -- start start 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 Processor -- start 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 0x556c5e19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddc4390 0x556c5e19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e2131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b4540 con 0x556c5dde5070 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b3e40 con 0x556c5ddbc1b0 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41b1080 con 0x556c5ddc4390 2024-09-21T06:58:38.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.276+0000 7ff7c0488740 10 client.0 _ll_register_callbacks cb 0x7ff7ac0098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:38.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 0x556c5e19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddc4390 0x556c5e19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7b2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 0x556c5e19bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45438/0 (socket says 172.21.15.114:45438) 2024-09-21T06:58:38.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7b2a00640 1 -- 172.21.15.114:0/514723314 learned_addr learned my addr 172.21.15.114:0/514723314 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:38.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7b3400640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e2131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 -- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 msgr2=0x556c5e19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 0x556c5e19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 -- 172.21.15.114:0/514723314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 msgr2=0x556c5e2131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:38.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e2131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 -- 172.21.15.114:0/514723314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c5e213850 con 0x556c5ddc4390 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 0x556c5e19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7aa000640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddc4390 0x556c5e19c5c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0eb5f0 tx=0x7ff79c009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26059e83dde68cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.277+0000 7ff7a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7a0c00640 1 -- 172.21.15.114:0/514723314 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0a96c0 con 0x556c5ddc4390 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7a0c00640 1 -- 172.21.15.114:0/514723314 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff79c0aeb50 con 0x556c5ddc4390 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a40054f0 con 0x556c5ddc4390 2024-09-21T06:58:38.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7a0c00640 1 -- 172.21.15.114:0/514723314 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff79c0acf90 con 0x556c5ddc4390 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 10 client.26713 fetch_fsmap learned FSMap version 37 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556c5e214b40 con 0x556c5ddc4390 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7a0c00640 1 -- 172.21.15.114:0/514723314 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a40054f0 con 0x556c5ddc4390 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7a0c00640 1 -- 172.21.15.114:0/514723314 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7ff79c0b3490 con 0x556c5ddc4390 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 10 client.26713 fetch_fsmap finished waiting for FSMap version 37 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:38.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 20 client.26713 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dd78da0 0x556c5e217380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 4 client.26713 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.278+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c5e100970 con 0x556c5dd78da0 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.279+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dd78da0 0x556c5e217380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.279+0000 7ff7b2a00640 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dd78da0 0x556c5e217380 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a62c982cde76f8 server_cookie=be7a0143fdecfa3a in_seq=0 out_seq=0 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.279+0000 7ff7a0c00640 10 client.26713 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.279+0000 7ff7a0c00640 1 -- 172.21.15.114:0/514723314 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x556c5e100970 con 0x556c5dd78da0 2024-09-21T06:58:38.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.279+0000 7ff7a0c00640 10 client.26713 handle_command_reply: tid=0 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.279+0000 7ff7c0488740 1 client.26713 shutdown 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddc4390 msgr2=0x556c5e19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddc4390 0x556c5e19c5c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0eb5f0 tx=0x7ff79c009740 comp rx=0 tx=0).stop 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 shutdown_connections 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556c5dde5070 0x556c5e2131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556c5dd78da0 0x556c5e217380 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556c5ddc4390 0x556c5e19c5c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 --2- 172.21.15.114:0/514723314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c5ddbc1b0 0x556c5e19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7a0c00640 0 client.26713 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 >> 172.21.15.114:0/514723314 conn(0x556c5ddd9460 msgr2=0x556c5e1989b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 shutdown_connections 2024-09-21T06:58:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 1 -- 172.21.15.114:0/514723314 wait complete. 2024-09-21T06:58:38.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.280+0000 7ff7c0488740 20 client.26713 trim_cache size 0 max 16384 2024-09-21T06:58:38.293 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:58:38.293 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T06:58:38.293 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T06:58:38.293 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:58:38.293 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:38.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:38.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:38.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:38.297 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:58:38.298 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:58:38.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:58:38.299 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:58:38.299 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:58:38.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7980b6700 msgr2=0x7ff7980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7980b6700 0x7ff7980b8c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0094e0 tx=0x7ff7ac005c30 comp rx=0 tx=0).stop 2024-09-21T06:58:38.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 msgr2=0x7ff7b419cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:38.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 0x7ff7b419cbf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e9f0 tx=0x7ff79c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 shutdown_connections 2024-09-21T06:58:38.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7980b6700 0x7ff7980b8c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7b410c020 0x7ff7b41a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b410b6d0 0x7ff7b419cbf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 --2- 172.21.15.114:0/1389700911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7b4105d90 0x7ff7b419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 >> 172.21.15.114:0/1389700911 conn(0x7ff7b41017c0 msgr2=0x7ff7b4108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 shutdown_connections 2024-09-21T06:58:38.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:38.281+0000 7ff7b9800640 1 -- 172.21.15.114:0/1389700911 wait complete. 2024-09-21T06:58:39.204 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:58:39.204 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:58:39.256 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:38 smithi114 ceph-mon[30654]: pgmap v531: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.5 MiB/s wr, 633 op/s 2024-09-21T06:58:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:38 smithi183 ceph-mon[34219]: pgmap v531: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.5 MiB/s wr, 633 op/s 2024-09-21T06:58:39.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:38 smithi193 ceph-mon[34601]: pgmap v531: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.5 MiB/s wr, 633 op/s 2024-09-21T06:58:39.601 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:40.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 -- 172.21.15.114:0/890774491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 msgr2=0x7f2038075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 --2- 172.21.15.114:0/890774491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f2038075bc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f202c009900 tx=0x7f202c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 -- 172.21.15.114:0/890774491 shutdown_connections 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 --2- 172.21.15.114:0/890774491 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f20381100d0 0x7f2038112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 --2- 172.21.15.114:0/890774491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2038076100 0x7f203810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 --2- 172.21.15.114:0/890774491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f2038075bc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 -- 172.21.15.114:0/890774491 >> 172.21.15.114:0/890774491 conn(0x7f20380ff5e0 msgr2=0x7f2038101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 -- 172.21.15.114:0/890774491 shutdown_connections 2024-09-21T06:58:40.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.333+0000 7f203fa00640 1 -- 172.21.15.114:0/890774491 wait complete. 2024-09-21T06:58:40.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.334+0000 7f203fa00640 1 Processor -- start 2024-09-21T06:58:40.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.334+0000 7f203fa00640 1 -- start start 2024-09-21T06:58:40.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.334+0000 7f203fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f203806d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:40.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.334+0000 7f203fa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2038076100 0x7f203806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:40.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.334+0000 7f203fa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20381100d0 0x7f2038072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:40.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203fa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2038115cd0 con 0x7f2038076100 2024-09-21T06:58:40.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203fa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2038115b50 con 0x7f20380757b0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203fa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2038115e50 con 0x7f20381100d0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f203806d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f203806d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45458/0 (socket says 172.21.15.114:45458) 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20381100d0 0x7f2038072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 -- 172.21.15.114:0/971312307 learned_addr learned my addr 172.21.15.114:0/971312307 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 -- 172.21.15.114:0/971312307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20381100d0 msgr2=0x7f2038072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20381100d0 0x7f2038072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 -- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2038076100 msgr2=0x7f203806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2038076100 0x7f203806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.335+0000 7f203d000640 1 -- 172.21.15.114:0/971312307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038073040 con 0x7f20380757b0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.336+0000 7f203d000640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f203806d0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f202c005ab0 tx=0x7f202c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.336+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c061070 con 0x7f20380757b0 2024-09-21T06:58:40.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.336+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f202c031d60 con 0x7f20380757b0 2024-09-21T06:58:40.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.336+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c035030 con 0x7f20380757b0 2024-09-21T06:58:40.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.336+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2038073270 con 0x7f20380757b0 2024-09-21T06:58:40.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.337+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2038073700 con 0x7f20380757b0 2024-09-21T06:58:40.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.338+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2038075bc0 con 0x7f20380757b0 2024-09-21T06:58:40.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.340+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f202c064070 con 0x7f20380757b0 2024-09-21T06:58:40.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.342+0000 7f2035600640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f20200b67d0 0x7f20200b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:40.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.342+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f202c12a880 con 0x7f20380757b0 2024-09-21T06:58:40.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.342+0000 7f2037e00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f20200b67d0 0x7f20200b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:40.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.343+0000 7f2037e00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f20200b67d0 0x7f20200b8ce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2024005fd0 tx=0x7f202400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:40.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.344+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f202c06b030 con 0x7f20380757b0 2024-09-21T06:58:40.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.568+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f203806e200 con 0x7f20380757b0 2024-09-21T06:58:40.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.569+0000 7f2035600640 1 -- 172.21.15.114:0/971312307 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11991 (secure 0 0 0) 0x7f202c0f0ae0 con 0x7f20380757b0 2024-09-21T06:58:40.572 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:58:40.572 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":37,"btime":"2024-09-21T06:58:35:977875+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:58:35.614409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:58:40.572 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 37 2024-09-21T06:58:40.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f20200b67d0 msgr2=0x7f20200b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:40.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f20200b67d0 0x7f20200b8ce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2024005fd0 tx=0x7f202400b410 comp rx=0 tx=0).stop 2024-09-21T06:58:40.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 msgr2=0x7f203806d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:40.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f203806d0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f202c005ab0 tx=0x7f202c003cd0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 shutdown_connections 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20381100d0 0x7f2038072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2038076100 0x7f203806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f20200b67d0 0x7f20200b8ce0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.572+0000 7f203fa00640 1 --2- 172.21.15.114:0/971312307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20380757b0 0x7f203806d0d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.573+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 >> 172.21.15.114:0/971312307 conn(0x7f20380ff5e0 msgr2=0x7f2038109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.573+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 shutdown_connections 2024-09-21T06:58:40.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:40.573+0000 7f203fa00640 1 -- 172.21.15.114:0/971312307 wait complete. 2024-09-21T06:58:41.257 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:41 smithi114 ceph-mon[30654]: pgmap v532: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-09-21T06:58:41.257 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/971312307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:58:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:41 smithi183 ceph-mon[34219]: pgmap v532: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-09-21T06:58:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/971312307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:58:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:41 smithi193 ceph-mon[34601]: pgmap v532: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.9 MiB/s wr, 582 op/s 2024-09-21T06:58:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/971312307' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:58:41.805 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:58:42.215 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:42.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 245.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:58:43.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:43 smithi114 ceph-mon[30654]: pgmap v533: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.9 MiB/s wr, 468 op/s 2024-09-21T06:58:43.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:43 smithi183 ceph-mon[34219]: pgmap v533: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.9 MiB/s wr, 468 op/s 2024-09-21T06:58:43.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:43 smithi193 ceph-mon[34601]: pgmap v533: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.9 MiB/s wr, 468 op/s 2024-09-21T06:58:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 -- 172.21.15.114:0/3445304800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 msgr2=0x7f3a4c110280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 --2- 172.21.15.114:0/3445304800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c110280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009ca0 tx=0x7f3a4802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 -- 172.21.15.114:0/3445304800 shutdown_connections 2024-09-21T06:58:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 --2- 172.21.15.114:0/3445304800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c110280 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 --2- 172.21.15.114:0/3445304800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a4c105280 0x7f3a4c10d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 --2- 172.21.15.114:0/3445304800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 -- 172.21.15.114:0/3445304800 >> 172.21.15.114:0/3445304800 conn(0x7f3a4c0fe3c0 msgr2=0x7f3a4c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 -- 172.21.15.114:0/3445304800 shutdown_connections 2024-09-21T06:58:43.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.671+0000 7f3a54800640 1 -- 172.21.15.114:0/3445304800 wait complete. 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.672+0000 7f3a54800640 1 Processor -- start 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.672+0000 7f3a54800640 1 -- start start 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a54800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a54800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a4c105280 0x7f3a4c102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a54800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a54800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c113a10 con 0x7f3a4c10dd90 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a54800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c113890 con 0x7f3a4c105280 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a54800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c113b90 con 0x7f3a4c104930 2024-09-21T06:58:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a52800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c102660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40192/0 (socket says 172.21.15.114:40192) 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a52800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38294/0 (socket says 172.21.15.114:38294) 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a51e00640 1 -- 172.21.15.114:0/279204414 learned_addr learned my addr 172.21.15.114:0/279204414 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.673+0000 7f3a51400640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a4c105280 0x7f3a4c102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a51e00640 1 -- 172.21.15.114:0/279204414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a4c105280 msgr2=0x7f3a4c102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a51e00640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a4c105280 0x7f3a4c102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a51e00640 1 -- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 msgr2=0x7f3a4c1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a51e00640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c1030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a51e00640 1 -- 172.21.15.114:0/279204414 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c1a8680 con 0x7f3a4c104930 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a51e00640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c102660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000e9f0 tx=0x7f3a4000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a46a00640 1 -- 172.21.15.114:0/279204414 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000cd80 con 0x7f3a4c104930 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c1a8970 con 0x7f3a4c104930 2024-09-21T06:58:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a46a00640 1 -- 172.21.15.114:0/279204414 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a400054d0 con 0x7f3a4c104930 2024-09-21T06:58:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a46a00640 1 -- 172.21.15.114:0/279204414 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40010690 con 0x7f3a4c104930 2024-09-21T06:58:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.674+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1a8ec0 con 0x7f3a4c104930 2024-09-21T06:58:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.677+0000 7f3a5b868740 1 Processor -- start 2024-09-21T06:58:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.677+0000 7f3a5b868740 1 -- start start 2024-09-21T06:58:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a5b868740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ac9c2286b0 con 0x55ac9beed4d0 2024-09-21T06:58:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a5b868740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac9c217ae0 con 0x55ac9bee52f0 2024-09-21T06:58:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a5b868740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ac9c21b8f0 con 0x55ac9bf0e1b0 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38306/0 (socket says 172.21.15.114:38306) 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 -- 172.21.15.114:0/2904377730 learned_addr learned my addr 172.21.15.114:0/2904377730 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a52800640 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 -- 172.21.15.114:0/2904377730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 msgr2=0x55ac9c2b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a46a00640 1 -- 172.21.15.114:0/279204414 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f3a40004e50 con 0x7f3a4c104930 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 -- 172.21.15.114:0/2904377730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 msgr2=0x55ac9c2b3170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2b3170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.678+0000 7f3a51400640 1 -- 172.21.15.114:0/2904377730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9c1e4780 con 0x55ac9beed4d0 2024-09-21T06:58:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.679+0000 7f3a51400640 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2b36b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009c20 tx=0x7f3a3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5c9bc28c7a4de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.679+0000 7f3a3b000640 1 -- 172.21.15.114:0/2904377730 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c004230 con 0x55ac9beed4d0 2024-09-21T06:58:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.679+0000 7f3a3b000640 1 -- 172.21.15.114:0/2904377730 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3c0043d0 con 0x55ac9beed4d0 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.679+0000 7f3a3b000640 1 -- 172.21.15.114:0/2904377730 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c004d40 con 0x55ac9beed4d0 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 -- 172.21.15.114:0/2904377730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 msgr2=0x55ac9c2b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2b36b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009c20 tx=0x7f3a3c011810 comp rx=0 tx=0).stop 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 -- 172.21.15.114:0/2904377730 shutdown_connections 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2b36b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 --2- 172.21.15.114:0/2904377730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 -- 172.21.15.114:0/2904377730 >> 172.21.15.114:0/2904377730 conn(0x55ac9bf02300 msgr2=0x55ac9c2b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 -- 172.21.15.114:0/2904377730 shutdown_connections 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a5b868740 1 -- 172.21.15.114:0/2904377730 wait complete. 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.680+0000 7f3a46a00640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3a340b65c0 0x7f3a340b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a46a00640 1 -- 172.21.15.114:0/279204414 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3a400e7e90 con 0x7f3a4c104930 2024-09-21T06:58:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a51400640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3a340b65c0 0x7f3a340b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 -- start start 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 Processor -- start 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a51400640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3a340b65c0 0x7f3a340b8ad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c011d10 tx=0x7f3a3c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2be210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9beed4d0 0x55ac9c2be750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ac9c14fab0 con 0x55ac9bf0e1b0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac9c1be570 con 0x55ac9bee52f0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.681+0000 7f3a5b868740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ac9c2c13b0 con 0x55ac9beed4d0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a5b868740 10 client.0 _ll_register_callbacks cb 0x55ac9c1cd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a52800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9beed4d0 0x55ac9c2be750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a52800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38316/0 (socket says 172.21.15.114:38316) 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a52800640 1 -- 172.21.15.114:0/789163125 learned_addr learned my addr 172.21.15.114:0/789163125 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 -- 172.21.15.114:0/789163125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9beed4d0 msgr2=0x55ac9c2be750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9beed4d0 0x55ac9c2be750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 -- 172.21.15.114:0/789163125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 msgr2=0x55ac9c2bec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 -- 172.21.15.114:0/789163125 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9bf79790 con 0x55ac9bee52f0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a52800640 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a51e00640 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2be210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a400e5210 tx=0x7f3a400ad7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd5092aaf471ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.682+0000 7f3a2f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a2f400640 1 -- 172.21.15.114:0/789163125 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400ae010 con 0x55ac9bee52f0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a3c0043d0 con 0x55ac9bee52f0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a2f400640 1 -- 172.21.15.114:0/789163125 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3a400acb70 con 0x55ac9bee52f0 2024-09-21T06:58:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a2f400640 1 -- 172.21.15.114:0/789163125 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3a400b23f0 con 0x55ac9bee52f0 2024-09-21T06:58:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 10 client.26771 fetch_fsmap learned FSMap version 38 2024-09-21T06:58:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac9bff2620 con 0x55ac9bee52f0 2024-09-21T06:58:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a2f400640 1 -- 172.21.15.114:0/789163125 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a3c0043d0 con 0x55ac9bee52f0 2024-09-21T06:58:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a2f400640 1 -- 172.21.15.114:0/789163125 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f3a400ac5e0 con 0x55ac9bee52f0 2024-09-21T06:58:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 10 client.26771 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:58:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 10 client.26771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 20 client.26771 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9beccd80 0x55ac9c2d02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 4 client.26771 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.683+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac9c22a600 con 0x55ac9beccd80 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.684+0000 7f3a51400640 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9beccd80 0x55ac9c2d02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.684+0000 7f3a51400640 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9beccd80 0x55ac9c2d02e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528a5c6f294ce059 server_cookie=a8ac4a9e6bff6786 in_seq=0 out_seq=0 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.684+0000 7f3a2f400640 10 client.26771 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.685+0000 7f3a2f400640 1 -- 172.21.15.114:0/789163125 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac9c22a600 con 0x55ac9beccd80 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.685+0000 7f3a2f400640 10 client.26771 handle_command_reply: tid=0 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 client.26771 shutdown 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 msgr2=0x55ac9c2be210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2be210 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a400e5210 tx=0x7f3a400ad7a0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 shutdown_connections 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9beccd80 0x55ac9c2d02e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9beed4d0 0x55ac9c2be750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 --2- 172.21.15.114:0/789163125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2be210 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a2f400640 0 client.26771 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 >> 172.21.15.114:0/789163125 conn(0x55ac9bf02300 msgr2=0x55ac9c2b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 shutdown_connections 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 1 -- 172.21.15.114:0/789163125 wait complete. 2024-09-21T06:58:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.686+0000 7f3a5b868740 20 client.26771 trim_cache size 0 max 16384 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 Processor -- start 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 -- start start 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 0x55ac9c2afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b2e90 con 0x55ac9beed4d0 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b4a60 con 0x55ac9bf0e1b0 2024-09-21T06:58:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.701+0000 7f3a5b868740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b3560 con 0x55ac9bee52f0 2024-09-21T06:58:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 0x55ac9c2afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 0x55ac9c2afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40216/0 (socket says 172.21.15.114:40216) 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 -- 172.21.15.114:0/3253573134 learned_addr learned my addr 172.21.15.114:0/3253573134 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a52800640 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 -- 172.21.15.114:0/3253573134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bf0e1b0 msgr2=0x55ac9c2bb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 -- 172.21.15.114:0/3253573134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 msgr2=0x55ac9c2bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 -- 172.21.15.114:0/3253573134 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9c2b5a40 con 0x55ac9bee52f0 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51400640 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a51e00640 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 0x55ac9c2afaa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a400aab80 tx=0x7f3a40004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d726c55788343ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.702+0000 7f3a3a600640 1 -- 172.21.15.114:0/3253573134 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400b7330 con 0x55ac9bee52f0 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a3a600640 1 -- 172.21.15.114:0/3253573134 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a40005680 con 0x55ac9bee52f0 2024-09-21T06:58:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a3a600640 1 -- 172.21.15.114:0/3253573134 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400a9080 con 0x55ac9bee52f0 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 -- 172.21.15.114:0/3253573134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 msgr2=0x55ac9c2afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 0x55ac9c2afaa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a400aab80 tx=0x7f3a40004a00 comp rx=0 tx=0).stop 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 -- 172.21.15.114:0/3253573134 shutdown_connections 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c2bb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 --2- 172.21.15.114:0/3253573134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bee52f0 0x55ac9c2afaa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 -- 172.21.15.114:0/3253573134 >> 172.21.15.114:0/3253573134 conn(0x55ac9bf02300 msgr2=0x55ac9c2bf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 -- 172.21.15.114:0/3253573134 shutdown_connections 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.703+0000 7f3a5b868740 1 -- 172.21.15.114:0/3253573134 wait complete. 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 -- start start 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 Processor -- start 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c332cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b4360 con 0x55ac9beed4d0 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b3c30 con 0x55ac9bee52f0 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.704+0000 7f3a5b868740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c1b0e70 con 0x55ac9bf0e1b0 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a5b868740 10 client.0 _ll_register_callbacks cb 0x7f3a4000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45508/0 (socket says 172.21.15.114:45508) 2024-09-21T06:58:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 -- 172.21.15.114:0/1958369294 learned_addr learned my addr 172.21.15.114:0/1958369294 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a52800640 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c332cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 -- 172.21.15.114:0/1958369294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 msgr2=0x55ac9c332cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c332cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 -- 172.21.15.114:0/1958369294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 msgr2=0x55ac9c2c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 -- 172.21.15.114:0/1958369294 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9c333350 con 0x55ac9bee52f0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a51e00640 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2c3f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a400b25b0 tx=0x7f3a400ace20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99a8b30dbe9ae73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.705+0000 7f3a2f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a2f400640 1 -- 172.21.15.114:0/1958369294 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400bc060 con 0x55ac9bee52f0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a2f400640 1 -- 172.21.15.114:0/1958369294 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3a400a9470 con 0x55ac9bee52f0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a40005680 con 0x55ac9bee52f0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a2f400640 1 -- 172.21.15.114:0/1958369294 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3a400c3050 con 0x55ac9bee52f0 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 10 client.26777 fetch_fsmap learned FSMap version 38 2024-09-21T06:58:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ac9c333350 con 0x55ac9bee52f0 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a2f400640 1 -- 172.21.15.114:0/1958369294 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a40005680 con 0x55ac9bee52f0 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a2f400640 1 -- 172.21.15.114:0/1958369294 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7f3a400b3580 con 0x55ac9bee52f0 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 10 client.26777 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 10 client.26777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 20 client.26777 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9bea21c0 0x55ac9c33ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 4 client.26777 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.706+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac9c2260b0 con 0x55ac9bea21c0 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.707+0000 7f3a51400640 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9bea21c0 0x55ac9c33ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.707+0000 7f3a2f400640 10 client.26777 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:43.707+0000 7f3a51400640 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9bea21c0 0x55ac9c33ca40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b6784a8e4cf319d server_cookie=9dcf9b6d5a8e6392 in_seq=0 out_seq=0 2024-09-21T06:58:44.171 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:44 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 -- 172.21.15.114:0/2176058458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 msgr2=0x7fb668110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 --2- 172.21.15.114:0/2176058458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668110470 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb660009ca0 tx=0x7fb66002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 -- 172.21.15.114:0/2176058458 shutdown_connections 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 --2- 172.21.15.114:0/2176058458 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668110470 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 --2- 172.21.15.114:0/2176058458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681092f0 0x7fb668109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 --2- 172.21.15.114:0/2176058458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681039b0 0x7fb668103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 -- 172.21.15.114:0/2176058458 >> 172.21.15.114:0/2176058458 conn(0x7fb6680ff440 msgr2=0x7fb668101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 -- 172.21.15.114:0/2176058458 shutdown_connections 2024-09-21T06:58:44.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.346+0000 7fb66ce00640 1 -- 172.21.15.114:0/2176058458 wait complete. 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.347+0000 7fb66ce00640 1 Processor -- start 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.347+0000 7fb66ce00640 1 -- start start 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.347+0000 7fb66ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681039b0 0x7fb668075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb66ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681092f0 0x7fb668075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb66ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb66ce00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb668113900 con 0x7fb668109c40 2024-09-21T06:58:44.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb66ce00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb668113780 con 0x7fb6681039b0 2024-09-21T06:58:44.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb66ce00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb668113a80 con 0x7fb6681092f0 2024-09-21T06:58:44.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb667e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb666a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681092f0 0x7fb668075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb667e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38362/0 (socket says 172.21.15.114:38362) 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb667e00640 1 -- 172.21.15.114:0/1490015790 learned_addr learned my addr 172.21.15.114:0/1490015790 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb667400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681039b0 0x7fb668075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb666a00640 1 -- 172.21.15.114:0/1490015790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681039b0 msgr2=0x7fb668075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb666a00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681039b0 0x7fb668075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb666a00640 1 -- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 msgr2=0x7fb668076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb666a00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb666a00640 1 -- 172.21.15.114:0/1490015790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6681a8530 con 0x7fb6681092f0 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.348+0000 7fb667e00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.349+0000 7fb666a00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681092f0 0x7fb668075c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb66806b990 tx=0x7fb65400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.349+0000 7fb65fe00640 1 -- 172.21.15.114:0/1490015790 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65400cc40 con 0x7fb6681092f0 2024-09-21T06:58:44.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.349+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6681a8820 con 0x7fb6681092f0 2024-09-21T06:58:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.349+0000 7fb65fe00640 1 -- 172.21.15.114:0/1490015790 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb65400cde0 con 0x7fb6681092f0 2024-09-21T06:58:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.349+0000 7fb65fe00640 1 -- 172.21.15.114:0/1490015790 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654018760 con 0x7fb6681092f0 2024-09-21T06:58:44.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.349+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6681a8db0 con 0x7fb6681092f0 2024-09-21T06:58:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.352+0000 7fb675021740 1 Processor -- start 2024-09-21T06:58:44.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.352+0000 7fb675021740 1 -- start start 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb675021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9cf5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb675021740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 0x5610b9cf64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb675021740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb675021740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610b9c6cfb0 con 0x5610b992e390 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb675021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610b9c5c3e0 con 0x5610b994f070 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb675021740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5610b9c601f0 con 0x5610b99261b0 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 0x5610b9cf64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9cf5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 0x5610b9cf64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40248/0 (socket says 172.21.15.114:40248) 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9cf5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45544/0 (socket says 172.21.15.114:45544) 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667400640 1 -- 172.21.15.114:0/2276365678 learned_addr learned my addr 172.21.15.114:0/2276365678 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 -- 172.21.15.114:0/2276365678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 msgr2=0x5610b9cf64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 0x5610b9cf64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 -- 172.21.15.114:0/2276365678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 msgr2=0x5610b9cf69f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf69f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667e00640 1 -- 172.21.15.114:0/2276365678 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b9c26cd0 con 0x5610b994f070 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.353+0000 7fb667400640 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 0x5610b9cf64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:44.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.354+0000 7fb65fe00640 1 -- 172.21.15.114:0/1490015790 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb654010a90 con 0x7fb6681092f0 2024-09-21T06:58:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.354+0000 7fb667e00640 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9cf5f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb660005cc0 tx=0x7fb66002f390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0bbb0a4a1a9061e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.354+0000 7fb65cc00640 1 -- 172.21.15.114:0/2276365678 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660031650 con 0x5610b994f070 2024-09-21T06:58:44.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.354+0000 7fb65cc00640 1 -- 172.21.15.114:0/2276365678 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb660036040 con 0x5610b994f070 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.354+0000 7fb65cc00640 1 -- 172.21.15.114:0/2276365678 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600044a0 con 0x5610b994f070 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 -- 172.21.15.114:0/2276365678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 msgr2=0x5610b9cf5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9cf5f70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb660005cc0 tx=0x7fb66002f390 comp rx=0 tx=0).stop 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 -- 172.21.15.114:0/2276365678 shutdown_connections 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b99261b0 0x5610b9cf64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 --2- 172.21.15.114:0/2276365678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9cf5f70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 -- 172.21.15.114:0/2276365678 >> 172.21.15.114:0/2276365678 conn(0x5610b9943460 msgr2=0x5610b9cf3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.355+0000 7fb675021740 1 -- 172.21.15.114:0/2276365678 shutdown_connections 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.356+0000 7fb675021740 1 -- 172.21.15.114:0/2276365678 wait complete. 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.356+0000 7fb65fe00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb64c0b6590 0x7fb64c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.356+0000 7fb65fe00640 1 -- 172.21.15.114:0/1490015790 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb6540e7d30 con 0x7fb6681092f0 2024-09-21T06:58:44.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.356+0000 7fb667400640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb64c0b6590 0x7fb64c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.356+0000 7fb675021740 1 -- start start 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.356+0000 7fb675021740 1 Processor -- start 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb667400640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb64c0b6590 0x7fb64c0b8aa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5610b9cf4bf0 tx=0x7fb65800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b992e390 0x5610b9d0abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9d0fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5610b9abd700 con 0x5610b99261b0 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610b9bf7580 con 0x5610b994f070 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5610b9d04570 con 0x5610b992e390 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb675021740 10 client.0 _ll_register_callbacks cb 0x5610b9c07470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb667400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb667400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38364/0 (socket says 172.21.15.114:38364) 2024-09-21T06:58:44.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb667400640 1 -- 172.21.15.114:0/2774740633 learned_addr learned my addr 172.21.15.114:0/2774740633 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.357+0000 7fb666a00640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b992e390 0x5610b9d0abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb667e00640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9d0fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb666a00640 1 -- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 msgr2=0x5610b9d0fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb666a00640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9d0fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb666a00640 1 -- 172.21.15.114:0/2774740633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 msgr2=0x5610b9d0a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb666a00640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb666a00640 1 -- 172.21.15.114:0/2774740633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b99ba650 con 0x5610b992e390 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb666a00640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b992e390 0x5610b9d0abc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb6540e5110 tx=0x7fb6540acef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aab37f673380da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:44.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb651600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb651600640 1 -- 172.21.15.114:0/2774740633 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540ad210 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.358+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610b9bf9b90 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb651600640 1 -- 172.21.15.114:0/2774740633 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb6540b3480 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb651600640 1 -- 172.21.15.114:0/2774740633 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb6540ad030 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 10 client.26749 fetch_fsmap learned FSMap version 38 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5610b9c14950 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb651600640 1 -- 172.21.15.114:0/2774740633 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5610b9bf9b90 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb651600640 1 -- 172.21.15.114:0/2774740633 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fb6540b6770 con 0x5610b992e390 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 10 client.26749 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 10 client.26749 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:58:44.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 20 client.26749 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b990d9f0 0x5610b9d1be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 4 client.26749 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.359+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b9c6ef00 con 0x5610b990d9f0 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.360+0000 7fb667400640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b990d9f0 0x5610b9d1be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.360+0000 7fb667400640 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b990d9f0 0x5610b9d1be80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79b7da5043de11de server_cookie=31d8c4cfd5600a8e in_seq=0 out_seq=0 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.360+0000 7fb651600640 10 client.26749 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.361+0000 7fb651600640 1 -- 172.21.15.114:0/2774740633 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5610b9c6ef00 con 0x5610b990d9f0 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.361+0000 7fb651600640 10 client.26749 handle_command_reply: tid=0 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.361+0000 7fb675021740 1 client.26749 shutdown 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.361+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b992e390 msgr2=0x5610b9d0abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.361+0000 7fb675021740 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b992e390 0x5610b9d0abc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb6540e5110 tx=0x7fb6540acef0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:44 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:44.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.361+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 shutdown_connections 2024-09-21T06:58:44.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b990d9f0 0x5610b9d1be80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b994f070 0x5610b9d0fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb651600640 0 client.26749 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b992e390 0x5610b9d0abc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 --2- 172.21.15.114:0/2774740633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 >> 172.21.15.114:0/2774740633 conn(0x5610b9943460 msgr2=0x5610b9cf9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 shutdown_connections 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 1 -- 172.21.15.114:0/2774740633 wait complete. 2024-09-21T06:58:44.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.362+0000 7fb675021740 20 client.26749 trim_cache size 0 max 16384 2024-09-21T06:58:44.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 Processor -- start 2024-09-21T06:58:44.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 -- start start 2024-09-21T06:58:44.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b99261b0 0x5610b9d0c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9cf74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681ae970 con 0x5610b992e390 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681b0540 con 0x5610b99261b0 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.376+0000 7fb675021740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681af040 con 0x5610b994f070 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38372/0 (socket says 172.21.15.114:38372) 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 -- 172.21.15.114:0/4258877030 learned_addr learned my addr 172.21.15.114:0/4258877030 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb667400640 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b99261b0 0x5610b9d0c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb667e00640 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9cf74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 -- 172.21.15.114:0/4258877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 msgr2=0x5610b9cf74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9cf74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 -- 172.21.15.114:0/4258877030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b99261b0 msgr2=0x5610b9d0c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b99261b0 0x5610b9d0c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 -- 172.21.15.114:0/4258877030 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b9cf8260 con 0x5610b992e390 2024-09-21T06:58:44.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.377+0000 7fb666a00640 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf6fa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb6540ad5c0 tx=0x7fb654010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5aecbd196f8125 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.378+0000 7fb653e00640 1 -- 172.21.15.114:0/4258877030 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540ad3d0 con 0x5610b992e390 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.378+0000 7fb653e00640 1 -- 172.21.15.114:0/4258877030 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6540b22f0 con 0x5610b992e390 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.378+0000 7fb653e00640 1 -- 172.21.15.114:0/4258877030 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654010850 con 0x5610b992e390 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.378+0000 7fb675021740 1 -- 172.21.15.114:0/4258877030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 msgr2=0x5610b9cf6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.378+0000 7fb675021740 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf6fa0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb6540ad5c0 tx=0x7fb654010e30 comp rx=0 tx=0).stop 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 -- 172.21.15.114:0/4258877030 shutdown_connections 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9cf74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b992e390 0x5610b9cf6fa0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 --2- 172.21.15.114:0/4258877030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b99261b0 0x5610b9d0c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 -- 172.21.15.114:0/4258877030 >> 172.21.15.114:0/4258877030 conn(0x5610b9943460 msgr2=0x5610b9d0b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 -- 172.21.15.114:0/4258877030 shutdown_connections 2024-09-21T06:58:44.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 -- 172.21.15.114:0/4258877030 wait complete. 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 -- start start 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.379+0000 7fb675021740 1 Processor -- start 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b992e390 0x5610b9d0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9d0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681afe40 con 0x5610b99261b0 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681af710 con 0x5610b992e390 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb6681ac970 con 0x5610b994f070 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb675021740 10 client.0 _ll_register_callbacks cb 0x7fb6540a9d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb667400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb667400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:38386/0 (socket says 172.21.15.114:38386) 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb667400640 1 -- 172.21.15.114:0/3000751091 learned_addr learned my addr 172.21.15.114:0/3000751091 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb667e00640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9d0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.380+0000 7fb666a00640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b992e390 0x5610b9d0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb667400640 1 -- 172.21.15.114:0/3000751091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 msgr2=0x5610b9d0bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb667400640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9d0bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb667400640 1 -- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b992e390 msgr2=0x5610b9d0b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb667400640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b992e390 0x5610b9d0b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb667400640 1 -- 172.21.15.114:0/3000751091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b9d0d630 con 0x5610b99261b0 2024-09-21T06:58:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb667400640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0cff0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb6580039c0 tx=0x7fb6580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a02f37068d8de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb651600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:58:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb651600640 1 -- 172.21.15.114:0/3000751091 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658035060 con 0x5610b99261b0 2024-09-21T06:58:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb651600640 1 -- 172.21.15.114:0/3000751091 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb65801eab0 con 0x5610b99261b0 2024-09-21T06:58:44.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.381+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6540b22f0 con 0x5610b99261b0 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb651600640 1 -- 172.21.15.114:0/3000751091 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb65801ecb0 con 0x5610b99261b0 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 10 client.16602 fetch_fsmap learned FSMap version 38 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5610b9d871b0 con 0x5610b99261b0 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb651600640 1 -- 172.21.15.114:0/3000751091 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6540b22f0 con 0x5610b99261b0 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb651600640 1 -- 172.21.15.114:0/3000751091 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fb658011440 con 0x5610b99261b0 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 10 client.16602 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 10 client.16602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 20 client.16602 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b98e2da0 0x5610b9d89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 4 client.16602 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.382+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610b9c6a9b0 con 0x5610b98e2da0 2024-09-21T06:58:44.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb666a00640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b98e2da0 0x5610b9d89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb666a00640 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b98e2da0 0x5610b9d89950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=704683148977a03 server_cookie=dc0c5bf86ea63189 in_seq=0 out_seq=0 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb651600640 10 client.16602 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb651600640 1 -- 172.21.15.114:0/3000751091 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610b9c6a9b0 con 0x5610b98e2da0 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb651600640 10 client.16602 handle_command_reply: tid=0 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb675021740 1 client.16602 shutdown 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 msgr2=0x5610b9d0cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.383+0000 7fb675021740 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0cff0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb6580039c0 tx=0x7fb6580046c0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 shutdown_connections 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5610b994f070 0x5610b9d0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5610b98e2da0 0x5610b9d89950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610b992e390 0x5610b9d0b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 --2- 172.21.15.114:0/3000751091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5610b99261b0 0x5610b9d0cff0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 >> 172.21.15.114:0/3000751091 conn(0x5610b9943460 msgr2=0x5610b9d08c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb651600640 0 client.16602 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:58:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 shutdown_connections 2024-09-21T06:58:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 1 -- 172.21.15.114:0/3000751091 wait complete. 2024-09-21T06:58:44.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.384+0000 7fb675021740 20 client.16602 trim_cache size 0 max 16384 2024-09-21T06:58:44.390 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:58:44.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb64c0b6590 msgr2=0x7fb64c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb64c0b6590 0x7fb64c0b8aa0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5610b9cf4bf0 tx=0x7fb65800b390 comp rx=0 tx=0).stop 2024-09-21T06:58:44.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681092f0 msgr2=0x7fb668075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681092f0 0x7fb668075c50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb66806b990 tx=0x7fb65400ee50 comp rx=0 tx=0).stop 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 shutdown_connections 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb668109c40 0x7fb668076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6681092f0 0x7fb668075c50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb64c0b6590 0x7fb64c0b8aa0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 --2- 172.21.15.114:0/1490015790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6681039b0 0x7fb668075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 >> 172.21.15.114:0/1490015790 conn(0x7fb6680ff440 msgr2=0x7fb668106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 shutdown_connections 2024-09-21T06:58:44.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:44.385+0000 7fb66ce00640 1 -- 172.21.15.114:0/1490015790 wait complete. 2024-09-21T06:58:44.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:44 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:45.285 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:45 smithi114 ceph-mon[30654]: pgmap v534: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.9 MiB/s wr, 547 op/s 2024-09-21T06:58:45.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:45 smithi183 ceph-mon[34219]: pgmap v534: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.9 MiB/s wr, 547 op/s 2024-09-21T06:58:45.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:45 smithi193 ceph-mon[34601]: pgmap v534: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 4.9 MiB/s wr, 547 op/s 2024-09-21T06:58:45.448 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:58:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.808+0000 7f3a2f400640 1 -- 172.21.15.114:0/1958369294 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac9c2260b0 con 0x55ac9bea21c0 2024-09-21T06:58:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.808+0000 7f3a2f400640 10 client.26777 handle_command_reply: tid=0 2024-09-21T06:58:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.808+0000 7f3a5b868740 1 client.26777 shutdown 2024-09-21T06:58:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.808+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 msgr2=0x55ac9c2c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.808+0000 7f3a5b868740 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2c3f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a400b25b0 tx=0x7f3a400ace20 comp rx=0 tx=0).stop 2024-09-21T06:58:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 shutdown_connections 2024-09-21T06:58:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a5b868740 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ac9bea21c0 0x55ac9c33ca40 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a5b868740 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac9bf0e1b0 0x55ac9c332cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a5b868740 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac9beed4d0 0x55ac9c2c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a5b868740 1 --2- 172.21.15.114:0/1958369294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac9bee52f0 0x55ac9c2c3f50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 >> 172.21.15.114:0/1958369294 conn(0x55ac9bf02300 msgr2=0x55ac9c2af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.809+0000 7f3a2f400640 0 client.26777 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.810+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 shutdown_connections 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.810+0000 7f3a5b868740 1 -- 172.21.15.114:0/1958369294 wait complete. 2024-09-21T06:58:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.810+0000 7f3a5b868740 20 client.26777 trim_cache size 0 max 16384 2024-09-21T06:58:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:58:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:58:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:58:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3a340b65c0 msgr2=0x7f3a340b8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3a340b65c0 0x7f3a340b8ad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c011d10 tx=0x7f3a3c005850 comp rx=0 tx=0).stop 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 msgr2=0x7f3a4c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c102660 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000e9f0 tx=0x7f3a4000edf0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 shutdown_connections 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a4c10dd90 0x7f3a4c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3a340b65c0 0x7f3a340b8ad0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a4c105280 0x7f3a4c102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 --2- 172.21.15.114:0/279204414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a4c104930 0x7f3a4c102660 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.812+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 >> 172.21.15.114:0/279204414 conn(0x7f3a4c0fe3c0 msgr2=0x7f3a4c107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.814+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 shutdown_connections 2024-09-21T06:58:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:45.814+0000 7f3a54800640 1 -- 172.21.15.114:0/279204414 wait complete. 2024-09-21T06:58:46.093 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:47.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:47 smithi114 ceph-mon[30654]: pgmap v535: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-09-21T06:58:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:47 smithi183 ceph-mon[34219]: pgmap v535: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-09-21T06:58:47.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:47 smithi193 ceph-mon[34601]: pgmap v535: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.3 MiB/s wr, 464 op/s 2024-09-21T06:58:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 250.0 s, 503.4 tps, lat 422.624 ms stddev 1713.246 2024-09-21T06:58:48.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 -- 172.21.15.114:0/2629060040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2480757b0 msgr2=0x7fd248075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 --2- 172.21.15.114:0/2629060040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2480757b0 0x7fd248075bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd238009900 tx=0x7fd23802a420 comp rx=0 tx=0).stop 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 -- 172.21.15.114:0/2629060040 shutdown_connections 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 --2- 172.21.15.114:0/2629060040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248110120 0x7fd248112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 --2- 172.21.15.114:0/2629060040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd248076100 0x7fd24810fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 --2- 172.21.15.114:0/2629060040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2480757b0 0x7fd248075bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.565+0000 7fd24d200640 1 -- 172.21.15.114:0/2629060040 >> 172.21.15.114:0/2629060040 conn(0x7fd24807b910 msgr2=0x7fd24807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.566+0000 7fd24d200640 1 -- 172.21.15.114:0/2629060040 shutdown_connections 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.566+0000 7fd24d200640 1 -- 172.21.15.114:0/2629060040 wait complete. 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 Processor -- start 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 -- start start 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2480757b0 0x7fd2481075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248076100 0x7fd248103890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 0x7fd248103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd248115d40 con 0x7fd2480757b0 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd248115bc0 con 0x7fd248076100 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.567+0000 7fd24d200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd248115ec0 con 0x7fd248110120 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 0x7fd248103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 0x7fd248103fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42416/0 (socket says 172.21.15.114:42416) 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 -- 172.21.15.114:0/2850722223 learned_addr learned my addr 172.21.15.114:0/2850722223 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd23fe00640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248076100 0x7fd248103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd245e00640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2480757b0 0x7fd2481075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 -- 172.21.15.114:0/2850722223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248076100 msgr2=0x7fd248103890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248076100 0x7fd248103890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 -- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2480757b0 msgr2=0x7fd2481075b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2480757b0 0x7fd2481075b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.568+0000 7fd246800640 1 -- 172.21.15.114:0/2850722223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2481acca0 con 0x7fd248110120 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.569+0000 7fd246800640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 0x7fd248103fb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b990 tx=0x7fd24000be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:48.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.571+0000 7fd23ea00640 1 -- 172.21.15.114:0/2850722223 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240019070 con 0x7fd248110120 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.571+0000 7fd23ea00640 1 -- 172.21.15.114:0/2850722223 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd24000cba0 con 0x7fd248110120 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.571+0000 7fd23ea00640 1 -- 172.21.15.114:0/2850722223 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400075d0 con 0x7fd248110120 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.571+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2481aced0 con 0x7fd248110120 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.571+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2481ad270 con 0x7fd248110120 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.590+0000 7fd253fe1740 1 Processor -- start 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.593+0000 7fd23ea00640 1 -- 172.21.15.114:0/2850722223 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd24001c070 con 0x7fd248110120 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 -- start start 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c34070 0x558d82fdada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 0x558d82fdb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fdb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d82f51fb0 con 0x558d82c34070 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558d82f413e0 con 0x558d82c0b1b0 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.597+0000 7fd253fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558d82f451f0 con 0x558d82c13390 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.598+0000 7fd23ea00640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd22c0b6780 0x7fd22c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.599+0000 7fd245e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 0x558d82fdb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.600+0000 7fd245e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 0x558d82fdb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35378/0 (socket says 172.21.15.114:35378) 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.600+0000 7fd245e00640 1 -- 172.21.15.114:0/320399810 learned_addr learned my addr 172.21.15.114:0/320399810 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.599+0000 7fd23ea00640 1 -- 172.21.15.114:0/2850722223 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd2400e6fe0 con 0x7fd248110120 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.599+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fdb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.600+0000 7fd246800640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd22c0b6780 0x7fd22c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.601+0000 7fd246800640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd22c0b6780 0x7fd22c0b8c90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c6a0 tx=0x7fd24000c610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.601+0000 7fd23fe00640 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c34070 0x558d82fdada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.602+0000 7fd245e00640 1 -- 172.21.15.114:0/320399810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 msgr2=0x558d82fdb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.603+0000 7fd245e00640 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fdb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.604+0000 7fd245e00640 1 -- 172.21.15.114:0/320399810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c34070 msgr2=0x558d82fdada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.604+0000 7fd245e00640 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c34070 0x558d82fdada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.604+0000 7fd245e00640 1 -- 172.21.15.114:0/320399810 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d82f0bcd0 con 0x558d82c0b1b0 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.604+0000 7fd23fe00640 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c34070 0x558d82fdada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.605+0000 7fd245e00640 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 0x558d82fdb2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd238004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77810d5215209287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.606+0000 7fd237400640 1 -- 172.21.15.114:0/320399810 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238035040 con 0x558d82c0b1b0 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.606+0000 7fd237400640 1 -- 172.21.15.114:0/320399810 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd238004770 con 0x558d82c0b1b0 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.606+0000 7fd237400640 1 -- 172.21.15.114:0/320399810 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238003850 con 0x558d82c0b1b0 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 -- 172.21.15.114:0/320399810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 msgr2=0x558d82fdb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 0x558d82fdb2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd238004390 comp rx=0 tx=0).stop 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 -- 172.21.15.114:0/320399810 shutdown_connections 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fdb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c0b1b0 0x558d82fdb2e0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd238004390 comp rx=0 tx=0).stop 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 --2- 172.21.15.114:0/320399810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c34070 0x558d82fdada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.608+0000 7fd253fe1740 1 -- 172.21.15.114:0/320399810 >> 172.21.15.114:0/320399810 conn(0x558d82c28460 msgr2=0x558d82fd8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.609+0000 7fd253fe1740 1 -- 172.21.15.114:0/320399810 shutdown_connections 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.609+0000 7fd253fe1740 1 -- 172.21.15.114:0/320399810 wait complete. 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.610+0000 7fd253fe1740 1 -- start start 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.610+0000 7fd253fe1740 1 Processor -- start 2024-09-21T06:58:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.612+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fe6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.612+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 0x558d82fe6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fe6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fe6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42428/0 (socket says 172.21.15.114:42428) 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd245e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 0x558d82fe6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd245e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 0x558d82fe6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35382/0 (socket says 172.21.15.114:35382) 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82ff2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd253fe1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558d82da2700 con 0x558d82bf29f0 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd253fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558d82edc580 con 0x558d82c34070 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd253fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558d82fe9520 con 0x558d82c13390 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.613+0000 7fd253fe1740 10 client.0 _ll_register_callbacks cb 0x558d82eec470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.614+0000 7fd245e00640 1 -- 172.21.15.114:0/391577724 learned_addr learned my addr 172.21.15.114:0/391577724 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.614+0000 7fd23fe00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82ff2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.614+0000 7fd23fe00640 1 -- 172.21.15.114:0/391577724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 msgr2=0x558d82fe6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.617+0000 7fd23fe00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 0x558d82fe6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.617+0000 7fd23fe00640 1 -- 172.21.15.114:0/391577724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 msgr2=0x558d82ff2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.617+0000 7fd23fe00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82ff2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.617+0000 7fd23fe00640 1 -- 172.21.15.114:0/391577724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d82c9f650 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.617+0000 7fd245e00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 0x558d82fe6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.618+0000 7fd22be00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.618+0000 7fd23fe00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fe6660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd248111df0 tx=0x7fd230004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bad66c13bb61d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.619+0000 7fd22be00640 1 -- 172.21.15.114:0/391577724 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230010b30 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.619+0000 7fd22be00640 1 -- 172.21.15.114:0/391577724 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd230010cd0 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.619+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558d82edeb90 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.620+0000 7fd22be00640 1 -- 172.21.15.114:0/391577724 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd230004020 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.620+0000 7fd253fe1740 10 client.26767 fetch_fsmap learned FSMap version 38 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.620+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558d82ef9950 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.620+0000 7fd22be00640 1 -- 172.21.15.114:0/391577724 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558d82edeb90 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.620+0000 7fd22be00640 1 -- 172.21.15.114:0/391577724 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fd230012480 con 0x558d82c13390 2024-09-21T06:58:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.622+0000 7fd253fe1740 10 client.26767 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.622+0000 7fd253fe1740 10 client.26767 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.622+0000 7fd253fe1740 20 client.26767 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.622+0000 7fd253fe1740 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82bc7da0 0x558d82ff86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.622+0000 7fd253fe1740 4 client.26767 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.622+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d82f53f00 con 0x558d82bc7da0 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.623+0000 7fd245e00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82bc7da0 0x558d82ff86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.623+0000 7fd245e00640 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82bc7da0 0x558d82ff86e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fc3a3d7ab64f8b6a server_cookie=2e43d7ad96fad368 in_seq=0 out_seq=0 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.623+0000 7fd22be00640 10 client.26767 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.625+0000 7fd22be00640 1 -- 172.21.15.114:0/391577724 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d82f53f00 con 0x558d82bc7da0 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.625+0000 7fd22be00640 10 client.26767 handle_command_reply: tid=0 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.628+0000 7fd253fe1740 1 client.26767 shutdown 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 msgr2=0x558d82fe6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fe6660 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd248111df0 tx=0x7fd230004730 comp rx=0 tx=0).stop 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 shutdown_connections 2024-09-21T06:58:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82bc7da0 0x558d82ff86e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82ff2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd22be00640 0 client.26767 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c34070 0x558d82fe6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 --2- 172.21.15.114:0/391577724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82c13390 0x558d82fe6660 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 >> 172.21.15.114:0/391577724 conn(0x558d82c28460 msgr2=0x558d82fdbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 shutdown_connections 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.629+0000 7fd253fe1740 1 -- 172.21.15.114:0/391577724 wait complete. 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.630+0000 7fd253fe1740 20 client.26767 trim_cache size 0 max 16384 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.653+0000 7fd253fe1740 1 Processor -- start 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.653+0000 7fd253fe1740 1 -- start start 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.654+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82bc7da0 0x558d82fe8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.654+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 0x558d82ff2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.654+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c13390 0x558d82ff2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.654+0000 7fd253fe1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481b74a0 con 0x558d82c13390 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.654+0000 7fd253fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481b9070 con 0x558d82bc7da0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.654+0000 7fd253fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481b7b70 con 0x558d82bf29f0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 0x558d82ff2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 0x558d82ff2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42430/0 (socket says 172.21.15.114:42430) 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 -- 172.21.15.114:0/1709841839 learned_addr learned my addr 172.21.15.114:0/1709841839 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd23fe00640 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82bc7da0 0x558d82fe8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd23fe00640 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c13390 0x558d82ff2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 -- 172.21.15.114:0/1709841839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82bc7da0 msgr2=0x558d82fe8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82bc7da0 0x558d82fe8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 -- 172.21.15.114:0/1709841839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c13390 msgr2=0x558d82ff2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c13390 0x558d82ff2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd245e00640 1 -- 172.21.15.114:0/1709841839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d82fdd090 con 0x558d82bf29f0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.655+0000 7fd23fe00640 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c13390 0x558d82ff2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.656+0000 7fd245e00640 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 0x558d82ff2710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd23802aa10 tx=0x7fd238004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81f7f3037330d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.656+0000 7fd236a00640 1 -- 172.21.15.114:0/1709841839 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238035040 con 0x558d82bf29f0 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.656+0000 7fd236a00640 1 -- 172.21.15.114:0/1709841839 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd238004c20 con 0x558d82bf29f0 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.656+0000 7fd236a00640 1 -- 172.21.15.114:0/1709841839 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238031790 con 0x558d82bf29f0 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.657+0000 7fd253fe1740 1 -- 172.21.15.114:0/1709841839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 msgr2=0x558d82ff2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.657+0000 7fd253fe1740 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 0x558d82ff2710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd23802aa10 tx=0x7fd238004580 comp rx=0 tx=0).stop 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.657+0000 7fd253fe1740 1 -- 172.21.15.114:0/1709841839 shutdown_connections 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.657+0000 7fd253fe1740 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82c13390 0x558d82ff2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.657+0000 7fd253fe1740 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bf29f0 0x558d82ff2710 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.658+0000 7fd253fe1740 1 --2- 172.21.15.114:0/1709841839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82bc7da0 0x558d82fe8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.658+0000 7fd253fe1740 1 -- 172.21.15.114:0/1709841839 >> 172.21.15.114:0/1709841839 conn(0x558d82c28460 msgr2=0x558d82fe7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.658+0000 7fd253fe1740 1 -- 172.21.15.114:0/1709841839 shutdown_connections 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.658+0000 7fd253fe1740 1 -- 172.21.15.114:0/1709841839 wait complete. 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.658+0000 7fd253fe1740 1 -- start start 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 Processor -- start 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bc7da0 0x558d82feaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82feb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 0x558d83073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481b8970 con 0x558d82bf29f0 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481b8240 con 0x558d82c13390 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.659+0000 7fd253fe1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2481b5480 con 0x558d82bc7da0 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.660+0000 7fd253fe1740 10 client.0 _ll_register_callbacks cb 0x7fd23000bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.660+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bc7da0 0x558d82feaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.660+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 0x558d83073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.660+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 0x558d83073060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35388/0 (socket says 172.21.15.114:35388) 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.660+0000 7fd23fe00640 1 -- 172.21.15.114:0/2992147301 learned_addr learned my addr 172.21.15.114:0/2992147301 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.660+0000 7fd245e00640 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82feb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd23fe00640 1 -- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bc7da0 msgr2=0x558d82feaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd23fe00640 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bc7da0 0x558d82feaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd23fe00640 1 -- 172.21.15.114:0/2992147301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 msgr2=0x558d82feb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd23fe00640 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82feb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd23fe00640 1 -- 172.21.15.114:0/2992147301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d83073670 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd23fe00640 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 0x558d83073060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2300129c0 tx=0x7fd230004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb18f7ba7bff6c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.661+0000 7fd22be00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.662+0000 7fd22be00640 1 -- 172.21.15.114:0/2992147301 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230012c70 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.662+0000 7fd22be00640 1 -- 172.21.15.114:0/2992147301 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd230010b30 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.662+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd238004c20 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.662+0000 7fd22be00640 1 -- 172.21.15.114:0/2992147301 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd230010d30 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.662+0000 7fd253fe1740 10 client.26810 fetch_fsmap learned FSMap version 38 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.662+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558d83074890 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.663+0000 7fd22be00640 1 -- 172.21.15.114:0/2992147301 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd238004c20 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.663+0000 7fd22be00640 1 -- 172.21.15.114:0/2992147301 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 38) ==== 8382+0+0 (secure 0 0 0) 0x7fd23001e430 con 0x558d82c13390 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.663+0000 7fd253fe1740 10 client.26810 fetch_fsmap finished waiting for FSMap version 38 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.663+0000 7fd253fe1740 10 client.26810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.663+0000 7fd253fe1740 20 client.26810 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.663+0000 7fd253fe1740 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82c34070 0x558d83077080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.664+0000 7fd253fe1740 4 client.26810 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.664+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d82f4f9b0 con 0x558d82c34070 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.664+0000 7fd245e00640 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82c34070 0x558d83077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.667+0000 7fd245e00640 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82c34070 0x558d83077080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=595ea2884288373a server_cookie=cd1dd61f0f3524d9 in_seq=0 out_seq=0 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.667+0000 7fd22be00640 10 client.26810 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.668+0000 7fd22be00640 1 -- 172.21.15.114:0/2992147301 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d82f4f9b0 con 0x558d82c34070 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.668+0000 7fd22be00640 10 client.26810 handle_command_reply: tid=0 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.668+0000 7fd253fe1740 1 client.26810 shutdown 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 msgr2=0x558d83073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 0x558d83073060 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd2300129c0 tx=0x7fd230004730 comp rx=0 tx=0).stop 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 shutdown_connections 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x558d82c34070 0x558d83077080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d82c13390 0x558d83073060 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd22be00640 0 client.26810 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:58:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558d82bf29f0 0x558d82feb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 --2- 172.21.15.114:0/2992147301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558d82bc7da0 0x558d82feaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.669+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 >> 172.21.15.114:0/2992147301 conn(0x558d82c28460 msgr2=0x558d82fd7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.670+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 shutdown_connections 2024-09-21T06:58:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.670+0000 7fd253fe1740 1 -- 172.21.15.114:0/2992147301 wait complete. 2024-09-21T06:58:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.670+0000 7fd253fe1740 20 client.26810 trim_cache size 0 max 16384 2024-09-21T06:58:49.000 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.671+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd22c0b6780 msgr2=0x7fd22c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.671+0000 7fd24d200640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd22c0b6780 0x7fd22c0b8c90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c6a0 tx=0x7fd24000c610 comp rx=0 tx=0).stop 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.671+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 msgr2=0x7fd248103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.671+0000 7fd24d200640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 0x7fd248103fb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b990 tx=0x7fd24000be50 comp rx=0 tx=0).stop 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 shutdown_connections 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd248110120 0x7fd248103fb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd22c0b6780 0x7fd22c0b8c90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd248076100 0x7fd248103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 --2- 172.21.15.114:0/2850722223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd2480757b0 0x7fd2481075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 >> 172.21.15.114:0/2850722223 conn(0x7fd24807b910 msgr2=0x7fd248109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 shutdown_connections 2024-09-21T06:58:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:48.672+0000 7fd24d200640 1 -- 172.21.15.114:0/2850722223 wait complete. 2024-09-21T06:58:49.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:49 smithi114 ceph-mon[30654]: pgmap v536: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-09-21T06:58:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:49 smithi183 ceph-mon[34219]: pgmap v536: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-09-21T06:58:49.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:49 smithi193 ceph-mon[34601]: pgmap v536: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 4.8 MiB/s wr, 505 op/s 2024-09-21T06:58:49.890 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:58:50.569 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:51.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:51 smithi114 ceph-mon[30654]: pgmap v537: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.1 MiB/s wr, 519 op/s 2024-09-21T06:58:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:51 smithi183 ceph-mon[34219]: pgmap v537: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.1 MiB/s wr, 519 op/s 2024-09-21T06:58:51.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:51 smithi193 ceph-mon[34601]: pgmap v537: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.1 MiB/s wr, 519 op/s 2024-09-21T06:58:51.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.435+0000 7fb241000640 1 -- 172.21.15.114:0/3763590234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340a5360 msgr2=0x7fb2340a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.435+0000 7fb241000640 1 --2- 172.21.15.114:0/3763590234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340a5360 0x7fb2340a5770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb228009370 tx=0x7fb22802b640 comp rx=0 tx=0).stop 2024-09-21T06:58:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.438+0000 7fb241000640 1 -- 172.21.15.114:0/3763590234 shutdown_connections 2024-09-21T06:58:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.439+0000 7fb241000640 1 --2- 172.21.15.114:0/3763590234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340ab5f0 0x7fb2340b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.439+0000 7fb241000640 1 --2- 172.21.15.114:0/3763590234 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2340aaca0 0x7fb2340ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.439+0000 7fb241000640 1 --2- 172.21.15.114:0/3763590234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340a5360 0x7fb2340a5770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.439+0000 7fb241000640 1 -- 172.21.15.114:0/3763590234 >> 172.21.15.114:0/3763590234 conn(0x7fb23401d090 msgr2=0x7fb2340a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:51.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.441+0000 7fb241000640 1 -- 172.21.15.114:0/3763590234 shutdown_connections 2024-09-21T06:58:51.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.441+0000 7fb241000640 1 -- 172.21.15.114:0/3763590234 wait complete. 2024-09-21T06:58:51.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.443+0000 7fb241000640 1 Processor -- start 2024-09-21T06:58:51.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.443+0000 7fb241000640 1 -- start start 2024-09-21T06:58:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb241000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340a5360 0x7fb234144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb241000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2340aaca0 0x7fb234144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb241000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 0x7fb2341499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb241000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b4f50 con 0x7fb2340aaca0 2024-09-21T06:58:51.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb241000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b4dd0 con 0x7fb2340a5360 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb241000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2340b50d0 con 0x7fb2340ab5f0 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb23a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 0x7fb2341499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb23a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 0x7fb2341499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42466/0 (socket says 172.21.15.114:42466) 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.444+0000 7fb23a800640 1 -- 172.21.15.114:0/2228166040 learned_addr learned my addr 172.21.15.114:0/2228166040 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb239e00640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340a5360 0x7fb234144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb23a800640 1 -- 172.21.15.114:0/2228166040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340a5360 msgr2=0x7fb234144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb23a800640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340a5360 0x7fb234144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb23a800640 1 -- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2340aaca0 msgr2=0x7fb234144730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb239400640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2340aaca0 0x7fb234144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb23a800640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2340aaca0 0x7fb234144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb23a800640 1 -- 172.21.15.114:0/2228166040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23414a100 con 0x7fb2340ab5f0 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.445+0000 7fb239e00640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340a5360 0x7fb234144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.446+0000 7fb23a800640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 0x7fb2341499d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb23c02ebd0 tx=0x7fb23c06deb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.446+0000 7fb232a00640 1 -- 172.21.15.114:0/2228166040 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0729c0 con 0x7fb2340ab5f0 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.446+0000 7fb232a00640 1 -- 172.21.15.114:0/2228166040 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c072b60 con 0x7fb2340ab5f0 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.446+0000 7fb232a00640 1 -- 172.21.15.114:0/2228166040 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c0775e0 con 0x7fb2340ab5f0 2024-09-21T06:58:51.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.446+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23414a3f0 con 0x7fb2340ab5f0 2024-09-21T06:58:51.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.446+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb234150b50 con 0x7fb2340ab5f0 2024-09-21T06:58:51.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.454+0000 7fb247cad740 1 Processor -- start 2024-09-21T06:58:51.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.455+0000 7fb232a00640 1 -- 172.21.15.114:0/2228166040 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb23c07f030 con 0x7fb2340ab5f0 2024-09-21T06:58:51.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.457+0000 7fb247cad740 1 -- start start 2024-09-21T06:58:51.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.457+0000 7fb247cad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e2673f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.458+0000 7fb247cad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e26744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.458+0000 7fb247cad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e26749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.458+0000 7fb247cad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5628e25eafb0 con 0x5628e22a41b0 2024-09-21T06:58:51.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.458+0000 7fb247cad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5628e25da3e0 con 0x5628e22ac390 2024-09-21T06:58:51.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.458+0000 7fb247cad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5628e25de1f0 con 0x5628e22cd070 2024-09-21T06:58:51.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.459+0000 7fb239400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e26749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.459+0000 7fb239400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e26749f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35418/0 (socket says 172.21.15.114:35418) 2024-09-21T06:58:51.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.459+0000 7fb239400640 1 -- 172.21.15.114:0/2974568684 learned_addr learned my addr 172.21.15.114:0/2974568684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:51.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.460+0000 7fb239400640 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e2673f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.460+0000 7fb239400640 1 -- 172.21.15.114:0/2974568684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 msgr2=0x5628e2673f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.460+0000 7fb239400640 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e2673f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.460+0000 7fb239400640 1 -- 172.21.15.114:0/2974568684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 msgr2=0x5628e26744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.460+0000 7fb239400640 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e26744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.460+0000 7fb239400640 1 -- 172.21.15.114:0/2974568684 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e25a4cd0 con 0x5628e22ac390 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.461+0000 7fb232a00640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2200b66e0 0x7fb2200b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.461+0000 7fb232a00640 1 -- 172.21.15.114:0/2228166040 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb23c14bd80 con 0x7fb2340ab5f0 2024-09-21T06:58:51.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.461+0000 7fb239400640 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e26749f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00cf00 tx=0x7fb22c00c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97302bb4da4ca7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.461+0000 7fb239e00640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2200b66e0 0x7fb2200b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.462+0000 7fb227400640 1 -- 172.21.15.114:0/2974568684 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c01d070 con 0x5628e22ac390 2024-09-21T06:58:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.462+0000 7fb227400640 1 -- 172.21.15.114:0/2974568684 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c018b20 con 0x5628e22ac390 2024-09-21T06:58:51.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.462+0000 7fb227400640 1 -- 172.21.15.114:0/2974568684 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00f040 con 0x5628e22ac390 2024-09-21T06:58:51.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.463+0000 7fb247cad740 1 -- 172.21.15.114:0/2974568684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 msgr2=0x5628e26749f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.463+0000 7fb247cad740 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e26749f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00cf00 tx=0x7fb22c00c0f0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 -- 172.21.15.114:0/2974568684 shutdown_connections 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e26749f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e26744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 --2- 172.21.15.114:0/2974568684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e2673f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 -- 172.21.15.114:0/2974568684 >> 172.21.15.114:0/2974568684 conn(0x5628e22c1460 msgr2=0x5628e2671e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 -- 172.21.15.114:0/2974568684 shutdown_connections 2024-09-21T06:58:51.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.464+0000 7fb247cad740 1 -- 172.21.15.114:0/2974568684 wait complete. 2024-09-21T06:58:51.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.465+0000 7fb239e00640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2200b66e0 0x7fb2200b8bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5628e2672bf0 tx=0x7fb228009ee0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:51.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.466+0000 7fb247cad740 1 -- start start 2024-09-21T06:58:51.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.466+0000 7fb247cad740 1 Processor -- start 2024-09-21T06:58:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e267f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e267fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5628e243b700 con 0x5628e22a41b0 2024-09-21T06:58:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5628e2575580 con 0x5628e22ac390 2024-09-21T06:58:51.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5628e2682750 con 0x5628e22cd070 2024-09-21T06:58:51.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.467+0000 7fb247cad740 10 client.0 _ll_register_callbacks cb 0x5628e2585470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:51.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.468+0000 7fb239e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e267f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.468+0000 7fb239e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e267f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58786/0 (socket says 172.21.15.114:58786) 2024-09-21T06:58:51.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.468+0000 7fb239e00640 1 -- 172.21.15.114:0/3335938540 learned_addr learned my addr 172.21.15.114:0/3335938540 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:51.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.470+0000 7fb239400640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e267fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.470+0000 7fb23a800640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.471+0000 7fb239e00640 1 -- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 msgr2=0x5628e268bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.471+0000 7fb239e00640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.471+0000 7fb239e00640 1 -- 172.21.15.114:0/3335938540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 msgr2=0x5628e267fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.471+0000 7fb239e00640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e267fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.473+0000 7fb239e00640 1 -- 172.21.15.114:0/3335938540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e2338650 con 0x5628e22a41b0 2024-09-21T06:58:51.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.473+0000 7fb239e00640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e267f760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb228009370 tx=0x7fb22802b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecd09a6d337adb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:51.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.473+0000 7fb21be00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:58:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.474+0000 7fb21be00640 1 -- 172.21.15.114:0/3335938540 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22804c030 con 0x5628e22a41b0 2024-09-21T06:58:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.474+0000 7fb21be00640 1 -- 172.21.15.114:0/3335938540 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb22803cb90 con 0x5628e22a41b0 2024-09-21T06:58:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.474+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5628e2577b90 con 0x5628e22a41b0 2024-09-21T06:58:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.474+0000 7fb21be00640 1 -- 172.21.15.114:0/3335938540 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb228059050 con 0x5628e22a41b0 2024-09-21T06:58:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.475+0000 7fb247cad740 10 client.16617 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:51.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.475+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5628e2592950 con 0x5628e22a41b0 2024-09-21T06:58:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.475+0000 7fb21be00640 1 -- 172.21.15.114:0/3335938540 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5628e2577b90 con 0x5628e22a41b0 2024-09-21T06:58:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb21be00640 1 -- 172.21.15.114:0/3335938540 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fb2280503c0 con 0x5628e22a41b0 2024-09-21T06:58:51.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb247cad740 10 client.16617 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb247cad740 10 client.16617 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:58:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb247cad740 20 client.16617 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb247cad740 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e228b9f0 0x5628e2691970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb247cad740 4 client.16617 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.476+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e25ecf00 con 0x5628e228b9f0 2024-09-21T06:58:51.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.477+0000 7fb239400640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e228b9f0 0x5628e2691970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.478+0000 7fb239400640 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e228b9f0 0x5628e2691970 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f84ae7599a0d7d31 server_cookie=3a4d1d13502c213d in_seq=0 out_seq=0 2024-09-21T06:58:51.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.479+0000 7fb21be00640 10 client.16617 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:58:51.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.480+0000 7fb21be00640 1 -- 172.21.15.114:0/3335938540 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628e25ecf00 con 0x5628e228b9f0 2024-09-21T06:58:51.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.480+0000 7fb21be00640 10 client.16617 handle_command_reply: tid=0 2024-09-21T06:58:51.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.480+0000 7fb247cad740 1 client.16617 shutdown 2024-09-21T06:58:51.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 msgr2=0x5628e267f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e267f760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb228009370 tx=0x7fb22802b640 comp rx=0 tx=0).stop 2024-09-21T06:58:51.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 shutdown_connections 2024-09-21T06:58:51.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e228b9f0 0x5628e2691970 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e267fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 --2- 172.21.15.114:0/3335938540 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e267f760 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb21be00640 0 client.16617 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:58:51.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.481+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 >> 172.21.15.114:0/3335938540 conn(0x5628e22c1460 msgr2=0x5628e2677100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:51.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.482+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 shutdown_connections 2024-09-21T06:58:51.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.482+0000 7fb247cad740 1 -- 172.21.15.114:0/3335938540 wait complete. 2024-09-21T06:58:51.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.482+0000 7fb247cad740 20 client.16617 trim_cache size 0 max 16384 2024-09-21T06:58:51.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.504+0000 7fb247cad740 1 Processor -- start 2024-09-21T06:58:51.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.504+0000 7fb247cad740 1 -- start start 2024-09-21T06:58:51.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.505+0000 7fb247cad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2670d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.505+0000 7fb247cad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e268b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.505+0000 7fb247cad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.505+0000 7fb247cad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb23415aa70 con 0x5628e22a41b0 2024-09-21T06:58:51.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.505+0000 7fb247cad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb23415c6d0 con 0x5628e22ac390 2024-09-21T06:58:51.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.505+0000 7fb247cad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb23415b140 con 0x5628e22cd070 2024-09-21T06:58:51.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.506+0000 7fb239400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e268b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.506+0000 7fb239400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e268b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35434/0 (socket says 172.21.15.114:35434) 2024-09-21T06:58:51.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.506+0000 7fb239400640 1 -- 172.21.15.114:0/680983066 learned_addr learned my addr 172.21.15.114:0/680983066 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.506+0000 7fb23a800640 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.506+0000 7fb239e00640 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2670d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb239400640 1 -- 172.21.15.114:0/680983066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 msgr2=0x5628e268bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb239400640 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb239400640 1 -- 172.21.15.114:0/680983066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 msgr2=0x5628e2670d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb239400640 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2670d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb239400640 1 -- 172.21.15.114:0/680983066 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e2676260 con 0x5628e22ac390 2024-09-21T06:58:51.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb23a800640 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:51.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.507+0000 7fb239400640 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e268b9d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb22c016010 tx=0x7fb22c016410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca2356b503990f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:51.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.508+0000 7fb226a00640 1 -- 172.21.15.114:0/680983066 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c01d040 con 0x5628e22ac390 2024-09-21T06:58:51.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.508+0000 7fb226a00640 1 -- 172.21.15.114:0/680983066 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c006b20 con 0x5628e22ac390 2024-09-21T06:58:51.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.508+0000 7fb226a00640 1 -- 172.21.15.114:0/680983066 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c018470 con 0x5628e22ac390 2024-09-21T06:58:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.509+0000 7fb247cad740 1 -- 172.21.15.114:0/680983066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 msgr2=0x5628e268b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.509+0000 7fb247cad740 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e268b9d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb22c016010 tx=0x7fb22c016410 comp rx=0 tx=0).stop 2024-09-21T06:58:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.510+0000 7fb247cad740 1 -- 172.21.15.114:0/680983066 shutdown_connections 2024-09-21T06:58:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.510+0000 7fb247cad740 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e268bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.510+0000 7fb247cad740 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e268b9d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.510+0000 7fb247cad740 1 --2- 172.21.15.114:0/680983066 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2670d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.510+0000 7fb247cad740 1 -- 172.21.15.114:0/680983066 >> 172.21.15.114:0/680983066 conn(0x5628e22c1460 msgr2=0x5628e2670600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:51.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.511+0000 7fb247cad740 1 -- 172.21.15.114:0/680983066 shutdown_connections 2024-09-21T06:58:51.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.511+0000 7fb247cad740 1 -- 172.21.15.114:0/680983066 wait complete. 2024-09-21T06:58:51.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.512+0000 7fb247cad740 1 -- start start 2024-09-21T06:58:51.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.512+0000 7fb247cad740 1 Processor -- start 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2687a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e2687f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e26829f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb23415bf40 con 0x5628e22a41b0 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb23415b810 con 0x5628e22ac390 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb234158a50 con 0x5628e22cd070 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb247cad740 10 client.0 _ll_register_callbacks cb 0x7fb228032960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb239e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2687a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb239e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2687a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58802/0 (socket says 172.21.15.114:58802) 2024-09-21T06:58:51.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb239e00640 1 -- 172.21.15.114:0/3184892052 learned_addr learned my addr 172.21.15.114:0/3184892052 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb239400640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e2687f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.513+0000 7fb23a800640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e26829f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb239e00640 1 -- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 msgr2=0x5628e26829f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb239e00640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e26829f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb239e00640 1 -- 172.21.15.114:0/3184892052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 msgr2=0x5628e2687f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb239e00640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e2687f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb239e00640 1 -- 172.21.15.114:0/3184892052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e2682f30 con 0x5628e22a41b0 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb239400640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e2687f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.514+0000 7fb23a800640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e26829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.515+0000 7fb239e00640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2687a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb228009370 tx=0x7fb228050670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9426cb7d2d5a2ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:51.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.515+0000 7fb21be00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:58:51.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.516+0000 7fb21be00640 1 -- 172.21.15.114:0/3184892052 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228062060 con 0x5628e22a41b0 2024-09-21T06:58:51.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.517+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb22c006b20 con 0x5628e22a41b0 2024-09-21T06:58:51.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.517+0000 7fb21be00640 1 -- 172.21.15.114:0/3184892052 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb22800d100 con 0x5628e22a41b0 2024-09-21T06:58:51.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.518+0000 7fb21be00640 1 -- 172.21.15.114:0/3184892052 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb228062060 con 0x5628e22a41b0 2024-09-21T06:58:51.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.518+0000 7fb247cad740 10 client.16623 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:51.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.518+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5628e26fca50 con 0x5628e22a41b0 2024-09-21T06:58:51.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb21be00640 1 -- 172.21.15.114:0/3184892052 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c006b20 con 0x5628e22a41b0 2024-09-21T06:58:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb21be00640 1 -- 172.21.15.114:0/3184892052 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fb2280485d0 con 0x5628e22a41b0 2024-09-21T06:58:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb247cad740 10 client.16623 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb247cad740 10 client.16623 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:58:51.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb247cad740 20 client.16623 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb247cad740 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e2260da0 0x5628e26feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb247cad740 4 client.16623 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:58:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.519+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5628e25e89b0 con 0x5628e2260da0 2024-09-21T06:58:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.520+0000 7fb239400640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e2260da0 0x5628e26feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:51.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.521+0000 7fb239400640 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e2260da0 0x5628e26feff0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dec0c3190798d36 server_cookie=c6568db59f8c32e6 in_seq=0 out_seq=0 2024-09-21T06:58:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.521+0000 7fb21be00640 10 client.16623 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:58:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.521+0000 7fb21be00640 1 -- 172.21.15.114:0/3184892052 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5628e25e89b0 con 0x5628e2260da0 2024-09-21T06:58:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.521+0000 7fb21be00640 10 client.16623 handle_command_reply: tid=0 2024-09-21T06:58:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.521+0000 7fb247cad740 1 client.16623 shutdown 2024-09-21T06:58:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 msgr2=0x5628e2687a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2687a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb228009370 tx=0x7fb228050670 comp rx=0 tx=0).stop 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 shutdown_connections 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5628e2260da0 0x5628e26feff0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5628e22cd070 0x5628e26829f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5628e22ac390 0x5628e2687f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 --2- 172.21.15.114:0/3184892052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5628e22a41b0 0x5628e2687a40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb21be00640 0 client.16623 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:58:51.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 >> 172.21.15.114:0/3184892052 conn(0x5628e22c1460 msgr2=0x5628e2670600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:51.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.522+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 shutdown_connections 2024-09-21T06:58:51.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.523+0000 7fb247cad740 1 -- 172.21.15.114:0/3184892052 wait complete. 2024-09-21T06:58:51.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.523+0000 7fb247cad740 20 client.16623 trim_cache size 0 max 16384 2024-09-21T06:58:51.526 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2200b66e0 msgr2=0x7fb2200b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2200b66e0 0x7fb2200b8bf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x5628e2672bf0 tx=0x7fb228009ee0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 msgr2=0x7fb2341499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 0x7fb2341499d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb23c02ebd0 tx=0x7fb23c06deb0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 shutdown_connections 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2340ab5f0 0x7fb2341499d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2340aaca0 0x7fb234144730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2200b66e0 0x7fb2200b8bf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 --2- 172.21.15.114:0/2228166040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2340a5360 0x7fb234144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.524+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 >> 172.21.15.114:0/2228166040 conn(0x7fb23401d090 msgr2=0x7fb2340aa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.525+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 shutdown_connections 2024-09-21T06:58:51.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:51.525+0000 7fb241000640 1 -- 172.21.15.114:0/2228166040 wait complete. 2024-09-21T06:58:52.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:52 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:52.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:52 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:52.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:52 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:58:52.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:58:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 255.0 s, 1067.0 tps, lat 93.468 ms stddev 122.470 2024-09-21T06:58:53.123 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:53.353 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:53 smithi114 ceph-mon[30654]: pgmap v538: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.6 MiB/s wr, 444 op/s 2024-09-21T06:58:53.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:53 smithi183 ceph-mon[34219]: pgmap v538: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.6 MiB/s wr, 444 op/s 2024-09-21T06:58:53.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:53 smithi193 ceph-mon[34601]: pgmap v538: 129 pgs: 129 active+clean; 2.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 4.6 MiB/s wr, 444 op/s 2024-09-21T06:58:54.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.020+0000 7efe3f200640 1 -- 172.21.15.114:0/2846808082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 msgr2=0x7efe401029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.020+0000 7efe3f200640 1 --2- 172.21.15.114:0/2846808082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe401029c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efe30009930 tx=0x7efe3002a420 comp rx=0 tx=0).stop 2024-09-21T06:58:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.022+0000 7efe3f200640 1 -- 172.21.15.114:0/2846808082 shutdown_connections 2024-09-21T06:58:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.022+0000 7efe3f200640 1 --2- 172.21.15.114:0/2846808082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe401079f0 0x7efe40107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.022+0000 7efe3f200640 1 --2- 172.21.15.114:0/2846808082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe40102f00 0x7efe40107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.022+0000 7efe3f200640 1 --2- 172.21.15.114:0/2846808082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe401029c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.022+0000 7efe3f200640 1 -- 172.21.15.114:0/2846808082 >> 172.21.15.114:0/2846808082 conn(0x7efe400fe2c0 msgr2=0x7efe40100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:54.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.026+0000 7efe3f200640 1 -- 172.21.15.114:0/2846808082 shutdown_connections 2024-09-21T06:58:54.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.026+0000 7efe3f200640 1 -- 172.21.15.114:0/2846808082 wait complete. 2024-09-21T06:58:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.029+0000 7efe3f200640 1 Processor -- start 2024-09-21T06:58:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.031+0000 7efe3f200640 1 -- start start 2024-09-21T06:58:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.031+0000 7efe3f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe40116ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.031+0000 7efe3f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe40102f00 0x7efe4010fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe401079f0 0x7efe40110330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe40119ee0 con 0x7efe401079f0 2024-09-21T06:58:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe40119d60 con 0x7efe401025b0 2024-09-21T06:58:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe4011a060 con 0x7efe40102f00 2024-09-21T06:58:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe40116ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe40116ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35450/0 (socket says 172.21.15.114:35450) 2024-09-21T06:58:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3de00640 1 -- 172.21.15.114:0/1536258326 learned_addr learned my addr 172.21.15.114:0/1536258326 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.032+0000 7efe3d400640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe40102f00 0x7efe4010fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.034+0000 7efe3de00640 1 -- 172.21.15.114:0/1536258326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe40102f00 msgr2=0x7efe4010fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.034+0000 7efe3de00640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe40102f00 0x7efe4010fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.034+0000 7efe3de00640 1 -- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe401079f0 msgr2=0x7efe40110330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.034+0000 7efe3de00640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe401079f0 0x7efe40110330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.034+0000 7efe3de00640 1 -- 172.21.15.114:0/1536258326 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe40110b50 con 0x7efe401025b0 2024-09-21T06:58:54.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.035+0000 7efe3de00640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe40116ac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe30005ab0 tx=0x7efe30003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:54.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.035+0000 7efe36a00640 1 -- 172.21.15.114:0/1536258326 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3003f070 con 0x7efe401025b0 2024-09-21T06:58:54.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.035+0000 7efe36a00640 1 -- 172.21.15.114:0/1536258326 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe30031b50 con 0x7efe401025b0 2024-09-21T06:58:54.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.035+0000 7efe36a00640 1 -- 172.21.15.114:0/1536258326 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30035030 con 0x7efe401025b0 2024-09-21T06:58:54.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.037+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe40115790 con 0x7efe401025b0 2024-09-21T06:58:54.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.037+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe40115c20 con 0x7efe401025b0 2024-09-21T06:58:54.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.046+0000 7efe4be20740 1 Processor -- start 2024-09-21T06:58:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.055+0000 7efe36a00640 1 -- 172.21.15.114:0/1536258326 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7efe30042070 con 0x7efe401025b0 2024-09-21T06:58:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 -- start start 2024-09-21T06:58:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f338f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56042f2affb0 con 0x56042ef691b0 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56042f29f3e0 con 0x56042ef92070 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.056+0000 7efe4be20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56042f2a31f0 con 0x56042ef71390 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.057+0000 7efe3d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.057+0000 7efe3d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42532/0 (socket says 172.21.15.114:42532) 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.057+0000 7efe3d400640 1 -- 172.21.15.114:0/1913257051 learned_addr learned my addr 172.21.15.114:0/1913257051 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.057+0000 7efe3e800640 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f338f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.058+0000 7efe3d400640 1 -- 172.21.15.114:0/1913257051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 msgr2=0x56042f338f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.058+0000 7efe3d400640 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f338f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.058+0000 7efe3d400640 1 -- 172.21.15.114:0/1913257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 msgr2=0x56042f3394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.058+0000 7efe3d400640 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.058+0000 7efe3d400640 1 -- 172.21.15.114:0/1913257051 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042f269cd0 con 0x56042ef71390 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.059+0000 7efe3d400640 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3399f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009c20 tx=0x7efe2c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c609d3b649c11a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.059+0000 7efe2b400640 1 -- 172.21.15.114:0/1913257051 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c004030 con 0x56042ef71390 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.060+0000 7efe2b400640 1 -- 172.21.15.114:0/1913257051 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe2c005020 con 0x56042ef71390 2024-09-21T06:58:54.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.060+0000 7efe2b400640 1 -- 172.21.15.114:0/1913257051 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c004ca0 con 0x56042ef71390 2024-09-21T06:58:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.060+0000 7efe36a00640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efe240b6780 0x7efe240b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.060+0000 7efe36a00640 1 -- 172.21.15.114:0/1536258326 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7efe30108110 con 0x7efe401025b0 2024-09-21T06:58:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.061+0000 7efe4be20740 1 -- 172.21.15.114:0/1913257051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 msgr2=0x56042f3399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.061+0000 7efe4be20740 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3399f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009c20 tx=0x7efe2c011810 comp rx=0 tx=0).stop 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 -- 172.21.15.114:0/1913257051 shutdown_connections 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3399f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 --2- 172.21.15.114:0/1913257051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f338f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 -- 172.21.15.114:0/1913257051 >> 172.21.15.114:0/1913257051 conn(0x56042ef86460 msgr2=0x56042f336e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 -- 172.21.15.114:0/1913257051 shutdown_connections 2024-09-21T06:58:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.064+0000 7efe4be20740 1 -- 172.21.15.114:0/1913257051 wait complete. 2024-09-21T06:58:54.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.067+0000 7efe4be20740 1 -- start start 2024-09-21T06:58:54.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.067+0000 7efe4be20740 1 Processor -- start 2024-09-21T06:58:54.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.068+0000 7efe3e800640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efe240b6780 0x7efe240b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef691b0 0x56042f3447b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef71390 0x56042f344d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f350a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56042f100700 con 0x56042ef71390 2024-09-21T06:58:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56042f23a580 con 0x56042ef691b0 2024-09-21T06:58:54.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56042f347750 con 0x56042ef92070 2024-09-21T06:58:54.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.070+0000 7efe4be20740 10 client.0 _ll_register_callbacks cb 0x56042f24a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.072+0000 7efe3de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef691b0 0x56042f3447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.073+0000 7efe3e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f350a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.073+0000 7efe3e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f350a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42538/0 (socket says 172.21.15.114:42538) 2024-09-21T06:58:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.073+0000 7efe3e800640 1 -- 172.21.15.114:0/276368880 learned_addr learned my addr 172.21.15.114:0/276368880 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.073+0000 7efe3e800640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efe240b6780 0x7efe240b8c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efe40117210 tx=0x7efe38009b00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.073+0000 7efe3d400640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef71390 0x56042f344d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.074+0000 7efe3de00640 1 -- 172.21.15.114:0/276368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 msgr2=0x56042f350a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.074+0000 7efe3de00640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f350a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.075+0000 7efe3de00640 1 -- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef71390 msgr2=0x56042f344d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.075+0000 7efe3de00640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef71390 0x56042f344d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.075+0000 7efe3de00640 1 -- 172.21.15.114:0/276368880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042effd650 con 0x56042ef691b0 2024-09-21T06:58:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.075+0000 7efe3d400640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef71390 0x56042f344d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.075+0000 7efe3e800640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f350a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.076+0000 7efe3de00640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef691b0 0x56042f3447b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe30105490 tx=0x7efe30009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38cf1e9f688b362d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.076+0000 7efe1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.080+0000 7efe1fe00640 1 -- 172.21.15.114:0/276368880 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe300c9300 con 0x56042ef691b0 2024-09-21T06:58:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.080+0000 7efe1fe00640 1 -- 172.21.15.114:0/276368880 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7efe300f10b0 con 0x56042ef691b0 2024-09-21T06:58:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.080+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56042f23cb90 con 0x56042ef691b0 2024-09-21T06:58:54.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.088+0000 7efe1fe00640 1 -- 172.21.15.114:0/276368880 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe300cd6a0 con 0x56042ef691b0 2024-09-21T06:58:54.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.090+0000 7efe4be20740 10 client.26837 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:54.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.090+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56042f257950 con 0x56042ef691b0 2024-09-21T06:58:54.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe1fe00640 1 -- 172.21.15.114:0/276368880 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56042f23cb90 con 0x56042ef691b0 2024-09-21T06:58:54.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe1fe00640 1 -- 172.21.15.114:0/276368880 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7efe30105be0 con 0x56042ef691b0 2024-09-21T06:58:54.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe4be20740 10 client.26837 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:54.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe4be20740 10 client.26837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:58:54.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe4be20740 20 client.26837 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:54.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe4be20740 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef509f0 0x56042f356900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe4be20740 4 client.26837 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:54.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.091+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56042f2b1f00 con 0x56042ef509f0 2024-09-21T06:58:54.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.092+0000 7efe3d400640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef509f0 0x56042f356900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.095+0000 7efe3d400640 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef509f0 0x56042f356900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2a54cd0da1a3dea server_cookie=4f1b06de3254ede9 in_seq=0 out_seq=0 2024-09-21T06:58:54.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.095+0000 7efe1fe00640 10 client.26837 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:58:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.101+0000 7efe1fe00640 1 -- 172.21.15.114:0/276368880 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56042f2b1f00 con 0x56042ef509f0 2024-09-21T06:58:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.101+0000 7efe1fe00640 10 client.26837 handle_command_reply: tid=0 2024-09-21T06:58:54.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 client.26837 shutdown 2024-09-21T06:58:54.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef691b0 msgr2=0x56042f3447b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef691b0 0x56042f3447b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7efe30105490 tx=0x7efe30009260 comp rx=0 tx=0).stop 2024-09-21T06:58:54.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 shutdown_connections 2024-09-21T06:58:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef509f0 0x56042f356900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f350a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef71390 0x56042f344d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 --2- 172.21.15.114:0/276368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef691b0 0x56042f3447b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.102+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 >> 172.21.15.114:0/276368880 conn(0x56042ef86460 msgr2=0x56042f335600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.103+0000 7efe1fe00640 0 client.26837 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:58:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.103+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 shutdown_connections 2024-09-21T06:58:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.103+0000 7efe4be20740 1 -- 172.21.15.114:0/276368880 wait complete. 2024-09-21T06:58:54.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.103+0000 7efe4be20740 20 client.26837 trim_cache size 0 max 16384 2024-09-21T06:58:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 Processor -- start 2024-09-21T06:58:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 -- start start 2024-09-21T06:58:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3436c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f350970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe401bfa60 con 0x56042ef691b0 2024-09-21T06:58:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe401c16c0 con 0x56042ef92070 2024-09-21T06:58:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.137+0000 7efe4be20740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe401c0130 con 0x56042ef71390 2024-09-21T06:58:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.138+0000 7efe3d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.138+0000 7efe3d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42552/0 (socket says 172.21.15.114:42552) 2024-09-21T06:58:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.138+0000 7efe3d400640 1 -- 172.21.15.114:0/1801854457 learned_addr learned my addr 172.21.15.114:0/1801854457 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:54.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.139+0000 7efe3e800640 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f350970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3de00640 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3d400640 1 -- 172.21.15.114:0/1801854457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 msgr2=0x56042f350970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3d400640 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f350970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3d400640 1 -- 172.21.15.114:0/1801854457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 msgr2=0x56042f3436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3d400640 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3d400640 1 -- 172.21.15.114:0/1801854457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042f33b260 con 0x56042ef71390 2024-09-21T06:58:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3de00640 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.140+0000 7efe3d400640 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3463e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097e0 tx=0x7efe2c004350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66316b0954103cab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.141+0000 7efe2aa00640 1 -- 172.21.15.114:0/1801854457 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c031070 con 0x56042ef71390 2024-09-21T06:58:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.141+0000 7efe2aa00640 1 -- 172.21.15.114:0/1801854457 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe2c003aa0 con 0x56042ef71390 2024-09-21T06:58:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.141+0000 7efe2aa00640 1 -- 172.21.15.114:0/1801854457 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c01e040 con 0x56042ef71390 2024-09-21T06:58:54.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 -- 172.21.15.114:0/1801854457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 msgr2=0x56042f3463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3463e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097e0 tx=0x7efe2c004350 comp rx=0 tx=0).stop 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 -- 172.21.15.114:0/1801854457 shutdown_connections 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef92070 0x56042f350970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef71390 0x56042f3463e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 --2- 172.21.15.114:0/1801854457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f3436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.142+0000 7efe4be20740 1 -- 172.21.15.114:0/1801854457 >> 172.21.15.114:0/1801854457 conn(0x56042ef86460 msgr2=0x56042f345cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.143+0000 7efe4be20740 1 -- 172.21.15.114:0/1801854457 shutdown_connections 2024-09-21T06:58:54.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.143+0000 7efe4be20740 1 -- 172.21.15.114:0/1801854457 wait complete. 2024-09-21T06:58:54.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.143+0000 7efe4be20740 1 -- start start 2024-09-21T06:58:54.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.144+0000 7efe4be20740 1 Processor -- start 2024-09-21T06:58:54.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.145+0000 7efe4be20740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f34c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.145+0000 7efe4be20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 0x56042f34cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe3d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 0x56042f34cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe3d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 0x56042f34cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35480/0 (socket says 172.21.15.114:35480) 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe3de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f34c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe3de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f34c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58852/0 (socket says 172.21.15.114:58852) 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe4be20740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f347990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe3d400640 1 -- 172.21.15.114:0/2139773561 learned_addr learned my addr 172.21.15.114:0/2139773561 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.146+0000 7efe3e800640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f347990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.147+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7efe401c0f30 con 0x56042ef691b0 2024-09-21T06:58:54.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.147+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe401c0800 con 0x56042ef71390 2024-09-21T06:58:54.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.147+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe401bda40 con 0x56042ef92070 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.148+0000 7efe4be20740 10 client.0 _ll_register_callbacks cb 0x7efe300ca270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.148+0000 7efe3d400640 1 -- 172.21.15.114:0/2139773561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 msgr2=0x56042f347990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.148+0000 7efe3d400640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f347990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.148+0000 7efe3d400640 1 -- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 msgr2=0x56042f34c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.148+0000 7efe3d400640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f34c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.149+0000 7efe3d400640 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042f34d4e0 con 0x56042ef71390 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.149+0000 7efe3e800640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f347990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.149+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe2c003aa0 con 0x56042ef71390 2024-09-21T06:58:54.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.149+0000 7efe3de00640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f34c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:54.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.150+0000 7efe3d400640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 0x56042f34cf20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efe2c014040 tx=0x7efe2c004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5752d388a034315 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:54.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.150+0000 7efe1fe00640 10 client.26843 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:54.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.150+0000 7efe1fe00640 1 -- 172.21.15.114:0/2139773561 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c03a060 con 0x56042ef71390 2024-09-21T06:58:54.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.151+0000 7efe1fe00640 1 -- 172.21.15.114:0/2139773561 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7efe2c003c50 con 0x56042ef71390 2024-09-21T06:58:54.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.151+0000 7efe1fe00640 1 -- 172.21.15.114:0/2139773561 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe2c031040 con 0x56042ef71390 2024-09-21T06:58:54.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.151+0000 7efe4be20740 10 client.26843 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:54.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=39}) -- 0x56042f3c1450 con 0x56042ef71390 2024-09-21T06:58:54.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe1fe00640 1 -- 172.21.15.114:0/2139773561 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7efe2c034070 con 0x56042ef71390 2024-09-21T06:58:54.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 10 client.26843 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:54.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 10 client.26843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:58:54.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 20 client.26843 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:54.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef25da0 0x56042f3c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:54.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 4 client.26843 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:58:54.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.152+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56042f2ad9b0 con 0x56042ef25da0 2024-09-21T06:58:54.158 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.155+0000 7efe3de00640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef25da0 0x56042f3c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.157+0000 7efe3de00640 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef25da0 0x56042f3c3b20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1417f3e590bb471a server_cookie=9c6d1c7a0d1f1543 in_seq=0 out_seq=0 2024-09-21T06:58:54.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.157+0000 7efe1fe00640 10 client.26843 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:58:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.157+0000 7efe1fe00640 1 -- 172.21.15.114:0/2139773561 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56042f2ad9b0 con 0x56042ef25da0 2024-09-21T06:58:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.157+0000 7efe1fe00640 10 client.26843 handle_command_reply: tid=0 2024-09-21T06:58:54.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.157+0000 7efe4be20740 1 client.26843 shutdown 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.158+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 msgr2=0x56042f34cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.158+0000 7efe4be20740 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 0x56042f34cf20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efe2c014040 tx=0x7efe2c004150 comp rx=0 tx=0).stop 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.159+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 shutdown_connections 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.159+0000 7efe4be20740 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x56042ef25da0 0x56042f3c3b20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.159+0000 7efe1fe00640 0 client.26843 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.160+0000 7efe4be20740 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56042ef92070 0x56042f347990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.160+0000 7efe4be20740 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56042ef71390 0x56042f34cf20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.160+0000 7efe4be20740 1 --2- 172.21.15.114:0/2139773561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56042ef691b0 0x56042f34c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.160+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 >> 172.21.15.114:0/2139773561 conn(0x56042ef86460 msgr2=0x56042f335fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:54.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.161+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 shutdown_connections 2024-09-21T06:58:54.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.161+0000 7efe4be20740 1 -- 172.21.15.114:0/2139773561 wait complete. 2024-09-21T06:58:54.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.161+0000 7efe4be20740 20 client.26843 trim_cache size 0 max 16384 2024-09-21T06:58:54.165 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:58:54.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efe240b6780 msgr2=0x7efe240b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efe240b6780 0x7efe240b8c90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efe40117210 tx=0x7efe38009b00 comp rx=0 tx=0).stop 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 msgr2=0x7efe40116ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe40116ac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efe30005ab0 tx=0x7efe30003b90 comp rx=0 tx=0).stop 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 shutdown_connections 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7efe401079f0 0x7efe40110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe40102f00 0x7efe4010fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7efe240b6780 0x7efe240b8c90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 --2- 172.21.15.114:0/1536258326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe401025b0 0x7efe40116ac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.163+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 >> 172.21.15.114:0/1536258326 conn(0x7efe400fe2c0 msgr2=0x7efe40105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.164+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 shutdown_connections 2024-09-21T06:58:54.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:54.164+0000 7efe3f200640 1 -- 172.21.15.114:0/1536258326 wait complete. 2024-09-21T06:58:55.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:55 smithi183 ceph-mon[34219]: pgmap v539: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-09-21T06:58:55.381 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:55 smithi114 ceph-mon[30654]: pgmap v539: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-09-21T06:58:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:55 smithi193 ceph-mon[34601]: pgmap v539: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.0 MiB/s wr, 598 op/s 2024-09-21T06:58:55.439 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:58:56.077 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:58:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 -- 172.21.15.114:0/4091241727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410fad0 msgr2=0x7ff8e410fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 --2- 172.21.15.114:0/4091241727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410fad0 0x7ff8e410fee0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 -- 172.21.15.114:0/4091241727 shutdown_connections 2024-09-21T06:58:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 --2- 172.21.15.114:0/4091241727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e4110420 0x7ff8e4116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 --2- 172.21.15.114:0/4091241727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410fad0 0x7ff8e410fee0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 --2- 172.21.15.114:0/4091241727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410a190 0x7ff8e410a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 -- 172.21.15.114:0/4091241727 >> 172.21.15.114:0/4091241727 conn(0x7ff8e40780f0 msgr2=0x7ff8e41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 -- 172.21.15.114:0/4091241727 shutdown_connections 2024-09-21T06:58:57.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.300+0000 7ff8eba00640 1 -- 172.21.15.114:0/4091241727 wait complete. 2024-09-21T06:58:57.304 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:57 smithi114 ceph-mon[30654]: pgmap v540: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-09-21T06:58:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.302+0000 7ff8eba00640 1 Processor -- start 2024-09-21T06:58:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.302+0000 7ff8eba00640 1 -- start start 2024-09-21T06:58:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8eba00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410a190 0x7ff8e41a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8eba00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 0x7ff8e41a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8eba00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e4110420 0x7ff8e41a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8eba00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e411a100 con 0x7ff8e410fad0 2024-09-21T06:58:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8eba00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4119f80 con 0x7ff8e4110420 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8eba00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e411a280 con 0x7ff8e410a190 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 0x7ff8e41a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8e9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410a190 0x7ff8e41a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8e9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410a190 0x7ff8e41a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43648/0 (socket says 172.21.15.114:43648) 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8e9000640 1 -- 172.21.15.114:0/2186915575 learned_addr learned my addr 172.21.15.114:0/2186915575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.303+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 0x7ff8e41a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33676/0 (socket says 172.21.15.114:33676) 2024-09-21T06:58:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.304+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e4110420 0x7ff8e41a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.304+0000 7ff8e3e00640 1 -- 172.21.15.114:0/2186915575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410a190 msgr2=0x7ff8e41a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.304+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410a190 0x7ff8e41a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.304+0000 7ff8e3e00640 1 -- 172.21.15.114:0/2186915575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e4110420 msgr2=0x7ff8e41a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.304+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e4110420 0x7ff8e41a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.304+0000 7ff8e3e00640 1 -- 172.21.15.114:0/2186915575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e41a6aa0 con 0x7ff8e410fad0 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.305+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 0x7ff8e41a10d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009230 tx=0x7ff8d0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.305+0000 7ff8e1600640 1 -- 172.21.15.114:0/2186915575 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d003f070 con 0x7ff8e410fad0 2024-09-21T06:58:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.305+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e41a6d30 con 0x7ff8e410fad0 2024-09-21T06:58:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.306+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e41ad4f0 con 0x7ff8e410fad0 2024-09-21T06:58:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.306+0000 7ff8e1600640 1 -- 172.21.15.114:0/2186915575 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d0031b50 con 0x7ff8e410fad0 2024-09-21T06:58:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.307+0000 7ff8e1600640 1 -- 172.21.15.114:0/2186915575 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0035030 con 0x7ff8e410fad0 2024-09-21T06:58:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.311+0000 7ff8f2666740 1 Processor -- start 2024-09-21T06:58:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.311+0000 7ff8f2666740 1 -- start start 2024-09-21T06:58:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.311+0000 7ff8e1600640 1 -- 172.21.15.114:0/2186915575 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff8d0042070 con 0x7ff8e410fad0 2024-09-21T06:58:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567eb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8f2666740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ec56761fb0 con 0x55ec5641b1b0 2024-09-21T06:58:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8f2666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec567513e0 con 0x55ec56444070 2024-09-21T06:58:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8f2666740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec567551f0 con 0x55ec56423390 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8e9a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8e9a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567eaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45670/0 (socket says 172.21.15.114:45670) 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8e9a00640 1 -- 172.21.15.114:0/1186165069 learned_addr learned my addr 172.21.15.114:0/1186165069 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.312+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567eb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8e9a00640 1 -- 172.21.15.114:0/1186165069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 msgr2=0x55ec567eb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567eb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8e9a00640 1 -- 172.21.15.114:0/1186165069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 msgr2=0x55ec567eb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8e9a00640 1 -- 172.21.15.114:0/1186165069 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec5671bcd0 con 0x55ec56444070 2024-09-21T06:58:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567eaba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009f50 tx=0x7ff8dc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5792e3897ae682ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.313+0000 7ff8d6000640 1 -- 172.21.15.114:0/1186165069 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004b20 con 0x55ec56444070 2024-09-21T06:58:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.314+0000 7ff8d6000640 1 -- 172.21.15.114:0/1186165069 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8dc004cc0 con 0x55ec56444070 2024-09-21T06:58:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.314+0000 7ff8d6000640 1 -- 172.21.15.114:0/1186165069 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc005020 con 0x55ec56444070 2024-09-21T06:58:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.314+0000 7ff8e1600640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8cc0b6880 0x7ff8cc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.314+0000 7ff8e1600640 1 -- 172.21.15.114:0/2186915575 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8d003b8e0 con 0x7ff8e410fad0 2024-09-21T06:58:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.314+0000 7ff8e9000640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8cc0b6880 0x7ff8cc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 -- 172.21.15.114:0/1186165069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 msgr2=0x55ec567eaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567eaba0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009f50 tx=0x7ff8dc0118d0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 -- 172.21.15.114:0/1186165069 shutdown_connections 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567eb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 --2- 172.21.15.114:0/1186165069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567eaba0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 -- 172.21.15.114:0/1186165069 >> 172.21.15.114:0/1186165069 conn(0x55ec56438460 msgr2=0x55ec567e8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 -- 172.21.15.114:0/1186165069 shutdown_connections 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8e9000640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8cc0b6880 0x7ff8cc0b8d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55ec567e9820 tx=0x7ff8d8009900 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.315+0000 7ff8f2666740 1 -- 172.21.15.114:0/1186165069 wait complete. 2024-09-21T06:58:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.316+0000 7ff8f2666740 1 -- start start 2024-09-21T06:58:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.316+0000 7ff8f2666740 1 Processor -- start 2024-09-21T06:58:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.317+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.317+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 0x55ec567f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.317+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56444070 0x55ec56802700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8f2666740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ec565b2700 con 0x55ec5641b1b0 2024-09-21T06:58:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8f2666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ec566ec580 con 0x55ec56423390 2024-09-21T06:58:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8f2666740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ec567f9380 con 0x55ec56444070 2024-09-21T06:58:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8f2666740 10 client.0 _ll_register_callbacks cb 0x55ec566fc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 0x55ec567f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8e3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 0x55ec567f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45684/0 (socket says 172.21.15.114:45684) 2024-09-21T06:58:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8e3e00640 1 -- 172.21.15.114:0/3874401926 learned_addr learned my addr 172.21.15.114:0/3874401926 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8e9000640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.318+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56444070 0x55ec56802700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8e3e00640 1 -- 172.21.15.114:0/3874401926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56444070 msgr2=0x55ec56802700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56444070 0x55ec56802700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8e3e00640 1 -- 172.21.15.114:0/3874401926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 msgr2=0x55ec567f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8e3e00640 1 -- 172.21.15.114:0/3874401926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec564af650 con 0x55ec56423390 2024-09-21T06:58:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 0x55ec567f6c70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e7858e1846ab61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.319+0000 7ff8caa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.320+0000 7ff8caa00640 1 -- 172.21.15.114:0/3874401926 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00c9310 con 0x55ec56423390 2024-09-21T06:58:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.320+0000 7ff8caa00640 1 -- 172.21.15.114:0/3874401926 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8d00c7b70 con 0x55ec56423390 2024-09-21T06:58:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.320+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ec566eeb90 con 0x55ec56423390 2024-09-21T06:58:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.320+0000 7ff8caa00640 1 -- 172.21.15.114:0/3874401926 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff8d00ccbe0 con 0x55ec56423390 2024-09-21T06:58:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.320+0000 7ff8f2666740 10 client.26855 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.320+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec56709950 con 0x55ec56423390 2024-09-21T06:58:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8caa00640 1 -- 172.21.15.114:0/3874401926 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d00ccf80 con 0x55ec56423390 2024-09-21T06:58:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8caa00640 1 -- 172.21.15.114:0/3874401926 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7ff8d00b4bb0 con 0x55ec56423390 2024-09-21T06:58:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8f2666740 10 client.26855 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8f2666740 10 client.26855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8f2666740 20 client.26855 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8f2666740 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec564029f0 0x55ec56808800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8f2666740 4 client.26855 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.321+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec566eeb90 con 0x55ec564029f0 2024-09-21T06:58:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.322+0000 7ff8e9000640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec564029f0 0x55ec56808800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.323+0000 7ff8e9000640 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec564029f0 0x55ec56808800 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b45ba3390a9790 server_cookie=f66fd741f77ecdff in_seq=0 out_seq=0 2024-09-21T06:58:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.323+0000 7ff8caa00640 10 client.26855 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.324+0000 7ff8caa00640 1 -- 172.21.15.114:0/3874401926 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ec566eeb90 con 0x55ec564029f0 2024-09-21T06:58:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.324+0000 7ff8caa00640 10 client.26855 handle_command_reply: tid=0 2024-09-21T06:58:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.325+0000 7ff8f2666740 1 client.26855 shutdown 2024-09-21T06:58:57.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.325+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 msgr2=0x55ec567f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.325+0000 7ff8f2666740 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 0x55ec567f6c70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d0031400 comp rx=0 tx=0).stop 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 shutdown_connections 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec564029f0 0x55ec56808800 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56444070 0x55ec56802700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56423390 0x55ec567f6c70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 --2- 172.21.15.114:0/3874401926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8caa00640 0 client.26855 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 >> 172.21.15.114:0/3874401926 conn(0x55ec56438460 msgr2=0x55ec567edd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 shutdown_connections 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 1 -- 172.21.15.114:0/3874401926 wait complete. 2024-09-21T06:58:57.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.326+0000 7ff8f2666740 20 client.26855 trim_cache size 0 max 16384 2024-09-21T06:58:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 Processor -- start 2024-09-21T06:58:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 -- start start 2024-09-21T06:58:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 0x55ec567e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec56423390 0x55ec567e8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567f3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c400fe00 con 0x55ec56423390 2024-09-21T06:58:57.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4011a60 con 0x55ec56444070 2024-09-21T06:58:57.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.350+0000 7ff8f2666740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40104d0 con 0x55ec5641b1b0 2024-09-21T06:58:57.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.351+0000 7ff8e9a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.351+0000 7ff8e9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 0x55ec567e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.351+0000 7ff8e9000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 0x55ec567e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43672/0 (socket says 172.21.15.114:43672) 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.351+0000 7ff8e9a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45690/0 (socket says 172.21.15.114:45690) 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.351+0000 7ff8e9000640 1 -- 172.21.15.114:0/2307804797 learned_addr learned my addr 172.21.15.114:0/2307804797 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.351+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec56423390 0x55ec567e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.352+0000 7ff8e9000640 1 -- 172.21.15.114:0/2307804797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 msgr2=0x55ec567f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.352+0000 7ff8e9000640 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.352+0000 7ff8e9000640 1 -- 172.21.15.114:0/2307804797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec56423390 msgr2=0x55ec567e8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.352+0000 7ff8e9000640 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec56423390 0x55ec567e8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.352+0000 7ff8e9000640 1 -- 172.21.15.114:0/2307804797 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec567e2c20 con 0x55ec5641b1b0 2024-09-21T06:58:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.352+0000 7ff8e9000640 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 0x55ec567e8040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff8d801e490 tx=0x7ff8d8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b118ef7af40a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.354+0000 7ff8d5600640 1 -- 172.21.15.114:0/2307804797 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d801ee10 con 0x55ec5641b1b0 2024-09-21T06:58:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.354+0000 7ff8d5600640 1 -- 172.21.15.114:0/2307804797 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d802a030 con 0x55ec5641b1b0 2024-09-21T06:58:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.354+0000 7ff8d5600640 1 -- 172.21.15.114:0/2307804797 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d801c660 con 0x55ec5641b1b0 2024-09-21T06:58:57.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.356+0000 7ff8f2666740 1 -- 172.21.15.114:0/2307804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 msgr2=0x55ec567e8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.356+0000 7ff8f2666740 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 0x55ec567e8040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff8d801e490 tx=0x7ff8d8017000 comp rx=0 tx=0).stop 2024-09-21T06:58:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.360+0000 7ff8f2666740 1 -- 172.21.15.114:0/2307804797 shutdown_connections 2024-09-21T06:58:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.360+0000 7ff8f2666740 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.360+0000 7ff8f2666740 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec56423390 0x55ec567e8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.360+0000 7ff8f2666740 1 --2- 172.21.15.114:0/2307804797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec5641b1b0 0x55ec567e8040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.360+0000 7ff8f2666740 1 -- 172.21.15.114:0/2307804797 >> 172.21.15.114:0/2307804797 conn(0x55ec56438460 msgr2=0x55ec567f4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.361+0000 7ff8f2666740 1 -- 172.21.15.114:0/2307804797 shutdown_connections 2024-09-21T06:58:57.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.361+0000 7ff8f2666740 1 -- 172.21.15.114:0/2307804797 wait complete. 2024-09-21T06:58:57.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:57 smithi183 ceph-mon[34219]: pgmap v540: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-09-21T06:58:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.362+0000 7ff8f2666740 1 -- start start 2024-09-21T06:58:57.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.363+0000 7ff8f2666740 1 Processor -- start 2024-09-21T06:58:57.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.364+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.365+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.365+0000 7ff8f2666740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567fb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.366+0000 7ff8f2666740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40112d0 con 0x55ec5641b1b0 2024-09-21T06:58:57.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.366+0000 7ff8e9000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.366+0000 7ff8e9000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33704/0 (socket says 172.21.15.114:33704) 2024-09-21T06:58:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.366+0000 7ff8e9000640 1 -- 172.21.15.114:0/4104251425 learned_addr learned my addr 172.21.15.114:0/4104251425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.367+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c4010ba0 con 0x55ec56444070 2024-09-21T06:58:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.367+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567fb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.367+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c400dde0 con 0x55ec56423390 2024-09-21T06:58:57.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.367+0000 7ff8f2666740 10 client.0 _ll_register_callbacks cb 0x7ff8d0005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.369+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.369+0000 7ff8e9a00640 1 -- 172.21.15.114:0/4104251425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 msgr2=0x55ec567fb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.369+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567fb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.369+0000 7ff8e9a00640 1 -- 172.21.15.114:0/4104251425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 msgr2=0x55ec567feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.369+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.369+0000 7ff8e9a00640 1 -- 172.21.15.114:0/4104251425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec567fba90 con 0x55ec56444070 2024-09-21T06:58:57.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.370+0000 7ff8e9000640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:58:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.370+0000 7ff8e3e00640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.371+0000 7ff8e9a00640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567fb550 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55ec567f8f50 tx=0x7ff8dc005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91fbc0ec06bdb03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:57.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.371+0000 7ff8caa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.372+0000 7ff8caa00640 1 -- 172.21.15.114:0/4104251425 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc029070 con 0x55ec56444070 2024-09-21T06:58:57.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.372+0000 7ff8caa00640 1 -- 172.21.15.114:0/4104251425 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8dc0053c0 con 0x55ec56444070 2024-09-21T06:58:57.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.373+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d802a030 con 0x55ec56444070 2024-09-21T06:58:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.375+0000 7ff8caa00640 1 -- 172.21.15.114:0/4104251425 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc033050 con 0x55ec56444070 2024-09-21T06:58:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.375+0000 7ff8f2666740 10 client.26861 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:57.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.375+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec56873e50 con 0x55ec56444070 2024-09-21T06:58:57.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.375+0000 7ff8caa00640 1 -- 172.21.15.114:0/4104251425 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d802a030 con 0x55ec56444070 2024-09-21T06:58:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.376+0000 7ff8caa00640 1 -- 172.21.15.114:0/4104251425 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7ff8dc02eb60 con 0x55ec56444070 2024-09-21T06:58:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.376+0000 7ff8f2666740 10 client.26861 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.377+0000 7ff8f2666740 10 client.26861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.377+0000 7ff8f2666740 20 client.26861 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.377+0000 7ff8f2666740 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec563d7da0 0x55ec568764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:57.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.377+0000 7ff8f2666740 4 client.26861 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:58:57.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.377+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec56763f00 con 0x55ec563d7da0 2024-09-21T06:58:57.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.379+0000 7ff8e9000640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec563d7da0 0x55ec568764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:57.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.382+0000 7ff8caa00640 10 client.26861 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:57.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.382+0000 7ff8e9000640 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec563d7da0 0x55ec568764a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103ed945db59f28d server_cookie=b7c387baacfcb6be in_seq=0 out_seq=0 2024-09-21T06:58:57.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.383+0000 7ff8caa00640 1 -- 172.21.15.114:0/4104251425 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec56763f00 con 0x55ec563d7da0 2024-09-21T06:58:57.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.383+0000 7ff8caa00640 10 client.26861 handle_command_reply: tid=0 2024-09-21T06:58:57.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.383+0000 7ff8f2666740 1 client.26861 shutdown 2024-09-21T06:58:57.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 msgr2=0x55ec567fb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567fb550 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55ec567f8f50 tx=0x7ff8dc005040 comp rx=0 tx=0).stop 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 shutdown_connections 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ec56444070 0x55ec567fb550 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ec563d7da0 0x55ec568764a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ec56423390 0x55ec567fb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 --2- 172.21.15.114:0/4104251425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ec5641b1b0 0x55ec567feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 >> 172.21.15.114:0/4104251425 conn(0x55ec56438460 msgr2=0x55ec567edca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:57.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.384+0000 7ff8caa00640 0 client.26861 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:57.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.388+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 shutdown_connections 2024-09-21T06:58:57.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.388+0000 7ff8f2666740 1 -- 172.21.15.114:0/4104251425 wait complete. 2024-09-21T06:58:57.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.389+0000 7ff8f2666740 20 client.26861 trim_cache size 0 max 16384 2024-09-21T06:58:57.392 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:58:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8cc0b6880 msgr2=0x7ff8cc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8cc0b6880 0x7ff8cc0b8d90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55ec567e9820 tx=0x7ff8d8009900 comp rx=0 tx=0).stop 2024-09-21T06:58:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 msgr2=0x7ff8e41a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:57.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 0x7ff8e41a10d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009230 tx=0x7ff8d0003b90 comp rx=0 tx=0).stop 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 shutdown_connections 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8cc0b6880 0x7ff8cc0b8d90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e4110420 0x7ff8e41a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8e410fad0 0x7ff8e41a10d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 --2- 172.21.15.114:0/2186915575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e410a190 0x7ff8e41a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 >> 172.21.15.114:0/2186915575 conn(0x7ff8e40780f0 msgr2=0x7ff8e410de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 shutdown_connections 2024-09-21T06:58:57.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:58:57.390+0000 7ff8eba00640 1 -- 172.21.15.114:0/2186915575 wait complete. 2024-09-21T06:58:57.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:57 smithi193 ceph-mon[34601]: pgmap v540: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.3 MiB/s wr, 567 op/s 2024-09-21T06:58:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 260.0 s, 1239.6 tps, lat 79.594 ms stddev 102.144 2024-09-21T06:58:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.121+0000 7fe441800640 1 -- 172.21.15.114:0/3947759671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c109d60 msgr2=0x7fe43c110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.121+0000 7fe441800640 1 --2- 172.21.15.114:0/3947759671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c109d60 0x7fe43c110590 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe434009ca0 tx=0x7fe43402a8e0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.125+0000 7fe441800640 1 -- 172.21.15.114:0/3947759671 shutdown_connections 2024-09-21T06:58:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.125+0000 7fe441800640 1 --2- 172.21.15.114:0/3947759671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c109d60 0x7fe43c110590 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.125+0000 7fe441800640 1 --2- 172.21.15.114:0/3947759671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 0x7fe43c109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.125+0000 7fe441800640 1 --2- 172.21.15.114:0/3947759671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe43c103ae0 0x7fe43c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.126+0000 7fe441800640 1 -- 172.21.15.114:0/3947759671 >> 172.21.15.114:0/3947759671 conn(0x7fe43c0ff520 msgr2=0x7fe43c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.127+0000 7fe441800640 1 -- 172.21.15.114:0/3947759671 shutdown_connections 2024-09-21T06:58:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.127+0000 7fe441800640 1 -- 172.21.15.114:0/3947759671 wait complete. 2024-09-21T06:58:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.130+0000 7fe441800640 1 Processor -- start 2024-09-21T06:58:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.130+0000 7fe441800640 1 -- start start 2024-09-21T06:58:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.132+0000 7fe441800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 0x7fe43c074e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.132+0000 7fe441800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 0x7fe43c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.132+0000 7fe441800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe43c109d60 0x7fe43c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.132+0000 7fe43aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 0x7fe43c074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.132+0000 7fe43aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 0x7fe43c074e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33724/0 (socket says 172.21.15.114:33724) 2024-09-21T06:58:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe43a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 0x7fe43c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe441800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c113af0 con 0x7fe43c103ae0 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe441800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c113970 con 0x7fe43c109410 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe441800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c113c70 con 0x7fe43c109d60 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe43a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 0x7fe43c06e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45724/0 (socket says 172.21.15.114:45724) 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe43a000640 1 -- 172.21.15.114:0/1810016959 learned_addr learned my addr 172.21.15.114:0/1810016959 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.133+0000 7fe43b400640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe43c109d60 0x7fe43c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.134+0000 7fe43aa00640 1 -- 172.21.15.114:0/1810016959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe43c109d60 msgr2=0x7fe43c06e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.134+0000 7fe43aa00640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe43c109d60 0x7fe43c06e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.134+0000 7fe43aa00640 1 -- 172.21.15.114:0/1810016959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 msgr2=0x7fe43c06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.134+0000 7fe43aa00640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 0x7fe43c06e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.134+0000 7fe43aa00640 1 -- 172.21.15.114:0/1810016959 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c06ee40 con 0x7fe43c103ae0 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.134+0000 7fe43aa00640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 0x7fe43c074e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009fd0 tx=0x7fe42c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.135+0000 7fe433400640 1 -- 172.21.15.114:0/1810016959 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c015020 con 0x7fe43c103ae0 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.135+0000 7fe433400640 1 -- 172.21.15.114:0/1810016959 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c011ce0 con 0x7fe43c103ae0 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.135+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c073ae0 con 0x7fe43c103ae0 2024-09-21T06:58:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.135+0000 7fe433400640 1 -- 172.21.15.114:0/1810016959 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c010e40 con 0x7fe43c103ae0 2024-09-21T06:58:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.135+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c0740b0 con 0x7fe43c103ae0 2024-09-21T06:58:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.144+0000 7fe4487c2740 1 Processor -- start 2024-09-21T06:58:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.145+0000 7fe4487c2740 1 -- start start 2024-09-21T06:58:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.146+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5ca7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.146+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5ca7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.146+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5ca7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.146+0000 7fe4487c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5587e5c146b0 con 0x5587e58d12f0 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.146+0000 7fe4487c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587e5c03ae0 con 0x5587e58d94d0 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.146+0000 7fe4487c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587e5c078f0 con 0x5587e58fa1b0 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.148+0000 7fe43a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5ca7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.148+0000 7fe43a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5ca7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45734/0 (socket says 172.21.15.114:45734) 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.148+0000 7fe43a000640 1 -- 172.21.15.114:0/3548577367 learned_addr learned my addr 172.21.15.114:0/3548577367 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.148+0000 7fe433400640 1 -- 172.21.15.114:0/1810016959 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe42c018020 con 0x7fe43c103ae0 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.148+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5ca7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.149+0000 7fe43b400640 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5ca7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.149+0000 7fe43aa00640 1 -- 172.21.15.114:0/3548577367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 msgr2=0x5587e5ca7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.149+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5ca7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.149+0000 7fe43aa00640 1 -- 172.21.15.114:0/3548577367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 msgr2=0x5587e5ca7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.149+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5ca7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.149+0000 7fe43aa00640 1 -- 172.21.15.114:0/3548577367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e5bd0780 con 0x5587e58d12f0 2024-09-21T06:58:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.151+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5ca7980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0e4490 tx=0x7fe42c0ef6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500a3a865ee91714 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.151+0000 7fe433400640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4200b6780 0x7fe4200b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.151+0000 7fe433400640 1 -- 172.21.15.114:0/1810016959 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe42c0e7110 con 0x7fe43c103ae0 2024-09-21T06:58:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.151+0000 7fe43b400640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4200b6780 0x7fe4200b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.153+0000 7fe427e00640 1 -- 172.21.15.114:0/3548577367 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0fa070 con 0x5587e58d12f0 2024-09-21T06:58:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.153+0000 7fe427e00640 1 -- 172.21.15.114:0/3548577367 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe42c0ac6a0 con 0x5587e58d12f0 2024-09-21T06:58:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.153+0000 7fe427e00640 1 -- 172.21.15.114:0/3548577367 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c0a91e0 con 0x5587e58d12f0 2024-09-21T06:58:59.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.153+0000 7fe43b400640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4200b6780 0x7fe4200b8c90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe434005e30 tx=0x7fe43403e040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 -- 172.21.15.114:0/3548577367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 msgr2=0x5587e5ca7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5ca7980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0e4490 tx=0x7fe42c0ef6f0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 -- 172.21.15.114:0/3548577367 shutdown_connections 2024-09-21T06:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5ca7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5ca7980 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3548577367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5ca7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.155+0000 7fe4487c2740 1 -- 172.21.15.114:0/3548577367 >> 172.21.15.114:0/3548577367 conn(0x5587e58ee300 msgr2=0x5587e5ca5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.157+0000 7fe4487c2740 1 -- 172.21.15.114:0/3548577367 shutdown_connections 2024-09-21T06:58:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.157+0000 7fe4487c2740 1 -- 172.21.15.114:0/3548577367 wait complete. 2024-09-21T06:58:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.158+0000 7fe4487c2740 1 -- start start 2024-09-21T06:58:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.158+0000 7fe4487c2740 1 Processor -- start 2024-09-21T06:58:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.161+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5cb2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5cb2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe43a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5cb2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe43a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5cb2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45736/0 (socket says 172.21.15.114:45736) 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe43a000640 1 -- 172.21.15.114:0/3514040273 learned_addr learned my addr 172.21.15.114:0/3514040273 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5587e5b3bab0 con 0x5587e58d12f0 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587e5baa570 con 0x5587e58d94d0 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.162+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587e5cb5a40 con 0x5587e58fa1b0 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.163+0000 7fe4487c2740 10 client.0 _ll_register_callbacks cb 0x5587e5bb9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:59.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.164+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5cb2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43a000640 1 -- 172.21.15.114:0/3514040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 msgr2=0x5587e5cb34f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:58:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43a000640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb34f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43a000640 1 -- 172.21.15.114:0/3514040273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 msgr2=0x5587e5cb2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43a000640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5cb2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43a000640 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e5965790 con 0x5587e58d94d0 2024-09-21T06:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5cb2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe424c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.165+0000 7fe43a000640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5cb2f90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe428012450 tx=0x7fe428012910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbd1b7059b49c795 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.166+0000 7fe424c00640 1 -- 172.21.15.114:0/3514040273 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe428010a20 con 0x5587e58d94d0 2024-09-21T06:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.166+0000 7fe424c00640 1 -- 172.21.15.114:0/3514040273 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe428010820 con 0x5587e58d94d0 2024-09-21T06:58:59.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.167+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe42c0ac6a0 con 0x5587e58d94d0 2024-09-21T06:58:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.168+0000 7fe424c00640 1 -- 172.21.15.114:0/3514040273 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe428010a20 con 0x5587e58d94d0 2024-09-21T06:58:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.168+0000 7fe4487c2740 10 client.26873 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.168+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5587e59de620 con 0x5587e58d94d0 2024-09-21T06:58:59.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.168+0000 7fe424c00640 1 -- 172.21.15.114:0/3514040273 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe428010bc0 con 0x5587e58d94d0 2024-09-21T06:58:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe424c00640 1 -- 172.21.15.114:0/3514040273 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fe42801c3f0 con 0x5587e58d94d0 2024-09-21T06:58:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe4487c2740 10 client.26873 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe4487c2740 10 client.26873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe4487c2740 20 client.26873 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e58b8d80 0x5587e5cc4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe4487c2740 4 client.26873 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:58:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.170+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe42c0ac6a0 con 0x5587e58b8d80 2024-09-21T06:58:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.171+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e58b8d80 0x5587e5cc4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.171+0000 7fe43aa00640 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e58b8d80 0x5587e5cc4ba0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0d5a48b11348c8 server_cookie=5855bc66e8328bfb in_seq=0 out_seq=0 2024-09-21T06:58:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.171+0000 7fe424c00640 10 client.26873 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.172+0000 7fe424c00640 1 -- 172.21.15.114:0/3514040273 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe42c0ac6a0 con 0x5587e58b8d80 2024-09-21T06:58:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.172+0000 7fe424c00640 10 client.26873 handle_command_reply: tid=0 2024-09-21T06:58:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.172+0000 7fe4487c2740 1 client.26873 shutdown 2024-09-21T06:58:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.172+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 msgr2=0x5587e5cb2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5cb2f90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe428012450 tx=0x7fe428012910 comp rx=0 tx=0).stop 2024-09-21T06:58:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 shutdown_connections 2024-09-21T06:58:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e58b8d80 0x5587e5cc4ba0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d94d0 0x5587e5cb2f90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe424c00640 0 client.26873 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 --2- 172.21.15.114:0/3514040273 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d12f0 0x5587e5cb2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 >> 172.21.15.114:0/3514040273 conn(0x5587e58ee300 msgr2=0x5587e5ca8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 shutdown_connections 2024-09-21T06:58:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 1 -- 172.21.15.114:0/3514040273 wait complete. 2024-09-21T06:58:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.173+0000 7fe4487c2740 20 client.26873 trim_cache size 0 max 16384 2024-09-21T06:58:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.203+0000 7fe4487c2740 1 Processor -- start 2024-09-21T06:58:59.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.203+0000 7fe4487c2740 1 -- start start 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.204+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.204+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cafc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.204+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.204+0000 7fe4487c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1b7340 con 0x5587e58d94d0 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.204+0000 7fe4487c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1b8fa0 con 0x5587e58d12f0 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.204+0000 7fe4487c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1b7a10 con 0x5587e58fa1b0 2024-09-21T06:58:59.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.205+0000 7fe43b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.205+0000 7fe43b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43714/0 (socket says 172.21.15.114:43714) 2024-09-21T06:58:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.205+0000 7fe43b400640 1 -- 172.21.15.114:0/1442989365 learned_addr learned my addr 172.21.15.114:0/1442989365 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:59.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.205+0000 7fe43a000640 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cafc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.205+0000 7fe43b400640 1 -- 172.21.15.114:0/1442989365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 msgr2=0x5587e5cb4680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.205+0000 7fe43b400640 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb4680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.206+0000 7fe43b400640 1 -- 172.21.15.114:0/1442989365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 msgr2=0x5587e5cafc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.206+0000 7fe43b400640 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cafc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.206+0000 7fe43b400640 1 -- 172.21.15.114:0/1442989365 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e5c97fa0 con 0x5587e58fa1b0 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.206+0000 7fe43b400640 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb0160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe434009ca0 tx=0x7fe434041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7b5deb50bb0645 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.207+0000 7fe427400640 1 -- 172.21.15.114:0/1442989365 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434031cb0 con 0x5587e58fa1b0 2024-09-21T06:58:59.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.207+0000 7fe427400640 1 -- 172.21.15.114:0/1442989365 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe434036040 con 0x5587e58fa1b0 2024-09-21T06:58:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.207+0000 7fe427400640 1 -- 172.21.15.114:0/1442989365 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4340309e0 con 0x5587e58fa1b0 2024-09-21T06:58:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.208+0000 7fe4487c2740 1 -- 172.21.15.114:0/1442989365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 msgr2=0x5587e5cb0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.208+0000 7fe4487c2740 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb0160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe434009ca0 tx=0x7fe434041000 comp rx=0 tx=0).stop 2024-09-21T06:58:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 -- 172.21.15.114:0/1442989365 shutdown_connections 2024-09-21T06:58:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb0160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cafc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 --2- 172.21.15.114:0/1442989365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 -- 172.21.15.114:0/1442989365 >> 172.21.15.114:0/1442989365 conn(0x5587e58ee300 msgr2=0x5587e5cb3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 -- 172.21.15.114:0/1442989365 shutdown_connections 2024-09-21T06:58:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.209+0000 7fe4487c2740 1 -- 172.21.15.114:0/1442989365 wait complete. 2024-09-21T06:58:59.212 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.210+0000 7fe4487c2740 1 -- start start 2024-09-21T06:58:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.210+0000 7fe4487c2740 1 Processor -- start 2024-09-21T06:58:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cb9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1b8810 con 0x5587e58d94d0 2024-09-21T06:58:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1b80e0 con 0x5587e58d12f0 2024-09-21T06:58:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c1b53d0 con 0x5587e58fa1b0 2024-09-21T06:58:59.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe4487c2740 10 client.0 _ll_register_callbacks cb 0x7fe42800b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe43aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe43aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45748/0 (socket says 172.21.15.114:45748) 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe43aa00640 1 -- 172.21.15.114:0/2667093551 learned_addr learned my addr 172.21.15.114:0/2667093551 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.211+0000 7fe43b400640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43a000640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cb9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43b400640 1 -- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 msgr2=0x5587e5cb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43b400640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43b400640 1 -- 172.21.15.114:0/2667093551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 msgr2=0x5587e5cb9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:58:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43b400640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cb9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43b400640 1 -- 172.21.15.114:0/2667093551 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e5d2e710 con 0x5587e58fa1b0 2024-09-21T06:58:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43a000640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cb9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:58:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe43b400640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb9890 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe43402ae00 tx=0x7fe434004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552b8692e4737d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:58:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe424c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:58:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.212+0000 7fe424c00640 1 -- 172.21.15.114:0/2667093551 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43405a020 con 0x5587e58fa1b0 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587e5cb53c0 con 0x5587e58fa1b0 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe424c00640 1 -- 172.21.15.114:0/2667093551 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe4340306e0 con 0x5587e58fa1b0 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe424c00640 1 -- 172.21.15.114:0/2667093551 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe434065050 con 0x5587e58fa1b0 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe4487c2740 10 client.26836 fetch_fsmap learned FSMap version 39 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5587e5d2f9a0 con 0x5587e58fa1b0 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe424c00640 1 -- 172.21.15.114:0/2667093551 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5587e5cb53c0 con 0x5587e58fa1b0 2024-09-21T06:58:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.213+0000 7fe424c00640 1 -- 172.21.15.114:0/2667093551 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fe434051070 con 0x5587e58fa1b0 2024-09-21T06:58:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.214+0000 7fe4487c2740 10 client.26836 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:58:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.214+0000 7fe4487c2740 10 client.26836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:58:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.214+0000 7fe4487c2740 20 client.26836 populate_metadata read hostname 'smithi114' 2024-09-21T06:58:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.214+0000 7fe4487c2740 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e588e1c0 0x5587e5d32170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:58:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.214+0000 7fe4487c2740 4 client.26836 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:58:59.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.214+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587e5c16600 con 0x5587e588e1c0 2024-09-21T06:58:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.215+0000 7fe43aa00640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e588e1c0 0x5587e5d32170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:58:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.216+0000 7fe43aa00640 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e588e1c0 0x5587e5d32170 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8cec720dbae8f5c server_cookie=cd1150262307ffd2 in_seq=0 out_seq=0 2024-09-21T06:58:59.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.216+0000 7fe424c00640 10 client.26836 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.225+0000 7fe424c00640 1 -- 172.21.15.114:0/2667093551 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587e5c16600 con 0x5587e588e1c0 2024-09-21T06:58:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.225+0000 7fe424c00640 10 client.26836 handle_command_reply: tid=0 2024-09-21T06:58:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.225+0000 7fe4487c2740 1 client.26836 shutdown 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 msgr2=0x5587e5cb9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb9890 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe43402ae00 tx=0x7fe434004510 comp rx=0 tx=0).stop 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 shutdown_connections 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587e588e1c0 0x5587e5d32170 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587e58fa1b0 0x5587e5cb9890 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587e58d94d0 0x5587e5cb9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 --2- 172.21.15.114:0/2667093551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587e58d12f0 0x5587e5cb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 >> 172.21.15.114:0/2667093551 conn(0x5587e58ee300 msgr2=0x5587e5ca8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.226+0000 7fe424c00640 0 client.26836 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:58:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.227+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 shutdown_connections 2024-09-21T06:58:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.227+0000 7fe4487c2740 1 -- 172.21.15.114:0/2667093551 wait complete. 2024-09-21T06:58:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.228+0000 7fe4487c2740 20 client.26836 trim_cache size 0 max 16384 2024-09-21T06:58:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:58:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:58:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:58:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:58:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.230+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4200b6780 msgr2=0x7fe4200b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.230+0000 7fe441800640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4200b6780 0x7fe4200b8c90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe434005e30 tx=0x7fe43403e040 comp rx=0 tx=0).stop 2024-09-21T06:58:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.230+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 msgr2=0x7fe43c074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:58:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.230+0000 7fe441800640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 0x7fe43c074e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009fd0 tx=0x7fe42c00b870 comp rx=0 tx=0).stop 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 shutdown_connections 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe43c109d60 0x7fe43c06e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4200b6780 0x7fe4200b8c90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe43c109410 0x7fe43c06e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 --2- 172.21.15.114:0/1810016959 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe43c103ae0 0x7fe43c074e20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 >> 172.21.15.114:0/1810016959 conn(0x7fe43c0ff520 msgr2=0x7fe43c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:58:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.231+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 shutdown_connections 2024-09-21T06:58:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:58:59.232+0000 7fe441800640 1 -- 172.21.15.114:0/1810016959 wait complete. 2024-09-21T06:58:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:59 smithi183 ceph-mon[34219]: pgmap v541: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-09-21T06:58:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:58:59 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:58:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:59 smithi193 ceph-mon[34601]: pgmap v541: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-09-21T06:58:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:58:59 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:58:59.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:59 smithi114 ceph-mon[30654]: pgmap v541: 129 pgs: 129 active+clean; 2.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.3 MiB/s wr, 672 op/s 2024-09-21T06:58:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:58:59 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:58:59.813 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:58:59.814 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:59:00.503 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 -- 172.21.15.114:0/3558280328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c1039b0 msgr2=0x7ff21c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 --2- 172.21.15.114:0/3558280328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c1039b0 0x7ff21c103dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009900 tx=0x7ff20c02a400 comp rx=0 tx=0).stop 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 -- 172.21.15.114:0/3558280328 shutdown_connections 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 --2- 172.21.15.114:0/3558280328 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff21c109c40 0x7ff21c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 --2- 172.21.15.114:0/3558280328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 --2- 172.21.15.114:0/3558280328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c1039b0 0x7ff21c103dc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 -- 172.21.15.114:0/3558280328 >> 172.21.15.114:0/3558280328 conn(0x7ff21c0ff440 msgr2=0x7ff21c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 -- 172.21.15.114:0/3558280328 shutdown_connections 2024-09-21T06:59:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.296+0000 7ff221600640 1 -- 172.21.15.114:0/3558280328 wait complete. 2024-09-21T06:59:01.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: pgmap v542: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-09-21T06:59:01.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:59:01.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:01 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:59:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.297+0000 7ff221600640 1 Processor -- start 2024-09-21T06:59:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 -- start start 2024-09-21T06:59:01.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff21c1039b0 0x7ff21c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c109c40 0x7ff21c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c113950 con 0x7ff21c1039b0 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1137d0 con 0x7ff21c1092f0 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff221600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c113ad0 con 0x7ff21c109c40 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.298+0000 7ff213e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff220c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c109c40 0x7ff21c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45778/0 (socket says 172.21.15.114:45778) 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 -- 172.21.15.114:0/3503457758 learned_addr learned my addr 172.21.15.114:0/3503457758 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff21be00640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff21c1039b0 0x7ff21c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 -- 172.21.15.114:0/3503457758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c109c40 msgr2=0x7ff21c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c109c40 0x7ff21c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 -- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff21c1039b0 msgr2=0x7ff21c19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff21c1039b0 0x7ff21c19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 -- 172.21.15.114:0/3503457758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21c1a2490 con 0x7ff21c1092f0 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.299+0000 7ff213e00640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c19cac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e9c0 tx=0x7ff20400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.300+0000 7ff219600640 1 -- 172.21.15.114:0/3503457758 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20400ccc0 con 0x7ff21c1092f0 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.300+0000 7ff219600640 1 -- 172.21.15.114:0/3503457758 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2040054d0 con 0x7ff21c1092f0 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.300+0000 7ff219600640 1 -- 172.21.15.114:0/3503457758 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204010430 con 0x7ff21c1092f0 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.300+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21c1a2780 con 0x7ff21c1092f0 2024-09-21T06:59:01.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.300+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21c1a8ee0 con 0x7ff21c1092f0 2024-09-21T06:59:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.304+0000 7ff229867740 1 Processor -- start 2024-09-21T06:59:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.304+0000 7ff229867740 1 -- start start 2024-09-21T06:59:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.304+0000 7ff229867740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 0x55fd43cec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.304+0000 7ff229867740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd43913490 0x55fd43cecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.304+0000 7ff229867740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43ced190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff229867740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fd43c59e90 con 0x55fd4391b670 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff229867740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd43c492c0 con 0x55fd43913490 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff229867740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd43c4d0d0 con 0x55fd4393c350 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 0x55fd43cec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 0x55fd43cec710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43740/0 (socket says 172.21.15.114:43740) 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 -- 172.21.15.114:0/1139403094 learned_addr learned my addr 172.21.15.114:0/1139403094 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff21be00640 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd43913490 0x55fd43cecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 -- 172.21.15.114:0/1139403094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd43913490 msgr2=0x55fd43cecc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd43913490 0x55fd43cecc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 -- 172.21.15.114:0/1139403094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 msgr2=0x55fd43ced190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43ced190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 -- 172.21.15.114:0/1139403094 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd439a7930 con 0x55fd4393c350 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff21be00640 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd43913490 0x55fd43cecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff219600640 1 -- 172.21.15.114:0/3503457758 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff2040105d0 con 0x7ff21c1092f0 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.305+0000 7ff220c00640 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 0x55fd43cec710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff214009fd0 tx=0x7ff2140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fd7f801f8916ff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.306+0000 7ff211200640 1 -- 172.21.15.114:0/1139403094 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214004d20 con 0x55fd4393c350 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.306+0000 7ff211200640 1 -- 172.21.15.114:0/1139403094 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff214004050 con 0x55fd4393c350 2024-09-21T06:59:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.306+0000 7ff211200640 1 -- 172.21.15.114:0/1139403094 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff214005220 con 0x55fd4393c350 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 -- 172.21.15.114:0/1139403094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 msgr2=0x55fd43cec710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 0x55fd43cec710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff214009fd0 tx=0x7ff2140118d0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 -- 172.21.15.114:0/1139403094 shutdown_connections 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43ced190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd43913490 0x55fd43cecc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 --2- 172.21.15.114:0/1139403094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4393c350 0x55fd43cec710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 -- 172.21.15.114:0/1139403094 >> 172.21.15.114:0/1139403094 conn(0x55fd43930740 msgr2=0x55fd43cea600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 -- 172.21.15.114:0/1139403094 shutdown_connections 2024-09-21T06:59:01.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.307+0000 7ff229867740 1 -- 172.21.15.114:0/1139403094 wait complete. 2024-09-21T06:59:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.308+0000 7ff219600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2000b6610 0x7ff2000b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.308+0000 7ff219600640 1 -- 172.21.15.114:0/3503457758 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff2040e7dd0 con 0x7ff21c1092f0 2024-09-21T06:59:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.308+0000 7ff220c00640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2000b6610 0x7ff2000b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.308+0000 7ff229867740 1 -- start start 2024-09-21T06:59:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.309+0000 7ff229867740 1 Processor -- start 2024-09-21T06:59:01.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff220c00640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2000b6610 0x7ff2000b8b20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff2140049d0 tx=0x7ff2140048d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4391b670 0x55fd43cf8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4393c350 0x55fd43d041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fd43aaa890 con 0x55fd4393c350 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd43be4160 con 0x55fd4391b670 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fd43cfaef0 con 0x55fd43913490 2024-09-21T06:59:01.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.310+0000 7ff229867740 10 client.0 _ll_register_callbacks cb 0x55fd43c00a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43742/0 (socket says 172.21.15.114:43742) 2024-09-21T06:59:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 -- 172.21.15.114:0/3570514713 learned_addr learned my addr 172.21.15.114:0/3570514713 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff213e00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4391b670 0x55fd43cf8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff220c00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4393c350 0x55fd43d041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 -- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4391b670 msgr2=0x55fd43cf8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4391b670 0x55fd43cf8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 -- 172.21.15.114:0/3570514713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4393c350 msgr2=0x55fd43d041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4393c350 0x55fd43d041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 -- 172.21.15.114:0/3570514713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd43d047d0 con 0x55fd43913490 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.311+0000 7ff21be00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf82e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda70b960d3ad0db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:01.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff209600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff209600640 1 -- 172.21.15.114:0/3570514713 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c035040 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff209600640 1 -- 172.21.15.114:0/3570514713 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff20c005450 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff214004050 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff209600640 1 -- 172.21.15.114:0/3570514713 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff20c04e050 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff229867740 10 client.26848 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.312+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fd43d070e0 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.313+0000 7ff209600640 1 -- 172.21.15.114:0/3570514713 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff214004050 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.313+0000 7ff209600640 1 -- 172.21.15.114:0/3570514713 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7ff20c003820 con 0x55fd43913490 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.313+0000 7ff229867740 10 client.26848 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:01.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.313+0000 7ff229867740 10 client.26848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:01.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.313+0000 7ff229867740 20 client.26848 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:01.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.313+0000 7ff229867740 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438facd0 0x55fd43d0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.314+0000 7ff229867740 4 client.26848 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:01.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.314+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd43c5bde0 con 0x55fd438facd0 2024-09-21T06:59:01.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.314+0000 7ff213e00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438facd0 0x55fd43d0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.314+0000 7ff213e00640 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438facd0 0x55fd43d0a690 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c11a2f6c37ef5d9 server_cookie=3f3a183e8133c1b8 in_seq=0 out_seq=0 2024-09-21T06:59:01.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.314+0000 7ff209600640 10 client.26848 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:01.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.315+0000 7ff209600640 1 -- 172.21.15.114:0/3570514713 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd43c5bde0 con 0x55fd438facd0 2024-09-21T06:59:01.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.315+0000 7ff209600640 10 client.26848 handle_command_reply: tid=0 2024-09-21T06:59:01.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.315+0000 7ff229867740 1 client.26848 shutdown 2024-09-21T06:59:01.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 msgr2=0x55fd43cf82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf82e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff20c000c00 tx=0x7ff20c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 shutdown_connections 2024-09-21T06:59:01.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4393c350 0x55fd43d041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4391b670 0x55fd43cf8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438facd0 0x55fd43d0a690 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 --2- 172.21.15.114:0/3570514713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf82e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff209600640 0 client.26848 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:01.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.316+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 >> 172.21.15.114:0/3570514713 conn(0x55fd43930740 msgr2=0x55fd43cef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:01.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.317+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 shutdown_connections 2024-09-21T06:59:01.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.317+0000 7ff229867740 1 -- 172.21.15.114:0/3570514713 wait complete. 2024-09-21T06:59:01.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.317+0000 7ff229867740 20 client.26848 trim_cache size 0 max 16384 2024-09-21T06:59:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.341+0000 7ff229867740 1 Processor -- start 2024-09-21T06:59:01.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.341+0000 7ff229867740 1 -- start start 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff229867740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff229867740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43cf1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff229867740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43ce9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff229867740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1aea90 con 0x55fd4391b670 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff229867740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1b06f0 con 0x55fd4393c350 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff229867740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1af160 con 0x55fd43913490 2024-09-21T06:59:01.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff213e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43cf1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.342+0000 7ff213e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43cf1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33806/0 (socket says 172.21.15.114:33806) 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff220c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43ce9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff213e00640 1 -- 172.21.15.114:0/1780767194 learned_addr learned my addr 172.21.15.114:0/1780767194 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 -- 172.21.15.114:0/1780767194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 msgr2=0x55fd43ce9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43ce9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 -- 172.21.15.114:0/1780767194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 msgr2=0x55fd43cf1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43cf1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 -- 172.21.15.114:0/1780767194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd43cf93a0 con 0x55fd43913490 2024-09-21T06:59:01.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.343+0000 7ff21be00640 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf1820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff20c02a9f0 tx=0x7ff20c0314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b855bc892986f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:01.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.344+0000 7ff20be00640 1 -- 172.21.15.114:0/1780767194 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c035070 con 0x55fd43913490 2024-09-21T06:59:01.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.344+0000 7ff20be00640 1 -- 172.21.15.114:0/1780767194 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20c040070 con 0x55fd43913490 2024-09-21T06:59:01.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.344+0000 7ff20be00640 1 -- 172.21.15.114:0/1780767194 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0046c0 con 0x55fd43913490 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 -- 172.21.15.114:0/1780767194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 msgr2=0x55fd43cf1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf1820 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff20c02a9f0 tx=0x7ff20c0314a0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 -- 172.21.15.114:0/1780767194 shutdown_connections 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43ce9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd4391b670 0x55fd43cf1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 --2- 172.21.15.114:0/1780767194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd43913490 0x55fd43cf1820 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.345+0000 7ff229867740 1 -- 172.21.15.114:0/1780767194 >> 172.21.15.114:0/1780767194 conn(0x55fd43930740 msgr2=0x55fd43cf69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.347+0000 7ff229867740 1 -- 172.21.15.114:0/1780767194 shutdown_connections 2024-09-21T06:59:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.347+0000 7ff229867740 1 -- 172.21.15.114:0/1780767194 wait complete. 2024-09-21T06:59:01.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.348+0000 7ff229867740 1 -- start start 2024-09-21T06:59:01.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.348+0000 7ff229867740 1 Processor -- start 2024-09-21T06:59:01.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd43913490 0x55fd43d00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 0x55fd43d00890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43cfb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1aff60 con 0x55fd43913490 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1af830 con 0x55fd4393c350 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff21c1aca90 con 0x55fd4391b670 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.349+0000 7ff229867740 10 client.0 _ll_register_callbacks cb 0x7ff204005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 0x55fd43d00890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 0x55fd43d00890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43756/0 (socket says 172.21.15.114:43756) 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 -- 172.21.15.114:0/3861692572 learned_addr learned my addr 172.21.15.114:0/3861692572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 -- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 msgr2=0x55fd43cfb100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43cfb100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 -- 172.21.15.114:0/3861692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd43913490 msgr2=0x55fd43d00350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd43913490 0x55fd43d00350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 -- 172.21.15.114:0/3861692572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd43cfb670 con 0x55fd4391b670 2024-09-21T06:59:01.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff213e00640 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 0x55fd43d00890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff204004650 tx=0x7ff2040a9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674d890a098b6d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.350+0000 7ff209600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.351+0000 7ff209600640 1 -- 172.21.15.114:0/3861692572 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2040ad360 con 0x55fd4391b670 2024-09-21T06:59:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.351+0000 7ff209600640 1 -- 172.21.15.114:0/3861692572 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff2040e53d0 con 0x55fd4391b670 2024-09-21T06:59:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.351+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff20c040070 con 0x55fd4391b670 2024-09-21T06:59:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.351+0000 7ff209600640 1 -- 172.21.15.114:0/3861692572 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff204005680 con 0x55fd4391b670 2024-09-21T06:59:01.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.351+0000 7ff229867740 10 client.26854 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.351+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fd43d75870 con 0x55fd4391b670 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff209600640 1 -- 172.21.15.114:0/3861692572 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20c040070 con 0x55fd4391b670 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff209600640 1 -- 172.21.15.114:0/3861692572 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7ff2040aeb80 con 0x55fd4391b670 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff229867740 10 client.26854 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff229867740 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff229867740 20 client.26854 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff229867740 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438d0080 0x55fd43d78150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff229867740 4 client.26854 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.352+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fd43c57890 con 0x55fd438d0080 2024-09-21T06:59:01.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.353+0000 7ff21be00640 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438d0080 0x55fd43d78150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:01.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.354+0000 7ff21be00640 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438d0080 0x55fd43d78150 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f14bb6c97ff2f6 server_cookie=cbaea684bfb55137 in_seq=0 out_seq=0 2024-09-21T06:59:01.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.354+0000 7ff209600640 10 client.26854 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.355+0000 7ff209600640 1 -- 172.21.15.114:0/3861692572 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fd43c57890 con 0x55fd438d0080 2024-09-21T06:59:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.355+0000 7ff209600640 10 client.26854 handle_command_reply: tid=0 2024-09-21T06:59:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.355+0000 7ff229867740 1 client.26854 shutdown 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 msgr2=0x55fd43d00890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 0x55fd43d00890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff204004650 tx=0x7ff2040a9f90 comp rx=0 tx=0).stop 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 shutdown_connections 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd4393c350 0x55fd43cfb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55fd438d0080 0x55fd43d78150 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fd4391b670 0x55fd43d00890 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff204004650 tx=0x7ff2040a9f90 comp rx=0 tx=0).stop 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 --2- 172.21.15.114:0/3861692572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fd43913490 0x55fd43d00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 >> 172.21.15.114:0/3861692572 conn(0x55fd43930740 msgr2=0x55fd43cf1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff209600640 0 client.26854 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 shutdown_connections 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.356+0000 7ff229867740 1 -- 172.21.15.114:0/3861692572 wait complete. 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "7f551d89-35a0-4b4f-9844-0afa60437f35", 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T06:59:01.359 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:59:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.357+0000 7ff229867740 20 client.26854 trim_cache size 0 max 16384 2024-09-21T06:59:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.357+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2000b6610 msgr2=0x7ff2000b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.357+0000 7ff221600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2000b6610 0x7ff2000b8b20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff2140049d0 tx=0x7ff2140048d0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.357+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 msgr2=0x7ff21c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.357+0000 7ff221600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c19cac0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff20400e9c0 tx=0x7ff20400ee80 comp rx=0 tx=0).stop 2024-09-21T06:59:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 shutdown_connections 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff21c109c40 0x7ff21c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff2000b6610 0x7ff2000b8b20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff21c1092f0 0x7ff21c19cac0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 --2- 172.21.15.114:0/3503457758 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff21c1039b0 0x7ff21c19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 >> 172.21.15.114:0/3503457758 conn(0x7ff21c0ff440 msgr2=0x7ff21c108940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 shutdown_connections 2024-09-21T06:59:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:01.358+0000 7ff221600640 1 -- 172.21.15.114:0/3503457758 wait complete. 2024-09-21T06:59:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: pgmap v542: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-09-21T06:59:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:59:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:01 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: pgmap v542: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.7 MiB/s wr, 640 op/s 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:59:01.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:59:01.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:59:01.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:01 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T06:59:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi114 to 176.4M 2024-09-21T06:59:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: Unable to set osd_memory_target on smithi114 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi114 to 176.4M 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: Unable to set osd_memory_target on smithi114 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:59:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T06:59:02.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.541 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi114 to 176.4M 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: Unable to set osd_memory_target on smithi114 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.542 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:02.590 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:59:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 265.0 s, 486.0 tps, lat 204.668 ms stddev 217.121 2024-09-21T06:59:03.134 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:03 smithi193 ceph-mon[34601]: pgmap v543: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 514 op/s 2024-09-21T06:59:03.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:03 smithi114 ceph-mon[30654]: pgmap v543: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 514 op/s 2024-09-21T06:59:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T06:59:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:03 smithi183 ceph-mon[34219]: pgmap v543: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 5.4 MiB/s wr, 514 op/s 2024-09-21T06:59:04.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.302+0000 7fedfd600640 1 -- 172.21.15.114:0/1200897889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8114440 msgr2=0x7fedf8116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 --2- 172.21.15.114:0/1200897889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8114440 0x7fedf8116910 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009ca0 tx=0x7fedf002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 -- 172.21.15.114:0/1200897889 shutdown_connections 2024-09-21T06:59:04.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 --2- 172.21.15.114:0/1200897889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8114440 0x7fedf8116910 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 --2- 172.21.15.114:0/1200897889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedf8075dd0 0x7fedf80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 --2- 172.21.15.114:0/1200897889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf80757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 -- 172.21.15.114:0/1200897889 >> 172.21.15.114:0/1200897889 conn(0x7fedf80ff5e0 msgr2=0x7fedf8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 -- 172.21.15.114:0/1200897889 shutdown_connections 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.303+0000 7fedfd600640 1 -- 172.21.15.114:0/1200897889 wait complete. 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.305+0000 7fedfd600640 1 Processor -- start 2024-09-21T06:59:04.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.306+0000 7fedfd600640 1 -- start start 2024-09-21T06:59:04.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.308+0000 7fedfd600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075dd0 0x7fedf8074dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.309+0000 7fedfd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.309+0000 7fedfd600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedf8114440 0x7fedf806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.310+0000 7fedfd600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fedf811a130 con 0x7fedf8114440 2024-09-21T06:59:04.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.311+0000 7fedfd600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedf8119fb0 con 0x7fedf8075dd0 2024-09-21T06:59:04.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.312+0000 7fedfd600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedf811a2b0 con 0x7fedf8077380 2024-09-21T06:59:04.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.312+0000 7fedf6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.312+0000 7fedf6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf806e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43786/0 (socket says 172.21.15.114:43786) 2024-09-21T06:59:04.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.312+0000 7fedf6000640 1 -- 172.21.15.114:0/3881680457 learned_addr learned my addr 172.21.15.114:0/3881680457 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:04.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.310+0000 7fedf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075dd0 0x7fedf8074dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf7400640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedf8114440 0x7fedf806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf6000640 1 -- 172.21.15.114:0/3881680457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075dd0 msgr2=0x7fedf8074dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf6000640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075dd0 0x7fedf8074dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf6000640 1 -- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedf8114440 msgr2=0x7fedf806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf6a00640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075dd0 0x7fedf8074dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:04.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf6000640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedf8114440 0x7fedf806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.313+0000 7fedf6000640 1 -- 172.21.15.114:0/3881680457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf806edb0 con 0x7fedf8077380 2024-09-21T06:59:04.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.314+0000 7fedf6000640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf806e0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fede400b450 tx=0x7fede400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:04.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.317+0000 7fedef400640 1 -- 172.21.15.114:0/3881680457 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4015020 con 0x7fedf8077380 2024-09-21T06:59:04.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.317+0000 7fedef400640 1 -- 172.21.15.114:0/3881680457 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede4011d40 con 0x7fedf8077380 2024-09-21T06:59:04.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.317+0000 7fedef400640 1 -- 172.21.15.114:0/3881680457 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede400f340 con 0x7fedf8077380 2024-09-21T06:59:04.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.317+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf8073a40 con 0x7fedf8077380 2024-09-21T06:59:04.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.318+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf8073fb0 con 0x7fedf8077380 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.326+0000 7fee04286740 1 Processor -- start 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.326+0000 7fee04286740 1 -- start start 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.327+0000 7fee04286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.327+0000 7fee04286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649aded70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.327+0000 7fee04286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649aded75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.327+0000 7fee04286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649ade4df70 con 0x5649adb30070 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.327+0000 7fee04286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649ade3d3a0 con 0x5649adb0f390 2024-09-21T06:59:04.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.327+0000 7fee04286740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649ade411b0 con 0x5649adb071b0 2024-09-21T06:59:04.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.334+0000 7fedf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.334+0000 7fedf7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33828/0 (socket says 172.21.15.114:33828) 2024-09-21T06:59:04.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.334+0000 7fedf7400640 1 -- 172.21.15.114:0/3475563358 learned_addr learned my addr 172.21.15.114:0/3475563358 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:04.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.334+0000 7fedf6a00640 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649aded70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.335+0000 7fedf7400640 1 -- 172.21.15.114:0/3475563358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 msgr2=0x5649aded70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.335+0000 7fedf7400640 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649aded70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.335+0000 7fedf7400640 1 -- 172.21.15.114:0/3475563358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 msgr2=0x5649aded75e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:04.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.335+0000 7fedf7400640 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649aded75e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.335+0000 7fedf7400640 1 -- 172.21.15.114:0/3475563358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ade07c90 con 0x5649adb30070 2024-09-21T06:59:04.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.337+0000 7fedef400640 1 -- 172.21.15.114:0/3881680457 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fede4018020 con 0x7fedf8077380 2024-09-21T06:59:04.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.339+0000 7fedf7400640 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded6b60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fedf0031530 tx=0x7fedf0031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b0945278f271bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:04.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.340+0000 7fede3400640 1 -- 172.21.15.114:0/3475563358 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0048070 con 0x5649adb30070 2024-09-21T06:59:04.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.340+0000 7fede3400640 1 -- 172.21.15.114:0/3475563358 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf0036040 con 0x5649adb30070 2024-09-21T06:59:04.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.340+0000 7fede3400640 1 -- 172.21.15.114:0/3475563358 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0048070 con 0x5649adb30070 2024-09-21T06:59:04.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.342+0000 7fee04286740 1 -- 172.21.15.114:0/3475563358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 msgr2=0x5649aded6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.342+0000 7fee04286740 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded6b60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fedf0031530 tx=0x7fedf0031560 comp rx=0 tx=0).stop 2024-09-21T06:59:04.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.343+0000 7fedef400640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feddc0b6700 0x7feddc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.344+0000 7fedef400640 1 -- 172.21.15.114:0/3881680457 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fede40e7370 con 0x7fedf8077380 2024-09-21T06:59:04.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.344+0000 7fee04286740 1 -- 172.21.15.114:0/3475563358 shutdown_connections 2024-09-21T06:59:04.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.344+0000 7fee04286740 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649aded75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.344+0000 7fee04286740 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649aded70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.344+0000 7fee04286740 1 --2- 172.21.15.114:0/3475563358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded6b60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.344+0000 7fee04286740 1 -- 172.21.15.114:0/3475563358 >> 172.21.15.114:0/3475563358 conn(0x5649adb24460 msgr2=0x5649aded4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:04.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.345+0000 7fee04286740 1 -- 172.21.15.114:0/3475563358 shutdown_connections 2024-09-21T06:59:04.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.345+0000 7fee04286740 1 -- 172.21.15.114:0/3475563358 wait complete. 2024-09-21T06:59:04.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.346+0000 7fee04286740 1 -- start start 2024-09-21T06:59:04.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.346+0000 7fee04286740 1 Processor -- start 2024-09-21T06:59:04.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.348+0000 7fee04286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.348+0000 7fee04286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649adef3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.348+0000 7fee04286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adeff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.348+0000 7fedf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649adef3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.348+0000 7fee04286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649adc9e700 con 0x5649adb30070 2024-09-21T06:59:04.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.348+0000 7fee04286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649addd8540 con 0x5649adb0f390 2024-09-21T06:59:04.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.349+0000 7fedf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649adef3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45828/0 (socket says 172.21.15.114:45828) 2024-09-21T06:59:04.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.349+0000 7fedf6000640 1 -- 172.21.15.114:0/3000484953 learned_addr learned my addr 172.21.15.114:0/3000484953 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:04.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.349+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649adef6240 con 0x5649adb071b0 2024-09-21T06:59:04.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.349+0000 7fedf7400640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adeff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.349+0000 7fee04286740 10 client.0 _ll_register_callbacks cb 0x5649adde8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:04.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.349+0000 7fedf6a00640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.350+0000 7fedf6000640 1 -- 172.21.15.114:0/3000484953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 msgr2=0x5649adef3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.350+0000 7fedf6000640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.350+0000 7fedf6000640 1 -- 172.21.15.114:0/3000484953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 msgr2=0x5649adeff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.350+0000 7fedf6000640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adeff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.350+0000 7fedf6000640 1 -- 172.21.15.114:0/3000484953 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649adb9b650 con 0x5649adb0f390 2024-09-21T06:59:04.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.351+0000 7fedd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:04.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.351+0000 7fedf6000640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649adef3c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fede40a93c0 tx=0x7fede4011c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc135ef787699af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:04.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.355+0000 7fedd7e00640 1 -- 172.21.15.114:0/3000484953 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede40b0030 con 0x5649adb0f390 2024-09-21T06:59:04.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.355+0000 7fedd7e00640 1 -- 172.21.15.114:0/3000484953 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fede40cee00 con 0x5649adb0f390 2024-09-21T06:59:04.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.356+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649adddab50 con 0x5649adb0f390 2024-09-21T06:59:04.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.356+0000 7fedd7e00640 1 -- 172.21.15.114:0/3000484953 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fede40ac900 con 0x5649adb0f390 2024-09-21T06:59:04.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.356+0000 7fee04286740 10 client.26900 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:04.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.356+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5649addf5910 con 0x5649adb0f390 2024-09-21T06:59:04.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.357+0000 7fedd7e00640 1 -- 172.21.15.114:0/3000484953 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5649adddab50 con 0x5649adb0f390 2024-09-21T06:59:04.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.357+0000 7fedd7e00640 1 -- 172.21.15.114:0/3000484953 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fede40d41a0 con 0x5649adb0f390 2024-09-21T06:59:04.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.358+0000 7fee04286740 10 client.26900 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:04.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.358+0000 7fee04286740 10 client.26900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:04.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.358+0000 7fee04286740 20 client.26900 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:04.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.358+0000 7fee04286740 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adaee9f0 0x5649adf05730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.358+0000 7fee04286740 4 client.26900 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:04.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.358+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649ade4fec0 con 0x5649adaee9f0 2024-09-21T06:59:04.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.359+0000 7fedf6a00640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adaee9f0 0x5649adf05730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.361+0000 7fedf6a00640 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adaee9f0 0x5649adf05730 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17edb8a9906cd6 server_cookie=9a9241a685e197b9 in_seq=0 out_seq=0 2024-09-21T06:59:04.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.361+0000 7fedd7e00640 10 client.26900 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:04.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.365+0000 7fedd7e00640 1 -- 172.21.15.114:0/3000484953 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649ade4fec0 con 0x5649adaee9f0 2024-09-21T06:59:04.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.365+0000 7fedd7e00640 10 client.26900 handle_command_reply: tid=0 2024-09-21T06:59:04.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.365+0000 7fee04286740 1 client.26900 shutdown 2024-09-21T06:59:04.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 msgr2=0x5649adef3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649adef3c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fede40a93c0 tx=0x7fede4011c60 comp rx=0 tx=0).stop 2024-09-21T06:59:04.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 shutdown_connections 2024-09-21T06:59:04.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adeff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb0f390 0x5649adef3c80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adaee9f0 0x5649adf05730 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 --2- 172.21.15.114:0/3000484953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 >> 172.21.15.114:0/3000484953 conn(0x5649adb24460 msgr2=0x5649aded8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:04.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.366+0000 7fedd7e00640 0 client.26900 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:04.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.367+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 shutdown_connections 2024-09-21T06:59:04.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.367+0000 7fee04286740 1 -- 172.21.15.114:0/3000484953 wait complete. 2024-09-21T06:59:04.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.367+0000 7fee04286740 20 client.26900 trim_cache size 0 max 16384 2024-09-21T06:59:04.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.369+0000 7fedf7400640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feddc0b6700 0x7feddc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.379+0000 7fedf7400640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feddc0b6700 0x7feddc0b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fedf00094e0 tx=0x7fedf002a8f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:04.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.417+0000 7fee04286740 1 Processor -- start 2024-09-21T06:59:04.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 -- start start 2024-09-21T06:59:04.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 0x5649aded4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adac3da0 0x5649adef0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fedd800fec0 con 0x5649adb30070 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8011b20 con 0x5649adb071b0 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.420+0000 7fee04286740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8010590 con 0x5649adac3da0 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 0x5649aded4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 0x5649aded4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45844/0 (socket says 172.21.15.114:45844) 2024-09-21T06:59:04.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 -- 172.21.15.114:0/1257674366 learned_addr learned my addr 172.21.15.114:0/1257674366 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:04.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 -- 172.21.15.114:0/1257674366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adac3da0 msgr2=0x5649adef0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adac3da0 0x5649adef0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 -- 172.21.15.114:0/1257674366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 msgr2=0x5649aded4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.421+0000 7fedf6a00640 1 -- 172.21.15.114:0/1257674366 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649adef4800 con 0x5649adb071b0 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.422+0000 7fedf6a00640 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 0x5649aded4000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fede80045a0 tx=0x7fede800bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e7081ddfefad71b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.422+0000 7fede2a00640 1 -- 172.21.15.114:0/1257674366 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8003b30 con 0x5649adb071b0 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.422+0000 7fede2a00640 1 -- 172.21.15.114:0/1257674366 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede8003cd0 con 0x5649adb071b0 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.422+0000 7fede2a00640 1 -- 172.21.15.114:0/1257674366 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8011680 con 0x5649adb071b0 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 -- 172.21.15.114:0/1257674366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 msgr2=0x5649aded4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 0x5649aded4000 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fede80045a0 tx=0x7fede800bd90 comp rx=0 tx=0).stop 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 -- 172.21.15.114:0/1257674366 shutdown_connections 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adac3da0 0x5649adef0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649aded4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 --2- 172.21.15.114:0/1257674366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adb071b0 0x5649aded4000 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 -- 172.21.15.114:0/1257674366 >> 172.21.15.114:0/1257674366 conn(0x5649adb24460 msgr2=0x5649adeee680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 -- 172.21.15.114:0/1257674366 shutdown_connections 2024-09-21T06:59:04.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.424+0000 7fee04286740 1 -- 172.21.15.114:0/1257674366 wait complete. 2024-09-21T06:59:04.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.425+0000 7fee04286740 1 -- start start 2024-09-21T06:59:04.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.425+0000 7fee04286740 1 Processor -- start 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 0x5649adefbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adef7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8011390 con 0x5649adb30070 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8010c60 con 0x5649adac3da0 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fedd800dea0 con 0x5649adb071b0 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fee04286740 10 client.0 _ll_register_callbacks cb 0x7fede40a9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:04.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fedf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 0x5649adefbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fedf6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 0x5649adefbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45858/0 (socket says 172.21.15.114:45858) 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.427+0000 7fedf6a00640 1 -- 172.21.15.114:0/2729555186 learned_addr learned my addr 172.21.15.114:0/2729555186 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.428+0000 7fedf6000640 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.428+0000 7fedf6a00640 1 -- 172.21.15.114:0/2729555186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 msgr2=0x5649adef72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.428+0000 7fedf6a00640 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.428+0000 7fedf6a00640 1 -- 172.21.15.114:0/2729555186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 msgr2=0x5649adef7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.428+0000 7fedf6a00640 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adef7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.428+0000 7fedf6a00640 1 -- 172.21.15.114:0/2729555186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649adef7fe0 con 0x5649adac3da0 2024-09-21T06:59:04.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.429+0000 7fedf6a00640 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 0x5649adefbd80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fede8004630 tx=0x7fede8003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be336266c637f710 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:04.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.429+0000 7fedd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:04.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.433+0000 7fedd7e00640 1 -- 172.21.15.114:0/2729555186 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede801c6f0 con 0x5649adac3da0 2024-09-21T06:59:04.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.433+0000 7fedd7e00640 1 -- 172.21.15.114:0/2729555186 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fede801c890 con 0x5649adac3da0 2024-09-21T06:59:04.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.435+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede8003cd0 con 0x5649adac3da0 2024-09-21T06:59:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.437+0000 7fedd7e00640 1 -- 172.21.15.114:0/2729555186 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fede8005040 con 0x5649adac3da0 2024-09-21T06:59:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.437+0000 7fee04286740 10 client.26906 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:04.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.437+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5649adf70080 con 0x5649adac3da0 2024-09-21T06:59:04.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.439+0000 7fedd7e00640 1 -- 172.21.15.114:0/2729555186 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede8003cd0 con 0x5649adac3da0 2024-09-21T06:59:04.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.439+0000 7fedd7e00640 1 -- 172.21.15.114:0/2729555186 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fede802b070 con 0x5649adac3da0 2024-09-21T06:59:04.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.440+0000 7fee04286740 10 client.26906 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:04.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.440+0000 7fee04286740 10 client.26906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:04.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.441+0000 7fee04286740 20 client.26906 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:04.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.441+0000 7fee04286740 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adf70890 0x5649adf72cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:04.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.441+0000 7fee04286740 4 client.26906 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:59:04.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.441+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5649ade4b970 con 0x5649adf70890 2024-09-21T06:59:04.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.443+0000 7fedf6000640 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adf70890 0x5649adf72cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:04.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.444+0000 7fedf6000640 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adf70890 0x5649adf72cb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dda3e4e2efa4998 server_cookie=d7199ac7479e414c in_seq=0 out_seq=0 2024-09-21T06:59:04.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.444+0000 7fedd7e00640 10 client.26906 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:04.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.445+0000 7fedd7e00640 1 -- 172.21.15.114:0/2729555186 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x5649ade4b970 con 0x5649adf70890 2024-09-21T06:59:04.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.445+0000 7fedd7e00640 10 client.26906 handle_command_reply: tid=0 2024-09-21T06:59:04.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.445+0000 7fee04286740 1 client.26906 shutdown 2024-09-21T06:59:04.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.445+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 msgr2=0x5649adefbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.445+0000 7fee04286740 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 0x5649adefbd80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fede8004630 tx=0x7fede8003e80 comp rx=0 tx=0).stop 2024-09-21T06:59:04.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 shutdown_connections 2024-09-21T06:59:04.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fee04286740 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649adb30070 0x5649adef7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fee04286740 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5649adf70890 0x5649adf72cb0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fee04286740 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649adb071b0 0x5649adef72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fee04286740 1 --2- 172.21.15.114:0/2729555186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649adac3da0 0x5649adefbd80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fedd7e00640 0 client.26906 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.448+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 >> 172.21.15.114:0/2729555186 conn(0x5649adb24460 msgr2=0x5649adef1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.449+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 shutdown_connections 2024-09-21T06:59:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.449+0000 7fee04286740 1 -- 172.21.15.114:0/2729555186 wait complete. 2024-09-21T06:59:04.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.449+0000 7fee04286740 20 client.26906 trim_cache size 0 max 16384 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "7f551d89-35a0-4b4f-9844-0afa60437f35", 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:04.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T06:59:04.454 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T06:59:04.455 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:04.456 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:04.457 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:04.458 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:59:04.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.451+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feddc0b6700 msgr2=0x7feddc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.451+0000 7fedfd600640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feddc0b6700 0x7feddc0b8c10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fedf00094e0 tx=0x7fedf002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 msgr2=0x7fedf806e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf806e0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fede400b450 tx=0x7fede400b910 comp rx=0 tx=0).stop 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 shutdown_connections 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fedf8114440 0x7fedf806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedf8077380 0x7fedf806e0a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feddc0b6700 0x7feddc0b8c10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 --2- 172.21.15.114:0/3881680457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf8075dd0 0x7fedf8074dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.452+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 >> 172.21.15.114:0/3881680457 conn(0x7fedf80ff5e0 msgr2=0x7fedf8110340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.453+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 shutdown_connections 2024-09-21T06:59:04.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:04.453+0000 7fedfd600640 1 -- 172.21.15.114:0/3881680457 wait complete. 2024-09-21T06:59:05.366 INFO:tasks.cephfs.filesystem:scrub status for tag:7f551d89-35a0-4b4f-9844-0afa60437f35 - {'path': '/', 'tag': '7f551d89-35a0-4b4f-9844-0afa60437f35', 'options': 'recursive,force'} 2024-09-21T06:59:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:05 smithi193 ceph-mon[34601]: pgmap v544: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-21T06:59:05.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:05 smithi114 ceph-mon[30654]: pgmap v544: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-21T06:59:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:05 smithi183 ceph-mon[34219]: pgmap v544: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.4 MiB/s wr, 662 op/s 2024-09-21T06:59:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:07 smithi193 ceph-mon[34601]: pgmap v545: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.8 MiB/s wr, 611 op/s 2024-09-21T06:59:07.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:07 smithi114 ceph-mon[30654]: pgmap v545: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.8 MiB/s wr, 611 op/s 2024-09-21T06:59:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:07 smithi183 ceph-mon[34219]: pgmap v545: 129 pgs: 129 active+clean; 2.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.8 MiB/s wr, 611 op/s 2024-09-21T06:59:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 270.0 s, 159.2 tps, lat 635.376 ms stddev 919.845 2024-09-21T06:59:09.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:09 smithi193 ceph-mon[34601]: pgmap v546: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-09-21T06:59:09.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:09 smithi114 ceph-mon[30654]: pgmap v546: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-09-21T06:59:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:09 smithi183 ceph-mon[34219]: pgmap v546: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 12 MiB/s wr, 1.16k op/s 2024-09-21T06:59:11.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:11 smithi193 ceph-mon[34601]: pgmap v547: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-09-21T06:59:11.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:11 smithi114 ceph-mon[30654]: pgmap v547: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-09-21T06:59:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:11 smithi183 ceph-mon[34219]: pgmap v547: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 14 MiB/s wr, 1.29k op/s 2024-09-21T06:59:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 -- 172.21.15.114:0/4241775498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 msgr2=0x7f56380757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 --2- 172.21.15.114:0/4241775498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56380757b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f562802a410 comp rx=0 tx=0).stop 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 -- 172.21.15.114:0/4241775498 shutdown_connections 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 --2- 172.21.15.114:0/4241775498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381141d0 0x7f56381166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 --2- 172.21.15.114:0/4241775498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 0x7f56380761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 --2- 172.21.15.114:0/4241775498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56380757b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.536+0000 7f5637200640 1 -- 172.21.15.114:0/4241775498 >> 172.21.15.114:0/4241775498 conn(0x7f56380ff330 msgr2=0x7f5638101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.537+0000 7f5637200640 1 -- 172.21.15.114:0/4241775498 shutdown_connections 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.537+0000 7f5637200640 1 -- 172.21.15.114:0/4241775498 wait complete. 2024-09-21T06:59:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.538+0000 7f5637200640 1 Processor -- start 2024-09-21T06:59:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.538+0000 7f5637200640 1 -- start start 2024-09-21T06:59:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.539+0000 7f5637200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 0x7f56381a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.539+0000 7f5637200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56381a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.539+0000 7f5637200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381141d0 0x7f56381a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5635400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56381a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5635400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56381a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47976/0 (socket says 172.21.15.114:47976) 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5637200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5638119e60 con 0x7f5638075dd0 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5637200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5638119ce0 con 0x7f56381141d0 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5637200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5638119fe0 con 0x7f5638077390 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5635400640 1 -- 172.21.15.114:0/3293443162 learned_addr learned my addr 172.21.15.114:0/3293443162 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5636800640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381141d0 0x7f56381a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5635400640 1 -- 172.21.15.114:0/3293443162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381141d0 msgr2=0x7f56381a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5635400640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381141d0 0x7f56381a60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.540+0000 7f5635400640 1 -- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 msgr2=0x7f56381a08a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.542+0000 7f5635e00640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 0x7f56381a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.542+0000 7f5635400640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 0x7f56381a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.542+0000 7f5635400640 1 -- 172.21.15.114:0/3293443162 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56381a67d0 con 0x7f5638077390 2024-09-21T06:59:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.542+0000 7f5635e00640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 0x7f56381a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.542+0000 7f5635400640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56381a0e40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f562400e9f0 tx=0x7f562400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.543+0000 7f562ea00640 1 -- 172.21.15.114:0/3293443162 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562400cd80 con 0x7f5638077390 2024-09-21T06:59:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.543+0000 7f562ea00640 1 -- 172.21.15.114:0/3293443162 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56240108e0 con 0x7f5638077390 2024-09-21T06:59:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.543+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56381a6ac0 con 0x7f5638077390 2024-09-21T06:59:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.543+0000 7f562ea00640 1 -- 172.21.15.114:0/3293443162 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624018980 con 0x7f5638077390 2024-09-21T06:59:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.543+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56381ad250 con 0x7f5638077390 2024-09-21T06:59:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.548+0000 7f562ea00640 1 -- 172.21.15.114:0/3293443162 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5624010430 con 0x7f5638077390 2024-09-21T06:59:12.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.550+0000 7f5643c77740 1 Processor -- start 2024-09-21T06:59:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.550+0000 7f5643c77740 1 -- start start 2024-09-21T06:59:12.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f562ea00640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f561c0b6850 0x7f561c0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f5643c77740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 0x55dd37afbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f5643c77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37afc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f5643c77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377364d0 0x55dd37afc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f5643c77740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dd37a716b0 con 0x55dd377571b0 2024-09-21T06:59:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f5643c77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dd37a60ae0 con 0x55dd377364d0 2024-09-21T06:59:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.551+0000 7f5643c77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dd37a648f0 con 0x55dd3772e2f0 2024-09-21T06:59:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.553+0000 7f562ea00640 1 -- 172.21.15.114:0/3293443162 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f56240e7dc0 con 0x7f5638077390 2024-09-21T06:59:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.554+0000 7f5635e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 0x55dd37afbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.554+0000 7f5635e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 0x55dd37afbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45498/0 (socket says 172.21.15.114:45498) 2024-09-21T06:59:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.554+0000 7f5635e00640 1 -- 172.21.15.114:0/691510408 learned_addr learned my addr 172.21.15.114:0/691510408 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.555+0000 7f5635400640 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37afc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.557+0000 7f5636800640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f561c0b6850 0x7f561c0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.557+0000 7f5635e00640 1 -- 172.21.15.114:0/691510408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 msgr2=0x55dd37afc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.557+0000 7f5635e00640 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37afc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.557+0000 7f5635e00640 1 -- 172.21.15.114:0/691510408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377364d0 msgr2=0x55dd37afc890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.557+0000 7f5635e00640 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377364d0 0x55dd37afc890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.557+0000 7f5635e00640 1 -- 172.21.15.114:0/691510408 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd37a2d780 con 0x55dd377571b0 2024-09-21T06:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.560+0000 7f5635e00640 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 0x55dd37afbe10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5628009a20 tx=0x7f56280042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adec8fc353f5589b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.566+0000 7f5623000640 1 -- 172.21.15.114:0/691510408 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562803f070 con 0x55dd377571b0 2024-09-21T06:59:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.566+0000 7f5623000640 1 -- 172.21.15.114:0/691510408 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56280044d0 con 0x55dd377571b0 2024-09-21T06:59:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.566+0000 7f5623000640 1 -- 172.21.15.114:0/691510408 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628035030 con 0x55dd377571b0 2024-09-21T06:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.567+0000 7f5643c77740 1 -- 172.21.15.114:0/691510408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 msgr2=0x55dd37afbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.567+0000 7f5643c77740 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 0x55dd37afbe10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5628009a20 tx=0x7f56280042e0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.567+0000 7f5636800640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f561c0b6850 0x7f561c0b8d60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5630005fd0 tx=0x7f5630009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.568+0000 7f5643c77740 1 -- 172.21.15.114:0/691510408 shutdown_connections 2024-09-21T06:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.568+0000 7f5643c77740 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377364d0 0x55dd37afc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.568+0000 7f5643c77740 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37afc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.568+0000 7f5643c77740 1 --2- 172.21.15.114:0/691510408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377571b0 0x55dd37afbe10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.569+0000 7f5643c77740 1 -- 172.21.15.114:0/691510408 >> 172.21.15.114:0/691510408 conn(0x55dd3774b300 msgr2=0x55dd37af9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.569+0000 7f5643c77740 1 -- 172.21.15.114:0/691510408 shutdown_connections 2024-09-21T06:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.569+0000 7f5643c77740 1 -- 172.21.15.114:0/691510408 wait complete. 2024-09-21T06:59:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.570+0000 7f5643c77740 1 -- start start 2024-09-21T06:59:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.571+0000 7f5643c77740 1 Processor -- start 2024-09-21T06:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.571+0000 7f5643c77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b07490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.571+0000 7f5643c77740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.571+0000 7f5643c77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b13820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5643c77740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55dd37998ab0 con 0x55dd377364d0 2024-09-21T06:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5643c77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dd37a07570 con 0x55dd3772e2f0 2024-09-21T06:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5643c77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55dd37b0a530 con 0x55dd377571b0 2024-09-21T06:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5643c77740 10 client.0 _ll_register_callbacks cb 0x55dd37a16100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5635e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b07490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5635e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b07490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33714/0 (socket says 172.21.15.114:33714) 2024-09-21T06:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.572+0000 7f5635e00640 1 -- 172.21.15.114:0/4042538241 learned_addr learned my addr 172.21.15.114:0/4042538241 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.573+0000 7f5636800640 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b13820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.574+0000 7f5636800640 1 -- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 msgr2=0x55dd37b07490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.574+0000 7f5636800640 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b07490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.574+0000 7f5636800640 1 -- 172.21.15.114:0/4042538241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 msgr2=0x55dd37b079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.574+0000 7f5636800640 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.574+0000 7f5636800640 1 -- 172.21.15.114:0/4042538241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd377c2790 con 0x55dd377571b0 2024-09-21T06:59:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.576+0000 7f5636800640 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b13820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f56300042d0 tx=0x7f5630004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf65948fce657324 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.576+0000 7f5617400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.579+0000 7f5617400640 1 -- 172.21.15.114:0/4042538241 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630029060 con 0x55dd377571b0 2024-09-21T06:59:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.580+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56280044d0 con 0x55dd377571b0 2024-09-21T06:59:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.580+0000 7f5617400640 1 -- 172.21.15.114:0/4042538241 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5630021070 con 0x55dd377571b0 2024-09-21T06:59:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.581+0000 7f5617400640 1 -- 172.21.15.114:0/4042538241 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5630030050 con 0x55dd377571b0 2024-09-21T06:59:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.581+0000 7f5643c77740 10 client.26881 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.581+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd3783b620 con 0x55dd377571b0 2024-09-21T06:59:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.583+0000 7f5617400640 1 -- 172.21.15.114:0/4042538241 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56280044d0 con 0x55dd377571b0 2024-09-21T06:59:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.583+0000 7f5617400640 1 -- 172.21.15.114:0/4042538241 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f5630018040 con 0x55dd377571b0 2024-09-21T06:59:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5643c77740 10 client.26881 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5643c77740 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5643c77740 20 client.26881 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5643c77740 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd37715d80 0x55dd37b19690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5643c77740 4 client.26881 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd37a73600 con 0x55dd37715d80 2024-09-21T06:59:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.586+0000 7f5635e00640 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd37715d80 0x55dd37b19690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.587+0000 7f5635e00640 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd37715d80 0x55dd37b19690 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79276900b021a1cf server_cookie=436cfa8341b2bea0 in_seq=0 out_seq=0 2024-09-21T06:59:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.588+0000 7f5617400640 10 client.26881 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.589+0000 7f5617400640 1 -- 172.21.15.114:0/4042538241 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dd37a73600 con 0x55dd37715d80 2024-09-21T06:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.589+0000 7f5617400640 10 client.26881 handle_command_reply: tid=0 2024-09-21T06:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.590+0000 7f5643c77740 1 client.26881 shutdown 2024-09-21T06:59:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 msgr2=0x55dd37b13820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b13820 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f56300042d0 tx=0x7f5630004ab0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 shutdown_connections 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd37715d80 0x55dd37b19690 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5617400640 0 client.26881 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b13820 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 --2- 172.21.15.114:0/4042538241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b07490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.591+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 >> 172.21.15.114:0/4042538241 conn(0x55dd3774b300 msgr2=0x55dd37afce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.593+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 shutdown_connections 2024-09-21T06:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.593+0000 7f5643c77740 1 -- 172.21.15.114:0/4042538241 wait complete. 2024-09-21T06:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.594+0000 7f5643c77740 20 client.26881 trim_cache size 0 max 16384 2024-09-21T06:59:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.624+0000 7f5643c77740 1 Processor -- start 2024-09-21T06:59:12.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 -- start start 2024-09-21T06:59:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37af8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37afd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 0x55dd37b1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56381b72a0 con 0x55dd377364d0 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56381b8f00 con 0x55dd377571b0 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.625+0000 7f5643c77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56381b7970 con 0x55dd3772e2f0 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.626+0000 7f5636800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 0x55dd37b1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.626+0000 7f5636800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 0x55dd37b1ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33720/0 (socket says 172.21.15.114:33720) 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.626+0000 7f5636800640 1 -- 172.21.15.114:0/2263719626 learned_addr learned my addr 172.21.15.114:0/2263719626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.626+0000 7f5635e00640 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37af8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5635400640 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37afd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5636800640 1 -- 172.21.15.114:0/2263719626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 msgr2=0x55dd37af8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5636800640 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37af8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5636800640 1 -- 172.21.15.114:0/2263719626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 msgr2=0x55dd37afd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5636800640 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37afd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5636800640 1 -- 172.21.15.114:0/2263719626 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd37b05970 con 0x55dd377571b0 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5635e00640 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37af8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5636800640 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 0x55dd37b1ee90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5630004020 tx=0x7f5630011d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1595b13ad6f6c3d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.627+0000 7f5622600640 1 -- 172.21.15.114:0/2263719626 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630039070 con 0x55dd377571b0 2024-09-21T06:59:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.628+0000 7f5622600640 1 -- 172.21.15.114:0/2263719626 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563001cd90 con 0x55dd377571b0 2024-09-21T06:59:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.628+0000 7f5622600640 1 -- 172.21.15.114:0/2263719626 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630029060 con 0x55dd377571b0 2024-09-21T06:59:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 -- 172.21.15.114:0/2263719626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 msgr2=0x55dd37b1ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 0x55dd37b1ee90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5630004020 tx=0x7f5630011d20 comp rx=0 tx=0).stop 2024-09-21T06:59:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 -- 172.21.15.114:0/2263719626 shutdown_connections 2024-09-21T06:59:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd377571b0 0x55dd37b1ee90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37afd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 --2- 172.21.15.114:0/2263719626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd3772e2f0 0x55dd37af8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.630+0000 7f5643c77740 1 -- 172.21.15.114:0/2263719626 >> 172.21.15.114:0/2263719626 conn(0x55dd3774b300 msgr2=0x55dd37afce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:12.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.631+0000 7f5643c77740 1 -- 172.21.15.114:0/2263719626 shutdown_connections 2024-09-21T06:59:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.631+0000 7f5643c77740 1 -- 172.21.15.114:0/2263719626 wait complete. 2024-09-21T06:59:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.631+0000 7f5643c77740 1 -- start start 2024-09-21T06:59:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.632+0000 7f5643c77740 1 Processor -- start 2024-09-21T06:59:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.633+0000 7f5643c77740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b0d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.633+0000 7f5643c77740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b0dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.633+0000 7f5643c77740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.634+0000 7f5643c77740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56381b8770 con 0x55dd377364d0 2024-09-21T06:59:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.634+0000 7f5643c77740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56381b8040 con 0x55dd3772e2f0 2024-09-21T06:59:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.634+0000 7f5643c77740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56381b5280 con 0x55dd377571b0 2024-09-21T06:59:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.634+0000 7f5643c77740 10 client.0 _ll_register_callbacks cb 0x7f5630003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.634+0000 7f5636800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.635+0000 7f5635e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b0d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.635+0000 7f5635400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b0dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.635+0000 7f5635e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b0d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:33736/0 (socket says 172.21.15.114:33736) 2024-09-21T06:59:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.635+0000 7f5635e00640 1 -- 172.21.15.114:0/2305780821 learned_addr learned my addr 172.21.15.114:0/2305780821 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5635e00640 1 -- 172.21.15.114:0/2305780821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 msgr2=0x55dd37b0e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5635e00640 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b0e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5635e00640 1 -- 172.21.15.114:0/2305780821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 msgr2=0x55dd37b0dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5635e00640 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b0dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5635e00640 1 -- 172.21.15.114:0/2305780821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd37b830a0 con 0x55dd3772e2f0 2024-09-21T06:59:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5635e00640 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b0d6a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f56280058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67a863b521db602 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:12.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.636+0000 7f5617400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.637+0000 7f5617400640 1 -- 172.21.15.114:0/2305780821 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562803f040 con 0x55dd3772e2f0 2024-09-21T06:59:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.637+0000 7f5617400640 1 -- 172.21.15.114:0/2305780821 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f56280048d0 con 0x55dd3772e2f0 2024-09-21T06:59:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.637+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563001cd90 con 0x55dd3772e2f0 2024-09-21T06:59:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.638+0000 7f5617400640 1 -- 172.21.15.114:0/2305780821 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5628035040 con 0x55dd3772e2f0 2024-09-21T06:59:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.638+0000 7f5643c77740 10 client.26921 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.638+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd37b84200 con 0x55dd3772e2f0 2024-09-21T06:59:12.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.639+0000 7f5617400640 1 -- 172.21.15.114:0/2305780821 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563001cd90 con 0x55dd3772e2f0 2024-09-21T06:59:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.639+0000 7f5617400640 1 -- 172.21.15.114:0/2305780821 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f562803b460 con 0x55dd3772e2f0 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.639+0000 7f5643c77740 10 client.26921 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.639+0000 7f5643c77740 10 client.26921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.639+0000 7f5643c77740 20 client.26921 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.640+0000 7f5643c77740 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd376eb1c0 0x55dd37b86610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.640+0000 7f5643c77740 4 client.26921 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.640+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd37a6f0b0 con 0x55dd376eb1c0 2024-09-21T06:59:12.642 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.641+0000 7f5635400640 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd376eb1c0 0x55dd37b86610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:12.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.642+0000 7f5617400640 10 client.26921 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:12.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.643+0000 7f5635400640 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd376eb1c0 0x55dd37b86610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a3dfd8bcec2e95 server_cookie=c7b733757a4d5dd0 in_seq=0 out_seq=0 2024-09-21T06:59:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.700+0000 7f5617400640 1 -- 172.21.15.114:0/2305780821 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd37a6f0b0 con 0x55dd376eb1c0 2024-09-21T06:59:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.700+0000 7f5617400640 10 client.26921 handle_command_reply: tid=0 2024-09-21T06:59:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.700+0000 7f5643c77740 1 client.26921 shutdown 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 msgr2=0x55dd37b0d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5643c77740 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b0d6a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5628009900 tx=0x7f56280058b0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 shutdown_connections 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5643c77740 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55dd376eb1c0 0x55dd37b86610 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5643c77740 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd377571b0 0x55dd37b0e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5617400640 0 client.26921 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.701+0000 7f5643c77740 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55dd377364d0 0x55dd37b0dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.702+0000 7f5643c77740 1 --2- 172.21.15.114:0/2305780821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dd3772e2f0 0x55dd37b0d6a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.702+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 >> 172.21.15.114:0/2305780821 conn(0x55dd3774b300 msgr2=0x55dd37af8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.702+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 shutdown_connections 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.702+0000 7f5643c77740 1 -- 172.21.15.114:0/2305780821 wait complete. 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.702+0000 7f5643c77740 20 client.26921 trim_cache size 0 max 16384 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:59:12.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f561c0b6850 msgr2=0x7f561c0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f561c0b6850 0x7f561c0b8d60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5630005fd0 tx=0x7f5630009500 comp rx=0 tx=0).stop 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 msgr2=0x7f56381a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56381a0e40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f562400e9f0 tx=0x7f562400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 shutdown_connections 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f561c0b6850 0x7f561c0b8d60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56381141d0 0x7f56381a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5638077390 0x7f56381a0e40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 --2- 172.21.15.114:0/3293443162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5638075dd0 0x7f56381a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 >> 172.21.15.114:0/3293443162 conn(0x7f56380ff330 msgr2=0x7f56381100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.703+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 shutdown_connections 2024-09-21T06:59:12.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:12.704+0000 7f5637200640 1 -- 172.21.15.114:0/3293443162 wait complete. 2024-09-21T06:59:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 275.0 s, 727.8 tps, lat 136.722 ms stddev 160.241 2024-09-21T06:59:13.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:13 smithi193 ceph-mon[34601]: pgmap v548: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-09-21T06:59:13.416 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:13 smithi183 ceph-mon[34219]: pgmap v548: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-09-21T06:59:13.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:13 smithi114 ceph-mon[30654]: pgmap v548: 129 pgs: 129 active+clean; 2.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 14 MiB/s wr, 1.31k op/s 2024-09-21T06:59:15.495 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:15 smithi114 ceph-mon[30654]: pgmap v549: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-21T06:59:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:15 smithi183 ceph-mon[34219]: pgmap v549: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-21T06:59:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:15 smithi193 ceph-mon[34601]: pgmap v549: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 15 MiB/s wr, 1.45k op/s 2024-09-21T06:59:16.497 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:59:16.497 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:59:16.497 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:59:16.498 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:59:16.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:59:16.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:59:16.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:59:16.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:59:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:59:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:59:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:59:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:59:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:17 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T06:59:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:17 smithi114 ceph-mon[30654]: pgmap v550: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.37k op/s 2024-09-21T06:59:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:17 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T06:59:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:17 smithi183 ceph-mon[34219]: pgmap v550: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.37k op/s 2024-09-21T06:59:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:17 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T06:59:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:17 smithi193 ceph-mon[34601]: pgmap v550: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 14 MiB/s wr, 1.37k op/s 2024-09-21T06:59:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 280.0 s, 1037.2 tps, lat 99.203 ms stddev 155.692 2024-09-21T06:59:19.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:19 smithi114 ceph-mon[30654]: pgmap v551: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-09-21T06:59:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:19 smithi183 ceph-mon[34219]: pgmap v551: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-09-21T06:59:19.636 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-21T06:59:19.637 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:59:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:19 smithi193 ceph-mon[34601]: pgmap v551: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 15 MiB/s wr, 1.38k op/s 2024-09-21T06:59:20.274 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:21.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.160+0000 7fb98e000640 1 -- 172.21.15.114:0/1560664214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9881094b0 msgr2=0x7fb9881098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:21.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.161+0000 7fb98e000640 1 --2- 172.21.15.114:0/1560664214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9881094b0 0x7fb9881098c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb980009ca0 tx=0x7fb98002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.161+0000 7fb98e000640 1 -- 172.21.15.114:0/1560664214 shutdown_connections 2024-09-21T06:59:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.161+0000 7fb98e000640 1 --2- 172.21.15.114:0/1560664214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb988109e00 0x7fb988110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.162+0000 7fb98e000640 1 --2- 172.21.15.114:0/1560664214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9881094b0 0x7fb9881098c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.162+0000 7fb98e000640 1 --2- 172.21.15.114:0/1560664214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb988103b70 0x7fb988103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.162+0000 7fb98e000640 1 -- 172.21.15.114:0/1560664214 >> 172.21.15.114:0/1560664214 conn(0x7fb9880ff5e0 msgr2=0x7fb988101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:21.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.163+0000 7fb98e000640 1 -- 172.21.15.114:0/1560664214 shutdown_connections 2024-09-21T06:59:21.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.163+0000 7fb98e000640 1 -- 172.21.15.114:0/1560664214 wait complete. 2024-09-21T06:59:21.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.165+0000 7fb98e000640 1 Processor -- start 2024-09-21T06:59:21.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 -- start start 2024-09-21T06:59:21.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 0x7fb988074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:21.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9881094b0 0x7fb98806e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:21.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb988109e00 0x7fb98806e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:21.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb988113b10 con 0x7fb988109e00 2024-09-21T06:59:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb988113990 con 0x7fb9881094b0 2024-09-21T06:59:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.166+0000 7fb98e000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb988113c90 con 0x7fb988103b70 2024-09-21T06:59:21.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.167+0000 7fb987400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 0x7fb988074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:21.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.167+0000 7fb987400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 0x7fb988074ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46632/0 (socket says 172.21.15.114:46632) 2024-09-21T06:59:21.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.167+0000 7fb987400640 1 -- 172.21.15.114:0/1458686386 learned_addr learned my addr 172.21.15.114:0/1458686386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:21.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.167+0000 7fb986a00640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9881094b0 0x7fb98806e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:21.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.167+0000 7fb986a00640 1 -- 172.21.15.114:0/1458686386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 msgr2=0x7fb988074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:21.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.167+0000 7fb986a00640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 0x7fb988074ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb986a00640 1 -- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb988109e00 msgr2=0x7fb98806e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:21.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb986a00640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb988109e00 0x7fb98806e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.172 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb986a00640 1 -- 172.21.15.114:0/1458686386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98806ef50 con 0x7fb9881094b0 2024-09-21T06:59:21.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb987400640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 0x7fb988074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:21.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb986a00640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9881094b0 0x7fb98806e170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb98002ade0 tx=0x7fb9800319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:21.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98004a030 con 0x7fb9881094b0 2024-09-21T06:59:21.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb980046d00 con 0x7fb9881094b0 2024-09-21T06:59:21.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980004640 con 0x7fb9881094b0 2024-09-21T06:59:21.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb988073b70 con 0x7fb9881094b0 2024-09-21T06:59:21.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.168+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb988073fd0 con 0x7fb9881094b0 2024-09-21T06:59:21.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.175+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb98004d030 con 0x7fb9881094b0 2024-09-21T06:59:21.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.176+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb988104df0 con 0x7fb9881094b0 2024-09-21T06:59:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.178+0000 7fb97fe00640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb96c0c7790 0x7fb96c0c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:21.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.179+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb980107cd0 con 0x7fb9881094b0 2024-09-21T06:59:21.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.179+0000 7fb987400640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb96c0c7790 0x7fb96c0c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:21.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.183+0000 7fb987400640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb96c0c7790 0x7fb96c0c9ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb970005fd0 tx=0x7fb97000b480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:21.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.185+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb98010e080 con 0x7fb9881094b0 2024-09-21T06:59:21.569 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:21 smithi114 ceph-mon[30654]: pgmap v552: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 828 op/s 2024-09-21T06:59:21.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:21 smithi183 ceph-mon[34219]: pgmap v552: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 828 op/s 2024-09-21T06:59:21.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.624+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb98810b640 con 0x7fb9881094b0 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:59:21.627 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"btime":"2024-09-21T06:58:51:043259+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:58:50.613377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:59:21.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.625+0000 7fb97fe00640 1 -- 172.21.15.114:0/1458686386 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11999 (secure 0 0 0) 0x7fb9800cdee0 con 0x7fb9881094b0 2024-09-21T06:59:21.628 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-21T06:59:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb96c0c7790 msgr2=0x7fb96c0c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb96c0c7790 0x7fb96c0c9ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb970005fd0 tx=0x7fb97000b480 comp rx=0 tx=0).stop 2024-09-21T06:59:21.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9881094b0 msgr2=0x7fb98806e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9881094b0 0x7fb98806e170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb98002ade0 tx=0x7fb9800319f0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 shutdown_connections 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb988109e00 0x7fb98806e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb96c0c7790 0x7fb96c0c9ca0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9881094b0 0x7fb98806e170 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 --2- 172.21.15.114:0/1458686386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb988103b70 0x7fb988074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:21.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 >> 172.21.15.114:0/1458686386 conn(0x7fb9880ff5e0 msgr2=0x7fb988101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:21.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.630+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 shutdown_connections 2024-09-21T06:59:21.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:21.631+0000 7fb98e000640 1 -- 172.21.15.114:0/1458686386 wait complete. 2024-09-21T06:59:21.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:21 smithi193 ceph-mon[34601]: pgmap v552: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.1 MiB/s wr, 828 op/s 2024-09-21T06:59:22.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:22 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1458686386' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:22.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:22 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1458686386' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:22.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:22 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1458686386' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 285.0 s, 1541.8 tps, lat 65.015 ms stddev 74.018 2024-09-21T06:59:23.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:23 smithi114 ceph-mon[30654]: pgmap v553: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-09-21T06:59:23.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:23 smithi183 ceph-mon[34219]: pgmap v553: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-09-21T06:59:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:23 smithi193 ceph-mon[34601]: pgmap v553: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.9 MiB/s wr, 644 op/s 2024-09-21T06:59:24.936 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-21T06:59:24.936 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-21T06:59:25.569 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:25 smithi114 ceph-mon[30654]: pgmap v554: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 660 op/s 2024-09-21T06:59:25.592 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:25 smithi183 ceph-mon[34219]: pgmap v554: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 660 op/s 2024-09-21T06:59:25.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:25 smithi193 ceph-mon[34601]: pgmap v554: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.7 MiB/s wr, 660 op/s 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 -- 172.21.15.114:0/2597055852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40810dbc0 msgr2=0x7fb4081100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 --2- 172.21.15.114:0/2597055852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40810dbc0 0x7fb4081100a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009900 tx=0x7fb3f402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 -- 172.21.15.114:0/2597055852 shutdown_connections 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 --2- 172.21.15.114:0/2597055852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40810dbc0 0x7fb4081100a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 --2- 172.21.15.114:0/2597055852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081050f0 0x7fb40810d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 --2- 172.21.15.114:0/2597055852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4081047a0 0x7fb408104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 -- 172.21.15.114:0/2597055852 >> 172.21.15.114:0/2597055852 conn(0x7fb4080fe230 msgr2=0x7fb408100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 -- 172.21.15.114:0/2597055852 shutdown_connections 2024-09-21T06:59:26.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.027+0000 7fb407e00640 1 -- 172.21.15.114:0/2597055852 wait complete. 2024-09-21T06:59:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.029+0000 7fb407e00640 1 Processor -- start 2024-09-21T06:59:26.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.029+0000 7fb407e00640 1 -- start start 2024-09-21T06:59:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 0x7fb40819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081050f0 0x7fb40819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40810dbc0 0x7fb4081a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb408113820 con 0x7fb40810dbc0 2024-09-21T06:59:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081136a0 con 0x7fb4081050f0 2024-09-21T06:59:26.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081139a0 con 0x7fb4081047a0 2024-09-21T06:59:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb406a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 0x7fb40819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb406000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081050f0 0x7fb40819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb406a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 0x7fb40819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35450/0 (socket says 172.21.15.114:35450) 2024-09-21T06:59:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb406a00640 1 -- 172.21.15.114:0/3215422950 learned_addr learned my addr 172.21.15.114:0/3215422950 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.030+0000 7fb407400640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40810dbc0 0x7fb4081a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.031+0000 7fb406a00640 1 -- 172.21.15.114:0/3215422950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081050f0 msgr2=0x7fb40819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.031+0000 7fb406a00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081050f0 0x7fb40819c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.031+0000 7fb406a00640 1 -- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40810dbc0 msgr2=0x7fb4081a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.031+0000 7fb406a00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40810dbc0 0x7fb4081a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.031+0000 7fb406a00640 1 -- 172.21.15.114:0/3215422950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4081a2350 con 0x7fb4081047a0 2024-09-21T06:59:26.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.036+0000 7fb406a00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 0x7fb40819c420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb400009d50 tx=0x7fb40000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.039+0000 7fb40ca00640 1 -- 172.21.15.114:0/3215422950 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400003b70 con 0x7fb4081047a0 2024-09-21T06:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.039+0000 7fb40ca00640 1 -- 172.21.15.114:0/3215422950 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb400003d10 con 0x7fb4081047a0 2024-09-21T06:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.039+0000 7fb40ca00640 1 -- 172.21.15.114:0/3215422950 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000108a0 con 0x7fb4081047a0 2024-09-21T06:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.039+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4081a2640 con 0x7fb4081047a0 2024-09-21T06:59:26.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.039+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4081a8dd0 con 0x7fb4081047a0 2024-09-21T06:59:26.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.044+0000 7fb414ff5740 1 Processor -- start 2024-09-21T06:59:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.045+0000 7fb414ff5740 1 -- start start 2024-09-21T06:59:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.045+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.045+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dd62f0 0x5617c71a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dde4d0 0x5617c71a4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb414ff5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5617c71196b0 con 0x5617c6dde4d0 2024-09-21T06:59:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb414ff5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5617c7108ae0 con 0x5617c6dff1b0 2024-09-21T06:59:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb414ff5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617c710c8f0 con 0x5617c6dd62f0 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46042/0 (socket says 172.21.15.114:46042) 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 -- 172.21.15.114:0/3003899162 learned_addr learned my addr 172.21.15.114:0/3003899162 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 -- 172.21.15.114:0/3003899162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dd62f0 msgr2=0x5617c71a4350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dd62f0 0x5617c71a4350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 -- 172.21.15.114:0/3003899162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dde4d0 msgr2=0x5617c71a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dde4d0 0x5617c71a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.046+0000 7fb407400640 1 -- 172.21.15.114:0/3003899162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c70d5780 con 0x5617c6dff1b0 2024-09-21T06:59:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.047+0000 7fb407400640 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a3e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb3f402a9e0 tx=0x7fb3f40040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d3996b1eac30515 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.047+0000 7fb3f3e00640 1 -- 172.21.15.114:0/3003899162 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f403f070 con 0x5617c6dff1b0 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.048+0000 7fb3f3e00640 1 -- 172.21.15.114:0/3003899162 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f4004440 con 0x5617c6dff1b0 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.048+0000 7fb3f3e00640 1 -- 172.21.15.114:0/3003899162 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4035070 con 0x5617c6dff1b0 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 -- 172.21.15.114:0/3003899162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 msgr2=0x5617c71a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a3e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb3f402a9e0 tx=0x7fb3f40040d0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 -- 172.21.15.114:0/3003899162 shutdown_connections 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dde4d0 0x5617c71a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dd62f0 0x5617c71a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 --2- 172.21.15.114:0/3003899162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a3e10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.049+0000 7fb414ff5740 1 -- 172.21.15.114:0/3003899162 >> 172.21.15.114:0/3003899162 conn(0x5617c6df3300 msgr2=0x5617c71a1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.052+0000 7fb40ca00640 1 -- 172.21.15.114:0/3215422950 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb400004000 con 0x7fb4081047a0 2024-09-21T06:59:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.055+0000 7fb414ff5740 1 -- 172.21.15.114:0/3003899162 shutdown_connections 2024-09-21T06:59:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.055+0000 7fb414ff5740 1 -- 172.21.15.114:0/3003899162 wait complete. 2024-09-21T06:59:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.056+0000 7fb414ff5740 1 -- start start 2024-09-21T06:59:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.057+0000 7fb414ff5740 1 Processor -- start 2024-09-21T06:59:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.058+0000 7fb40ca00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6640 0x7fb3ec0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.058+0000 7fb40ca00640 1 -- 172.21.15.114:0/3215422950 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb4000e8250 con 0x7fb4081047a0 2024-09-21T06:59:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.059+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.059+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dff1b0 0x5617c71b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb406a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 0x5617c71bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb414ff5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5617c7040ab0 con 0x5617c6dbdd80 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb414ff5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5617c70af570 con 0x5617c6dde4d0 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb414ff5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617c71b2530 con 0x5617c6dff1b0 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb406000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dff1b0 0x5617c71b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb406a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71b86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46048/0 (socket says 172.21.15.114:46048) 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb406a00640 1 -- 172.21.15.114:0/1740149988 learned_addr learned my addr 172.21.15.114:0/1740149988 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb407400640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6640 0x7fb3ec0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.060+0000 7fb414ff5740 10 client.0 _ll_register_callbacks cb 0x5617c70be100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.061+0000 7fb406000640 1 -- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 msgr2=0x5617c71b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.061+0000 7fb406000640 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.061+0000 7fb406000640 1 -- 172.21.15.114:0/1740149988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 msgr2=0x5617c71bdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.061+0000 7fb406000640 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 0x5617c71bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.061+0000 7fb406000640 1 -- 172.21.15.114:0/1740149988 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c6e6a790 con 0x5617c6dff1b0 2024-09-21T06:59:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.061+0000 7fb406a00640 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.062+0000 7fb406000640 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dff1b0 0x5617c71b8ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb408101930 tx=0x7fb3f800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91039fc845d229c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.062+0000 7fb3f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.062+0000 7fb3f0c00640 1 -- 172.21.15.114:0/1740149988 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8016070 con 0x5617c6dff1b0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.062+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f4004440 con 0x5617c6dff1b0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.063+0000 7fb3f0c00640 1 -- 172.21.15.114:0/1740149988 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb3f8005220 con 0x5617c6dff1b0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.063+0000 7fb407400640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6640 0x7fb3ec0b8b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40096c0 tx=0x7fb3f4004a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.063+0000 7fb3f0c00640 1 -- 172.21.15.114:0/1740149988 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb3f8021050 con 0x5617c6dff1b0 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.063+0000 7fb414ff5740 10 client.26899 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:26.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.063+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5617c6ee3620 con 0x5617c6dff1b0 2024-09-21T06:59:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.063+0000 7fb3f0c00640 1 -- 172.21.15.114:0/1740149988 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f4004440 con 0x5617c6dff1b0 2024-09-21T06:59:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.064+0000 7fb3f0c00640 1 -- 172.21.15.114:0/1740149988 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fb3f800f040 con 0x5617c6dff1b0 2024-09-21T06:59:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.067+0000 7fb414ff5740 10 client.26899 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.067+0000 7fb414ff5740 10 client.26899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.067+0000 7fb414ff5740 20 client.26899 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.067+0000 7fb414ff5740 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c6d931c0 0x5617c71ca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.067+0000 7fb414ff5740 4 client.26899 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.067+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617c711b600 con 0x5617c6d931c0 2024-09-21T06:59:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.068+0000 7fb406a00640 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c6d931c0 0x5617c71ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.068+0000 7fb406a00640 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c6d931c0 0x5617c71ca020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a728c6bb2f7ddbb5 server_cookie=745a6d0c67f2aab5 in_seq=0 out_seq=0 2024-09-21T06:59:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.068+0000 7fb3f0c00640 10 client.26899 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.069+0000 7fb3f0c00640 1 -- 172.21.15.114:0/1740149988 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617c711b600 con 0x5617c6d931c0 2024-09-21T06:59:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.069+0000 7fb3f0c00640 10 client.26899 handle_command_reply: tid=0 2024-09-21T06:59:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.070+0000 7fb414ff5740 1 client.26899 shutdown 2024-09-21T06:59:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.070+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dff1b0 msgr2=0x5617c71b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.070+0000 7fb414ff5740 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dff1b0 0x5617c71b8ca0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb408101930 tx=0x7fb3f800e6c0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.070+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 shutdown_connections 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.070+0000 7fb414ff5740 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 0x5617c71bdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.070+0000 7fb414ff5740 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c6d931c0 0x5617c71ca020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.071+0000 7fb414ff5740 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dff1b0 0x5617c71b8ca0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.071+0000 7fb3f0c00640 0 client.26899 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.071+0000 7fb414ff5740 1 --2- 172.21.15.114:0/1740149988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.071+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 >> 172.21.15.114:0/1740149988 conn(0x5617c6df3300 msgr2=0x5617c71a4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.073+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 shutdown_connections 2024-09-21T06:59:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.074+0000 7fb414ff5740 1 -- 172.21.15.114:0/1740149988 wait complete. 2024-09-21T06:59:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.075+0000 7fb414ff5740 20 client.26899 trim_cache size 0 max 16384 2024-09-21T06:59:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.105+0000 7fb414ff5740 1 Processor -- start 2024-09-21T06:59:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.106+0000 7fb414ff5740 1 -- start start 2024-09-21T06:59:26.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.108+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 0x5617c71b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.108+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 0x5617c71bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb406000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 0x5617c71bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb406000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 0x5617c71bdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35470/0 (socket says 172.21.15.114:35470) 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb406000640 1 -- 172.21.15.114:0/2230730122 learned_addr learned my addr 172.21.15.114:0/2230730122 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081aea60 con 0x5617c6dbdd80 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081b06c0 con 0x5617c6dff1b0 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081af130 con 0x5617c6dde4d0 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.109+0000 7fb407400640 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.110+0000 7fb406000640 1 -- 172.21.15.114:0/2230730122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 msgr2=0x5617c71a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.110+0000 7fb406000640 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.110+0000 7fb406000640 1 -- 172.21.15.114:0/2230730122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 msgr2=0x5617c71b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.110+0000 7fb406000640 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 0x5617c71b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.110+0000 7fb406000640 1 -- 172.21.15.114:0/2230730122 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c71be780 con 0x5617c6dde4d0 2024-09-21T06:59:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.110+0000 7fb406000640 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 0x5617c71bdb20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8005940 tx=0x7fb3f800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e24a392f64be210 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.111+0000 7fb3f3400640 1 -- 172.21.15.114:0/2230730122 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8016070 con 0x5617c6dde4d0 2024-09-21T06:59:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.111+0000 7fb3f3400640 1 -- 172.21.15.114:0/2230730122 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f8011b70 con 0x5617c6dde4d0 2024-09-21T06:59:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.111+0000 7fb3f3400640 1 -- 172.21.15.114:0/2230730122 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8004df0 con 0x5617c6dde4d0 2024-09-21T06:59:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.112+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 msgr2=0x5617c71bdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.112+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 0x5617c71bdb20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8005940 tx=0x7fb3f800eca0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.114+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 shutdown_connections 2024-09-21T06:59:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.114+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dff1b0 0x5617c71a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.115+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dde4d0 0x5617c71bdb20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.115+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2230730122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dbdd80 0x5617c71b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.115+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 >> 172.21.15.114:0/2230730122 conn(0x5617c6df3300 msgr2=0x5617c71a1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.116+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 shutdown_connections 2024-09-21T06:59:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.116+0000 7fb414ff5740 1 -- 172.21.15.114:0/2230730122 wait complete. 2024-09-21T06:59:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.118+0000 7fb414ff5740 1 -- start start 2024-09-21T06:59:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.119+0000 7fb414ff5740 1 Processor -- start 2024-09-21T06:59:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.121+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 0x5617c71c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.121+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.121+0000 7fb414ff5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dff1b0 0x5617c71cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.122+0000 7fb406a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 0x5617c71c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.122+0000 7fb406a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 0x5617c71c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35474/0 (socket says 172.21.15.114:35474) 2024-09-21T06:59:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081aff30 con 0x5617c6dff1b0 2024-09-21T06:59:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081af800 con 0x5617c6dde4d0 2024-09-21T06:59:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081aca80 con 0x5617c6dbdd80 2024-09-21T06:59:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406a00640 1 -- 172.21.15.114:0/2260795912 learned_addr learned my addr 172.21.15.114:0/2260795912 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406000640 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb414ff5740 10 client.0 _ll_register_callbacks cb 0x7fb3f8005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406a00640 1 -- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 msgr2=0x5617c71cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406a00640 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406a00640 1 -- 172.21.15.114:0/2260795912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dff1b0 msgr2=0x5617c71cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406a00640 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dff1b0 0x5617c71cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.123+0000 7fb406a00640 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c71d0020 con 0x5617c6dbdd80 2024-09-21T06:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.124+0000 7fb406a00640 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 0x5617c71c46a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb400009270 tx=0x7fb400004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22dca4b8fc473fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.124+0000 7fb3f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.125+0000 7fb3f0c00640 1 -- 172.21.15.114:0/2260795912 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000ad840 con 0x5617c6dbdd80 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.125+0000 7fb3f0c00640 1 -- 172.21.15.114:0/2260795912 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb4000e58b0 con 0x5617c6dbdd80 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.125+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f8011b70 con 0x5617c6dbdd80 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.125+0000 7fb3f0c00640 1 -- 172.21.15.114:0/2260795912 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb4000ae430 con 0x5617c6dbdd80 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.125+0000 7fb414ff5740 10 client.26905 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.125+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5617c71c3fd0 con 0x5617c6dbdd80 2024-09-21T06:59:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb3f0c00640 1 -- 172.21.15.114:0/2260795912 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f8011b70 con 0x5617c6dbdd80 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb3f0c00640 1 -- 172.21.15.114:0/2260795912 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fb4000a9fa0 con 0x5617c6dbdd80 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb414ff5740 10 client.26905 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb414ff5740 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb414ff5740 20 client.26905 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c7234e70 0x5617c7237290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb414ff5740 4 client.26905 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:59:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617c71170b0 con 0x5617c7234e70 2024-09-21T06:59:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.126+0000 7fb406000640 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c7234e70 0x5617c7237290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.127+0000 7fb406000640 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c7234e70 0x5617c7237290 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfa073ddbfc9b63 server_cookie=cf904de2d7951204 in_seq=0 out_seq=0 2024-09-21T06:59:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.127+0000 7fb3f0c00640 10 client.26905 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.153+0000 7fb3f0c00640 1 -- 172.21.15.114:0/2260795912 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617c71170b0 con 0x5617c7234e70 2024-09-21T06:59:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.153+0000 7fb3f0c00640 10 client.26905 handle_command_reply: tid=0 2024-09-21T06:59:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.153+0000 7fb414ff5740 1 client.26905 shutdown 2024-09-21T06:59:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 msgr2=0x5617c71c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 0x5617c71c46a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb400009270 tx=0x7fb400004790 comp rx=0 tx=0).stop 2024-09-21T06:59:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 shutdown_connections 2024-09-21T06:59:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5617c6dff1b0 0x5617c71cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5617c6dde4d0 0x5617c71cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5617c7234e70 0x5617c7237290 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 --2- 172.21.15.114:0/2260795912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617c6dbdd80 0x5617c71c46a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 >> 172.21.15.114:0/2260795912 conn(0x5617c6df3300 msgr2=0x5617c71bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.154+0000 7fb3f0c00640 0 client.26905 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.155+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 shutdown_connections 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.155+0000 7fb414ff5740 1 -- 172.21.15.114:0/2260795912 wait complete. 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.155+0000 7fb414ff5740 20 client.26905 trim_cache size 0 max 16384 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:59:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:59:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:59:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:59:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6640 msgr2=0x7fb3ec0b8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6640 0x7fb3ec0b8b50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40096c0 tx=0x7fb3f4004a20 comp rx=0 tx=0).stop 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 msgr2=0x7fb40819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 0x7fb40819c420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb400009d50 tx=0x7fb40000c6a0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 shutdown_connections 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb40810dbc0 0x7fb4081a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb3ec0b6640 0x7fb3ec0b8b50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4081050f0 0x7fb40819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 --2- 172.21.15.114:0/3215422950 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4081047a0 0x7fb40819c420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.157+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 >> 172.21.15.114:0/3215422950 conn(0x7fb4080fe230 msgr2=0x7fb4081023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.158+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 shutdown_connections 2024-09-21T06:59:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:26.158+0000 7fb407e00640 1 -- 172.21.15.114:0/3215422950 wait complete. 2024-09-21T06:59:26.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.511+0000 7fc167e00640 1 -- 172.21.15.114:0/935055018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 msgr2=0x7fc16810d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.511+0000 7fc167e00640 1 --2- 172.21.15.114:0/935055018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 0x7fc16810d6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc158009860 tx=0x7fc15802a340 comp rx=0 tx=0).stop 2024-09-21T06:59:26.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.512+0000 7fc167e00640 1 -- 172.21.15.114:0/935055018 shutdown_connections 2024-09-21T06:59:26.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.512+0000 7fc167e00640 1 --2- 172.21.15.114:0/935055018 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc16810dc30 0x7fc168110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.512+0000 7fc167e00640 1 --2- 172.21.15.114:0/935055018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 0x7fc16810d6f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.512+0000 7fc167e00640 1 --2- 172.21.15.114:0/935055018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc168104680 0x7fc168104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.512+0000 7fc167e00640 1 -- 172.21.15.114:0/935055018 >> 172.21.15.114:0/935055018 conn(0x7fc1680fe330 msgr2=0x7fc168100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.512+0000 7fc167e00640 1 -- 172.21.15.114:0/935055018 shutdown_connections 2024-09-21T06:59:26.518 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.514+0000 7fc167e00640 1 -- 172.21.15.114:0/935055018 wait complete. 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.516+0000 7fc167e00640 1 Processor -- start 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.516+0000 7fc167e00640 1 -- start start 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc168104680 0x7fc16810d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 0x7fc168107710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 0x7fc168107c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc16806d6e0 con 0x7fc168104680 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc16806d560 con 0x7fc16810dc30 2024-09-21T06:59:26.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc16806d860 con 0x7fc1681050b0 2024-09-21T06:59:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 0x7fc168107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 0x7fc168107c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46100/0 (socket says 172.21.15.114:46100) 2024-09-21T06:59:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.517+0000 7fc167400640 1 -- 172.21.15.114:0/1266842214 learned_addr learned my addr 172.21.15.114:0/1266842214 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.518+0000 7fc167400640 1 -- 172.21.15.114:0/1266842214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 msgr2=0x7fc168107710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:59:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.518+0000 7fc167400640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 0x7fc168107710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.518+0000 7fc167400640 1 -- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc168104680 msgr2=0x7fc16810d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.518+0000 7fc167400640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc168104680 0x7fc16810d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.518+0000 7fc167400640 1 -- 172.21.15.114:0/1266842214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc168108450 con 0x7fc16810dc30 2024-09-21T06:59:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.519+0000 7fc167400640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 0x7fc168107c50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc150009fd0 tx=0x7fc15000b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.520+0000 7fc15f400640 1 -- 172.21.15.114:0/1266842214 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150011830 con 0x7fc16810dc30 2024-09-21T06:59:26.521 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.520+0000 7fc15f400640 1 -- 172.21.15.114:0/1266842214 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc150016070 con 0x7fc16810dc30 2024-09-21T06:59:26.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.520+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc168119080 con 0x7fc16810dc30 2024-09-21T06:59:26.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.520+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1681195c0 con 0x7fc16810dc30 2024-09-21T06:59:26.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.521+0000 7fc15f400640 1 -- 172.21.15.114:0/1266842214 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc150010de0 con 0x7fc16810dc30 2024-09-21T06:59:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.526+0000 7fc174286740 1 Processor -- start 2024-09-21T06:59:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.526+0000 7fc174286740 1 -- start start 2024-09-21T06:59:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc174286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2ead840 0x5604a3255510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc174286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 0x5604a3255a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc15f400640 1 -- 172.21.15.114:0/1266842214 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc15001b4a0 con 0x7fc16810dc30 2024-09-21T06:59:26.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc174286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e8cb60 0x5604a3255f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc174286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5604a31caca0 con 0x5604a2ead840 2024-09-21T06:59:26.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc174286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604a31ba0d0 con 0x5604a2e8cb60 2024-09-21T06:59:26.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.527+0000 7fc174286740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5604a31bdee0 con 0x5604a2e84980 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 0x5604a3255a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 0x5604a3255a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35494/0 (socket says 172.21.15.114:35494) 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 -- 172.21.15.114:0/2429931377 learned_addr learned my addr 172.21.15.114:0/2429931377 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 -- 172.21.15.114:0/2429931377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e8cb60 msgr2=0x5604a3255f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e8cb60 0x5604a3255f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 -- 172.21.15.114:0/2429931377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2ead840 msgr2=0x5604a3255510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2ead840 0x5604a3255510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.528+0000 7fc166a00640 1 -- 172.21.15.114:0/2429931377 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a316c400 con 0x5604a2e84980 2024-09-21T06:59:26.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.529+0000 7fc166a00640 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 0x5604a3255a50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc160009df0 tx=0x7fc160012b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363ea92ebeb0af87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.529+0000 7fc157400640 1 -- 172.21.15.114:0/2429931377 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16001f020 con 0x5604a2e84980 2024-09-21T06:59:26.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.529+0000 7fc15f400640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc14c0b6770 0x7fc14c0b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.529+0000 7fc157400640 1 -- 172.21.15.114:0/2429931377 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16000cda0 con 0x5604a2e84980 2024-09-21T06:59:26.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.529+0000 7fc15f400640 1 -- 172.21.15.114:0/1266842214 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc1500e6f50 con 0x7fc16810dc30 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.530+0000 7fc157400640 1 -- 172.21.15.114:0/2429931377 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16001f020 con 0x5604a2e84980 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.530+0000 7fc174286740 1 -- 172.21.15.114:0/2429931377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 msgr2=0x5604a3255a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.530+0000 7fc174286740 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 0x5604a3255a50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc160009df0 tx=0x7fc160012b70 comp rx=0 tx=0).stop 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.531+0000 7fc174286740 1 -- 172.21.15.114:0/2429931377 shutdown_connections 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.531+0000 7fc174286740 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e8cb60 0x5604a3255f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.531+0000 7fc174286740 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e84980 0x5604a3255a50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.531+0000 7fc174286740 1 --2- 172.21.15.114:0/2429931377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2ead840 0x5604a3255510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.531+0000 7fc174286740 1 -- 172.21.15.114:0/2429931377 >> 172.21.15.114:0/2429931377 conn(0x5604a2ea1990 msgr2=0x5604a3253400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.539+0000 7fc166000640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc14c0b6770 0x7fc14c0b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.539+0000 7fc174286740 1 -- 172.21.15.114:0/2429931377 shutdown_connections 2024-09-21T06:59:26.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.540+0000 7fc174286740 1 -- 172.21.15.114:0/2429931377 wait complete. 2024-09-21T06:59:26.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.540+0000 7fc174286740 1 -- start start 2024-09-21T06:59:26.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.541+0000 7fc174286740 1 Processor -- start 2024-09-21T06:59:26.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.541+0000 7fc166000640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc14c0b6770 0x7fc14c0b8c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5604a32564f0 tx=0x7fc15802c040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.542+0000 7fc174286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3260b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.542+0000 7fc174286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e6c1c0 0x5604a3261040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.542+0000 7fc174286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 0x5604a3261580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.542+0000 7fc166a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3260b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.542+0000 7fc166a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3260b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46116/0 (socket says 172.21.15.114:46116) 2024-09-21T06:59:26.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.543+0000 7fc167400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 0x5604a3261580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.543+0000 7fc167400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 0x5604a3261580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35508/0 (socket says 172.21.15.114:35508) 2024-09-21T06:59:26.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.543+0000 7fc166a00640 1 -- 172.21.15.114:0/3689907791 learned_addr learned my addr 172.21.15.114:0/3689907791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.544+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5604a301b7a0 con 0x5604a2e6c1c0 2024-09-21T06:59:26.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.544+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604a315fa10 con 0x5604a2e84980 2024-09-21T06:59:26.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.544+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5604a3263d90 con 0x5604a2e41570 2024-09-21T06:59:26.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.544+0000 7fc166000640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e6c1c0 0x5604a3261040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc174286740 10 client.0 _ll_register_callbacks cb 0x5604a316fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:26.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc167400640 1 -- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 msgr2=0x5604a3260b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc167400640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3260b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc167400640 1 -- 172.21.15.114:0/3689907791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e6c1c0 msgr2=0x5604a3261040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc167400640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e6c1c0 0x5604a3261040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc167400640 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a2f27b90 con 0x5604a2e41570 2024-09-21T06:59:26.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.545+0000 7fc166000640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e6c1c0 0x5604a3261040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:26.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.546+0000 7fc167400640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 0x5604a3261580 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc1500e42d0 tx=0x7fc1500ac3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9141052c43ae8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.546+0000 7fc14be00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.547+0000 7fc14be00640 1 -- 172.21.15.114:0/3689907791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500d0070 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.547+0000 7fc14be00640 1 -- 172.21.15.114:0/3689907791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc1500e4550 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.547+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc16000cda0 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc14be00640 1 -- 172.21.15.114:0/3689907791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1500e4750 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc174286740 10 client.26914 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5604a326fa00 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc14be00640 1 -- 172.21.15.114:0/3689907791 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc16000cda0 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc14be00640 1 -- 172.21.15.114:0/3689907791 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fc1500ab6a0 con 0x5604a2e41570 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc174286740 10 client.26914 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc174286740 10 client.26914 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:59:26.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc174286740 20 client.26914 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:26.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.548+0000 7fc174286740 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a3270f90 0x5604a3273480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.550+0000 7fc174286740 4 client.26914 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:26.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.550+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604a31ccbf0 con 0x5604a3270f90 2024-09-21T06:59:26.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.551+0000 7fc166a00640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a3270f90 0x5604a3273480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.551+0000 7fc166a00640 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a3270f90 0x5604a3273480 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f166d32d676e16 server_cookie=7357fcacddbd4f91 in_seq=0 out_seq=0 2024-09-21T06:59:26.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.551+0000 7fc14be00640 10 client.26914 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.552+0000 7fc14be00640 1 -- 172.21.15.114:0/3689907791 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604a31ccbf0 con 0x5604a3270f90 2024-09-21T06:59:26.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.552+0000 7fc14be00640 10 client.26914 handle_command_reply: tid=0 2024-09-21T06:59:26.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.552+0000 7fc174286740 1 client.26914 shutdown 2024-09-21T06:59:26.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 msgr2=0x5604a3261580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 0x5604a3261580 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc1500e42d0 tx=0x7fc1500ac3e0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 shutdown_connections 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a3270f90 0x5604a3273480 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e41570 0x5604a3261580 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e6c1c0 0x5604a3261040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 --2- 172.21.15.114:0/3689907791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3260b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 >> 172.21.15.114:0/3689907791 conn(0x5604a2ea1990 msgr2=0x5604a3256540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.553+0000 7fc14be00640 0 client.26914 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.554+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 shutdown_connections 2024-09-21T06:59:26.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.554+0000 7fc174286740 1 -- 172.21.15.114:0/3689907791 wait complete. 2024-09-21T06:59:26.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.555+0000 7fc174286740 20 client.26914 trim_cache size 0 max 16384 2024-09-21T06:59:26.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.586+0000 7fc174286740 1 Processor -- start 2024-09-21T06:59:26.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.586+0000 7fc174286740 1 -- start start 2024-09-21T06:59:26.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc174286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3258e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc174286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc174286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a32629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc174286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400ff80 con 0x5604a2e41570 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc174286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc144011be0 con 0x5604a2e84980 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc174286740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc144010650 con 0x5604a2e6c1c0 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc166000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.587+0000 7fc167400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a32629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.588+0000 7fc167400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a32629e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46130/0 (socket says 172.21.15.114:46130) 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.588+0000 7fc166a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3258e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.588+0000 7fc166000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35512/0 (socket says 172.21.15.114:35512) 2024-09-21T06:59:26.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.588+0000 7fc166000640 1 -- 172.21.15.114:0/3720440488 learned_addr learned my addr 172.21.15.114:0/3720440488 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc166000640 1 -- 172.21.15.114:0/3720440488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 msgr2=0x5604a32629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc166000640 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a32629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc166000640 1 -- 172.21.15.114:0/3720440488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 msgr2=0x5604a3258e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc166000640 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3258e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc166000640 1 -- 172.21.15.114:0/3720440488 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604a3257800 con 0x5604a2e6c1c0 2024-09-21T06:59:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc167400640 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a32629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.589+0000 7fc166000640 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32532d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc158009250 tx=0x7fc158038000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea1fc24beda32c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.590+0000 7fc156a00640 1 -- 172.21.15.114:0/3720440488 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158037040 con 0x5604a2e6c1c0 2024-09-21T06:59:26.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.590+0000 7fc156a00640 1 -- 172.21.15.114:0/3720440488 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1580054f0 con 0x5604a2e6c1c0 2024-09-21T06:59:26.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.590+0000 7fc156a00640 1 -- 172.21.15.114:0/3720440488 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158004da0 con 0x5604a2e6c1c0 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 -- 172.21.15.114:0/3720440488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 msgr2=0x5604a32532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32532d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc158009250 tx=0x7fc158038000 comp rx=0 tx=0).stop 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 -- 172.21.15.114:0/3720440488 shutdown_connections 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a32629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32532d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 --2- 172.21.15.114:0/3720440488 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3258e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 -- 172.21.15.114:0/3720440488 >> 172.21.15.114:0/3720440488 conn(0x5604a2ea1990 msgr2=0x5604a3256540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 -- 172.21.15.114:0/3720440488 shutdown_connections 2024-09-21T06:59:26.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.591+0000 7fc174286740 1 -- 172.21.15.114:0/3720440488 wait complete. 2024-09-21T06:59:26.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.592+0000 7fc174286740 1 -- start start 2024-09-21T06:59:26.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.592+0000 7fc174286740 1 Processor -- start 2024-09-21T06:59:26.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.594+0000 7fc174286740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3267280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.594+0000 7fc174286740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.594+0000 7fc174286740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3267d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.594+0000 7fc174286740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc144011450 con 0x5604a2e41570 2024-09-21T06:59:26.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc174286740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc144010d20 con 0x5604a2e84980 2024-09-21T06:59:26.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc174286740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400df60 con 0x5604a2e6c1c0 2024-09-21T06:59:26.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc174286740 10 client.0 _ll_register_callbacks cb 0x7fc150009300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:26.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc167400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3267d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc167400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3267d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46140/0 (socket says 172.21.15.114:46140) 2024-09-21T06:59:26.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc167400640 1 -- 172.21.15.114:0/759026802 learned_addr learned my addr 172.21.15.114:0/759026802 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:26.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.595+0000 7fc166000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.596+0000 7fc166a00640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3267280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.597+0000 7fc167400640 1 -- 172.21.15.114:0/759026802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 msgr2=0x5604a32677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.597+0000 7fc167400640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.597+0000 7fc167400640 1 -- 172.21.15.114:0/759026802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 msgr2=0x5604a3267280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:26.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.597+0000 7fc167400640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3267280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.597+0000 7fc167400640 1 -- 172.21.15.114:0/759026802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604a32dd0a0 con 0x5604a2e84980 2024-09-21T06:59:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.598+0000 7fc167400640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3267d00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1500a8fa0 tx=0x7fc150010870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ee772afbeb37b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.598+0000 7fc166000640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:26.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.598+0000 7fc166a00640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3267280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.598+0000 7fc14be00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.598+0000 7fc14be00640 1 -- 172.21.15.114:0/759026802 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500ac7a0 con 0x5604a2e84980 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.598+0000 7fc14be00640 1 -- 172.21.15.114:0/759026802 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc1500e4550 con 0x5604a2e84980 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.599+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1580054f0 con 0x5604a2e84980 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.599+0000 7fc14be00640 1 -- 172.21.15.114:0/759026802 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc1500a9020 con 0x5604a2e84980 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.599+0000 7fc174286740 10 client.26957 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:26.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.599+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5604a32de300 con 0x5604a2e84980 2024-09-21T06:59:26.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.599+0000 7fc14be00640 1 -- 172.21.15.114:0/759026802 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1580054f0 con 0x5604a2e84980 2024-09-21T06:59:26.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc14be00640 1 -- 172.21.15.114:0/759026802 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fc1500c2b80 con 0x5604a2e84980 2024-09-21T06:59:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc174286740 10 client.26957 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc174286740 10 client.26957 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:59:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc174286740 20 client.26957 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc174286740 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a32deab0 0x5604a32e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc174286740 4 client.26957 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-21T06:59:26.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.602+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x5604a31c86a0 con 0x5604a32deab0 2024-09-21T06:59:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.603+0000 7fc166a00640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a32deab0 0x5604a32e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.603+0000 7fc166a00640 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a32deab0 0x5604a32e0ed0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e600a64ed1d042 server_cookie=2bf71bd3a8abf608 in_seq=0 out_seq=0 2024-09-21T06:59:26.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.603+0000 7fc14be00640 10 client.26957 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc14be00640 1 -- 172.21.15.114:0/759026802 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5604a31c86a0 con 0x5604a32deab0 2024-09-21T06:59:26.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc14be00640 10 client.26957 handle_command_reply: tid=0 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 client.26957 shutdown 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 msgr2=0x5604a3267d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3267d00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1500a8fa0 tx=0x7fc150010870 comp rx=0 tx=0).stop 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 shutdown_connections 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604a2e84980 0x5604a3267d00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5604a32deab0 0x5604a32e0ed0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604a2e6c1c0 0x5604a32677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 --2- 172.21.15.114:0/759026802 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604a2e41570 0x5604a3267280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc14be00640 0 client.26957 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.835+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 >> 172.21.15.114:0/759026802 conn(0x5604a2ea1990 msgr2=0x5604a326c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 shutdown_connections 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc174286740 1 -- 172.21.15.114:0/759026802 wait complete. 2024-09-21T06:59:26.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc174286740 20 client.26957 trim_cache size 0 max 16384 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 39, 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 18, 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "ca4b3545": { 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 18, 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:59:26.839 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 130.0, 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:26.840 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc14c0b6770 msgr2=0x7fc14c0b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc167e00640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc14c0b6770 0x7fc14c0b8c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x5604a32564f0 tx=0x7fc15802c040 comp rx=0 tx=0).stop 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 msgr2=0x7fc168107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:26.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.836+0000 7fc167e00640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 0x7fc168107c50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc150009fd0 tx=0x7fc15000b8e0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 shutdown_connections 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc14c0b6770 0x7fc14c0b8c80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc16810dc30 0x7fc168107c50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1681050b0 0x7fc168107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 --2- 172.21.15.114:0/1266842214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc168104680 0x7fc16810d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 >> 172.21.15.114:0/1266842214 conn(0x7fc1680fe330 msgr2=0x7fc168100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 shutdown_connections 2024-09-21T06:59:26.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:26.837+0000 7fc167e00640 1 -- 172.21.15.114:0/1266842214 wait complete. 2024-09-21T06:59:27.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:27 smithi114 ceph-mon[30654]: pgmap v555: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-21T06:59:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:27 smithi183 ceph-mon[34219]: pgmap v555: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-21T06:59:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:27 smithi193 ceph-mon[34601]: pgmap v555: 129 pgs: 129 active+clean; 2.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 599 op/s 2024-09-21T06:59:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 290.0 s, 796.6 tps, lat 92.955 ms stddev 128.944 2024-09-21T06:59:29.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:29 smithi114 ceph-mon[30654]: pgmap v556: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 647 op/s 2024-09-21T06:59:29.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:29 smithi183 ceph-mon[34219]: pgmap v556: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 647 op/s 2024-09-21T06:59:29.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:29 smithi193 ceph-mon[34601]: pgmap v556: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 647 op/s 2024-09-21T06:59:30.256 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ca4b3545', quiesce took 10.6 seconds. Will release after: -0.5999999999999996 2024-09-21T06:59:30.257 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:59:30.673 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 -- 172.21.15.114:0/3500798811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34075dd0 msgr2=0x7fec340761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 --2- 172.21.15.114:0/3500798811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34075dd0 0x7fec340761e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fec20009900 tx=0x7fec2002a410 comp rx=0 tx=0).stop 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 -- 172.21.15.114:0/3500798811 shutdown_connections 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 --2- 172.21.15.114:0/3500798811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34114280 0x7fec34116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 --2- 172.21.15.114:0/3500798811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34075dd0 0x7fec340761e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 --2- 172.21.15.114:0/3500798811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec34077380 0x7fec340757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 -- 172.21.15.114:0/3500798811 >> 172.21.15.114:0/3500798811 conn(0x7fec340ff440 msgr2=0x7fec34101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 -- 172.21.15.114:0/3500798811 shutdown_connections 2024-09-21T06:59:31.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.340+0000 7fec3a800640 1 -- 172.21.15.114:0/3500798811 wait complete. 2024-09-21T06:59:31.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 Processor -- start 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 -- start start 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 0x7fec34113a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34077380 0x7fec3410cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec34114280 0x7fec3410d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fec34119f10 con 0x7fec34075dd0 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec34119d90 con 0x7fec34114280 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.342+0000 7fec3a800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec3411a090 con 0x7fec34077380 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.343+0000 7fec39400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 0x7fec34113a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.343+0000 7fec39400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 0x7fec34113a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45084/0 (socket says 172.21.15.114:45084) 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.343+0000 7fec39400640 1 -- 172.21.15.114:0/411381419 learned_addr learned my addr 172.21.15.114:0/411381419 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.343+0000 7fec39e00640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec34114280 0x7fec3410d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:31.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.344+0000 7fec39400640 1 -- 172.21.15.114:0/411381419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34077380 msgr2=0x7fec3410cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.344+0000 7fec39400640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34077380 0x7fec3410cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.344+0000 7fec39400640 1 -- 172.21.15.114:0/411381419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec34114280 msgr2=0x7fec3410d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.344+0000 7fec39400640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec34114280 0x7fec3410d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.344+0000 7fec39400640 1 -- 172.21.15.114:0/411381419 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3410db80 con 0x7fec34075dd0 2024-09-21T06:59:31.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.344+0000 7fec39400640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 0x7fec34113a90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fec2800e9f0 tx=0x7fec2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:31.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.348+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2800cd80 con 0x7fec34075dd0 2024-09-21T06:59:31.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.348+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec34112820 con 0x7fec34075dd0 2024-09-21T06:59:31.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.348+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec34112d60 con 0x7fec34075dd0 2024-09-21T06:59:31.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.349+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec280054d0 con 0x7fec34075dd0 2024-09-21T06:59:31.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.349+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28010430 con 0x7fec34075dd0 2024-09-21T06:59:31.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.352+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec140052c0 con 0x7fec34075dd0 2024-09-21T06:59:31.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.372+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fec280046c0 con 0x7fec34075dd0 2024-09-21T06:59:31.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.375+0000 7fec32000640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec1c0b6700 0x7fec1c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.378+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fec280e7e80 con 0x7fec34075dd0 2024-09-21T06:59:31.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.379+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec280e8380 con 0x7fec34075dd0 2024-09-21T06:59:31.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.381+0000 7fec38a00640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec1c0b6700 0x7fec1c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:31.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.383+0000 7fec38a00640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec1c0b6700 0x7fec1c0b8c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fec20009230 tx=0x7fec20031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:31.571 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:31 smithi114 ceph-mon[30654]: pgmap v557: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 562 op/s 2024-09-21T06:59:31.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:31 smithi183 ceph-mon[34219]: pgmap v557: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 562 op/s 2024-09-21T06:59:31.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:31 smithi193 ceph-mon[34601]: pgmap v557: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.9 MiB/s wr, 562 op/s 2024-09-21T06:59:31.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.731+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec140055b0 con 0x7fec34075dd0 2024-09-21T06:59:31.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.732+0000 7fec32000640 1 -- 172.21.15.114:0/411381419 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11999 (secure 0 0 0) 0x7fec280ae060 con 0x7fec34075dd0 2024-09-21T06:59:31.735 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:59:31.735 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"btime":"2024-09-21T06:58:51:043259+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:58:50.613377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:59:31.735 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-21T06:59:31.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec1c0b6700 msgr2=0x7fec1c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:31.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec1c0b6700 0x7fec1c0b8c10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fec20009230 tx=0x7fec20031400 comp rx=0 tx=0).stop 2024-09-21T06:59:31.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 msgr2=0x7fec34113a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:31.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 0x7fec34113a90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fec2800e9f0 tx=0x7fec2800edf0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 shutdown_connections 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fec1c0b6700 0x7fec1c0b8c10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec34114280 0x7fec3410d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec34077380 0x7fec3410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 --2- 172.21.15.114:0/411381419 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fec34075dd0 0x7fec34113a90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.734+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 >> 172.21.15.114:0/411381419 conn(0x7fec340ff440 msgr2=0x7fec341033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.735+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 shutdown_connections 2024-09-21T06:59:31.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:31.735+0000 7fec3a800640 1 -- 172.21.15.114:0/411381419 wait complete. 2024-09-21T06:59:32.562 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:32 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/411381419' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:32.601 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id ca4b3545 --release --await' 2024-09-21T06:59:32.601 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id ca4b3545 --release --await 2024-09-21T06:59:32.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:32 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/411381419' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:32.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:32 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/411381419' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:32.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 295.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T06:59:32.972 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:33.392 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:33 smithi114 ceph-mon[30654]: pgmap v558: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 538 op/s 2024-09-21T06:59:33.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 -- 172.21.15.114:0/3477652939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b010b500 msgr2=0x7f82b010b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 --2- 172.21.15.114:0/3477652939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b010b500 0x7f82b010b910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 -- 172.21.15.114:0/3477652939 shutdown_connections 2024-09-21T06:59:33.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 --2- 172.21.15.114:0/3477652939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82b010be50 0x7f82b0112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 --2- 172.21.15.114:0/3477652939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b010b500 0x7f82b010b910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 --2- 172.21.15.114:0/3477652939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b0105bc0 0x7f82b0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 -- 172.21.15.114:0/3477652939 >> 172.21.15.114:0/3477652939 conn(0x7f82b0101670 msgr2=0x7f82b0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 -- 172.21.15.114:0/3477652939 shutdown_connections 2024-09-21T06:59:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.570+0000 7f82afe00640 1 -- 172.21.15.114:0/3477652939 wait complete. 2024-09-21T06:59:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.571+0000 7f82afe00640 1 Processor -- start 2024-09-21T06:59:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.571+0000 7f82afe00640 1 -- start start 2024-09-21T06:59:33.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82afe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 0x7f82b019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82afe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 0x7f82b019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82b010be50 0x7f82b01a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82afe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0115b10 con 0x7f82b0105bc0 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82afe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0115990 con 0x7f82b010be50 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82afe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0115c90 con 0x7f82b010b500 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82ae000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 0x7f82b019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82aea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 0x7f82b019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82ae000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 0x7f82b019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35564/0 (socket says 172.21.15.114:35564) 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82ae000640 1 -- 172.21.15.114:0/2504743286 learned_addr learned my addr 172.21.15.114:0/2504743286 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82aea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 0x7f82b019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45088/0 (socket says 172.21.15.114:45088) 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.572+0000 7f82af400640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82b010be50 0x7f82b01a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82ae000640 1 -- 172.21.15.114:0/2504743286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82b010be50 msgr2=0x7f82b01a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82ae000640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82b010be50 0x7f82b01a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82ae000640 1 -- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 msgr2=0x7f82b019c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82ae000640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 0x7f82b019c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82ae000640 1 -- 172.21.15.114:0/2504743286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b01a2450 con 0x7f82b010b500 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82aea00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 0x7f82b019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82ae000640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 0x7f82b019ca80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f829c02a910 tx=0x7f829c031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82a7400640 1 -- 172.21.15.114:0/2504743286 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c03f030 con 0x7f82b010b500 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b01a26e0 con 0x7f82b010b500 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82a7400640 1 -- 172.21.15.114:0/2504743286 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c004490 con 0x7f82b010b500 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.573+0000 7f82a7400640 1 -- 172.21.15.114:0/2504743286 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c035070 con 0x7f82b010b500 2024-09-21T06:59:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.574+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01a8ea0 con 0x7f82b010b500 2024-09-21T06:59:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 Processor -- start 2024-09-21T06:59:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 -- start start 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a19340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a19dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555799990290 con 0x555799672520 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55579997f6c0 con 0x555799649660 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5557999834d0 con 0x555799651840 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82a7400640 1 -- 172.21.15.114:0/2504743286 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f829c042030 con 0x7f82b010b500 2024-09-21T06:59:33.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.577+0000 7f82af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a19340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a19340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45094/0 (socket says 172.21.15.114:45094) 2024-09-21T06:59:33.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 -- 172.21.15.114:0/1598686279 learned_addr learned my addr 172.21.15.114:0/1598686279 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:33.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82aea00640 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82ae000640 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a19dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 -- 172.21.15.114:0/1598686279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 msgr2=0x555799a19dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a19dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 -- 172.21.15.114:0/1598686279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 msgr2=0x555799a19880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a19880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 -- 172.21.15.114:0/1598686279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55579990c460 con 0x555799672520 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82ae000640 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a19dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.578+0000 7f82af400640 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a19340 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b0ce8588cd6fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.579+0000 7f829b400640 1 -- 172.21.15.114:0/1598686279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a801e070 con 0x555799672520 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.579+0000 7f829b400640 1 -- 172.21.15.114:0/1598686279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8009e50 con 0x555799672520 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.579+0000 7f829b400640 1 -- 172.21.15.114:0/1598686279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a800ea10 con 0x555799672520 2024-09-21T06:59:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.579+0000 7f82a7400640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f82940b6680 0x7f82940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.579+0000 7f82a7400640 1 -- 172.21.15.114:0/2504743286 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f829c107db0 con 0x7f82b010b500 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.579+0000 7f82aea00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f82940b6680 0x7f82940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1598686279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 msgr2=0x555799a19340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a19340 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a8011810 comp rx=0 tx=0).stop 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1598686279 shutdown_connections 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a19dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a19880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1598686279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a19340 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1598686279 >> 172.21.15.114:0/1598686279 conn(0x555799666910 msgr2=0x555799a17230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1598686279 shutdown_connections 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1598686279 wait complete. 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.580+0000 7f82aea00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f82940b6680 0x7f82940b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x555799a17fc0 tx=0x7f82a0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 -- start start 2024-09-21T06:59:33.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 Processor -- start 2024-09-21T06:59:33.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799649660 0x555799a24f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799651840 0x555799a25470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5557997e0bb0 con 0x555799672520 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55579991a6c0 con 0x555799651840 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.581+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555799a27b20 con 0x555799649660 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82bc3bd740 10 client.0 _ll_register_callbacks cb 0x555799935490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82aea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799649660 0x555799a24f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45102/0 (socket says 172.21.15.114:45102) 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82af400640 1 -- 172.21.15.114:0/1333405326 learned_addr learned my addr 172.21.15.114:0/1333405326 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82ae000640 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799651840 0x555799a25470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82aea00640 1 -- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799651840 msgr2=0x555799a25470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82aea00640 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799651840 0x555799a25470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82aea00640 1 -- 172.21.15.114:0/1333405326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 msgr2=0x555799a30e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82aea00640 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.582+0000 7f82aea00640 1 -- 172.21.15.114:0/1333405326 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557996ddb00 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f82aea00640 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799649660 0x555799a24f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f82a0005fd0 tx=0x7f82a0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71fec42f520ac168 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f828fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f828fe00640 1 -- 172.21.15.114:0/1333405326 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0004520 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a8009e50 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f828fe00640 1 -- 172.21.15.114:0/1333405326 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f82a0018030 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f828fe00640 1 -- 172.21.15.114:0/1333405326 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f82a00046c0 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f82bc3bd740 10 client.26935 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555799754960 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.583+0000 7f828fe00640 1 -- 172.21.15.114:0/1333405326 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a8009e50 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f828fe00640 1 -- 172.21.15.114:0/1333405326 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f82a0011ce0 con 0x555799649660 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82bc3bd740 10 client.26935 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82bc3bd740 10 client.26935 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82bc3bd740 20 client.26935 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:33.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799630ea0 0x555799a36f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82bc3bd740 4 client.26935 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:33.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557999921e0 con 0x555799630ea0 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.584+0000 7f82ae000640 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799630ea0 0x555799a36f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.585+0000 7f82ae000640 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799630ea0 0x555799a36f80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7444d180d31b51cc server_cookie=c14948a25f5d3b94 in_seq=0 out_seq=0 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.585+0000 7f828fe00640 10 client.26935 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f828fe00640 1 -- 172.21.15.114:0/1333405326 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5557999921e0 con 0x555799630ea0 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f828fe00640 10 client.26935 handle_command_reply: tid=0 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 client.26935 shutdown 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799649660 msgr2=0x555799a24f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799649660 0x555799a24f30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f82a0005fd0 tx=0x7f82a0004ab0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 shutdown_connections 2024-09-21T06:59:33.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799651840 0x555799a25470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799630ea0 0x555799a36f80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/1333405326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799649660 0x555799a24f30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f828fe00640 0 client.26935 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 >> 172.21.15.114:0/1333405326 conn(0x555799666910 msgr2=0x555799a1c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 shutdown_connections 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 1 -- 172.21.15.114:0/1333405326 wait complete. 2024-09-21T06:59:33.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.586+0000 7f82bc3bd740 20 client.26935 trim_cache size 0 max 16384 2024-09-21T06:59:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.604+0000 7f82bc3bd740 1 Processor -- start 2024-09-21T06:59:33.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.604+0000 7f82bc3bd740 1 -- start start 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a1aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01b2eb0 con 0x555799672520 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01b4b10 con 0x555799649660 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01b3580 con 0x555799651840 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82af400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45118/0 (socket says 172.21.15.114:45118) 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82af400640 1 -- 172.21.15.114:0/934523433 learned_addr learned my addr 172.21.15.114:0/934523433 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82aea00640 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82ae000640 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a1aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82af400640 1 -- 172.21.15.114:0/934523433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 msgr2=0x555799a1aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.605+0000 7f82af400640 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a1aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f82af400640 1 -- 172.21.15.114:0/934523433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 msgr2=0x555799a1fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f82af400640 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a1fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f82af400640 1 -- 172.21.15.114:0/934523433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555799a1b630 con 0x555799672520 2024-09-21T06:59:33.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f82aea00640 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a1fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:33.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f82af400640 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7fa970709ce116 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:33.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f829aa00640 1 -- 172.21.15.114:0/934523433 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a8028030 con 0x555799672520 2024-09-21T06:59:33.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f829aa00640 1 -- 172.21.15.114:0/934523433 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a80098f0 con 0x555799672520 2024-09-21T06:59:33.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.606+0000 7f829aa00640 1 -- 172.21.15.114:0/934523433 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a801e040 con 0x555799672520 2024-09-21T06:59:33.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 -- 172.21.15.114:0/934523433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 msgr2=0x555799a30d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009ca0 tx=0x7f82a8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 -- 172.21.15.114:0/934523433 shutdown_connections 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799672520 0x555799a30d40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799651840 0x555799a1aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/934523433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a1fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 -- 172.21.15.114:0/934523433 >> 172.21.15.114:0/934523433 conn(0x555799666910 msgr2=0x555799a1a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 -- 172.21.15.114:0/934523433 shutdown_connections 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.607+0000 7f82bc3bd740 1 -- 172.21.15.114:0/934523433 wait complete. 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 -- start start 2024-09-21T06:59:33.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 Processor -- start 2024-09-21T06:59:33.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a29d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 0x555799a2a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799672520 0x555799a2a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01b4380 con 0x555799651840 2024-09-21T06:59:33.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01b3c50 con 0x555799649660 2024-09-21T06:59:33.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01b0e90 con 0x555799672520 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.608+0000 7f82bc3bd740 10 client.0 _ll_register_callbacks cb 0x7f82a0012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 0x555799a2a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 0x555799a2a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45134/0 (socket says 172.21.15.114:45134) 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 -- 172.21.15.114:0/4107592777 learned_addr learned my addr 172.21.15.114:0/4107592777 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82af400640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799672520 0x555799a2a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82aea00640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a29d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 -- 172.21.15.114:0/4107592777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799672520 msgr2=0x555799a2a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799672520 0x555799a2a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 -- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 msgr2=0x555799a29d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a29d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 -- 172.21.15.114:0/4107592777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555799aa1990 con 0x555799651840 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f82ae000640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 0x555799a2a290 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f829c0cab50 tx=0x7f829c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc0db3ffff38cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:33.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.609+0000 7f828fe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f828fe00640 1 -- 172.21.15.114:0/4107592777 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c049040 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f828fe00640 1 -- 172.21.15.114:0/4107592777 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f829c0ceb60 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a80098f0 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f828fe00640 1 -- 172.21.15.114:0/4107592777 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f829c1055b0 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 10 client.16728 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555799aa2c80 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f828fe00640 1 -- 172.21.15.114:0/4107592777 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a80098f0 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f828fe00640 1 -- 172.21.15.114:0/4107592777 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f829c0c9230 con 0x555799651840 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 10 client.16728 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 10 client.16728 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 20 client.16728 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799606250 0x555799aa5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 4 client.16728 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ca4b3545", "release": true, "await": true}] 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.610+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ca4b3545", "release": true, "await": true}) -- 0x55579998dc90 con 0x555799606250 2024-09-21T06:59:33.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.611+0000 7f82aea00640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799606250 0x555799aa5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:33.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:33 smithi183 ceph-mon[34219]: pgmap v558: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 538 op/s 2024-09-21T06:59:33.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.611+0000 7f82aea00640 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799606250 0x555799aa5480 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc5d476b6ea6ab8 server_cookie=755c1bd20ede4154 in_seq=0 out_seq=0 2024-09-21T06:59:33.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.611+0000 7f828fe00640 10 client.16728 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.642+0000 7f828fe00640 1 -- 172.21.15.114:0/4107592777 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55579998dc90 con 0x555799606250 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.642+0000 7f828fe00640 10 client.16728 handle_command_reply: tid=0 2024-09-21T06:59:33.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.642+0000 7f82bc3bd740 1 client.16728 shutdown 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.643+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 msgr2=0x555799a2a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.643+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 0x555799a2a290 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f829c0cab50 tx=0x7f829c02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 shutdown_connections 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555799606250 0x555799aa5480 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555799672520 0x555799a2a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555799651840 0x555799a2a290 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 --2- 172.21.15.114:0/4107592777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555799649660 0x555799a29d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f828fe00640 0 client.16728 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 >> 172.21.15.114:0/4107592777 conn(0x555799666910 msgr2=0x555799a1ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 shutdown_connections 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 1 -- 172.21.15.114:0/4107592777 wait complete. 2024-09-21T06:59:33.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.644+0000 7f82bc3bd740 20 client.16728 trim_cache size 0 max 16384 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 39, 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 20, 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "ca4b3545": { 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 20, 2024-09-21T06:59:33.650 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 130.0, 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:33.651 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:33.652 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.650+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f82940b6680 msgr2=0x7f82940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.650+0000 7f82afe00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f82940b6680 0x7f82940b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x555799a17fc0 tx=0x7f82a0009500 comp rx=0 tx=0).stop 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.650+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 msgr2=0x7f82b019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:33.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.650+0000 7f82afe00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 0x7f82b019ca80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f829c02a910 tx=0x7f829c031a20 comp rx=0 tx=0).stop 2024-09-21T06:59:33.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.654+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 shutdown_connections 2024-09-21T06:59:33.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.654+0000 7f82afe00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f82940b6680 0x7f82940b8b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.654+0000 7f82afe00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82b010be50 0x7f82b01a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.654+0000 7f82afe00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b010b500 0x7f82b019ca80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.654+0000 7f82afe00640 1 --2- 172.21.15.114:0/2504743286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f82b0105bc0 0x7f82b019c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:33.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.654+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 >> 172.21.15.114:0/2504743286 conn(0x7f82b0101670 msgr2=0x7f82b0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:33.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.655+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 shutdown_connections 2024-09-21T06:59:33.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:33.655+0000 7f82afe00640 1 -- 172.21.15.114:0/2504743286 wait complete. 2024-09-21T06:59:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:33 smithi193 ceph-mon[34601]: pgmap v558: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 4.7 MiB/s wr, 538 op/s 2024-09-21T06:59:35.367 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:59:35.466 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ca4b3545', total seconds elapsed: 15.8 2024-09-21T06:59:35.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:35 smithi114 ceph-mon[30654]: pgmap v559: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 491 op/s 2024-09-21T06:59:35.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:35 smithi183 ceph-mon[34219]: pgmap v559: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 491 op/s 2024-09-21T06:59:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:35 smithi193 ceph-mon[34601]: pgmap v559: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.2 MiB/s wr, 491 op/s 2024-09-21T06:59:36.066 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.969+0000 7fd579800640 1 -- 172.21.15.114:0/1318180570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd574111cd0 msgr2=0x7fd57410d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.969+0000 7fd579800640 1 --2- 172.21.15.114:0/1318180570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd574111cd0 0x7fd57410d6b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd564009860 tx=0x7fd56402a340 comp rx=0 tx=0).stop 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.970+0000 7fd579800640 1 -- 172.21.15.114:0/1318180570 shutdown_connections 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.970+0000 7fd579800640 1 --2- 172.21.15.114:0/1318180570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd57411cba0 0x7fd57411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.970+0000 7fd579800640 1 --2- 172.21.15.114:0/1318180570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd57410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.970+0000 7fd579800640 1 --2- 172.21.15.114:0/1318180570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd574111cd0 0x7fd57410d6b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.970+0000 7fd579800640 1 -- 172.21.15.114:0/1318180570 >> 172.21.15.114:0/1318180570 conn(0x7fd57406e9f0 msgr2=0x7fd57406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:36.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.970+0000 7fd579800640 1 -- 172.21.15.114:0/1318180570 shutdown_connections 2024-09-21T06:59:36.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.971+0000 7fd579800640 1 -- 172.21.15.114:0/1318180570 wait complete. 2024-09-21T06:59:36.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.973+0000 7fd579800640 1 Processor -- start 2024-09-21T06:59:36.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.974+0000 7fd579800640 1 -- start start 2024-09-21T06:59:36.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd579800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd5741a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd579800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd574111cd0 0x7fd5741a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd579800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd57411cba0 0x7fd5741ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd579800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741226f0 con 0x7fd574111cd0 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd579800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd574122570 con 0x7fd57410dbf0 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd579800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd574122870 con 0x7fd57411cba0 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd573e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd5741a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd573e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd5741a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54840/0 (socket says 172.21.15.114:54840) 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd573e00640 1 -- 172.21.15.114:0/3806308718 learned_addr learned my addr 172.21.15.114:0/3806308718 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.975+0000 7fd573400640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd574111cd0 0x7fd5741a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.976+0000 7fd573e00640 1 -- 172.21.15.114:0/3806308718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd57411cba0 msgr2=0x7fd5741ae9a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.976+0000 7fd573e00640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd57411cba0 0x7fd5741ae9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.976+0000 7fd573e00640 1 -- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd574111cd0 msgr2=0x7fd5741a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:36.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.976+0000 7fd573e00640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd574111cd0 0x7fd5741a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.977+0000 7fd573e00640 1 -- 172.21.15.114:0/3806308718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741af010 con 0x7fd57410dbf0 2024-09-21T06:59:36.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.978+0000 7fd573e00640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd5741a9160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd564035040 tx=0x7fd564032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:36.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.980+0000 7fd570c00640 1 -- 172.21.15.114:0/3806308718 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564037040 con 0x7fd57410dbf0 2024-09-21T06:59:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.980+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741af2a0 con 0x7fd57410dbf0 2024-09-21T06:59:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.981+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5741af770 con 0x7fd57410dbf0 2024-09-21T06:59:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.981+0000 7fd570c00640 1 -- 172.21.15.114:0/3806308718 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd564041070 con 0x7fd57410dbf0 2024-09-21T06:59:36.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.981+0000 7fd570c00640 1 -- 172.21.15.114:0/3806308718 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640035f0 con 0x7fd57410dbf0 2024-09-21T06:59:36.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.988+0000 7fd581a38740 1 Processor -- start 2024-09-21T06:59:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.989+0000 7fd581a38740 1 -- start start 2024-09-21T06:59:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.989+0000 7fd581a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57a93f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.989+0000 7fd581a38740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576c41b0 0x55ce57a94470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd581a38740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576cc390 0x55ce57a949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd578e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57a93f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd578e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57a93f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54842/0 (socket says 172.21.15.114:54842) 2024-09-21T06:59:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd578e00640 1 -- 172.21.15.114:0/3435798140 learned_addr learned my addr 172.21.15.114:0/3435798140 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ce57a0af70 con 0x55ce576cc390 2024-09-21T06:59:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce579fa3a0 con 0x55ce576ed070 2024-09-21T06:59:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd570c00640 1 -- 172.21.15.114:0/3806308718 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd564004050 con 0x7fd57410dbf0 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.990+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ce579fe1b0 con 0x55ce576c41b0 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.991+0000 7fd573e00640 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576c41b0 0x55ce57a94470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.991+0000 7fd573e00640 1 -- 172.21.15.114:0/3435798140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 msgr2=0x55ce57a93f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.991+0000 7fd573e00640 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57a93f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.991+0000 7fd573e00640 1 -- 172.21.15.114:0/3435798140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576cc390 msgr2=0x55ce57a949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.991+0000 7fd573e00640 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576cc390 0x55ce57a949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.991+0000 7fd573e00640 1 -- 172.21.15.114:0/3435798140 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce579c4c90 con 0x55ce576c41b0 2024-09-21T06:59:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.992+0000 7fd578e00640 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57a93f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.992+0000 7fd573e00640 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576c41b0 0x55ce57a94470 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd5641045f0 tx=0x7fd56410f3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b21a0b2ad1dc41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.995+0000 7fd569600640 1 -- 172.21.15.114:0/3435798140 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5641090d0 con 0x55ce576c41b0 2024-09-21T06:59:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.995+0000 7fd569600640 1 -- 172.21.15.114:0/3435798140 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd564114030 con 0x55ce576c41b0 2024-09-21T06:59:36.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.995+0000 7fd569600640 1 -- 172.21.15.114:0/3435798140 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564118630 con 0x55ce576c41b0 2024-09-21T06:59:36.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.995+0000 7fd570c00640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5580b6730 0x7fd5580b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.996+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576c41b0 msgr2=0x55ce57a94470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.996+0000 7fd581a38740 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576c41b0 0x55ce57a94470 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd5641045f0 tx=0x7fd56410f3b0 comp rx=0 tx=0).stop 2024-09-21T06:59:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.997+0000 7fd578e00640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5580b6730 0x7fd5580b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.997+0000 7fd570c00640 1 -- 172.21.15.114:0/3806308718 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd564107210 con 0x7fd57410dbf0 2024-09-21T06:59:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.998+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 shutdown_connections 2024-09-21T06:59:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.998+0000 7fd581a38740 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576cc390 0x55ce57a949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.998+0000 7fd581a38740 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576c41b0 0x55ce57a94470 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.999+0000 7fd581a38740 1 --2- 172.21.15.114:0/3435798140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57a93f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.999+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 >> 172.21.15.114:0/3435798140 conn(0x55ce576e1460 msgr2=0x55ce57a91e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:37.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:36.999+0000 7fd578e00640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5580b6730 0x7fd5580b8c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd56c007fd0 tx=0x7fd56c007560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:37.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.000+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 shutdown_connections 2024-09-21T06:59:37.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.000+0000 7fd581a38740 1 -- 172.21.15.114:0/3435798140 wait complete. 2024-09-21T06:59:37.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.001+0000 7fd581a38740 1 -- start start 2024-09-21T06:59:37.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.001+0000 7fd581a38740 1 Processor -- start 2024-09-21T06:59:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.003+0000 7fd581a38740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a9f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.003+0000 7fd581a38740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576cc390 0x55ce57a9fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.003+0000 7fd581a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57aab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd573e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a9f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd573400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576cc390 0x55ce57a9fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd581a38740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ce5785b700 con 0x55ce576c41b0 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd573e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a9f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56816/0 (socket says 172.21.15.114:56816) 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd581a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ce57995540 con 0x55ce576ed070 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd581a38740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ce57aa24d0 con 0x55ce576cc390 2024-09-21T06:59:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd573e00640 1 -- 172.21.15.114:0/33807108 learned_addr learned my addr 172.21.15.114:0/33807108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:37.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.004+0000 7fd578e00640 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57aab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.006+0000 7fd581a38740 10 client.0 _ll_register_callbacks cb 0x55ce579a5430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd573400640 1 -- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 msgr2=0x55ce57aab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd573400640 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57aab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd573400640 1 -- 172.21.15.114:0/33807108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 msgr2=0x55ce57a9f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd573400640 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a9f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd573400640 1 -- 172.21.15.114:0/33807108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce57758650 con 0x55ce576cc390 2024-09-21T06:59:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd55e000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:37.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.009+0000 7fd573400640 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576cc390 0x55ce57a9fd10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd57406c680 tx=0x7fd560011a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b4bd4aab1ee45c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:37.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.010+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ce57997b50 con 0x55ce576cc390 2024-09-21T06:59:37.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.010+0000 7fd55e000640 1 -- 172.21.15.114:0/33807108 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560004030 con 0x55ce576cc390 2024-09-21T06:59:37.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.010+0000 7fd55e000640 1 -- 172.21.15.114:0/33807108 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd5600041d0 con 0x55ce576cc390 2024-09-21T06:59:37.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.011+0000 7fd55e000640 1 -- 172.21.15.114:0/33807108 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd560024050 con 0x55ce576cc390 2024-09-21T06:59:37.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.011+0000 7fd581a38740 10 client.26950 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:37.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.011+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=39}) -- 0x55ce579b2910 con 0x55ce576cc390 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd55e000640 1 -- 172.21.15.114:0/33807108 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fd560005020 con 0x55ce576cc390 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd581a38740 10 client.26950 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd581a38740 10 client.26950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd581a38740 20 client.26950 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd581a38740 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce576ab9f0 0x55ce57ab1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd581a38740 4 client.26950 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce57a0cec0 con 0x55ce576ab9f0 2024-09-21T06:59:37.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.012+0000 7fd573e00640 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce576ab9f0 0x55ce57ab1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.016+0000 7fd573e00640 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce576ab9f0 0x55ce57ab1510 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f844d86702cd9 server_cookie=b4de0db069340d3c in_seq=0 out_seq=0 2024-09-21T06:59:37.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.016+0000 7fd55e000640 10 client.26950 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:37.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.016+0000 7fd55e000640 1 -- 172.21.15.114:0/33807108 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce57a0cec0 con 0x55ce576ab9f0 2024-09-21T06:59:37.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.016+0000 7fd55e000640 10 client.26950 handle_command_reply: tid=0 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 client.26950 shutdown 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576cc390 msgr2=0x55ce57a9fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576cc390 0x55ce57a9fd10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd57406c680 tx=0x7fd560011a90 comp rx=0 tx=0).stop 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 shutdown_connections 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ed070 0x55ce57aab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce576ab9f0 0x55ce57ab1510 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576cc390 0x55ce57a9fd10 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd57406c680 tx=0x7fd560011a90 comp rx=0 tx=0).stop 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 --2- 172.21.15.114:0/33807108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a9f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd55e000640 0 client.26950 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:37.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.017+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 >> 172.21.15.114:0/33807108 conn(0x55ce576e1460 msgr2=0x55ce57a905c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:37.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.018+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 shutdown_connections 2024-09-21T06:59:37.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.018+0000 7fd581a38740 1 -- 172.21.15.114:0/33807108 wait complete. 2024-09-21T06:59:37.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.019+0000 7fd581a38740 20 client.26950 trim_cache size 0 max 16384 2024-09-21T06:59:37.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.052+0000 7fd581a38740 1 Processor -- start 2024-09-21T06:59:37.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.053+0000 7fd581a38740 1 -- start start 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.054+0000 7fd581a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57a9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd581a38740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a95670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd581a38740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd581a38740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd55400ff00 con 0x55ce576c41b0 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd581a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd554011b60 con 0x55ce576ab9f0 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd581a38740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540105d0 con 0x55ce576ed070 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57a9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57a9e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54862/0 (socket says 172.21.15.114:54862) 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 -- 172.21.15.114:0/4149742401 learned_addr learned my addr 172.21.15.114:0/4149742401 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:37.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd578e00640 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 -- 172.21.15.114:0/4149742401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 msgr2=0x55ce57aab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:37.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 -- 172.21.15.114:0/4149742401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 msgr2=0x55ce57a95670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a95670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.055+0000 7fd573e00640 1 -- 172.21.15.114:0/4149742401 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce57aa0060 con 0x55ce576ab9f0 2024-09-21T06:59:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.058+0000 7fd573e00640 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57a9e680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd564104bf0 tx=0x7fd56402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e88673da0450015 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.058+0000 7fd568c00640 1 -- 172.21.15.114:0/4149742401 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640c9340 con 0x55ce576ab9f0 2024-09-21T06:59:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.058+0000 7fd568c00640 1 -- 172.21.15.114:0/4149742401 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd564114040 con 0x55ce576ab9f0 2024-09-21T06:59:37.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.058+0000 7fd568c00640 1 -- 172.21.15.114:0/4149742401 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640cc460 con 0x55ce576ab9f0 2024-09-21T06:59:37.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 -- 172.21.15.114:0/4149742401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 msgr2=0x55ce57a9e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:37.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57a9e680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd564104bf0 tx=0x7fd56402a340 comp rx=0 tx=0).stop 2024-09-21T06:59:37.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 -- 172.21.15.114:0/4149742401 shutdown_connections 2024-09-21T06:59:37.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57a95670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 --2- 172.21.15.114:0/4149742401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57a9e680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.060+0000 7fd581a38740 1 -- 172.21.15.114:0/4149742401 >> 172.21.15.114:0/4149742401 conn(0x55ce576e1460 msgr2=0x55ce57a94f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:37.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.061+0000 7fd581a38740 1 -- 172.21.15.114:0/4149742401 shutdown_connections 2024-09-21T06:59:37.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.061+0000 7fd581a38740 1 -- 172.21.15.114:0/4149742401 wait complete. 2024-09-21T06:59:37.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.062+0000 7fd581a38740 1 -- start start 2024-09-21T06:59:37.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.062+0000 7fd581a38740 1 Processor -- start 2024-09-21T06:59:37.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57aa7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57aa7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aa26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd5540113d0 con 0x55ce576c41b0 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd554010ca0 con 0x55ce576ab9f0 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd55400dee0 con 0x55ce576ed070 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd581a38740 10 client.0 _ll_register_callbacks cb 0x7fd560009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd573e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57aa7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd573400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57aa7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd578e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aa26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd578e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aa26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50738/0 (socket says 172.21.15.114:50738) 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd573e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57aa7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54864/0 (socket says 172.21.15.114:54864) 2024-09-21T06:59:37.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.063+0000 7fd578e00640 1 -- 172.21.15.114:0/1909441961 learned_addr learned my addr 172.21.15.114:0/1909441961 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:37.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.064+0000 7fd573e00640 1 -- 172.21.15.114:0/1909441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 msgr2=0x55ce57aa26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:37.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.064+0000 7fd573e00640 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aa26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.064+0000 7fd573e00640 1 -- 172.21.15.114:0/1909441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 msgr2=0x55ce57aa7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:37.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.064+0000 7fd573e00640 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57aa7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.064+0000 7fd573e00640 1 -- 172.21.15.114:0/1909441961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce57aa8480 con 0x55ce576ab9f0 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.064+0000 7fd578e00640 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aa26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.065+0000 7fd573e00640 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57aa7730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd564009780 tx=0x7fd5640095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3abdc61d973d5e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.065+0000 7fd55e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.065+0000 7fd55e000640 1 -- 172.21.15.114:0/1909441961 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640cc640 con 0x55ce576ab9f0 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.065+0000 7fd55e000640 1 -- 172.21.15.114:0/1909441961 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd564004340 con 0x55ce576ab9f0 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.066+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd564114040 con 0x55ce576ab9f0 2024-09-21T06:59:37.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.066+0000 7fd55e000640 1 -- 172.21.15.114:0/1909441961 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd5640ccac0 con 0x55ce576ab9f0 2024-09-21T06:59:37.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.066+0000 7fd581a38740 10 client.26999 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:37.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.066+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ce57b1c4b0 con 0x55ce576ab9f0 2024-09-21T06:59:37.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.067+0000 7fd55e000640 1 -- 172.21.15.114:0/1909441961 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd564114040 con 0x55ce576ab9f0 2024-09-21T06:59:37.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd55e000640 1 -- 172.21.15.114:0/1909441961 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fd564104870 con 0x55ce576ab9f0 2024-09-21T06:59:37.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd581a38740 10 client.26999 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd581a38740 10 client.26999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd581a38740 20 client.26999 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd581a38740 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce57680da0 0x55ce57b1e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd581a38740 4 client.26999 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:59:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.068+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce57a08970 con 0x55ce57680da0 2024-09-21T06:59:37.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.069+0000 7fd573400640 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce57680da0 0x55ce57b1e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:37.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.070+0000 7fd573400640 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce57680da0 0x55ce57b1e770 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baeca6fea1df8fa4 server_cookie=c62077f762c96e1d in_seq=0 out_seq=0 2024-09-21T06:59:37.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.070+0000 7fd55e000640 10 client.26999 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:37.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.070+0000 7fd55e000640 1 -- 172.21.15.114:0/1909441961 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55ce57a08970 con 0x55ce57680da0 2024-09-21T06:59:37.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.070+0000 7fd55e000640 10 client.26999 handle_command_reply: tid=0 2024-09-21T06:59:37.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.071+0000 7fd581a38740 1 client.26999 shutdown 2024-09-21T06:59:37.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 msgr2=0x55ce57aa7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:37.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57aa7730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd564009780 tx=0x7fd5640095b0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 shutdown_connections 2024-09-21T06:59:37.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ce57680da0 0x55ce57b1e770 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ce576ed070 0x55ce57aa26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ce576c41b0 0x55ce57aa7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 --2- 172.21.15.114:0/1909441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ce576ab9f0 0x55ce57aa7730 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 >> 172.21.15.114:0/1909441961 conn(0x55ce576e1460 msgr2=0x55ce57aa15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.073+0000 7fd55e000640 0 client.26999 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:37.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.074+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 shutdown_connections 2024-09-21T06:59:37.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.074+0000 7fd581a38740 1 -- 172.21.15.114:0/1909441961 wait complete. 2024-09-21T06:59:37.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.077+0000 7fd581a38740 20 client.26999 trim_cache size 0 max 16384 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:37.081 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:37.082 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:37.083 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T06:59:37.084 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.078+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5580b6730 msgr2=0x7fd5580b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.078+0000 7fd579800640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5580b6730 0x7fd5580b8c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd56c007fd0 tx=0x7fd56c007560 comp rx=0 tx=0).stop 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 msgr2=0x7fd5741a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd5741a9160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd564035040 tx=0x7fd564032dc0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 shutdown_connections 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd57411cba0 0x7fd5741ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd574111cd0 0x7fd5741a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd5580b6730 0x7fd5580b8c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 --2- 172.21.15.114:0/3806308718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd57410dbf0 0x7fd5741a9160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 >> 172.21.15.114:0/3806308718 conn(0x7fd57406e9f0 msgr2=0x7fd57410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 shutdown_connections 2024-09-21T06:59:37.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:37.079+0000 7fd579800640 1 -- 172.21.15.114:0/3806308718 wait complete. 2024-09-21T06:59:37.284 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:37 smithi114 ceph-mon[30654]: pgmap v560: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-21T06:59:37.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:37 smithi183 ceph-mon[34219]: pgmap v560: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-21T06:59:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:37 smithi193 ceph-mon[34601]: pgmap v560: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 379 op/s 2024-09-21T06:59:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 300.0 s, 506.5 tps, lat 441.254 ms stddev 1426.563 2024-09-21T06:59:38.077 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:59:38.077 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:59:38.743 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:39.473 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:39 smithi114 ceph-mon[30654]: pgmap v561: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.7 MiB/s wr, 390 op/s 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 -- 172.21.15.114:0/4134015705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810ca40 msgr2=0x7f526810ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 --2- 172.21.15.114:0/4134015705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810ca40 0x7f526810ce50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f52540074e0 tx=0x7f525402b080 comp rx=0 tx=0).stop 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 -- 172.21.15.114:0/4134015705 shutdown_connections 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 --2- 172.21.15.114:0/4134015705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f526810d390 0x7f5268116e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 --2- 172.21.15.114:0/4134015705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810ca40 0x7f526810ce50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 --2- 172.21.15.114:0/4134015705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810a660 0x7f526810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.490+0000 7f526e400640 1 -- 172.21.15.114:0/4134015705 >> 172.21.15.114:0/4134015705 conn(0x7f5268070c00 msgr2=0x7f5268071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.491+0000 7f526e400640 1 -- 172.21.15.114:0/4134015705 shutdown_connections 2024-09-21T06:59:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.491+0000 7f526e400640 1 -- 172.21.15.114:0/4134015705 wait complete. 2024-09-21T06:59:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.493+0000 7f526e400640 1 Processor -- start 2024-09-21T06:59:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.493+0000 7f526e400640 1 -- start start 2024-09-21T06:59:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.494+0000 7f526e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810a660 0x7f5268114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.494+0000 7f526e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 0x7f52681149c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.494+0000 7f526e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f526810d390 0x7f52681b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.494+0000 7f526e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5268119b60 con 0x7f526810a660 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.494+0000 7f526e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52681199e0 con 0x7f526810ca40 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.494+0000 7f526e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5268119ce0 con 0x7f526810d390 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 0x7f52681149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 0x7f52681149c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54884/0 (socket says 172.21.15.114:54884) 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f526da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f526810d390 0x7f52681b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 -- 172.21.15.114:0/3106582450 learned_addr learned my addr 172.21.15.114:0/3106582450 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f526d000640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810a660 0x7f5268114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 -- 172.21.15.114:0/3106582450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f526810d390 msgr2=0x7f52681b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f526810d390 0x7f52681b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 -- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810a660 msgr2=0x7f5268114480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810a660 0x7f5268114480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.495+0000 7f5267e00640 1 -- 172.21.15.114:0/3106582450 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52681152b0 con 0x7f526810ca40 2024-09-21T06:59:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.496+0000 7f5267e00640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 0x7f52681149c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5254006010 tx=0x7f525402bbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.497+0000 7f5265600640 1 -- 172.21.15.114:0/3106582450 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525400a2e0 con 0x7f526810ca40 2024-09-21T06:59:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.497+0000 7f5265600640 1 -- 172.21.15.114:0/3106582450 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f525400a930 con 0x7f526810ca40 2024-09-21T06:59:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.500+0000 7f5265600640 1 -- 172.21.15.114:0/3106582450 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254008750 con 0x7f526810ca40 2024-09-21T06:59:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.500+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52681b54c0 con 0x7f526810ca40 2024-09-21T06:59:39.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.501+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681b5930 con 0x7f526810ca40 2024-09-21T06:59:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.508+0000 7f5265600640 1 -- 172.21.15.114:0/3106582450 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5254043020 con 0x7f526810ca40 2024-09-21T06:59:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.509+0000 7f5276be8740 1 Processor -- start 2024-09-21T06:59:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.509+0000 7f5276be8740 1 -- start start 2024-09-21T06:59:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.510+0000 7f5276be8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e0493fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.511+0000 7f5265600640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f52500b65a0 0x7f52500b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.511+0000 7f5265600640 1 -- 172.21.15.114:0/3106582450 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5254106990 con 0x7f526810ca40 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.511+0000 7f526d000640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f52500b65a0 0x7f52500b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f5276be8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e04902e2f0 0x55e0493fc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f5276be8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e0493fc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f5276be8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e0493716b0 con 0x55e0490364d0 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f5276be8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e049360ae0 con 0x55e0490571b0 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f5276be8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e0493648f0 con 0x55e04902e2f0 2024-09-21T06:59:39.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f526da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e0493fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f526da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e0493fbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54886/0 (socket says 172.21.15.114:54886) 2024-09-21T06:59:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.512+0000 7f526da00640 1 -- 172.21.15.114:0/3346595096 learned_addr learned my addr 172.21.15.114:0/3346595096 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526da00640 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e0493fc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f5267e00640 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e04902e2f0 0x55e0493fc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526da00640 1 -- 172.21.15.114:0/3346595096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e04902e2f0 msgr2=0x55e0493fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526d000640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f52500b65a0 0x7f52500b8ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5268116ee0 tx=0x7f525c0094f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526da00640 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e04902e2f0 0x55e0493fc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526da00640 1 -- 172.21.15.114:0/3346595096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 msgr2=0x55e0493fc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526da00640 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e0493fc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f526da00640 1 -- 172.21.15.114:0/3346595096 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04932d780 con 0x55e0490571b0 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.513+0000 7f5267e00640 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e04902e2f0 0x55e0493fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.514+0000 7f526da00640 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e0493fbe10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f52600125e0 tx=0x7f526000b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90328ede8dbb8745 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.514+0000 7f525a000640 1 -- 172.21.15.114:0/3346595096 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260004250 con 0x55e0490571b0 2024-09-21T06:59:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.514+0000 7f525a000640 1 -- 172.21.15.114:0/3346595096 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52600043f0 con 0x55e0490571b0 2024-09-21T06:59:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.515+0000 7f525a000640 1 -- 172.21.15.114:0/3346595096 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260010970 con 0x55e0490571b0 2024-09-21T06:59:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 -- 172.21.15.114:0/3346595096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 msgr2=0x55e0493fbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e0493fbe10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f52600125e0 tx=0x7f526000b850 comp rx=0 tx=0).stop 2024-09-21T06:59:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 -- 172.21.15.114:0/3346595096 shutdown_connections 2024-09-21T06:59:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e0493fc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e04902e2f0 0x55e0493fc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 --2- 172.21.15.114:0/3346595096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e0493fbe10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 -- 172.21.15.114:0/3346595096 >> 172.21.15.114:0/3346595096 conn(0x55e04904b300 msgr2=0x55e0493f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 -- 172.21.15.114:0/3346595096 shutdown_connections 2024-09-21T06:59:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.516+0000 7f5276be8740 1 -- 172.21.15.114:0/3346595096 wait complete. 2024-09-21T06:59:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.517+0000 7f5276be8740 1 -- start start 2024-09-21T06:59:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.517+0000 7f5276be8740 1 Processor -- start 2024-09-21T06:59:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.522+0000 7f5276be8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e049418370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f5276be8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e0494188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f526d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e049418370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f526d000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e049418370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56874/0 (socket says 172.21.15.114:56874) 2024-09-21T06:59:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f5276be8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049418df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f5267e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e0494188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f5267e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e0494188b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50764/0 (socket says 172.21.15.114:50764) 2024-09-21T06:59:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.523+0000 7f5267e00640 1 -- 172.21.15.114:0/2653787752 learned_addr learned my addr 172.21.15.114:0/2653787752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.524+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e049298ab0 con 0x55e04902e2f0 2024-09-21T06:59:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.524+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e049307570 con 0x55e0490571b0 2024-09-21T06:59:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.524+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e04941b610 con 0x55e0490364d0 2024-09-21T06:59:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.524+0000 7f5276be8740 10 client.0 _ll_register_callbacks cb 0x55e049316100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.524+0000 7f526da00640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049418df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.524+0000 7f5267e00640 1 -- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 msgr2=0x55e049418df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f5267e00640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049418df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f5267e00640 1 -- 172.21.15.114:0/2653787752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 msgr2=0x55e049418370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f5267e00640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e049418370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f5267e00640 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0490c2790 con 0x55e0490364d0 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f526da00640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049418df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f5267e00640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e0494188b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f52540cc400 tx=0x7f52540cbe80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f399495ddb013d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.525+0000 7f524ea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.526+0000 7f524ea00640 1 -- 172.21.15.114:0/2653787752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540ccb70 con 0x55e0490364d0 2024-09-21T06:59:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.526+0000 7f524ea00640 1 -- 172.21.15.114:0/2653787752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f52540d5a00 con 0x55e0490364d0 2024-09-21T06:59:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.526+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52600043f0 con 0x55e0490364d0 2024-09-21T06:59:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.527+0000 7f524ea00640 1 -- 172.21.15.114:0/2653787752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f52540d0f50 con 0x55e0490364d0 2024-09-21T06:59:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.528+0000 7f5276be8740 10 client.26965 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.528+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e04913b620 con 0x55e0490364d0 2024-09-21T06:59:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f524ea00640 1 -- 172.21.15.114:0/2653787752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52600043f0 con 0x55e0490364d0 2024-09-21T06:59:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f524ea00640 1 -- 172.21.15.114:0/2653787752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f52540cb6d0 con 0x55e0490364d0 2024-09-21T06:59:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f5276be8740 10 client.26965 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f5276be8740 10 client.26965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f5276be8740 20 client.26965 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f5276be8740 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e049015d80 0x55e04942a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f5276be8740 4 client.26965 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e049373600 con 0x55e049015d80 2024-09-21T06:59:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.529+0000 7f526d000640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e049015d80 0x55e04942a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.530+0000 7f524ea00640 10 client.26965 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.530+0000 7f526d000640 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e049015d80 0x55e04942a530 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a3c36bc459253b server_cookie=1cfbac902a0e32a7 in_seq=0 out_seq=0 2024-09-21T06:59:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.531+0000 7f524ea00640 1 -- 172.21.15.114:0/2653787752 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e049373600 con 0x55e049015d80 2024-09-21T06:59:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.532+0000 7f524ea00640 10 client.26965 handle_command_reply: tid=0 2024-09-21T06:59:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.533+0000 7f5276be8740 1 client.26965 shutdown 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.534+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 msgr2=0x55e0494188b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.534+0000 7f5276be8740 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e0494188b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f52540cc400 tx=0x7f52540cbe80 comp rx=0 tx=0).stop 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.534+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 shutdown_connections 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049418df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e049015d80 0x55e04942a530 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e0494188b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 --2- 172.21.15.114:0/2653787752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e049418370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 >> 172.21.15.114:0/2653787752 conn(0x55e04904b300 msgr2=0x55e0493fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f524ea00640 0 client.26965 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 shutdown_connections 2024-09-21T06:59:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.535+0000 7f5276be8740 1 -- 172.21.15.114:0/2653787752 wait complete. 2024-09-21T06:59:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.536+0000 7f5276be8740 20 client.26965 trim_cache size 0 max 16384 2024-09-21T06:59:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.580+0000 7f5276be8740 1 Processor -- start 2024-09-21T06:59:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.581+0000 7f5276be8740 1 -- start start 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.581+0000 7f5276be8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e04902e2f0 0x55e049413860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5276be8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e049415540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5276be8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490571b0 0x55e049415a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5276be8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b7010 con 0x55e0490364d0 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5276be8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b8c70 con 0x55e04902e2f0 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5276be8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b76e0 con 0x55e0490571b0 2024-09-21T06:59:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5267e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e049415540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.582+0000 7f5267e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e049415540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56884/0 (socket says 172.21.15.114:56884) 2024-09-21T06:59:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.583+0000 7f5267e00640 1 -- 172.21.15.114:0/2675241457 learned_addr learned my addr 172.21.15.114:0/2675241457 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.583+0000 7f526d000640 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e04902e2f0 0x55e049413860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.583+0000 7f526da00640 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490571b0 0x55e049415a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.583+0000 7f5267e00640 1 -- 172.21.15.114:0/2675241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490571b0 msgr2=0x55e049415a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.583+0000 7f5267e00640 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490571b0 0x55e049415a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.583+0000 7f5267e00640 1 -- 172.21.15.114:0/2675241457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e04902e2f0 msgr2=0x55e049413860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.584+0000 7f5267e00640 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e04902e2f0 0x55e049413860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.584+0000 7f5267e00640 1 -- 172.21.15.114:0/2675241457 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0493fec70 con 0x55e0490364d0 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.584+0000 7f526da00640 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490571b0 0x55e049415a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.585+0000 7f5267e00640 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e049415540 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f52540cc170 tx=0x7f52540cbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27c12e7d58d9775 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.586+0000 7f5259600640 1 -- 172.21.15.114:0/2675241457 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540cc240 con 0x55e0490364d0 2024-09-21T06:59:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.586+0000 7f5259600640 1 -- 172.21.15.114:0/2675241457 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52540ccb70 con 0x55e0490364d0 2024-09-21T06:59:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.586+0000 7f5259600640 1 -- 172.21.15.114:0/2675241457 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52540d5a00 con 0x55e0490364d0 2024-09-21T06:59:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.588+0000 7f5276be8740 1 -- 172.21.15.114:0/2675241457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 msgr2=0x55e049415540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.588+0000 7f5276be8740 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e049415540 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f52540cc170 tx=0x7f52540cbef0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.589+0000 7f5276be8740 1 -- 172.21.15.114:0/2675241457 shutdown_connections 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.589+0000 7f5276be8740 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490571b0 0x55e049415a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.589+0000 7f5276be8740 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e0490364d0 0x55e049415540 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.589+0000 7f5276be8740 1 --2- 172.21.15.114:0/2675241457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e04902e2f0 0x55e049413860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.589+0000 7f5276be8740 1 -- 172.21.15.114:0/2675241457 >> 172.21.15.114:0/2675241457 conn(0x55e04904b300 msgr2=0x55e049413150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.590+0000 7f5276be8740 1 -- 172.21.15.114:0/2675241457 shutdown_connections 2024-09-21T06:59:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.590+0000 7f5276be8740 1 -- 172.21.15.114:0/2675241457 wait complete. 2024-09-21T06:59:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.591+0000 7f5276be8740 1 -- start start 2024-09-21T06:59:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.592+0000 7f5276be8740 1 Processor -- start 2024-09-21T06:59:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.593+0000 7f5276be8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e04941eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.593+0000 7f5276be8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e04941f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.593+0000 7f5276be8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049493f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.594+0000 7f5276be8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b84e0 con 0x55e04902e2f0 2024-09-21T06:59:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.594+0000 7f5276be8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b7db0 con 0x55e0490571b0 2024-09-21T06:59:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.594+0000 7f5276be8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52681265b0 con 0x55e0490364d0 2024-09-21T06:59:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.594+0000 7f5276be8740 10 client.0 _ll_register_callbacks cb 0x7f5254005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.594+0000 7f526da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049493f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.595+0000 7f5267e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e04941f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.595+0000 7f526da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049493f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54914/0 (socket says 172.21.15.114:54914) 2024-09-21T06:59:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.595+0000 7f526da00640 1 -- 172.21.15.114:0/1143241266 learned_addr learned my addr 172.21.15.114:0/1143241266 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.595+0000 7f526d000640 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e04941eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.596+0000 7f526da00640 1 -- 172.21.15.114:0/1143241266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 msgr2=0x55e04941f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.596+0000 7f526da00640 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e04941f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.596+0000 7f526da00640 1 -- 172.21.15.114:0/1143241266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 msgr2=0x55e04941eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.596+0000 7f526da00640 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e04941eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.596+0000 7f526da00640 1 -- 172.21.15.114:0/1143241266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e04941f870 con 0x55e0490571b0 2024-09-21T06:59:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.597+0000 7f526da00640 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049493f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52600105e0 tx=0x7f5260005820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2393d579eef98969 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.597+0000 7f524ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.598+0000 7f524ea00640 1 -- 172.21.15.114:0/1143241266 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260010a90 con 0x55e0490571b0 2024-09-21T06:59:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.598+0000 7f524ea00640 1 -- 172.21.15.114:0/1143241266 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5260010c30 con 0x55e0490571b0 2024-09-21T06:59:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.598+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52540ccb70 con 0x55e0490571b0 2024-09-21T06:59:39.602 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.600+0000 7f524ea00640 1 -- 172.21.15.114:0/1143241266 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5260010e30 con 0x55e0490571b0 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.600+0000 7f5276be8740 10 client.27014 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.600+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e049495370 con 0x55e0490571b0 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.600+0000 7f524ea00640 1 -- 172.21.15.114:0/1143241266 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52540ccb70 con 0x55e0490571b0 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f524ea00640 1 -- 172.21.15.114:0/1143241266 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f526001c020 con 0x55e0490571b0 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f5276be8740 10 client.27014 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f5276be8740 10 client.27014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f5276be8740 20 client.27014 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f5276be8740 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e048feb1c0 0x55e049497a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f5276be8740 4 client.27014 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:59:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e04936f0b0 con 0x55e048feb1c0 2024-09-21T06:59:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.601+0000 7f526d000640 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e048feb1c0 0x55e049497a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.602+0000 7f524ea00640 10 client.27014 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.604+0000 7f526d000640 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e048feb1c0 0x55e049497a10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572c3d0a5607648d server_cookie=bad40b4483121c21 in_seq=0 out_seq=0 2024-09-21T06:59:39.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:39 smithi183 ceph-mon[34219]: pgmap v561: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.7 MiB/s wr, 390 op/s 2024-09-21T06:59:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.618+0000 7f524ea00640 1 -- 172.21.15.114:0/1143241266 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e04936f0b0 con 0x55e048feb1c0 2024-09-21T06:59:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.618+0000 7f524ea00640 10 client.27014 handle_command_reply: tid=0 2024-09-21T06:59:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.618+0000 7f5276be8740 1 client.27014 shutdown 2024-09-21T06:59:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 msgr2=0x55e049493f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049493f00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52600105e0 tx=0x7f5260005820 comp rx=0 tx=0).stop 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 shutdown_connections 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e0490571b0 0x55e049493f00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e048feb1c0 0x55e049497a10 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0490364d0 0x55e04941f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 --2- 172.21.15.114:0/1143241266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e04902e2f0 0x55e04941eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 >> 172.21.15.114:0/1143241266 conn(0x55e04904b300 msgr2=0x55e0493fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.619+0000 7f524ea00640 0 client.27014 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.620+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 shutdown_connections 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.620+0000 7f5276be8740 1 -- 172.21.15.114:0/1143241266 wait complete. 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.620+0000 7f5276be8740 20 client.27014 trim_cache size 0 max 16384 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:59:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:59:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.623+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f52500b65a0 msgr2=0x7f52500b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.623+0000 7f526e400640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f52500b65a0 0x7f52500b8ab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5268116ee0 tx=0x7f525c0094f0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.623+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 msgr2=0x7f52681149c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.623+0000 7f526e400640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 0x7f52681149c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5254006010 tx=0x7f525402bbf0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 shutdown_connections 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f526810d390 0x7f52681b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f52500b65a0 0x7f52500b8ab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526810ca40 0x7f52681149c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 --2- 172.21.15.114:0/3106582450 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f526810a660 0x7f5268114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 >> 172.21.15.114:0/3106582450 conn(0x7f5268070c00 msgr2=0x7f526810ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 shutdown_connections 2024-09-21T06:59:39.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:39.624+0000 7f526e400640 1 -- 172.21.15.114:0/3106582450 wait complete. 2024-09-21T06:59:39.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:39 smithi193 ceph-mon[34601]: pgmap v561: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.7 MiB/s wr, 390 op/s 2024-09-21T06:59:39.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.662+0000 7f798fa00640 1 -- 172.21.15.114:0/1797558411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 msgr2=0x7f7988110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.662+0000 7f798fa00640 1 --2- 172.21.15.114:0/1797558411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 0x7f7988110620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7980009900 tx=0x7f798002a400 comp rx=0 tx=0).stop 2024-09-21T06:59:39.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.663+0000 7f798fa00640 1 -- 172.21.15.114:0/1797558411 shutdown_connections 2024-09-21T06:59:39.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.663+0000 7f798fa00640 1 --2- 172.21.15.114:0/1797558411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 0x7f7988110620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.663+0000 7f798fa00640 1 --2- 172.21.15.114:0/1797558411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f79881098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.663+0000 7f798fa00640 1 --2- 172.21.15.114:0/1797558411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7988101b60 0x7f7988101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.663+0000 7f798fa00640 1 -- 172.21.15.114:0/1797558411 >> 172.21.15.114:0/1797558411 conn(0x7f7988077800 msgr2=0x7f7988077c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:39.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.664+0000 7f798fa00640 1 -- 172.21.15.114:0/1797558411 shutdown_connections 2024-09-21T06:59:39.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.664+0000 7f798fa00640 1 -- 172.21.15.114:0/1797558411 wait complete. 2024-09-21T06:59:39.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.665+0000 7f798fa00640 1 Processor -- start 2024-09-21T06:59:39.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.666+0000 7f798fa00640 1 -- start start 2024-09-21T06:59:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f798fa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7988101b60 0x7f7988199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f798fa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f7988199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f798fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 0x7f7988199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f798fa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7988113b60 con 0x7f7988101b60 2024-09-21T06:59:39.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f797fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f7988199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f797fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f7988199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50790/0 (socket says 172.21.15.114:50790) 2024-09-21T06:59:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.668+0000 7f797fe00640 1 -- 172.21.15.114:0/551339236 learned_addr learned my addr 172.21.15.114:0/551339236 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.669+0000 7f798da00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 0x7f7988199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.669+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79881139e0 con 0x7f7988109e10 2024-09-21T06:59:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.669+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7988113ce0 con 0x7f79881094c0 2024-09-21T06:59:39.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.669+0000 7f798d000640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7988101b60 0x7f7988199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.670+0000 7f797fe00640 1 -- 172.21.15.114:0/551339236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 msgr2=0x7f7988199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.670+0000 7f797fe00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 0x7f7988199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.670+0000 7f797fe00640 1 -- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7988101b60 msgr2=0x7f7988199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:39.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.670+0000 7f797fe00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7988101b60 0x7f7988199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:39.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.670+0000 7f797fe00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79881a0320 con 0x7f79881094c0 2024-09-21T06:59:39.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.672+0000 7f797fe00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f7988199570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f797000e9f0 tx=0x7f797000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.673+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000ccc0 con 0x7f79881094c0 2024-09-21T06:59:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.673+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79700108e0 con 0x7f79881094c0 2024-09-21T06:59:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.674+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700188e0 con 0x7f79881094c0 2024-09-21T06:59:39.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.674+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79881a0610 con 0x7f79881094c0 2024-09-21T06:59:39.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.674+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a0c20 con 0x7f79881094c0 2024-09-21T06:59:39.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.676+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7988101f70 con 0x7f79881094c0 2024-09-21T06:59:39.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.682+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7970010430 con 0x7f79881094c0 2024-09-21T06:59:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.685+0000 7f7986000640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f796c0b6700 0x7f796c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.685+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f79700e8290 con 0x7f79881094c0 2024-09-21T06:59:39.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.685+0000 7f798d000640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f796c0b6700 0x7f796c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:39.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.688+0000 7f798d000640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f796c0b6700 0x7f796c0b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7988110610 tx=0x7f79780094f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:39.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:39.691+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f79700ae470 con 0x7f79881094c0 2024-09-21T06:59:40.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.114+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798810fec0 con 0x7f79881094c0 2024-09-21T06:59:40.121 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:59:40.121 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"btime":"2024-09-21T06:58:51:043259+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:58:50.613377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.115+0000 7f7986000640 1 -- 172.21.15.114:0/551339236 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11999 (secure 0 0 0) 0x7f79700b2850 con 0x7f79881094c0 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.118+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f796c0b6700 msgr2=0x7f796c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.118+0000 7f798fa00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f796c0b6700 0x7f796c0b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7988110610 tx=0x7f79780094f0 comp rx=0 tx=0).stop 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.118+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 msgr2=0x7f7988199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.118+0000 7f798fa00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f7988199570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f797000e9f0 tx=0x7f797000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 shutdown_connections 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f796c0b6700 0x7f796c0b8c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7988109e10 0x7f7988199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:40.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79881094c0 0x7f7988199570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:40.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 --2- 172.21.15.114:0/551339236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7988101b60 0x7f7988199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:40.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 >> 172.21.15.114:0/551339236 conn(0x7f7988077800 msgr2=0x7f79881073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:40.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.119+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 shutdown_connections 2024-09-21T06:59:40.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:40.120+0000 7f798fa00640 1 -- 172.21.15.114:0/551339236 wait complete. 2024-09-21T06:59:40.554 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:40 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/551339236' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:40.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:40 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/551339236' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:40.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:40 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/551339236' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:41.111 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:59:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:41 smithi183 ceph-mon[34219]: pgmap v562: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.8 MiB/s wr, 355 op/s 2024-09-21T06:59:41.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:41 smithi193 ceph-mon[34601]: pgmap v562: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.8 MiB/s wr, 355 op/s 2024-09-21T06:59:41.776 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:41.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:41 smithi114 ceph-mon[30654]: pgmap v562: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 6.8 MiB/s wr, 355 op/s 2024-09-21T06:59:42.517 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:42 smithi114 ceph-mon[30654]: pgmap v563: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.8 MiB/s wr, 356 op/s 2024-09-21T06:59:42.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:42 smithi183 ceph-mon[34219]: pgmap v563: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.8 MiB/s wr, 356 op/s 2024-09-21T06:59:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:42 smithi193 ceph-mon[34601]: pgmap v563: 129 pgs: 129 active+clean; 2.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 6.8 MiB/s wr, 356 op/s 2024-09-21T06:59:42.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.704+0000 7f2fee800640 1 -- 172.21.15.114:0/2274362941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe81094d0 msgr2=0x7f2fe81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.704+0000 7f2fee800640 1 --2- 172.21.15.114:0/2274362941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe81094d0 0x7f2fe81098e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009900 tx=0x7f2fd402a410 comp rx=0 tx=0).stop 2024-09-21T06:59:42.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 -- 172.21.15.114:0/2274362941 shutdown_connections 2024-09-21T06:59:42.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 --2- 172.21.15.114:0/2274362941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8109e20 0x7f2fe8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 --2- 172.21.15.114:0/2274362941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe81094d0 0x7f2fe81098e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 --2- 172.21.15.114:0/2274362941 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fe8103b90 0x7f2fe8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 -- 172.21.15.114:0/2274362941 >> 172.21.15.114:0/2274362941 conn(0x7f2fe80ff5e0 msgr2=0x7f2fe8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:42.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 -- 172.21.15.114:0/2274362941 shutdown_connections 2024-09-21T06:59:42.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.705+0000 7f2fee800640 1 -- 172.21.15.114:0/2274362941 wait complete. 2024-09-21T06:59:42.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.707+0000 7f2fee800640 1 Processor -- start 2024-09-21T06:59:42.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.707+0000 7f2fee800640 1 -- start start 2024-09-21T06:59:42.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 0x7f2fe819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fee800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fe81094d0 0x7f2fe819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fee800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe8109e20 0x7f2fe81a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fee800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8113b30 con 0x7f2fe81094d0 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fee800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81139b0 con 0x7f2fe8103b90 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fee800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8113cb0 con 0x7f2fe8109e20 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fe7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 0x7f2fe819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fe7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 0x7f2fe819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54948/0 (socket says 172.21.15.114:54948) 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.708+0000 7f2fe7400640 1 -- 172.21.15.114:0/2672057720 learned_addr learned my addr 172.21.15.114:0/2672057720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.710+0000 7f2fe7400640 1 -- 172.21.15.114:0/2672057720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe8109e20 msgr2=0x7f2fe81a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.710+0000 7f2fe7400640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe8109e20 0x7f2fe81a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.710+0000 7f2fe7400640 1 -- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fe81094d0 msgr2=0x7f2fe819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.710+0000 7f2fe7400640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fe81094d0 0x7f2fe819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.710+0000 7f2fe7400640 1 -- 172.21.15.114:0/2672057720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe81a2630 con 0x7f2fe8103b90 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.711+0000 7f2fe7400640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 0x7f2fe819c6c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80058c0 tx=0x7f2fd8007590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.711+0000 7f2fdfe00640 1 -- 172.21.15.114:0/2672057720 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8007d90 con 0x7f2fe8103b90 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.712+0000 7f2fdfe00640 1 -- 172.21.15.114:0/2672057720 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd8013020 con 0x7f2fe8103b90 2024-09-21T06:59:42.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.712+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe81a2920 con 0x7f2fe8103b90 2024-09-21T06:59:42.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.712+0000 7f2fdfe00640 1 -- 172.21.15.114:0/2672057720 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd801e680 con 0x7f2fe8103b90 2024-09-21T06:59:42.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.712+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe810a500 con 0x7f2fe8103b90 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.717+0000 7f2ff5499740 1 Processor -- start 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.717+0000 7f2ff5499740 1 -- start start 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.717+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd3f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.718+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.718+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd3f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.718+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd3f5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54954/0 (socket says 172.21.15.114:54954) 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.718+0000 7f2fe7e00640 1 -- 172.21.15.114:0/752448213 learned_addr learned my addr 172.21.15.114:0/752448213 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:42.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.718+0000 7f2ff5499740 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 0x557bfd3f6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.718+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557bfd36cfb0 con 0x557bfd02e390 2024-09-21T06:59:42.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.719+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557bfd35c3e0 con 0x557bfd04f070 2024-09-21T06:59:42.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.720+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557bfd3601f0 con 0x557bfd0261b0 2024-09-21T06:59:42.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.721+0000 7f2fe7e00640 1 -- 172.21.15.114:0/752448213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 msgr2=0x557bfd3f62e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:59:42.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.721+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f62e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.721+0000 7f2fe7e00640 1 -- 172.21.15.114:0/752448213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 msgr2=0x557bfd3f6820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:42.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.721+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 0x557bfd3f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.722+0000 7f2fe7e00640 1 -- 172.21.15.114:0/752448213 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfd326cd0 con 0x557bfd04f070 2024-09-21T06:59:42.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.728+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd3f5da0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009ea0 tx=0x7f2fe0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3816f9e778867040 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.728+0000 7f2fdcc00640 1 -- 172.21.15.114:0/752448213 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004d20 con 0x557bfd04f070 2024-09-21T06:59:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.728+0000 7f2fdcc00640 1 -- 172.21.15.114:0/752448213 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe000ee60 con 0x557bfd04f070 2024-09-21T06:59:42.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.728+0000 7f2fdcc00640 1 -- 172.21.15.114:0/752448213 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0010850 con 0x557bfd04f070 2024-09-21T06:59:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.729+0000 7f2fdfe00640 1 -- 172.21.15.114:0/2672057720 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f2fd8012a60 con 0x7f2fe8103b90 2024-09-21T06:59:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 msgr2=0x557bfd3f5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd3f5da0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009ea0 tx=0x7f2fe0009f90 comp rx=0 tx=0).stop 2024-09-21T06:59:42.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 shutdown_connections 2024-09-21T06:59:42.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 0x557bfd3f6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 --2- 172.21.15.114:0/752448213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd3f5da0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.730+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 >> 172.21.15.114:0/752448213 conn(0x557bfd043460 msgr2=0x557bfd3f3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:42.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.732+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 shutdown_connections 2024-09-21T06:59:42.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.733+0000 7f2ff5499740 1 -- 172.21.15.114:0/752448213 wait complete. 2024-09-21T06:59:42.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.734+0000 7f2ff5499740 1 -- start start 2024-09-21T06:59:42.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.734+0000 7f2ff5499740 1 Processor -- start 2024-09-21T06:59:42.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.735+0000 7f2fdfe00640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2fcc0b6610 0x7f2fcc0b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.735+0000 7f2fdfe00640 1 -- 172.21.15.114:0/2672057720 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2fd80ef610 con 0x7f2fe8103b90 2024-09-21T06:59:42.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd0261b0 0x557bfd401490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd02e390 0x557bfd4019d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 0x557bfd40d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557bfd1bd700 con 0x557bfd0261b0 2024-09-21T06:59:42.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557bfd2f7580 con 0x557bfd02e390 2024-09-21T06:59:42.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557bfd404580 con 0x557bfd04f070 2024-09-21T06:59:42.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.738+0000 7f2ff5499740 10 client.0 _ll_register_callbacks cb 0x557bfd307470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:42.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.739+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 0x557bfd40d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.739+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 0x557bfd40d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50814/0 (socket says 172.21.15.114:50814) 2024-09-21T06:59:42.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.739+0000 7f2fe7e00640 1 -- 172.21.15.114:0/1280056598 learned_addr learned my addr 172.21.15.114:0/1280056598 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:42.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.739+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2fcc0b6610 0x7f2fcc0b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.739+0000 7f2fe6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd02e390 0x557bfd4019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7e00640 1 -- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd02e390 msgr2=0x557bfd4019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:42.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd02e390 0x557bfd4019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd0261b0 0x557bfd401490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7e00640 1 -- 172.21.15.114:0/1280056598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd0261b0 msgr2=0x557bfd401490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:42.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd0261b0 0x557bfd401490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7e00640 1 -- 172.21.15.114:0/1280056598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfd0ba650 con 0x557bfd04f070 2024-09-21T06:59:42.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fd1600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:42.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.740+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 0x557bfd40d8d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009ce0 tx=0x7f2fe0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf3c1bf9a9ba0da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:42.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.742+0000 7f2fd1600640 1 -- 172.21.15.114:0/1280056598 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0005860 con 0x557bfd04f070 2024-09-21T06:59:42.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.742+0000 7f2fd1600640 1 -- 172.21.15.114:0/1280056598 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2fe0003970 con 0x557bfd04f070 2024-09-21T06:59:42.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.742+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557bfd2f9b90 con 0x557bfd04f070 2024-09-21T06:59:42.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.742+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2fcc0b6610 0x7f2fcc0b8b20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2fe002f000 tx=0x7f2fe002b000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:42.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.743+0000 7f2fd1600640 1 -- 172.21.15.114:0/1280056598 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2fe0003c50 con 0x557bfd04f070 2024-09-21T06:59:42.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.743+0000 7f2ff5499740 10 client.26980 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.743+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557bfd314950 con 0x557bfd04f070 2024-09-21T06:59:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.744+0000 7f2fd1600640 1 -- 172.21.15.114:0/1280056598 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557bfd2f9b90 con 0x557bfd04f070 2024-09-21T06:59:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.745+0000 7f2fd1600640 1 -- 172.21.15.114:0/1280056598 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f2fe0023020 con 0x557bfd04f070 2024-09-21T06:59:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.745+0000 7f2ff5499740 10 client.26980 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:42.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.745+0000 7f2ff5499740 10 client.26980 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:59:42.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.746+0000 7f2ff5499740 20 client.26980 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:42.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.746+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfd00d9f0 0x557bfd4135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.746+0000 7f2ff5499740 4 client.26980 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:42.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.746+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bfd36ef00 con 0x557bfd00d9f0 2024-09-21T06:59:42.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.746+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfd00d9f0 0x557bfd4135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.747+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfd00d9f0 0x557bfd4135c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f832bff3b70e585 server_cookie=4cc3d3af65ef5492 in_seq=0 out_seq=0 2024-09-21T06:59:42.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.747+0000 7f2fd1600640 10 client.26980 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:59:42.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.748+0000 7f2fd1600640 1 -- 172.21.15.114:0/1280056598 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557bfd36ef00 con 0x557bfd00d9f0 2024-09-21T06:59:42.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.748+0000 7f2fd1600640 10 client.26980 handle_command_reply: tid=0 2024-09-21T06:59:42.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.748+0000 7f2ff5499740 1 client.26980 shutdown 2024-09-21T06:59:42.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.748+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 msgr2=0x557bfd40d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.748+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 0x557bfd40d8d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009ce0 tx=0x7f2fe0000f40 comp rx=0 tx=0).stop 2024-09-21T06:59:42.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 shutdown_connections 2024-09-21T06:59:42.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd04f070 0x557bfd40d8d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfd00d9f0 0x557bfd4135c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd02e390 0x557bfd4019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1280056598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd0261b0 0x557bfd401490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 >> 172.21.15.114:0/1280056598 conn(0x557bfd043460 msgr2=0x557bfd3f4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.749+0000 7f2fd1600640 0 client.26980 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:59:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.751+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 shutdown_connections 2024-09-21T06:59:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.751+0000 7f2ff5499740 1 -- 172.21.15.114:0/1280056598 wait complete. 2024-09-21T06:59:42.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.751+0000 7f2ff5499740 20 client.26980 trim_cache size 0 max 16384 2024-09-21T06:59:42.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 Processor -- start 2024-09-21T06:59:42.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 -- start start 2024-09-21T06:59:42.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 0x557bfd40d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd4029b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81aed50 con 0x557bfd02e390 2024-09-21T06:59:42.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81b09b0 con 0x557bfd04f070 2024-09-21T06:59:42.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.779+0000 7f2ff5499740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81af420 con 0x557bfd0261b0 2024-09-21T06:59:42.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50818/0 (socket says 172.21.15.114:50818) 2024-09-21T06:59:42.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 -- 172.21.15.114:0/1344402496 learned_addr learned my addr 172.21.15.114:0/1344402496 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:42.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd4029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 -- 172.21.15.114:0/1344402496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 msgr2=0x557bfd4029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:42.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd4029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 -- 172.21.15.114:0/1344402496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 msgr2=0x557bfd40d840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:42.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 0x557bfd40d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.780+0000 7f2fe7400640 1 -- 172.21.15.114:0/1344402496 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfd3ff990 con 0x557bfd0261b0 2024-09-21T06:59:42.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.781+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f6490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80ef930 tx=0x7f2fd8013c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50f42cc0d97d0c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.781+0000 7f2fd3e00640 1 -- 172.21.15.114:0/1344402496 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80b9bd0 con 0x557bfd0261b0 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.781+0000 7f2fd3e00640 1 -- 172.21.15.114:0/1344402496 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd80beb10 con 0x557bfd0261b0 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.781+0000 7f2fd3e00640 1 -- 172.21.15.114:0/1344402496 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80a0180 con 0x557bfd0261b0 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 -- 172.21.15.114:0/1344402496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 msgr2=0x557bfd3f6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f6490 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80ef930 tx=0x7f2fd8013c10 comp rx=0 tx=0).stop 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 -- 172.21.15.114:0/1344402496 shutdown_connections 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd04f070 0x557bfd4029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd02e390 0x557bfd40d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1344402496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd0261b0 0x557bfd3f6490 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 -- 172.21.15.114:0/1344402496 >> 172.21.15.114:0/1344402496 conn(0x557bfd043460 msgr2=0x557bfd3f5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.783+0000 7f2ff5499740 1 -- 172.21.15.114:0/1344402496 shutdown_connections 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.784+0000 7f2ff5499740 1 -- 172.21.15.114:0/1344402496 wait complete. 2024-09-21T06:59:42.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.784+0000 7f2ff5499740 1 -- start start 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.785+0000 7f2ff5499740 1 Processor -- start 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 0x557bfd3fee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd02e390 0x557bfd3fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd04f070 0x557bfd3fa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2fe7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 0x557bfd3fee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2fe7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 0x557bfd3fee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54996/0 (socket says 172.21.15.114:54996) 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81b0220 con 0x557bfd04f070 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81afaf0 con 0x557bfd0261b0 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81acdb0 con 0x557bfd02e390 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2ff5499740 10 client.0 _ll_register_callbacks cb 0x7f2fe0011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2fe6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd02e390 0x557bfd3fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.786+0000 7f2fe7400640 1 -- 172.21.15.114:0/1800081623 learned_addr learned my addr 172.21.15.114:0/1800081623 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.788+0000 7f2fe7e00640 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd04f070 0x557bfd3fa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.788+0000 7f2fe7400640 1 -- 172.21.15.114:0/1800081623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd02e390 msgr2=0x557bfd3fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.788+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd02e390 0x557bfd3fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.788+0000 7f2fe7400640 1 -- 172.21.15.114:0/1800081623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd04f070 msgr2=0x557bfd3fa890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.788+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd04f070 0x557bfd3fa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.788+0000 7f2fe7400640 1 -- 172.21.15.114:0/1800081623 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfd3fb020 con 0x557bfd0261b0 2024-09-21T06:59:42.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.790+0000 7f2fe7400640 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 0x557bfd3fee60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80b4df0 tx=0x7f2fd8013400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df2cc5ec6c3cb146 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:42.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.790+0000 7f2fd1600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:42.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.791+0000 7f2fd1600640 1 -- 172.21.15.114:0/1800081623 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd80b1360 con 0x557bfd0261b0 2024-09-21T06:59:42.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.791+0000 7f2fd1600640 1 -- 172.21.15.114:0/1800081623 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f2fd80134d0 con 0x557bfd0261b0 2024-09-21T06:59:42.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.791+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fd80beb10 con 0x557bfd0261b0 2024-09-21T06:59:42.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.792+0000 7f2fd1600640 1 -- 172.21.15.114:0/1800081623 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2fd80b1360 con 0x557bfd0261b0 2024-09-21T06:59:42.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.792+0000 7f2ff5499740 10 client.27038 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:42.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.792+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557bfd48ee00 con 0x557bfd0261b0 2024-09-21T06:59:42.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.792+0000 7f2fd1600640 1 -- 172.21.15.114:0/1800081623 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fd80beb10 con 0x557bfd0261b0 2024-09-21T06:59:42.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2fd1600640 1 -- 172.21.15.114:0/1800081623 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f2fd80b0a40 con 0x557bfd0261b0 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2ff5499740 10 client.27038 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2ff5499740 10 client.27038 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2ff5499740 20 client.27038 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfcfe2da0 0x557bfd4915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2ff5499740 4 client.27038 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bfd36a9b0 con 0x557bfcfe2da0 2024-09-21T06:59:42.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.793+0000 7f2fe6a00640 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfcfe2da0 0x557bfd4915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:42.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.794+0000 7f2fe6a00640 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfcfe2da0 0x557bfd4915c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f30e9186b6b3bd1 server_cookie=ba4bc1e29e20844 in_seq=0 out_seq=0 2024-09-21T06:59:42.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.794+0000 7f2fd1600640 10 client.27038 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T06:59:42.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2fd1600640 1 -- 172.21.15.114:0/1800081623 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bfd36a9b0 con 0x557bfcfe2da0 2024-09-21T06:59:42.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2fd1600640 10 client.27038 handle_command_reply: tid=0 2024-09-21T06:59:42.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2ff5499740 1 client.27038 shutdown 2024-09-21T06:59:42.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 msgr2=0x557bfd3fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 0x557bfd3fee60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80b4df0 tx=0x7f2fd8013400 comp rx=0 tx=0).stop 2024-09-21T06:59:42.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 shutdown_connections 2024-09-21T06:59:42.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.795+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557bfd04f070 0x557bfd3fa890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557bfd02e390 0x557bfd3fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557bfcfe2da0 0x557bfd4915c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2ff5499740 1 --2- 172.21.15.114:0/1800081623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557bfd0261b0 0x557bfd3fee60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 >> 172.21.15.114:0/1800081623 conn(0x557bfd043460 msgr2=0x557bfd402920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:42.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2fd1600640 0 client.27038 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T06:59:42.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 shutdown_connections 2024-09-21T06:59:42.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.796+0000 7f2ff5499740 1 -- 172.21.15.114:0/1800081623 wait complete. 2024-09-21T06:59:42.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.798+0000 7f2ff5499740 20 client.27038 trim_cache size 0 max 16384 2024-09-21T06:59:42.802 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:59:42.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2fcc0b6610 msgr2=0x7f2fcc0b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2fcc0b6610 0x7f2fcc0b8b20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2fe002f000 tx=0x7f2fe002b000 comp rx=0 tx=0).stop 2024-09-21T06:59:42.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 msgr2=0x7f2fe819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:42.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 0x7f2fe819c6c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80058c0 tx=0x7f2fd8007590 comp rx=0 tx=0).stop 2024-09-21T06:59:42.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 shutdown_connections 2024-09-21T06:59:42.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fe8109e20 0x7f2fe81a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2fe81094d0 0x7f2fe819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2fcc0b6610 0x7f2fcc0b8b20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 --2- 172.21.15.114:0/2672057720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fe8103b90 0x7f2fe819c6c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:42.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 >> 172.21.15.114:0/2672057720 conn(0x7f2fe80ff5e0 msgr2=0x7f2fe8108b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:42.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.800+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 shutdown_connections 2024-09-21T06:59:42.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:42.801+0000 7f2fee800640 1 -- 172.21.15.114:0/2672057720 wait complete. 2024-09-21T06:59:42.835 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 305.0 s, 715.2 tps, lat 142.610 ms stddev 176.943 2024-09-21T06:59:43.940 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:59:44.590 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:45.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:44 smithi114 ceph-mon[30654]: pgmap v564: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 553 op/s 2024-09-21T06:59:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:44 smithi183 ceph-mon[34219]: pgmap v564: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 553 op/s 2024-09-21T06:59:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:44 smithi193 ceph-mon[34601]: pgmap v564: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 553 op/s 2024-09-21T06:59:46.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.390+0000 7f9b7fe00640 1 -- 172.21.15.114:0/4267222276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801039b0 msgr2=0x7f9b80103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.390+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/4267222276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801039b0 0x7f9b80103dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009900 tx=0x7f9b6c02a410 comp rx=0 tx=0).stop 2024-09-21T06:59:46.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 -- 172.21.15.114:0/4267222276 shutdown_connections 2024-09-21T06:59:46.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/4267222276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b80109c40 0x7f9b80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/4267222276 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b801092f0 0x7f9b80109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/4267222276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801039b0 0x7f9b80103dc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 -- 172.21.15.114:0/4267222276 >> 172.21.15.114:0/4267222276 conn(0x7f9b800ff440 msgr2=0x7f9b80101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:46.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 -- 172.21.15.114:0/4267222276 shutdown_connections 2024-09-21T06:59:46.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.391+0000 7f9b7fe00640 1 -- 172.21.15.114:0/4267222276 wait complete. 2024-09-21T06:59:46.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.395+0000 7f9b7fe00640 1 Processor -- start 2024-09-21T06:59:46.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.395+0000 7f9b7fe00640 1 -- start start 2024-09-21T06:59:46.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.396+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 0x7f9b8019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.396+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801092f0 0x7f9b8019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.397+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b80109c40 0x7f9b801a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.397+0000 7f9b7fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801139b0 con 0x7f9b80109c40 2024-09-21T06:59:46.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.397+0000 7f9b7fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b80113830 con 0x7f9b801092f0 2024-09-21T06:59:46.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.397+0000 7f9b7fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b80113b30 con 0x7f9b801039b0 2024-09-21T06:59:46.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.398+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 0x7f9b8019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.398+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 0x7f9b8019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50080/0 (socket says 172.21.15.114:50080) 2024-09-21T06:59:46.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.398+0000 7f9b7ea00640 1 -- 172.21.15.114:0/860546457 learned_addr learned my addr 172.21.15.114:0/860546457 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:46.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.398+0000 7f9b7ea00640 1 -- 172.21.15.114:0/860546457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801092f0 msgr2=0x7f9b8019cb60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:59:46.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801092f0 0x7f9b8019cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7ea00640 1 -- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b80109c40 msgr2=0x7f9b801a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7f400640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b80109c40 0x7f9b801a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b80109c40 0x7f9b801a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7ea00640 1 -- 172.21.15.114:0/860546457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b801a2530 con 0x7f9b801039b0 2024-09-21T06:59:46.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7f400640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b80109c40 0x7f9b801a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:46.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.399+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 0x7f9b8019c5c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005ab0 tx=0x7f9b6c003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:46.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.400+0000 7f9b77400640 1 -- 172.21.15.114:0/860546457 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c03f070 con 0x7f9b801039b0 2024-09-21T06:59:46.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.400+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b801a27c0 con 0x7f9b801039b0 2024-09-21T06:59:46.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.400+0000 7f9b77400640 1 -- 172.21.15.114:0/860546457 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6c031d60 con 0x7f9b801039b0 2024-09-21T06:59:46.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.400+0000 7f9b77400640 1 -- 172.21.15.114:0/860546457 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c035030 con 0x7f9b801039b0 2024-09-21T06:59:46.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.400+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b801a8f80 con 0x7f9b801039b0 2024-09-21T06:59:46.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.415+0000 7f9b8c63b740 1 Processor -- start 2024-09-21T06:59:46.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.415+0000 7f9b77400640 1 -- 172.21.15.114:0/860546457 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9b6c042070 con 0x7f9b801039b0 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.416+0000 7f9b8c63b740 1 -- start start 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b578910780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5785381b0 0x55b578910cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578911200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b57887efb0 con 0x55b578561070 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b57886e3e0 con 0x55b578540390 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b5788721f0 con 0x55b5785381b0 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578911200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578911200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42904/0 (socket says 172.21.15.114:42904) 2024-09-21T06:59:46.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.417+0000 7f9b7e000640 1 -- 172.21.15.114:0/1006835849 learned_addr learned my addr 172.21.15.114:0/1006835849 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b7e000640 1 -- 172.21.15.114:0/1006835849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5785381b0 msgr2=0x55b578910cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b7e000640 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5785381b0 0x55b578910cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b7e000640 1 -- 172.21.15.114:0/1006835849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 msgr2=0x55b578910780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b7e000640 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b578910780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b7e000640 1 -- 172.21.15.114:0/1006835849 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b578838cd0 con 0x55b578540390 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b7e000640 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578911200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b7002b790 tx=0x7f9b7002bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93b22d02dec417f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b77400640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b640b6780 0x7f9b640b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.418+0000 7f9b6be00640 1 -- 172.21.15.114:0/1006835849 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70037020 con 0x55b578540390 2024-09-21T06:59:46.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.419+0000 7f9b77400640 1 -- 172.21.15.114:0/860546457 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9b6c108380 con 0x7f9b801039b0 2024-09-21T06:59:46.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.419+0000 7f9b6be00640 1 -- 172.21.15.114:0/1006835849 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7002dd90 con 0x55b578540390 2024-09-21T06:59:46.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.419+0000 7f9b6be00640 1 -- 172.21.15.114:0/1006835849 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70033e30 con 0x55b578540390 2024-09-21T06:59:46.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.419+0000 7f9b7f400640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b640b6780 0x7f9b640b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1006835849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 msgr2=0x55b578911200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578911200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9b7002b790 tx=0x7f9b7002bc50 comp rx=0 tx=0).stop 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1006835849 shutdown_connections 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578911200 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b5785381b0 0x55b578910cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1006835849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b578910780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.420+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1006835849 >> 172.21.15.114:0/1006835849 conn(0x55b578555460 msgr2=0x55b57890e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.421+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1006835849 shutdown_connections 2024-09-21T06:59:46.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.422+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1006835849 wait complete. 2024-09-21T06:59:46.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.424+0000 7f9b8c63b740 1 -- start start 2024-09-21T06:59:46.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.425+0000 7f9b8c63b740 1 Processor -- start 2024-09-21T06:59:46.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.426+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5785381b0 0x55b578914040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.426+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578914620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.427+0000 7f9b7f400640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b640b6780 0x7f9b640b8c90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b78005fd0 tx=0x7f9b78004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:46.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.427+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5785381b0 0x55b578914040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.427+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578914620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.427+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578914620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42906/0 (socket says 172.21.15.114:42906) 2024-09-21T06:59:46.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.428+0000 7f9b7e000640 1 -- 172.21.15.114:0/1127647106 learned_addr learned my addr 172.21.15.114:0/1127647106 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:46.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.428+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b578919640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.431+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b5786cf700 con 0x55b5785381b0 2024-09-21T06:59:46.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.431+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b578809580 con 0x55b578540390 2024-09-21T06:59:46.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.431+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b57890dab0 con 0x55b578561070 2024-09-21T06:59:46.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.431+0000 7f9b8c63b740 10 client.0 _ll_register_callbacks cb 0x55b578819470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:46.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.431+0000 7f9b7f400640 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b578919640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.432+0000 7f9b7f400640 1 -- 172.21.15.114:0/1127647106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 msgr2=0x55b578914620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.432+0000 7f9b7f400640 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578914620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.432+0000 7f9b7f400640 1 -- 172.21.15.114:0/1127647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5785381b0 msgr2=0x55b578914040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.432+0000 7f9b7f400640 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5785381b0 0x55b578914040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.432+0000 7f9b7f400640 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5785cc650 con 0x55b578561070 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.433+0000 7f9b7f400640 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b578919640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b780116c0 tx=0x7f9b78004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8a66da1ca39058c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:46.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.433+0000 7f9b68c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:46.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.433+0000 7f9b68c00640 1 -- 172.21.15.114:0/1127647106 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78022070 con 0x55b578561070 2024-09-21T06:59:46.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.433+0000 7f9b68c00640 1 -- 172.21.15.114:0/1127647106 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9b780109e0 con 0x55b578561070 2024-09-21T06:59:46.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.434+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b57880bb90 con 0x55b578561070 2024-09-21T06:59:46.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.434+0000 7f9b68c00640 1 -- 172.21.15.114:0/1127647106 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b7802c050 con 0x55b578561070 2024-09-21T06:59:46.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.434+0000 7f9b8c63b740 10 client.26995 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:46.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.434+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b578826950 con 0x55b578561070 2024-09-21T06:59:46.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.435+0000 7f9b68c00640 1 -- 172.21.15.114:0/1127647106 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b57880bb90 con 0x55b578561070 2024-09-21T06:59:46.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.438+0000 7f9b68c00640 1 -- 172.21.15.114:0/1127647106 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f9b78027440 con 0x55b578561070 2024-09-21T06:59:46.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.439+0000 7f9b8c63b740 10 client.26995 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.439+0000 7f9b8c63b740 10 client.26995 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:59:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.439+0000 7f9b8c63b740 20 client.26995 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.439+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b57851f9f0 0x55b578925920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.439+0000 7f9b8c63b740 4 client.26995 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:46.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.439+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b578880f00 con 0x55b57851f9f0 2024-09-21T06:59:46.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.441+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b57851f9f0 0x55b578925920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.441+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b57851f9f0 0x55b578925920 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=73c6f4da46d055aa server_cookie=2e4ef479c9edcc11 in_seq=0 out_seq=0 2024-09-21T06:59:46.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.441+0000 7f9b68c00640 10 client.26995 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:59:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.442+0000 7f9b68c00640 1 -- 172.21.15.114:0/1127647106 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b578880f00 con 0x55b57851f9f0 2024-09-21T06:59:46.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.442+0000 7f9b68c00640 10 client.26995 handle_command_reply: tid=0 2024-09-21T06:59:46.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.443+0000 7f9b8c63b740 1 client.26995 shutdown 2024-09-21T06:59:46.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 msgr2=0x55b578919640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b578919640 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9b780116c0 tx=0x7f9b78004ab0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 shutdown_connections 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b57851f9f0 0x55b578925920 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b578919640 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b578540390 0x55b578914620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/1127647106 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5785381b0 0x55b578914040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 >> 172.21.15.114:0/1127647106 conn(0x55b578555460 msgr2=0x55b5789117b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:46.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b68c00640 0 client.26995 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:59:46.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 shutdown_connections 2024-09-21T06:59:46.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.445+0000 7f9b8c63b740 1 -- 172.21.15.114:0/1127647106 wait complete. 2024-09-21T06:59:46.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.446+0000 7f9b8c63b740 20 client.26995 trim_cache size 0 max 16384 2024-09-21T06:59:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.472+0000 7f9b8c63b740 1 Processor -- start 2024-09-21T06:59:46.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.472+0000 7f9b8c63b740 1 -- start start 2024-09-21T06:59:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.473+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b57890ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.473+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 0x55b57890b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.473+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b57892b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.473+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801aeaa0 con 0x55b578561070 2024-09-21T06:59:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.473+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801b0700 con 0x55b5785381b0 2024-09-21T06:59:46.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.473+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801af170 con 0x55b578540390 2024-09-21T06:59:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b57890ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 0x55b57890b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 0x55b57890b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50096/0 (socket says 172.21.15.114:50096) 2024-09-21T06:59:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 -- 172.21.15.114:0/2305109334 learned_addr learned my addr 172.21.15.114:0/2305109334 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:46.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7f400640 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b57892b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 -- 172.21.15.114:0/2305109334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 msgr2=0x55b57890ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b57890ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 -- 172.21.15.114:0/2305109334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 msgr2=0x55b57892b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b57892b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.474+0000 7f9b7e000640 1 -- 172.21.15.114:0/2305109334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57890c180 con 0x55b578540390 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.475+0000 7f9b7f400640 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b57892b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.475+0000 7f9b7e000640 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 0x55b57890b3b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006bc0 tx=0x7f9b7002b3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5281f5885a6b3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:46.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.476+0000 7f9b6b400640 1 -- 172.21.15.114:0/2305109334 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7004a070 con 0x55b578540390 2024-09-21T06:59:46.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.476+0000 7f9b6b400640 1 -- 172.21.15.114:0/2305109334 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70033430 con 0x55b578540390 2024-09-21T06:59:46.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.476+0000 7f9b6b400640 1 -- 172.21.15.114:0/2305109334 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70037040 con 0x55b578540390 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.477+0000 7f9b8c63b740 1 -- 172.21.15.114:0/2305109334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 msgr2=0x55b57890b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.477+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 0x55b57890b3b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006bc0 tx=0x7f9b7002b3a0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.478+0000 7f9b8c63b740 1 -- 172.21.15.114:0/2305109334 shutdown_connections 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.478+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b578561070 0x55b57892b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.478+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578540390 0x55b57890b3b0 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b70006bc0 tx=0x7f9b7002b3a0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.478+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/2305109334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b57890ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.478+0000 7f9b8c63b740 1 -- 172.21.15.114:0/2305109334 >> 172.21.15.114:0/2305109334 conn(0x55b578555460 msgr2=0x55b5789196c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.479+0000 7f9b8c63b740 1 -- 172.21.15.114:0/2305109334 shutdown_connections 2024-09-21T06:59:46.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.479+0000 7f9b8c63b740 1 -- 172.21.15.114:0/2305109334 wait complete. 2024-09-21T06:59:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.480+0000 7f9b8c63b740 1 -- start start 2024-09-21T06:59:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.480+0000 7f9b8c63b740 1 Processor -- start 2024-09-21T06:59:46.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b578920220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b57891c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5784f4da0 0x55b57891cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801aff70 con 0x55b5784f4da0 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801af840 con 0x55b5785381b0 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b801acac0 con 0x55b578561070 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b8c63b740 10 client.0 _ll_register_callbacks cb 0x7f9b7800ff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.481+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b578920220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.482+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b57891c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.482+0000 7f9b7e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b57891c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50106/0 (socket says 172.21.15.114:50106) 2024-09-21T06:59:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.482+0000 7f9b7e000640 1 -- 172.21.15.114:0/4182751978 learned_addr learned my addr 172.21.15.114:0/4182751978 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.482+0000 7f9b7f400640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5784f4da0 0x55b57891cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.483+0000 7f9b7f400640 1 -- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 msgr2=0x55b57891c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.483+0000 7f9b7f400640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b57891c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.483+0000 7f9b7f400640 1 -- 172.21.15.114:0/4182751978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 msgr2=0x55b578920220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.483+0000 7f9b7f400640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b578920220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.483+0000 7f9b7f400640 1 -- 172.21.15.114:0/4182751978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57891d310 con 0x55b5784f4da0 2024-09-21T06:59:46.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.484+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b578920220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:46.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.484+0000 7f9b7f400640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5784f4da0 0x55b57891cc20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009600 tx=0x7f9b780276f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe28fe8672e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:46.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.485+0000 7f9b68c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:59:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.486+0000 7f9b68c00640 1 -- 172.21.15.114:0/4182751978 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78022040 con 0x55b5784f4da0 2024-09-21T06:59:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.486+0000 7f9b68c00640 1 -- 172.21.15.114:0/4182751978 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9b78010bf0 con 0x55b5784f4da0 2024-09-21T06:59:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.486+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b70033430 con 0x55b5784f4da0 2024-09-21T06:59:46.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.487+0000 7f9b68c00640 1 -- 172.21.15.114:0/4182751978 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9b78010e10 con 0x55b5784f4da0 2024-09-21T06:59:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.487+0000 7f9b8c63b740 10 client.16767 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.487+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b578991150 con 0x55b5784f4da0 2024-09-21T06:59:46.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.488+0000 7f9b68c00640 1 -- 172.21.15.114:0/4182751978 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b70033430 con 0x55b5784f4da0 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.488+0000 7f9b68c00640 1 -- 172.21.15.114:0/4182751978 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f9b780316b0 con 0x55b5784f4da0 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b8c63b740 10 client.16767 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b8c63b740 10 client.16767 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b8c63b740 20 client.16767 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b5789918d0 0x55b578993cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b8c63b740 4 client.16767 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b57887c9b0 con 0x55b5789918d0 2024-09-21T06:59:46.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.489+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b5789918d0 0x55b578993cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.491+0000 7f9b7ea00640 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b5789918d0 0x55b578993cf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=627b376c84fc3820 server_cookie=541a8b06e4d221e4 in_seq=0 out_seq=0 2024-09-21T06:59:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.491+0000 7f9b68c00640 10 client.16767 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T06:59:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.492+0000 7f9b68c00640 1 -- 172.21.15.114:0/4182751978 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b57887c9b0 con 0x55b5789918d0 2024-09-21T06:59:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.492+0000 7f9b68c00640 10 client.16767 handle_command_reply: tid=0 2024-09-21T06:59:46.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.492+0000 7f9b8c63b740 1 client.16767 shutdown 2024-09-21T06:59:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5784f4da0 msgr2=0x55b57891cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5784f4da0 0x55b57891cc20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9b78009600 tx=0x7f9b780276f0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 shutdown_connections 2024-09-21T06:59:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55b5789918d0 0x55b578993cf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b5784f4da0 0x55b57891cc20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b578561070 0x55b57891c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 --2- 172.21.15.114:0/4182751978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b5785381b0 0x55b578920220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 >> 172.21.15.114:0/4182751978 conn(0x55b578555460 msgr2=0x55b578917450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:46.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.494+0000 7f9b68c00640 0 client.16767 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T06:59:46.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.495+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 shutdown_connections 2024-09-21T06:59:46.500 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:59:46.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.495+0000 7f9b8c63b740 1 -- 172.21.15.114:0/4182751978 wait complete. 2024-09-21T06:59:46.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.496+0000 7f9b8c63b740 20 client.16767 trim_cache size 0 max 16384 2024-09-21T06:59:46.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.497+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b640b6780 msgr2=0x7f9b640b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.497+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b640b6780 0x7f9b640b8c90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9b78005fd0 tx=0x7f9b78004a40 comp rx=0 tx=0).stop 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.497+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 msgr2=0x7f9b8019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.497+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 0x7f9b8019c5c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005ab0 tx=0x7f9b6c003c80 comp rx=0 tx=0).stop 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 shutdown_connections 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b80109c40 0x7f9b801a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b640b6780 0x7f9b640b8c90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b801092f0 0x7f9b8019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 --2- 172.21.15.114:0/860546457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b801039b0 0x7f9b8019c5c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:46.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 >> 172.21.15.114:0/860546457 conn(0x7f9b800ff440 msgr2=0x7f9b80108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:46.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.498+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 shutdown_connections 2024-09-21T06:59:46.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:46.500+0000 7f9b7fe00640 1 -- 172.21.15.114:0/860546457 wait complete. 2024-09-21T06:59:47.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:46 smithi114 ceph-mon[30654]: pgmap v565: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 580 op/s 2024-09-21T06:59:47.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:46 smithi183 ceph-mon[34219]: pgmap v565: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 580 op/s 2024-09-21T06:59:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:46 smithi193 ceph-mon[34601]: pgmap v565: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 14 MiB/s wr, 580 op/s 2024-09-21T06:59:47.510 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:59:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 310.0 s, 756.2 tps, lat 129.635 ms stddev 168.686 2024-09-21T06:59:48.159 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:48.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.975+0000 7f71fb200640 1 -- 172.21.15.114:0/2774720468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 msgr2=0x7f71f4007970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:48.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.975+0000 7f71fb200640 1 --2- 172.21.15.114:0/2774720468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4007970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009900 tx=0x7f71ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 -- 172.21.15.114:0/2774720468 shutdown_connections 2024-09-21T06:59:48.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 --2- 172.21.15.114:0/2774720468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 0x7f71f40a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 --2- 172.21.15.114:0/2774720468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f4007f90 0x7f71f40083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 --2- 172.21.15.114:0/2774720468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4007970 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 -- 172.21.15.114:0/2774720468 >> 172.21.15.114:0/2774720468 conn(0x7f71f400bb50 msgr2=0x7f71f400bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:48.978 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:48 smithi114 ceph-mon[30654]: pgmap v566: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 696 op/s 2024-09-21T06:59:48.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 -- 172.21.15.114:0/2774720468 shutdown_connections 2024-09-21T06:59:48.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.976+0000 7f71fb200640 1 -- 172.21.15.114:0/2774720468 wait complete. 2024-09-21T06:59:48.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.978+0000 7f71fb200640 1 Processor -- start 2024-09-21T06:59:48.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 -- start start 2024-09-21T06:59:48.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f4007f90 0x7f71f4132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4133580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 0x7f71f4138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71f40ac5d0 con 0x7f71f4007f90 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71f40ac450 con 0x7f71f4009540 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.979+0000 7f71fb200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71f40ac750 con 0x7f71f40a69a0 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4133580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42952/0 (socket says 172.21.15.114:42952) 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 -- 172.21.15.114:0/4155501394 learned_addr learned my addr 172.21.15.114:0/4155501394 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9e00640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f4007f90 0x7f71f4132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71fa800640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 0x7f71f4138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 -- 172.21.15.114:0/4155501394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 msgr2=0x7f71f4138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 0x7f71f4138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 -- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f4007f90 msgr2=0x7f71f4132fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f4007f90 0x7f71f4132fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71f9400640 1 -- 172.21.15.114:0/4155501394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f4138f50 con 0x7f71f4009540 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.980+0000 7f71fa800640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 0x7f71f4138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.981+0000 7f71f9400640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4133580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9a0 tx=0x7f71e800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.981+0000 7f71f2a00640 1 -- 172.21.15.114:0/4155501394 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800cd80 con 0x7f71f4009540 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.981+0000 7f71f2a00640 1 -- 172.21.15.114:0/4155501394 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e80035f0 con 0x7f71f4009540 2024-09-21T06:59:48.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.981+0000 7f71f2a00640 1 -- 172.21.15.114:0/4155501394 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8010690 con 0x7f71f4009540 2024-09-21T06:59:48.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.981+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f4139240 con 0x7f71f4009540 2024-09-21T06:59:48.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.981+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f413f9a0 con 0x7f71f4009540 2024-09-21T06:59:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.988+0000 7f72079e1740 1 Processor -- start 2024-09-21T06:59:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.988+0000 7f72079e1740 1 -- start start 2024-09-21T06:59:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.989+0000 7f72079e1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f01f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f72079e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f72079e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b3a390 0x55b122f029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f72079e1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b122e78fb0 con 0x55b122b5b070 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f72079e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b122e683e0 con 0x55b122b321b0 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f72079e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b122e6c1f0 con 0x55b122b3a390 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f71f2a00640 1 -- 172.21.15.114:0/4155501394 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f71e8004000 con 0x7f71f4009540 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42968/0 (socket says 172.21.15.114:42968) 2024-09-21T06:59:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f71f9e00640 1 -- 172.21.15.114:0/1479894425 learned_addr learned my addr 172.21.15.114:0/1479894425 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.990+0000 7f71fa800640 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f01f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.991+0000 7f71f9400640 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b3a390 0x55b122f029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71f9400640 1 -- 172.21.15.114:0/1479894425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 msgr2=0x55b122f024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71f9400640 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71f9400640 1 -- 172.21.15.114:0/1479894425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 msgr2=0x55b122f01f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71f9400640 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f01f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71f9400640 1 -- 172.21.15.114:0/1479894425 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b122e32cd0 con 0x55b122b3a390 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71f9400640 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b3a390 0x55b122f029f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f71e80e50b0 tx=0x7f71e80ebd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fe9563ee0c72ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:48.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.993+0000 7f71f2a00640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71e00b6850 0x7f71e00b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.992+0000 7f71e7400640 1 -- 172.21.15.114:0/1479894425 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80f0d60 con 0x55b122b3a390 2024-09-21T06:59:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.993+0000 7f71e7400640 1 -- 172.21.15.114:0/1479894425 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e80f5070 con 0x55b122b3a390 2024-09-21T06:59:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.993+0000 7f71e7400640 1 -- 172.21.15.114:0/1479894425 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80ee6c0 con 0x55b122b3a390 2024-09-21T06:59:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.993+0000 7f71f2a00640 1 -- 172.21.15.114:0/4155501394 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f71e80e7d30 con 0x7f71f4009540 2024-09-21T06:59:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.993+0000 7f71fa800640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71e00b6850 0x7f71e00b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.994+0000 7f72079e1740 1 -- 172.21.15.114:0/1479894425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b3a390 msgr2=0x55b122f029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.994+0000 7f71fa800640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71e00b6850 0x7f71e00b8d60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f71fc051230 tx=0x7f71fc069570 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.994+0000 7f72079e1740 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b3a390 0x55b122f029f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f71e80e50b0 tx=0x7f71e80ebd50 comp rx=0 tx=0).stop 2024-09-21T06:59:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.995+0000 7f72079e1740 1 -- 172.21.15.114:0/1479894425 shutdown_connections 2024-09-21T06:59:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.995+0000 7f72079e1740 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b3a390 0x55b122f029f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.995+0000 7f72079e1740 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.995+0000 7f72079e1740 1 --2- 172.21.15.114:0/1479894425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f01f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.995+0000 7f72079e1740 1 -- 172.21.15.114:0/1479894425 >> 172.21.15.114:0/1479894425 conn(0x55b122b4f460 msgr2=0x55b122effe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.996+0000 7f72079e1740 1 -- 172.21.15.114:0/1479894425 shutdown_connections 2024-09-21T06:59:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.996+0000 7f72079e1740 1 -- 172.21.15.114:0/1479894425 wait complete. 2024-09-21T06:59:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.997+0000 7f72079e1740 1 -- start start 2024-09-21T06:59:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.998+0000 7f72079e1740 1 Processor -- start 2024-09-21T06:59:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f0d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f0dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f19ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b122cc9700 con 0x55b122b5b070 2024-09-21T06:59:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b122e03580 con 0x55b122b3a390 2024-09-21T06:59:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b122f10750 con 0x55b122b321b0 2024-09-21T06:59:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f72079e1740 10 client.0 _ll_register_callbacks cb 0x55b122e13470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f0d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f0d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50152/0 (socket says 172.21.15.114:50152) 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:48.999+0000 7f71f9e00640 1 -- 172.21.15.114:0/511710834 learned_addr learned my addr 172.21.15.114:0/511710834 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.001+0000 7f71f9400640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f0dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.001+0000 7f71f9e00640 1 -- 172.21.15.114:0/511710834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 msgr2=0x55b122f0dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.001+0000 7f71f9e00640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f0dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.001+0000 7f71f9e00640 1 -- 172.21.15.114:0/511710834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 msgr2=0x55b122f19ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.001+0000 7f71f9e00640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f19ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.001+0000 7f71f9e00640 1 -- 172.21.15.114:0/511710834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b122bc6650 con 0x55b122b321b0 2024-09-21T06:59:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.002+0000 7f71f9400640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f0dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.003+0000 7f71f9e00640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f0d6c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71ec000c00 tx=0x7f71ec005f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71d7e12660ca175 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.003+0000 7f71dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.003+0000 7f71dbe00640 1 -- 172.21.15.114:0/511710834 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec035040 con 0x55b122b321b0 2024-09-21T06:59:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.003+0000 7f71dbe00640 1 -- 172.21.15.114:0/511710834 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f71ec031720 con 0x55b122b321b0 2024-09-21T06:59:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b122e05b90 con 0x55b122b321b0 2024-09-21T06:59:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f71dbe00640 1 -- 172.21.15.114:0/511710834 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71ec04f050 con 0x55b122b321b0 2024-09-21T06:59:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f72079e1740 10 client.27013 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b122e20950 con 0x55b122b321b0 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f71dbe00640 1 -- 172.21.15.114:0/511710834 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b122e05b90 con 0x55b122b321b0 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f71dbe00640 1 -- 172.21.15.114:0/511710834 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f71ec004750 con 0x55b122b321b0 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f72079e1740 10 client.27013 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.004+0000 7f72079e1740 10 client.27013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.005+0000 7f72079e1740 20 client.27013 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.005+0000 7f72079e1740 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122b199f0 0x55b122f1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.005+0000 7f72079e1740 4 client.27013 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.005+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b122e7af00 con 0x55b122b199f0 2024-09-21T06:59:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.005+0000 7f71f9400640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122b199f0 0x55b122f1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.006+0000 7f71dbe00640 10 client.27013 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:59:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.006+0000 7f71f9400640 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122b199f0 0x55b122f1fa40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7817262e8f402f8 server_cookie=5cc676f3e03ea44b in_seq=0 out_seq=0 2024-09-21T06:59:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.007+0000 7f71dbe00640 1 -- 172.21.15.114:0/511710834 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b122e7af00 con 0x55b122b199f0 2024-09-21T06:59:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.007+0000 7f71dbe00640 10 client.27013 handle_command_reply: tid=0 2024-09-21T06:59:49.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.007+0000 7f72079e1740 1 client.27013 shutdown 2024-09-21T06:59:49.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.007+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 msgr2=0x55b122f0d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:49.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.007+0000 7f72079e1740 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f0d6c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71ec000c00 tx=0x7f71ec005f20 comp rx=0 tx=0).stop 2024-09-21T06:59:49.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 shutdown_connections 2024-09-21T06:59:49.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122b199f0 0x55b122f1fa40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f19ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f0dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f71dbe00640 0 client.27013 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:59:49.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 --2- 172.21.15.114:0/511710834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f0d6c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 >> 172.21.15.114:0/511710834 conn(0x55b122b4f460 msgr2=0x55b122f05100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:49.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 shutdown_connections 2024-09-21T06:59:49.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.008+0000 7f72079e1740 1 -- 172.21.15.114:0/511710834 wait complete. 2024-09-21T06:59:49.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.009+0000 7f72079e1740 20 client.27013 trim_cache size 0 max 16384 2024-09-21T06:59:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.029+0000 7f72079e1740 1 Processor -- start 2024-09-21T06:59:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.029+0000 7f72079e1740 1 -- start start 2024-09-21T06:59:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f72079e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f0bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f72079e1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b3a390 0x55b122f199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f72079e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b5b070 0x55b122f0a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f72079e1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4149870 con 0x55b122b3a390 2024-09-21T06:59:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f72079e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71f414b4d0 con 0x55b122b321b0 2024-09-21T06:59:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f72079e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4149f40 con 0x55b122b5b070 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f0bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f0bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42996/0 (socket says 172.21.15.114:42996) 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f71f9e00640 1 -- 172.21.15.114:0/2327976477 learned_addr learned my addr 172.21.15.114:0/2327976477 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.030+0000 7f71f9e00640 1 -- 172.21.15.114:0/2327976477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b5b070 msgr2=0x55b122f0a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.031+0000 7f71f9e00640 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b5b070 0x55b122f0a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.031+0000 7f71f9e00640 1 -- 172.21.15.114:0/2327976477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b3a390 msgr2=0x55b122f199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.031+0000 7f71f9e00640 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b3a390 0x55b122f199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.031+0000 7f71f9e00640 1 -- 172.21.15.114:0/2327976477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b122efb2c0 con 0x55b122b321b0 2024-09-21T06:59:49.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.031+0000 7f71f9e00640 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f0bf40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f71ec004cc0 tx=0x7f71ec004940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb881126d61fbe6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:49.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.031+0000 7f71e6a00640 1 -- 172.21.15.114:0/2327976477 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec058070 con 0x55b122b321b0 2024-09-21T06:59:49.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.032+0000 7f71e6a00640 1 -- 172.21.15.114:0/2327976477 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec031d10 con 0x55b122b321b0 2024-09-21T06:59:49.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.032+0000 7f71e6a00640 1 -- 172.21.15.114:0/2327976477 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec035030 con 0x55b122b321b0 2024-09-21T06:59:49.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 -- 172.21.15.114:0/2327976477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 msgr2=0x55b122f0bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f0bf40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f71ec004cc0 tx=0x7f71ec004940 comp rx=0 tx=0).stop 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 -- 172.21.15.114:0/2327976477 shutdown_connections 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b5b070 0x55b122f0a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b3a390 0x55b122f199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 --2- 172.21.15.114:0/2327976477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b321b0 0x55b122f0bf40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 -- 172.21.15.114:0/2327976477 >> 172.21.15.114:0/2327976477 conn(0x55b122b4f460 msgr2=0x55b122f03960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:49.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 -- 172.21.15.114:0/2327976477 shutdown_connections 2024-09-21T06:59:49.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.033+0000 7f72079e1740 1 -- 172.21.15.114:0/2327976477 wait complete. 2024-09-21T06:59:49.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.034+0000 7f72079e1740 1 -- start start 2024-09-21T06:59:49.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.034+0000 7f72079e1740 1 Processor -- start 2024-09-21T06:59:49.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f16140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f10f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71f414ad40 con 0x55b122b5b070 2024-09-21T06:59:49.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71f414a610 con 0x55b122b3a390 2024-09-21T06:59:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4147850 con 0x55b122b321b0 2024-09-21T06:59:49.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f72079e1740 10 client.0 _ll_register_callbacks cb 0x7f71ec032c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f16140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f71f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f16140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50182/0 (socket says 172.21.15.114:50182) 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.035+0000 7f71f9e00640 1 -- 172.21.15.114:0/880447117 learned_addr learned my addr 172.21.15.114:0/880447117 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71fa800640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f10f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71f9e00640 1 -- 172.21.15.114:0/880447117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 msgr2=0x55b122f109f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71f9e00640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f109f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71f9e00640 1 -- 172.21.15.114:0/880447117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 msgr2=0x55b122f10f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:49.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71f9e00640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f10f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71f9e00640 1 -- 172.21.15.114:0/880447117 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b122f115a0 con 0x55b122b321b0 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71fa800640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f10f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71f9e00640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f16140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f71ec006010 tx=0x7f71ec031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d0b8bcecde3c378 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.036+0000 7f71dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.037+0000 7f71dbe00640 1 -- 172.21.15.114:0/880447117 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec058070 con 0x55b122b321b0 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.037+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec031d10 con 0x55b122b321b0 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.037+0000 7f71dbe00640 1 -- 172.21.15.114:0/880447117 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f71ec004230 con 0x55b122b321b0 2024-09-21T06:59:49.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.037+0000 7f71dbe00640 1 -- 172.21.15.114:0/880447117 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f71ec065050 con 0x55b122b321b0 2024-09-21T06:59:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.039+0000 7f72079e1740 10 client.27016 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:49.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.039+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b122f8b240 con 0x55b122b321b0 2024-09-21T06:59:49.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.041+0000 7f71dbe00640 1 -- 172.21.15.114:0/880447117 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71ec031d10 con 0x55b122b321b0 2024-09-21T06:59:49.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.041+0000 7f71dbe00640 1 -- 172.21.15.114:0/880447117 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f71ec004e20 con 0x55b122b321b0 2024-09-21T06:59:49.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.042+0000 7f72079e1740 10 client.27016 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:49.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.042+0000 7f72079e1740 10 client.27016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T06:59:49.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.042+0000 7f72079e1740 20 client.27016 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:49.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.042+0000 7f72079e1740 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122aeeda0 0x55b122f8d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:49.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.042+0000 7f72079e1740 4 client.27016 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:59:49.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.042+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b122e769b0 con 0x55b122aeeda0 2024-09-21T06:59:49.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.045+0000 7f71f9400640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122aeeda0 0x55b122f8d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:49.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.047+0000 7f71f9400640 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122aeeda0 0x55b122f8d8f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3f799b7b93f6393 server_cookie=cc1c18be60cc046c in_seq=0 out_seq=0 2024-09-21T06:59:49.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.047+0000 7f71dbe00640 10 client.27016 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T06:59:49.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.047+0000 7f71dbe00640 1 -- 172.21.15.114:0/880447117 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b122e769b0 con 0x55b122aeeda0 2024-09-21T06:59:49.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.047+0000 7f71dbe00640 10 client.27016 handle_command_reply: tid=0 2024-09-21T06:59:49.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.047+0000 7f72079e1740 1 client.27016 shutdown 2024-09-21T06:59:49.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 msgr2=0x55b122f16140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:49.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f16140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f71ec006010 tx=0x7f71ec031400 comp rx=0 tx=0).stop 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 shutdown_connections 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55b122aeeda0 0x55b122f8d8f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b122b5b070 0x55b122f10f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b122b3a390 0x55b122f109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 --2- 172.21.15.114:0/880447117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b122b321b0 0x55b122f16140 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f71dbe00640 0 client.27016 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T06:59:49.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.049+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 >> 172.21.15.114:0/880447117 conn(0x55b122b4f460 msgr2=0x55b122f02fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:49.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.051+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 shutdown_connections 2024-09-21T06:59:49.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.051+0000 7f72079e1740 1 -- 172.21.15.114:0/880447117 wait complete. 2024-09-21T06:59:49.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.051+0000 7f72079e1740 20 client.27016 trim_cache size 0 max 16384 2024-09-21T06:59:49.054 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:59:49.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.052+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71e00b6850 msgr2=0x7f71e00b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:49.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.052+0000 7f71fb200640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71e00b6850 0x7f71e00b8d60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f71fc051230 tx=0x7f71fc069570 comp rx=0 tx=0).stop 2024-09-21T06:59:49.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.052+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 msgr2=0x7f71f4133580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:49.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.052+0000 7f71fb200640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4133580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9a0 tx=0x7f71e800ee60 comp rx=0 tx=0).stop 2024-09-21T06:59:49.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.054+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 shutdown_connections 2024-09-21T06:59:49.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.054+0000 7f71fb200640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71f40a69a0 0x7f71f4138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.054+0000 7f71fb200640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71e00b6850 0x7f71e00b8d60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.054+0000 7f71fb200640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71f4009540 0x7f71f4133580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.054+0000 7f71fb200640 1 --2- 172.21.15.114:0/4155501394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71f4007f90 0x7f71f4132fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:49.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.054+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 >> 172.21.15.114:0/4155501394 conn(0x7f71f400bb50 msgr2=0x7f71f40a5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:49.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.055+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 shutdown_connections 2024-09-21T06:59:49.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:49.055+0000 7f71fb200640 1 -- 172.21.15.114:0/4155501394 wait complete. 2024-09-21T06:59:49.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:48 smithi183 ceph-mon[34219]: pgmap v566: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 696 op/s 2024-09-21T06:59:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:48 smithi193 ceph-mon[34601]: pgmap v566: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 17 MiB/s wr, 696 op/s 2024-09-21T06:59:50.337 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:59:50.975 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:51.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:50 smithi183 ceph-mon[34219]: pgmap v567: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 661 op/s 2024-09-21T06:59:51.161 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:50 smithi114 ceph-mon[30654]: pgmap v567: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 661 op/s 2024-09-21T06:59:51.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:50 smithi193 ceph-mon[34601]: pgmap v567: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 16 MiB/s wr, 661 op/s 2024-09-21T06:59:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.803+0000 7f9dcd000640 1 -- 172.21.15.114:0/1691332727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc8116510 msgr2=0x7f9dc8118990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.803+0000 7f9dcd000640 1 --2- 172.21.15.114:0/1691332727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc8116510 0x7f9dc8118990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000f0a0 tx=0x7f9dc002e710 comp rx=0 tx=0).stop 2024-09-21T06:59:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.806+0000 7f9dcd000640 1 -- 172.21.15.114:0/1691332727 shutdown_connections 2024-09-21T06:59:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.806+0000 7f9dcd000640 1 --2- 172.21.15.114:0/1691332727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc8116510 0x7f9dc8118990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.806+0000 7f9dcd000640 1 --2- 172.21.15.114:0/1691332727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc8115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.806+0000 7f9dcd000640 1 --2- 172.21.15.114:0/1691332727 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dc810a640 0x7f9dc810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.806+0000 7f9dcd000640 1 -- 172.21.15.114:0/1691332727 >> 172.21.15.114:0/1691332727 conn(0x7f9dc806e9f0 msgr2=0x7f9dc806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.807+0000 7f9dcd000640 1 -- 172.21.15.114:0/1691332727 shutdown_connections 2024-09-21T06:59:52.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.807+0000 7f9dcd000640 1 -- 172.21.15.114:0/1691332727 wait complete. 2024-09-21T06:59:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.811+0000 7f9dcd000640 1 Processor -- start 2024-09-21T06:59:52.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 -- start start 2024-09-21T06:59:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc810a640 0x7f9dc81a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc81a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dc8116510 0x7f9dc81a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8119ef0 con 0x7f9dc8116510 2024-09-21T06:59:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8119d70 con 0x7f9dc810af90 2024-09-21T06:59:52.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.812+0000 7f9dcd000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc811a070 con 0x7f9dc810a640 2024-09-21T06:59:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.813+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc81a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.813+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc81a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43028/0 (socket says 172.21.15.114:43028) 2024-09-21T06:59:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.813+0000 7f9dbfe00640 1 -- 172.21.15.114:0/3939777626 learned_addr learned my addr 172.21.15.114:0/3939777626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.813+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc810a640 0x7f9dc81a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.813+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dc8116510 0x7f9dc81a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dc7400640 1 -- 172.21.15.114:0/3939777626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 msgr2=0x7f9dc81a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc81a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dc7400640 1 -- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dc8116510 msgr2=0x7f9dc81a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dc8116510 0x7f9dc81a6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dc7400640 1 -- 172.21.15.114:0/3939777626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc81a6850 con 0x7f9dc810a640 2024-09-21T06:59:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc81a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:52.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.814+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc810a640 0x7f9dc81a08e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c970 tx=0x7f9db800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.815+0000 7f9dc4c00640 1 -- 172.21.15.114:0/3939777626 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8007bb0 con 0x7f9dc810a640 2024-09-21T06:59:52.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.815+0000 7f9dc4c00640 1 -- 172.21.15.114:0/3939777626 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db8007d50 con 0x7f9dc810a640 2024-09-21T06:59:52.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.815+0000 7f9dc4c00640 1 -- 172.21.15.114:0/3939777626 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db8004df0 con 0x7f9dc810a640 2024-09-21T06:59:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.816+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc81a6b40 con 0x7f9dc810a640 2024-09-21T06:59:52.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.817+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc81ad2a0 con 0x7f9dc810a640 2024-09-21T06:59:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 315.0 s, 576.2 tps, lat 166.586 ms stddev 216.389 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.836+0000 7f9dd53cc740 1 Processor -- start 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.836+0000 7f9dc4c00640 1 -- 172.21.15.114:0/3939777626 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9db8004f90 con 0x7f9dc810a640 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.839+0000 7f9dd53cc740 1 -- start start 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.839+0000 7f9dc4c00640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9dac0b6730 0x7f9dac0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.840+0000 7f9dc4c00640 1 -- 172.21.15.114:0/3939777626 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9db80e7ab0 con 0x7f9dc810a640 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.840+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9dac0b6730 0x7f9dac0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d17d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 0x55fc1d17dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d17e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d0ebfb0 con 0x55fc1cdce070 2024-09-21T06:59:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d0db3e0 con 0x55fc1cda51b0 2024-09-21T06:59:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d0df1f0 con 0x55fc1cdad390 2024-09-21T06:59:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 0x55fc1d17dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 0x55fc1d17dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43034/0 (socket says 172.21.15.114:43034) 2024-09-21T06:59:52.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.841+0000 7f9dbfe00640 1 -- 172.21.15.114:0/3656507231 learned_addr learned my addr 172.21.15.114:0/3656507231 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.842+0000 7f9dbfe00640 1 -- 172.21.15.114:0/3656507231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 msgr2=0x55fc1d17e200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.842+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d17e200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.842+0000 7f9dbfe00640 1 -- 172.21.15.114:0/3656507231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 msgr2=0x55fc1d17d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.842+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d17d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.842+0000 7f9dbfe00640 1 -- 172.21.15.114:0/3656507231 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1d0a5cd0 con 0x55fc1cda51b0 2024-09-21T06:59:52.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.842+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9dac0b6730 0x7f9dac0b8c40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0004af0 tx=0x7f9dc0005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.844+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 0x55fc1d17dcc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009df0 tx=0x7f9db000e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f46129965c64e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.844+0000 7f9dbcc00640 1 -- 172.21.15.114:0/3656507231 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0017070 con 0x55fc1cda51b0 2024-09-21T06:59:52.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.847+0000 7f9dbcc00640 1 -- 172.21.15.114:0/3656507231 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db0013920 con 0x55fc1cda51b0 2024-09-21T06:59:52.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.847+0000 7f9dbcc00640 1 -- 172.21.15.114:0/3656507231 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0017070 con 0x55fc1cda51b0 2024-09-21T06:59:52.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.848+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3656507231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 msgr2=0x55fc1d17dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.849+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 0x55fc1d17dcc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009df0 tx=0x7f9db000e5a0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.850+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3656507231 shutdown_connections 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.850+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d17e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.850+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cda51b0 0x55fc1d17dcc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.850+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3656507231 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d17d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.850+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3656507231 >> 172.21.15.114:0/3656507231 conn(0x55fc1cdc2460 msgr2=0x55fc1d17b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.851+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3656507231 shutdown_connections 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.851+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3656507231 wait complete. 2024-09-21T06:59:52.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.852+0000 7f9dd53cc740 1 -- start start 2024-09-21T06:59:52.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.852+0000 7f9dd53cc740 1 Processor -- start 2024-09-21T06:59:52.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 0x55fc1d189000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d195250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fc1cf3c700 con 0x55fc1cdad390 2024-09-21T06:59:52.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d076580 con 0x55fc1cdce070 2024-09-21T06:59:52.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fc1d18bf60 con 0x55fc1cda51b0 2024-09-21T06:59:52.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.853+0000 7f9dd53cc740 10 client.0 _ll_register_callbacks cb 0x55fc1d086470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:52.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.854+0000 7f9dc7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 0x55fc1d189000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.854+0000 7f9dc7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 0x55fc1d189000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50218/0 (socket says 172.21.15.114:50218) 2024-09-21T06:59:52.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.854+0000 7f9dc7400640 1 -- 172.21.15.114:0/2246229082 learned_addr learned my addr 172.21.15.114:0/2246229082 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.854+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.855+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.855+0000 7f9dc7400640 1 -- 172.21.15.114:0/2246229082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 msgr2=0x55fc1d195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.856+0000 7f9dc7400640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.856+0000 7f9dc7400640 1 -- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 msgr2=0x55fc1d1895a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.856+0000 7f9dc7400640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1895a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.856+0000 7f9dc7400640 1 -- 172.21.15.114:0/2246229082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1ce39650 con 0x55fc1cda51b0 2024-09-21T06:59:52.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.857+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:52.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.857+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:52.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.858+0000 7f9dc7400640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 0x55fc1d189000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9db80ad4c0 tx=0x7f9db80e52e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b21b99859b96f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.858+0000 7f9db5600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T06:59:52.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.858+0000 7f9db5600640 1 -- 172.21.15.114:0/2246229082 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db80046c0 con 0x55fc1cda51b0 2024-09-21T06:59:52.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.858+0000 7f9db5600640 1 -- 172.21.15.114:0/2246229082 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9db80ac200 con 0x55fc1cda51b0 2024-09-21T06:59:52.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.859+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc1d078b90 con 0x55fc1cda51b0 2024-09-21T06:59:52.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.859+0000 7f9db5600640 1 -- 172.21.15.114:0/2246229082 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9db80ad040 con 0x55fc1cda51b0 2024-09-21T06:59:52.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.859+0000 7f9dd53cc740 10 client.27025 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:52.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.860+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fc1d093950 con 0x55fc1cda51b0 2024-09-21T06:59:52.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.860+0000 7f9db5600640 1 -- 172.21.15.114:0/2246229082 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fc1d078b90 con 0x55fc1cda51b0 2024-09-21T06:59:52.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.862+0000 7f9db5600640 1 -- 172.21.15.114:0/2246229082 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f9db80b6490 con 0x55fc1cda51b0 2024-09-21T06:59:52.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.862+0000 7f9dd53cc740 10 client.27025 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:52.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.862+0000 7f9dd53cc740 10 client.27025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:59:52.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.863+0000 7f9dd53cc740 20 client.27025 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:52.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.863+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1cd8c9f0 0x55fc1d19b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.863+0000 7f9dd53cc740 4 client.27025 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:52.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.863+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc1d0edf00 con 0x55fc1cd8c9f0 2024-09-21T06:59:52.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.864+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1cd8c9f0 0x55fc1d19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.866+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1cd8c9f0 0x55fc1d19b140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a9caae59cbdd7e8 server_cookie=8c2267619cdb21f8 in_seq=0 out_seq=0 2024-09-21T06:59:52.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.866+0000 7f9db5600640 10 client.27025 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:59:52.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.868+0000 7f9db5600640 1 -- 172.21.15.114:0/2246229082 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc1d0edf00 con 0x55fc1cd8c9f0 2024-09-21T06:59:52.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.868+0000 7f9db5600640 10 client.27025 handle_command_reply: tid=0 2024-09-21T06:59:52.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.868+0000 7f9dd53cc740 1 client.27025 shutdown 2024-09-21T06:59:52.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.869+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 msgr2=0x55fc1d189000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.869+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 0x55fc1d189000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9db80ad4c0 tx=0x7f9db80e52e0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 shutdown_connections 2024-09-21T06:59:52.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1cd8c9f0 0x55fc1d19b140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/2246229082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cda51b0 0x55fc1d189000 secure :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9db80ad4c0 tx=0x7f9db80e52e0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 >> 172.21.15.114:0/2246229082 conn(0x55fc1cdc2460 msgr2=0x55fc1d171600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.870+0000 7f9db5600640 0 client.27025 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:59:52.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.871+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 shutdown_connections 2024-09-21T06:59:52.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.872+0000 7f9dd53cc740 1 -- 172.21.15.114:0/2246229082 wait complete. 2024-09-21T06:59:52.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.873+0000 7f9dd53cc740 20 client.27025 trim_cache size 0 max 16384 2024-09-21T06:59:52.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.902+0000 7f9dd53cc740 1 Processor -- start 2024-09-21T06:59:52.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.902+0000 7f9dd53cc740 1 -- start start 2024-09-21T06:59:52.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.902+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d171d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cd61da0 0x55fc1d195150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b7350 con 0x55fc1cdad390 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b8fb0 con 0x55fc1cdce070 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b7a20 con 0x55fc1cd61da0 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d171d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.903+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cd61da0 0x55fc1d195150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.904+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cd61da0 0x55fc1d195150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50220/0 (socket says 172.21.15.114:50220) 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.904+0000 7f9dc7e00640 1 -- 172.21.15.114:0/1968824114 learned_addr learned my addr 172.21.15.114:0/1968824114 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.904+0000 7f9dc7400640 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.905+0000 7f9dbfe00640 1 -- 172.21.15.114:0/1968824114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cd61da0 msgr2=0x55fc1d195150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.905+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cd61da0 0x55fc1d195150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.905+0000 7f9dbfe00640 1 -- 172.21.15.114:0/1968824114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 msgr2=0x55fc1d1838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.905+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.905+0000 7f9dbfe00640 1 -- 172.21.15.114:0/1968824114 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc1d18b5e0 con 0x55fc1cdce070 2024-09-21T06:59:52.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.906+0000 7f9dbfe00640 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d171d10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009980 tx=0x7f9db0009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62da35aa3cbe36e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.907+0000 7f9db7e00640 1 -- 172.21.15.114:0/1968824114 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0017040 con 0x55fc1cdce070 2024-09-21T06:59:52.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.907+0000 7f9db7e00640 1 -- 172.21.15.114:0/1968824114 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db0004500 con 0x55fc1cdce070 2024-09-21T06:59:52.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.908+0000 7f9dd53cc740 1 -- 172.21.15.114:0/1968824114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 msgr2=0x55fc1d171d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.908+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d171d10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009980 tx=0x7f9db0009190 comp rx=0 tx=0).stop 2024-09-21T06:59:52.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 -- 172.21.15.114:0/1968824114 shutdown_connections 2024-09-21T06:59:52.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cd61da0 0x55fc1d195150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cdce070 0x55fc1d171d10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/1968824114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdad390 0x55fc1d1838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 -- 172.21.15.114:0/1968824114 >> 172.21.15.114:0/1968824114 conn(0x55fc1cdc2460 msgr2=0x55fc1d171600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 -- 172.21.15.114:0/1968824114 shutdown_connections 2024-09-21T06:59:52.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.910+0000 7f9dd53cc740 1 -- 172.21.15.114:0/1968824114 wait complete. 2024-09-21T06:59:52.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.911+0000 7f9dd53cc740 1 -- start start 2024-09-21T06:59:52.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.911+0000 7f9dd53cc740 1 Processor -- start 2024-09-21T06:59:52.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.912+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cd61da0 0x55fc1d1ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.912+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d200160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.913+0000 7f9dd53cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d2006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.913+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b8820 con 0x55fc1cdce070 2024-09-21T06:59:52.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.913+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b80f0 con 0x55fc1cd61da0 2024-09-21T06:59:52.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.913+0000 7f9dd53cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc81b52f0 con 0x55fc1cdad390 2024-09-21T06:59:52.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.913+0000 7f9dd53cc740 10 client.0 _ll_register_callbacks cb 0x7f9db8005490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.914+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d2006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.914+0000 7f9dc7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d2006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44606/0 (socket says 172.21.15.114:44606) 2024-09-21T06:59:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.914+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d200160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.914+0000 7f9dc7e00640 1 -- 172.21.15.114:0/3634964590 learned_addr learned my addr 172.21.15.114:0/3634964590 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.916+0000 7f9dc7e00640 1 -- 172.21.15.114:0/3634964590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 msgr2=0x55fc1d200160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.916+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d200160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.916+0000 7f9dc7e00640 1 -- 172.21.15.114:0/3634964590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cd61da0 msgr2=0x55fc1d1ffc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.916+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cd61da0 0x55fc1d1ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.916+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cd61da0 0x55fc1d1ffc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.916+0000 7f9dc7e00640 1 -- 172.21.15.114:0/3634964590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc1d190250 con 0x55fc1cdce070 2024-09-21T06:59:52.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.917+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cd61da0 0x55fc1d1ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:52.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.920+0000 7f9db5600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T06:59:52.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.920+0000 7f9dc7e00640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d2006a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0035aa0 tx=0x7f9dc000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96c8a79a4404740 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.921+0000 7f9db5600640 1 -- 172.21.15.114:0/3634964590 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc003a030 con 0x55fc1cdce070 2024-09-21T06:59:52.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.924+0000 7f9db5600640 1 -- 172.21.15.114:0/3634964590 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9dc003b070 con 0x55fc1cdce070 2024-09-21T06:59:52.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.924+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db0004500 con 0x55fc1cdce070 2024-09-21T06:59:52.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.927+0000 7f9db5600640 1 -- 172.21.15.114:0/3634964590 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9dc0004040 con 0x55fc1cdce070 2024-09-21T06:59:52.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.928+0000 7f9dd53cc740 10 client.16785 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:52.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.928+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fc1d1914d0 con 0x55fc1cdce070 2024-09-21T06:59:52.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.928+0000 7f9db5600640 1 -- 172.21.15.114:0/3634964590 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db0004500 con 0x55fc1cdce070 2024-09-21T06:59:52.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9db5600640 1 -- 172.21.15.114:0/3634964590 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7f9dc00579d0 con 0x55fc1cdce070 2024-09-21T06:59:52.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dd53cc740 10 client.16785 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:52.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dd53cc740 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T06:59:52.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dd53cc740 20 client.16785 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:52.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1d191c50 0x55fc1d177630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dd53cc740 4 client.16785 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:59:52.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc1d0e99b0 con 0x55fc1d191c50 2024-09-21T06:59:52.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.932+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1d191c50 0x55fc1d177630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.936+0000 7f9dc7400640 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1d191c50 0x55fc1d177630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5af142293e99c061 server_cookie=19b76494477b6c75 in_seq=0 out_seq=0 2024-09-21T06:59:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.936+0000 7f9db5600640 10 client.16785 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T06:59:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.937+0000 7f9db5600640 1 -- 172.21.15.114:0/3634964590 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc1d0e99b0 con 0x55fc1d191c50 2024-09-21T06:59:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.937+0000 7f9db5600640 10 client.16785 handle_command_reply: tid=0 2024-09-21T06:59:52.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.937+0000 7f9dd53cc740 1 client.16785 shutdown 2024-09-21T06:59:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.937+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 msgr2=0x55fc1d2006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.937+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d2006a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0035aa0 tx=0x7f9dc000d9d0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 shutdown_connections 2024-09-21T06:59:52.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fc1d191c50 0x55fc1d177630 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fc1cdce070 0x55fc1d2006a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fc1cdad390 0x55fc1d200160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 --2- 172.21.15.114:0/3634964590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc1cd61da0 0x55fc1d1ffc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 >> 172.21.15.114:0/3634964590 conn(0x55fc1cdc2460 msgr2=0x55fc1d199130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9db5600640 0 client.16785 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 shutdown_connections 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 1 -- 172.21.15.114:0/3634964590 wait complete. 2024-09-21T06:59:52.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.939+0000 7f9dd53cc740 20 client.16785 trim_cache size 0 max 16384 2024-09-21T06:59:52.942 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:59:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.940+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9dac0b6730 msgr2=0x7f9dac0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.940+0000 7f9dcd000640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9dac0b6730 0x7f9dac0b8c40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0004af0 tx=0x7f9dc0005fd0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.940+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc810a640 msgr2=0x7f9dc81a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.940+0000 7f9dcd000640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc810a640 0x7f9dc81a08e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c970 tx=0x7f9db800ce30 comp rx=0 tx=0).stop 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.941+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 shutdown_connections 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.941+0000 7f9dcd000640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9dc8116510 0x7f9dc81a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.941+0000 7f9dcd000640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9dac0b6730 0x7f9dac0b8c40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.941+0000 7f9dcd000640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9dc810af90 0x7f9dc81a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.941+0000 7f9dcd000640 1 --2- 172.21.15.114:0/3939777626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc810a640 0x7f9dc81a08e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.941+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 >> 172.21.15.114:0/3939777626 conn(0x7f9dc806e9f0 msgr2=0x7f9dc8073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.942+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 shutdown_connections 2024-09-21T06:59:52.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:52.942+0000 7f9dcd000640 1 -- 172.21.15.114:0/3939777626 wait complete. 2024-09-21T06:59:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.953+0000 7fe925400640 1 -- 172.21.15.114:0/3397230869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92006ac80 msgr2=0x7fe92010a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.953+0000 7fe925400640 1 --2- 172.21.15.114:0/3397230869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92006ac80 0x7fe92010a980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe90800a9a0 tx=0x7fe90802b740 comp rx=0 tx=0).stop 2024-09-21T06:59:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.954+0000 7fe925400640 1 -- 172.21.15.114:0/3397230869 shutdown_connections 2024-09-21T06:59:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.954+0000 7fe925400640 1 --2- 172.21.15.114:0/3397230869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe920114d60 0x7fe920117180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.954+0000 7fe925400640 1 --2- 172.21.15.114:0/3397230869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe92006ac80 0x7fe92010a980 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.954+0000 7fe925400640 1 --2- 172.21.15.114:0/3397230869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92010d650 0x7fe92010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.954+0000 7fe925400640 1 -- 172.21.15.114:0/3397230869 >> 172.21.15.114:0/3397230869 conn(0x7fe920071080 msgr2=0x7fe920073bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.955+0000 7fe925400640 1 -- 172.21.15.114:0/3397230869 shutdown_connections 2024-09-21T06:59:52.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.955+0000 7fe925400640 1 -- 172.21.15.114:0/3397230869 wait complete. 2024-09-21T06:59:52.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.956+0000 7fe925400640 1 Processor -- start 2024-09-21T06:59:52.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.958+0000 7fe925400640 1 -- start start 2024-09-21T06:59:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.958+0000 7fe925400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92006ac80 0x7fe9201a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.958+0000 7fe925400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 0x7fe9201a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.958+0000 7fe925400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9201a7d50 0x7fe9201a8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe925400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe920119ef0 con 0x7fe92006ac80 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe925400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe920119d70 con 0x7fe9201a7d50 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe925400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe92011a070 con 0x7fe920114d60 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe917e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 0x7fe9201a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe917e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 0x7fe9201a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50240/0 (socket says 172.21.15.114:50240) 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe917e00640 1 -- 172.21.15.114:0/1783371517 learned_addr learned my addr 172.21.15.114:0/1783371517 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.959+0000 7fe91e800640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92006ac80 0x7fe9201a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.960+0000 7fe917e00640 1 -- 172.21.15.114:0/1783371517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9201a7d50 msgr2=0x7fe9201a8160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:59:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.960+0000 7fe917e00640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9201a7d50 0x7fe9201a8160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.960+0000 7fe917e00640 1 -- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92006ac80 msgr2=0x7fe9201a2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.960+0000 7fe917e00640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92006ac80 0x7fe9201a2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.964+0000 7fe917e00640 1 -- 172.21.15.114:0/1783371517 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9201a8860 con 0x7fe920114d60 2024-09-21T06:59:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.964+0000 7fe91e800640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92006ac80 0x7fe9201a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.964+0000 7fe917e00640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 0x7fe9201a2ac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9080052f0 tx=0x7fe908005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.965+0000 7fe917400640 1 -- 172.21.15.114:0/1783371517 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908040070 con 0x7fe920114d60 2024-09-21T06:59:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.965+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9201a8af0 con 0x7fe920114d60 2024-09-21T06:59:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.965+0000 7fe917400640 1 -- 172.21.15.114:0/1783371517 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9080046c0 con 0x7fe920114d60 2024-09-21T06:59:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.965+0000 7fe917400640 1 -- 172.21.15.114:0/1783371517 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908030bc0 con 0x7fe920114d60 2024-09-21T06:59:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.965+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9201af1c0 con 0x7fe920114d60 2024-09-21T06:59:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 Processor -- start 2024-09-21T06:59:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 -- start start 2024-09-21T06:59:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405dae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405db350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405db890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564b405506b0 con 0x564b402154d0 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b4053fae0 con 0x564b402361b0 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.972+0000 7fe92c48d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b405438f0 con 0x564b4020d2f0 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405db350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50254/0 (socket says 172.21.15.114:50254) 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 -- 172.21.15.114:0/1023667411 learned_addr learned my addr 172.21.15.114:0/1023667411 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 -- 172.21.15.114:0/1023667411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 msgr2=0x564b405dae10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405dae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 -- 172.21.15.114:0/1023667411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 msgr2=0x564b405db890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405db890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.973+0000 7fe91e800640 1 -- 172.21.15.114:0/1023667411 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4050c780 con 0x564b4020d2f0 2024-09-21T06:59:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.974+0000 7fe917400640 1 -- 172.21.15.114:0/1783371517 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe908004b70 con 0x7fe920114d60 2024-09-21T06:59:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.980+0000 7fe917400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9000b6780 0x7fe9000b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.981+0000 7fe917400640 1 -- 172.21.15.114:0/1783371517 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe908109240 con 0x7fe920114d60 2024-09-21T06:59:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.981+0000 7fe91e800640 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405db350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe910009d20 tx=0x7fe9100117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ed8e80309a5f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.982+0000 7fe90f400640 1 -- 172.21.15.114:0/1023667411 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910012650 con 0x564b4020d2f0 2024-09-21T06:59:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.982+0000 7fe90f400640 1 -- 172.21.15.114:0/1023667411 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe910009e10 con 0x564b4020d2f0 2024-09-21T06:59:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.982+0000 7fe90f400640 1 -- 172.21.15.114:0/1023667411 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91000fbf0 con 0x564b4020d2f0 2024-09-21T06:59:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.983+0000 7fe92c48d740 1 -- 172.21.15.114:0/1023667411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 msgr2=0x564b405db350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.983+0000 7fe92c48d740 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405db350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe910009d20 tx=0x7fe9100117b0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.984+0000 7fe92c48d740 1 -- 172.21.15.114:0/1023667411 shutdown_connections 2024-09-21T06:59:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.984+0000 7fe92c48d740 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405db890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.984+0000 7fe92c48d740 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405db350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.984+0000 7fe92c48d740 1 --2- 172.21.15.114:0/1023667411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405dae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.984+0000 7fe92c48d740 1 -- 172.21.15.114:0/1023667411 >> 172.21.15.114:0/1023667411 conn(0x564b4022a300 msgr2=0x564b405d8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.985+0000 7fe92c48d740 1 -- 172.21.15.114:0/1023667411 shutdown_connections 2024-09-21T06:59:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.986+0000 7fe91f200640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9000b6780 0x7fe9000b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.986+0000 7fe92c48d740 1 -- 172.21.15.114:0/1023667411 wait complete. 2024-09-21T06:59:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.987+0000 7fe92c48d740 1 -- start start 2024-09-21T06:59:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.987+0000 7fe92c48d740 1 Processor -- start 2024-09-21T06:59:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.988+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe92c48d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564b40477ab0 con 0x564b402154d0 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe92c48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b404e6570 con 0x564b402361b0 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe92c48d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564b405e5590 con 0x564b4020d2f0 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe92c48d740 10 client.0 _ll_register_callbacks cb 0x564b404f5100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43076/0 (socket says 172.21.15.114:43076) 2024-09-21T06:59:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 -- 172.21.15.114:0/710250139 learned_addr learned my addr 172.21.15.114:0/710250139 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91e800640 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 -- 172.21.15.114:0/710250139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 msgr2=0x564b405e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405e6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 -- 172.21.15.114:0/710250139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 msgr2=0x564b405e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:52.989+0000 7fe91f200640 1 -- 172.21.15.114:0/710250139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b402a1790 con 0x564b402361b0 2024-09-21T06:59:53.001 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.000+0000 7fe907e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.000+0000 7fe91f200640 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405f14b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe91800d930 tx=0x7fe918004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35bed1ba2e7bc9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.001+0000 7fe91f200640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9000b6780 0x7fe9000b8c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe91800aa90 tx=0x7fe918009250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.001+0000 7fe907e00640 1 -- 172.21.15.114:0/710250139 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918018c50 con 0x564b402361b0 2024-09-21T06:59:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.001+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe910009e10 con 0x564b402361b0 2024-09-21T06:59:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.001+0000 7fe907e00640 1 -- 172.21.15.114:0/710250139 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe918018a50 con 0x564b402361b0 2024-09-21T06:59:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.002+0000 7fe907e00640 1 -- 172.21.15.114:0/710250139 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe918018c50 con 0x564b402361b0 2024-09-21T06:59:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.002+0000 7fe92c48d740 10 client.27077 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.003+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b4031a620 con 0x564b402361b0 2024-09-21T06:59:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.004+0000 7fe907e00640 1 -- 172.21.15.114:0/710250139 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe910009e10 con 0x564b402361b0 2024-09-21T06:59:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.004+0000 7fe907e00640 1 -- 172.21.15.114:0/710250139 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fe91802d420 con 0x564b402361b0 2024-09-21T06:59:53.008 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe92c48d740 10 client.27077 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe92c48d740 10 client.27077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe92c48d740 20 client.27077 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe92c48d740 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b401f4d80 0x564b40609810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe92c48d740 4 client.27077 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b40552600 con 0x564b401f4d80 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.007+0000 7fe91e800640 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b401f4d80 0x564b40609810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.008+0000 7fe91e800640 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b401f4d80 0x564b40609810 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2592fd1413ff0d9f server_cookie=422845533b1ecd73 in_seq=0 out_seq=0 2024-09-21T06:59:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.008+0000 7fe907e00640 10 client.27077 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.009+0000 7fe907e00640 1 -- 172.21.15.114:0/710250139 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564b40552600 con 0x564b401f4d80 2024-09-21T06:59:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.009+0000 7fe907e00640 10 client.27077 handle_command_reply: tid=0 2024-09-21T06:59:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.009+0000 7fe92c48d740 1 client.27077 shutdown 2024-09-21T06:59:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.009+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 msgr2=0x564b405f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.009+0000 7fe92c48d740 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405f14b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe91800d930 tx=0x7fe918004b50 comp rx=0 tx=0).stop 2024-09-21T06:59:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 shutdown_connections 2024-09-21T06:59:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b405f14b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b401f4d80 0x564b40609810 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 --2- 172.21.15.114:0/710250139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b4020d2f0 0x564b405e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 >> 172.21.15.114:0/710250139 conn(0x564b4022a300 msgr2=0x564b405d7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe907e00640 0 client.27077 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 shutdown_connections 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.010+0000 7fe92c48d740 1 -- 172.21.15.114:0/710250139 wait complete. 2024-09-21T06:59:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.011+0000 7fe92c48d740 20 client.27077 trim_cache size 0 max 16384 2024-09-21T06:59:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.047+0000 7fe92c48d740 1 Processor -- start 2024-09-21T06:59:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.047+0000 7fe92c48d740 1 -- start start 2024-09-21T06:59:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.047+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 0x564b405ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.047+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b402361b0 0x564b40609780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe92c48d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201b70e0 con 0x564b402154d0 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe92c48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201b8d40 con 0x564b4020d2f0 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe92c48d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201b77b0 con 0x564b402361b0 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe91e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 0x564b405ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe91e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 0x564b405ec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43084/0 (socket says 172.21.15.114:43084) 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe91e800640 1 -- 172.21.15.114:0/682286138 learned_addr learned my addr 172.21.15.114:0/682286138 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe917e00640 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.048+0000 7fe91e800640 1 -- 172.21.15.114:0/682286138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b402361b0 msgr2=0x564b40609780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.049+0000 7fe91e800640 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b402361b0 0x564b40609780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.049+0000 7fe91e800640 1 -- 172.21.15.114:0/682286138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 msgr2=0x564b405e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.049+0000 7fe91e800640 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.049+0000 7fe91e800640 1 -- 172.21.15.114:0/682286138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b405e7660 con 0x564b4020d2f0 2024-09-21T06:59:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.049+0000 7fe91e800640 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 0x564b405ec190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe910009c40 tx=0x7fe9100117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911c4e136e256fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.050+0000 7fe90ea00640 1 -- 172.21.15.114:0/682286138 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910030020 con 0x564b4020d2f0 2024-09-21T06:59:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.051+0000 7fe90ea00640 1 -- 172.21.15.114:0/682286138 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe910009e10 con 0x564b4020d2f0 2024-09-21T06:59:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.051+0000 7fe90ea00640 1 -- 172.21.15.114:0/682286138 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9100035f0 con 0x564b4020d2f0 2024-09-21T06:59:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.053+0000 7fe92c48d740 1 -- 172.21.15.114:0/682286138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 msgr2=0x564b405ec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.053+0000 7fe92c48d740 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 0x564b405ec190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe910009c40 tx=0x7fe9100117b0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.054+0000 7fe92c48d740 1 -- 172.21.15.114:0/682286138 shutdown_connections 2024-09-21T06:59:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.054+0000 7fe92c48d740 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b402361b0 0x564b40609780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.054+0000 7fe92c48d740 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b405e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.054+0000 7fe92c48d740 1 --2- 172.21.15.114:0/682286138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4020d2f0 0x564b405ec190 secure :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe910009c40 tx=0x7fe9100117b0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.054+0000 7fe92c48d740 1 -- 172.21.15.114:0/682286138 >> 172.21.15.114:0/682286138 conn(0x564b4022a300 msgr2=0x564b405dff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.055+0000 7fe92c48d740 1 -- 172.21.15.114:0/682286138 shutdown_connections 2024-09-21T06:59:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.055+0000 7fe92c48d740 1 -- 172.21.15.114:0/682286138 wait complete. 2024-09-21T06:59:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 -- start start 2024-09-21T06:59:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 Processor -- start 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b406168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b40616ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b401ca1c0 0x564b4066c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201b85b0 con 0x564b402154d0 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201b7e80 con 0x564b402361b0 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.056+0000 7fe92c48d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9201b5060 con 0x564b401ca1c0 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe92c48d740 10 client.0 _ll_register_callbacks cb 0x7fe91800a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe91f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b401ca1c0 0x564b4066c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.059 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b40616ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe91f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b401ca1c0 0x564b4066c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50274/0 (socket says 172.21.15.114:50274) 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b40616ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43092/0 (socket says 172.21.15.114:43092) 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe91f200640 1 -- 172.21.15.114:0/3337662883 learned_addr learned my addr 172.21.15.114:0/3337662883 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe91e800640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b406168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 -- 172.21.15.114:0/3337662883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b401ca1c0 msgr2=0x564b4066c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b401ca1c0 0x564b4066c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 -- 172.21.15.114:0/3337662883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 msgr2=0x564b406168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b406168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe917e00640 1 -- 172.21.15.114:0/3337662883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4066c8a0 con 0x564b402361b0 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.057+0000 7fe91e800640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b406168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.058+0000 7fe917e00640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b40616ec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9081065c0 tx=0x7fe9080cea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e538ba61c9c981f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.058+0000 7fe907e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.058+0000 7fe907e00640 1 -- 172.21.15.114:0/3337662883 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9080ca430 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.058+0000 7fe907e00640 1 -- 172.21.15.114:0/3337662883 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe9080d8450 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.058+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe910009e10 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.058+0000 7fe907e00640 1 -- 172.21.15.114:0/3337662883 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe9080ce7d0 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.059+0000 7fe92c48d740 10 client.27083 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.059+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564b40673c40 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.059+0000 7fe907e00640 1 -- 172.21.15.114:0/3337662883 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe910009e10 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.059+0000 7fe907e00640 1 -- 172.21.15.114:0/3337662883 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fe90804d030 con 0x564b402361b0 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.060+0000 7fe92c48d740 10 client.27083 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.060+0000 7fe92c48d740 10 client.27083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.060+0000 7fe92c48d740 20 client.27083 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.060+0000 7fe92c48d740 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b40674480 0x564b406768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.060+0000 7fe92c48d740 4 client.27083 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:59:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.060+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b4054e0b0 con 0x564b40674480 2024-09-21T06:59:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.063+0000 7fe91e800640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b40674480 0x564b406768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:53.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:52 smithi114 ceph-mon[30654]: pgmap v568: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 630 op/s 2024-09-21T06:59:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.064+0000 7fe91e800640 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b40674480 0x564b406768a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c1a3635e97df8e server_cookie=9e870f3defd99013 in_seq=0 out_seq=0 2024-09-21T06:59:53.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.065+0000 7fe907e00640 10 client.27083 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.073+0000 7fe907e00640 1 -- 172.21.15.114:0/3337662883 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b4054e0b0 con 0x564b40674480 2024-09-21T06:59:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.074+0000 7fe907e00640 10 client.27083 handle_command_reply: tid=0 2024-09-21T06:59:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.074+0000 7fe92c48d740 1 client.27083 shutdown 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.075+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 msgr2=0x564b40616ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.075+0000 7fe92c48d740 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b40616ec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9081065c0 tx=0x7fe9080cea90 comp rx=0 tx=0).stop 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 shutdown_connections 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564b40674480 0x564b406768a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564b401ca1c0 0x564b4066c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b402361b0 0x564b40616ec0 secure :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9081065c0 tx=0x7fe9080cea90 comp rx=0 tx=0).stop 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 --2- 172.21.15.114:0/3337662883 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564b402154d0 0x564b406168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 >> 172.21.15.114:0/3337662883 conn(0x564b4022a300 msgr2=0x564b405e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:53.078 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe907e00640 0 client.27083 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 shutdown_connections 2024-09-21T06:59:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.076+0000 7fe92c48d740 1 -- 172.21.15.114:0/3337662883 wait complete. 2024-09-21T06:59:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.079+0000 7fe92c48d740 20 client.27083 trim_cache size 0 max 16384 2024-09-21T06:59:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T06:59:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T06:59:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T06:59:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.080+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9000b6780 msgr2=0x7fe9000b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9000b6780 0x7fe9000b8c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe91800aa90 tx=0x7fe918009250 comp rx=0 tx=0).stop 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 msgr2=0x7fe9201a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 0x7fe9201a2ac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9080052f0 tx=0x7fe908005320 comp rx=0 tx=0).stop 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 shutdown_connections 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9000b6780 0x7fe9000b8c90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9201a7d50 0x7fe9201a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe920114d60 0x7fe9201a2ac0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 --2- 172.21.15.114:0/1783371517 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe92006ac80 0x7fe9201a2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 >> 172.21.15.114:0/1783371517 conn(0x7fe920071080 msgr2=0x7fe92010d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 shutdown_connections 2024-09-21T06:59:53.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T06:59:53.081+0000 7fe925400640 1 -- 172.21.15.114:0/1783371517 wait complete. 2024-09-21T06:59:53.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:52 smithi183 ceph-mon[34219]: pgmap v568: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 630 op/s 2024-09-21T06:59:53.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:52 smithi193 ceph-mon[34601]: pgmap v568: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 MiB/s wr, 630 op/s 2024-09-21T06:59:54.039 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T06:59:54.039 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T06:59:54.039 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:59:54.271 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:59:54.703 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:55.098 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:54 smithi114 ceph-mon[30654]: pgmap v569: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 728 op/s 2024-09-21T06:59:55.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:54 smithi183 ceph-mon[34219]: pgmap v569: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 728 op/s 2024-09-21T06:59:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:54 smithi193 ceph-mon[34601]: pgmap v569: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 17 MiB/s wr, 728 op/s 2024-09-21T06:59:55.513 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 -- 172.21.15.114:0/4068579096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384102fa0 msgr2=0x7fd38410fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 --2- 172.21.15.114:0/4068579096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384102fa0 0x7fd38410fcc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd370009900 tx=0x7fd37002a480 comp rx=0 tx=0).stop 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 -- 172.21.15.114:0/4068579096 shutdown_connections 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 --2- 172.21.15.114:0/4068579096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384110200 0x7fd384112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 --2- 172.21.15.114:0/4068579096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384102fa0 0x7fd38410fcc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 --2- 172.21.15.114:0/4068579096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd384102650 0x7fd384102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:55.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 -- 172.21.15.114:0/4068579096 >> 172.21.15.114:0/4068579096 conn(0x7fd384079730 msgr2=0x7fd384079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:55.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 -- 172.21.15.114:0/4068579096 shutdown_connections 2024-09-21T06:59:55.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.568+0000 7fd38ac00640 1 -- 172.21.15.114:0/4068579096 wait complete. 2024-09-21T06:59:55.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.569+0000 7fd38ac00640 1 Processor -- start 2024-09-21T06:59:55.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.570+0000 7fd38ac00640 1 -- start start 2024-09-21T06:59:55.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.570+0000 7fd38ac00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd384102650 0x7fd38410d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:55.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.570+0000 7fd38ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 0x7fd38410d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:55.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.571+0000 7fd38ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384110200 0x7fd38410dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:55.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.571+0000 7fd38ac00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd384115cc0 con 0x7fd384102650 2024-09-21T06:59:55.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.571+0000 7fd38ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd384115b40 con 0x7fd384110200 2024-09-21T06:59:55.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.571+0000 7fd38ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd384115e40 con 0x7fd384102fa0 2024-09-21T06:59:55.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.571+0000 7fd383400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 0x7fd38410d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:55.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.572+0000 7fd383400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 0x7fd38410d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36328/0 (socket says 172.21.15.114:36328) 2024-09-21T06:59:55.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.572+0000 7fd383400640 1 -- 172.21.15.114:0/3573873322 learned_addr learned my addr 172.21.15.114:0/3573873322 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:55.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.572+0000 7fd388c00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384110200 0x7fd38410dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:55.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd383400640 1 -- 172.21.15.114:0/3573873322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384110200 msgr2=0x7fd38410dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:55.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd383400640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384110200 0x7fd38410dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:55.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd383400640 1 -- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd384102650 msgr2=0x7fd38410d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:55.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd383400640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd384102650 0x7fd38410d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:55.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd383400640 1 -- 172.21.15.114:0/3573873322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3841acad0 con 0x7fd384102fa0 2024-09-21T06:59:55.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd388c00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384110200 0x7fd38410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:55.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.573+0000 7fd383400640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 0x7fd38410d660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd370031440 tx=0x7fd370003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:55.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.574+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37003f070 con 0x7fd384102fa0 2024-09-21T06:59:55.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.574+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd370003e40 con 0x7fd384102fa0 2024-09-21T06:59:55.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.574+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370035030 con 0x7fd384102fa0 2024-09-21T06:59:55.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.574+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3841acd60 con 0x7fd384102fa0 2024-09-21T06:59:55.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.574+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3841ad190 con 0x7fd384102fa0 2024-09-21T06:59:55.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.576+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd384102a60 con 0x7fd384102fa0 2024-09-21T06:59:55.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.581+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd370042070 con 0x7fd384102fa0 2024-09-21T06:59:55.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.583+0000 7fd380c00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd3680b6880 0x7fd3680b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:55.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.585+0000 7fd383e00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd3680b6880 0x7fd3680b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:55.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.585+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd3701085f0 con 0x7fd384102fa0 2024-09-21T06:59:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.586+0000 7fd383e00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd3680b6880 0x7fd3680b8d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd38410fb80 tx=0x7fd374005bf0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:55.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:55.586+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd370049030 con 0x7fd384102fa0 2024-09-21T06:59:56.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.034+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd38410e420 con 0x7fd384102fa0 2024-09-21T06:59:56.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.035+0000 7fd380c00640 1 -- 172.21.15.114:0/3573873322 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11999 (secure 0 0 0) 0x7fd3700ce7f0 con 0x7fd384102fa0 2024-09-21T06:59:56.041 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T06:59:56.041 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":39,"btime":"2024-09-21T06:58:51:043259+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:58:50.613377+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T06:59:56.041 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 39 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.041+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd3680b6880 msgr2=0x7fd3680b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.041+0000 7fd38ac00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd3680b6880 0x7fd3680b8d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd38410fb80 tx=0x7fd374005bf0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.041+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 msgr2=0x7fd38410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.041+0000 7fd38ac00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 0x7fd38410d660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd370031440 tx=0x7fd370003cd0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 shutdown_connections 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd3680b6880 0x7fd3680b8d90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd384110200 0x7fd38410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd384102fa0 0x7fd38410d660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 --2- 172.21.15.114:0/3573873322 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd384102650 0x7fd38410d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 >> 172.21.15.114:0/3573873322 conn(0x7fd384079730 msgr2=0x7fd3841016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 shutdown_connections 2024-09-21T06:59:56.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.042+0000 7fd38ac00640 1 -- 172.21.15.114:0/3573873322 wait complete. 2024-09-21T06:59:56.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 -- 172.21.15.114:0/1940738997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf40a2060 msgr2=0x7fcaf40a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 --2- 172.21.15.114:0/1940738997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf40a2060 0x7fcaf40a2470 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009900 tx=0x7fcae802a420 comp rx=0 tx=0).stop 2024-09-21T06:59:56.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 -- 172.21.15.114:0/1940738997 shutdown_connections 2024-09-21T06:59:56.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 --2- 172.21.15.114:0/1940738997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf40a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 --2- 172.21.15.114:0/1940738997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf40a2060 0x7fcaf40a2470 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 --2- 172.21.15.114:0/1940738997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf409cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 -- 172.21.15.114:0/1940738997 >> 172.21.15.114:0/1940738997 conn(0x7fcaf4008ce0 msgr2=0x7fcaf40090f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 -- 172.21.15.114:0/1940738997 shutdown_connections 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.374+0000 7fcb03000640 1 -- 172.21.15.114:0/1940738997 wait complete. 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.376+0000 7fcb03000640 1 Processor -- start 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.376+0000 7fcb03000640 1 -- start start 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.377+0000 7fcb03000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf4015bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb03000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf400ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb03000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf400f480 0x7fcaf40147a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf4015bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf4015bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36344/0 (socket says 172.21.15.114:36344) 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb01200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf400ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb01200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf400ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34886/0 (socket says 172.21.15.114:34886) 2024-09-21T06:59:56.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb03000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40ac700 con 0x7fcaf400f480 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb03000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40ac580 con 0x7fcaf40a29b0 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb03000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf40ac880 con 0x7fcaf409c720 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb01c00640 1 -- 172.21.15.114:0/1697737859 learned_addr learned my addr 172.21.15.114:0/1697737859 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.378+0000 7fcb02600640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf400f480 0x7fcaf40147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb02600640 1 -- 172.21.15.114:0/1697737859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 msgr2=0x7fcaf4015bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb02600640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf4015bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb02600640 1 -- 172.21.15.114:0/1697737859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 msgr2=0x7fcaf400ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb02600640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf400ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb02600640 1 -- 172.21.15.114:0/1697737859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf4014ce0 con 0x7fcaf400f480 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb01c00640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf4015bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.379+0000 7fcb01200640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf400ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.380+0000 7fcb02600640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf400f480 0x7fcaf40147a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcafc069e80 tx=0x7fcafc06b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.381+0000 7fcafa000640 1 -- 172.21.15.114:0/1697737859 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc084070 con 0x7fcaf400f480 2024-09-21T06:59:56.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.381+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf4014fd0 con 0x7fcaf400f480 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.381+0000 7fcafa000640 1 -- 172.21.15.114:0/1697737859 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcafc080ae0 con 0x7fcaf400f480 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.381+0000 7fcafa000640 1 -- 172.21.15.114:0/1697737859 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc07fa30 con 0x7fcaf400f480 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.381+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf4015490 con 0x7fcaf400f480 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.386+0000 7fcb0b248740 1 Processor -- start 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.396+0000 7fcafa000640 1 -- 172.21.15.114:0/1697737859 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fcafc087070 con 0x7fcaf400f480 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 -- start start 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa08650d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa086512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55aa085c7fb0 con 0x55aa08289390 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aa085b73e0 con 0x55aa082811b0 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.398+0000 7fcb0b248740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aa085bb1f0 con 0x55aa082aa070 2024-09-21T06:59:56.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.399+0000 7fcafa000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcae40b6730 0x7fcae40b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.399+0000 7fcafa000640 1 -- 172.21.15.114:0/1697737859 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcafc155b10 con 0x7fcaf400f480 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.401+0000 7fcb02600640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcae40b6730 0x7fcae40b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.401+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa086512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.401+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa086512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34892/0 (socket says 172.21.15.114:34892) 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.401+0000 7fcb01c00640 1 -- 172.21.15.114:0/3440415756 learned_addr learned my addr 172.21.15.114:0/3440415756 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.403+0000 7fcb02600640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcae40b6730 0x7fcae40b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0692a0 tx=0x7fcafc068f60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.404+0000 7fcb01200640 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.404+0000 7fcb01200640 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa08650d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.406+0000 7fcb01c00640 1 -- 172.21.15.114:0/3440415756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 msgr2=0x55aa08650d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.406+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa08650d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.406+0000 7fcb01c00640 1 -- 172.21.15.114:0/3440415756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 msgr2=0x55aa086517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.406+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.406+0000 7fcb01c00640 1 -- 172.21.15.114:0/3440415756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa08581cd0 con 0x55aa082811b0 2024-09-21T06:59:56.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.408+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa086512a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009bd0 tx=0x7fcaf0009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680135e32eee6f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.408+0000 7fcaeea00640 1 -- 172.21.15.114:0/3440415756 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0017020 con 0x55aa082811b0 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.408+0000 7fcaeea00640 1 -- 172.21.15.114:0/3440415756 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf000ebe0 con 0x55aa082811b0 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.408+0000 7fcaeea00640 1 -- 172.21.15.114:0/3440415756 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000f040 con 0x55aa082811b0 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.411+0000 7fcb0b248740 1 -- 172.21.15.114:0/3440415756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 msgr2=0x55aa086512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.411+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa086512a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009bd0 tx=0x7fcaf0009190 comp rx=0 tx=0).stop 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.411+0000 7fcb0b248740 1 -- 172.21.15.114:0/3440415756 shutdown_connections 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.412+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.412+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa086512a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.412+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3440415756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa08650d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.412+0000 7fcb0b248740 1 -- 172.21.15.114:0/3440415756 >> 172.21.15.114:0/3440415756 conn(0x55aa0829e460 msgr2=0x55aa0864ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.412+0000 7fcb0b248740 1 -- 172.21.15.114:0/3440415756 shutdown_connections 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.412+0000 7fcb0b248740 1 -- 172.21.15.114:0/3440415756 wait complete. 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.413+0000 7fcb0b248740 1 -- start start 2024-09-21T06:59:56.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.414+0000 7fcb0b248740 1 Processor -- start 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082811b0 0x55aa0865c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 0x55aa0865cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa086798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55aa08418700 con 0x55aa082811b0 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aa08552580 con 0x55aa08289390 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aa08670550 con 0x55aa082aa070 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb0b248740 10 client.0 _ll_register_callbacks cb 0x55aa08562470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.415+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 0x55aa0865cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.416+0000 7fcb01c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 0x55aa0865cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34902/0 (socket says 172.21.15.114:34902) 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.416+0000 7fcb01c00640 1 -- 172.21.15.114:0/3142131487 learned_addr learned my addr 172.21.15.114:0/3142131487 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.416+0000 7fcb01200640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa086798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01c00640 1 -- 172.21.15.114:0/3142131487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 msgr2=0x55aa086798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa086798d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01c00640 1 -- 172.21.15.114:0/3142131487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082811b0 msgr2=0x55aa0865c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082811b0 0x55aa0865c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01c00640 1 -- 172.21.15.114:0/3142131487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa08315650 con 0x55aa08289390 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01200640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa086798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 0x55aa0865cdd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0013c40 tx=0x7fcaf0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfe5fe1cc9c32f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.418+0000 7fcae3400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.419+0000 7fcae3400640 1 -- 172.21.15.114:0/3142131487 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0017050 con 0x55aa08289390 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.419+0000 7fcae3400640 1 -- 172.21.15.114:0/3142131487 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcaf0013820 con 0x55aa08289390 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.420+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa08554b90 con 0x55aa08289390 2024-09-21T06:59:56.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.421+0000 7fcae3400640 1 -- 172.21.15.114:0/3142131487 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcaf0027480 con 0x55aa08289390 2024-09-21T06:59:56.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.422+0000 7fcb0b248740 10 client.27101 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.422+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55aa0856f950 con 0x55aa08289390 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.423+0000 7fcae3400640 1 -- 172.21.15.114:0/3142131487 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa08554b90 con 0x55aa08289390 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.423+0000 7fcae3400640 1 -- 172.21.15.114:0/3142131487 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fcaf0012050 con 0x55aa08289390 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb0b248740 10 client.27101 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb0b248740 10 client.27101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb0b248740 20 client.27101 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa082689f0 0x55aa0867f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb0b248740 4 client.27101 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa085c9f00 con 0x55aa082689f0 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.424+0000 7fcb01200640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa082689f0 0x55aa0867f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.425+0000 7fcb01200640 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa082689f0 0x55aa0867f740 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec65fc6a85c4ecf server_cookie=e573721466c22e6 in_seq=0 out_seq=0 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.425+0000 7fcae3400640 10 client.27101 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:56.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.426+0000 7fcae3400640 1 -- 172.21.15.114:0/3142131487 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aa085c9f00 con 0x55aa082689f0 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.426+0000 7fcae3400640 10 client.27101 handle_command_reply: tid=0 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.426+0000 7fcb0b248740 1 client.27101 shutdown 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.427+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 msgr2=0x55aa0865cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.427+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 0x55aa0865cdd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0013c40 tx=0x7fcaf0004770 comp rx=0 tx=0).stop 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 shutdown_connections 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa082689f0 0x55aa0867f740 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082aa070 0x55aa086798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa08289390 0x55aa0865cdd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3142131487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082811b0 0x55aa0865c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 >> 172.21.15.114:0/3142131487 conn(0x55aa0829e460 msgr2=0x55aa08653ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcae3400640 0 client.27101 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.428+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 shutdown_connections 2024-09-21T06:59:56.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.429+0000 7fcb0b248740 1 -- 172.21.15.114:0/3142131487 wait complete. 2024-09-21T06:59:56.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.429+0000 7fcb0b248740 20 client.27101 trim_cache size 0 max 16384 2024-09-21T06:59:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.462+0000 7fcb0b248740 1 Processor -- start 2024-09-21T06:59:56.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.462+0000 7fcb0b248740 1 -- start start 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.462+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 0x55aa0864eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082aa070 0x55aa08679d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb0b248740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf41527a0 con 0x55aa08289390 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb0b248740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4154400 con 0x55aa082aa070 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb0b248740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4152e70 con 0x55aa082811b0 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb01200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 0x55aa0864eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb01200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 0x55aa0864eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36358/0 (socket says 172.21.15.114:36358) 2024-09-21T06:59:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.463+0000 7fcb01200640 1 -- 172.21.15.114:0/600681308 learned_addr learned my addr 172.21.15.114:0/600681308 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.464+0000 7fcb01200640 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082aa070 0x55aa08679d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.464+0000 7fcb01200640 1 -- 172.21.15.114:0/600681308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082aa070 msgr2=0x55aa08679d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.464+0000 7fcb01200640 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082aa070 0x55aa08679d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.464+0000 7fcb01200640 1 -- 172.21.15.114:0/600681308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 msgr2=0x55aa086797d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.464+0000 7fcb01200640 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.464+0000 7fcb01200640 1 -- 172.21.15.114:0/600681308 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa086535f0 con 0x55aa082811b0 2024-09-21T06:59:56.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.465+0000 7fcb01200640 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 0x55aa0864eb60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcae80096c0 tx=0x7fcae8004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc482feb5d9dd0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:56.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.468+0000 7fcaee000640 1 -- 172.21.15.114:0/600681308 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae804c040 con 0x55aa082811b0 2024-09-21T06:59:56.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.468+0000 7fcaee000640 1 -- 172.21.15.114:0/600681308 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8004c30 con 0x55aa082811b0 2024-09-21T06:59:56.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.468+0000 7fcaee000640 1 -- 172.21.15.114:0/600681308 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035040 con 0x55aa082811b0 2024-09-21T06:59:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.472+0000 7fcb0b248740 1 -- 172.21.15.114:0/600681308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 msgr2=0x55aa0864eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.472+0000 7fcb0b248740 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 0x55aa0864eb60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcae80096c0 tx=0x7fcae8004490 comp rx=0 tx=0).stop 2024-09-21T06:59:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.475+0000 7fcb0b248740 1 -- 172.21.15.114:0/600681308 shutdown_connections 2024-09-21T06:59:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.475+0000 7fcb0b248740 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082aa070 0x55aa08679d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.475+0000 7fcb0b248740 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa08289390 0x55aa086797d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.475+0000 7fcb0b248740 1 --2- 172.21.15.114:0/600681308 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa082811b0 0x55aa0864eb60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.475+0000 7fcb0b248740 1 -- 172.21.15.114:0/600681308 >> 172.21.15.114:0/600681308 conn(0x55aa0829e460 msgr2=0x55aa0864e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.476+0000 7fcb0b248740 1 -- 172.21.15.114:0/600681308 shutdown_connections 2024-09-21T06:59:56.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.476+0000 7fcb0b248740 1 -- 172.21.15.114:0/600681308 wait complete. 2024-09-21T06:59:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.477+0000 7fcb0b248740 1 -- start start 2024-09-21T06:59:56.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.477+0000 7fcb0b248740 1 Processor -- start 2024-09-21T06:59:56.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.480+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa08675b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.481+0000 7fcb01200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa08675b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.481+0000 7fcb01200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa08675b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34922/0 (socket says 172.21.15.114:34922) 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.481+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa08289390 0x55aa08676160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.482+0000 7fcb0b248740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082aa070 0x55aa086708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.482+0000 7fcb0b248740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4153c70 con 0x55aa082aa070 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.482+0000 7fcb0b248740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4153540 con 0x55aa082811b0 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.482+0000 7fcb0b248740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf4150780 con 0x55aa08289390 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.482+0000 7fcb0b248740 10 client.0 _ll_register_callbacks cb 0x7fcaf0009200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.482+0000 7fcb01200640 1 -- 172.21.15.114:0/3103124196 learned_addr learned my addr 172.21.15.114:0/3103124196 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa08289390 0x55aa08676160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01200640 1 -- 172.21.15.114:0/3103124196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa08289390 msgr2=0x55aa08676160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01200640 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa08289390 0x55aa08676160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01200640 1 -- 172.21.15.114:0/3103124196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082aa070 msgr2=0x55aa086708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:59:56.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01200640 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082aa070 0x55aa086708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01200640 1 -- 172.21.15.114:0/3103124196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa08671040 con 0x55aa082811b0 2024-09-21T06:59:56.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.483+0000 7fcb01200640 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa08675b80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcae802a460 tx=0x7fcae80055c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ecf5ff6c60eecb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:59:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcae3400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T06:59:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcae3400640 1 -- 172.21.15.114:0/3103124196 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035040 con 0x55aa082811b0 2024-09-21T06:59:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcae3400640 1 -- 172.21.15.114:0/3103124196 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcae80035f0 con 0x55aa082811b0 2024-09-21T06:59:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae8004c30 con 0x55aa082811b0 2024-09-21T06:59:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcae3400640 1 -- 172.21.15.114:0/3103124196 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcae805f050 con 0x55aa082811b0 2024-09-21T06:59:56.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcb0b248740 10 client.27107 fetch_fsmap learned FSMap version 39 2024-09-21T06:59:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.489+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55aa086ea9d0 con 0x55aa082811b0 2024-09-21T06:59:56.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.490+0000 7fcae3400640 1 -- 172.21.15.114:0/3103124196 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcae8004c30 con 0x55aa082811b0 2024-09-21T06:59:56.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcae3400640 1 -- 172.21.15.114:0/3103124196 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 39) ==== 8394+0+0 (secure 0 0 0) 0x7fcae8031950 con 0x55aa082811b0 2024-09-21T06:59:56.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcb0b248740 10 client.27107 fetch_fsmap finished waiting for FSMap version 39 2024-09-21T06:59:56.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcb0b248740 10 client.27107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:59:56.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcb0b248740 20 client.27107 populate_metadata read hostname 'smithi114' 2024-09-21T06:59:56.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa0823dda0 0x55aa086ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:59:56.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcb0b248740 4 client.27107 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:59:56.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.491+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa085c59b0 con 0x55aa0823dda0 2024-09-21T06:59:56.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.493+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa0823dda0 0x55aa086ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:59:56.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.494+0000 7fcb01c00640 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa0823dda0 0x55aa086ecfc0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b52676e10dc87c server_cookie=c4c0bac67f4ceb30 in_seq=0 out_seq=0 2024-09-21T06:59:56.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.494+0000 7fcae3400640 10 client.27107 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:56.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.494+0000 7fcae3400640 1 -- 172.21.15.114:0/3103124196 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa085c59b0 con 0x55aa0823dda0 2024-09-21T06:59:56.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.494+0000 7fcae3400640 10 client.27107 handle_command_reply: tid=0 2024-09-21T06:59:56.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.494+0000 7fcb0b248740 1 client.27107 shutdown 2024-09-21T06:59:56.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 msgr2=0x55aa08675b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa08675b80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcae802a460 tx=0x7fcae80055c0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 shutdown_connections 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aa082aa070 0x55aa086708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aa0823dda0 0x55aa086ecfc0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aa08289390 0x55aa08676160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 --2- 172.21.15.114:0/3103124196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aa082811b0 0x55aa08675b80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 >> 172.21.15.114:0/3103124196 conn(0x55aa0829e460 msgr2=0x55aa08651d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcae3400640 0 client.27107 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.495+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 shutdown_connections 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.496+0000 7fcb0b248740 1 -- 172.21.15.114:0/3103124196 wait complete. 2024-09-21T06:59:56.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.496+0000 7fcb0b248740 20 client.27107 trim_cache size 0 max 16384 2024-09-21T06:59:56.499 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T06:59:56.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.497+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcae40b6730 msgr2=0x7fcae40b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.497+0000 7fcb03000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcae40b6730 0x7fcae40b8c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0692a0 tx=0x7fcafc068f60 comp rx=0 tx=0).stop 2024-09-21T06:59:56.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.498+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf400f480 msgr2=0x7fcaf40147a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:59:56.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.498+0000 7fcb03000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf400f480 0x7fcaf40147a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcafc069e80 tx=0x7fcafc06b9b0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.500+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 shutdown_connections 2024-09-21T06:59:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.500+0000 7fcb03000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcaf400f480 0x7fcaf40147a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.500+0000 7fcb03000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcae40b6730 0x7fcae40b8c40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.500+0000 7fcb03000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf40a29b0 0x7fcaf400ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.500+0000 7fcb03000640 1 --2- 172.21.15.114:0/1697737859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcaf409c720 0x7fcaf4015bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:59:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.500+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 >> 172.21.15.114:0/1697737859 conn(0x7fcaf4008ce0 msgr2=0x7fcaf4005400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:59:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.501+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 shutdown_connections 2024-09-21T06:59:56.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T06:59:56.501+0000 7fcb03000640 1 -- 172.21.15.114:0/1697737859 wait complete. 2024-09-21T06:59:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:56 smithi183 ceph-mon[34219]: pgmap v570: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 546 op/s 2024-09-21T06:59:57.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:56 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3573873322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:56 smithi193 ceph-mon[34601]: pgmap v570: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 546 op/s 2024-09-21T06:59:57.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:56 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3573873322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:57.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:56 smithi114 ceph-mon[30654]: pgmap v570: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 546 op/s 2024-09-21T06:59:57.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:56 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3573873322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:59:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 320.0 s, 596.6 tps, lat 173.794 ms stddev 337.725 2024-09-21T06:59:58.509 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T06:59:58.510 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T06:59:59.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:58 smithi193 ceph-mon[34601]: pgmap v571: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-21T06:59:59.241 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:59:59.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:58 smithi114 ceph-mon[30654]: pgmap v571: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-21T06:59:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:58 smithi183 ceph-mon[34219]: pgmap v571: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 591 op/s 2024-09-21T07:00:00.076 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:00:00.077 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:00:00.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.127+0000 7f6d1e400640 1 -- 172.21.15.114:0/743641390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810a3c0 msgr2=0x7f6d1810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.128+0000 7f6d1e400640 1 --2- 172.21.15.114:0/743641390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810a3c0 0x7f6d1810a7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d000074e0 tx=0x7f6d0002b070 comp rx=0 tx=0).stop 2024-09-21T07:00:00.131 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 06:59:59 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:00.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.129+0000 7f6d1e400640 1 -- 172.21.15.114:0/743641390 shutdown_connections 2024-09-21T07:00:00.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.129+0000 7f6d1e400640 1 --2- 172.21.15.114:0/743641390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810ad10 0x7f6d18116fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.129+0000 7f6d1e400640 1 --2- 172.21.15.114:0/743641390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810a3c0 0x7f6d1810a7d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.129+0000 7f6d1e400640 1 --2- 172.21.15.114:0/743641390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810f160 0x7f6d1810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.129+0000 7f6d1e400640 1 -- 172.21.15.114:0/743641390 >> 172.21.15.114:0/743641390 conn(0x7f6d180712b0 msgr2=0x7f6d18073e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:00.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.130+0000 7f6d1e400640 1 -- 172.21.15.114:0/743641390 shutdown_connections 2024-09-21T07:00:00.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.130+0000 7f6d1e400640 1 -- 172.21.15.114:0/743641390 wait complete. 2024-09-21T07:00:00.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.132+0000 7f6d1e400640 1 Processor -- start 2024-09-21T07:00:00.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.135+0000 7f6d1e400640 1 -- start start 2024-09-21T07:00:00.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.136+0000 7f6d1e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 0x7f6d181a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.136+0000 7f6d1e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810ad10 0x7f6d181a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.137+0000 7f6d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 0x7f6d181a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.137+0000 7f6d1e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810f160 0x7f6d181aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.137+0000 7f6d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 0x7f6d181a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36372/0 (socket says 172.21.15.114:36372) 2024-09-21T07:00:00.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.138+0000 7f6d1e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1811a0d0 con 0x7f6d1810f160 2024-09-21T07:00:00.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.138+0000 7f6d1e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18119f50 con 0x7f6d1810ad10 2024-09-21T07:00:00.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.138+0000 7f6d17400640 1 -- 172.21.15.114:0/1239746004 learned_addr learned my addr 172.21.15.114:0/1239746004 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:00.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.139+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1811a250 con 0x7f6d1810a3c0 2024-09-21T07:00:00.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.139+0000 7f6d16a00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810ad10 0x7f6d181a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.139+0000 7f6d17e00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810f160 0x7f6d181aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.140+0000 7f6d17400640 1 -- 172.21.15.114:0/1239746004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810ad10 msgr2=0x7f6d181a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.141+0000 7f6d17400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810ad10 0x7f6d181a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.142+0000 7f6d17400640 1 -- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810f160 msgr2=0x7f6d181aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.142+0000 7f6d17400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810f160 0x7f6d181aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.142+0000 7f6d17400640 1 -- 172.21.15.114:0/1239746004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d181af280 con 0x7f6d1810a3c0 2024-09-21T07:00:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.142+0000 7f6d16a00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810ad10 0x7f6d181a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.142+0000 7f6d17e00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810f160 0x7f6d181aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.142+0000 7f6d17400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 0x7f6d181a9310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000db80 tx=0x7f6d1000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.143+0000 7f6d0fe00640 1 -- 172.21.15.114:0/1239746004 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10017070 con 0x7f6d1810a3c0 2024-09-21T07:00:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.143+0000 7f6d0fe00640 1 -- 172.21.15.114:0/1239746004 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d10003b70 con 0x7f6d1810a3c0 2024-09-21T07:00:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.143+0000 7f6d0fe00640 1 -- 172.21.15.114:0/1239746004 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100106c0 con 0x7f6d1810a3c0 2024-09-21T07:00:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.144+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d181af570 con 0x7f6d1810a3c0 2024-09-21T07:00:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.144+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d181b5cd0 con 0x7f6d1810a3c0 2024-09-21T07:00:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.148+0000 7f6d0fe00640 1 -- 172.21.15.114:0/1239746004 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6d10004020 con 0x7f6d1810a3c0 2024-09-21T07:00:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.149+0000 7f6d25057740 1 Processor -- start 2024-09-21T07:00:00.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.149+0000 7f6d25057740 1 -- start start 2024-09-21T07:00:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.150+0000 7f6d25057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362acc850 0x565362e739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.150+0000 7f6d25057740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e73ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.150+0000 7f6d25057740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e74430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.150+0000 7f6d25057740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x565362de9a90 con 0x565362aabb70 2024-09-21T07:00:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.150+0000 7f6d25057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565362dd8ec0 con 0x565362acc850 2024-09-21T07:00:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.150+0000 7f6d25057740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565362ddccd0 con 0x565362aa3990 2024-09-21T07:00:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.151+0000 7f6d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e73ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.151+0000 7f6d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e73ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36386/0 (socket says 172.21.15.114:36386) 2024-09-21T07:00:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.151+0000 7f6d17400640 1 -- 172.21.15.114:0/1853192454 learned_addr learned my addr 172.21.15.114:0/1853192454 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.151+0000 7f6d17e00640 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362acc850 0x565362e739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.151+0000 7f6d0fe00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cfc0b67b0 0x7f6cfc0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.153 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.151+0000 7f6d16a00640 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e74430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d0fe00640 1 -- 172.21.15.114:0/1239746004 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6d100e78b0 con 0x7f6d1810a3c0 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17400640 1 -- 172.21.15.114:0/1853192454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362acc850 msgr2=0x565362e739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17400640 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362acc850 0x565362e739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17400640 1 -- 172.21.15.114:0/1853192454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 msgr2=0x565362e74430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17400640 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e74430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17400640 1 -- 172.21.15.114:0/1853192454 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565362d7e9d0 con 0x565362aa3990 2024-09-21T07:00:00.154 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17e00640 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362acc850 0x565362e739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.152+0000 7f6d17e00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cfc0b67b0 0x7f6cfc0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.153+0000 7f6d17400640 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e73ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6d100aa600 tx=0x7f6d10000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d44e9c2b98cc83b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:00.155 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.153+0000 7f6d0cc00640 1 -- 172.21.15.114:0/1853192454 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100acb00 con 0x565362aa3990 2024-09-21T07:00:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.154+0000 7f6d0cc00640 1 -- 172.21.15.114:0/1853192454 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d100acea0 con 0x565362aa3990 2024-09-21T07:00:00.156 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.154+0000 7f6d0cc00640 1 -- 172.21.15.114:0/1853192454 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100a9600 con 0x565362aa3990 2024-09-21T07:00:00.161 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.160+0000 7f6d25057740 1 -- 172.21.15.114:0/1853192454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 msgr2=0x565362e73ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 06:59:59 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.160+0000 7f6d25057740 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e73ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6d100aa600 tx=0x7f6d10000f40 comp rx=0 tx=0).stop 2024-09-21T07:00:00.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.160+0000 7f6d17e00640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cfc0b67b0 0x7f6cfc0b8cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009d60 tx=0x7f6d080046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 -- 172.21.15.114:0/1853192454 shutdown_connections 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e74430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e73ef0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 --2- 172.21.15.114:0/1853192454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362acc850 0x565362e739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 -- 172.21.15.114:0/1853192454 >> 172.21.15.114:0/1853192454 conn(0x565362ac0c40 msgr2=0x565362e718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 -- 172.21.15.114:0/1853192454 shutdown_connections 2024-09-21T07:00:00.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.161+0000 7f6d25057740 1 -- 172.21.15.114:0/1853192454 wait complete. 2024-09-21T07:00:00.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.162+0000 7f6d25057740 1 -- start start 2024-09-21T07:00:00.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.162+0000 7f6d25057740 1 Processor -- start 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e7f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e7f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362acc850 0x565362e8b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x565362c3a7b0 con 0x565362acc850 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565362d7d650 con 0x565362aabb70 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565362e82130 con 0x565362aa3990 2024-09-21T07:00:00.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.163+0000 7f6d25057740 10 client.0 _ll_register_callbacks cb 0x565362ae40a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e7f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362acc850 0x565362e8b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e7f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36402/0 (socket says 172.21.15.114:36402) 2024-09-21T07:00:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17400640 1 -- 172.21.15.114:0/58536811 learned_addr learned my addr 172.21.15.114:0/58536811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d16a00640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e7f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17400640 1 -- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 msgr2=0x565362e7f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17400640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e7f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.165+0000 7f6d17400640 1 -- 172.21.15.114:0/58536811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362acc850 msgr2=0x565362e8b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.166+0000 7f6d17400640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362acc850 0x565362e8b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.166+0000 7f6d17400640 1 -- 172.21.15.114:0/58536811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565362b46ba0 con 0x565362aa3990 2024-09-21T07:00:00.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.166+0000 7f6d17e00640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362acc850 0x565362e8b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:00.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.168+0000 7f6d17400640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e7f4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6d100e5290 tx=0x7f6d10003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0002b3ad62f2650 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:00.173 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.168+0000 7f6d05600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.168+0000 7f6d05600640 1 -- 172.21.15.114:0/58536811 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100b1e70 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.168+0000 7f6d05600640 1 -- 172.21.15.114:0/58536811 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6d10098580 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.168+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d100acea0 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.169+0000 7f6d05600640 1 -- 172.21.15.114:0/58536811 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6d100ada90 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.169+0000 7f6d25057740 10 client.27076 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.169+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565362e9f040 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.169+0000 7f6d05600640 1 -- 172.21.15.114:0/58536811 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d100acea0 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.169+0000 7f6d05600640 1 -- 172.21.15.114:0/58536811 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f6d100ac5f0 con 0x565362aa3990 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d25057740 10 client.27076 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d25057740 10 client.27076 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d25057740 20 client.27076 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d25057740 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a8b1d0 0x565362ea26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d25057740 4 client.27076 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:00.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565362deb9e0 con 0x565362a8b1d0 2024-09-21T07:00:00.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.170+0000 7f6d16a00640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a8b1d0 0x565362ea26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.171+0000 7f6d05600640 10 client.27076 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:00.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.171+0000 7f6d16a00640 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a8b1d0 0x565362ea26c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75c44171787331a server_cookie=b10271cf6af4d73d in_seq=0 out_seq=0 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.172+0000 7f6d05600640 1 -- 172.21.15.114:0/58536811 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565362deb9e0 con 0x565362a8b1d0 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.172+0000 7f6d05600640 10 client.27076 handle_command_reply: tid=0 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.172+0000 7f6d25057740 1 client.27076 shutdown 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 msgr2=0x565362e7f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e7f4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6d100e5290 tx=0x7f6d10003f50 comp rx=0 tx=0).stop 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 shutdown_connections 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362acc850 0x565362e8b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e7f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a8b1d0 0x565362ea26c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 --2- 172.21.15.114:0/58536811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362aa3990 0x565362e7f4a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 >> 172.21.15.114:0/58536811 conn(0x565362ac0c40 msgr2=0x565362e76b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.173+0000 7f6d05600640 0 client.27076 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.174+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 shutdown_connections 2024-09-21T07:00:00.176 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.174+0000 7f6d25057740 1 -- 172.21.15.114:0/58536811 wait complete. 2024-09-21T07:00:00.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.174+0000 7f6d25057740 20 client.27076 trim_cache size 0 max 16384 2024-09-21T07:00:00.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.212+0000 7f6d25057740 1 Processor -- start 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 -- start start 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aa3990 0x565362e71150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e71690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e7c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b7650 con 0x565362aa3990 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b92e0 con 0x565362aabb70 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.213+0000 7f6d25057740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b7d20 con 0x565362acc850 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.214+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e7c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.214+0000 7f6d17e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e7c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36404/0 (socket says 172.21.15.114:36404) 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.214+0000 7f6d17e00640 1 -- 172.21.15.114:0/387233060 learned_addr learned my addr 172.21.15.114:0/387233060 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:00.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.214+0000 7f6d17400640 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aa3990 0x565362e71150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.215+0000 7f6d16a00640 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e71690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.215+0000 7f6d17e00640 1 -- 172.21.15.114:0/387233060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 msgr2=0x565362e71690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.215+0000 7f6d17e00640 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e71690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.215+0000 7f6d17e00640 1 -- 172.21.15.114:0/387233060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aa3990 msgr2=0x565362e71150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.215+0000 7f6d17e00640 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aa3990 0x565362e71150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.215+0000 7f6d17e00640 1 -- 172.21.15.114:0/387233060 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565362e7aaf0 con 0x565362acc850 2024-09-21T07:00:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.216+0000 7f6d17e00640 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e7c490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d0801c700 tx=0x7f6d08015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd44a23014f206e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.217+0000 7f6d07e00640 1 -- 172.21.15.114:0/387233060 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0801c0d0 con 0x565362acc850 2024-09-21T07:00:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.217+0000 7f6d07e00640 1 -- 172.21.15.114:0/387233060 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0800f8f0 con 0x565362acc850 2024-09-21T07:00:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.217+0000 7f6d07e00640 1 -- 172.21.15.114:0/387233060 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0801a5f0 con 0x565362acc850 2024-09-21T07:00:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 -- 172.21.15.114:0/387233060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 msgr2=0x565362e7c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e7c490 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d0801c700 tx=0x7f6d08015000 comp rx=0 tx=0).stop 2024-09-21T07:00:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 -- 172.21.15.114:0/387233060 shutdown_connections 2024-09-21T07:00:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e7c490 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aabb70 0x565362e71690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 --2- 172.21.15.114:0/387233060 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aa3990 0x565362e71150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.219+0000 7f6d25057740 1 -- 172.21.15.114:0/387233060 >> 172.21.15.114:0/387233060 conn(0x565362ac0c40 msgr2=0x565362e7db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.222+0000 7f6d25057740 1 -- 172.21.15.114:0/387233060 shutdown_connections 2024-09-21T07:00:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.222+0000 7f6d25057740 1 -- 172.21.15.114:0/387233060 wait complete. 2024-09-21T07:00:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.222+0000 7f6d25057740 1 -- start start 2024-09-21T07:00:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.223+0000 7f6d25057740 1 Processor -- start 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.223+0000 7f6d25057740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aa3990 0x565362e873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d25057740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e87920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d25057740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d25057740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b8b50 con 0x565362aabb70 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d25057740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b83f0 con 0x565362aa3990 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d25057740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181a7500 con 0x565362acc850 2024-09-21T07:00:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d25057740 10 client.0 _ll_register_callbacks cb 0x7f6d100b1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:00.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d17400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aa3990 0x565362e873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d17400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aa3990 0x565362e873e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34966/0 (socket says 172.21.15.114:34966) 2024-09-21T07:00:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.224+0000 7f6d17400640 1 -- 172.21.15.114:0/618136038 learned_addr learned my addr 172.21.15.114:0/618136038 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.225+0000 7f6d16a00640 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e87920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.225+0000 7f6d16a00640 1 -- 172.21.15.114:0/618136038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 msgr2=0x565362e82390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:00:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.225+0000 7f6d16a00640 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e82390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.225+0000 7f6d16a00640 1 -- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aa3990 msgr2=0x565362e873e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.225+0000 7f6d16a00640 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aa3990 0x565362e873e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.225+0000 7f6d16a00640 1 -- 172.21.15.114:0/618136038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565362e88090 con 0x565362aabb70 2024-09-21T07:00:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.226+0000 7f6d16a00640 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e87920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6d0002b590 tx=0x7f6d0002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b690deeee801453 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.226+0000 7f6d05600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:00.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.227+0000 7f6d05600640 1 -- 172.21.15.114:0/618136038 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0004e060 con 0x565362aabb70 2024-09-21T07:00:00.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.227+0000 7f6d05600640 1 -- 172.21.15.114:0/618136038 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6d00038040 con 0x565362aabb70 2024-09-21T07:00:00.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.227+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d0800f8f0 con 0x565362aabb70 2024-09-21T07:00:00.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.228+0000 7f6d05600640 1 -- 172.21.15.114:0/618136038 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6d0002baa0 con 0x565362aabb70 2024-09-21T07:00:00.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.228+0000 7f6d25057740 10 client.16815 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:00.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.228+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565362f0d5a0 con 0x565362aabb70 2024-09-21T07:00:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.229+0000 7f6d05600640 1 -- 172.21.15.114:0/618136038 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d0800f8f0 con 0x565362aabb70 2024-09-21T07:00:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.229+0000 7f6d05600640 1 -- 172.21.15.114:0/618136038 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f6d00008040 con 0x565362aabb70 2024-09-21T07:00:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.229+0000 7f6d25057740 10 client.16815 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.230+0000 7f6d25057740 10 client.16815 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.230+0000 7f6d25057740 20 client.16815 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.230+0000 7f6d25057740 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a60580 0x565362f0fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.230+0000 7f6d25057740 4 client.16815 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T07:00:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.230+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x565362de7490 con 0x565362a60580 2024-09-21T07:00:00.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.231+0000 7f6d17400640 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a60580 0x565362f0fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.232+0000 7f6d17400640 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a60580 0x565362f0fdc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1353864b54121a2 server_cookie=e37cec262ef77077 in_seq=0 out_seq=0 2024-09-21T07:00:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.232+0000 7f6d05600640 10 client.16815 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:00.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 06:59:59 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:00.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d05600640 1 -- 172.21.15.114:0/618136038 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565362de7490 con 0x565362a60580 2024-09-21T07:00:00.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d05600640 10 client.16815 handle_command_reply: tid=0 2024-09-21T07:00:00.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 client.16815 shutdown 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 msgr2=0x565362e87920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e87920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6d0002b590 tx=0x7f6d0002b9b0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 shutdown_connections 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x565362a60580 0x565362f0fdc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565362acc850 0x565362e82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x565362aabb70 0x565362e87920 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 --2- 172.21.15.114:0/618136038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565362aa3990 0x565362e873e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 >> 172.21.15.114:0/618136038 conn(0x565362ac0c40 msgr2=0x565362e71150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d05600640 0 client.16815 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.442+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 shutdown_connections 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.443+0000 7f6d25057740 1 -- 172.21.15.114:0/618136038 wait complete. 2024-09-21T07:00:00.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.443+0000 7f6d25057740 20 client.16815 trim_cache size 0 max 16384 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 40, 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 22, 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "505c114e": { 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 22, 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:00:00.446 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 139.2, 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:00.447 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cfc0b67b0 msgr2=0x7f6cfc0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cfc0b67b0 0x7f6cfc0b8cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009d60 tx=0x7f6d080046c0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 msgr2=0x7f6d181a9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 0x7f6d181a9310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000db80 tx=0x7f6d1000b630 comp rx=0 tx=0).stop 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 shutdown_connections 2024-09-21T07:00:00.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d1810f160 0x7f6d181aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cfc0b67b0 0x7f6cfc0b8cc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d1810ad10 0x7f6d181a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 --2- 172.21.15.114:0/1239746004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d1810a3c0 0x7f6d181a9310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:00.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 >> 172.21.15.114:0/1239746004 conn(0x7f6d180712b0 msgr2=0x7f6d1810d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:00.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 shutdown_connections 2024-09-21T07:00:00.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:00.444+0000 7f6d1e400640 1 -- 172.21.15.114:0/1239746004 wait complete. 2024-09-21T07:00:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: pgmap v572: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 509 op/s 2024-09-21T07:00:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: overall HEALTH_OK 2024-09-21T07:00:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:00:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.163 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:00 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: pgmap v572: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 509 op/s 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: overall HEALTH_OK 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:00 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: pgmap v572: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 509 op/s 2024-09-21T07:00:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: overall HEALTH_OK 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:00 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:00:01.481 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '505c114e', quiesce took 7.4 seconds. Will release after: 11.85 2024-09-21T07:00:01.507 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: Unable to set osd_memory_target on smithi193 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:00:02.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:01 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: Unable to set osd_memory_target on smithi193 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:00:02.168 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:01 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:00:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T07:00:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-21T07:00:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: Unable to set osd_memory_target on smithi193 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-21T07:00:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:00:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:00:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:00:02.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:01 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:00:02.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 -- 172.21.15.114:0/361321319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 msgr2=0x7fcf1410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 --2- 172.21.15.114:0/361321319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 0x7fcf1410b910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009900 tx=0x7fcefc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 -- 172.21.15.114:0/361321319 shutdown_connections 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 --2- 172.21.15.114:0/361321319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf1410be50 0x7fcf14112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 --2- 172.21.15.114:0/361321319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 0x7fcf1410b910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 --2- 172.21.15.114:0/361321319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf14105bc0 0x7fcf14105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 -- 172.21.15.114:0/361321319 >> 172.21.15.114:0/361321319 conn(0x7fcf14101670 msgr2=0x7fcf14103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.539+0000 7fcf19000640 1 -- 172.21.15.114:0/361321319 shutdown_connections 2024-09-21T07:00:02.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.540+0000 7fcf19000640 1 -- 172.21.15.114:0/361321319 wait complete. 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 Processor -- start 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 -- start start 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 0x7fcf1419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 0x7fcf1419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf1410be50 0x7fcf141a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14115b10 con 0x7fcf14105bc0 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14115990 con 0x7fcf1410b500 2024-09-21T07:00:02.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf19000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14115c90 con 0x7fcf1410be50 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.541+0000 7fcf13400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 0x7fcf1419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 0x7fcf1419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42836/0 (socket says 172.21.15.114:42836) 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 -- 172.21.15.114:0/317223581 learned_addr learned my addr 172.21.15.114:0/317223581 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13e00640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf1410be50 0x7fcf141a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 0x7fcf1419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 -- 172.21.15.114:0/317223581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf1410be50 msgr2=0x7fcf141a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf1410be50 0x7fcf141a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 -- 172.21.15.114:0/317223581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 msgr2=0x7fcf1419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 0x7fcf1419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 -- 172.21.15.114:0/317223581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf141a2470 con 0x7fcf14105bc0 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13e00640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf1410be50 0x7fcf141a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.542+0000 7fcf13400640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 0x7fcf1419c500 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400ea60 tx=0x7fcf0400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.543+0000 7fcf10c00640 1 -- 172.21.15.114:0/317223581 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400cd20 con 0x7fcf14105bc0 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.543+0000 7fcf10c00640 1 -- 172.21.15.114:0/317223581 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf040035f0 con 0x7fcf14105bc0 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.543+0000 7fcf10c00640 1 -- 172.21.15.114:0/317223581 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04010690 con 0x7fcf14105bc0 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.543+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf141a2760 con 0x7fcf14105bc0 2024-09-21T07:00:02.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.543+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a8ec0 con 0x7fcf14105bc0 2024-09-21T07:00:02.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.546+0000 7fcf213f8740 1 Processor -- start 2024-09-21T07:00:02.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.546+0000 7fcf213f8740 1 -- start start 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.546+0000 7fcf10c00640 1 -- 172.21.15.114:0/317223581 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fcf040037a0 con 0x7fcf14105bc0 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.546+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 0x56215d906f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.546+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d907440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.546+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d907980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf213f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56215d87ce90 con 0x56215d53e670 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf213f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56215d86c2c0 con 0x56215d55f350 2024-09-21T07:00:02.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf213f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56215d8700d0 con 0x56215d536490 2024-09-21T07:00:02.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 0x56215d906f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d907440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d907980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 0x56215d906f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35008/0 (socket says 172.21.15.114:35008) 2024-09-21T07:00:02.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 -- 172.21.15.114:0/2711258350 learned_addr learned my addr 172.21.15.114:0/2711258350 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d907980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:42846/0 (socket says 172.21.15.114:42846) 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 -- 172.21.15.114:0/2711258350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 msgr2=0x56215d907440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d907440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 -- 172.21.15.114:0/2711258350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 msgr2=0x56215d907980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d907980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.547+0000 7fcf13e00640 1 -- 172.21.15.114:0/2711258350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215d5ca930 con 0x56215d55f350 2024-09-21T07:00:02.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.548+0000 7fcf13e00640 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 0x56215d906f00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c011810 tx=0x7fcf0c011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e72860447b061de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:02.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.548+0000 7fcf08c00640 1 -- 172.21.15.114:0/2711258350 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c011310 con 0x56215d55f350 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.548+0000 7fcf08c00640 1 -- 172.21.15.114:0/2711258350 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c00f9a0 con 0x56215d55f350 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.548+0000 7fcf08c00640 1 -- 172.21.15.114:0/2711258350 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c005630 con 0x56215d55f350 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.548+0000 7fcf10c00640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcef80b6700 0x7fcef80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf10c00640 1 -- 172.21.15.114:0/317223581 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcf040e7d50 con 0x7fcf14105bc0 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf13400640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcef80b6700 0x7fcef80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 -- 172.21.15.114:0/2711258350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 msgr2=0x56215d906f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 0x56215d906f00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c011810 tx=0x7fcf0c011cd0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 -- 172.21.15.114:0/2711258350 shutdown_connections 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d907980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf13400640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcef80b6700 0x7fcef80b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf040e51e0 tx=0x7fcf040eb8a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d907440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 --2- 172.21.15.114:0/2711258350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d55f350 0x56215d906f00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 -- 172.21.15.114:0/2711258350 >> 172.21.15.114:0/2711258350 conn(0x56215d553740 msgr2=0x56215d904df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:02.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 -- 172.21.15.114:0/2711258350 shutdown_connections 2024-09-21T07:00:02.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.549+0000 7fcf213f8740 1 -- 172.21.15.114:0/2711258350 wait complete. 2024-09-21T07:00:02.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.550+0000 7fcf213f8740 1 -- start start 2024-09-21T07:00:02.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.550+0000 7fcf213f8740 1 Processor -- start 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d912a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d912fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d55f350 0x56215d91e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56215d6cd890 con 0x56215d53e670 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56215d807160 con 0x56215d536490 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56215d9156e0 con 0x56215d55f350 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf213f8740 10 client.0 _ll_register_callbacks cb 0x56215d823a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d912a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d912a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:35018/0 (socket says 172.21.15.114:35018) 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf13e00640 1 -- 172.21.15.114:0/1180409377 learned_addr learned my addr 172.21.15.114:0/1180409377 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d912fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.551+0000 7fcf13e00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d55f350 0x56215d91e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf0aa00640 1 -- 172.21.15.114:0/1180409377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d55f350 msgr2=0x56215d91e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d55f350 0x56215d91e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf0aa00640 1 -- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 msgr2=0x56215d912a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d912a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf0aa00640 1 -- 172.21.15.114:0/1180409377 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215d91ef60 con 0x56215d53e670 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d912fb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02a910 tx=0x7fcefc032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21b54dcf857695c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.552+0000 7fcf01600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf01600640 1 -- 172.21.15.114:0/1180409377 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc035040 con 0x56215d53e670 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf01600640 1 -- 172.21.15.114:0/1180409377 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcefc031760 con 0x56215d53e670 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c00f9a0 con 0x56215d53e670 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf01600640 1 -- 172.21.15.114:0/1180409377 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcefc04f050 con 0x56215d53e670 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf213f8740 10 client.16827 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56215d921870 con 0x56215d53e670 2024-09-21T07:00:02.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf01600640 1 -- 172.21.15.114:0/1180409377 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c00f9a0 con 0x56215d53e670 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf01600640 1 -- 172.21.15.114:0/1180409377 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7fcefc0054b0 con 0x56215d53e670 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.553+0000 7fcf213f8740 10 client.16827 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.554+0000 7fcf213f8740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.554+0000 7fcf213f8740 20 client.16827 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.554+0000 7fcf213f8740 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d51dcd0 0x56215d924e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.554+0000 7fcf213f8740 4 client.16827 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.554+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56215d87ede0 con 0x56215d51dcd0 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.554+0000 7fcf13e00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d51dcd0 0x56215d924e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.555+0000 7fcf13e00640 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d51dcd0 0x56215d924e20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969a221a6803d12f server_cookie=cdc93ffe77ef1ea4 in_seq=0 out_seq=0 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.555+0000 7fcf01600640 10 client.16827 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:02.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.555+0000 7fcf01600640 1 -- 172.21.15.114:0/1180409377 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56215d87ede0 con 0x56215d51dcd0 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.555+0000 7fcf01600640 10 client.16827 handle_command_reply: tid=0 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.555+0000 7fcf213f8740 1 client.16827 shutdown 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 msgr2=0x56215d912fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d912fb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02a910 tx=0x7fcefc032e90 comp rx=0 tx=0).stop 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 shutdown_connections 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d51dcd0 0x56215d924e20 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d55f350 0x56215d91e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d53e670 0x56215d912fb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 --2- 172.21.15.114:0/1180409377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d912a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 >> 172.21.15.114:0/1180409377 conn(0x56215d553740 msgr2=0x56215d907f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf01600640 0 client.16827 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 shutdown_connections 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 1 -- 172.21.15.114:0/1180409377 wait complete. 2024-09-21T07:00:02.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.556+0000 7fcf213f8740 20 client.16827 trim_cache size 0 max 16384 2024-09-21T07:00:02.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.574+0000 7fcf213f8740 1 Processor -- start 2024-09-21T07:00:02.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.574+0000 7fcf213f8740 1 -- start start 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d90d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d53e670 0x56215d9110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d911630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf213f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b2eb0 con 0x56215d55f350 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf213f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b4b40 con 0x56215d53e670 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf213f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b3580 con 0x56215d536490 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d90d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d90d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36452/0 (socket says 172.21.15.114:36452) 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d53e670 0x56215d9110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf13e00640 1 -- 172.21.15.114:0/110915250 learned_addr learned my addr 172.21.15.114:0/110915250 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:02.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.575+0000 7fcf13e00640 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d911630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf0aa00640 1 -- 172.21.15.114:0/110915250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 msgr2=0x56215d90d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d90d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf0aa00640 1 -- 172.21.15.114:0/110915250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 msgr2=0x56215d911630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d911630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf0aa00640 1 -- 172.21.15.114:0/110915250 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215d910d70 con 0x56215d53e670 2024-09-21T07:00:02.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d53e670 0x56215d9110f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02a430 tx=0x7fcefc004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f52399d6168c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:02.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf03e00640 1 -- 172.21.15.114:0/110915250 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc058070 con 0x56215d53e670 2024-09-21T07:00:02.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf03e00640 1 -- 172.21.15.114:0/110915250 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcefc005850 con 0x56215d53e670 2024-09-21T07:00:02.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.576+0000 7fcf03e00640 1 -- 172.21.15.114:0/110915250 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc035030 con 0x56215d53e670 2024-09-21T07:00:02.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 -- 172.21.15.114:0/110915250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d53e670 msgr2=0x56215d9110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d53e670 0x56215d9110f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcefc02a430 tx=0x7fcefc004b80 comp rx=0 tx=0).stop 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 -- 172.21.15.114:0/110915250 shutdown_connections 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d911630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d53e670 0x56215d9110f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 --2- 172.21.15.114:0/110915250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d536490 0x56215d90d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 -- 172.21.15.114:0/110915250 >> 172.21.15.114:0/110915250 conn(0x56215d553740 msgr2=0x56215d90da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 -- 172.21.15.114:0/110915250 shutdown_connections 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.577+0000 7fcf213f8740 1 -- 172.21.15.114:0/110915250 wait complete. 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.578+0000 7fcf213f8740 1 -- start start 2024-09-21T07:00:02.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.578+0000 7fcf213f8740 1 Processor -- start 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d916b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 0x56215d917040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d917580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b43b0 con 0x56215d55f350 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b3c50 con 0x56215d536490 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b0e90 con 0x56215d53e670 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf213f8740 10 client.0 _ll_register_callbacks cb 0x7fcefc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 0x56215d917040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 0x56215d917040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36454/0 (socket says 172.21.15.114:36454) 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 -- 172.21.15.114:0/3557391362 learned_addr learned my addr 172.21.15.114:0/3557391362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:02.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf13e00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d916b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf13e00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d917580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 -- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 msgr2=0x56215d916b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d916b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 -- 172.21.15.114:0/3557391362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 msgr2=0x56215d917580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d917580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.579+0000 7fcf0aa00640 1 -- 172.21.15.114:0/3557391362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215d98e9e0 con 0x56215d53e670 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf0aa00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 0x56215d917040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcefc004780 tx=0x7fcefc031d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45f02b129d78392f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf01600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf01600640 1 -- 172.21.15.114:0/3557391362 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc058070 con 0x56215d53e670 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf01600640 1 -- 172.21.15.114:0/3557391362 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fcefc004fe0 con 0x56215d53e670 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc005850 con 0x56215d53e670 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf01600640 1 -- 172.21.15.114:0/3557391362 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcefc031970 con 0x56215d53e670 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf213f8740 10 client.27097 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:02.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.580+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56215d98fca0 con 0x56215d53e670 2024-09-21T07:00:02.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf01600640 1 -- 172.21.15.114:0/3557391362 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcefc005850 con 0x56215d53e670 2024-09-21T07:00:02.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf01600640 1 -- 172.21.15.114:0/3557391362 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7fcefc046770 con 0x56215d53e670 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf213f8740 10 client.27097 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf213f8740 10 client.27097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf213f8740 20 client.27097 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf213f8740 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d4f3080 0x56215d9922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf213f8740 4 client.27097 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56215d87a890 con 0x56215d4f3080 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.581+0000 7fcf13e00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d4f3080 0x56215d9922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.582+0000 7fcf13e00640 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d4f3080 0x56215d9922e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de00a0fc7ea7f84 server_cookie=d4985891695f791e in_seq=0 out_seq=0 2024-09-21T07:00:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.582+0000 7fcf01600640 10 client.27097 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.583+0000 7fcf01600640 1 -- 172.21.15.114:0/3557391362 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56215d87a890 con 0x56215d4f3080 2024-09-21T07:00:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.583+0000 7fcf01600640 10 client.27097 handle_command_reply: tid=0 2024-09-21T07:00:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.583+0000 7fcf213f8740 1 client.27097 shutdown 2024-09-21T07:00:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 msgr2=0x56215d917040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 0x56215d917040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcefc004780 tx=0x7fcefc031d40 comp rx=0 tx=0).stop 2024-09-21T07:00:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 shutdown_connections 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56215d55f350 0x56215d917580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56215d4f3080 0x56215d9922e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56215d53e670 0x56215d917040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 --2- 172.21.15.114:0/3557391362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56215d536490 0x56215d916b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 >> 172.21.15.114:0/3557391362 conn(0x56215d553740 msgr2=0x56215d911090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf01600640 0 client.27097 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 shutdown_connections 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 1 -- 172.21.15.114:0/3557391362 wait complete. 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.584+0000 7fcf213f8740 20 client.27097 trim_cache size 0 max 16384 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:02.587 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "4eb7c4dc-e1ac-4313-8502-e52283f23360", 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcef80b6700 msgr2=0x7fcef80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcef80b6700 0x7fcef80b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf040e51e0 tx=0x7fcf040eb8a0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 msgr2=0x7fcf1419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 0x7fcf1419c500 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400ea60 tx=0x7fcf0400ee60 comp rx=0 tx=0).stop 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 shutdown_connections 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf1410be50 0x7fcf141a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcef80b6700 0x7fcef80b8c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf1410b500 0x7fcf1419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 --2- 172.21.15.114:0/317223581 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcf14105bc0 0x7fcf1419c500 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 >> 172.21.15.114:0/317223581 conn(0x7fcf14101670 msgr2=0x7fcf14103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 shutdown_connections 2024-09-21T07:00:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:02.585+0000 7fcf19000640 1 -- 172.21.15.114:0/317223581 wait complete. 2024-09-21T07:00:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 325.0 s, 220.2 tps, lat 198.214 ms stddev 255.170 2024-09-21T07:00:03.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: pgmap v573: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 480 op/s 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: pgmap v573: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 480 op/s 2024-09-21T07:00:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.634 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: pgmap v573: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 480 op/s 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:00:03.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:03.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:04.044 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:04.515 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:00:04.515 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:04.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:00:04.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:04.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:00:04.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 -- 172.21.15.114:0/2671832154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d4109c40 msgr2=0x7f14d4110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 --2- 172.21.15.114:0/2671832154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d4109c40 0x7f14d4110470 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009ca0 tx=0x7f14cc02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 -- 172.21.15.114:0/2671832154 shutdown_connections 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 --2- 172.21.15.114:0/2671832154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d4109c40 0x7f14d4110470 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 --2- 172.21.15.114:0/2671832154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41092f0 0x7f14d4109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 --2- 172.21.15.114:0/2671832154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d41039b0 0x7f14d4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.899+0000 7f14d3200640 1 -- 172.21.15.114:0/2671832154 >> 172.21.15.114:0/2671832154 conn(0x7f14d40ff440 msgr2=0x7f14d4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.900+0000 7f14d3200640 1 -- 172.21.15.114:0/2671832154 shutdown_connections 2024-09-21T07:00:04.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.900+0000 7f14d3200640 1 -- 172.21.15.114:0/2671832154 wait complete. 2024-09-21T07:00:04.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 Processor -- start 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 -- start start 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 0x7f14d419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d41092f0 0x7f14d419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4109c40 0x7f14d419fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4113950 con 0x7f14d41092f0 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41137d0 con 0x7f14d4109c40 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.901+0000 7f14d3200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4113ad0 con 0x7f14d41039b0 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 0x7f14d419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d41092f0 0x7f14d419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 0x7f14d419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52672/0 (socket says 172.21.15.114:52672) 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 -- 172.21.15.114:0/135310791 learned_addr learned my addr 172.21.15.114:0/135310791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d2800640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4109c40 0x7f14d419fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 -- 172.21.15.114:0/135310791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4109c40 msgr2=0x7f14d419fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4109c40 0x7f14d419fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 -- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d41092f0 msgr2=0x7f14d419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d41092f0 0x7f14d419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 -- 172.21.15.114:0/135310791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d41a0230 con 0x7f14d41039b0 2024-09-21T07:00:04.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.902+0000 7f14d1e00640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 0x7f14d419a2c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e9f0 tx=0x7f14c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:04.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.903+0000 7f14caa00640 1 -- 172.21.15.114:0/135310791 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c400cd80 con 0x7f14d41039b0 2024-09-21T07:00:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.903+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d41a0520 con 0x7f14d41039b0 2024-09-21T07:00:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.903+0000 7f14caa00640 1 -- 172.21.15.114:0/135310791 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c40108e0 con 0x7f14d41039b0 2024-09-21T07:00:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.903+0000 7f14caa00640 1 -- 172.21.15.114:0/135310791 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4018980 con 0x7f14d41039b0 2024-09-21T07:00:04.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.903+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41a6c80 con 0x7f14d41039b0 2024-09-21T07:00:04.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.906+0000 7f14dfc60740 1 Processor -- start 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.906+0000 7f14dfc60740 1 -- start start 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.906+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a66f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.906+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14dfc60740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55efc19ddf70 con 0x55efc16971b0 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14dfc60740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efc19cd3a0 con 0x55efc169f390 2024-09-21T07:00:04.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14dfc60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efc19d11b0 con 0x55efc16c0070 2024-09-21T07:00:04.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a66f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a66f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52674/0 (socket says 172.21.15.114:52674) 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a679b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38594/0 (socket says 172.21.15.114:38594) 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d2800640 1 -- 172.21.15.114:0/3849924569 learned_addr learned my addr 172.21.15.114:0/3849924569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14caa00640 1 -- 172.21.15.114:0/135310791 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f14c4010430 con 0x7f14d41039b0 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d1400640 1 -- 172.21.15.114:0/3849924569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 msgr2=0x55efc1a66f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d1400640 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a66f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.907+0000 7f14d1400640 1 -- 172.21.15.114:0/3849924569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 msgr2=0x55efc1a67470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.908+0000 7f14d1400640 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.908+0000 7f14d1400640 1 -- 172.21.15.114:0/3849924569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc1997c90 con 0x55efc169f390 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.908+0000 7f14d1400640 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a679b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14d406b700 tx=0x7f14c0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751b337045313063 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:04.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.908+0000 7f14bf400640 1 -- 172.21.15.114:0/3849924569 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004250 con 0x55efc169f390 2024-09-21T07:00:04.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.908+0000 7f14bf400640 1 -- 172.21.15.114:0/3849924569 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c0009e10 con 0x55efc169f390 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.908+0000 7f14bf400640 1 -- 172.21.15.114:0/3849924569 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0004cd0 con 0x55efc169f390 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14caa00640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14b80b6700 0x7f14b80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 -- 172.21.15.114:0/3849924569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 msgr2=0x55efc1a679b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a679b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f14d406b700 tx=0x7f14c0011810 comp rx=0 tx=0).stop 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14caa00640 1 -- 172.21.15.114:0/135310791 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f14c40e7e50 con 0x7f14d41039b0 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 -- 172.21.15.114:0/3849924569 shutdown_connections 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a679b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 --2- 172.21.15.114:0/3849924569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a66f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 -- 172.21.15.114:0/3849924569 >> 172.21.15.114:0/3849924569 conn(0x55efc16b4460 msgr2=0x55efc1a64e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14d2800640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14b80b6700 0x7f14b80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 -- 172.21.15.114:0/3849924569 shutdown_connections 2024-09-21T07:00:04.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.909+0000 7f14dfc60740 1 -- 172.21.15.114:0/3849924569 wait complete. 2024-09-21T07:00:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.910+0000 7f14dfc60740 1 -- start start 2024-09-21T07:00:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.910+0000 7f14d2800640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14b80b6700 0x7f14b80b8c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f14cc02ade0 tx=0x7f14cc005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:04.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.910+0000 7f14dfc60740 1 Processor -- start 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a72be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55efc182e700 con 0x55efc169f390 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55efc1968540 con 0x55efc16971b0 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55efc1a75710 con 0x55efc16c0070 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14dfc60740 10 client.0 _ll_register_callbacks cb 0x55efc1978430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14d1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a72be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14d2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38604/0 (socket says 172.21.15.114:38604) 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14d2800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a7ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52686/0 (socket says 172.21.15.114:52686) 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.911+0000 7f14d1e00640 1 -- 172.21.15.114:0/159259297 learned_addr learned my addr 172.21.15.114:0/159259297 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d1e00640 1 -- 172.21.15.114:0/159259297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 msgr2=0x55efc1a7ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d1e00640 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a7ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d1e00640 1 -- 172.21.15.114:0/159259297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 msgr2=0x55efc1a72be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d1e00640 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a72be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d1e00640 1 -- 172.21.15.114:0/159259297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc172b650 con 0x55efc16971b0 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d2800640 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a7ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14d1e00640 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a726a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14c400c6a0 tx=0x7f14c40e5750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c419c48e5f0ad4af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14b3e00640 1 -- 172.21.15.114:0/159259297 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4010720 con 0x55efc16971b0 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14b3e00640 1 -- 172.21.15.114:0/159259297 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f14c4018b20 con 0x55efc16971b0 2024-09-21T07:00:04.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.912+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efc196ab50 con 0x55efc16971b0 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14b3e00640 1 -- 172.21.15.114:0/159259297 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14c40b21a0 con 0x55efc16971b0 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 10 client.27149 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55efc1985910 con 0x55efc16971b0 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14b3e00640 1 -- 172.21.15.114:0/159259297 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55efc196ab50 con 0x55efc16971b0 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14b3e00640 1 -- 172.21.15.114:0/159259297 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f14c40b3540 con 0x55efc16971b0 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 10 client.27149 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 20 client.27149 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc167e9f0 0x55efc1a84780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 4 client.27149 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:04.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.913+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc19dfec0 con 0x55efc167e9f0 2024-09-21T07:00:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.914+0000 7f14d1400640 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc167e9f0 0x55efc1a84780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.914+0000 7f14d1400640 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc167e9f0 0x55efc1a84780 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4966588696e3bc44 server_cookie=bcc29413a207d898 in_seq=0 out_seq=0 2024-09-21T07:00:04.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.914+0000 7f14b3e00640 10 client.27149 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.915+0000 7f14b3e00640 1 -- 172.21.15.114:0/159259297 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55efc19dfec0 con 0x55efc167e9f0 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.915+0000 7f14b3e00640 10 client.27149 handle_command_reply: tid=0 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.915+0000 7f14dfc60740 1 client.27149 shutdown 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.915+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 msgr2=0x55efc1a726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a726a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f14c400c6a0 tx=0x7f14c40e5750 comp rx=0 tx=0).stop 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 shutdown_connections 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc167e9f0 0x55efc1a84780 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a7ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a72be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14b3e00640 0 client.27149 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 --2- 172.21.15.114:0/159259297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a726a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 >> 172.21.15.114:0/159259297 conn(0x55efc16b4460 msgr2=0x55efc1a6a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:04.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 shutdown_connections 2024-09-21T07:00:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 1 -- 172.21.15.114:0/159259297 wait complete. 2024-09-21T07:00:04.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.916+0000 7f14dfc60740 20 client.27149 trim_cache size 0 max 16384 2024-09-21T07:00:04.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.934+0000 7f14dfc60740 1 Processor -- start 2024-09-21T07:00:04.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.934+0000 7f14dfc60740 1 -- start start 2024-09-21T07:00:04.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.934+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a71590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.934+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.934+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a63cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.934+0000 7f14dfc60740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41aeaa0 con 0x55efc169f390 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14dfc60740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41b0730 con 0x55efc16971b0 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14dfc60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41af170 con 0x55efc16c0070 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a74580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60738/0 (socket says 172.21.15.114:60738) 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 -- 172.21.15.114:0/1383436634 learned_addr learned my addr 172.21.15.114:0/1383436634 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1e00640 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a71590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d2800640 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a63cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 -- 172.21.15.114:0/1383436634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 msgr2=0x55efc1a63cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a63cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 -- 172.21.15.114:0/1383436634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 msgr2=0x55efc1a71590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a71590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 -- 172.21.15.114:0/1383436634 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc1a69d90 con 0x55efc169f390 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.935+0000 7f14d1400640 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a74580 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f14c00058c0 tx=0x7f14c0004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109eb94286aee76c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.936+0000 7f14bea00640 1 -- 172.21.15.114:0/1383436634 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c001e040 con 0x55efc169f390 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.936+0000 7f14bea00640 1 -- 172.21.15.114:0/1383436634 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c001f040 con 0x55efc169f390 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.936+0000 7f14bea00640 1 -- 172.21.15.114:0/1383436634 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c0003c60 con 0x55efc169f390 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.936+0000 7f14dfc60740 1 -- 172.21.15.114:0/1383436634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 msgr2=0x55efc1a74580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.936+0000 7f14dfc60740 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a74580 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f14c00058c0 tx=0x7f14c0004950 comp rx=0 tx=0).stop 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 -- 172.21.15.114:0/1383436634 shutdown_connections 2024-09-21T07:00:04.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a63cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc169f390 0x55efc1a74580 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 --2- 172.21.15.114:0/1383436634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc16971b0 0x55efc1a71590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 -- 172.21.15.114:0/1383436634 >> 172.21.15.114:0/1383436634 conn(0x55efc16b4460 msgr2=0x55efc1a635c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 -- 172.21.15.114:0/1383436634 shutdown_connections 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 -- 172.21.15.114:0/1383436634 wait complete. 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 -- start start 2024-09-21T07:00:04.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.937+0000 7f14dfc60740 1 Processor -- start 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a7a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a7aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41affa0 con 0x55efc16971b0 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41af840 con 0x55efc169f390 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14d41aca80 con 0x55efc16c0070 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14dfc60740 10 client.0 _ll_register_callbacks cb 0x7f14c4010ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:04.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14d1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a7a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14d1e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a7a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60750/0 (socket says 172.21.15.114:60750) 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14d1e00640 1 -- 172.21.15.114:0/2129246683 learned_addr learned my addr 172.21.15.114:0/2129246683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14d1400640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a7aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.938+0000 7f14d2800640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a75920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1e00640 1 -- 172.21.15.114:0/2129246683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 msgr2=0x55efc1a75920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1e00640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a75920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1e00640 1 -- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 msgr2=0x55efc1a7aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1e00640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a7aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1e00640 1 -- 172.21.15.114:0/2129246683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc1a75e60 con 0x55efc16971b0 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1400640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a7aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14d1e00640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a7a970 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f14c40b3a20 tx=0x7f14c40a9170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eda7ab997f6c432 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:04.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14b3e00640 1 -- 172.21.15.114:0/2129246683 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c40a9770 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14b3e00640 1 -- 172.21.15.114:0/2129246683 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f14c40ac7a0 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.939+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c001f040 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14b3e00640 1 -- 172.21.15.114:0/2129246683 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f14c40ad3e0 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 10 client.16848 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55efc1aef870 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14b3e00640 1 -- 172.21.15.114:0/2129246683 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c001f040 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14b3e00640 1 -- 172.21.15.114:0/2129246683 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f14c40ad8d0 con 0x55efc16971b0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 10 client.16848 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 20 client.16848 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc1653da0 0x55efc1af2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 4 client.16848 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.940+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efc19db970 con 0x55efc1653da0 2024-09-21T07:00:04.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.941+0000 7f14d1400640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc1653da0 0x55efc1af2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.941+0000 7f14d1400640 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc1653da0 0x55efc1af2080 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f966b0afa7ad1d server_cookie=e838747903ef5650 in_seq=0 out_seq=0 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.941+0000 7f14b3e00640 10 client.16848 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.941+0000 7f14b3e00640 1 -- 172.21.15.114:0/2129246683 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55efc19db970 con 0x55efc1653da0 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14b3e00640 10 client.16848 handle_command_reply: tid=0 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 client.16848 shutdown 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 msgr2=0x55efc1a7a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a7a970 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f14c40b3a20 tx=0x7f14c40a9170 comp rx=0 tx=0).stop 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 shutdown_connections 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55efc1653da0 0x55efc1af2080 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55efc16c0070 0x55efc1a75920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55efc169f390 0x55efc1a7aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 --2- 172.21.15.114:0/2129246683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55efc16971b0 0x55efc1a7a970 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 >> 172.21.15.114:0/2129246683 conn(0x55efc16b4460 msgr2=0x55efc1a73960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14b3e00640 0 client.16848 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 shutdown_connections 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 1 -- 172.21.15.114:0/2129246683 wait complete. 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.942+0000 7f14dfc60740 20 client.16848 trim_cache size 0 max 16384 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:00:04.947 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "4eb7c4dc-e1ac-4313-8502-e52283f23360", 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:00:04.948 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.949 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.950 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:04.951 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14b80b6700 msgr2=0x7f14b80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14b80b6700 0x7f14b80b8c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f14cc02ade0 tx=0x7f14cc005da0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 msgr2=0x7f14d419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 0x7f14d419a2c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e9f0 tx=0x7f14c400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 shutdown_connections 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14b80b6700 0x7f14b80b8c10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4109c40 0x7f14d419fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14d41092f0 0x7f14d419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 --2- 172.21.15.114:0/135310791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14d41039b0 0x7f14d419a2c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:04.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 >> 172.21.15.114:0/135310791 conn(0x7f14d40ff440 msgr2=0x7f14d4108980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:04.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 shutdown_connections 2024-09-21T07:00:04.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:04.943+0000 7f14d3200640 1 -- 172.21.15.114:0/135310791 wait complete. 2024-09-21T07:00:05.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:05 smithi114 ceph-mon[30654]: pgmap v574: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 543 op/s 2024-09-21T07:00:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:05 smithi183 ceph-mon[34219]: pgmap v574: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 543 op/s 2024-09-21T07:00:05.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:05 smithi193 ceph-mon[34601]: pgmap v574: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 543 op/s 2024-09-21T07:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 -- 172.21.15.114:0/2452913775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640109d50 msgr2=0x7f5640110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 --2- 172.21.15.114:0/2452913775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640109d50 0x7f5640110580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5638009ca0 tx=0x7f563802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 -- 172.21.15.114:0/2452913775 shutdown_connections 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 --2- 172.21.15.114:0/2452913775 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640109d50 0x7f5640110580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 --2- 172.21.15.114:0/2452913775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5640109400 0x7f5640109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 --2- 172.21.15.114:0/2452913775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640103ad0 0x7f5640103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.285+0000 7f5644c00640 1 -- 172.21.15.114:0/2452913775 >> 172.21.15.114:0/2452913775 conn(0x7f56400ff510 msgr2=0x7f5640101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.286+0000 7f5644c00640 1 -- 172.21.15.114:0/2452913775 shutdown_connections 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.286+0000 7f5644c00640 1 -- 172.21.15.114:0/2452913775 wait complete. 2024-09-21T07:00:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 Processor -- start 2024-09-21T07:00:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 -- start start 2024-09-21T07:00:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 0x7f564019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5640109400 0x7f564019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640109d50 0x7f564019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5640113a10 con 0x7f5640103ad0 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5640113890 con 0x7f5640109400 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f5644c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5640113b90 con 0x7f5640109d50 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f563de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 0x7f564019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.287+0000 7f563e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640109d50 0x7f564019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5640109400 0x7f564019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 0x7f564019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60774/0 (socket says 172.21.15.114:60774) 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 -- 172.21.15.114:0/1545022182 learned_addr learned my addr 172.21.15.114:0/1545022182 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640109d50 0x7f564019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52710/0 (socket says 172.21.15.114:52710) 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 -- 172.21.15.114:0/1545022182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640109d50 msgr2=0x7f564019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640109d50 0x7f564019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 -- 172.21.15.114:0/1545022182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5640109400 msgr2=0x7f564019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5640109400 0x7f564019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 -- 172.21.15.114:0/1545022182 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56401a0320 con 0x7f5640103ad0 2024-09-21T07:00:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f563de00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 0x7f564019a3f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f563000e9f0 tx=0x7f563000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.288+0000 7f5636a00640 1 -- 172.21.15.114:0/1545022182 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563000cd80 con 0x7f5640103ad0 2024-09-21T07:00:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.289+0000 7f5636a00640 1 -- 172.21.15.114:0/1545022182 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56300108e0 con 0x7f5640103ad0 2024-09-21T07:00:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.289+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56401a0610 con 0x7f5640103ad0 2024-09-21T07:00:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.289+0000 7f5636a00640 1 -- 172.21.15.114:0/1545022182 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630018980 con 0x7f5640103ad0 2024-09-21T07:00:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.289+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56400755f0 con 0x7f5640103ad0 2024-09-21T07:00:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 Processor -- start 2024-09-21T07:00:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 -- start start 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777357c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x556777358170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5567772cd6b0 con 0x556776f924d0 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567772bcae0 con 0x556776fb31b0 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f564bc4f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567772c08f0 con 0x556776f8a2f0 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f5636a00640 1 -- 172.21.15.114:0/1545022182 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5630010430 con 0x7f5640103ad0 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f563d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f563d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773586b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60790/0 (socket says 172.21.15.114:60790) 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f563d400640 1 -- 172.21.15.114:0/1489608202 learned_addr learned my addr 172.21.15.114:0/1489608202 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.292+0000 7f563e800640 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777357c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563de00640 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x556777358170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563d400640 1 -- 172.21.15.114:0/1489608202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 msgr2=0x556777358170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563d400640 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x556777358170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563d400640 1 -- 172.21.15.114:0/1489608202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 msgr2=0x556777357c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563d400640 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777357c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563d400640 1 -- 172.21.15.114:0/1489608202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556777289780 con 0x556776f924d0 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563e800640 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777357c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f563d400640 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773586b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f562c010d50 tx=0x7f562c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b75700d2849baa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.293+0000 7f562b000640 1 -- 172.21.15.114:0/1489608202 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c004050 con 0x556776f924d0 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f562b000640 1 -- 172.21.15.114:0/1489608202 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c0054d0 con 0x556776f924d0 2024-09-21T07:00:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f562b000640 1 -- 172.21.15.114:0/1489608202 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c004d40 con 0x556776f924d0 2024-09-21T07:00:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f5636a00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f56240b6700 0x7f56240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f5636a00640 1 -- 172.21.15.114:0/1545022182 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f56300e7d40 con 0x7f5640103ad0 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f564bc4f740 1 -- 172.21.15.114:0/1489608202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 msgr2=0x5567773586b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773586b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f562c010d50 tx=0x7f562c005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.294+0000 7f563e800640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f56240b6700 0x7f56240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 -- 172.21.15.114:0/1489608202 shutdown_connections 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773586b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x556777358170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1489608202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777357c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 -- 172.21.15.114:0/1489608202 >> 172.21.15.114:0/1489608202 conn(0x556776fa7300 msgr2=0x556777355b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 -- 172.21.15.114:0/1489608202 shutdown_connections 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 -- 172.21.15.114:0/1489608202 wait complete. 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f563e800640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f56240b6700 0x7f56240b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f563802f500 tx=0x7f563802f490 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.295+0000 7f564bc4f740 1 -- start start 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 Processor -- start 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x556777363670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777363bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5567771f4ab0 con 0x556776f8a2f0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556777263570 con 0x556776f924d0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556777366400 con 0x556776fb31b0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f564bc4f740 10 client.0 _ll_register_callbacks cb 0x556777272100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f563de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f563de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60802/0 (socket says 172.21.15.114:60802) 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f563de00640 1 -- 172.21.15.114:0/4043587611 learned_addr learned my addr 172.21.15.114:0/4043587611 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f563e800640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777363bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.296+0000 7f563d400640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x556777363670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f563de00640 1 -- 172.21.15.114:0/4043587611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 msgr2=0x556777363bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f563de00640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777363bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f563de00640 1 -- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 msgr2=0x556777363670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f563de00640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x556777363670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f563de00640 1 -- 172.21.15.114:0/4043587611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677701e790 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f563de00640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363130 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f56300ad5d0 tx=0x7f56300a9930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e538eba0630ab982 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f561f400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f561f400640 1 -- 172.21.15.114:0/4043587611 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300a9f20 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f561f400640 1 -- 172.21.15.114:0/4043587611 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f56300aeb50 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562c0054d0 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.297+0000 7f561f400640 1 -- 172.21.15.114:0/4043587611 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f56300ad040 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 10 client.16860 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556777097620 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f561f400640 1 -- 172.21.15.114:0/4043587611 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c0054d0 con 0x556776f8a2f0 2024-09-21T07:00:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f561f400640 1 -- 172.21.15.114:0/4043587611 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f56300a9220 con 0x556776f8a2f0 2024-09-21T07:00:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 10 client.16860 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 20 client.16860 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f71d80 0x556777375440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 4 client.16860 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.298+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567772cf600 con 0x556776f71d80 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.299+0000 7f563d400640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f71d80 0x556777375440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.299+0000 7f563d400640 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f71d80 0x556777375440 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27037b3b0f117b5f server_cookie=6c04cee3af43ef4e in_seq=0 out_seq=0 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.299+0000 7f561f400640 10 client.16860 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f561f400640 1 -- 172.21.15.114:0/4043587611 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567772cf600 con 0x556776f71d80 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f561f400640 10 client.16860 handle_command_reply: tid=0 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 client.16860 shutdown 2024-09-21T07:00:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 msgr2=0x556777363130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363130 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f56300ad5d0 tx=0x7f56300a9930 comp rx=0 tx=0).stop 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 shutdown_connections 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f71d80 0x556777375440 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777363bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x556777363670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f564bc4f740 1 --2- 172.21.15.114:0/4043587611 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363130 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.300+0000 7f561f400640 0 client.16860 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.301+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 >> 172.21.15.114:0/4043587611 conn(0x556776fa7300 msgr2=0x556777358c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.301+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 shutdown_connections 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.301+0000 7f564bc4f740 1 -- 172.21.15.114:0/4043587611 wait complete. 2024-09-21T07:00:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.301+0000 7f564bc4f740 20 client.16860 trim_cache size 0 max 16384 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.317+0000 7f564bc4f740 1 Processor -- start 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.317+0000 7f564bc4f740 1 -- start start 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x5567773540e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777359370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f564bc4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56401aebb0 con 0x556776f8a2f0 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f564bc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56401b0840 con 0x556776f924d0 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f564bc4f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56401af280 con 0x556776fb31b0 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60806/0 (socket says 172.21.15.114:60806) 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 -- 172.21.15.114:0/3315316469 learned_addr learned my addr 172.21.15.114:0/3315316469 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563d400640 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x5567773540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563e800640 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777359370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 -- 172.21.15.114:0/3315316469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 msgr2=0x556777359370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777359370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 -- 172.21.15.114:0/3315316469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 msgr2=0x5567773540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x5567773540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563de00640 1 -- 172.21.15.114:0/3315316469 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556777356050 con 0x556776f8a2f0 2024-09-21T07:00:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.318+0000 7f563d400640 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x5567773540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.319+0000 7f563de00640 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363a60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f563000c6a0 tx=0x7f56300a96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f34c24984ae763e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.319+0000 7f562a600640 1 -- 172.21.15.114:0/3315316469 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300ad220 con 0x556776f8a2f0 2024-09-21T07:00:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.319+0000 7f562a600640 1 -- 172.21.15.114:0/3315316469 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56300ad3e0 con 0x556776f8a2f0 2024-09-21T07:00:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.319+0000 7f562a600640 1 -- 172.21.15.114:0/3315316469 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300ad6a0 con 0x556776f8a2f0 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.319+0000 7f564bc4f740 1 -- 172.21.15.114:0/3315316469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 msgr2=0x556777363a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.319+0000 7f564bc4f740 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363a60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f563000c6a0 tx=0x7f56300a96d0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 -- 172.21.15.114:0/3315316469 shutdown_connections 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776fb31b0 0x556777359370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776f924d0 0x5567773540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 --2- 172.21.15.114:0/3315316469 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f8a2f0 0x556777363a60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 -- 172.21.15.114:0/3315316469 >> 172.21.15.114:0/3315316469 conn(0x556776fa7300 msgr2=0x556777358c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 -- 172.21.15.114:0/3315316469 shutdown_connections 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 -- 172.21.15.114:0/3315316469 wait complete. 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 -- start start 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.320+0000 7f564bc4f740 1 Processor -- start 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x55677736c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777369bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f56401b00b0 con 0x556776f924d0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56401af950 con 0x556776fb31b0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56401acb90 con 0x556776f8a2f0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f564bc4f740 10 client.0 _ll_register_callbacks cb 0x7f563000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x55677736c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773694d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60812/0 (socket says 172.21.15.114:60812) 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 -- 172.21.15.114:0/1390623939 learned_addr learned my addr 172.21.15.114:0/1390623939 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563e800640 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777369bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 -- 172.21.15.114:0/1390623939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 msgr2=0x55677736c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x55677736c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 -- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 msgr2=0x556777369bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777369bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.321+0000 7f563d400640 1 -- 172.21.15.114:0/1390623939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567773dee50 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f563d400640 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773694d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f562c0057c0 tx=0x7f562c009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9c38a6a029597e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f561f400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f561f400640 1 -- 172.21.15.114:0/1390623939 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c004650 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f561f400640 1 -- 172.21.15.114:0/1390623939 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f562c0042e0 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56300ad3e0 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f561f400640 1 -- 172.21.15.114:0/1390623939 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f562c03d050 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f564bc4f740 10 client.16866 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5567773dff50 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f561f400640 1 -- 172.21.15.114:0/1390623939 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56300ad3e0 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.322+0000 7f561f400640 1 -- 172.21.15.114:0/1390623939 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f562c0264d0 con 0x556776f924d0 2024-09-21T07:00:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f564bc4f740 10 client.16866 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f564bc4f740 10 client.16866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f564bc4f740 20 client.16866 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f471c0 0x5567773e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f564bc4f740 4 client.16866 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567772cb0b0 con 0x556776f471c0 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f563de00640 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f471c0 0x5567773e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f563de00640 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f471c0 0x5567773e2530 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f95adc9d932c2ff0 server_cookie=c31e84da04d14da6 in_seq=0 out_seq=0 2024-09-21T07:00:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.323+0000 7f561f400640 10 client.16866 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f561f400640 1 -- 172.21.15.114:0/1390623939 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5567772cb0b0 con 0x556776f471c0 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f561f400640 10 client.16866 handle_command_reply: tid=0 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 client.16866 shutdown 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 msgr2=0x5567773694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773694d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f562c0057c0 tx=0x7f562c009e40 comp rx=0 tx=0).stop 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 shutdown_connections 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556776fb31b0 0x556777369bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556776f924d0 0x5567773694d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556776f471c0 0x5567773e2530 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 --2- 172.21.15.114:0/1390623939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556776f8a2f0 0x55677736c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f561f400640 0 client.16866 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 >> 172.21.15.114:0/1390623939 conn(0x556776fa7300 msgr2=0x55677736efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 shutdown_connections 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 1 -- 172.21.15.114:0/1390623939 wait complete. 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.332+0000 7f564bc4f740 20 client.16866 trim_cache size 0 max 16384 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:00:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f56240b6700 msgr2=0x7f56240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f56240b6700 0x7f56240b8c10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f563802f500 tx=0x7f563802f490 comp rx=0 tx=0).stop 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 msgr2=0x7f564019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 0x7f564019a3f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f563000e9f0 tx=0x7f563000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 shutdown_connections 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5640109d50 0x7f564019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f56240b6700 0x7f56240b8c10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5640109400 0x7f564019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 --2- 172.21.15.114:0/1545022182 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5640103ad0 0x7f564019a3f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 >> 172.21.15.114:0/1545022182 conn(0x7f56400ff510 msgr2=0x7f5640101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 shutdown_connections 2024-09-21T07:00:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:06.333+0000 7f5644c00640 1 -- 172.21.15.114:0/1545022182 wait complete. 2024-09-21T07:00:06.465 INFO:tasks.cephfs.filesystem:scrub status for tag:4eb7c4dc-e1ac-4313-8502-e52283f23360 - {'path': '/', 'tag': '4eb7c4dc-e1ac-4313-8502-e52283f23360', 'options': 'recursive,force'} 2024-09-21T07:00:07.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:07 smithi114 ceph-mon[30654]: pgmap v575: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 436 op/s 2024-09-21T07:00:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:07 smithi183 ceph-mon[34219]: pgmap v575: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 436 op/s 2024-09-21T07:00:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:07 smithi193 ceph-mon[34601]: pgmap v575: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 8.5 MiB/s wr, 436 op/s 2024-09-21T07:00:07.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 330.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:00:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:09 smithi114 ceph-mon[30654]: pgmap v576: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.9 MiB/s wr, 445 op/s 2024-09-21T07:00:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:09 smithi183 ceph-mon[34219]: pgmap v576: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.9 MiB/s wr, 445 op/s 2024-09-21T07:00:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:09 smithi193 ceph-mon[34601]: pgmap v576: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.9 MiB/s wr, 445 op/s 2024-09-21T07:00:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:11 smithi114 ceph-mon[30654]: pgmap v577: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-09-21T07:00:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:11 smithi183 ceph-mon[34219]: pgmap v577: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-09-21T07:00:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:11 smithi193 ceph-mon[34601]: pgmap v577: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-09-21T07:00:12.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 335.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:00:13.331 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:00:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:13 smithi114 ceph-mon[30654]: pgmap v578: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-09-21T07:00:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:13 smithi183 ceph-mon[34219]: pgmap v578: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-09-21T07:00:13.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:13 smithi193 ceph-mon[34601]: pgmap v578: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-09-21T07:00:13.707 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 -- 172.21.15.114:0/1123039795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 msgr2=0x7fd1cc103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1123039795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc103f70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc009900 tx=0x7fd1bc02a410 comp rx=0 tx=0).stop 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 -- 172.21.15.114:0/1123039795 shutdown_connections 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1123039795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 0x7fd1cc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1123039795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 0x7fd1cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1123039795 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc103f70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 -- 172.21.15.114:0/1123039795 >> 172.21.15.114:0/1123039795 conn(0x7fd1cc07b850 msgr2=0x7fd1cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:14.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 -- 172.21.15.114:0/1123039795 shutdown_connections 2024-09-21T07:00:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.349+0000 7fd1d1800640 1 -- 172.21.15.114:0/1123039795 wait complete. 2024-09-21T07:00:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.350+0000 7fd1d1800640 1 Processor -- start 2024-09-21T07:00:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.350+0000 7fd1d1800640 1 -- start start 2024-09-21T07:00:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1d1800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:14.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1d1800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 0x7fd1cc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1d1800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 0x7fd1cc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1d1800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113ab0 con 0x7fd1cc103b60 2024-09-21T07:00:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1d1800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113930 con 0x7fd1cc109df0 2024-09-21T07:00:14.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1d1800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc113c30 con 0x7fd1cc1094a0 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1caa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1caa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60832/0 (socket says 172.21.15.114:60832) 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1caa00640 1 -- 172.21.15.114:0/1728164357 learned_addr learned my addr 172.21.15.114:0/1728164357 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1cb400640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 0x7fd1cc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.351+0000 7fd1ca000640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 0x7fd1cc19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1caa00640 1 -- 172.21.15.114:0/1728164357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 msgr2=0x7fd1cc19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1caa00640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 0x7fd1cc19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1caa00640 1 -- 172.21.15.114:0/1728164357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 msgr2=0x7fd1cc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1caa00640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 0x7fd1cc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1caa00640 1 -- 172.21.15.114:0/1728164357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc1a0410 con 0x7fd1cc103b60 2024-09-21T07:00:14.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1cb400640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 0x7fd1cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1ca000640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 0x7fd1cc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1caa00640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc19a4a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005940 tx=0x7fd1bc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc03f030 con 0x7fd1cc103b60 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc0044c0 con 0x7fd1cc103b60 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc1a06a0 con 0x7fd1cc103b60 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc035070 con 0x7fd1cc103b60 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.352+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a6e60 con 0x7fd1cc103b60 2024-09-21T07:00:14.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.353+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc104de0 con 0x7fd1cc103b60 2024-09-21T07:00:14.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.356+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd1bc042030 con 0x7fd1cc103b60 2024-09-21T07:00:14.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.358+0000 7fd1c3400640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1b00b6680 0x7fd1b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:14.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.358+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd1bc107ff0 con 0x7fd1cc103b60 2024-09-21T07:00:14.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.359+0000 7fd1ca000640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1b00b6680 0x7fd1b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:14.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.359+0000 7fd1ca000640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1b00b6680 0x7fd1b00b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc1091a0 tx=0x7fd1b8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:14.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.361+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1bc0d25b0 con 0x7fd1cc103b60 2024-09-21T07:00:14.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.587+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1cc10bbc0 con 0x7fd1cc103b60 2024-09-21T07:00:14.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.587+0000 7fd1c3400640 1 -- 172.21.15.114:0/1728164357 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12001 (secure 0 0 0) 0x7fd1bc0d74f0 con 0x7fd1cc103b60 2024-09-21T07:00:14.589 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:00:14.589 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"btime":"2024-09-21T06:59:58:855353+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:59:57.908080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:00:14.590 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 40 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1b00b6680 msgr2=0x7fd1b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1b00b6680 0x7fd1b00b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc1091a0 tx=0x7fd1b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 msgr2=0x7fd1cc19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc19a4a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc005940 tx=0x7fd1bc0040c0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 shutdown_connections 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd1b00b6680 0x7fd1b00b8b90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc109df0 0x7fd1cc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1cc1094a0 0x7fd1cc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 --2- 172.21.15.114:0/1728164357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd1cc103b60 0x7fd1cc19a4a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 >> 172.21.15.114:0/1728164357 conn(0x7fd1cc07b850 msgr2=0x7fd1cc107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 shutdown_connections 2024-09-21T07:00:14.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:14.590+0000 7fd1d1800640 1 -- 172.21.15.114:0/1728164357 wait complete. 2024-09-21T07:00:15.426 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 505c114e --release --await' 2024-09-21T07:00:15.427 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 505c114e --release --await 2024-09-21T07:00:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:15 smithi114 ceph-mon[30654]: pgmap v579: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.6 MiB/s wr, 246 op/s 2024-09-21T07:00:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:15 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1728164357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:15 smithi183 ceph-mon[34219]: pgmap v579: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.6 MiB/s wr, 246 op/s 2024-09-21T07:00:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:15 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1728164357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:15 smithi193 ceph-mon[34601]: pgmap v579: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.6 MiB/s wr, 246 op/s 2024-09-21T07:00:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:15 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1728164357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:15.798 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:16.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.464+0000 7f3ff6000640 1 -- 172.21.15.114:0/1992272557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff0105bb0 msgr2=0x7f3ff0105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.464+0000 7f3ff6000640 1 --2- 172.21.15.114:0/1992272557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff0105bb0 0x7f3ff0105fc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009900 tx=0x7f3fe002a410 comp rx=0 tx=0).stop 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 -- 172.21.15.114:0/1992272557 shutdown_connections 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 --2- 172.21.15.114:0/1992272557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff010be40 0x7f3ff0112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 --2- 172.21.15.114:0/1992272557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff010b4f0 0x7f3ff010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 --2- 172.21.15.114:0/1992272557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff0105bb0 0x7f3ff0105fc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 -- 172.21.15.114:0/1992272557 >> 172.21.15.114:0/1992272557 conn(0x7f3ff0101680 msgr2=0x7f3ff0103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 -- 172.21.15.114:0/1992272557 shutdown_connections 2024-09-21T07:00:16.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.465+0000 7f3ff6000640 1 -- 172.21.15.114:0/1992272557 wait complete. 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 Processor -- start 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 -- start start 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff0105bb0 0x7f3ff019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff010b4f0 0x7f3ff019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 0x7f3ff01a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0115b50 con 0x7f3ff010be40 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff01159d0 con 0x7f3ff010b4f0 2024-09-21T07:00:16.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.466+0000 7f3ff6000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff0115cd0 con 0x7f3ff0105bb0 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 0x7f3ff01a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 0x7f3ff01a1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45098/0 (socket says 172.21.15.114:45098) 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff4c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff0105bb0 0x7f3ff019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff5600640 1 -- 172.21.15.114:0/4291161684 learned_addr learned my addr 172.21.15.114:0/4291161684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3fefe00640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff010b4f0 0x7f3ff019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff4c00640 1 -- 172.21.15.114:0/4291161684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff010b4f0 msgr2=0x7f3ff019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff4c00640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff010b4f0 0x7f3ff019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff4c00640 1 -- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 msgr2=0x7f3ff01a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff4c00640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 0x7f3ff01a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff4c00640 1 -- 172.21.15.114:0/4291161684 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff01a2470 con 0x7f3ff0105bb0 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.467+0000 7f3ff5600640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 0x7f3ff01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:16.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.468+0000 7f3ff4c00640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff0105bb0 0x7f3ff019c500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3fe002aa00 tx=0x7f3fe0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:16.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.468+0000 7f3fed600640 1 -- 172.21.15.114:0/4291161684 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe003f070 con 0x7f3ff0105bb0 2024-09-21T07:00:16.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.468+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff01a2700 con 0x7f3ff0105bb0 2024-09-21T07:00:16.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.468+0000 7f3fed600640 1 -- 172.21.15.114:0/4291161684 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe0031b50 con 0x7f3ff0105bb0 2024-09-21T07:00:16.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.468+0000 7f3fed600640 1 -- 172.21.15.114:0/4291161684 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0035030 con 0x7f3ff0105bb0 2024-09-21T07:00:16.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.468+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff01a8ec0 con 0x7f3ff0105bb0 2024-09-21T07:00:16.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.471+0000 7f3ffe3ef740 1 Processor -- start 2024-09-21T07:00:16.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.471+0000 7f3ffe3ef740 1 -- start start 2024-09-21T07:00:16.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x558730597340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x558730597880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x558730597dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55873050e290 con 0x5587301cf840 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587304fd6c0 con 0x5587301f0520 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587305014d0 con 0x5587301c7660 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3fed600640 1 -- 172.21.15.114:0/4291161684 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f3fe0042070 con 0x7f3ff0105bb0 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x558730597340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x558730597340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40786/0 (socket says 172.21.15.114:40786) 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x558730597dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 -- 172.21.15.114:0/1962704544 learned_addr learned my addr 172.21.15.114:0/1962704544 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:16.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff4c00640 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x558730597880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 -- 172.21.15.114:0/1962704544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 msgr2=0x558730597880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x558730597880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 -- 172.21.15.114:0/1962704544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 msgr2=0x558730597dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x558730597dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.472+0000 7f3ff5600640 1 -- 172.21.15.114:0/1962704544 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873048a460 con 0x5587301f0520 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.473+0000 7f3ff5600640 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x558730597340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8011850 tx=0x7f3fe8011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3329f5d5456ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.473+0000 7f3fe5c00640 1 -- 172.21.15.114:0/1962704544 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8011330 con 0x5587301f0520 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.473+0000 7f3fe5c00640 1 -- 172.21.15.114:0/1962704544 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe800fe50 con 0x5587301f0520 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.473+0000 7f3fe5c00640 1 -- 172.21.15.114:0/1962704544 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80193e0 con 0x5587301f0520 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3fed600640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3fd40b6680 0x7f3fd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3fed600640 1 -- 172.21.15.114:0/4291161684 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3fe0107e80 con 0x7f3ff0105bb0 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ff4c00640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3fd40b6680 0x7f3fd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/1962704544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 msgr2=0x558730597340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x558730597340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8011850 tx=0x7f3fe8011d10 comp rx=0 tx=0).stop 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/1962704544 shutdown_connections 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x558730597dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x558730597880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/1962704544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x558730597340 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/1962704544 >> 172.21.15.114:0/1962704544 conn(0x5587301e4910 msgr2=0x558730595230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/1962704544 shutdown_connections 2024-09-21T07:00:16.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.474+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/1962704544 wait complete. 2024-09-21T07:00:16.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.475+0000 7f3ff4c00640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3fd40b6680 0x7f3fd40b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0105310 tx=0x7f3fe002a410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:16.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.475+0000 7f3ffe3ef740 1 -- start start 2024-09-21T07:00:16.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.475+0000 7f3ffe3ef740 1 Processor -- start 2024-09-21T07:00:16.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55873035ebb0 con 0x5587301cf840 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587304986c0 con 0x5587301f0520 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587305a5b20 con 0x5587301c7660 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ffe3ef740 10 client.0 _ll_register_callbacks cb 0x5587304b3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305a3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45124/0 (socket says 172.21.15.114:45124) 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3fefe00640 1 -- 172.21.15.114:0/3885106805 learned_addr learned my addr 172.21.15.114:0/3885106805 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.476+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fefe00640 1 -- 172.21.15.114:0/3885106805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 msgr2=0x5587305a2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305a2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fefe00640 1 -- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 msgr2=0x5587305aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fefe00640 1 -- 172.21.15.114:0/3885106805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873025bb00 con 0x5587301cf840 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305a3220 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00bf50 tx=0x7f3fdc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d056d6c7a905925c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fda000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fda000640 1 -- 172.21.15.114:0/3885106805 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc004020 con 0x5587301cf840 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe800fe50 con 0x5587301cf840 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.477+0000 7f3fda000640 1 -- 172.21.15.114:0/3885106805 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3fdc0041c0 con 0x5587301cf840 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3fda000640 1 -- 172.21.15.114:0/3885106805 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc024050 con 0x5587301cf840 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 10 client.16884 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5587302d2960 con 0x5587301cf840 2024-09-21T07:00:16.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3fda000640 1 -- 172.21.15.114:0/3885106805 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe800fe50 con 0x5587301cf840 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3fda000640 1 -- 172.21.15.114:0/3885106805 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f3fdc0037b0 con 0x5587301cf840 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 10 client.16884 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 10 client.16884 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 20 client.16884 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587301aeea0 0x5587305b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 4 client.16884 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.478+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587305101e0 con 0x5587301aeea0 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.479+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587301aeea0 0x5587305b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.479+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587301aeea0 0x5587305b4f10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59086dcc8ca0c489 server_cookie=391de23e4d6a1f74 in_seq=0 out_seq=0 2024-09-21T07:00:16.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.479+0000 7f3fda000640 10 client.16884 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.480+0000 7f3fda000640 1 -- 172.21.15.114:0/3885106805 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587305101e0 con 0x5587301aeea0 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.480+0000 7f3fda000640 10 client.16884 handle_command_reply: tid=0 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.480+0000 7f3ffe3ef740 1 client.16884 shutdown 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 msgr2=0x5587305a3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305a3220 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00bf50 tx=0x7f3fdc0118d0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 shutdown_connections 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305a3220 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587301aeea0 0x5587305b4f10 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3885106805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305a2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3fda000640 0 client.16884 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 >> 172.21.15.114:0/3885106805 conn(0x5587301e4910 msgr2=0x55873059a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 shutdown_connections 2024-09-21T07:00:16.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3885106805 wait complete. 2024-09-21T07:00:16.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.481+0000 7f3ffe3ef740 20 client.16884 trim_cache size 0 max 16384 2024-09-21T07:00:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 Processor -- start 2024-09-21T07:00:16.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 -- start start 2024-09-21T07:00:16.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 0x55873059aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301cf840 0x5587305a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301f0520 0x5587305aed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd000fed0 con 0x5587301c7660 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0011b60 con 0x5587301cf840 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00105a0 con 0x5587301f0520 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 0x55873059aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 0x55873059aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45132/0 (socket says 172.21.15.114:45132) 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.499+0000 7f3ff5600640 1 -- 172.21.15.114:0/3190161607 learned_addr learned my addr 172.21.15.114:0/3190161607 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301cf840 0x5587305a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301f0520 0x5587305aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 -- 172.21.15.114:0/3190161607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301f0520 msgr2=0x5587305aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301f0520 0x5587305aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 -- 172.21.15.114:0/3190161607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 msgr2=0x55873059aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 0x55873059aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 -- 172.21.15.114:0/3190161607 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558730594960 con 0x5587301cf840 2024-09-21T07:00:16.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 0x55873059aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:16.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301cf840 0x5587305a3120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc003bd0 tx=0x7f3fdc0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b77f740f45f80a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:16.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.500+0000 7f3fe5200640 1 -- 172.21.15.114:0/3190161607 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc02c070 con 0x5587301cf840 2024-09-21T07:00:16.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3fe5200640 1 -- 172.21.15.114:0/3190161607 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fdc01a3f0 con 0x5587301cf840 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3fe5200640 1 -- 172.21.15.114:0/3190161607 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0040c0 con 0x5587301cf840 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3190161607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301cf840 msgr2=0x5587305a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301cf840 0x5587305a3120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc003bd0 tx=0x7f3fdc0058d0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3190161607 shutdown_connections 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301f0520 0x5587305aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301cf840 0x5587305a3120 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3190161607 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301c7660 0x55873059aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3190161607 >> 172.21.15.114:0/3190161607 conn(0x5587301e4910 msgr2=0x5587305a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3190161607 shutdown_connections 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.501+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3190161607 wait complete. 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.502+0000 7f3ffe3ef740 1 -- start start 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.502+0000 7f3ffe3ef740 1 Processor -- start 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00113d0 con 0x5587301cf840 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd0010c70 con 0x5587301f0520 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd000deb0 con 0x5587301c7660 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ffe3ef740 10 client.0 _ll_register_callbacks cb 0x7f3fdc00ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3fefe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ff5600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39452/0 (socket says 172.21.15.114:39452) 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ff5600640 1 -- 172.21.15.114:0/3912231622 learned_addr learned my addr 172.21.15.114:0/3912231622 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:16.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3fefe00640 1 -- 172.21.15.114:0/3912231622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 msgr2=0x5587305aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3fefe00640 1 -- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 msgr2=0x5587305a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3fefe00640 1 -- 172.21.15.114:0/3912231622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587305a6480 con 0x5587301cf840 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.503+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3fefe00640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305ab580 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b450 tx=0x7f3fdc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d536c4eca89d5d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3fda000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3fda000640 1 -- 172.21.15.114:0/3912231622 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc035070 con 0x5587301cf840 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fdc01a3f0 con 0x5587301cf840 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3fda000640 1 -- 172.21.15.114:0/3912231622 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f3fdc005260 con 0x5587301cf840 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3fda000640 1 -- 172.21.15.114:0/3912231622 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3fdc041050 con 0x5587301cf840 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3ffe3ef740 10 client.16890 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.504+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558730620230 con 0x5587301cf840 2024-09-21T07:00:16.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3fda000640 1 -- 172.21.15.114:0/3912231622 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fdc01a3f0 con 0x5587301cf840 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3fda000640 1 -- 172.21.15.114:0/3912231622 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f3fdc010890 con 0x5587301cf840 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ffe3ef740 10 client.16890 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ffe3ef740 10 client.16890 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ffe3ef740 20 client.16890 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558730184250 0x5587306229c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ffe3ef740 4 client.16890 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "505c114e", "release": true, "await": true}] 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "505c114e", "release": true, "await": true}) -- 0x55873050bc90 con 0x558730184250 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.505+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558730184250 0x5587306229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.506+0000 7f3ff5600640 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558730184250 0x5587306229c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f3792fb7660b27 server_cookie=f4e451569fa293d6 in_seq=0 out_seq=0 2024-09-21T07:00:16.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.506+0000 7f3fda000640 10 client.16890 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.706+0000 7f3fda000640 1 -- 172.21.15.114:0/3912231622 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55873050bc90 con 0x558730184250 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.706+0000 7f3fda000640 10 client.16890 handle_command_reply: tid=0 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.706+0000 7f3ffe3ef740 1 client.16890 shutdown 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.706+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 msgr2=0x5587305ab580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.706+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305ab580 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00b450 tx=0x7f3fdc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 shutdown_connections 2024-09-21T07:00:16.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587301f0520 0x5587305a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587301cf840 0x5587305ab580 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558730184250 0x5587306229c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 --2- 172.21.15.114:0/3912231622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587301c7660 0x5587305aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3fda000640 0 client.16890 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 >> 172.21.15.114:0/3912231622 conn(0x5587301e4910 msgr2=0x558730598370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 shutdown_connections 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 1 -- 172.21.15.114:0/3912231622 wait complete. 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.707+0000 7f3ffe3ef740 20 client.16890 trim_cache size 0 max 16384 2024-09-21T07:00:16.710 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 40, 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 24, 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "505c114e": { 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 24, 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:16.711 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 139.2, 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:16.712 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3fd40b6680 msgr2=0x7f3fd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3fd40b6680 0x7f3fd40b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0105310 tx=0x7f3fe002a410 comp rx=0 tx=0).stop 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff0105bb0 msgr2=0x7f3ff019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff0105bb0 0x7f3ff019c500 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3fe002aa00 tx=0x7f3fe0003b90 comp rx=0 tx=0).stop 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 shutdown_connections 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3ff010be40 0x7f3ff01a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f3fd40b6680 0x7f3fd40b8b90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ff010b4f0 0x7f3ff019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 --2- 172.21.15.114:0/4291161684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ff0105bb0 0x7f3ff019c500 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:16.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.708+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 >> 172.21.15.114:0/4291161684 conn(0x7f3ff0101680 msgr2=0x7f3ff0103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:16.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.709+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 shutdown_connections 2024-09-21T07:00:16.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:16.709+0000 7f3ff6000640 1 -- 172.21.15.114:0/4291161684 wait complete. 2024-09-21T07:00:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:17 smithi114 ceph-mon[30654]: pgmap v580: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 654 KiB/s wr, 143 op/s 2024-09-21T07:00:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:17 smithi183 ceph-mon[34219]: pgmap v580: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 654 KiB/s wr, 143 op/s 2024-09-21T07:00:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:17 smithi193 ceph-mon[34601]: pgmap v580: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 654 KiB/s wr, 143 op/s 2024-09-21T07:00:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 340.0 s, 60.9 tps, lat 5867.671 ms stddev 7660.054 2024-09-21T07:00:17.917 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '505c114e', total seconds elapsed: 23.9 2024-09-21T07:00:17.917 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.25 2024-09-21T07:00:17.917 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:00:18.639 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:19.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.459+0000 7fb173200640 1 -- 172.21.15.114:0/55152705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 msgr2=0x7fb16c115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.459+0000 7fb173200640 1 --2- 172.21.15.114:0/55152705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c115fd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb160009900 tx=0x7fb16002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.460+0000 7fb173200640 1 -- 172.21.15.114:0/55152705 shutdown_connections 2024-09-21T07:00:19.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.460+0000 7fb173200640 1 --2- 172.21.15.114:0/55152705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c116510 0x7fb16c118950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.460+0000 7fb173200640 1 --2- 172.21.15.114:0/55152705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c115fd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.460+0000 7fb173200640 1 --2- 172.21.15.114:0/55152705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c10a3c0 0x7fb16c10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.460+0000 7fb173200640 1 -- 172.21.15.114:0/55152705 >> 172.21.15.114:0/55152705 conn(0x7fb16c06e9f0 msgr2=0x7fb16c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.465 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:19 smithi114 ceph-mon[30654]: pgmap v581: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-09-21T07:00:19.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.461+0000 7fb173200640 1 -- 172.21.15.114:0/55152705 shutdown_connections 2024-09-21T07:00:19.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.461+0000 7fb173200640 1 -- 172.21.15.114:0/55152705 wait complete. 2024-09-21T07:00:19.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.463+0000 7fb173200640 1 Processor -- start 2024-09-21T07:00:19.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.464+0000 7fb173200640 1 -- start start 2024-09-21T07:00:19.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.465+0000 7fb173200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c10a3c0 0x7fb16c1a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.466+0000 7fb173200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c1a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.466+0000 7fb173200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 0x7fb16c1ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.466+0000 7fb172800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 0x7fb16c1ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.467+0000 7fb172800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 0x7fb16c1ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40818/0 (socket says 172.21.15.114:40818) 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.467+0000 7fb172800640 1 -- 172.21.15.114:0/1451371025 learned_addr learned my addr 172.21.15.114:0/1451371025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.467+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c122710 con 0x7fb16c10a3c0 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.467+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c122590 con 0x7fb16c116510 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.467+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c122890 con 0x7fb16c10ad10 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.468+0000 7fb172800640 1 -- 172.21.15.114:0/1451371025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 msgr2=0x7fb16c1a9700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.468+0000 7fb171e00640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c10a3c0 0x7fb16c1a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.469+0000 7fb171400640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c1a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.469+0000 7fb172800640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c1a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.469+0000 7fb172800640 1 -- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c10a3c0 msgr2=0x7fb16c1a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.469+0000 7fb172800640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c10a3c0 0x7fb16c1a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.469+0000 7fb172800640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c1af0d0 con 0x7fb16c116510 2024-09-21T07:00:19.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.469+0000 7fb171e00640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c10a3c0 0x7fb16c1a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:19.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.470+0000 7fb171400640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c1a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:19.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.470+0000 7fb172800640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 0x7fb16c1ae9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e9d0 tx=0x7fb16800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.473+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16800cd80 con 0x7fb16c116510 2024-09-21T07:00:19.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.473+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb168005020 con 0x7fb16c116510 2024-09-21T07:00:19.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.473+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1680116d0 con 0x7fb16c116510 2024-09-21T07:00:19.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.473+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c1af3c0 con 0x7fb16c116510 2024-09-21T07:00:19.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.473+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c1b5b20 con 0x7fb16c116510 2024-09-21T07:00:19.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.475+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16c10a7d0 con 0x7fb16c116510 2024-09-21T07:00:19.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.478+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb1680051d0 con 0x7fb16c116510 2024-09-21T07:00:19.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.481+0000 7fb166a00640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb1540b6840 0x7fb1540b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.481+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fb1680e7b90 con 0x7fb16c116510 2024-09-21T07:00:19.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.484+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb168016030 con 0x7fb16c116510 2024-09-21T07:00:19.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.487+0000 7fb171e00640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb1540b6840 0x7fb1540b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.488+0000 7fb171e00640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb1540b6840 0x7fb1540b8d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009b50 tx=0x7fb15c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:19 smithi183 ceph-mon[34219]: pgmap v581: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-09-21T07:00:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 -- 172.21.15.114:0/416594632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 msgr2=0x7f605c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 --2- 172.21.15.114:0/416594632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c102dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6054009ca0 tx=0x7f605402a8f0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 -- 172.21.15.114:0/416594632 shutdown_connections 2024-09-21T07:00:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 --2- 172.21.15.114:0/416594632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f605c108c30 0x7f605c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 --2- 172.21.15.114:0/416594632 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c1082e0 0x7f605c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 --2- 172.21.15.114:0/416594632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c102dc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.634+0000 7f6062a00640 1 -- 172.21.15.114:0/416594632 >> 172.21.15.114:0/416594632 conn(0x7f605c07a740 msgr2=0x7f605c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.636+0000 7f6062a00640 1 -- 172.21.15.114:0/416594632 shutdown_connections 2024-09-21T07:00:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.637+0000 7f6062a00640 1 -- 172.21.15.114:0/416594632 wait complete. 2024-09-21T07:00:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.638+0000 7f6062a00640 1 Processor -- start 2024-09-21T07:00:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 -- start start 2024-09-21T07:00:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f605c1082e0 0x7f605c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c108c30 0x7f605c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f605c112980 con 0x7f605c108c30 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f605c112800 con 0x7f605c1029b0 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.639+0000 7f6062a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f605c112b00 con 0x7f605c1082e0 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.640+0000 7f6060a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c108c30 0x7f605c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.640+0000 7f6060a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c108c30 0x7f605c19db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45152/0 (socket says 172.21.15.114:45152) 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.640+0000 7f6060a00640 1 -- 172.21.15.114:0/2822249023 learned_addr learned my addr 172.21.15.114:0/2822249023 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.640+0000 7f605be00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.641+0000 7f605be00640 1 -- 172.21.15.114:0/2822249023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f605c1082e0 msgr2=0x7f605c198940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.641+0000 7f605be00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f605c1082e0 0x7f605c198940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.641+0000 7f605be00640 1 -- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c108c30 msgr2=0x7f605c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.641+0000 7f605be00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c108c30 0x7f605c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.641+0000 7f605be00640 1 -- 172.21.15.114:0/2822249023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c19e1e0 con 0x7f605c1029b0 2024-09-21T07:00:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.641+0000 7f605be00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c1983c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6054000f80 tx=0x7f6054031a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.642+0000 7f6058c00640 1 -- 172.21.15.114:0/2822249023 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054003b70 con 0x7f605c1029b0 2024-09-21T07:00:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.642+0000 7f6058c00640 1 -- 172.21.15.114:0/2822249023 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6054003d10 con 0x7f605c1029b0 2024-09-21T07:00:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.642+0000 7f6058c00640 1 -- 172.21.15.114:0/2822249023 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605402fb80 con 0x7f605c1029b0 2024-09-21T07:00:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.642+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c19e470 con 0x7f605c1029b0 2024-09-21T07:00:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.643+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c19ec70 con 0x7f605c1029b0 2024-09-21T07:00:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.656+0000 7f6069a7a740 1 Processor -- start 2024-09-21T07:00:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.656+0000 7f6058c00640 1 -- 172.21.15.114:0/2822249023 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f605402fd20 con 0x7f605c1029b0 2024-09-21T07:00:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.656+0000 7f6069a7a740 1 -- start start 2024-09-21T07:00:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.656+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cd42f0 0x55da150a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cdc4d0 0x55da150a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f6069a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55da150176b0 con 0x55da14cfd1b0 2024-09-21T07:00:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f6069a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da15006ae0 con 0x55da14cdc4d0 2024-09-21T07:00:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f6069a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da1500a8f0 con 0x55da14cd42f0 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f605b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cdc4d0 0x55da150a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f6060a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f605b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cdc4d0 0x55da150a26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40848/0 (socket says 172.21.15.114:40848) 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.657+0000 7f605b400640 1 -- 172.21.15.114:0/3090082656 learned_addr learned my addr 172.21.15.114:0/3090082656 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.658+0000 7f6060a00640 1 -- 172.21.15.114:0/3090082656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cd42f0 msgr2=0x55da150a2170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.658+0000 7f6060a00640 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cd42f0 0x55da150a2170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.658+0000 7f6060a00640 1 -- 172.21.15.114:0/3090082656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cdc4d0 msgr2=0x55da150a26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.658+0000 7f6060a00640 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cdc4d0 0x55da150a26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.658+0000 7f6060a00640 1 -- 172.21.15.114:0/3090082656 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da14fd3780 con 0x55da14cfd1b0 2024-09-21T07:00:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:19 smithi193 ceph-mon[34601]: pgmap v581: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-09-21T07:00:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.658+0000 7f6058c00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60400b6850 0x7f60400b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.659+0000 7f6058c00640 1 -- 172.21.15.114:0/2822249023 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f60541086a0 con 0x7f605c1029b0 2024-09-21T07:00:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.660+0000 7f6060a00640 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a1c30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f604c009c70 tx=0x7f604c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df23bb0dc919701 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.660+0000 7f605be00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60400b6850 0x7f60400b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.661+0000 7f6051200640 1 -- 172.21.15.114:0/3090082656 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c005220 con 0x55da14cfd1b0 2024-09-21T07:00:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.661+0000 7f6051200640 1 -- 172.21.15.114:0/3090082656 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c004c50 con 0x55da14cfd1b0 2024-09-21T07:00:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.661+0000 7f6051200640 1 -- 172.21.15.114:0/3090082656 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c0197b0 con 0x55da14cfd1b0 2024-09-21T07:00:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.663+0000 7f6069a7a740 1 -- 172.21.15.114:0/3090082656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 msgr2=0x55da150a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.663+0000 7f605be00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60400b6850 0x7f60400b8d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55da150a08b0 tx=0x7f605402a8f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.663+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a1c30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f604c009c70 tx=0x7f604c011810 comp rx=0 tx=0).stop 2024-09-21T07:00:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.665+0000 7f6069a7a740 1 -- 172.21.15.114:0/3090082656 shutdown_connections 2024-09-21T07:00:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.665+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cdc4d0 0x55da150a26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.665+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cd42f0 0x55da150a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.665+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3090082656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a1c30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.665+0000 7f6069a7a740 1 -- 172.21.15.114:0/3090082656 >> 172.21.15.114:0/3090082656 conn(0x55da14cf1300 msgr2=0x55da1509fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.666+0000 7f6069a7a740 1 -- 172.21.15.114:0/3090082656 shutdown_connections 2024-09-21T07:00:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.666+0000 7f6069a7a740 1 -- 172.21.15.114:0/3090082656 wait complete. 2024-09-21T07:00:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.667+0000 7f6069a7a740 1 -- start start 2024-09-21T07:00:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.667+0000 7f6069a7a740 1 Processor -- start 2024-09-21T07:00:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cdc4d0 0x55da150ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cfd1b0 0x55da150adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55da14f3eab0 con 0x55da14cdc4d0 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55da14fad570 con 0x55da14cd42f0 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55da150b03b0 con 0x55da14cfd1b0 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6060a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6060a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40858/0 (socket says 172.21.15.114:40858) 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6060a00640 1 -- 172.21.15.114:0/1351514678 learned_addr learned my addr 172.21.15.114:0/1351514678 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.669+0000 7f6069a7a740 10 client.0 _ll_register_callbacks cb 0x55da14fbc100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cfd1b0 0x55da150adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 -- 172.21.15.114:0/1351514678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cfd1b0 msgr2=0x55da150adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cfd1b0 0x55da150adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 -- 172.21.15.114:0/1351514678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cdc4d0 msgr2=0x55da150ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cdc4d0 0x55da150ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 -- 172.21.15.114:0/1351514678 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da14d68790 con 0x55da14cd42f0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6060a00640 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150ad210 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f604c0098c0 tx=0x7f604c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b9507849c05625 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.670+0000 7f6049600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.671+0000 7f6049600640 1 -- 172.21.15.114:0/1351514678 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c02c020 con 0x55da14cd42f0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.671+0000 7f6049600640 1 -- 172.21.15.114:0/1351514678 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f604c019530 con 0x55da14cd42f0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.671+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604c004c50 con 0x55da14cd42f0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.671+0000 7f6049600640 1 -- 172.21.15.114:0/1351514678 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f604c019730 con 0x55da14cd42f0 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.671+0000 7f6069a7a740 10 client.27215 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.671+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55da14de1620 con 0x55da14cd42f0 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6049600640 1 -- 172.21.15.114:0/1351514678 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c004c50 con 0x55da14cd42f0 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6049600640 1 -- 172.21.15.114:0/1351514678 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f604c028a50 con 0x55da14cd42f0 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6069a7a740 10 client.27215 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6069a7a740 10 client.27215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6069a7a740 20 client.27215 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6069a7a740 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14cbbd80 0x55da150bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6069a7a740 4 client.27215 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.672+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da15019600 con 0x55da14cbbd80 2024-09-21T07:00:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.673+0000 7f605b400640 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14cbbd80 0x55da150bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.676+0000 7f605b400640 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14cbbd80 0x55da150bf290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba505b3cf9ecdfb server_cookie=de016d19e7414bba in_seq=0 out_seq=0 2024-09-21T07:00:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.676+0000 7f6049600640 10 client.27215 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.677+0000 7f6049600640 1 -- 172.21.15.114:0/1351514678 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da15019600 con 0x55da14cbbd80 2024-09-21T07:00:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.677+0000 7f6049600640 10 client.27215 handle_command_reply: tid=0 2024-09-21T07:00:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.677+0000 7f6069a7a740 1 client.27215 shutdown 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 msgr2=0x55da150ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150ad210 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f604c0098c0 tx=0x7f604c009790 comp rx=0 tx=0).stop 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 shutdown_connections 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14cbbd80 0x55da150bf290 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cfd1b0 0x55da150adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cdc4d0 0x55da150ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 --2- 172.21.15.114:0/1351514678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150ad210 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 >> 172.21.15.114:0/1351514678 conn(0x55da14cf1300 msgr2=0x55da150a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6049600640 0 client.27215 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 shutdown_connections 2024-09-21T07:00:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 1 -- 172.21.15.114:0/1351514678 wait complete. 2024-09-21T07:00:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.678+0000 7f6069a7a740 20 client.27215 trim_cache size 0 max 16384 2024-09-21T07:00:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.707+0000 7f6069a7a740 1 Processor -- start 2024-09-21T07:00:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.707+0000 7f6069a7a740 1 -- start start 2024-09-21T07:00:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da1509ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6069a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f605c075320 con 0x55da14cfd1b0 2024-09-21T07:00:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6069a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f605c1ca080 con 0x55da14cd42f0 2024-09-21T07:00:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6069a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f605c1c8ac0 con 0x55da14cdc4d0 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6060a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da1509ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6060a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da1509ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40870/0 (socket says 172.21.15.114:40870) 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6060a00640 1 -- 172.21.15.114:0/3498434586 learned_addr learned my addr 172.21.15.114:0/3498434586 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f6060a00640 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.708+0000 7f605b400640 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.709+0000 7f6060a00640 1 -- 172.21.15.114:0/3498434586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 msgr2=0x55da150af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.709+0000 7f6060a00640 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.709+0000 7f6060a00640 1 -- 172.21.15.114:0/3498434586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 msgr2=0x55da150a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.709+0000 7f6060a00640 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.709+0000 7f6060a00640 1 -- 172.21.15.114:0/3498434586 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da150a04a0 con 0x55da14cd42f0 2024-09-21T07:00:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.709+0000 7f6060a00640 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da1509ebb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f604c004b20 tx=0x7f604c009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ef88b03e3197ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.710+0000 7f604be00640 1 -- 172.21.15.114:0/3498434586 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c02c040 con 0x55da14cd42f0 2024-09-21T07:00:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.710+0000 7f604be00640 1 -- 172.21.15.114:0/3498434586 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c019d80 con 0x55da14cd42f0 2024-09-21T07:00:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.710+0000 7f604be00640 1 -- 172.21.15.114:0/3498434586 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c026a30 con 0x55da14cd42f0 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 -- 172.21.15.114:0/3498434586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 msgr2=0x55da1509ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da1509ebb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f604c004b20 tx=0x7f604c009720 comp rx=0 tx=0).stop 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 -- 172.21.15.114:0/3498434586 shutdown_connections 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 --2- 172.21.15.114:0/3498434586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da1509ebb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.712+0000 7f6069a7a740 1 -- 172.21.15.114:0/3498434586 >> 172.21.15.114:0/3498434586 conn(0x55da14cf1300 msgr2=0x55da150a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.713+0000 7f6069a7a740 1 -- 172.21.15.114:0/3498434586 shutdown_connections 2024-09-21T07:00:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.713+0000 7f6069a7a740 1 -- 172.21.15.114:0/3498434586 wait complete. 2024-09-21T07:00:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.714+0000 7f6069a7a740 1 -- start start 2024-09-21T07:00:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.714+0000 7f6069a7a740 1 Processor -- start 2024-09-21T07:00:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f605c1c98f0 con 0x55da14cfd1b0 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f605c1c9190 con 0x55da14cd42f0 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f605c073300 con 0x55da14cdc4d0 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f6069a7a740 10 client.0 _ll_register_callbacks cb 0x7f604c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f605b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f605b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39522/0 (socket says 172.21.15.114:39522) 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.715+0000 7f605b400640 1 -- 172.21.15.114:0/571148435 learned_addr learned my addr 172.21.15.114:0/571148435 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.716+0000 7f6060a00640 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.716+0000 7f6060a00640 1 -- 172.21.15.114:0/571148435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 msgr2=0x55da150b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.716+0000 7f6060a00640 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.716+0000 7f6060a00640 1 -- 172.21.15.114:0/571148435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 msgr2=0x55da150b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.716+0000 7f6060a00640 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.716+0000 7f6060a00640 1 -- 172.21.15.114:0/571148435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da150b3b40 con 0x55da14cd42f0 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.717+0000 7f6049600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.717+0000 7f6060a00640 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150b5b60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f604c026450 tx=0x7f604c009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50000bf6a623f169 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.717+0000 7f6049600640 1 -- 172.21.15.114:0/571148435 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c02c040 con 0x55da14cd42f0 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.717+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604c019d80 con 0x55da14cd42f0 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.717+0000 7f6049600640 1 -- 172.21.15.114:0/571148435 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f604c040040 con 0x55da14cd42f0 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6049600640 1 -- 172.21.15.114:0/571148435 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f604c02c040 con 0x55da14cd42f0 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 10 client.27221 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55da151298c0 con 0x55da14cd42f0 2024-09-21T07:00:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6049600640 1 -- 172.21.15.114:0/571148435 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c019d80 con 0x55da14cd42f0 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6049600640 1 -- 172.21.15.114:0/571148435 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7f604c028a20 con 0x55da14cd42f0 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 10 client.27221 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 10 client.27221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 20 client.27221 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14c911c0 0x55da1512c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 4 client.27221 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.718+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da150150b0 con 0x55da14c911c0 2024-09-21T07:00:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.719+0000 7f605b400640 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14c911c0 0x55da1512c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.720+0000 7f6049600640 10 client.27221 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.720+0000 7f605b400640 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14c911c0 0x55da1512c060 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4d7d470b84fcf6 server_cookie=6d6ef6f06a722b84 in_seq=0 out_seq=0 2024-09-21T07:00:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.729+0000 7f6049600640 1 -- 172.21.15.114:0/571148435 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da150150b0 con 0x55da14c911c0 2024-09-21T07:00:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.729+0000 7f6049600640 10 client.27221 handle_command_reply: tid=0 2024-09-21T07:00:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 client.27221 shutdown 2024-09-21T07:00:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 msgr2=0x55da150b5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150b5b60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f604c026450 tx=0x7f604c009790 comp rx=0 tx=0).stop 2024-09-21T07:00:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 shutdown_connections 2024-09-21T07:00:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55da14cfd1b0 0x55da150b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55da14c911c0 0x55da1512c060 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55da14cdc4d0 0x55da150b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 --2- 172.21.15.114:0/571148435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55da14cd42f0 0x55da150b5b60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 >> 172.21.15.114:0/571148435 conn(0x55da14cf1300 msgr2=0x55da150aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6049600640 0 client.27221 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 shutdown_connections 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.730+0000 7f6069a7a740 1 -- 172.21.15.114:0/571148435 wait complete. 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.731+0000 7f6069a7a740 20 client.27221 trim_cache size 0 max 16384 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:00:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60400b6850 msgr2=0x7f60400b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60400b6850 0x7f60400b8d60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55da150a08b0 tx=0x7f605402a8f0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 msgr2=0x7f605c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c1983c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6054000f80 tx=0x7f6054031a00 comp rx=0 tx=0).stop 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 shutdown_connections 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f605c108c30 0x7f605c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f605c1082e0 0x7f605c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60400b6850 0x7f60400b8d60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 --2- 172.21.15.114:0/2822249023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f605c1029b0 0x7f605c1983c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 >> 172.21.15.114:0/2822249023 conn(0x7f605c07a740 msgr2=0x7f605c1062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 shutdown_connections 2024-09-21T07:00:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:19.732+0000 7f6062a00640 1 -- 172.21.15.114:0/2822249023 wait complete. 2024-09-21T07:00:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.926+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb16c1b5db0 con 0x7fb16c116510 2024-09-21T07:00:19.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.927+0000 7fb166a00640 1 -- 172.21.15.114:0/1451371025 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+12001 (secure 0 0 0) 0x7fb1680add10 con 0x7fb16c116510 2024-09-21T07:00:19.931 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:00:19.931 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":40,"btime":"2024-09-21T06:59:58:855353+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T06:59:57.908080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:00:19.931 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 40 2024-09-21T07:00:19.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.931+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb1540b6840 msgr2=0x7fb1540b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.931+0000 7fb173200640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb1540b6840 0x7fb1540b8d50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009b50 tx=0x7fb15c005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:19.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.931+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 msgr2=0x7fb16c1ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:19.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.931+0000 7fb173200640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 0x7fb16c1ae9a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e9d0 tx=0x7fb16800ee90 comp rx=0 tx=0).stop 2024-09-21T07:00:19.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.932+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 shutdown_connections 2024-09-21T07:00:19.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.932+0000 7fb173200640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb1540b6840 0x7fb1540b8d50 secure :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009b50 tx=0x7fb15c005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:19.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.932+0000 7fb173200640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c116510 0x7fb16c1ae9a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.932+0000 7fb173200640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb16c10ad10 0x7fb16c1a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.932+0000 7fb173200640 1 --2- 172.21.15.114:0/1451371025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb16c10a3c0 0x7fb16c1a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:19.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.932+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 >> 172.21.15.114:0/1451371025 conn(0x7fb16c06e9f0 msgr2=0x7fb16c10e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:19.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.933+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 shutdown_connections 2024-09-21T07:00:19.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:19.933+0000 7fb173200640 1 -- 172.21.15.114:0/1451371025 wait complete. 2024-09-21T07:00:20.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:20 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1451371025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:20 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1451371025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:20 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1451371025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:20.872 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0' 2024-09-21T07:00:20.873 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 139.25 --await-for 45.0 2024-09-21T07:00:21.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:21 smithi114 ceph-mon[30654]: pgmap v582: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-21T07:00:21.567 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:21.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:21 smithi183 ceph-mon[34219]: pgmap v582: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-21T07:00:21.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:21 smithi193 ceph-mon[34601]: pgmap v582: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-21T07:00:22.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.674+0000 7fe144600640 1 -- 172.21.15.114:0/1608811513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c105da0 msgr2=0x7fe13c1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.674+0000 7fe144600640 1 --2- 172.21.15.114:0/1608811513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c105da0 0x7fe13c1061b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009900 tx=0x7fe12c02a410 comp rx=0 tx=0).stop 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.675+0000 7fe144600640 1 -- 172.21.15.114:0/1608811513 shutdown_connections 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.675+0000 7fe144600640 1 --2- 172.21.15.114:0/1608811513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c112860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.675+0000 7fe144600640 1 --2- 172.21.15.114:0/1608811513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c10b6e0 0x7fe13c10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.675+0000 7fe144600640 1 --2- 172.21.15.114:0/1608811513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c105da0 0x7fe13c1061b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.675+0000 7fe144600640 1 -- 172.21.15.114:0/1608811513 >> 172.21.15.114:0/1608811513 conn(0x7fe13c101790 msgr2=0x7fe13c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.676+0000 7fe144600640 1 -- 172.21.15.114:0/1608811513 shutdown_connections 2024-09-21T07:00:22.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.676+0000 7fe144600640 1 -- 172.21.15.114:0/1608811513 wait complete. 2024-09-21T07:00:22.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.678+0000 7fe144600640 1 Processor -- start 2024-09-21T07:00:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.679+0000 7fe144600640 1 -- start start 2024-09-21T07:00:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.680+0000 7fe144600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c105da0 0x7fe13c10fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.680+0000 7fe144600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c10b6e0 0x7fe13c1101c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.680+0000 7fe144600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c110700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.680+0000 7fe144600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c115d40 con 0x7fe13c10c030 2024-09-21T07:00:22.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.680+0000 7fe144600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c115bc0 con 0x7fe13c10b6e0 2024-09-21T07:00:22.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.680+0000 7fe144600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe13c115ec0 con 0x7fe13c105da0 2024-09-21T07:00:22.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.681+0000 7fe142600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c110700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.681+0000 7fe142600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c110700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45216/0 (socket says 172.21.15.114:45216) 2024-09-21T07:00:22.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.681+0000 7fe142600640 1 -- 172.21.15.114:0/4097965870 learned_addr learned my addr 172.21.15.114:0/4097965870 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:22.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.682+0000 7fe141c00640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c105da0 0x7fe13c10fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.682+0000 7fe141200640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c10b6e0 0x7fe13c1101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.683+0000 7fe142600640 1 -- 172.21.15.114:0/4097965870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c105da0 msgr2=0x7fe13c10fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.683+0000 7fe142600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c105da0 0x7fe13c10fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.683+0000 7fe142600640 1 -- 172.21.15.114:0/4097965870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c10b6e0 msgr2=0x7fe13c1101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.683+0000 7fe142600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c10b6e0 0x7fe13c1101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.683+0000 7fe142600640 1 -- 172.21.15.114:0/4097965870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe13c19b2f0 con 0x7fe13c10c030 2024-09-21T07:00:22.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.684+0000 7fe141c00640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c105da0 0x7fe13c10fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:22.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.684+0000 7fe141200640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c10b6e0 0x7fe13c1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:22.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.684+0000 7fe142600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c110700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe134009fd0 tx=0x7fe13400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:22.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.685+0000 7fe13a000640 1 -- 172.21.15.114:0/4097965870 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134017070 con 0x7fe13c10c030 2024-09-21T07:00:22.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.685+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe13c19b5e0 con 0x7fe13c10c030 2024-09-21T07:00:22.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.686+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13c19bb70 con 0x7fe13c10c030 2024-09-21T07:00:22.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.688+0000 7fe13a000640 1 -- 172.21.15.114:0/4097965870 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe134003e20 con 0x7fe13c10c030 2024-09-21T07:00:22.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.688+0000 7fe13a000640 1 -- 172.21.15.114:0/4097965870 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134010460 con 0x7fe13c10c030 2024-09-21T07:00:22.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 Processor -- start 2024-09-21T07:00:22.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 -- start start 2024-09-21T07:00:22.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4570990 0x5650f4940ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4578b70 0x5650f4941430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650f48b6a90 con 0x5650f4570990 2024-09-21T07:00:22.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650f48a5ec0 con 0x5650f4578b70 2024-09-21T07:00:22.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.692+0000 7fe14b201740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650f48a9cd0 con 0x5650f4599850 2024-09-21T07:00:22.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.693+0000 7fe141200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.693+0000 7fe141200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39540/0 (socket says 172.21.15.114:39540) 2024-09-21T07:00:22.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.693+0000 7fe141200640 1 -- 172.21.15.114:0/2602567956 learned_addr learned my addr 172.21.15.114:0/2602567956 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:22.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.693+0000 7fe141200640 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4578b70 0x5650f4941430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.694+0000 7fe141200640 1 -- 172.21.15.114:0/2602567956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4578b70 msgr2=0x5650f4941430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.694+0000 7fe141200640 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4578b70 0x5650f4941430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.694+0000 7fe141200640 1 -- 172.21.15.114:0/2602567956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4570990 msgr2=0x5650f4940ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.694+0000 7fe141200640 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4570990 0x5650f4940ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.695+0000 7fe141200640 1 -- 172.21.15.114:0/2602567956 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f484b9d0 con 0x5650f4599850 2024-09-21T07:00:22.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.695+0000 7fe141200640 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49409b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe130012b20 tx=0x7fe130004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5504b0b8f9cb1e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:22.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.696+0000 7fe12aa00640 1 -- 172.21.15.114:0/2602567956 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130004020 con 0x5650f4599850 2024-09-21T07:00:22.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.696+0000 7fe12aa00640 1 -- 172.21.15.114:0/2602567956 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1300054d0 con 0x5650f4599850 2024-09-21T07:00:22.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.696+0000 7fe12aa00640 1 -- 172.21.15.114:0/2602567956 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130010c20 con 0x5650f4599850 2024-09-21T07:00:22.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.698+0000 7fe14b201740 1 -- 172.21.15.114:0/2602567956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 msgr2=0x5650f49409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.698+0000 7fe14b201740 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49409b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe130012b20 tx=0x7fe130004730 comp rx=0 tx=0).stop 2024-09-21T07:00:22.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.710+0000 7fe14b201740 1 -- 172.21.15.114:0/2602567956 shutdown_connections 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.710+0000 7fe14b201740 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4578b70 0x5650f4941430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.710+0000 7fe14b201740 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4570990 0x5650f4940ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.710+0000 7fe14b201740 1 --2- 172.21.15.114:0/2602567956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49409b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.710+0000 7fe14b201740 1 -- 172.21.15.114:0/2602567956 >> 172.21.15.114:0/2602567956 conn(0x5650f458dc40 msgr2=0x5650f493e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.710+0000 7fe13a000640 1 -- 172.21.15.114:0/4097965870 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe134003970 con 0x7fe13c10c030 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.711+0000 7fe14b201740 1 -- 172.21.15.114:0/2602567956 shutdown_connections 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.711+0000 7fe14b201740 1 -- 172.21.15.114:0/2602567956 wait complete. 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.712+0000 7fe14b201740 1 -- start start 2024-09-21T07:00:22.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.712+0000 7fe14b201740 1 Processor -- start 2024-09-21T07:00:22.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.715+0000 7fe14b201740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4570990 0x5650f494c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.716+0000 7fe141200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4570990 0x5650f494c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.716+0000 7fe141200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4570990 0x5650f494c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40896/0 (socket says 172.21.15.114:40896) 2024-09-21T07:00:22.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.716+0000 7fe141200640 1 -- 172.21.15.114:0/3470742714 learned_addr learned my addr 172.21.15.114:0/3470742714 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.717+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f494ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.717+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f4958510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.717+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650f47077b0 con 0x5650f4578b70 2024-09-21T07:00:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.717+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650f484a650 con 0x5650f4570990 2024-09-21T07:00:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.717+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650f494f190 con 0x5650f4599850 2024-09-21T07:00:22.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.718+0000 7fe14b201740 10 client.0 _ll_register_callbacks cb 0x5650f45b10a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.718+0000 7fe13a000640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1240b6690 0x7fe1240b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.718+0000 7fe13a000640 1 -- 172.21.15.114:0/4097965870 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe1340e7880 con 0x7fe13c10c030 2024-09-21T07:00:22.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.720+0000 7fe141c00640 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f494ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.720+0000 7fe142600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1240b6690 0x7fe1240b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe141c00640 1 -- 172.21.15.114:0/3470742714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 msgr2=0x5650f4958510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe141c00640 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f4958510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe141c00640 1 -- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4570990 msgr2=0x5650f494c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe141c00640 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4570990 0x5650f494c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe141c00640 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f4613ba0 con 0x5650f4578b70 2024-09-21T07:00:22.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe141c00640 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f494ca50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe12c02a930 tx=0x7fe12c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c286446cf3a76c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:22.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.722+0000 7fe11f400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:22.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.724+0000 7fe11f400640 1 -- 172.21.15.114:0/3470742714 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c03f070 con 0x5650f4578b70 2024-09-21T07:00:22.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.725+0000 7fe142600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1240b6690 0x7fe1240b8ba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe1340ad000 tx=0x7fe13401dfe0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:22.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.725+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1300054d0 con 0x5650f4578b70 2024-09-21T07:00:22.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.725+0000 7fe11f400640 1 -- 172.21.15.114:0/3470742714 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe12c005450 con 0x5650f4578b70 2024-09-21T07:00:22.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.726+0000 7fe11f400640 1 -- 172.21.15.114:0/3470742714 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe12c04b050 con 0x5650f4578b70 2024-09-21T07:00:22.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.726+0000 7fe14b201740 10 client.16911 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:22.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.726+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5650f495b200 con 0x5650f4578b70 2024-09-21T07:00:22.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.728+0000 7fe11f400640 1 -- 172.21.15.114:0/3470742714 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1300054d0 con 0x5650f4578b70 2024-09-21T07:00:22.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.728+0000 7fe11f400640 1 -- 172.21.15.114:0/3470742714 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7fe12c0038d0 con 0x5650f4578b70 2024-09-21T07:00:22.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.728+0000 7fe14b201740 10 client.16911 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:22.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.728+0000 7fe14b201740 10 client.16911 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:22.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.729+0000 7fe14b201740 20 client.16911 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:22.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.729+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f45581d0 0x5650f495e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.729+0000 7fe14b201740 4 client.16911 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:22.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.729+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650f48b89e0 con 0x5650f45581d0 2024-09-21T07:00:22.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.729+0000 7fe141200640 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f45581d0 0x5650f495e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.731+0000 7fe141200640 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f45581d0 0x5650f495e7b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7dc3448346bb4c7 server_cookie=4b5fd99ad6108bb8 in_seq=0 out_seq=0 2024-09-21T07:00:22.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.731+0000 7fe11f400640 10 client.16911 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:22.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.734+0000 7fe11f400640 1 -- 172.21.15.114:0/3470742714 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650f48b89e0 con 0x5650f45581d0 2024-09-21T07:00:22.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.734+0000 7fe11f400640 10 client.16911 handle_command_reply: tid=0 2024-09-21T07:00:22.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.734+0000 7fe14b201740 1 client.16911 shutdown 2024-09-21T07:00:22.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 msgr2=0x5650f494ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f494ca50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe12c02a930 tx=0x7fe12c0043b0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 shutdown_connections 2024-09-21T07:00:22.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f45581d0 0x5650f495e7b0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f4958510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f494ca50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 --2- 172.21.15.114:0/3470742714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4570990 0x5650f494c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe11f400640 0 client.16911 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.735+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 >> 172.21.15.114:0/3470742714 conn(0x5650f458dc40 msgr2=0x5650f4943b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.736+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 shutdown_connections 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.736+0000 7fe14b201740 1 -- 172.21.15.114:0/3470742714 wait complete. 2024-09-21T07:00:22.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.736+0000 7fe14b201740 20 client.16911 trim_cache size 0 max 16384 2024-09-21T07:00:22.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.781+0000 7fe14b201740 1 Processor -- start 2024-09-21T07:00:22.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.781+0000 7fe14b201740 1 -- start start 2024-09-21T07:00:22.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe14b201740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 0x5650f493d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe14b201740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f493e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe14b201740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4599850 0x5650f494df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe14b201740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe12000ff10 con 0x5650f4578b70 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe14b201740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe120011ba0 con 0x5650f4599850 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe14b201740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe1200105e0 con 0x5650f4570990 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe141200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 0x5650f493d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe141200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 0x5650f493d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39556/0 (socket says 172.21.15.114:39556) 2024-09-21T07:00:22.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe141200640 1 -- 172.21.15.114:0/3437627822 learned_addr learned my addr 172.21.15.114:0/3437627822 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:22.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.782+0000 7fe141200640 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4599850 0x5650f494df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.783+0000 7fe141c00640 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f493e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.785+0000 7fe141200640 1 -- 172.21.15.114:0/3437627822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4599850 msgr2=0x5650f494df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.785+0000 7fe141200640 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4599850 0x5650f494df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.785+0000 7fe141200640 1 -- 172.21.15.114:0/3437627822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 msgr2=0x5650f493e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.785+0000 7fe141200640 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f493e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.785+0000 7fe141200640 1 -- 172.21.15.114:0/3437627822 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f494d0a0 con 0x5650f4570990 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.785+0000 7fe141200640 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 0x5650f493d8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe1300057c0 tx=0x7fe130015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e14c49672b0976 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:22.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.786+0000 7fe12a000640 1 -- 172.21.15.114:0/3437627822 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130010b40 con 0x5650f4570990 2024-09-21T07:00:22.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.786+0000 7fe12a000640 1 -- 172.21.15.114:0/3437627822 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe130016040 con 0x5650f4570990 2024-09-21T07:00:22.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.790+0000 7fe12a000640 1 -- 172.21.15.114:0/3437627822 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe130003a50 con 0x5650f4570990 2024-09-21T07:00:22.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 -- 172.21.15.114:0/3437627822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 msgr2=0x5650f493d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 0x5650f493d8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe1300057c0 tx=0x7fe130015000 comp rx=0 tx=0).stop 2024-09-21T07:00:22.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 -- 172.21.15.114:0/3437627822 shutdown_connections 2024-09-21T07:00:22.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f4599850 0x5650f494df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f493e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 --2- 172.21.15.114:0/3437627822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4570990 0x5650f493d8c0 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe1300057c0 tx=0x7fe130015000 comp rx=0 tx=0).stop 2024-09-21T07:00:22.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.791+0000 7fe14b201740 1 -- 172.21.15.114:0/3437627822 >> 172.21.15.114:0/3437627822 conn(0x5650f458dc40 msgr2=0x5650f494d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.792+0000 7fe14b201740 1 -- 172.21.15.114:0/3437627822 shutdown_connections 2024-09-21T07:00:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.792+0000 7fe14b201740 1 -- 172.21.15.114:0/3437627822 wait complete. 2024-09-21T07:00:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.793+0000 7fe14b201740 1 -- start start 2024-09-21T07:00:22.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.793+0000 7fe14b201740 1 Processor -- start 2024-09-21T07:00:22.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.794+0000 7fe14b201740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f4954c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.797+0000 7fe14b201740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.797+0000 7fe141200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f4954c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.797+0000 7fe14b201740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f452d580 0x5650f4952bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.797+0000 7fe14b201740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe120011410 con 0x5650f4578b70 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.797+0000 7fe14b201740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe120010cb0 con 0x5650f452d580 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.797+0000 7fe14b201740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe12000def0 con 0x5650f4599850 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.798+0000 7fe14b201740 10 client.0 _ll_register_callbacks cb 0x7fe12c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.798+0000 7fe141200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f4954c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45248/0 (socket says 172.21.15.114:45248) 2024-09-21T07:00:22.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.798+0000 7fe141200640 1 -- 172.21.15.114:0/4260774535 learned_addr learned my addr 172.21.15.114:0/4260774535 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:22.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.798+0000 7fe141c00640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.799+0000 7fe141200640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f452d580 0x5650f4952bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.799+0000 7fe141c00640 1 -- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f452d580 msgr2=0x5650f4952bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.799+0000 7fe141c00640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f452d580 0x5650f4952bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.799+0000 7fe141c00640 1 -- 172.21.15.114:0/4260774535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 msgr2=0x5650f4954c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.800+0000 7fe141c00640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f4954c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.800+0000 7fe141c00640 1 -- 172.21.15.114:0/4260774535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f4953330 con 0x5650f4599850 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.800+0000 7fe141200640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f452d580 0x5650f4952bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:22.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.800+0000 7fe11f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.801+0000 7fe141c00640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49526b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe12c003e40 tx=0x7fe12c0095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ff6d802ee23a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.801+0000 7fe11f400640 1 -- 172.21.15.114:0/4260774535 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c03f070 con 0x5650f4599850 2024-09-21T07:00:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.801+0000 7fe11f400640 1 -- 172.21.15.114:0/4260774535 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fe12c005450 con 0x5650f4599850 2024-09-21T07:00:22.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.802+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe130016040 con 0x5650f4599850 2024-09-21T07:00:22.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.802+0000 7fe11f400640 1 -- 172.21.15.114:0/4260774535 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe12c035030 con 0x5650f4599850 2024-09-21T07:00:22.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.802+0000 7fe14b201740 10 client.27184 fetch_fsmap learned FSMap version 40 2024-09-21T07:00:22.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.802+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5650f49ca040 con 0x5650f4599850 2024-09-21T07:00:22.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe11f400640 1 -- 172.21.15.114:0/4260774535 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe130016040 con 0x5650f4599850 2024-09-21T07:00:22.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe11f400640 1 -- 172.21.15.114:0/4260774535 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 40) ==== 8398+0+0 (secure 0 0 0) 0x7fe12c031c20 con 0x5650f4599850 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe14b201740 10 client.27184 fetch_fsmap finished waiting for FSMap version 40 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe14b201740 10 client.27184 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe14b201740 20 client.27184 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe14b201740 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f49ca840 0x5650f49ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe14b201740 4 client.27184 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}] 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.25, "await_for": 45.0}) -- 0x5650f48b4490 con 0x5650f49ca840 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.806+0000 7fe141200640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f49ca840 0x5650f49ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.807+0000 7fe141200640 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f49ca840 0x5650f49ccc60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c6d5bab2d8d526 server_cookie=f2a3fe158e703dbd in_seq=0 out_seq=0 2024-09-21T07:00:22.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.807+0000 7fe11f400640 10 client.27184 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 345.0 s, 688.2 tps, lat 143.241 ms stddev 183.788 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.985+0000 7fe11f400640 1 -- 172.21.15.114:0/4260774535 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650f48b4490 con 0x5650f49ca840 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.985+0000 7fe11f400640 10 client.27184 handle_command_reply: tid=0 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 client.27184 shutdown 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 msgr2=0x5650f49526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49526b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe12c003e40 tx=0x7fe12c0095b0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 shutdown_connections 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650f452d580 0x5650f4952bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650f49ca840 0x5650f49ccc60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650f4599850 0x5650f49526b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 --2- 172.21.15.114:0/4260774535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650f4578b70 0x5650f4954c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe11f400640 0 client.27184 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 >> 172.21.15.114:0/4260774535 conn(0x5650f458dc40 msgr2=0x5650f49423a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 shutdown_connections 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 1 -- 172.21.15.114:0/4260774535 wait complete. 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.986+0000 7fe14b201740 20 client.27184 trim_cache size 0 max 16384 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 40, 2024-09-21T07:00:22.990 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 26, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "6e774d75": { 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 26, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 139.2, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:22.991 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1240b6690 msgr2=0x7fe1240b8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1240b6690 0x7fe1240b8ba0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe1340ad000 tx=0x7fe13401dfe0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 msgr2=0x7fe13c110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c110700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe134009fd0 tx=0x7fe13400b260 comp rx=0 tx=0).stop 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 shutdown_connections 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe13c10c030 0x7fe13c110700 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe1240b6690 0x7fe1240b8ba0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe13c10b6e0 0x7fe13c1101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 --2- 172.21.15.114:0/4097965870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe13c105da0 0x7fe13c10fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.987+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 >> 172.21.15.114:0/4097965870 conn(0x7fe13c101790 msgr2=0x7fe13c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.988+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 shutdown_connections 2024-09-21T07:00:22.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:22.988+0000 7fe144600640 1 -- 172.21.15.114:0/4097965870 wait complete. 2024-09-21T07:00:23.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:23 smithi114 ceph-mon[30654]: pgmap v583: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-09-21T07:00:23.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:23 smithi183 ceph-mon[34219]: pgmap v583: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-09-21T07:00:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:23 smithi193 ceph-mon[34601]: pgmap v583: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-09-21T07:00:24.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6e774d75', quiesce took 6.3 seconds. Will release after: 12.95 2024-09-21T07:00:25.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:25 smithi114 ceph-mon[30654]: pgmap v584: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 659 op/s 2024-09-21T07:00:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:25 smithi183 ceph-mon[34219]: pgmap v584: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 659 op/s 2024-09-21T07:00:25.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:25 smithi193 ceph-mon[34601]: pgmap v584: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.2 MiB/s wr, 659 op/s 2024-09-21T07:00:26.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:00:26.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:00:26.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:00:26.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:00:26.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:00:26.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:00:26.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:00:26.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:00:26.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:00:26.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:00:26.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:00:26.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:00:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:27 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:00:27.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:27 smithi183 ceph-mon[34219]: pgmap v585: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2024-09-21T07:00:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:27 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:00:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:27 smithi193 ceph-mon[34601]: pgmap v585: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2024-09-21T07:00:27.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:27 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:00:27.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:27 smithi114 ceph-mon[30654]: pgmap v585: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 MiB/s wr, 664 op/s 2024-09-21T07:00:27.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 350.0 s, 4.8 tps, lat 321.426 ms stddev 111.427 2024-09-21T07:00:28.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:28 smithi193 ceph-mon[34601]: pgmap v586: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 670 op/s 2024-09-21T07:00:28.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:28 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:28.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:28 smithi114 ceph-mon[30654]: pgmap v586: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 670 op/s 2024-09-21T07:00:28.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:28 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:28 smithi183 ceph-mon[34219]: pgmap v586: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.9 MiB/s wr, 670 op/s 2024-09-21T07:00:28.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:28 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:31.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:30 smithi114 ceph-mon[30654]: pgmap v587: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 543 op/s 2024-09-21T07:00:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:30 smithi183 ceph-mon[34219]: pgmap v587: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 543 op/s 2024-09-21T07:00:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:30 smithi193 ceph-mon[34601]: pgmap v587: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 543 op/s 2024-09-21T07:00:32.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 355.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:00:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 -- 172.21.15.114:0/59892749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 msgr2=0x7ff8f4116b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 --2- 172.21.15.114:0/59892749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 0x7ff8f4116b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009be0 tx=0x7ff8ec02a6d0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 -- 172.21.15.114:0/59892749 shutdown_connections 2024-09-21T07:00:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 --2- 172.21.15.114:0/59892749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 0x7ff8f4116b90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 --2- 172.21.15.114:0/59892749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 --2- 172.21.15.114:0/59892749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8f410a0e0 0x7ff8f410a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 -- 172.21.15.114:0/59892749 >> 172.21.15.114:0/59892749 conn(0x7ff8f4079630 msgr2=0x7ff8f41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 -- 172.21.15.114:0/59892749 shutdown_connections 2024-09-21T07:00:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.948+0000 7ff8faa00640 1 -- 172.21.15.114:0/59892749 wait complete. 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.949+0000 7ff8faa00640 1 Processor -- start 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 -- start start 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8f410a0e0 0x7ff8f41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f41a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 0x7ff8f41a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4119fd0 con 0x7ff8f410fa10 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4119e50 con 0x7ff8f4110360 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8faa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f411a150 con 0x7ff8f410a0e0 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f41a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f41a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60798/0 (socket says 172.21.15.114:60798) 2024-09-21T07:00:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8f3400640 1 -- 172.21.15.114:0/2420782397 learned_addr learned my addr 172.21.15.114:0/2420782397 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8f410a0e0 0x7ff8f41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.950+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 0x7ff8f41a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f3400640 1 -- 172.21.15.114:0/2420782397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8f410a0e0 msgr2=0x7ff8f41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f3400640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8f410a0e0 0x7ff8f41a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f3400640 1 -- 172.21.15.114:0/2420782397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 msgr2=0x7ff8f41a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f3400640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 0x7ff8f41a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f3400640 1 -- 172.21.15.114:0/2420782397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41a6960 con 0x7ff8f410fa10 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f3400640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f41a0fd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9a0 tx=0x7ff8e000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f0c00640 1 -- 172.21.15.114:0/2420782397 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cd80 con 0x7ff8f410fa10 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f0c00640 1 -- 172.21.15.114:0/2420782397 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e00035f0 con 0x7ff8f410fa10 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8f0c00640 1 -- 172.21.15.114:0/2420782397 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0010690 con 0x7ff8f410fa10 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41a6c50 con 0x7ff8f410fa10 2024-09-21T07:00:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.951+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41ad3e0 con 0x7ff8f410fa10 2024-09-21T07:00:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 Processor -- start 2024-09-21T07:00:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 -- start start 2024-09-21T07:00:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 0x55647678ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647678f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564767046b0 con 0x5564763c94d0 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5564766f3ae0 con 0x5564763c12f0 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.955+0000 7ff901bd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564766f78f0 con 0x5564763ea1b0 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 0x55647678ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 0x55647678ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35010/0 (socket says 172.21.15.114:35010) 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3809367343 learned_addr learned my addr 172.21.15.114:0/3809367343 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f0c00640 1 -- 172.21.15.114:0/2420782397 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff8e0004000 con 0x7ff8f410fa10 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3809367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 msgr2=0x55647678f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3809367343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 msgr2=0x55647678f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647678f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3809367343 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564766c0780 con 0x5564763ea1b0 2024-09-21T07:00:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.956+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 0x55647678ec30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec033000 tx=0x7ff8ec030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef49b47b62ad25c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.957+0000 7ff8e9200640 1 -- 172.21.15.114:0/3809367343 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec041070 con 0x5564763ea1b0 2024-09-21T07:00:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.957+0000 7ff8e9200640 1 -- 172.21.15.114:0/3809367343 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec003e20 con 0x5564763ea1b0 2024-09-21T07:00:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.957+0000 7ff8e9200640 1 -- 172.21.15.114:0/3809367343 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec036040 con 0x5564763ea1b0 2024-09-21T07:00:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.957+0000 7ff8f0c00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8d80b6700 0x7ff8d80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.957+0000 7ff901bd7740 1 -- 172.21.15.114:0/3809367343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 msgr2=0x55647678ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.957+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 0x55647678ec30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec033000 tx=0x7ff8ec030430 comp rx=0 tx=0).stop 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff8f0c00640 1 -- 172.21.15.114:0/2420782397 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8e00e7c30 con 0x7ff8f410fa10 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 -- 172.21.15.114:0/3809367343 shutdown_connections 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647678f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3809367343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763ea1b0 0x55647678ec30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 -- 172.21.15.114:0/3809367343 >> 172.21.15.114:0/3809367343 conn(0x5564763de300 msgr2=0x55647678cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8d80b6700 0x7ff8d80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 -- 172.21.15.114:0/3809367343 shutdown_connections 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 -- 172.21.15.114:0/3809367343 wait complete. 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8d80b6700 0x7ff8d80b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55647678d8b0 tx=0x7ff8e4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.958+0000 7ff901bd7740 1 -- start start 2024-09-21T07:00:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 Processor -- start 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 0x55647679a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x55647679a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x55647679ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55647662bab0 con 0x5564763c12f0 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55647669a570 con 0x5564763ea1b0 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55647679d350 con 0x5564763c94d0 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff901bd7740 10 client.0 _ll_register_callbacks cb 0x5564766a9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff8f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 0x55647679a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff8f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 0x55647679a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60816/0 (socket says 172.21.15.114:60816) 2024-09-21T07:00:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.959+0000 7ff8f3e00640 1 -- 172.21.15.114:0/547810427 learned_addr learned my addr 172.21.15.114:0/547810427 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x55647679ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3400640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x55647679a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3e00640 1 -- 172.21.15.114:0/547810427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 msgr2=0x55647679a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x55647679a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3e00640 1 -- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 msgr2=0x55647679ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x55647679ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3e00640 1 -- 172.21.15.114:0/547810427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556476455790 con 0x5564763c12f0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 0x55647679a250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8e401ea00 tx=0x7ff8e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566193bae6cec442 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8dd600640 1 -- 172.21.15.114:0/547810427 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e402e060 con 0x5564763c12f0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff8dd600640 1 -- 172.21.15.114:0/547810427 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8e4018030 con 0x5564763c12f0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.960+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ec003e20 con 0x5564763c12f0 2024-09-21T07:00:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff8dd600640 1 -- 172.21.15.114:0/547810427 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff8e4035050 con 0x5564763c12f0 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 10 client.16926 fetch_fsmap learned FSMap version 41 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5564764ce620 con 0x5564763c12f0 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff8dd600640 1 -- 172.21.15.114:0/547810427 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec003e20 con 0x5564763c12f0 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff8dd600640 1 -- 172.21.15.114:0/547810427 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 41) ==== 8394+0+0 (secure 0 0 0) 0x7ff8e401c4c0 con 0x5564763c12f0 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 10 client.16926 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 20 client.16926 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564763a8d80 0x5564767ac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 4 client.16926 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.961+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556476706600 con 0x5564763a8d80 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.962+0000 7ff8f3400640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564763a8d80 0x5564767ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.962+0000 7ff8dd600640 10 client.16926 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.962+0000 7ff8f3400640 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564763a8d80 0x5564767ac480 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b2545066b4aa66 server_cookie=5860b59308e016be in_seq=0 out_seq=0 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.963+0000 7ff8dd600640 1 -- 172.21.15.114:0/547810427 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556476706600 con 0x5564763a8d80 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.963+0000 7ff8dd600640 10 client.16926 handle_command_reply: tid=0 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.963+0000 7ff901bd7740 1 client.16926 shutdown 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 msgr2=0x55647679a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 0x55647679a250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8e401ea00 tx=0x7ff8e4004730 comp rx=0 tx=0).stop 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 shutdown_connections 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x55647679ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564763a8d80 0x5564767ac480 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x55647679a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 --2- 172.21.15.114:0/547810427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c12f0 0x55647679a250 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 >> 172.21.15.114:0/547810427 conn(0x5564763de300 msgr2=0x55647678fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff8dd600640 0 client.16926 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 shutdown_connections 2024-09-21T07:00:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 1 -- 172.21.15.114:0/547810427 wait complete. 2024-09-21T07:00:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.964+0000 7ff901bd7740 20 client.16926 trim_cache size 0 max 16384 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 Processor -- start 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 -- start start 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x556476790370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 0x5564767b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b7460 con 0x5564763ea1b0 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b90c0 con 0x5564763c12f0 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff901bd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b7b30 con 0x5564763c94d0 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 0x5564767b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 0x5564767b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60822/0 (socket says 172.21.15.114:60822) 2024-09-21T07:00:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3091363726 learned_addr learned my addr 172.21.15.114:0/3091363726 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f3400640 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x556476790370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3091363726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 msgr2=0x556476790370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x556476790370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3091363726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 msgr2=0x55647678ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f8a00640 1 -- 172.21.15.114:0/3091363726 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556476795d70 con 0x5564763ea1b0 2024-09-21T07:00:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.980+0000 7ff8f3400640 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x556476790370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.981+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 0x5564767b1c40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec034040 tx=0x7ff8ec030f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76d104a54e0a4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.981+0000 7ff8dfe00640 1 -- 172.21.15.114:0/3091363726 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec054070 con 0x5564763ea1b0 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.981+0000 7ff8dfe00640 1 -- 172.21.15.114:0/3091363726 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec003970 con 0x5564763ea1b0 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.981+0000 7ff8dfe00640 1 -- 172.21.15.114:0/3091363726 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec036040 con 0x5564763ea1b0 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 -- 172.21.15.114:0/3091363726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 msgr2=0x5564767b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 0x5564767b1c40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec034040 tx=0x7ff8ec030f40 comp rx=0 tx=0).stop 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 -- 172.21.15.114:0/3091363726 shutdown_connections 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763ea1b0 0x5564767b1c40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c94d0 0x556476790370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 --2- 172.21.15.114:0/3091363726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763c12f0 0x55647678ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 -- 172.21.15.114:0/3091363726 >> 172.21.15.114:0/3091363726 conn(0x5564763de300 msgr2=0x55647678fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 -- 172.21.15.114:0/3091363726 shutdown_connections 2024-09-21T07:00:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 -- 172.21.15.114:0/3091363726 wait complete. 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 -- start start 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.982+0000 7ff901bd7740 1 Processor -- start 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c12f0 0x5564767a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647679ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x5564767a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b8930 con 0x5564763c94d0 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b8200 con 0x5564763ea1b0 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b5440 con 0x5564763c12f0 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff901bd7740 10 client.0 _ll_register_callbacks cb 0x7ff8e4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647679ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647679ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60834/0 (socket says 172.21.15.114:60834) 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 -- 172.21.15.114:0/2445993647 learned_addr learned my addr 172.21.15.114:0/2445993647 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c12f0 0x5564767a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f8a00640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x5564767a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 -- 172.21.15.114:0/2445993647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c12f0 msgr2=0x5564767a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c12f0 0x5564767a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 -- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 msgr2=0x5564767a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x5564767a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3400640 1 -- 172.21.15.114:0/2445993647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564767a0a30 con 0x5564763c94d0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.983+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c12f0 0x5564767a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff8f3400640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647679ffb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c6a0 tx=0x7ff8e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc22176d13db4072 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff8dd600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff8dd600640 1 -- 172.21.15.114:0/2445993647 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e00ace80 con 0x5564763c94d0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff8dd600640 1 -- 172.21.15.114:0/2445993647 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7ff8e00e5290 con 0x5564763c94d0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ec003970 con 0x5564763c94d0 2024-09-21T07:00:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff8dd600640 1 -- 172.21.15.114:0/2445993647 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff8e00ad220 con 0x5564763c94d0 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff901bd7740 10 client.16932 fetch_fsmap learned FSMap version 41 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556476816b30 con 0x5564763c94d0 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.984+0000 7ff8dd600640 1 -- 172.21.15.114:0/2445993647 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec003970 con 0x5564763c94d0 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff8dd600640 1 -- 172.21.15.114:0/2445993647 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 41) ==== 8394+0+0 (secure 0 0 0) 0x7ff8e00a9980 con 0x5564763c94d0 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff901bd7740 10 client.16932 fetch_fsmap finished waiting for FSMap version 41 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff901bd7740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff901bd7740 20 client.16932 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff901bd7740 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55647637e1c0 0x556476819330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff901bd7740 4 client.16932 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564767020b0 con 0x55647637e1c0 2024-09-21T07:00:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55647637e1c0 0x556476819330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff8f3e00640 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55647637e1c0 0x556476819330 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abe09dc39fb6ecb server_cookie=14446117d973af61 in_seq=0 out_seq=0 2024-09-21T07:00:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.985+0000 7ff8dd600640 10 client.16932 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff8dd600640 1 -- 172.21.15.114:0/2445993647 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564767020b0 con 0x55647637e1c0 2024-09-21T07:00:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff8dd600640 10 client.16932 handle_command_reply: tid=0 2024-09-21T07:00:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 client.16932 shutdown 2024-09-21T07:00:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 msgr2=0x55647679ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647679ffb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000c6a0 tx=0x7ff8e0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 shutdown_connections 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564763ea1b0 0x5564767a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564763c94d0 0x55647679ffb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55647637e1c0 0x556476819330 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 --2- 172.21.15.114:0/2445993647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564763c12f0 0x5564767a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff8dd600640 0 client.16932 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 >> 172.21.15.114:0/2445993647 conn(0x5564763de300 msgr2=0x55647678cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 shutdown_connections 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.994+0000 7ff901bd7740 1 -- 172.21.15.114:0/2445993647 wait complete. 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff901bd7740 20 client.16932 trim_cache size 0 max 16384 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:00:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:00:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:00:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:00:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8d80b6700 msgr2=0x7ff8d80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8d80b6700 0x7ff8d80b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55647678d8b0 tx=0x7ff8e4005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 msgr2=0x7ff8f41a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f41a0fd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9a0 tx=0x7ff8e000ee60 comp rx=0 tx=0).stop 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 shutdown_connections 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8d80b6700 0x7ff8d80b8c10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8f4110360 0x7ff8f41a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8f410fa10 0x7ff8f41a0fd0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 --2- 172.21.15.114:0/2420782397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8f410a0e0 0x7ff8f41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 >> 172.21.15.114:0/2420782397 conn(0x7ff8f4079630 msgr2=0x7ff8f410ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 shutdown_connections 2024-09-21T07:00:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:32.995+0000 7ff8faa00640 1 -- 172.21.15.114:0/2420782397 wait complete. 2024-09-21T07:00:33.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:32 smithi114 ceph-mon[30654]: pgmap v588: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 420 op/s 2024-09-21T07:00:33.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:32 smithi183 ceph-mon[34219]: pgmap v588: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 420 op/s 2024-09-21T07:00:33.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:32 smithi193 ceph-mon[34601]: pgmap v588: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.6 MiB/s wr, 420 op/s 2024-09-21T07:00:35.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:34 smithi114 ceph-mon[30654]: pgmap v589: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 304 op/s 2024-09-21T07:00:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:34 smithi183 ceph-mon[34219]: pgmap v589: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 304 op/s 2024-09-21T07:00:35.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:34 smithi193 ceph-mon[34601]: pgmap v589: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 304 op/s 2024-09-21T07:00:36.466 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:00:36.840 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:37.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:36 smithi183 ceph-mon[34219]: pgmap v590: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 68 op/s 2024-09-21T07:00:37.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:36 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:37.122 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:36 smithi114 ceph-mon[30654]: pgmap v590: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 68 op/s 2024-09-21T07:00:37.123 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:36 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:36 smithi193 ceph-mon[34601]: pgmap v590: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.3 MiB/s wr, 68 op/s 2024-09-21T07:00:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:36 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:37.193 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:00:37.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 -- 172.21.15.114:0/4049860205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10ad10 msgr2=0x7f99cc116fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 --2- 172.21.15.114:0/4049860205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10ad10 0x7f99cc116fc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009ca0 tx=0x7f99c402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 -- 172.21.15.114:0/4049860205 shutdown_connections 2024-09-21T07:00:37.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 --2- 172.21.15.114:0/4049860205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10ad10 0x7f99cc116fc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 --2- 172.21.15.114:0/4049860205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10a3c0 0x7f99cc10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 --2- 172.21.15.114:0/4049860205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99cc10f150 0x7f99cc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 -- 172.21.15.114:0/4049860205 >> 172.21.15.114:0/4049860205 conn(0x7f99cc0712f0 msgr2=0x7f99cc073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:37.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 -- 172.21.15.114:0/4049860205 shutdown_connections 2024-09-21T07:00:37.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.462+0000 7f99d3400640 1 -- 172.21.15.114:0/4049860205 wait complete. 2024-09-21T07:00:37.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.464+0000 7f99d3400640 1 Processor -- start 2024-09-21T07:00:37.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.464+0000 7f99d3400640 1 -- start start 2024-09-21T07:00:37.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.464+0000 7f99d3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99cc10a3c0 0x7f99cc1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d3400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 0x7f99cc1a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10f150 0x7f99cc1a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d3400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc11a110 con 0x7f99cc10ad10 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d3400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc119f90 con 0x7f99cc10f150 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d3400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99cc11a290 con 0x7f99cc10a3c0 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 0x7f99cc1a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 0x7f99cc1a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43450/0 (socket says 172.21.15.114:43450) 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 -- 172.21.15.114:0/1668707226 learned_addr learned my addr 172.21.15.114:0/1668707226 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10f150 0x7f99cc1a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99cc10a3c0 0x7f99cc1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 -- 172.21.15.114:0/1668707226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99cc10a3c0 msgr2=0x7f99cc1a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99cc10a3c0 0x7f99cc1a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 -- 172.21.15.114:0/1668707226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10f150 msgr2=0x7f99cc1a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10f150 0x7f99cc1a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99c3e00640 1 -- 172.21.15.114:0/1668707226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99cc1a69d0 con 0x7f99cc10ad10 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.465+0000 7f99d1400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10f150 0x7f99cc1a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.466+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 0x7f99cc1a1000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f99b400d760 tx=0x7f99b400dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.466+0000 7f99ca000640 1 -- 172.21.15.114:0/1668707226 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4008d70 con 0x7f99cc10ad10 2024-09-21T07:00:37.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.466+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99cc1a6c60 con 0x7f99cc10ad10 2024-09-21T07:00:37.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.466+0000 7f99ca000640 1 -- 172.21.15.114:0/1668707226 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b400e710 con 0x7f99cc10ad10 2024-09-21T07:00:37.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.466+0000 7f99ca000640 1 -- 172.21.15.114:0/1668707226 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4003850 con 0x7f99cc10ad10 2024-09-21T07:00:37.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.466+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99cc1ad420 con 0x7f99cc10ad10 2024-09-21T07:00:37.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.470+0000 7f99d9ff9740 1 Processor -- start 2024-09-21T07:00:37.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 -- start start 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da711070 0x5650daab7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6e81b0 0x5650daab82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daab87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650daa2ef70 con 0x5650da6f0390 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650daa1e3a0 con 0x5650da6e81b0 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650daa221b0 con 0x5650da711070 2024-09-21T07:00:37.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.471+0000 7f99ca000640 1 -- 172.21.15.114:0/1668707226 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f99b401a020 con 0x7f99cc10ad10 2024-09-21T07:00:37.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da711070 0x5650daab7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6e81b0 0x5650daab82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d1400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da711070 0x5650daab7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54888/0 (socket says 172.21.15.114:54888) 2024-09-21T07:00:37.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d1400640 1 -- 172.21.15.114:0/3960899056 learned_addr learned my addr 172.21.15.114:0/3960899056 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:37.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99c3e00640 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daab87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d0a00640 1 -- 172.21.15.114:0/3960899056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da711070 msgr2=0x5650daab7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d0a00640 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da711070 0x5650daab7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d0a00640 1 -- 172.21.15.114:0/3960899056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 msgr2=0x5650daab87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.472+0000 7f99d0a00640 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daab87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99d0a00640 1 -- 172.21.15.114:0/3960899056 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650da9e8c90 con 0x5650da6e81b0 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99c3e00640 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daab87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99d0a00640 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6e81b0 0x5650daab82a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009b80 tx=0x7f99bc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470f9fc68948125e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:37.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99c1c00640 1 -- 172.21.15.114:0/3960899056 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc01b020 con 0x5650da6e81b0 2024-09-21T07:00:37.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99c1c00640 1 -- 172.21.15.114:0/3960899056 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99bc005020 con 0x5650da6e81b0 2024-09-21T07:00:37.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99c1c00640 1 -- 172.21.15.114:0/3960899056 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc004a00 con 0x5650da6e81b0 2024-09-21T07:00:37.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.473+0000 7f99ca000640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99b00b6680 0x7f99b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.474+0000 7f99ca000640 1 -- 172.21.15.114:0/1668707226 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f99b40e6d20 con 0x7f99cc10ad10 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.474+0000 7f99d1400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99b00b6680 0x7f99b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 -- 172.21.15.114:0/3960899056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6e81b0 msgr2=0x5650daab82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6e81b0 0x5650daab82a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009b80 tx=0x7f99bc005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d1400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99b00b6680 0x7f99b00b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99c402aeb0 tx=0x7f99c4005d80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 -- 172.21.15.114:0/3960899056 shutdown_connections 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daab87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6e81b0 0x5650daab82a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/3960899056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da711070 0x5650daab7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 -- 172.21.15.114:0/3960899056 >> 172.21.15.114:0/3960899056 conn(0x5650da705460 msgr2=0x5650daab5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 -- 172.21.15.114:0/3960899056 shutdown_connections 2024-09-21T07:00:37.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.475+0000 7f99d9ff9740 1 -- 172.21.15.114:0/3960899056 wait complete. 2024-09-21T07:00:37.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 -- start start 2024-09-21T07:00:37.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 Processor -- start 2024-09-21T07:00:37.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daac3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 0x5650daac3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da711070 0x5650daad3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5650da87f700 con 0x5650da711070 2024-09-21T07:00:37.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5650da9b9540 con 0x5650da6f0390 2024-09-21T07:00:37.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.476+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5650daaca950 con 0x5650da6e81b0 2024-09-21T07:00:37.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99d9ff9740 10 client.0 _ll_register_callbacks cb 0x5650da9c9430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 0x5650daac3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daac3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 0x5650daac3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54782/0 (socket says 172.21.15.114:54782) 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 -- 172.21.15.114:0/1875073926 learned_addr learned my addr 172.21.15.114:0/1875073926 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99d1400640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da711070 0x5650daad3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 -- 172.21.15.114:0/1875073926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 msgr2=0x5650daac3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daac3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 -- 172.21.15.114:0/1875073926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da711070 msgr2=0x5650daad3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da711070 0x5650daad3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99c3e00640 1 -- 172.21.15.114:0/1875073926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650da77c650 con 0x5650da6f0390 2024-09-21T07:00:37.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.477+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daac3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:37.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 0x5650daac3a80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99b40a8d70 tx=0x7f99b40abdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173c16a8f6f76156 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:37.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99ba000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:37.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99ba000640 1 -- 172.21.15.114:0/1875073926 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b40a8640 con 0x5650da6f0390 2024-09-21T07:00:37.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650da9bbb50 con 0x5650da6f0390 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99ba000640 1 -- 172.21.15.114:0/1875073926 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f99b40b2410 con 0x5650da6f0390 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99ba000640 1 -- 172.21.15.114:0/1875073926 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f99b40abf10 con 0x5650da6f0390 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99d9ff9740 10 client.27266 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.478+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5650da9d6910 con 0x5650da6f0390 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99ba000640 1 -- 172.21.15.114:0/1875073926 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5650da9bbb50 con 0x5650da6f0390 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99ba000640 1 -- 172.21.15.114:0/1875073926 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f99b40b2610 con 0x5650da6f0390 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99d9ff9740 10 client.27266 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99d9ff9740 10 client.27266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99d9ff9740 20 client.27266 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6cf9f0 0x5650daad9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99d9ff9740 4 client.27266 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.479+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650daa30ec0 con 0x5650da6cf9f0 2024-09-21T07:00:37.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.480+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6cf9f0 0x5650daad9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.481+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6cf9f0 0x5650daad9b20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599e57146e560bf3 server_cookie=706a1b76cabee67 in_seq=0 out_seq=0 2024-09-21T07:00:37.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.481+0000 7f99ba000640 10 client.27266 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99ba000640 1 -- 172.21.15.114:0/1875073926 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650daa30ec0 con 0x5650da6cf9f0 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99ba000640 10 client.27266 handle_command_reply: tid=0 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99d9ff9740 1 client.27266 shutdown 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 msgr2=0x5650daac3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 0x5650daac3a80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f99b40a8d70 tx=0x7f99b40abdd0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 shutdown_connections 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da711070 0x5650daad3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.482+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da6f0390 0x5650daac3a80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6cf9f0 0x5650daad9b20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1875073926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daac3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 >> 172.21.15.114:0/1875073926 conn(0x5650da705460 msgr2=0x5650daabaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99ba000640 0 client.27266 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:37.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 shutdown_connections 2024-09-21T07:00:37.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1875073926 wait complete. 2024-09-21T07:00:37.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.483+0000 7f99d9ff9740 20 client.27266 trim_cache size 0 max 16384 2024-09-21T07:00:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.504+0000 7f99d9ff9740 1 Processor -- start 2024-09-21T07:00:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.504+0000 7f99d9ff9740 1 -- start start 2024-09-21T07:00:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.504+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 0x5650daac5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.504+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6f0390 0x5650daad3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daac0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99a800fe70 con 0x5650da6e81b0 2024-09-21T07:00:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8011a40 con 0x5650da711070 2024-09-21T07:00:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8010540 con 0x5650da6f0390 2024-09-21T07:00:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 0x5650daac5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 0x5650daac5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43476/0 (socket says 172.21.15.114:43476) 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 -- 172.21.15.114:0/1563196144 learned_addr learned my addr 172.21.15.114:0/1563196144 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d1400640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daac0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6f0390 0x5650daad3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 -- 172.21.15.114:0/1563196144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6f0390 msgr2=0x5650daad3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6f0390 0x5650daad3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 -- 172.21.15.114:0/1563196144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 msgr2=0x5650daac0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daac0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.505+0000 7f99d0a00640 1 -- 172.21.15.114:0/1563196144 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650daac4800 con 0x5650da6e81b0 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.506+0000 7f99d1400640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daac0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.506+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6f0390 0x5650daad3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.506+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 0x5650daac5170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009b20 tx=0x7f99bc009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86ed0fc24fcbaed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.506+0000 7f99c1200640 1 -- 172.21.15.114:0/1563196144 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc01b070 con 0x5650da6e81b0 2024-09-21T07:00:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.506+0000 7f99c1200640 1 -- 172.21.15.114:0/1563196144 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99bc0054d0 con 0x5650da6e81b0 2024-09-21T07:00:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.506+0000 7f99c1200640 1 -- 172.21.15.114:0/1563196144 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc004c20 con 0x5650da6e81b0 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1563196144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 msgr2=0x5650daac5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 0x5650daac5170 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009b20 tx=0x7f99bc009660 comp rx=0 tx=0).stop 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1563196144 shutdown_connections 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daac0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6f0390 0x5650daad3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1563196144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6e81b0 0x5650daac5170 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1563196144 >> 172.21.15.114:0/1563196144 conn(0x5650da705460 msgr2=0x5650daac4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1563196144 shutdown_connections 2024-09-21T07:00:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.508+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1563196144 wait complete. 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 -- start start 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 Processor -- start 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daad00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daacc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daaccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8011340 con 0x5650da6f0390 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8010c10 con 0x5650da711070 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99a800de50 con 0x5650da6e81b0 2024-09-21T07:00:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.509+0000 7f99d9ff9740 10 client.0 _ll_register_callbacks cb 0x7f99b40e4550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d0a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daad00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daaccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daaccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54796/0 (socket says 172.21.15.114:54796) 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 -- 172.21.15.114:0/1041745049 learned_addr learned my addr 172.21.15.114:0/1041745049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99c3e00640 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daacc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 -- 172.21.15.114:0/1041745049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 msgr2=0x5650daad00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daad00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 -- 172.21.15.114:0/1041745049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 msgr2=0x5650daacc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daacc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.510+0000 7f99d1400640 1 -- 172.21.15.114:0/1041745049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650daacd030 con 0x5650da711070 2024-09-21T07:00:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99d1400640 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daaccaf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009ca0 tx=0x7f99c402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c05856e6ce28906 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99ba000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:37.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99ba000640 1 -- 172.21.15.114:0/1041745049 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4059060 con 0x5650da711070 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99ba000640 1 -- 172.21.15.114:0/1041745049 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f99c4043030 con 0x5650da711070 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99bc0054d0 con 0x5650da711070 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99ba000640 1 -- 172.21.15.114:0/1041745049 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f99c4004250 con 0x5650da711070 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99d9ff9740 10 client.27272 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.511+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5650dab45120 con 0x5650da711070 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99ba000640 1 -- 172.21.15.114:0/1041745049 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99bc0054d0 con 0x5650da711070 2024-09-21T07:00:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99ba000640 1 -- 172.21.15.114:0/1041745049 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f99c402fbe0 con 0x5650da711070 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99d9ff9740 10 client.27272 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99d9ff9740 10 client.27272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99d9ff9740 20 client.27272 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6a4da0 0x5650dab47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99d9ff9740 4 client.27272 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.512+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650daa2c970 con 0x5650da6a4da0 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.513+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6a4da0 0x5650dab47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.513+0000 7f99ba000640 10 client.27272 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:37.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.513+0000 7f99d0a00640 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6a4da0 0x5650dab47940 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2771c4169e72838 server_cookie=491dc3375d812624 in_seq=0 out_seq=0 2024-09-21T07:00:37.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.513+0000 7f99ba000640 1 -- 172.21.15.114:0/1041745049 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x5650daa2c970 con 0x5650da6a4da0 2024-09-21T07:00:37.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.513+0000 7f99ba000640 10 client.27272 handle_command_reply: tid=0 2024-09-21T07:00:37.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 client.27272 shutdown 2024-09-21T07:00:37.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 msgr2=0x5650daaccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daaccaf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009ca0 tx=0x7f99c402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 shutdown_connections 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5650da711070 0x5650daaccaf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5650da6f0390 0x5650daacc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5650da6a4da0 0x5650dab47940 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 --2- 172.21.15.114:0/1041745049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650da6e81b0 0x5650daad00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 >> 172.21.15.114:0/1041745049 conn(0x5650da705460 msgr2=0x5650daac49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99ba000640 0 client.27272 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 shutdown_connections 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.514+0000 7f99d9ff9740 1 -- 172.21.15.114:0/1041745049 wait complete. 2024-09-21T07:00:37.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d9ff9740 20 client.27272 trim_cache size 0 max 16384 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.518 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:00:37.519 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:37.520 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.521 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:37.522 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99b00b6680 msgr2=0x7f99b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99b00b6680 0x7f99b00b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f99c402aeb0 tx=0x7f99c4005d80 comp rx=0 tx=0).stop 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 msgr2=0x7f99cc1a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 0x7f99cc1a1000 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f99b400d760 tx=0x7f99b400dc20 comp rx=0 tx=0).stop 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 shutdown_connections 2024-09-21T07:00:37.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99b00b6680 0x7f99b00b8b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99cc10f150 0x7f99cc1a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f99cc10ad10 0x7f99cc1a1000 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.515+0000 7f99d3400640 1 --2- 172.21.15.114:0/1668707226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99cc10a3c0 0x7f99cc1a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.516+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 >> 172.21.15.114:0/1668707226 conn(0x7f99cc0712f0 msgr2=0x7f99cc10c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.516+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 shutdown_connections 2024-09-21T07:00:37.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:37.516+0000 7f99d3400640 1 -- 172.21.15.114:0/1668707226 wait complete. 2024-09-21T07:00:37.788 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:37.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 360.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:00:38.446 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:00:38.446 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:00:39.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:38 smithi114 ceph-mon[30654]: pgmap v591: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 844 KiB/s wr, 87 op/s 2024-09-21T07:00:39.086 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:39.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.088+0000 7f1d6f200640 1 -- 172.21.15.114:0/2906624184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d70102880 msgr2=0x7f1d70102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.088+0000 7f1d6f200640 1 --2- 172.21.15.114:0/2906624184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d70102880 0x7f1d70102c90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009900 tx=0x7f1d6002a410 comp rx=0 tx=0).stop 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 -- 172.21.15.114:0/2906624184 shutdown_connections 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 --2- 172.21.15.114:0/2906624184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70108b10 0x7f1d7010f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 --2- 172.21.15.114:0/2906624184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d701081c0 0x7f1d701085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 --2- 172.21.15.114:0/2906624184 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d70102880 0x7f1d70102c90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 -- 172.21.15.114:0/2906624184 >> 172.21.15.114:0/2906624184 conn(0x7f1d700fe2d0 msgr2=0x7f1d70100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 -- 172.21.15.114:0/2906624184 shutdown_connections 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.089+0000 7f1d6f200640 1 -- 172.21.15.114:0/2906624184 wait complete. 2024-09-21T07:00:39.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 Processor -- start 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 -- start start 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70102880 0x7f1d7019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 0x7f1d7019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d70108b10 0x7f1d7019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70113920 con 0x7f1d701081c0 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d701137a0 con 0x7f1d70108b10 2024-09-21T07:00:39.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.090+0000 7f1d6f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70113aa0 con 0x7f1d70102880 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 0x7f1d7019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70102880 0x7f1d7019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 0x7f1d7019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43508/0 (socket says 172.21.15.114:43508) 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 -- 172.21.15.114:0/1684442324 learned_addr learned my addr 172.21.15.114:0/1684442324 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6e800640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d70108b10 0x7f1d7019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 -- 172.21.15.114:0/1684442324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70102880 msgr2=0x7f1d7019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:39.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70102880 0x7f1d7019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 -- 172.21.15.114:0/1684442324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d70108b10 msgr2=0x7f1d7019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d70108b10 0x7f1d7019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6d400640 1 -- 172.21.15.114:0/1684442324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d701a0260 con 0x7f1d701081c0 2024-09-21T07:00:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6de00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70102880 0x7f1d7019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.091+0000 7f1d6e800640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d70108b10 0x7f1d7019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:39.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.092+0000 7f1d6d400640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 0x7f1d7019a890 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9f0 tx=0x7f1d5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:39.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.092+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00ccc0 con 0x7f1d701081c0 2024-09-21T07:00:39.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.092+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c0108e0 con 0x7f1d701081c0 2024-09-21T07:00:39.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.092+0000 7f1d6f200640 1 -- 172.21.15.114:0/1684442324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d701a0550 con 0x7f1d701081c0 2024-09-21T07:00:39.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.092+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c018860 con 0x7f1d701081c0 2024-09-21T07:00:39.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.092+0000 7f1d6f200640 1 -- 172.21.15.114:0/1684442324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7010fd00 con 0x7f1d701081c0 2024-09-21T07:00:39.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.094+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d70102c90 con 0x7f1d701081c0 2024-09-21T07:00:39.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.096+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1d5c010430 con 0x7f1d701081c0 2024-09-21T07:00:39.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.097+0000 7f1d66a00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d540b6680 0x7f1d540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:39.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.098+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f1d5c0e7ea0 con 0x7f1d701081c0 2024-09-21T07:00:39.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.098+0000 7f1d6de00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d540b6680 0x7f1d540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:39.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.098+0000 7f1d6de00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d540b6680 0x7f1d540b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1d60005ab0 tx=0x7f1d6003c040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:39.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.102+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d5c0b2460 con 0x7f1d701081c0 2024-09-21T07:00:39.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:38 smithi183 ceph-mon[34219]: pgmap v591: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 844 KiB/s wr, 87 op/s 2024-09-21T07:00:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:38 smithi193 ceph-mon[34601]: pgmap v591: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 844 KiB/s wr, 87 op/s 2024-09-21T07:00:39.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.343+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d70111730 con 0x7f1d701081c0 2024-09-21T07:00:39.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.344+0000 7f1d66a00640 1 -- 172.21.15.114:0/1684442324 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11993 (secure 0 0 0) 0x7f1d5c0b73a0 con 0x7f1d701081c0 2024-09-21T07:00:39.346 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:00:39.346 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"btime":"2024-09-21T07:00:35:813607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:00:35.616527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:00:39.347 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d540b6680 msgr2=0x7f1d540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d540b6680 0x7f1d540b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1d60005ab0 tx=0x7f1d6003c040 comp rx=0 tx=0).stop 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 msgr2=0x7f1d7019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 0x7f1d7019a890 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9f0 tx=0x7f1d5c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 shutdown_connections 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d540b6680 0x7f1d540b8b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d70108b10 0x7f1d7019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d701081c0 0x7f1d7019a890 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 --2- 172.21.15.114:0/1684442324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d70102880 0x7f1d7019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:39.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 >> 172.21.15.114:0/1684442324 conn(0x7f1d700fe2d0 msgr2=0x7f1d701006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:39.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 shutdown_connections 2024-09-21T07:00:39.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:39.347+0000 7f1d5be00640 1 -- 172.21.15.114:0/1684442324 wait complete. 2024-09-21T07:00:40.019 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:39 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1684442324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:40.102 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 6e774d75 --release --await' 2024-09-21T07:00:40.102 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 6e774d75 --release --await 2024-09-21T07:00:40.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:39 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1684442324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:40.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:39 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1684442324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:40.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 -- 172.21.15.114:0/1084567683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1092f0 msgr2=0x7fbefc109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:40.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 --2- 172.21.15.114:0/1084567683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1092f0 0x7fbefc109700 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009900 tx=0x7fbee802a410 comp rx=0 tx=0).stop 2024-09-21T07:00:40.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 -- 172.21.15.114:0/1084567683 shutdown_connections 2024-09-21T07:00:40.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 --2- 172.21.15.114:0/1084567683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbefc109c40 0x7fbefc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 --2- 172.21.15.114:0/1084567683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1092f0 0x7fbefc109700 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 --2- 172.21.15.114:0/1084567683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbefc1039b0 0x7fbefc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 -- 172.21.15.114:0/1084567683 >> 172.21.15.114:0/1084567683 conn(0x7fbefc0ff440 msgr2=0x7fbefc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:40.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.357+0000 7fbf01400640 1 -- 172.21.15.114:0/1084567683 shutdown_connections 2024-09-21T07:00:40.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.358+0000 7fbf01400640 1 -- 172.21.15.114:0/1084567683 wait complete. 2024-09-21T07:00:40.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.359+0000 7fbf01400640 1 Processor -- start 2024-09-21T07:00:40.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.359+0000 7fbf01400640 1 -- start start 2024-09-21T07:00:40.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf01400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 0x7fbefc19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:40.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf01400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbefc1092f0 0x7fbefc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf01400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbefc109c40 0x7fbefc1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf01400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc113950 con 0x7fbefc1039b0 2024-09-21T07:00:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf01400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc1137d0 con 0x7fbefc109c40 2024-09-21T07:00:40.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf01400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc113ad0 con 0x7fbefc1092f0 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbefbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 0x7fbefc19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbefbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 0x7fbefc19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43526/0 (socket says 172.21.15.114:43526) 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbefbe00640 1 -- 172.21.15.114:0/2098222853 learned_addr learned my addr 172.21.15.114:0/2098222853 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbefb400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbefc1092f0 0x7fbefc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.360+0000 7fbf00a00640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbefc109c40 0x7fbefc1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbefbe00640 1 -- 172.21.15.114:0/2098222853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbefc1092f0 msgr2=0x7fbefc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbefbe00640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbefc1092f0 0x7fbefc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbefbe00640 1 -- 172.21.15.114:0/2098222853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbefc109c40 msgr2=0x7fbefc1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbefbe00640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbefc109c40 0x7fbefc1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbefbe00640 1 -- 172.21.15.114:0/2098222853 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc1a2490 con 0x7fbefc1039b0 2024-09-21T07:00:40.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbefbe00640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 0x7fbefc19c520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00ea60 tx=0x7fbeec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:40.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00cde0 con 0x7fbefc1039b0 2024-09-21T07:00:40.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeec0035f0 con 0x7fbefc1039b0 2024-09-21T07:00:40.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc1a2780 con 0x7fbefc1039b0 2024-09-21T07:00:40.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.361+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc1a8ee0 con 0x7fbefc1039b0 2024-09-21T07:00:40.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.362+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec010690 con 0x7fbefc1039b0 2024-09-21T07:00:40.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.363+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc103dc0 con 0x7fbefc1039b0 2024-09-21T07:00:40.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.365+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fbeec0037a0 con 0x7fbefc1039b0 2024-09-21T07:00:40.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.368+0000 7fbef8c00640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee00b6680 0x7fbee00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:40.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.368+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fbeec0e81f0 con 0x7fbefc1039b0 2024-09-21T07:00:40.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.368+0000 7fbefb400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee00b6680 0x7fbee00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:40.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.369+0000 7fbefb400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee00b6680 0x7fbee00b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbee802aa00 tx=0x7fbee8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:40.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.371+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbeec0ae370 con 0x7fbefc1039b0 2024-09-21T07:00:40.491 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:40.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.622+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbefc1a91c0 con 0x7fbefc1039b0 2024-09-21T07:00:40.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.623+0000 7fbef8c00640 1 -- 172.21.15.114:0/2098222853 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11993 (secure 0 0 0) 0x7fbeec0c3e20 con 0x7fbefc1039b0 2024-09-21T07:00:40.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:00:40.625 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":42,"btime":"2024-09-21T07:00:35:813607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:00:35.616527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:00:40.625 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 42 2024-09-21T07:00:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee00b6680 msgr2=0x7fbee00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee00b6680 0x7fbee00b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbee802aa00 tx=0x7fbee8005a20 comp rx=0 tx=0).stop 2024-09-21T07:00:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 msgr2=0x7fbefc19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 0x7fbefc19c520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00ea60 tx=0x7fbeec00ef20 comp rx=0 tx=0).stop 2024-09-21T07:00:40.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 shutdown_connections 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbee00b6680 0x7fbee00b8b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbefc109c40 0x7fbefc1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.625+0000 7fbf01400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbefc1092f0 0x7fbefc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.626+0000 7fbf01400640 1 --2- 172.21.15.114:0/2098222853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbefc1039b0 0x7fbefc19c520 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.626+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 >> 172.21.15.114:0/2098222853 conn(0x7fbefc0ff440 msgr2=0x7fbefc108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.626+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 shutdown_connections 2024-09-21T07:00:40.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:40.626+0000 7fbf01400640 1 -- 172.21.15.114:0/2098222853 wait complete. 2024-09-21T07:00:41.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:40 smithi114 ceph-mon[30654]: pgmap v592: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 533 KiB/s wr, 79 op/s 2024-09-21T07:00:41.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:40 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2098222853' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:40 smithi183 ceph-mon[34219]: pgmap v592: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 533 KiB/s wr, 79 op/s 2024-09-21T07:00:41.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:40 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2098222853' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:40 smithi193 ceph-mon[34601]: pgmap v592: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 533 KiB/s wr, 79 op/s 2024-09-21T07:00:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:40 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2098222853' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:00:41.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 -- 172.21.15.114:0/2450338562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc10b500 msgr2=0x7f10fc10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 --2- 172.21.15.114:0/2450338562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc10b500 0x7f10fc10b910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009900 tx=0x7f10ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 -- 172.21.15.114:0/2450338562 shutdown_connections 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 --2- 172.21.15.114:0/2450338562 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 0x7f10fc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 --2- 172.21.15.114:0/2450338562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc10b500 0x7f10fc10b910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 --2- 172.21.15.114:0/2450338562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc105bc0 0x7f10fc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.386+0000 7f1103200640 1 -- 172.21.15.114:0/2450338562 >> 172.21.15.114:0/2450338562 conn(0x7f10fc101670 msgr2=0x7f10fc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.387+0000 7f1103200640 1 -- 172.21.15.114:0/2450338562 shutdown_connections 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.387+0000 7f1103200640 1 -- 172.21.15.114:0/2450338562 wait complete. 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 Processor -- start 2024-09-21T07:00:41.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 -- start start 2024-09-21T07:00:41.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc105bc0 0x7f10fc19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc10b500 0x7f10fc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 0x7f10fc1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc115b60 con 0x7f10fc10be50 2024-09-21T07:00:41.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1159e0 con 0x7f10fc105bc0 2024-09-21T07:00:41.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.388+0000 7f1103200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc115ce0 con 0x7f10fc10b500 2024-09-21T07:00:41.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1102800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 0x7f10fc1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc10b500 0x7f10fc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc105bc0 0x7f10fc19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1102800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 0x7f10fc1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43546/0 (socket says 172.21.15.114:43546) 2024-09-21T07:00:41.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1102800640 1 -- 172.21.15.114:0/3406384078 learned_addr learned my addr 172.21.15.114:0/3406384078 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101400640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc10b500 0x7f10fc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54968/0 (socket says 172.21.15.114:54968) 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101e00640 1 -- 172.21.15.114:0/3406384078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc10b500 msgr2=0x7f10fc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101e00640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc10b500 0x7f10fc19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101e00640 1 -- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 msgr2=0x7f10fc1a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101e00640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 0x7f10fc1a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.389+0000 7f1101e00640 1 -- 172.21.15.114:0/3406384078 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10fc1a2580 con 0x7f10fc105bc0 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.390+0000 7f1101e00640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc105bc0 0x7f10fc19c610 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10f000e9f0 tx=0x7f10f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.390+0000 7f10f6a00640 1 -- 172.21.15.114:0/3406384078 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f000ccc0 con 0x7f10fc105bc0 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.390+0000 7f10f6a00640 1 -- 172.21.15.114:0/3406384078 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f0005020 con 0x7f10fc105bc0 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.390+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10fc1a2870 con 0x7f10fc105bc0 2024-09-21T07:00:41.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.390+0000 7f10f6a00640 1 -- 172.21.15.114:0/3406384078 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f0004a00 con 0x7f10fc105bc0 2024-09-21T07:00:41.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.390+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10fc1a8fd0 con 0x7f10fc105bc0 2024-09-21T07:00:41.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.393+0000 7f110b49d740 1 Processor -- start 2024-09-21T07:00:41.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.393+0000 7f110b49d740 1 -- start start 2024-09-21T07:00:41.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f110b49d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd2673340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f110b49d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2673880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f110b49d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22ab840 0x556bd2673dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f110b49d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556bd25ea290 con 0x556bd22a3660 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f110b49d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556bd25d96c0 con 0x556bd22cc520 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f110b49d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556bd25dd4d0 con 0x556bd22ab840 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1102800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd2673340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1102800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd2673340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54874/0 (socket says 172.21.15.114:54874) 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1102800640 1 -- 172.21.15.114:0/300859037 learned_addr learned my addr 172.21.15.114:0/300859037 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f10f6a00640 1 -- 172.21.15.114:0/3406384078 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f10f0004ba0 con 0x7f10fc105bc0 2024-09-21T07:00:41.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1101400640 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22ab840 0x556bd2673dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1101e00640 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2673880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1101400640 1 -- 172.21.15.114:0/300859037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 msgr2=0x556bd2673340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1101400640 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd2673340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.394+0000 7f1101400640 1 -- 172.21.15.114:0/300859037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 msgr2=0x556bd2673880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f1101400640 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2673880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f1101400640 1 -- 172.21.15.114:0/300859037 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd2566460 con 0x556bd22ab840 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f1101e00640 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2673880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f1101400640 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22ab840 0x556bd2673dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009230 tx=0x7f10ec004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8469dd2e72f6234 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:41.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f10eb000640 1 -- 172.21.15.114:0/300859037 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec035040 con 0x556bd22ab840 2024-09-21T07:00:41.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f10eb000640 1 -- 172.21.15.114:0/300859037 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec0054f0 con 0x556bd22ab840 2024-09-21T07:00:41.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.395+0000 7f10eb000640 1 -- 172.21.15.114:0/300859037 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0035f0 con 0x556bd22ab840 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.396+0000 7f10f6a00640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10e40b6680 0x7f10e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.396+0000 7f10f6a00640 1 -- 172.21.15.114:0/3406384078 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f10f00e7a40 con 0x7f10fc105bc0 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.396+0000 7f1102800640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10e40b6680 0x7f10e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.396+0000 7f110b49d740 1 -- 172.21.15.114:0/300859037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22ab840 msgr2=0x556bd2673dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.396+0000 7f110b49d740 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22ab840 0x556bd2673dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009230 tx=0x7f10ec004040 comp rx=0 tx=0).stop 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 -- 172.21.15.114:0/300859037 shutdown_connections 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22ab840 0x556bd2673dc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2673880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 --2- 172.21.15.114:0/300859037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd2673340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 -- 172.21.15.114:0/300859037 >> 172.21.15.114:0/300859037 conn(0x556bd22c0910 msgr2=0x556bd2671230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 -- 172.21.15.114:0/300859037 shutdown_connections 2024-09-21T07:00:41.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 -- 172.21.15.114:0/300859037 wait complete. 2024-09-21T07:00:41.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f1102800640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10e40b6680 0x7f10e40b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009fd0 tx=0x7f10f8009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:41.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.397+0000 7f110b49d740 1 -- start start 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 Processor -- start 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 0x556bd267ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 0x556bd267f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd268adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556bd243abb0 con 0x556bd22ab840 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556bd25746c0 con 0x556bd22cc520 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556bd2681ac0 con 0x556bd22a3660 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f110b49d740 10 client.0 _ll_register_callbacks cb 0x556bd258f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f1101e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 0x556bd267ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.398+0000 7f1101400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 0x556bd267f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 0x556bd267ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54986/0 (socket says 172.21.15.114:54986) 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 0x556bd267f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43562/0 (socket says 172.21.15.114:43562) 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101e00640 1 -- 172.21.15.114:0/67901723 learned_addr learned my addr 172.21.15.114:0/67901723 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1102800640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd268adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 -- 172.21.15.114:0/67901723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 msgr2=0x556bd267ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 0x556bd267ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 -- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 msgr2=0x556bd268adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd268adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 -- 172.21.15.114:0/67901723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd2337b00 con 0x556bd22ab840 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101e00640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 0x556bd267ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f1101400640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 0x556bd267f440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f10ec02a910 tx=0x7f10ec004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0959150f4b447e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:41.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.399+0000 7f10df400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f10df400640 1 -- 172.21.15.114:0/67901723 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec053060 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f10df400640 1 -- 172.21.15.114:0/67901723 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f10ec004ea0 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ec0054f0 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f10df400640 1 -- 172.21.15.114:0/67901723 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f10ec05a050 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f110b49d740 10 client.16971 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bd23ae960 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.400+0000 7f10df400640 1 -- 172.21.15.114:0/67901723 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec0054f0 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f10df400640 1 -- 172.21.15.114:0/67901723 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f10ec003d10 con 0x556bd22ab840 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f110b49d740 10 client.16971 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f110b49d740 10 client.16971 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f110b49d740 20 client.16971 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:41.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f110b49d740 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd228aea0 0x556bd2690be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f110b49d740 4 client.16971 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bd25ec1e0 con 0x556bd228aea0 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.401+0000 7f1101e00640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd228aea0 0x556bd2690be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.402+0000 7f1101e00640 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd228aea0 0x556bd2690be0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef56be2212622377 server_cookie=cebedb6b6d76d422 in_seq=0 out_seq=0 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.402+0000 7f10df400640 10 client.16971 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.402+0000 7f10df400640 1 -- 172.21.15.114:0/67901723 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556bd25ec1e0 con 0x556bd228aea0 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.402+0000 7f10df400640 10 client.16971 handle_command_reply: tid=0 2024-09-21T07:00:41.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.402+0000 7f110b49d740 1 client.16971 shutdown 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 msgr2=0x556bd267f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 0x556bd267f440 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f10ec02a910 tx=0x7f10ec004980 comp rx=0 tx=0).stop 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 shutdown_connections 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22cc520 0x556bd268adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22ab840 0x556bd267f440 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd228aea0 0x556bd2690be0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 --2- 172.21.15.114:0/67901723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22a3660 0x556bd267ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 >> 172.21.15.114:0/67901723 conn(0x556bd22c0910 msgr2=0x556bd2676520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f10df400640 0 client.16971 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 shutdown_connections 2024-09-21T07:00:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 1 -- 172.21.15.114:0/67901723 wait complete. 2024-09-21T07:00:41.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.403+0000 7f110b49d740 20 client.16971 trim_cache size 0 max 16384 2024-09-21T07:00:41.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.421+0000 7f110b49d740 1 Processor -- start 2024-09-21T07:00:41.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.421+0000 7f110b49d740 1 -- start start 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f110b49d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2678420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f110b49d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd2678960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f110b49d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd2676440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f110b49d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1b2dd0 con 0x556bd22a3660 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f110b49d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1b49a0 con 0x556bd22ab840 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f110b49d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1b34a0 con 0x556bd22cc520 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2678420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd2678960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2678420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43574/0 (socket says 172.21.15.114:43574) 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101e00640 1 -- 172.21.15.114:0/3761061989 learned_addr learned my addr 172.21.15.114:0/3761061989 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:41.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1102800640 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd2676440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101400640 1 -- 172.21.15.114:0/3761061989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 msgr2=0x556bd2676440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101400640 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd2676440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101400640 1 -- 172.21.15.114:0/3761061989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 msgr2=0x556bd2678420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.422+0000 7f1101400640 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2678420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f1101400640 1 -- 172.21.15.114:0/3761061989 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd267a590 con 0x556bd22ab840 2024-09-21T07:00:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f1102800640 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd2676440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f1101e00640 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2678420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f1101400640 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd2678960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009230 tx=0x7f10ec02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd33af3823d0673 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f10ea600640 1 -- 172.21.15.114:0/3761061989 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec053030 con 0x556bd22ab840 2024-09-21T07:00:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f10ea600640 1 -- 172.21.15.114:0/3761061989 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec04a070 con 0x556bd22ab840 2024-09-21T07:00:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.423+0000 7f10ea600640 1 -- 172.21.15.114:0/3761061989 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec035050 con 0x556bd22ab840 2024-09-21T07:00:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 -- 172.21.15.114:0/3761061989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 msgr2=0x556bd2678960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd2678960 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009230 tx=0x7f10ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 -- 172.21.15.114:0/3761061989 shutdown_connections 2024-09-21T07:00:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd2676440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd2678960 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 --2- 172.21.15.114:0/3761061989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2678420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 -- 172.21.15.114:0/3761061989 >> 172.21.15.114:0/3761061989 conn(0x556bd22c0910 msgr2=0x556bd267ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 -- 172.21.15.114:0/3761061989 shutdown_connections 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.424+0000 7f110b49d740 1 -- 172.21.15.114:0/3761061989 wait complete. 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 -- start start 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 Processor -- start 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2691f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd26924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd26fa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1b42a0 con 0x556bd22a3660 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1b3b70 con 0x556bd22ab840 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f10fc1b0db0 con 0x556bd22cc520 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.425+0000 7f110b49d740 10 client.0 _ll_register_callbacks cb 0x7f10ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2691f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2691f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:43576/0 (socket says 172.21.15.114:43576) 2024-09-21T07:00:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd26924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101e00640 1 -- 172.21.15.114:0/1513117551 learned_addr learned my addr 172.21.15.114:0/1513117551 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:41.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1102800640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd26fa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 -- 172.21.15.114:0/1513117551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 msgr2=0x556bd26fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd26fa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 -- 172.21.15.114:0/1513117551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 msgr2=0x556bd2691f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:41.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2691f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 -- 172.21.15.114:0/1513117551 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd26fa8a0 con 0x556bd22ab840 2024-09-21T07:00:41.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101e00640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2691f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f1101400640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd26924d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x556bd255f770 tx=0x7f10ec0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee50627885a3e6eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.426+0000 7f10df400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f10df400640 1 -- 172.21.15.114:0/1513117551 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec035070 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f10df400640 1 -- 172.21.15.114:0/1513117551 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f10ec04e670 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10ec04a070 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f10df400640 1 -- 172.21.15.114:0/1513117551 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f10ec06e050 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f110b49d740 10 client.27302 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556bd26fb9b0 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.427+0000 7f10df400640 1 -- 172.21.15.114:0/1513117551 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ec04a070 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f10df400640 1 -- 172.21.15.114:0/1513117551 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f10ec03cb20 con 0x556bd22ab840 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f110b49d740 10 client.27302 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f110b49d740 10 client.27302 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f110b49d740 20 client.27302 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f110b49d740 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd2260250 0x556bd26fe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f110b49d740 4 client.27302 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6e774d75", "release": true, "await": true}] 2024-09-21T07:00:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6e774d75", "release": true, "await": true}) -- 0x556bd25e7c90 con 0x556bd2260250 2024-09-21T07:00:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f1101e00640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd2260250 0x556bd26fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f1101e00640 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd2260250 0x556bd26fe040 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37b083a4ec64391e server_cookie=e4dae8f7ac81baed in_seq=0 out_seq=0 2024-09-21T07:00:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.428+0000 7f10df400640 10 client.27302 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:41.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.463+0000 7f10df400640 1 -- 172.21.15.114:0/1513117551 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556bd25e7c90 con 0x556bd2260250 2024-09-21T07:00:41.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.463+0000 7f10df400640 10 client.27302 handle_command_reply: tid=0 2024-09-21T07:00:41.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.463+0000 7f110b49d740 1 client.27302 shutdown 2024-09-21T07:00:41.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.464+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 msgr2=0x556bd26924d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.464+0000 7f110b49d740 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd26924d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x556bd255f770 tx=0x7f10ec0095c0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 shutdown_connections 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556bd2260250 0x556bd26fe040 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556bd22cc520 0x556bd26fa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bd22ab840 0x556bd26924d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 --2- 172.21.15.114:0/1513117551 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556bd22a3660 0x556bd2691f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 >> 172.21.15.114:0/1513117551 conn(0x556bd22c0910 msgr2=0x556bd2674370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:41.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f10df400640 0 client.27302 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:41.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.465+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 shutdown_connections 2024-09-21T07:00:41.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.467+0000 7f110b49d740 1 -- 172.21.15.114:0/1513117551 wait complete. 2024-09-21T07:00:41.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.467+0000 7f110b49d740 20 client.27302 trim_cache size 0 max 16384 2024-09-21T07:00:41.470 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:00:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 42, 2024-09-21T07:00:41.470 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 28, 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "6e774d75": { 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 28, 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:00:41.471 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 139.2, 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:00:41.472 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:41.473 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:00:41.474 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:00:41.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10e40b6680 msgr2=0x7f10e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10e40b6680 0x7f10e40b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009fd0 tx=0x7f10f8009500 comp rx=0 tx=0).stop 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc105bc0 msgr2=0x7f10fc19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc105bc0 0x7f10fc19c610 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10f000e9f0 tx=0x7f10f000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 shutdown_connections 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f10fc10be50 0x7f10fc1a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10fc10b500 0x7f10fc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f10e40b6680 0x7f10e40b8b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 --2- 172.21.15.114:0/3406384078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10fc105bc0 0x7f10fc19c610 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:41.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.471+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 >> 172.21.15.114:0/3406384078 conn(0x7f10fc101670 msgr2=0x7f10fc108b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:41.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.474+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 shutdown_connections 2024-09-21T07:00:41.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:41.474+0000 7f1103200640 1 -- 172.21.15.114:0/3406384078 wait complete. 2024-09-21T07:00:41.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:00:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 365.0 s, 196.6 tps, lat 2044.592 ms stddev 5668.352 2024-09-21T07:00:43.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:42 smithi183 ceph-mon[34219]: pgmap v593: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 573 KiB/s wr, 83 op/s 2024-09-21T07:00:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:42 smithi193 ceph-mon[34601]: pgmap v593: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 573 KiB/s wr, 83 op/s 2024-09-21T07:00:43.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:42 smithi114 ceph-mon[30654]: pgmap v593: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 573 KiB/s wr, 83 op/s 2024-09-21T07:00:43.457 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6e774d75', total seconds elapsed: 25.5 2024-09-21T07:00:43.540 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.035+0000 7fd954200640 1 -- 172.21.15.114:0/569364684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c075dd0 msgr2=0x7fd94c0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.035+0000 7fd954200640 1 --2- 172.21.15.114:0/569364684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c075dd0 0x7fd94c0761e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd94401acb0 tx=0x7fd94403b900 comp rx=0 tx=0).stop 2024-09-21T07:00:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.036+0000 7fd954200640 1 -- 172.21.15.114:0/569364684 shutdown_connections 2024-09-21T07:00:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.036+0000 7fd954200640 1 --2- 172.21.15.114:0/569364684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c114440 0x7fd94c116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.036+0000 7fd954200640 1 --2- 172.21.15.114:0/569364684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c075dd0 0x7fd94c0761e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.036+0000 7fd954200640 1 --2- 172.21.15.114:0/569364684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c077380 0x7fd94c0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.036+0000 7fd954200640 1 -- 172.21.15.114:0/569364684 >> 172.21.15.114:0/569364684 conn(0x7fd94c0ff5e0 msgr2=0x7fd94c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:45.040 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:44 smithi114 ceph-mon[30654]: pgmap v594: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-09-21T07:00:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.037+0000 7fd954200640 1 -- 172.21.15.114:0/569364684 shutdown_connections 2024-09-21T07:00:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.037+0000 7fd954200640 1 -- 172.21.15.114:0/569364684 wait complete. 2024-09-21T07:00:45.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.039+0000 7fd954200640 1 Processor -- start 2024-09-21T07:00:45.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 -- start start 2024-09-21T07:00:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c075dd0 0x7fd94c074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 0x7fd94c0722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c114440 0x7fd94c113c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10a420 con 0x7fd94c075dd0 2024-09-21T07:00:45.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10a2a0 con 0x7fd94c114440 2024-09-21T07:00:45.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.041+0000 7fd954200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10a5a0 con 0x7fd94c077380 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 0x7fd94c0722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 0x7fd94c0722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52080/0 (socket says 172.21.15.114:52080) 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 -- 172.21.15.114:0/2124038799 learned_addr learned my addr 172.21.15.114:0/2124038799 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd951800640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c075dd0 0x7fd94c074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 -- 172.21.15.114:0/2124038799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c114440 msgr2=0x7fd94c113c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c114440 0x7fd94c113c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 -- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c075dd0 msgr2=0x7fd94c074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c075dd0 0x7fd94c074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.042+0000 7fd950e00640 1 -- 172.21.15.114:0/2124038799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c072920 con 0x7fd94c077380 2024-09-21T07:00:45.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.043+0000 7fd951800640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c075dd0 0x7fd94c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.044+0000 7fd950e00640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 0x7fd94c0722a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd944005e30 tx=0x7fd944041970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.044+0000 7fd94a000640 1 -- 172.21.15.114:0/2124038799 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944045070 con 0x7fd94c077380 2024-09-21T07:00:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.044+0000 7fd94a000640 1 -- 172.21.15.114:0/2124038799 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd944040800 con 0x7fd94c077380 2024-09-21T07:00:45.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.044+0000 7fd94a000640 1 -- 172.21.15.114:0/2124038799 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440043c0 con 0x7fd94c077380 2024-09-21T07:00:45.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.045+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c072bb0 con 0x7fd94c077380 2024-09-21T07:00:45.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.046+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c073040 con 0x7fd94c077380 2024-09-21T07:00:45.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.052+0000 7fd94a000640 1 -- 172.21.15.114:0/2124038799 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd944048020 con 0x7fd94c077380 2024-09-21T07:00:45.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.054+0000 7fd95b013740 1 Processor -- start 2024-09-21T07:00:45.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.054+0000 7fd95b013740 1 -- start start 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd95b013740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 0x55ac02a86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd95b013740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 0x55ac02a872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd95b013740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026bf390 0x55ac02a87820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd95b013740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ac029fdfb0 con 0x55ac026bf390 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd95b013740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac029ed3e0 con 0x55ac026b71b0 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd95b013740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ac029f11f0 con 0x55ac026e0070 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd951800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 0x55ac02a872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd951800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 0x55ac02a872e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40746/0 (socket says 172.21.15.114:40746) 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.055+0000 7fd951800640 1 -- 172.21.15.114:0/962814483 learned_addr learned my addr 172.21.15.114:0/962814483 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:45.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.056+0000 7fd951800640 1 -- 172.21.15.114:0/962814483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 msgr2=0x55ac02a86da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.056+0000 7fd951800640 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 0x55ac02a86da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.056+0000 7fd951800640 1 -- 172.21.15.114:0/962814483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026bf390 msgr2=0x55ac02a87820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.056+0000 7fd951800640 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026bf390 0x55ac02a87820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.056+0000 7fd951800640 1 -- 172.21.15.114:0/962814483 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac029b7cd0 con 0x55ac026b71b0 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.056+0000 7fd951800640 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 0x55ac02a872e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd940009c20 tx=0x7fd940010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6653fceb8eb54fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.058+0000 7fd93ea00640 1 -- 172.21.15.114:0/962814483 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940004020 con 0x55ac026b71b0 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.058+0000 7fd93ea00640 1 -- 172.21.15.114:0/962814483 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9400041c0 con 0x55ac026b71b0 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.059+0000 7fd93ea00640 1 -- 172.21.15.114:0/962814483 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9400126b0 con 0x55ac026b71b0 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.060+0000 7fd95b013740 1 -- 172.21.15.114:0/962814483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 msgr2=0x55ac02a872e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.060+0000 7fd95b013740 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 0x55ac02a872e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd940009c20 tx=0x7fd940010740 comp rx=0 tx=0).stop 2024-09-21T07:00:45.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.061+0000 7fd94a000640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9340b6780 0x7fd9340b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.061+0000 7fd94a000640 1 -- 172.21.15.114:0/2124038799 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd944119460 con 0x7fd94c077380 2024-09-21T07:00:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.065+0000 7fd95b013740 1 -- 172.21.15.114:0/962814483 shutdown_connections 2024-09-21T07:00:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.065+0000 7fd95b013740 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026bf390 0x55ac02a87820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.065+0000 7fd95b013740 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026b71b0 0x55ac02a872e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.065+0000 7fd95b013740 1 --2- 172.21.15.114:0/962814483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 0x55ac02a86da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.065+0000 7fd95b013740 1 -- 172.21.15.114:0/962814483 >> 172.21.15.114:0/962814483 conn(0x55ac026d4460 msgr2=0x55ac02a84c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:45.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.067+0000 7fd95b013740 1 -- 172.21.15.114:0/962814483 shutdown_connections 2024-09-21T07:00:45.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.067+0000 7fd952200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9340b6780 0x7fd9340b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.067+0000 7fd95b013740 1 -- 172.21.15.114:0/962814483 wait complete. 2024-09-21T07:00:45.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.068+0000 7fd95b013740 1 -- start start 2024-09-21T07:00:45.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.068+0000 7fd95b013740 1 Processor -- start 2024-09-21T07:00:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.069+0000 7fd952200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9340b6780 0x7fd9340b8c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd938006fd0 tx=0x7fd938006c40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.069+0000 7fd95b013740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a92840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.069+0000 7fd95b013740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.069+0000 7fd95b013740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 0x55ac02a9e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.069+0000 7fd95b013740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ac0284e700 con 0x55ac026b71b0 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.070+0000 7fd95b013740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ac02988580 con 0x55ac026bf390 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.070+0000 7fd95b013740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ac02a95580 con 0x55ac026e0070 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.070+0000 7fd95b013740 10 client.0 _ll_register_callbacks cb 0x55ac02998470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a92d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40756/0 (socket says 172.21.15.114:40756) 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 -- 172.21.15.114:0/1757479474 learned_addr learned my addr 172.21.15.114:0/1757479474 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd951800640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a92840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 -- 172.21.15.114:0/1757479474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 msgr2=0x55ac02a9e8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 0x55ac02a9e8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 -- 172.21.15.114:0/1757479474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 msgr2=0x55ac02a92840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a92840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.071+0000 7fd950e00640 1 -- 172.21.15.114:0/1757479474 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac0274b650 con 0x55ac026bf390 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.072+0000 7fd951800640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a92840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:45.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.072+0000 7fd950e00640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a92d80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd9440debe0 tx=0x7fd9440de8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff60b6a04f6cc3fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.072+0000 7fd933400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd933400640 1 -- 172.21.15.114:0/1757479474 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440e39c0 con 0x55ac026bf390 2024-09-21T07:00:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd933400640 1 -- 172.21.15.114:0/1757479474 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd944116a60 con 0x55ac026bf390 2024-09-21T07:00:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ac0298ab90 con 0x55ac026bf390 2024-09-21T07:00:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd933400640 1 -- 172.21.15.114:0/1757479474 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd944116c60 con 0x55ac026bf390 2024-09-21T07:00:45.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd95b013740 10 client.27311 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ac029a5950 con 0x55ac026bf390 2024-09-21T07:00:45.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.073+0000 7fd933400640 1 -- 172.21.15.114:0/1757479474 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ac0298ab90 con 0x55ac026bf390 2024-09-21T07:00:45.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.074+0000 7fd933400640 1 -- 172.21.15.114:0/1757479474 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7fd944004670 con 0x55ac026bf390 2024-09-21T07:00:45.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.078+0000 7fd95b013740 10 client.27311 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:45.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.078+0000 7fd95b013740 10 client.27311 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:00:45.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.078+0000 7fd95b013740 20 client.27311 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:45.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.078+0000 7fd95b013740 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac0269e9f0 0x55ac02aa47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.078+0000 7fd95b013740 4 client.27311 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:45.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.078+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac029fff00 con 0x55ac0269e9f0 2024-09-21T07:00:45.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.079+0000 7fd951800640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac0269e9f0 0x55ac02aa47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.080+0000 7fd933400640 10 client.27311 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:00:45.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.080+0000 7fd951800640 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac0269e9f0 0x55ac02aa47e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d886f00ea3dc7fee server_cookie=4e2bdeac0ba1b880 in_seq=0 out_seq=0 2024-09-21T07:00:45.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.081+0000 7fd933400640 1 -- 172.21.15.114:0/1757479474 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac029fff00 con 0x55ac0269e9f0 2024-09-21T07:00:45.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.081+0000 7fd933400640 10 client.27311 handle_command_reply: tid=0 2024-09-21T07:00:45.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.081+0000 7fd95b013740 1 client.27311 shutdown 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 msgr2=0x55ac02a92d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a92d80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd9440debe0 tx=0x7fd9440de8f0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 shutdown_connections 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026e0070 0x55ac02a9e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac0269e9f0 0x55ac02aa47e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a92d80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 --2- 172.21.15.114:0/1757479474 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a92840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd933400640 0 client.27311 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:00:45.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.082+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 >> 172.21.15.114:0/1757479474 conn(0x55ac026d4460 msgr2=0x55ac02a89f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:45.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.088+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 shutdown_connections 2024-09-21T07:00:45.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.088+0000 7fd95b013740 1 -- 172.21.15.114:0/1757479474 wait complete. 2024-09-21T07:00:45.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.088+0000 7fd95b013740 20 client.27311 trim_cache size 0 max 16384 2024-09-21T07:00:45.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:44 smithi183 ceph-mon[34219]: pgmap v594: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-09-21T07:00:45.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.120+0000 7fd95b013740 1 Processor -- start 2024-09-21T07:00:45.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 -- start start 2024-09-21T07:00:45.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 0x55ac02a8a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026e0070 0x55ac02a9ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c121870 con 0x55ac026e0070 2024-09-21T07:00:45.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c123440 con 0x55ac026bf390 2024-09-21T07:00:45.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.121+0000 7fd95b013740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c121f40 con 0x55ac026b71b0 2024-09-21T07:00:45.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd951800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 0x55ac02a8a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd950e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd951800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 0x55ac02a8a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52092/0 (socket says 172.21.15.114:52092) 2024-09-21T07:00:45.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd951800640 1 -- 172.21.15.114:0/2615706176 learned_addr learned my addr 172.21.15.114:0/2615706176 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:45.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd950e00640 1 -- 172.21.15.114:0/2615706176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 msgr2=0x55ac02a8a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd950e00640 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 0x55ac02a8a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd950e00640 1 -- 172.21.15.114:0/2615706176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026e0070 msgr2=0x55ac02a9ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd950e00640 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026e0070 0x55ac02a9ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd950e00640 1 -- 172.21.15.114:0/2615706176 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac02a89090 con 0x55ac026bf390 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.122+0000 7fd951800640 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 0x55ac02a8a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.123+0000 7fd950e00640 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a9e7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd94403bec0 tx=0x7fd9440de960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e6e2a6a4bc1424 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.124+0000 7fd93e000640 1 -- 172.21.15.114:0/2615706176 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440e8900 con 0x55ac026bf390 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.124+0000 7fd93e000640 1 -- 172.21.15.114:0/2615706176 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440fa2b0 con 0x55ac026bf390 2024-09-21T07:00:45.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.124+0000 7fd93e000640 1 -- 172.21.15.114:0/2615706176 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944116a60 con 0x55ac026bf390 2024-09-21T07:00:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.126+0000 7fd95b013740 1 -- 172.21.15.114:0/2615706176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 msgr2=0x55ac02a9e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.126+0000 7fd95b013740 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a9e7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd94403bec0 tx=0x7fd9440de960 comp rx=0 tx=0).stop 2024-09-21T07:00:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.129+0000 7fd95b013740 1 -- 172.21.15.114:0/2615706176 shutdown_connections 2024-09-21T07:00:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.129+0000 7fd95b013740 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026e0070 0x55ac02a9ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.129+0000 7fd95b013740 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026bf390 0x55ac02a9e7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.129+0000 7fd95b013740 1 --2- 172.21.15.114:0/2615706176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026b71b0 0x55ac02a8a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.129+0000 7fd95b013740 1 -- 172.21.15.114:0/2615706176 >> 172.21.15.114:0/2615706176 conn(0x55ac026d4460 msgr2=0x55ac02a89ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.130+0000 7fd95b013740 1 -- 172.21.15.114:0/2615706176 shutdown_connections 2024-09-21T07:00:45.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.130+0000 7fd95b013740 1 -- 172.21.15.114:0/2615706176 wait complete. 2024-09-21T07:00:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.131+0000 7fd95b013740 1 -- start start 2024-09-21T07:00:45.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.131+0000 7fd95b013740 1 Processor -- start 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a9ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 0x55ac02a972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026e0070 0x55ac02a97820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c122d40 con 0x55ac026b71b0 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c122610 con 0x55ac026e0070 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c11f850 con 0x55ac026bf390 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.132+0000 7fd95b013740 10 client.0 _ll_register_callbacks cb 0x7fd9440db380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:45.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 0x55ac02a972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 0x55ac02a972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52106/0 (socket says 172.21.15.114:52106) 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 -- 172.21.15.114:0/2803633921 learned_addr learned my addr 172.21.15.114:0/2803633921 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd952200640 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026e0070 0x55ac02a97820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 -- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026e0070 msgr2=0x55ac02a97820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026e0070 0x55ac02a97820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 -- 172.21.15.114:0/2803633921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 msgr2=0x55ac02a9ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a9ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.133+0000 7fd950e00640 1 -- 172.21.15.114:0/2803633921 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac02a97de0 con 0x55ac026bf390 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.134+0000 7fd950e00640 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 0x55ac02a972e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd9440e3b80 tx=0x7fd944000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef94954c499893f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:45.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.134+0000 7fd933400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:45.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.136+0000 7fd933400640 1 -- 172.21.15.114:0/2803633921 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440ed060 con 0x55ac026bf390 2024-09-21T07:00:45.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.136+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9440fa2b0 con 0x55ac026bf390 2024-09-21T07:00:45.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.136+0000 7fd933400640 1 -- 172.21.15.114:0/2803633921 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd944004560 con 0x55ac026bf390 2024-09-21T07:00:45.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.138+0000 7fd933400640 1 -- 172.21.15.114:0/2803633921 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd9440db530 con 0x55ac026bf390 2024-09-21T07:00:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.138+0000 7fd95b013740 10 client.27256 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:45.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.138+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ac02a97de0 con 0x55ac026bf390 2024-09-21T07:00:45.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.140+0000 7fd933400640 1 -- 172.21.15.114:0/2803633921 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440fa2b0 con 0x55ac026bf390 2024-09-21T07:00:45.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd933400640 1 -- 172.21.15.114:0/2803633921 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7fd9440e8b50 con 0x55ac026bf390 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd95b013740 10 client.27256 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd95b013740 10 client.27256 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd95b013740 20 client.27256 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd95b013740 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac02673da0 0x55ac02b121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd95b013740 4 client.27256 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac029fb9b0 con 0x55ac02673da0 2024-09-21T07:00:45.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.142+0000 7fd951800640 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac02673da0 0x55ac02b121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:45.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.143+0000 7fd933400640 10 client.27256 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:00:45.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.143+0000 7fd951800640 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac02673da0 0x55ac02b121e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2153ffc71e2b6a02 server_cookie=663fbcc5ac0752b7 in_seq=0 out_seq=0 2024-09-21T07:00:45.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.144+0000 7fd933400640 1 -- 172.21.15.114:0/2803633921 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac029fb9b0 con 0x55ac02673da0 2024-09-21T07:00:45.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.144+0000 7fd933400640 10 client.27256 handle_command_reply: tid=0 2024-09-21T07:00:45.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.144+0000 7fd95b013740 1 client.27256 shutdown 2024-09-21T07:00:45.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.144+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 msgr2=0x55ac02a972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.144+0000 7fd95b013740 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 0x55ac02a972e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd9440e3b80 tx=0x7fd944000f40 comp rx=0 tx=0).stop 2024-09-21T07:00:45.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 shutdown_connections 2024-09-21T07:00:45.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd95b013740 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55ac02673da0 0x55ac02b121e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd95b013740 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ac026e0070 0x55ac02a97820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd95b013740 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ac026bf390 0x55ac02a972e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd95b013740 1 --2- 172.21.15.114:0/2803633921 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ac026b71b0 0x55ac02a9ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd933400640 0 client.27256 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.146+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 >> 172.21.15.114:0/2803633921 conn(0x55ac026d4460 msgr2=0x55ac02a87d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.147+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 shutdown_connections 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.147+0000 7fd95b013740 1 -- 172.21.15.114:0/2803633921 wait complete. 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.147+0000 7fd95b013740 20 client.27256 trim_cache size 0 max 16384 2024-09-21T07:00:45.150 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9340b6780 msgr2=0x7fd9340b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9340b6780 0x7fd9340b8c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd938006fd0 tx=0x7fd938006c40 comp rx=0 tx=0).stop 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 msgr2=0x7fd94c0722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 0x7fd94c0722a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd944005e30 tx=0x7fd944041970 comp rx=0 tx=0).stop 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 shutdown_connections 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9340b6780 0x7fd9340b8c90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.151 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd94c114440 0x7fd94c113c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd94c077380 0x7fd94c0722a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 --2- 172.21.15.114:0/2124038799 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd94c075dd0 0x7fd94c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:45.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.149+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 >> 172.21.15.114:0/2124038799 conn(0x7fd94c0ff5e0 msgr2=0x7fd94c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:45.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.150+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 shutdown_connections 2024-09-21T07:00:45.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:45.150+0000 7fd954200640 1 -- 172.21.15.114:0/2124038799 wait complete. 2024-09-21T07:00:45.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:44 smithi193 ceph-mon[34601]: pgmap v594: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-09-21T07:00:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 -- 172.21.15.114:0/3801257368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 msgr2=0x7f6370115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 --2- 172.21.15.114:0/3801257368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 0x7f6370115fd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6360009900 tx=0x7f636002a400 comp rx=0 tx=0).stop 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 -- 172.21.15.114:0/3801257368 shutdown_connections 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 --2- 172.21.15.114:0/3801257368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6370116510 0x7f6370118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 --2- 172.21.15.114:0/3801257368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 0x7f6370115fd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 --2- 172.21.15.114:0/3801257368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f637010a3c0 0x7f637010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.273+0000 7f6374e00640 1 -- 172.21.15.114:0/3801257368 >> 172.21.15.114:0/3801257368 conn(0x7f6370070c00 msgr2=0x7f6370071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.274+0000 7f6374e00640 1 -- 172.21.15.114:0/3801257368 shutdown_connections 2024-09-21T07:00:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.275+0000 7f6374e00640 1 -- 172.21.15.114:0/3801257368 wait complete. 2024-09-21T07:00:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.276+0000 7f6374e00640 1 Processor -- start 2024-09-21T07:00:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 -- start start 2024-09-21T07:00:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637010a3c0 0x7f6370121900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 0x7f637011abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6370116510 0x7f637011b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6370122670 con 0x7f6370116510 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63701224f0 con 0x7f637010ad10 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.277+0000 7f6374e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63701227f0 con 0x7f637010a3c0 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.278+0000 7f636f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637010a3c0 0x7f6370121900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.278+0000 7f636f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637010a3c0 0x7f6370121900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52118/0 (socket says 172.21.15.114:52118) 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.278+0000 7f636f400640 1 -- 172.21.15.114:0/486663010 learned_addr learned my addr 172.21.15.114:0/486663010 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:46.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.278+0000 7f636fe00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6370116510 0x7f637011b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.279+0000 7f636ea00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 0x7f637011abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.280+0000 7f636fe00640 1 -- 172.21.15.114:0/486663010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637010a3c0 msgr2=0x7f6370121900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.280+0000 7f636fe00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637010a3c0 0x7f6370121900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.280+0000 7f636fe00640 1 -- 172.21.15.114:0/486663010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 msgr2=0x7f637011abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.280+0000 7f636fe00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 0x7f637011abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.280+0000 7f636fe00640 1 -- 172.21.15.114:0/486663010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6370120560 con 0x7f6370116510 2024-09-21T07:00:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.281+0000 7f636fe00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6370116510 0x7f637011b1f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c710 tx=0x7f635c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.287+0000 7f6367e00640 1 -- 172.21.15.114:0/486663010 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c004220 con 0x7f6370116510 2024-09-21T07:00:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.287+0000 7f6367e00640 1 -- 172.21.15.114:0/486663010 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f635c00ce20 con 0x7f6370116510 2024-09-21T07:00:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.287+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63701207f0 con 0x7f6370116510 2024-09-21T07:00:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.288+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6370120d10 con 0x7f6370116510 2024-09-21T07:00:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.288+0000 7f6367e00640 1 -- 172.21.15.114:0/486663010 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c003db0 con 0x7f6370116510 2024-09-21T07:00:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.302+0000 7f6367e00640 1 -- 172.21.15.114:0/486663010 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f635c0035f0 con 0x7f6370116510 2024-09-21T07:00:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.308+0000 7f6367e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63540b6840 0x7f63540b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.309+0000 7f6367e00640 1 -- 172.21.15.114:0/486663010 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f635c0e7520 con 0x7f6370116510 2024-09-21T07:00:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.311+0000 7f637d5e5740 1 Processor -- start 2024-09-21T07:00:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.313+0000 7f636f400640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63540b6840 0x7f63540b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.314+0000 7f636f400640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63540b6840 0x7f63540b8d50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6368005c40 tx=0x7f6368007620 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:46.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 -- start start 2024-09-21T07:00:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e69a01b0 0x55e1e6d44e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d45350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e697f4d0 0x55e1e6d45890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e1e6cba6b0 con 0x55e1e69772f0 2024-09-21T07:00:46.320 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1e6ca9ae0 con 0x55e1e69a01b0 2024-09-21T07:00:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.318+0000 7f637d5e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1e6cad8f0 con 0x55e1e697f4d0 2024-09-21T07:00:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.320+0000 7f636ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d45350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d45350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56706/0 (socket says 172.21.15.114:56706) 2024-09-21T07:00:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 -- 172.21.15.114:0/2523124955 learned_addr learned my addr 172.21.15.114:0/2523124955 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636fe00640 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e697f4d0 0x55e1e6d45890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 -- 172.21.15.114:0/2523124955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e697f4d0 msgr2=0x55e1e6d45890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e697f4d0 0x55e1e6d45890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 -- 172.21.15.114:0/2523124955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e69a01b0 msgr2=0x55e1e6d44e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e69a01b0 0x55e1e6d44e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.321+0000 7f636ea00640 1 -- 172.21.15.114:0/2523124955 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e6c76780 con 0x55e1e69772f0 2024-09-21T07:00:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.325+0000 7f636ea00640 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d45350 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f6360004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e52eb6c327300cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.327+0000 7f6364c00640 1 -- 172.21.15.114:0/2523124955 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636003f070 con 0x55e1e69772f0 2024-09-21T07:00:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.328+0000 7f6364c00640 1 -- 172.21.15.114:0/2523124955 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63600054d0 con 0x55e1e69772f0 2024-09-21T07:00:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.328+0000 7f6364c00640 1 -- 172.21.15.114:0/2523124955 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360035070 con 0x55e1e69772f0 2024-09-21T07:00:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.329+0000 7f637d5e5740 1 -- 172.21.15.114:0/2523124955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 msgr2=0x55e1e6d45350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.329+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d45350 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f6360004200 comp rx=0 tx=0).stop 2024-09-21T07:00:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.330+0000 7f637d5e5740 1 -- 172.21.15.114:0/2523124955 shutdown_connections 2024-09-21T07:00:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.330+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e697f4d0 0x55e1e6d45890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.330+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d45350 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.330+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2523124955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e69a01b0 0x55e1e6d44e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.330+0000 7f637d5e5740 1 -- 172.21.15.114:0/2523124955 >> 172.21.15.114:0/2523124955 conn(0x55e1e6994300 msgr2=0x55e1e6d42d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:46.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.334+0000 7f637d5e5740 1 -- 172.21.15.114:0/2523124955 shutdown_connections 2024-09-21T07:00:46.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.337+0000 7f637d5e5740 1 -- 172.21.15.114:0/2523124955 wait complete. 2024-09-21T07:00:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.338+0000 7f637d5e5740 1 -- start start 2024-09-21T07:00:46.340 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.338+0000 7f637d5e5740 1 Processor -- start 2024-09-21T07:00:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.339+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.339+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d50930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f637d5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e1e6be1ab0 con 0x55e1e69772f0 2024-09-21T07:00:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f637d5e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1e6c50570 con 0x55e1e697f4d0 2024-09-21T07:00:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f637d5e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1e6d53530 con 0x55e1e69a01b0 2024-09-21T07:00:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f637d5e5740 10 client.0 _ll_register_callbacks cb 0x55e1e6c5f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f636ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d50930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f636ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d50930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40812/0 (socket says 172.21.15.114:40812) 2024-09-21T07:00:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f636ea00640 1 -- 172.21.15.114:0/487530082 learned_addr learned my addr 172.21.15.114:0/487530082 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.340+0000 7f636fe00640 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f636fe00640 1 -- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 msgr2=0x55e1e6d50930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f636fe00640 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d50930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f636fe00640 1 -- 172.21.15.114:0/487530082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 msgr2=0x55e1e6d503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f636fe00640 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f636fe00640 1 -- 172.21.15.114:0/487530082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e6a0b790 con 0x55e1e69a01b0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f6359600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.341+0000 7f636fe00640 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f635c0e48a0 tx=0x7f635c0e7420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a827b36cef1efd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.342+0000 7f6359600640 1 -- 172.21.15.114:0/487530082 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c01d040 con 0x55e1e69a01b0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.342+0000 7f6359600640 1 -- 172.21.15.114:0/487530082 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f635c0043c0 con 0x55e1e69a01b0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.342+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63600054d0 con 0x55e1e69a01b0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.342+0000 7f6359600640 1 -- 172.21.15.114:0/487530082 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f635c01d040 con 0x55e1e69a01b0 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.342+0000 7f637d5e5740 10 client.27265 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.342+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e1e6a84620 con 0x55e1e69a01b0 2024-09-21T07:00:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.343+0000 7f6359600640 1 -- 172.21.15.114:0/487530082 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f635c0b1a80 con 0x55e1e69a01b0 2024-09-21T07:00:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.344+0000 7f6359600640 1 -- 172.21.15.114:0/487530082 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f635c0abc70 con 0x55e1e69a01b0 2024-09-21T07:00:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.344+0000 7f637d5e5740 10 client.27265 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.344+0000 7f637d5e5740 10 client.27265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.344+0000 7f637d5e5740 20 client.27265 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.345+0000 7f637d5e5740 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e695ed80 0x55e1e6d62ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.345+0000 7f637d5e5740 4 client.27265 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.345+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f63600054d0 con 0x55e1e695ed80 2024-09-21T07:00:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.345+0000 7f636f400640 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e695ed80 0x55e1e6d62ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.346+0000 7f6359600640 10 client.27265 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:46.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.346+0000 7f636f400640 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e695ed80 0x55e1e6d62ae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83f20132fac5c62 server_cookie=3a115b262aa9cb7e in_seq=0 out_seq=0 2024-09-21T07:00:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.347+0000 7f6359600640 1 -- 172.21.15.114:0/487530082 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f63600054d0 con 0x55e1e695ed80 2024-09-21T07:00:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.347+0000 7f6359600640 10 client.27265 handle_command_reply: tid=0 2024-09-21T07:00:46.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.347+0000 7f637d5e5740 1 client.27265 shutdown 2024-09-21T07:00:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.353+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 msgr2=0x55e1e6d50e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.353+0000 7f637d5e5740 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f635c0e48a0 tx=0x7f635c0e7420 comp rx=0 tx=0).stop 2024-09-21T07:00:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 shutdown_connections 2024-09-21T07:00:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e695ed80 0x55e1e6d62ae0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d50930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f6359600640 0 client.27265 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 --2- 172.21.15.114:0/487530082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 >> 172.21.15.114:0/487530082 conn(0x55e1e6994300 msgr2=0x55e1e6d45e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.357+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 shutdown_connections 2024-09-21T07:00:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.358+0000 7f637d5e5740 1 -- 172.21.15.114:0/487530082 wait complete. 2024-09-21T07:00:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.358+0000 7f637d5e5740 20 client.27265 trim_cache size 0 max 16384 2024-09-21T07:00:46.376 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:00:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.400+0000 7f637d5e5740 1 Processor -- start 2024-09-21T07:00:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 -- start start 2024-09-21T07:00:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 0x55e1e6d46550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6350010060 con 0x55e1e69a01b0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6350011c30 con 0x55e1e697f4d0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.401+0000 7f637d5e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6350010730 con 0x55e1e69772f0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.402+0000 7f636f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 0x55e1e6d46550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.402+0000 7f636f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 0x55e1e6d46550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52152/0 (socket says 172.21.15.114:52152) 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.402+0000 7f636f400640 1 -- 172.21.15.114:0/1860819885 learned_addr learned my addr 172.21.15.114:0/1860819885 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.402+0000 7f636ea00640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.403+0000 7f636fe00640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.403+0000 7f636ea00640 1 -- 172.21.15.114:0/1860819885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 msgr2=0x55e1e6d46550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.403+0000 7f636ea00640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 0x55e1e6d46550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.403+0000 7f636ea00640 1 -- 172.21.15.114:0/1860819885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69a01b0 msgr2=0x55e1e6d50e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.403+0000 7f636ea00640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.404+0000 7f636ea00640 1 -- 172.21.15.114:0/1860819885 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e6d46a90 con 0x55e1e697f4d0 2024-09-21T07:00:46.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.404+0000 7f636fe00640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.404+0000 7f636f400640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 0x55e1e6d46550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.405+0000 7f636ea00640 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d508d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f636002a920 tx=0x7f6360005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd32d884b58174b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.405+0000 7f635be00640 1 -- 172.21.15.114:0/1860819885 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360035070 con 0x55e1e697f4d0 2024-09-21T07:00:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.406+0000 7f635be00640 1 -- 172.21.15.114:0/1860819885 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360003ca0 con 0x55e1e697f4d0 2024-09-21T07:00:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.406+0000 7f635be00640 1 -- 172.21.15.114:0/1860819885 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636003f040 con 0x55e1e697f4d0 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.407+0000 7f637d5e5740 1 -- 172.21.15.114:0/1860819885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 msgr2=0x55e1e6d508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.407+0000 7f637d5e5740 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d508d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f636002a920 tx=0x7f6360005730 comp rx=0 tx=0).stop 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.407+0000 7f637d5e5740 1 -- 172.21.15.114:0/1860819885 shutdown_connections 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.407+0000 7f637d5e5740 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69a01b0 0x55e1e6d50e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.407+0000 7f637d5e5740 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d508d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.408+0000 7f637d5e5740 1 --2- 172.21.15.114:0/1860819885 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69772f0 0x55e1e6d46550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.408+0000 7f637d5e5740 1 -- 172.21.15.114:0/1860819885 >> 172.21.15.114:0/1860819885 conn(0x55e1e6994300 msgr2=0x55e1e6d45e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.408+0000 7f637d5e5740 1 -- 172.21.15.114:0/1860819885 shutdown_connections 2024-09-21T07:00:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.408+0000 7f637d5e5740 1 -- 172.21.15.114:0/1860819885 wait complete. 2024-09-21T07:00:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.409+0000 7f637d5e5740 1 -- start start 2024-09-21T07:00:46.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.409+0000 7f637d5e5740 1 Processor -- start 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.410+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d59250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.410+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d56790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.410+0000 7f637d5e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d56ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.410+0000 7f637d5e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6350011530 con 0x55e1e69772f0 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.410+0000 7f637d5e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6350010e00 con 0x55e1e697f4d0 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.410+0000 7f637d5e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f635000e000 con 0x55e1e69a01b0 2024-09-21T07:00:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.411+0000 7f637d5e5740 10 client.0 _ll_register_callbacks cb 0x7f635c0a9690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.411+0000 7f636fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d56ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.411+0000 7f636ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d56790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.412+0000 7f636fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d56ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52160/0 (socket says 172.21.15.114:52160) 2024-09-21T07:00:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.412+0000 7f636ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d56790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40822/0 (socket says 172.21.15.114:40822) 2024-09-21T07:00:46.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.412+0000 7f636ea00640 1 -- 172.21.15.114:0/2617176441 learned_addr learned my addr 172.21.15.114:0/2617176441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.412+0000 7f636f400640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d59250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.413+0000 7f636ea00640 1 -- 172.21.15.114:0/2617176441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 msgr2=0x55e1e6d56ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.414+0000 7f636ea00640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d56ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.414+0000 7f636ea00640 1 -- 172.21.15.114:0/2617176441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 msgr2=0x55e1e6d59250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.414+0000 7f636ea00640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d59250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.415+0000 7f636ea00640 1 -- 172.21.15.114:0/2617176441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e6dcc300 con 0x55e1e697f4d0 2024-09-21T07:00:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.415+0000 7f636f400640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d59250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.415+0000 7f636fe00640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d56ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.416+0000 7f636ea00640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d56790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6360009800 tx=0x7f6360032b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c11623ce283d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.417+0000 7f6359600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.417+0000 7f6359600640 1 -- 172.21.15.114:0/2617176441 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360035070 con 0x55e1e697f4d0 2024-09-21T07:00:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.417+0000 7f6359600640 1 -- 172.21.15.114:0/2617176441 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f63600037f0 con 0x55e1e697f4d0 2024-09-21T07:00:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.420+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1e6d52d60 con 0x55e1e697f4d0 2024-09-21T07:00:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.421+0000 7f6359600640 1 -- 172.21.15.114:0/2617176441 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f636003f040 con 0x55e1e697f4d0 2024-09-21T07:00:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.421+0000 7f637d5e5740 10 client.27332 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.421+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e1e6dcd410 con 0x55e1e697f4d0 2024-09-21T07:00:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.422+0000 7f6359600640 1 -- 172.21.15.114:0/2617176441 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e1e6d52d60 con 0x55e1e697f4d0 2024-09-21T07:00:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.422+0000 7f6359600640 1 -- 172.21.15.114:0/2617176441 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f636005e030 con 0x55e1e697f4d0 2024-09-21T07:00:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f637d5e5740 10 client.27332 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f637d5e5740 10 client.27332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f637d5e5740 20 client.27332 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e69341c0 0x55e1e6dcf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f637d5e5740 4 client.27332 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:00:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1e6cbc600 con 0x55e1e69341c0 2024-09-21T07:00:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.425+0000 7f636f400640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e69341c0 0x55e1e6dcf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.426+0000 7f636f400640 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e69341c0 0x55e1e6dcf6a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42b69eacbc3ab86 server_cookie=711a61313160365 in_seq=0 out_seq=0 2024-09-21T07:00:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.426+0000 7f6359600640 10 client.27332 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.464+0000 7f6359600640 1 -- 172.21.15.114:0/2617176441 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1e6cbc600 con 0x55e1e69341c0 2024-09-21T07:00:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.464+0000 7f6359600640 10 client.27332 handle_command_reply: tid=0 2024-09-21T07:00:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.464+0000 7f637d5e5740 1 client.27332 shutdown 2024-09-21T07:00:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 msgr2=0x55e1e6d56790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d56790 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6360009800 tx=0x7f6360032b80 comp rx=0 tx=0).stop 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 shutdown_connections 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e1e69341c0 0x55e1e6dcf6a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1e69a01b0 0x55e1e6d56ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1e697f4d0 0x55e1e6d56790 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6360009800 tx=0x7f6360032b80 comp rx=0 tx=0).stop 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 --2- 172.21.15.114:0/2617176441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e1e69772f0 0x55e1e6d59250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 >> 172.21.15.114:0/2617176441 conn(0x55e1e6994300 msgr2=0x55e1e6d4d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f6359600640 0 client.27332 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.467+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 shutdown_connections 2024-09-21T07:00:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.468+0000 7f637d5e5740 1 -- 172.21.15.114:0/2617176441 wait complete. 2024-09-21T07:00:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.468+0000 7f637d5e5740 20 client.27332 trim_cache size 0 max 16384 2024-09-21T07:00:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:00:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:00:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:00:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:00:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.470+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63540b6840 msgr2=0x7f63540b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.470+0000 7f6374e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63540b6840 0x7f63540b8d50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6368005c40 tx=0x7f6368007620 comp rx=0 tx=0).stop 2024-09-21T07:00:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.470+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6370116510 msgr2=0x7f637011b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.470+0000 7f6374e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6370116510 0x7f637011b1f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f635c00c710 tx=0x7f635c00cbd0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 shutdown_connections 2024-09-21T07:00:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6370116510 0x7f637011b1f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63540b6840 0x7f63540b8d50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f637010ad10 0x7f637011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 --2- 172.21.15.114:0/486663010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f637010a3c0 0x7f6370121900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 >> 172.21.15.114:0/486663010 conn(0x7f6370070c00 msgr2=0x7f6370071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 shutdown_connections 2024-09-21T07:00:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:46.472+0000 7f6374e00640 1 -- 172.21.15.114:0/486663010 wait complete. 2024-09-21T07:00:47.083 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:47.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:46 smithi193 ceph-mon[34601]: pgmap v595: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-09-21T07:00:47.257 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:46 smithi114 ceph-mon[30654]: pgmap v595: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-09-21T07:00:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:46 smithi183 ceph-mon[34219]: pgmap v595: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.3 MiB/s wr, 149 op/s 2024-09-21T07:00:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 370.0 s, 664.2 tps, lat 148.885 ms stddev 204.583 2024-09-21T07:00:48.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 -- 172.21.15.114:0/3224519334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d8105d80 msgr2=0x7fd7d8106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 --2- 172.21.15.114:0/3224519334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d8105d80 0x7fd7d8106190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009900 tx=0x7fd7cc02a400 comp rx=0 tx=0).stop 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 -- 172.21.15.114:0/3224519334 shutdown_connections 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 --2- 172.21.15.114:0/3224519334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d810c010 0x7fd7d8112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 --2- 172.21.15.114:0/3224519334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7d810b6c0 0x7fd7d810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 --2- 172.21.15.114:0/3224519334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d8105d80 0x7fd7d8106190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.092+0000 7fd7df400640 1 -- 172.21.15.114:0/3224519334 >> 172.21.15.114:0/3224519334 conn(0x7fd7d8101810 msgr2=0x7fd7d8103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.093+0000 7fd7df400640 1 -- 172.21.15.114:0/3224519334 shutdown_connections 2024-09-21T07:00:48.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.093+0000 7fd7df400640 1 -- 172.21.15.114:0/3224519334 wait complete. 2024-09-21T07:00:48.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.095+0000 7fd7df400640 1 Processor -- start 2024-09-21T07:00:48.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.095+0000 7fd7df400640 1 -- start start 2024-09-21T07:00:48.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7df400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d8105d80 0x7fd7d819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d810b6c0 0x7fd7d819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7df400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7d810c010 0x7fd7d81a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7df400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8115d20 con 0x7fd7d810c010 2024-09-21T07:00:48.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7df400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8115ba0 con 0x7fd7d8105d80 2024-09-21T07:00:48.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7df400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8115ea0 con 0x7fd7d810b6c0 2024-09-21T07:00:48.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d8105d80 0x7fd7d819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d8105d80 0x7fd7d819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40848/0 (socket says 172.21.15.114:40848) 2024-09-21T07:00:48.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7dca00640 1 -- 172.21.15.114:0/2580199049 learned_addr learned my addr 172.21.15.114:0/2580199049 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:48.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.096+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d810b6c0 0x7fd7d819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.097+0000 7fd7d7e00640 1 -- 172.21.15.114:0/2580199049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d8105d80 msgr2=0x7fd7d819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.097+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d8105d80 0x7fd7d819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.097+0000 7fd7d7e00640 1 -- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7d810c010 msgr2=0x7fd7d81a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.097+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7d810c010 0x7fd7d81a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.097+0000 7fd7d7e00640 1 -- 172.21.15.114:0/2580199049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d81a2620 con 0x7fd7d810b6c0 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.097+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d810b6c0 0x7fd7d819cc50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005fd0 tx=0x7fd7c400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.098+0000 7fd7d5600640 1 -- 172.21.15.114:0/2580199049 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4009c00 con 0x7fd7d810b6c0 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.098+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d81a2910 con 0x7fd7d810b6c0 2024-09-21T07:00:48.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.098+0000 7fd7d5600640 1 -- 172.21.15.114:0/2580199049 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c4009da0 con 0x7fd7d810b6c0 2024-09-21T07:00:48.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.098+0000 7fd7d5600640 1 -- 172.21.15.114:0/2580199049 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4003db0 con 0x7fd7d810b6c0 2024-09-21T07:00:48.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.098+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81a9070 con 0x7fd7d810b6c0 2024-09-21T07:00:48.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.105+0000 7fd7d5600640 1 -- 172.21.15.114:0/2580199049 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd7c4004030 con 0x7fd7d810b6c0 2024-09-21T07:00:48.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.108+0000 7fd7e6079740 1 Processor -- start 2024-09-21T07:00:48.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.108+0000 7fd7d5600640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7c00b6700 0x7fd7c00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.108+0000 7fd7d5600640 1 -- 172.21.15.114:0/2580199049 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd7c40e7530 con 0x7fd7d810b6c0 2024-09-21T07:00:48.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.108+0000 7fd7dd400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7c00b6700 0x7fd7c00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.110+0000 7fd7e6079740 1 -- start start 2024-09-21T07:00:48.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.110+0000 7fd7dd400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7c00b6700 0x7fd7c00b8c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00092f0 tx=0x7fd7d0009220 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:48.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.111+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf9a070 0x55d7ed340da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.111+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed3412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.111+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 0x55d7ed341820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.112+0000 7fd7e6079740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d7ed2b7fb0 con 0x55d7ecf9a070 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.112+0000 7fd7e6079740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7ed2a73e0 con 0x55d7ecf79390 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.112+0000 7fd7e6079740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7ed2ab1f0 con 0x55d7ecf711b0 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.112+0000 7fd7dd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 0x55d7ed341820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.112+0000 7fd7dd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 0x55d7ed341820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40864/0 (socket says 172.21.15.114:40864) 2024-09-21T07:00:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.112+0000 7fd7dd400640 1 -- 172.21.15.114:0/2122143367 learned_addr learned my addr 172.21.15.114:0/2122143367 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.113+0000 7fd7dd400640 1 -- 172.21.15.114:0/2122143367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 msgr2=0x55d7ed3412e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.113+0000 7fd7dd400640 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed3412e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.113+0000 7fd7dd400640 1 -- 172.21.15.114:0/2122143367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf9a070 msgr2=0x55d7ed340da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.113+0000 7fd7dd400640 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf9a070 0x55d7ed340da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.113+0000 7fd7dd400640 1 -- 172.21.15.114:0/2122143367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ed271cd0 con 0x55d7ecf79390 2024-09-21T07:00:48.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.115+0000 7fd7dd400640 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 0x55d7ed341820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009fb0 tx=0x7fd7d0011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdece0003d1c23b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:48.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.116+0000 7fd7c9c00640 1 -- 172.21.15.114:0/2122143367 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d001e640 con 0x55d7ecf79390 2024-09-21T07:00:48.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.116+0000 7fd7c9c00640 1 -- 172.21.15.114:0/2122143367 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d0023070 con 0x55d7ecf79390 2024-09-21T07:00:48.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 -- 172.21.15.114:0/2122143367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 msgr2=0x55d7ed341820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 0x55d7ed341820 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009fb0 tx=0x7fd7d0011000 comp rx=0 tx=0).stop 2024-09-21T07:00:48.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 -- 172.21.15.114:0/2122143367 shutdown_connections 2024-09-21T07:00:48.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf79390 0x55d7ed341820 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed3412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2122143367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf9a070 0x55d7ed340da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.118+0000 7fd7e6079740 1 -- 172.21.15.114:0/2122143367 >> 172.21.15.114:0/2122143367 conn(0x55d7ecf8e460 msgr2=0x55d7ed33ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.119+0000 7fd7e6079740 1 -- 172.21.15.114:0/2122143367 shutdown_connections 2024-09-21T07:00:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.119+0000 7fd7e6079740 1 -- 172.21.15.114:0/2122143367 wait complete. 2024-09-21T07:00:48.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.120+0000 7fd7e6079740 1 -- start start 2024-09-21T07:00:48.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.120+0000 7fd7e6079740 1 Processor -- start 2024-09-21T07:00:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 0x55d7ed34c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed34ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf9a070 0x55d7ed3610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 0x55d7ed34c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.124 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 0x55d7ed34c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40868/0 (socket says 172.21.15.114:40868) 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7dca00640 1 -- 172.21.15.114:0/2790164694 learned_addr learned my addr 172.21.15.114:0/2790164694 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d7ed108700 con 0x55d7ecf79390 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7ed242580 con 0x55d7ecf711b0 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7ed357d80 con 0x55d7ecf9a070 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.122+0000 7fd7dd400640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf9a070 0x55d7ed3610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7dca00640 1 -- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf9a070 msgr2=0x55d7ed3610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7dca00640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf9a070 0x55d7ed3610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed34ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7e6079740 10 client.0 _ll_register_callbacks cb 0x55d7ed252470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:48.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7dca00640 1 -- 172.21.15.114:0/2790164694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 msgr2=0x55d7ed34ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7dca00640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed34ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7dca00640 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ed005650 con 0x55d7ecf711b0 2024-09-21T07:00:48.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.123+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7ed244b90 con 0x55d7ecf711b0 2024-09-21T07:00:48.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.124+0000 7fd7be000640 10 client.27344 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:48.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.124+0000 7fd7dca00640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 0x55d7ed34c540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006010 tx=0x7fd7cc004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46c8d5e93bbf542 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:48.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.127+0000 7fd7be000640 1 -- 172.21.15.114:0/2790164694 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc040070 con 0x55d7ecf711b0 2024-09-21T07:00:48.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.127+0000 7fd7be000640 1 -- 172.21.15.114:0/2790164694 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd7cc003ac0 con 0x55d7ecf711b0 2024-09-21T07:00:48.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.127+0000 7fd7be000640 1 -- 172.21.15.114:0/2790164694 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc035040 con 0x55d7ecf711b0 2024-09-21T07:00:48.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.127+0000 7fd7e6079740 10 client.27344 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:48.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.128+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55d7ed25f950 con 0x55d7ecf711b0 2024-09-21T07:00:48.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.129+0000 7fd7be000640 1 -- 172.21.15.114:0/2790164694 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7fd7cc0314a0 con 0x55d7ecf711b0 2024-09-21T07:00:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.129+0000 7fd7e6079740 10 client.27344 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.130+0000 7fd7e6079740 10 client.27344 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:00:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.130+0000 7fd7e6079740 20 client.27344 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.130+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf589f0 0x55d7ed366c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.130+0000 7fd7e6079740 4 client.27344 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:48.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.130+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7ed2b9f00 con 0x55d7ecf589f0 2024-09-21T07:00:48.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.135+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf589f0 0x55d7ed366c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.140+0000 7fd7be000640 10 client.27344 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:00:48.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.140+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf589f0 0x55d7ed366c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d81d35436ad1ce9 server_cookie=22e57e6c06087650 in_seq=0 out_seq=0 2024-09-21T07:00:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.141+0000 7fd7be000640 1 -- 172.21.15.114:0/2790164694 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7ed2b9f00 con 0x55d7ecf589f0 2024-09-21T07:00:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.141+0000 7fd7be000640 10 client.27344 handle_command_reply: tid=0 2024-09-21T07:00:48.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 client.27344 shutdown 2024-09-21T07:00:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 msgr2=0x55d7ed34c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 0x55d7ed34c540 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006010 tx=0x7fd7cc004480 comp rx=0 tx=0).stop 2024-09-21T07:00:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 shutdown_connections 2024-09-21T07:00:48.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf589f0 0x55d7ed366c60 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf9a070 0x55d7ed3610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed34ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2790164694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf711b0 0x55d7ed34c540 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7be000640 0 client.27344 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:00:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.142+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 >> 172.21.15.114:0/2790164694 conn(0x55d7ecf8e460 msgr2=0x55d7ed343f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:48.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.143+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 shutdown_connections 2024-09-21T07:00:48.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.146+0000 7fd7e6079740 1 -- 172.21.15.114:0/2790164694 wait complete. 2024-09-21T07:00:48.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.147+0000 7fd7e6079740 20 client.27344 trim_cache size 0 max 16384 2024-09-21T07:00:48.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.178+0000 7fd7e6079740 1 Processor -- start 2024-09-21T07:00:48.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.178+0000 7fd7e6079740 1 -- start start 2024-09-21T07:00:48.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.179+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 0x55d7ed34e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.179+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf79390 0x55d7ed349730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.179+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed349c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.179+0000 7fd7e6079740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81aed90 con 0x55d7ecf711b0 2024-09-21T07:00:48.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.180+0000 7fd7e6079740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b0960 con 0x55d7ecf9a070 2024-09-21T07:00:48.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.180+0000 7fd7e6079740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81af460 con 0x55d7ecf79390 2024-09-21T07:00:48.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.180+0000 7fd7dca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 0x55d7ed34e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.180+0000 7fd7d7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf79390 0x55d7ed349730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.180+0000 7fd7dca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 0x55d7ed34e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56778/0 (socket says 172.21.15.114:56778) 2024-09-21T07:00:48.182 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.180+0000 7fd7dca00640 1 -- 172.21.15.114:0/2690789410 learned_addr learned my addr 172.21.15.114:0/2690789410 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:48.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.181+0000 7fd7dca00640 1 -- 172.21.15.114:0/2690789410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf79390 msgr2=0x55d7ed349730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.181+0000 7fd7dca00640 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf79390 0x55d7ed349730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.181+0000 7fd7dca00640 1 -- 172.21.15.114:0/2690789410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 msgr2=0x55d7ed349c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.181+0000 7fd7dca00640 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed349c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.183 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.181+0000 7fd7dca00640 1 -- 172.21.15.114:0/2690789410 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ed33f970 con 0x55d7ecf711b0 2024-09-21T07:00:48.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.183+0000 7fd7dca00640 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 0x55d7ed34e170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc031c10 tx=0x7fd7cc038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d791988bd1292a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:48.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.186+0000 7fd7c9200640 1 -- 172.21.15.114:0/2690789410 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc005d50 con 0x55d7ecf711b0 2024-09-21T07:00:48.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.186+0000 7fd7c9200640 1 -- 172.21.15.114:0/2690789410 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7ed244b90 con 0x55d7ecf711b0 2024-09-21T07:00:48.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.186+0000 7fd7c9200640 1 -- 172.21.15.114:0/2690789410 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc04ea50 con 0x55d7ecf711b0 2024-09-21T07:00:48.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 -- 172.21.15.114:0/2690789410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 msgr2=0x55d7ed34e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 0x55d7ed34e170 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc031c10 tx=0x7fd7cc038000 comp rx=0 tx=0).stop 2024-09-21T07:00:48.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 -- 172.21.15.114:0/2690789410 shutdown_connections 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed349c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf79390 0x55d7ed349730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 --2- 172.21.15.114:0/2690789410 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf711b0 0x55d7ed34e170 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 -- 172.21.15.114:0/2690789410 >> 172.21.15.114:0/2690789410 conn(0x55d7ecf8e460 msgr2=0x55d7ed34da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 -- 172.21.15.114:0/2690789410 shutdown_connections 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.188+0000 7fd7e6079740 1 -- 172.21.15.114:0/2690789410 wait complete. 2024-09-21T07:00:48.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.189+0000 7fd7e6079740 1 -- start start 2024-09-21T07:00:48.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.189+0000 7fd7e6079740 1 Processor -- start 2024-09-21T07:00:48.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.191+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed359920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.191+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed359e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.191+0000 7fd7e6079740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed357470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7e6079740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b0260 con 0x55d7ecf79390 2024-09-21T07:00:48.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7e6079740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81afb30 con 0x55d7ecf9a070 2024-09-21T07:00:48.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7e6079740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81acd70 con 0x55d7ecf711b0 2024-09-21T07:00:48.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7e6079740 10 client.0 _ll_register_callbacks cb 0x7fd7cc004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:48.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed359e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed359e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56786/0 (socket says 172.21.15.114:56786) 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.192+0000 7fd7d7e00640 1 -- 172.21.15.114:0/3941099358 learned_addr learned my addr 172.21.15.114:0/3941099358 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.193+0000 7fd7dca00640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed359920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7d7e00640 1 -- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 msgr2=0x55d7ed359920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed359920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7dd400640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed357470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7d7e00640 1 -- 172.21.15.114:0/3941099358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 msgr2=0x55d7ed357470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed357470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7d7e00640 1 -- 172.21.15.114:0/3941099358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ed350360 con 0x55d7ecf79390 2024-09-21T07:00:48.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.194+0000 7fd7dd400640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:48.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.195+0000 7fd7d7e00640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed359e80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40acfa0 tx=0x7fd7c40acde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73448d5de629b5a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:48.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.195+0000 7fd7be000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:48.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.195+0000 7fd7be000640 1 -- 172.21.15.114:0/3941099358 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4020040 con 0x55d7ecf79390 2024-09-21T07:00:48.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.195+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7ed244b90 con 0x55d7ecf79390 2024-09-21T07:00:48.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.196+0000 7fd7be000640 1 -- 172.21.15.114:0/3941099358 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fd7c40a9280 con 0x55d7ecf79390 2024-09-21T07:00:48.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.196+0000 7fd7be000640 1 -- 172.21.15.114:0/3941099358 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd7c40b1af0 con 0x55d7ecf79390 2024-09-21T07:00:48.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.196+0000 7fd7e6079740 10 client.16998 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:48.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.196+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d7ed3da800 con 0x55d7ecf79390 2024-09-21T07:00:48.199 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.198+0000 7fd7be000640 1 -- 172.21.15.114:0/3941099358 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d7ed244b90 con 0x55d7ecf79390 2024-09-21T07:00:48.200 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7be000640 1 -- 172.21.15.114:0/3941099358 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7fd7c40a8960 con 0x55d7ecf79390 2024-09-21T07:00:48.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7e6079740 10 client.16998 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:48.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7e6079740 10 client.16998 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:00:48.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7e6079740 20 client.16998 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:48.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7e6079740 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf2dda0 0x55d7ed3dcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:48.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7e6079740 4 client.16998 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:00:48.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.199+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7ed3480f0 con 0x55d7ecf2dda0 2024-09-21T07:00:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.200+0000 7fd7dca00640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf2dda0 0x55d7ed3dcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.200+0000 7fd7dca00640 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf2dda0 0x55d7ed3dcf30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42df06acf4e3c033 server_cookie=9fc2986c42420448 in_seq=0 out_seq=0 2024-09-21T07:00:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.200+0000 7fd7be000640 10 client.16998 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:00:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7be000640 1 -- 172.21.15.114:0/3941099358 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7ed3480f0 con 0x55d7ecf2dda0 2024-09-21T07:00:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7be000640 10 client.16998 handle_command_reply: tid=0 2024-09-21T07:00:48.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 client.16998 shutdown 2024-09-21T07:00:48.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 msgr2=0x55d7ed359e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed359e80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40acfa0 tx=0x7fd7c40acde0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 shutdown_connections 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55d7ecf2dda0 0x55d7ed3dcf30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7ecf9a070 0x55d7ed357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d7ecf79390 0x55d7ed359e80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 --2- 172.21.15.114:0/3941099358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7ecf711b0 0x55d7ed359920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.201+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 >> 172.21.15.114:0/3941099358 conn(0x55d7ecf8e460 msgr2=0x55d7ed344630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.202+0000 7fd7be000640 0 client.16998 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.202+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 shutdown_connections 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.202+0000 7fd7e6079740 1 -- 172.21.15.114:0/3941099358 wait complete. 2024-09-21T07:00:48.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.202+0000 7fd7e6079740 20 client.16998 trim_cache size 0 max 16384 2024-09-21T07:00:48.205 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:00:48.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.203+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7c00b6700 msgr2=0x7fd7c00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.203+0000 7fd7df400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7c00b6700 0x7fd7c00b8c10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00092f0 tx=0x7fd7d0009220 comp rx=0 tx=0).stop 2024-09-21T07:00:48.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.203+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d810b6c0 msgr2=0x7fd7d819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:48.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.203+0000 7fd7df400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d810b6c0 0x7fd7d819cc50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4005fd0 tx=0x7fd7c400dde0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 shutdown_connections 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd7d810c010 0x7fd7d81a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7d810b6c0 0x7fd7d819cc50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7c00b6700 0x7fd7c00b8c10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 --2- 172.21.15.114:0/2580199049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7d8105d80 0x7fd7d819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 >> 172.21.15.114:0/2580199049 conn(0x7fd7d8101810 msgr2=0x7fd7d810ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:48.206 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 shutdown_connections 2024-09-21T07:00:48.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:48.204+0000 7fd7df400640 1 -- 172.21.15.114:0/2580199049 wait complete. 2024-09-21T07:00:49.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:48 smithi193 ceph-mon[34601]: pgmap v596: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 317 op/s 2024-09-21T07:00:49.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:48 smithi114 ceph-mon[30654]: pgmap v596: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 317 op/s 2024-09-21T07:00:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:48 smithi183 ceph-mon[34219]: pgmap v596: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 6.2 MiB/s wr, 317 op/s 2024-09-21T07:00:49.429 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:00:50.135 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:51.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:50 smithi193 ceph-mon[34601]: pgmap v597: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.9 MiB/s wr, 318 op/s 2024-09-21T07:00:51.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 -- 172.21.15.114:0/3201801316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4910111f40 msgr2=0x7f491010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 --2- 172.21.15.114:0/3201801316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4910111f40 0x7f491010a3c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4904009900 tx=0x7f490402a400 comp rx=0 tx=0).stop 2024-09-21T07:00:51.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 -- 172.21.15.114:0/3201801316 shutdown_connections 2024-09-21T07:00:51.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 --2- 172.21.15.114:0/3201801316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f491011cc30 0x7f491011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 --2- 172.21.15.114:0/3201801316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491010a9e0 0x7f491010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 --2- 172.21.15.114:0/3201801316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4910111f40 0x7f491010a3c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.254+0000 7f4917800640 1 -- 172.21.15.114:0/3201801316 >> 172.21.15.114:0/3201801316 conn(0x7f4910070ec0 msgr2=0x7f49100712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:51.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.255+0000 7f4917800640 1 -- 172.21.15.114:0/3201801316 shutdown_connections 2024-09-21T07:00:51.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.255+0000 7f4917800640 1 -- 172.21.15.114:0/3201801316 wait complete. 2024-09-21T07:00:51.257 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:50 smithi114 ceph-mon[30654]: pgmap v597: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.9 MiB/s wr, 318 op/s 2024-09-21T07:00:51.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.257+0000 7f4917800640 1 Processor -- start 2024-09-21T07:00:51.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.257+0000 7f4917800640 1 -- start start 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4917800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 0x7f49101a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4917800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4910111f40 0x7f49101a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4917800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491011cc30 0x7f49101aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4917800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4910122900 con 0x7f4910111f40 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4917800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4910122780 con 0x7f491010a9e0 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4917800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4910122a80 con 0x7f491011cc30 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4914e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 0x7f49101a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f490fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4910111f40 0x7f49101a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.260 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4914e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 0x7f49101a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40906/0 (socket says 172.21.15.114:40906) 2024-09-21T07:00:51.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f490fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4910111f40 0x7f49101a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56812/0 (socket says 172.21.15.114:56812) 2024-09-21T07:00:51.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.258+0000 7f4914e00640 1 -- 172.21.15.114:0/3059322768 learned_addr learned my addr 172.21.15.114:0/3059322768 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:51.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.259+0000 7f4914e00640 1 -- 172.21.15.114:0/3059322768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491011cc30 msgr2=0x7f49101aeb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:00:51.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.259+0000 7f4914e00640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491011cc30 0x7f49101aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.259+0000 7f4914e00640 1 -- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4910111f40 msgr2=0x7f49101a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.261 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.259+0000 7f4914e00640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4910111f40 0x7f49101a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.259+0000 7f4914e00640 1 -- 172.21.15.114:0/3059322768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49101af240 con 0x7f491010a9e0 2024-09-21T07:00:51.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.260+0000 7f4914e00640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 0x7f49101a92d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4904005ab0 tx=0x7f4904003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:51.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.260+0000 7f490d600640 1 -- 172.21.15.114:0/3059322768 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490403f070 con 0x7f491010a9e0 2024-09-21T07:00:51.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.261+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101af4d0 con 0x7f491010a9e0 2024-09-21T07:00:51.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.261+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101b5c90 con 0x7f491010a9e0 2024-09-21T07:00:51.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.261+0000 7f490d600640 1 -- 172.21.15.114:0/3059322768 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4904031b60 con 0x7f491010a9e0 2024-09-21T07:00:51.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.261+0000 7f490d600640 1 -- 172.21.15.114:0/3059322768 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904035030 con 0x7f491010a9e0 2024-09-21T07:00:51.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.266+0000 7f491e3c2740 1 Processor -- start 2024-09-21T07:00:51.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.267+0000 7f491e3c2740 1 -- start start 2024-09-21T07:00:51.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 0x5615f995ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f958b1b0 0x5615f995b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f491e3c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5615f98d1fb0 con 0x5615f9593390 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f491e3c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615f98c13e0 con 0x5615f95b4070 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f491e3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5615f98c51f0 con 0x5615f958b1b0 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 0x5615f995ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f490fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 0x5615f995ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40922/0 (socket says 172.21.15.114:40922) 2024-09-21T07:00:51.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.268+0000 7f4915800640 1 -- 172.21.15.114:0/921638403 learned_addr learned my addr 172.21.15.114:0/921638403 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.269+0000 7f4915800640 1 -- 172.21.15.114:0/921638403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f958b1b0 msgr2=0x5615f995b2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.269+0000 7f4915800640 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f958b1b0 0x5615f995b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.269+0000 7f4915800640 1 -- 172.21.15.114:0/921638403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 msgr2=0x5615f995b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.269+0000 7f4915800640 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.269+0000 7f4915800640 1 -- 172.21.15.114:0/921638403 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f988bcd0 con 0x5615f95b4070 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.270+0000 7f4915800640 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 0x5615f995ada0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4908005c40 tx=0x7f4908009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7debb12681c9b209 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:51.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.270+0000 7f490d600640 1 -- 172.21.15.114:0/3059322768 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4904042070 con 0x7f491010a9e0 2024-09-21T07:00:51.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.272+0000 7f4902000640 1 -- 172.21.15.114:0/921638403 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908003b70 con 0x5615f95b4070 2024-09-21T07:00:51.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.272+0000 7f4902000640 1 -- 172.21.15.114:0/921638403 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f490800ee60 con 0x5615f95b4070 2024-09-21T07:00:51.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.272+0000 7f4902000640 1 -- 172.21.15.114:0/921638403 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908010850 con 0x5615f95b4070 2024-09-21T07:00:51.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.273+0000 7f490d600640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48f80b6780 0x7f48f80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.273+0000 7f490d600640 1 -- 172.21.15.114:0/3059322768 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f49041080b0 con 0x7f491010a9e0 2024-09-21T07:00:51.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.273+0000 7f4914e00640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48f80b6780 0x7f48f80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 -- 172.21.15.114:0/921638403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 msgr2=0x5615f995ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 0x5615f995ada0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4908005c40 tx=0x7f4908009f90 comp rx=0 tx=0).stop 2024-09-21T07:00:51.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 -- 172.21.15.114:0/921638403 shutdown_connections 2024-09-21T07:00:51.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f958b1b0 0x5615f995b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 --2- 172.21.15.114:0/921638403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f95b4070 0x5615f995ada0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.274+0000 7f491e3c2740 1 -- 172.21.15.114:0/921638403 >> 172.21.15.114:0/921638403 conn(0x5615f95a8460 msgr2=0x5615f9958c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:51.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.275+0000 7f491e3c2740 1 -- 172.21.15.114:0/921638403 shutdown_connections 2024-09-21T07:00:51.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.275+0000 7f491e3c2740 1 -- 172.21.15.114:0/921638403 wait complete. 2024-09-21T07:00:51.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.276+0000 7f4914e00640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48f80b6780 0x7f48f80b8c90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5615f9959a20 tx=0x7f4904009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:51.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.276+0000 7f491e3c2740 1 -- start start 2024-09-21T07:00:51.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.276+0000 7f491e3c2740 1 Processor -- start 2024-09-21T07:00:51.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f9966b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f9972950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5615f9722700 con 0x5615f9593390 2024-09-21T07:00:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615f985c580 con 0x5615f958b1b0 2024-09-21T07:00:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5615f99695d0 con 0x5615f95b4070 2024-09-21T07:00:51.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.277+0000 7f491e3c2740 10 client.0 _ll_register_callbacks cb 0x5615f986c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:51.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.278+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.278+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40936/0 (socket says 172.21.15.114:40936) 2024-09-21T07:00:51.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.278+0000 7f4915800640 1 -- 172.21.15.114:0/1911427222 learned_addr learned my addr 172.21.15.114:0/1911427222 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:51.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.278+0000 7f4915800640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f9972950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.279+0000 7f4915800640 1 -- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 msgr2=0x5615f9972950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.279+0000 7f490fe00640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f9966b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.279+0000 7f4915800640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f9972950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.279+0000 7f4915800640 1 -- 172.21.15.114:0/1911427222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 msgr2=0x5615f9966b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.279+0000 7f4915800640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f9966b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.280+0000 7f4915800640 1 -- 172.21.15.114:0/1911427222 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f961f650 con 0x5615f958b1b0 2024-09-21T07:00:51.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.280+0000 7f490fe00640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f9966b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:51.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.280+0000 7f4915800640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f490800efe0 tx=0x7f4908010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f1fcc6804e6828 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:51.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.280+0000 7f48f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:51.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.281+0000 7f48f6a00640 1 -- 172.21.15.114:0/1911427222 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908004600 con 0x5615f958b1b0 2024-09-21T07:00:51.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.281+0000 7f48f6a00640 1 -- 172.21.15.114:0/1911427222 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4908004b20 con 0x5615f958b1b0 2024-09-21T07:00:51.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.282+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615f985eb90 con 0x5615f958b1b0 2024-09-21T07:00:51.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.282+0000 7f48f6a00640 1 -- 172.21.15.114:0/1911427222 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f49080047a0 con 0x5615f958b1b0 2024-09-21T07:00:51.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.282+0000 7f491e3c2740 10 client.27356 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:51.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.282+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5615f9879950 con 0x5615f958b1b0 2024-09-21T07:00:51.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f48f6a00640 1 -- 172.21.15.114:0/1911427222 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5615f985eb90 con 0x5615f958b1b0 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f48f6a00640 1 -- 172.21.15.114:0/1911427222 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f49080185d0 con 0x5615f958b1b0 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f491e3c2740 10 client.27356 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f491e3c2740 10 client.27356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f491e3c2740 20 client.27356 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f491e3c2740 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f95729f0 0x5615f9978630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f491e3c2740 4 client.27356 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f98d3f00 con 0x5615f95729f0 2024-09-21T07:00:51.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.283+0000 7f490fe00640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f95729f0 0x5615f9978630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.284+0000 7f48f6a00640 10 client.27356 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:00:51.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.284+0000 7f490fe00640 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f95729f0 0x5615f9978630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e259c2bf5cb0fe3 server_cookie=26ece16677d1881 in_seq=0 out_seq=0 2024-09-21T07:00:51.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.285+0000 7f48f6a00640 1 -- 172.21.15.114:0/1911427222 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615f98d3f00 con 0x5615f95729f0 2024-09-21T07:00:51.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.285+0000 7f48f6a00640 10 client.27356 handle_command_reply: tid=0 2024-09-21T07:00:51.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.286+0000 7f491e3c2740 1 client.27356 shutdown 2024-09-21T07:00:51.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.286+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 msgr2=0x5615f9966600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.286+0000 7f491e3c2740 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f490800efe0 tx=0x7f4908010430 comp rx=0 tx=0).stop 2024-09-21T07:00:51.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 shutdown_connections 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f95729f0 0x5615f9978630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f9972950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f9966b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 --2- 172.21.15.114:0/1911427222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f48f6a00640 0 client.27356 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 >> 172.21.15.114:0/1911427222 conn(0x5615f95a8460 msgr2=0x5615f995df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 shutdown_connections 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.287+0000 7f491e3c2740 1 -- 172.21.15.114:0/1911427222 wait complete. 2024-09-21T07:00:51.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.288+0000 7f491e3c2740 20 client.27356 trim_cache size 0 max 16384 2024-09-21T07:00:51.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 Processor -- start 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 -- start start 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f995e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f48f000ff90 con 0x5615f9593390 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0011b60 con 0x5615f958b1b0 2024-09-21T07:00:51.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.309+0000 7f491e3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0010660 con 0x5615f95b4070 2024-09-21T07:00:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.310+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.310+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40946/0 (socket says 172.21.15.114:40946) 2024-09-21T07:00:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.311+0000 7f4915800640 1 -- 172.21.15.114:0/3478942323 learned_addr learned my addr 172.21.15.114:0/3478942323 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:51.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.311+0000 7f4915800640 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f995e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.313+0000 7f490fe00640 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.313+0000 7f4915800640 1 -- 172.21.15.114:0/3478942323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 msgr2=0x5615f9966ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.313+0000 7f4915800640 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.313+0000 7f4915800640 1 -- 172.21.15.114:0/3478942323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 msgr2=0x5615f995dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.313+0000 7f4915800640 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.314+0000 7f4915800640 1 -- 172.21.15.114:0/3478942323 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f995d090 con 0x5615f95b4070 2024-09-21T07:00:51.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.317+0000 7f4915800640 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f995e3e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5615f99738a0 tx=0x7f4908009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60b3eac00557d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:51.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.319+0000 7f4901600640 1 -- 172.21.15.114:0/3478942323 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908034020 con 0x5615f95b4070 2024-09-21T07:00:51.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.321+0000 7f4901600640 1 -- 172.21.15.114:0/3478942323 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908009820 con 0x5615f95b4070 2024-09-21T07:00:51.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.321+0000 7f4901600640 1 -- 172.21.15.114:0/3478942323 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908010730 con 0x5615f95b4070 2024-09-21T07:00:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.323+0000 7f491e3c2740 1 -- 172.21.15.114:0/3478942323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 msgr2=0x5615f995e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.323+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f995e3e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5615f99738a0 tx=0x7f4908009200 comp rx=0 tx=0).stop 2024-09-21T07:00:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.326+0000 7f491e3c2740 1 -- 172.21.15.114:0/3478942323 shutdown_connections 2024-09-21T07:00:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.326+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f95b4070 0x5615f995e3e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.326+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f9593390 0x5615f995dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.326+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3478942323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9966ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.326+0000 7f491e3c2740 1 -- 172.21.15.114:0/3478942323 >> 172.21.15.114:0/3478942323 conn(0x5615f95a8460 msgr2=0x5615f99663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:51.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.326+0000 7f491e3c2740 1 -- 172.21.15.114:0/3478942323 shutdown_connections 2024-09-21T07:00:51.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.328+0000 7f491e3c2740 1 -- 172.21.15.114:0/3478942323 wait complete. 2024-09-21T07:00:51.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.328+0000 7f491e3c2740 1 -- start start 2024-09-21T07:00:51.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.329+0000 7f491e3c2740 1 Processor -- start 2024-09-21T07:00:51.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.330+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9979680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.330+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f9593390 0x5615f9979bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f491e3c2740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f95b4070 0x5615f997a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f491e3c2740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0011460 con 0x5615f95b4070 2024-09-21T07:00:51.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f491e3c2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0010d30 con 0x5615f958b1b0 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f491e3c2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48f000df70 con 0x5615f9593390 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f491e3c2740 10 client.0 _ll_register_callbacks cb 0x7f49080040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9979680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f4915800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9979680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40956/0 (socket says 172.21.15.114:40956) 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f4915800640 1 -- 172.21.15.114:0/3964106784 learned_addr learned my addr 172.21.15.114:0/3964106784 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f4915800640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f95b4070 0x5615f997a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.331+0000 7f490fe00640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f9593390 0x5615f9979bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f4915800640 1 -- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f9593390 msgr2=0x5615f9979bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f4915800640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f9593390 0x5615f9979bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f4915800640 1 -- 172.21.15.114:0/3964106784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f95b4070 msgr2=0x5615f997a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f4915800640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f95b4070 0x5615f997a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f4915800640 1 -- 172.21.15.114:0/3964106784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f99e1de0 con 0x5615f958b1b0 2024-09-21T07:00:51.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f4915800640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9979680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f490800efe0 tx=0x7f4908009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baec5064f57535e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:51.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.332+0000 7f48f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:51.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.333+0000 7f48f6a00640 1 -- 172.21.15.114:0/3964106784 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908044070 con 0x5615f958b1b0 2024-09-21T07:00:51.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.333+0000 7f48f6a00640 1 -- 172.21.15.114:0/3964106784 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f4908004b20 con 0x5615f958b1b0 2024-09-21T07:00:51.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.334+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4908009820 con 0x5615f958b1b0 2024-09-21T07:00:51.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.335+0000 7f48f6a00640 1 -- 172.21.15.114:0/3964106784 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4908034070 con 0x5615f958b1b0 2024-09-21T07:00:51.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.335+0000 7f491e3c2740 10 client.27362 fetch_fsmap learned FSMap version 42 2024-09-21T07:00:51.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.335+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5615f99e3060 con 0x5615f958b1b0 2024-09-21T07:00:51.337 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.335+0000 7f48f6a00640 1 -- 172.21.15.114:0/3964106784 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4908009820 con 0x5615f958b1b0 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.336+0000 7f48f6a00640 1 -- 172.21.15.114:0/3964106784 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8382+0+0 (secure 0 0 0) 0x7f4908047030 con 0x5615f958b1b0 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f491e3c2740 10 client.27362 fetch_fsmap finished waiting for FSMap version 42 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f491e3c2740 10 client.27362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f491e3c2740 20 client.27362 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f9547da0 0x5615f99e5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f491e3c2740 4 client.27362 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615f98cf9b0 con 0x5615f9547da0 2024-09-21T07:00:51.339 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.337+0000 7f490fe00640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f9547da0 0x5615f99e5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:51.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.339+0000 7f490fe00640 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f9547da0 0x5615f99e5730 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bed59902f70d94ce server_cookie=464fe754b67a6aa5 in_seq=0 out_seq=0 2024-09-21T07:00:51.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.339+0000 7f48f6a00640 10 client.27362 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:00:51.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.340+0000 7f48f6a00640 1 -- 172.21.15.114:0/3964106784 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615f98cf9b0 con 0x5615f9547da0 2024-09-21T07:00:51.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.340+0000 7f48f6a00640 10 client.27362 handle_command_reply: tid=0 2024-09-21T07:00:51.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.340+0000 7f491e3c2740 1 client.27362 shutdown 2024-09-21T07:00:51.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.342+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 msgr2=0x5615f9979680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9979680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f490800efe0 tx=0x7f4908009f90 comp rx=0 tx=0).stop 2024-09-21T07:00:51.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 shutdown_connections 2024-09-21T07:00:51.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5615f9547da0 0x5615f99e5730 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5615f95b4070 0x5615f997a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f48f6a00640 0 client.27362 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5615f9593390 0x5615f9979bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 --2- 172.21.15.114:0/3964106784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f958b1b0 0x5615f9979680 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.344+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 >> 172.21.15.114:0/3964106784 conn(0x5615f95a8460 msgr2=0x5615f995dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.345+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 shutdown_connections 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.345+0000 7f491e3c2740 1 -- 172.21.15.114:0/3964106784 wait complete. 2024-09-21T07:00:51.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.345+0000 7f491e3c2740 20 client.27362 trim_cache size 0 max 16384 2024-09-21T07:00:51.349 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48f80b6780 msgr2=0x7f48f80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48f80b6780 0x7f48f80b8c90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x5615f9959a20 tx=0x7f4904009260 comp rx=0 tx=0).stop 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 msgr2=0x7f49101a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 0x7f49101a92d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4904005ab0 tx=0x7f4904003c10 comp rx=0 tx=0).stop 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 shutdown_connections 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f491011cc30 0x7f49101aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4910111f40 0x7f49101a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48f80b6780 0x7f48f80b8c90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 --2- 172.21.15.114:0/3059322768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f491010a9e0 0x7f49101a92d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.350+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 >> 172.21.15.114:0/3059322768 conn(0x7f4910070ec0 msgr2=0x7f491011d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:51.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.351+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 shutdown_connections 2024-09-21T07:00:51.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:51.351+0000 7f4917800640 1 -- 172.21.15.114:0/3059322768 wait complete. 2024-09-21T07:00:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:50 smithi183 ceph-mon[34219]: pgmap v597: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.9 MiB/s wr, 318 op/s 2024-09-21T07:00:52.276 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:00:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 375.0 s, 683.8 tps, lat 144.935 ms stddev 171.255 2024-09-21T07:00:52.935 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:53.153 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:52 smithi114 ceph-mon[30654]: pgmap v598: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.0 MiB/s wr, 328 op/s 2024-09-21T07:00:53.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:52 smithi193 ceph-mon[34601]: pgmap v598: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.0 MiB/s wr, 328 op/s 2024-09-21T07:00:53.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:52 smithi183 ceph-mon[34219]: pgmap v598: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.0 MiB/s wr, 328 op/s 2024-09-21T07:00:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 -- 172.21.15.114:0/167549476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1025b0 msgr2=0x7fc8dc1029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/167549476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1025b0 0x7fc8dc1029c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:00:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 -- 172.21.15.114:0/167549476 shutdown_connections 2024-09-21T07:00:53.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/167549476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc1079f0 0x7fc8dc107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/167549476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8dc102f00 0x7fc8dc107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/167549476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1025b0 0x7fc8dc1029c0 secure :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:00:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.979+0000 7fc8e2a00640 1 -- 172.21.15.114:0/167549476 >> 172.21.15.114:0/167549476 conn(0x7fc8dc0fe2c0 msgr2=0x7fc8dc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.980+0000 7fc8e2a00640 1 -- 172.21.15.114:0/167549476 shutdown_connections 2024-09-21T07:00:53.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.980+0000 7fc8e2a00640 1 -- 172.21.15.114:0/167549476 wait complete. 2024-09-21T07:00:53.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.982+0000 7fc8e2a00640 1 Processor -- start 2024-09-21T07:00:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.983+0000 7fc8e2a00640 1 -- start start 2024-09-21T07:00:53.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.983+0000 7fc8e2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 0x7fc8dc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.983+0000 7fc8e2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8dc1079f0 0x7fc8dc1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.984+0000 7fc8e2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1a6210 0x7fc8dc1a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.984+0000 7fc8e2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc119f60 con 0x7fc8dc102f00 2024-09-21T07:00:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.984+0000 7fc8e2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc119de0 con 0x7fc8dc1079f0 2024-09-21T07:00:53.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.984+0000 7fc8e2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc11a0e0 con 0x7fc8dc1a6210 2024-09-21T07:00:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.984+0000 7fc8e0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8dc1079f0 0x7fc8dc1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.985+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 0x7fc8dc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.985+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 0x7fc8dc1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56850/0 (socket says 172.21.15.114:56850) 2024-09-21T07:00:53.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.985+0000 7fc8e1600640 1 -- 172.21.15.114:0/564695978 learned_addr learned my addr 172.21.15.114:0/564695978 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.987+0000 7fc8e2000640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1a6210 0x7fc8dc1a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.989+0000 7fc8e1600640 1 -- 172.21.15.114:0/564695978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1a6210 msgr2=0x7fc8dc1a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.989+0000 7fc8e1600640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1a6210 0x7fc8dc1a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.989+0000 7fc8e1600640 1 -- 172.21.15.114:0/564695978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8dc1079f0 msgr2=0x7fc8dc1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.989+0000 7fc8e1600640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8dc1079f0 0x7fc8dc1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.989+0000 7fc8e1600640 1 -- 172.21.15.114:0/564695978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc1a6df0 con 0x7fc8dc102f00 2024-09-21T07:00:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.989+0000 7fc8e2000640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1a6210 0x7fc8dc1a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.991+0000 7fc8e1600640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 0x7fc8dc1a0a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0004770 tx=0x7fc8d0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.992+0000 7fc8da000640 1 -- 172.21.15.114:0/564695978 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d003f020 con 0x7fc8dc102f00 2024-09-21T07:00:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.992+0000 7fc8da000640 1 -- 172.21.15.114:0/564695978 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d003bd20 con 0x7fc8dc102f00 2024-09-21T07:00:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.992+0000 7fc8da000640 1 -- 172.21.15.114:0/564695978 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0035070 con 0x7fc8dc102f00 2024-09-21T07:00:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.992+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc1ad390 con 0x7fc8dc102f00 2024-09-21T07:00:53.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:53.992+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1ad730 con 0x7fc8dc102f00 2024-09-21T07:00:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.003+0000 7fc8da000640 1 -- 172.21.15.114:0/564695978 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc8d0042020 con 0x7fc8dc102f00 2024-09-21T07:00:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 Processor -- start 2024-09-21T07:00:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 -- start start 2024-09-21T07:00:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab54d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563d7aacbfb0 con 0x563d7a7851b0 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d7aabb3e0 con 0x563d7a78d390 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.004+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563d7aabf1f0 con 0x563d7a7ae070 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.005+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.005+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab55270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:56856/0 (socket says 172.21.15.114:56856) 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.005+0000 7fc8e1600640 1 -- 172.21.15.114:0/1887770460 learned_addr learned my addr 172.21.15.114:0/1887770460 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.006+0000 7fc8e1600640 1 -- 172.21.15.114:0/1887770460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 msgr2=0x563d7ab54d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.006+0000 7fc8e1600640 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab54d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.006+0000 7fc8da000640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8c40b6850 0x7fc8c40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.006+0000 7fc8e1600640 1 -- 172.21.15.114:0/1887770460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 msgr2=0x563d7ab557b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:54.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.006+0000 7fc8e1600640 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab557b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.006+0000 7fc8e1600640 1 -- 172.21.15.114:0/1887770460 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7aa85cd0 con 0x563d7a7851b0 2024-09-21T07:00:54.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.007+0000 7fc8da000640 1 -- 172.21.15.114:0/564695978 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc8d0108410 con 0x7fc8dc102f00 2024-09-21T07:00:54.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.007+0000 7fc8e1600640 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab55270 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0105700 tx=0x7fc8d00ce080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b19c54705a2fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:54.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.007+0000 7fc8e2000640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8c40b6850 0x7fc8c40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.009+0000 7fc8e2000640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8c40b6850 0x7fc8c40b8d60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4007d90 tx=0x7fc8d4007320 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:54.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.008+0000 7fc8ce600640 1 -- 172.21.15.114:0/1887770460 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00cdc60 con 0x563d7a7851b0 2024-09-21T07:00:54.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.009+0000 7fc8ce600640 1 -- 172.21.15.114:0/1887770460 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d00cd9a0 con 0x563d7a7851b0 2024-09-21T07:00:54.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.009+0000 7fc8ce600640 1 -- 172.21.15.114:0/1887770460 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00cd600 con 0x563d7a7851b0 2024-09-21T07:00:54.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1887770460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 msgr2=0x563d7ab55270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:54.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab55270 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0105700 tx=0x7fc8d00ce080 comp rx=0 tx=0).stop 2024-09-21T07:00:54.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1887770460 shutdown_connections 2024-09-21T07:00:54.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab55270 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1887770460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab54d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.010+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1887770460 >> 172.21.15.114:0/1887770460 conn(0x563d7a7a2460 msgr2=0x563d7ab52bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:54.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.012+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1887770460 shutdown_connections 2024-09-21T07:00:54.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.012+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1887770460 wait complete. 2024-09-21T07:00:54.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.012+0000 7fc8eabcc740 1 -- start start 2024-09-21T07:00:54.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.013+0000 7fc8eabcc740 1 Processor -- start 2024-09-21T07:00:54.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab71a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7ae070 0x563d7ab77470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563d7a91c700 con 0x563d7a7ae070 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d7aa56580 con 0x563d7a78d390 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563d7ab74230 con 0x563d7a7851b0 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.015+0000 7fc8eabcc740 10 client.0 _ll_register_callbacks cb 0x563d7aa66470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7ae070 0x563d7ab77470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab71fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40976/0 (socket says 172.21.15.114:40976) 2024-09-21T07:00:54.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 -- 172.21.15.114:0/744978286 learned_addr learned my addr 172.21.15.114:0/744978286 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:54.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 -- 172.21.15.114:0/744978286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 msgr2=0x563d7ab71a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:54.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab71a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 -- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7ae070 msgr2=0x563d7ab77470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:54.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.016+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7ae070 0x563d7ab77470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.017+0000 7fc8e0c00640 1 -- 172.21.15.114:0/744978286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7a819650 con 0x563d7a78d390 2024-09-21T07:00:54.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.018+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab71fe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800be00 tx=0x7fc8c80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae5ce62ff0c0783 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8c2a00640 1 -- 172.21.15.114:0/744978286 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8011850 con 0x563d7a78d390 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8c2a00640 1 -- 172.21.15.114:0/744978286 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc8c80119f0 con 0x563d7a78d390 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d7aa58b90 con 0x563d7a78d390 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8c2a00640 1 -- 172.21.15.114:0/744978286 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc8c8011bf0 con 0x563d7a78d390 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8eabcc740 10 client.27371 fetch_fsmap learned FSMap version 43 2024-09-21T07:00:54.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.019+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563d7aa73950 con 0x563d7a78d390 2024-09-21T07:00:54.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.020+0000 7fc8c2a00640 1 -- 172.21.15.114:0/744978286 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d7aa58b90 con 0x563d7a78d390 2024-09-21T07:00:54.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.020+0000 7fc8c2a00640 1 -- 172.21.15.114:0/744978286 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc8c8026420 con 0x563d7a78d390 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.020+0000 7fc8eabcc740 10 client.27371 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.020+0000 7fc8eabcc740 10 client.27371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.020+0000 7fc8eabcc740 20 client.27371 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.020+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a76c9f0 0x563d7ab836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.021+0000 7fc8eabcc740 4 client.27371 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.021+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d7aacdf00 con 0x563d7a76c9f0 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.022+0000 7fc8e1600640 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a76c9f0 0x563d7ab836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.023+0000 7fc8e1600640 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a76c9f0 0x563d7ab836f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1848d08047f95c6 server_cookie=ddf9d0a1fffca902 in_seq=0 out_seq=0 2024-09-21T07:00:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.023+0000 7fc8c2a00640 10 client.27371 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:00:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.025+0000 7fc8c2a00640 1 -- 172.21.15.114:0/744978286 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d7aacdf00 con 0x563d7a76c9f0 2024-09-21T07:00:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.025+0000 7fc8c2a00640 10 client.27371 handle_command_reply: tid=0 2024-09-21T07:00:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.025+0000 7fc8eabcc740 1 client.27371 shutdown 2024-09-21T07:00:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.025+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 msgr2=0x563d7ab71fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.025+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab71fe0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800be00 tx=0x7fc8c80107a0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 shutdown_connections 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a76c9f0 0x563d7ab836f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7ae070 0x563d7ab77470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab71fe0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/744978286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab71a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 >> 172.21.15.114:0/744978286 conn(0x563d7a7a2460 msgr2=0x563d7ab51200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8c2a00640 0 client.27371 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:00:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 shutdown_connections 2024-09-21T07:00:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 1 -- 172.21.15.114:0/744978286 wait complete. 2024-09-21T07:00:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.026+0000 7fc8eabcc740 20 client.27371 trim_cache size 0 max 16384 2024-09-21T07:00:54.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.055+0000 7fc8eabcc740 1 Processor -- start 2024-09-21T07:00:54.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 -- start start 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab6c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab7e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab6cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1af150 con 0x563d7a7851b0 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1b0cf0 con 0x563d7a78d390 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.056+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1af820 con 0x563d7a7ae070 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.057+0000 7fc8e2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab6cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.057+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab6c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.057+0000 7fc8e2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab6cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52270/0 (socket says 172.21.15.114:52270) 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.057+0000 7fc8e2000640 1 -- 172.21.15.114:0/3967103334 learned_addr learned my addr 172.21.15.114:0/3967103334 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:54.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.057+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab7e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.058+0000 7fc8e2000640 1 -- 172.21.15.114:0/3967103334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 msgr2=0x563d7ab7e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.058+0000 7fc8e2000640 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab7e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.058+0000 7fc8e2000640 1 -- 172.21.15.114:0/3967103334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 msgr2=0x563d7ab6c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:54.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.058+0000 7fc8e2000640 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab6c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.058+0000 7fc8e2000640 1 -- 172.21.15.114:0/3967103334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7ab80190 con 0x563d7a7ae070 2024-09-21T07:00:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.058+0000 7fc8e2000640 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab6cea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400f790 tx=0x7fc8d400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48371831914fa37b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.059+0000 7fc8cdc00640 1 -- 172.21.15.114:0/3967103334 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4025400 con 0x563d7a7ae070 2024-09-21T07:00:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.059+0000 7fc8cdc00640 1 -- 172.21.15.114:0/3967103334 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d4015030 con 0x563d7a7ae070 2024-09-21T07:00:54.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.059+0000 7fc8cdc00640 1 -- 172.21.15.114:0/3967103334 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d401da60 con 0x563d7a7ae070 2024-09-21T07:00:54.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.060+0000 7fc8eabcc740 1 -- 172.21.15.114:0/3967103334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 msgr2=0x563d7ab6cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:54.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.060+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab6cea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400f790 tx=0x7fc8d400f000 comp rx=0 tx=0).stop 2024-09-21T07:00:54.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 -- 172.21.15.114:0/3967103334 shutdown_connections 2024-09-21T07:00:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7ae070 0x563d7ab6cea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a78d390 0x563d7ab7e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/3967103334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a7851b0 0x563d7ab6c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 -- 172.21.15.114:0/3967103334 >> 172.21.15.114:0/3967103334 conn(0x563d7a7a2460 msgr2=0x563d7ab59ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 -- 172.21.15.114:0/3967103334 shutdown_connections 2024-09-21T07:00:54.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 -- 172.21.15.114:0/3967103334 wait complete. 2024-09-21T07:00:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.061+0000 7fc8eabcc740 1 -- start start 2024-09-21T07:00:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.062+0000 7fc8eabcc740 1 Processor -- start 2024-09-21T07:00:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.062+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab77ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.062+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a78d390 0x563d7ab78420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.062+0000 7fc8eabcc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a7ae070 0x563d7ab7cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.062+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1b05f0 con 0x563d7a78d390 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.062+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1afef0 con 0x563d7a7ae070 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8eabcc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1a1b00 con 0x563d7a7851b0 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8eabcc740 10 client.0 _ll_register_callbacks cb 0x7fc8c800ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab77ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab77ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52280/0 (socket says 172.21.15.114:52280) 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e1600640 1 -- 172.21.15.114:0/1146714794 learned_addr learned my addr 172.21.15.114:0/1146714794 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e2000640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a7ae070 0x563d7ab7cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a78d390 0x563d7ab78420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e1600640 1 -- 172.21.15.114:0/1146714794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a7ae070 msgr2=0x563d7ab7cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.063+0000 7fc8e1600640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a7ae070 0x563d7ab7cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8e1600640 1 -- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a78d390 msgr2=0x563d7ab78420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8e1600640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a78d390 0x563d7ab78420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8e1600640 1 -- 172.21.15.114:0/1146714794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7ab7d320 con 0x563d7a7851b0 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8e1600640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab77ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00cb100 tx=0x7fc8d0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9a58462dc9abfd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8c2a00640 1 -- 172.21.15.114:0/1146714794 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00cd7e0 con 0x563d7a7851b0 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.064+0000 7fc8c2a00640 1 -- 172.21.15.114:0/1146714794 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7fc8d00ca100 con 0x563d7a7851b0 2024-09-21T07:00:54.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.065+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d4015030 con 0x563d7a7851b0 2024-09-21T07:00:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.065+0000 7fc8c2a00640 1 -- 172.21.15.114:0/1146714794 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc8d00cd600 con 0x563d7a7851b0 2024-09-21T07:00:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.065+0000 7fc8eabcc740 10 client.27301 fetch_fsmap learned FSMap version 43 2024-09-21T07:00:54.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.065+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563d7ab7e580 con 0x563d7a7851b0 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.066+0000 7fc8c2a00640 1 -- 172.21.15.114:0/1146714794 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8d4015030 con 0x563d7a7851b0 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8c2a00640 1 -- 172.21.15.114:0/1146714794 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7fc8d0105a10 con 0x563d7a7851b0 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8eabcc740 10 client.27301 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8eabcc740 10 client.27301 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8eabcc740 20 client.27301 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a741da0 0x563d7abf0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:54.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8eabcc740 4 client.27301 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:00:54.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.067+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d7aac99b0 con 0x563d7a741da0 2024-09-21T07:00:54.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.069+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a741da0 0x563d7abf0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.074+0000 7fc8c2a00640 10 client.27301 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:00:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.074+0000 7fc8e0c00640 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a741da0 0x563d7abf0530 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66a1885a53792988 server_cookie=300cf4f48518a0c3 in_seq=0 out_seq=0 2024-09-21T07:00:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.074+0000 7fc8c2a00640 1 -- 172.21.15.114:0/1146714794 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d7aac99b0 con 0x563d7a741da0 2024-09-21T07:00:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.075+0000 7fc8c2a00640 10 client.27301 handle_command_reply: tid=0 2024-09-21T07:00:54.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.075+0000 7fc8eabcc740 1 client.27301 shutdown 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.076+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 msgr2=0x563d7ab77ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.076+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab77ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00cb100 tx=0x7fc8d0005de0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 shutdown_connections 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x563d7a741da0 0x563d7abf0530 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d7a7ae070 0x563d7ab7cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563d7a78d390 0x563d7ab78420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 --2- 172.21.15.114:0/1146714794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563d7a7851b0 0x563d7ab77ee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 >> 172.21.15.114:0/1146714794 conn(0x563d7a7a2460 msgr2=0x563d7ab70060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8c2a00640 0 client.27301 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:00:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 shutdown_connections 2024-09-21T07:00:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.077+0000 7fc8eabcc740 1 -- 172.21.15.114:0/1146714794 wait complete. 2024-09-21T07:00:54.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.078+0000 7fc8eabcc740 20 client.27301 trim_cache size 0 max 16384 2024-09-21T07:00:54.080 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:00:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.079+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8c40b6850 msgr2=0x7fc8c40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.079+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8c40b6850 0x7fc8c40b8d60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4007d90 tx=0x7fc8d4007320 comp rx=0 tx=0).stop 2024-09-21T07:00:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.079+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 msgr2=0x7fc8dc1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.079+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 0x7fc8dc1a0a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0004770 tx=0x7fc8d0003d50 comp rx=0 tx=0).stop 2024-09-21T07:00:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 shutdown_connections 2024-09-21T07:00:54.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8dc1a6210 0x7fc8dc1a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8c40b6850 0x7fc8c40b8d60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8dc1079f0 0x7fc8dc1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 --2- 172.21.15.114:0/564695978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc8dc102f00 0x7fc8dc1a0a90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 >> 172.21.15.114:0/564695978 conn(0x7fc8dc0fe2c0 msgr2=0x7fc8dc105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 shutdown_connections 2024-09-21T07:00:54.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:54.080+0000 7fc8e2a00640 1 -- 172.21.15.114:0/564695978 wait complete. 2024-09-21T07:00:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:54 smithi193 ceph-mon[34601]: pgmap v599: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 492 op/s 2024-09-21T07:00:55.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:54 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:55.297 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:54 smithi114 ceph-mon[30654]: pgmap v599: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 492 op/s 2024-09-21T07:00:55.297 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:54 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:55.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:54 smithi183 ceph-mon[34219]: pgmap v599: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 11 MiB/s wr, 492 op/s 2024-09-21T07:00:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:54 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:00:55.369 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:00:56.021 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:00:56.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.897+0000 7f374ca00640 1 -- 172.21.15.114:0/1125388091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810d210 msgr2=0x7f3748116ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:56.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.897+0000 7f374ca00640 1 --2- 172.21.15.114:0/1125388091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810d210 0x7f3748116ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3740009c10 tx=0x7f374002a710 comp rx=0 tx=0).stop 2024-09-21T07:00:56.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 -- 172.21.15.114:0/1125388091 shutdown_connections 2024-09-21T07:00:56.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 --2- 172.21.15.114:0/1125388091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810d210 0x7f3748116ca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 --2- 172.21.15.114:0/1125388091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810c8c0 0x7f374810ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 --2- 172.21.15.114:0/1125388091 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810a490 0x7f374810a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 -- 172.21.15.114:0/1125388091 >> 172.21.15.114:0/1125388091 conn(0x7f37480796d0 msgr2=0x7f374807baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:56.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 -- 172.21.15.114:0/1125388091 shutdown_connections 2024-09-21T07:00:56.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.898+0000 7f374ca00640 1 -- 172.21.15.114:0/1125388091 wait complete. 2024-09-21T07:00:56.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.901+0000 7f374ca00640 1 Processor -- start 2024-09-21T07:00:56.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.901+0000 7f374ca00640 1 -- start start 2024-09-21T07:00:56.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.902+0000 7f374ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810a490 0x7f3748085c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.902+0000 7f374ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810c8c0 0x7f374807ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.902+0000 7f374ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810d210 0x7f374807f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.902+0000 7f374ca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f374811a0e0 con 0x7f374810c8c0 2024-09-21T07:00:56.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.902+0000 7f374ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3748119f60 con 0x7f374810d210 2024-09-21T07:00:56.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.902+0000 7f374ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f374811a260 con 0x7f374810a490 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.903+0000 7f3745e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810a490 0x7f3748085c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.903+0000 7f3745400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810c8c0 0x7f374807ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.903+0000 7f3745e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810a490 0x7f3748085c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40188/0 (socket says 172.21.15.114:40188) 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.903+0000 7f3745e00640 1 -- 172.21.15.114:0/1204472988 learned_addr learned my addr 172.21.15.114:0/1204472988 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.904+0000 7f3745400640 1 -- 172.21.15.114:0/1204472988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810a490 msgr2=0x7f3748085c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.904+0000 7f3745400640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810a490 0x7f3748085c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.904+0000 7f3745400640 1 -- 172.21.15.114:0/1204472988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810d210 msgr2=0x7f374807f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:56.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.903+0000 7f3746800640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810d210 0x7f374807f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.904+0000 7f3745400640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810d210 0x7f374807f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.904+0000 7f3745400640 1 -- 172.21.15.114:0/1204472988 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374807fca0 con 0x7f374810c8c0 2024-09-21T07:00:56.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.905+0000 7f3746800640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810d210 0x7f374807f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:56.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.905+0000 7f3745400640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810c8c0 0x7f374807ef40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f374806bb30 tx=0x7f373400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:56.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.906+0000 7f373ea00640 1 -- 172.21.15.114:0/1204472988 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373400cc90 con 0x7f374810c8c0 2024-09-21T07:00:56.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.906+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3748084940 con 0x7f374810c8c0 2024-09-21T07:00:56.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.906+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3748084ed0 con 0x7f374810c8c0 2024-09-21T07:00:56.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.907+0000 7f373ea00640 1 -- 172.21.15.114:0/1204472988 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373400ee70 con 0x7f374810c8c0 2024-09-21T07:00:56.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.908+0000 7f373ea00640 1 -- 172.21.15.114:0/1204472988 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734018680 con 0x7f374810c8c0 2024-09-21T07:00:56.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.914+0000 7f375380a740 1 Processor -- start 2024-09-21T07:00:56.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.914+0000 7f375380a740 1 -- start start 2024-09-21T07:00:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.915+0000 7f375380a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x561762151da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.916+0000 7f375380a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x5617621522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.916+0000 7f375380a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d8a390 0x561762152820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.916+0000 7f375380a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5617620c8fb0 con 0x561761dab070 2024-09-21T07:00:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.916+0000 7f375380a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5617620b83e0 con 0x561761d821b0 2024-09-21T07:00:56.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.916+0000 7f375380a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617620bc1f0 con 0x561761d8a390 2024-09-21T07:00:56.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.917+0000 7f3745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x5617621522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.917+0000 7f3745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x5617621522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41406/0 (socket says 172.21.15.114:41406) 2024-09-21T07:00:56.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.917+0000 7f3745e00640 1 -- 172.21.15.114:0/957706244 learned_addr learned my addr 172.21.15.114:0/957706244 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:56.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3746800640 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x561762151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3745e00640 1 -- 172.21.15.114:0/957706244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d8a390 msgr2=0x561762152820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:56.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3745e00640 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d8a390 0x561762152820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3745e00640 1 -- 172.21.15.114:0/957706244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 msgr2=0x561762151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3745e00640 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x561762151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3745e00640 1 -- 172.21.15.114:0/957706244 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561762082cd0 con 0x561761d821b0 2024-09-21T07:00:56.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.918+0000 7f3746800640 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x561762151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:56.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.919+0000 7f3745e00640 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x5617621522e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3738009d60 tx=0x7f3738011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0715ba462d315c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:56.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.920+0000 7f3733000640 1 -- 172.21.15.114:0/957706244 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738004230 con 0x561761d821b0 2024-09-21T07:00:56.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.920+0000 7f3733000640 1 -- 172.21.15.114:0/957706244 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37380043d0 con 0x561761d821b0 2024-09-21T07:00:56.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.920+0000 7f3733000640 1 -- 172.21.15.114:0/957706244 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738019820 con 0x561761d821b0 2024-09-21T07:00:56.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.920+0000 7f373ea00640 1 -- 172.21.15.114:0/1204472988 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f37340188e0 con 0x7f374810c8c0 2024-09-21T07:00:56.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.921+0000 7f375380a740 1 -- 172.21.15.114:0/957706244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 msgr2=0x5617621522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:56.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.921+0000 7f375380a740 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x5617621522e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3738009d60 tx=0x7f3738011810 comp rx=0 tx=0).stop 2024-09-21T07:00:56.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.922+0000 7f375380a740 1 -- 172.21.15.114:0/957706244 shutdown_connections 2024-09-21T07:00:56.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.922+0000 7f375380a740 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d8a390 0x561762152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.922+0000 7f375380a740 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x5617621522e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.922+0000 7f375380a740 1 --2- 172.21.15.114:0/957706244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x561762151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.922+0000 7f375380a740 1 -- 172.21.15.114:0/957706244 >> 172.21.15.114:0/957706244 conn(0x561761d9f460 msgr2=0x56176214fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:56.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.923+0000 7f375380a740 1 -- 172.21.15.114:0/957706244 shutdown_connections 2024-09-21T07:00:56.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.923+0000 7f375380a740 1 -- 172.21.15.114:0/957706244 wait complete. 2024-09-21T07:00:56.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.924+0000 7f373ea00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f372c0b6590 0x7f372c0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.925+0000 7f375380a740 1 -- start start 2024-09-21T07:00:56.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.926+0000 7f375380a740 1 Processor -- start 2024-09-21T07:00:56.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.926+0000 7f3745e00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f372c0b6590 0x7f372c0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.924+0000 7f373ea00640 1 -- 172.21.15.114:0/1204472988 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f37340e7b60 con 0x7f374810c8c0 2024-09-21T07:00:56.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.928+0000 7f375380a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x56176215d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.928+0000 7f375380a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x56176215db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.928+0000 7f375380a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761dab070 0x561762169950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f375380a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561761f19700 con 0x561761d8a390 2024-09-21T07:00:56.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f375380a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561762053580 con 0x561761dab070 2024-09-21T07:00:56.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f375380a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5617621605d0 con 0x561761d821b0 2024-09-21T07:00:56.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f3745e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x56176215d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f3745e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x56176215d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40202/0 (socket says 172.21.15.114:40202) 2024-09-21T07:00:56.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f3745e00640 1 -- 172.21.15.114:0/4030303478 learned_addr learned my addr 172.21.15.114:0/4030303478 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:56.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.929+0000 7f3745400640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x56176215db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f3746800640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761dab070 0x561762169950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f375380a740 10 client.0 _ll_register_callbacks cb 0x561762063470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f3745400640 1 -- 172.21.15.114:0/4030303478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 msgr2=0x56176215d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f3745400640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x56176215d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f3745400640 1 -- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761dab070 msgr2=0x561762169950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f3745400640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761dab070 0x561762169950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.930+0000 7f3745400640 1 -- 172.21.15.114:0/4030303478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561761e16650 con 0x561761d8a390 2024-09-21T07:00:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.932+0000 7f3745e00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f372c0b6590 0x7f372c0b8aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3738009b20 tx=0x7f3738005880 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:56.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.934+0000 7f3745400640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x56176215db60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f37340e4f40 tx=0x7f37340acc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598658f9205ec70d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:56.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.934+0000 7f3727400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:00:56.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.935+0000 7f3727400640 1 -- 172.21.15.114:0/4030303478 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37340acf90 con 0x561761d8a390 2024-09-21T07:00:56.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.935+0000 7f3727400640 1 -- 172.21.15.114:0/4030303478 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f37340b6050 con 0x561761d8a390 2024-09-21T07:00:56.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.935+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561762055b90 con 0x561761d8a390 2024-09-21T07:00:56.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.936+0000 7f3727400640 1 -- 172.21.15.114:0/4030303478 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f37340acdb0 con 0x561761d8a390 2024-09-21T07:00:56.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.936+0000 7f375380a740 10 client.17031 fetch_fsmap learned FSMap version 43 2024-09-21T07:00:56.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.936+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561762070950 con 0x561761d8a390 2024-09-21T07:00:56.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.937+0000 7f3727400640 1 -- 172.21.15.114:0/4030303478 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561762055b90 con 0x561761d8a390 2024-09-21T07:00:56.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f3727400640 1 -- 172.21.15.114:0/4030303478 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f37340b32b0 con 0x561761d8a390 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f375380a740 10 client.17031 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f375380a740 10 client.17031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f375380a740 20 client.17031 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f375380a740 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d699f0 0x56176216f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f375380a740 4 client.17031 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617620caf00 con 0x561761d699f0 2024-09-21T07:00:56.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.938+0000 7f3745e00640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d699f0 0x56176216f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.943+0000 7f3745e00640 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d699f0 0x56176216f660 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbb701fe291262d server_cookie=702de581e9b798e4 in_seq=0 out_seq=0 2024-09-21T07:00:56.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.943+0000 7f3727400640 10 client.17031 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.944+0000 7f3727400640 1 -- 172.21.15.114:0/4030303478 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5617620caf00 con 0x561761d699f0 2024-09-21T07:00:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.944+0000 7f3727400640 10 client.17031 handle_command_reply: tid=0 2024-09-21T07:00:56.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.944+0000 7f375380a740 1 client.17031 shutdown 2024-09-21T07:00:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.945+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 msgr2=0x56176215db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:56.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.945+0000 7f375380a740 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x56176215db60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f37340e4f40 tx=0x7f37340acc70 comp rx=0 tx=0).stop 2024-09-21T07:00:56.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 shutdown_connections 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761dab070 0x561762169950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x56176215db60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d699f0 0x56176216f660 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 --2- 172.21.15.114:0/4030303478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x56176215d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 >> 172.21.15.114:0/4030303478 conn(0x561761d9f460 msgr2=0x561762154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f3727400640 0 client.17031 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 shutdown_connections 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.946+0000 7f375380a740 1 -- 172.21.15.114:0/4030303478 wait complete. 2024-09-21T07:00:56.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.947+0000 7f375380a740 20 client.17031 trim_cache size 0 max 16384 2024-09-21T07:00:56.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.977+0000 7f375380a740 1 Processor -- start 2024-09-21T07:00:56.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.977+0000 7f375380a740 1 -- start start 2024-09-21T07:00:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.978+0000 7f375380a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x5617621555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.978+0000 7f375380a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d8a390 0x56176215a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.978+0000 7f375380a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x56176215ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.978+0000 7f375380a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f372800ff50 con 0x561761dab070 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.978+0000 7f375380a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3728011af0 con 0x561761d8a390 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.978+0000 7f375380a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3728010620 con 0x561761d821b0 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x5617621555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d8a390 0x56176215a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x5617621555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40204/0 (socket says 172.21.15.114:40204) 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745e00640 1 -- 172.21.15.114:0/2132688415 learned_addr learned my addr 172.21.15.114:0/2132688415 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:56.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745400640 1 -- 172.21.15.114:0/2132688415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 msgr2=0x5617621555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745400640 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x5617621555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745400640 1 -- 172.21.15.114:0/2132688415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 msgr2=0x56176215ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745400640 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x56176215ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.979+0000 7f3745400640 1 -- 172.21.15.114:0/2132688415 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176215e8e0 con 0x561761d8a390 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.980+0000 7f3745400640 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d8a390 0x56176215a810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f37340e4ff0 tx=0x7f37340ad540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ade43d8c11000 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.980+0000 7f3732600640 1 -- 172.21.15.114:0/2132688415 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37340acf90 con 0x561761d8a390 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.980+0000 7f3732600640 1 -- 172.21.15.114:0/2132688415 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37340ad150 con 0x561761d8a390 2024-09-21T07:00:56.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.980+0000 7f3732600640 1 -- 172.21.15.114:0/2132688415 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37340b6940 con 0x561761d8a390 2024-09-21T07:00:56.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 -- 172.21.15.114:0/2132688415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d8a390 msgr2=0x56176215a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:56.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d8a390 0x56176215a810 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f37340e4ff0 tx=0x7f37340ad540 comp rx=0 tx=0).stop 2024-09-21T07:00:56.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 -- 172.21.15.114:0/2132688415 shutdown_connections 2024-09-21T07:00:56.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761dab070 0x56176215ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d8a390 0x56176215a810 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 --2- 172.21.15.114:0/2132688415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761d821b0 0x5617621555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.982+0000 7f375380a740 1 -- 172.21.15.114:0/2132688415 >> 172.21.15.114:0/2132688415 conn(0x561761d9f460 msgr2=0x561762154ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:56.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.984+0000 7f375380a740 1 -- 172.21.15.114:0/2132688415 shutdown_connections 2024-09-21T07:00:56.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.984+0000 7f375380a740 1 -- 172.21.15.114:0/2132688415 wait complete. 2024-09-21T07:00:56.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.985+0000 7f375380a740 1 -- start start 2024-09-21T07:00:56.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.985+0000 7f375380a740 1 Processor -- start 2024-09-21T07:00:56.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.986+0000 7f375380a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x561762161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.986+0000 7f375380a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x561762162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.986+0000 7f375380a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761dab070 0x5617621d9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f3745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x561762161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f3745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x561762161fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41436/0 (socket says 172.21.15.114:41436) 2024-09-21T07:00:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f3745e00640 1 -- 172.21.15.114:0/958806557 learned_addr learned my addr 172.21.15.114:0/958806557 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37280113f0 con 0x561761d8a390 2024-09-21T07:00:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3728010cf0 con 0x561761d821b0 2024-09-21T07:00:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f372800df30 con 0x561761dab070 2024-09-21T07:00:56.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.988+0000 7f375380a740 10 client.0 _ll_register_callbacks cb 0x7f37340a9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:56.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.990+0000 7f3745400640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x561762162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.990+0000 7f3746800640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761dab070 0x5617621d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.991+0000 7f3745e00640 1 -- 172.21.15.114:0/958806557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761dab070 msgr2=0x5617621d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.991+0000 7f3745e00640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761dab070 0x5617621d9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.991+0000 7f3745e00640 1 -- 172.21.15.114:0/958806557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 msgr2=0x561762162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.991+0000 7f3745e00640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x561762162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.991+0000 7f3745e00640 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617621d9560 con 0x561761d821b0 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.991+0000 7f3745400640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x561762162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.992+0000 7f3746800640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761dab070 0x5617621d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.992+0000 7f3745e00640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x561762161fd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3738009cb0 tx=0x7f3738005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409e934cd338027e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:56.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.992+0000 7f3727400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:56.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.992+0000 7f3727400640 1 -- 172.21.15.114:0/958806557 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738003cd0 con 0x561761d821b0 2024-09-21T07:00:56.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.993+0000 7f3727400640 1 -- 172.21.15.114:0/958806557 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f373801f040 con 0x561761d821b0 2024-09-21T07:00:56.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.993+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37340ad150 con 0x561761d821b0 2024-09-21T07:00:56.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.994+0000 7f3727400640 1 -- 172.21.15.114:0/958806557 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3738004660 con 0x561761d821b0 2024-09-21T07:00:56.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.994+0000 7f375380a740 10 client.27392 fetch_fsmap learned FSMap version 43 2024-09-21T07:00:56.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.994+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5617621da7e0 con 0x561761d821b0 2024-09-21T07:00:56.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.995+0000 7f3727400640 1 -- 172.21.15.114:0/958806557 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37340ad150 con 0x561761d821b0 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f3727400640 1 -- 172.21.15.114:0/958806557 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f3738019760 con 0x561761d821b0 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f375380a740 10 client.27392 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f375380a740 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f375380a740 20 client.27392 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f375380a740 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d3eda0 0x5617621dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f375380a740 4 client.27392 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:00:56.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.996+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617620c69b0 con 0x561761d3eda0 2024-09-21T07:00:56.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.997+0000 7f3745400640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d3eda0 0x5617621dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:56.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.997+0000 7f3727400640 10 client.27392 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:56.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.997+0000 7f3745400640 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d3eda0 0x5617621dcdd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890a503264075e36 server_cookie=6280b91a57ef7ccc in_seq=0 out_seq=0 2024-09-21T07:00:56.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.998+0000 7f3727400640 1 -- 172.21.15.114:0/958806557 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617620c69b0 con 0x561761d3eda0 2024-09-21T07:00:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.998+0000 7f3727400640 10 client.27392 handle_command_reply: tid=0 2024-09-21T07:00:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.998+0000 7f375380a740 1 client.27392 shutdown 2024-09-21T07:00:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.998+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 msgr2=0x561762161fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:57.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.998+0000 7f375380a740 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x561762161fd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3738009cb0 tx=0x7f3738005f90 comp rx=0 tx=0).stop 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.998+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 shutdown_connections 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f375380a740 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561761d3eda0 0x5617621dcdd0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f375380a740 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561761dab070 0x5617621d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f375380a740 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x561761d8a390 0x561762162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f375380a740 1 --2- 172.21.15.114:0/958806557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561761d821b0 0x561762161fd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 >> 172.21.15.114:0/958806557 conn(0x561761d9f460 msgr2=0x56176215eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f3727400640 0 client.27392 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:57.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:56.999+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 shutdown_connections 2024-09-21T07:00:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.000+0000 7f375380a740 1 -- 172.21.15.114:0/958806557 wait complete. 2024-09-21T07:00:57.002 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:00:57.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.000+0000 7f375380a740 20 client.27392 trim_cache size 0 max 16384 2024-09-21T07:00:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.001+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f372c0b6590 msgr2=0x7f372c0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.001+0000 7f374ca00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f372c0b6590 0x7f372c0b8aa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3738009b20 tx=0x7f3738005880 comp rx=0 tx=0).stop 2024-09-21T07:00:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.001+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810c8c0 msgr2=0x7f374807ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.001+0000 7f374ca00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810c8c0 0x7f374807ef40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f374806bb30 tx=0x7f373400ec20 comp rx=0 tx=0).stop 2024-09-21T07:00:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 shutdown_connections 2024-09-21T07:00:57.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f372c0b6590 0x7f372c0b8aa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f374810d210 0x7f374807f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f374810c8c0 0x7f374807ef40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 --2- 172.21.15.114:0/1204472988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f374810a490 0x7f3748085c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 >> 172.21.15.114:0/1204472988 conn(0x7f37480796d0 msgr2=0x7f374810e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 shutdown_connections 2024-09-21T07:00:57.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:00:57.002+0000 7f374ca00640 1 -- 172.21.15.114:0/1204472988 wait complete. 2024-09-21T07:00:57.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:56 smithi114 ceph-mon[30654]: pgmap v600: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 512 op/s 2024-09-21T07:00:57.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:56 smithi183 ceph-mon[34219]: pgmap v600: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 512 op/s 2024-09-21T07:00:57.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:56 smithi193 ceph-mon[34601]: pgmap v600: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 12 MiB/s wr, 512 op/s 2024-09-21T07:00:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 380.0 s, 794.0 tps, lat 127.077 ms stddev 158.722 2024-09-21T07:00:59.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:00:58 smithi114 ceph-mon[30654]: pgmap v601: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 633 op/s 2024-09-21T07:00:59.317 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:00:59.317 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:00:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:00:58 smithi183 ceph-mon[34219]: pgmap v601: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 633 op/s 2024-09-21T07:00:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:00:58 smithi193 ceph-mon[34601]: pgmap v601: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 13 MiB/s wr, 633 op/s 2024-09-21T07:00:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 -- 172.21.15.114:0/2071120763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410aef0 msgr2=0x7f6f34115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 --2- 172.21.15.114:0/2071120763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410aef0 0x7f6f34115fd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009860 tx=0x7f6f2002a340 comp rx=0 tx=0).stop 2024-09-21T07:00:59.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 -- 172.21.15.114:0/2071120763 shutdown_connections 2024-09-21T07:00:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 --2- 172.21.15.114:0/2071120763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f341165a0 0x7f6f34118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 --2- 172.21.15.114:0/2071120763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410aef0 0x7f6f34115fd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 --2- 172.21.15.114:0/2071120763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f3410a5a0 0x7f6f3410a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.824+0000 7f6f39600640 1 -- 172.21.15.114:0/2071120763 >> 172.21.15.114:0/2071120763 conn(0x7f6f34070dd0 msgr2=0x7f6f340711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.827+0000 7f6f39600640 1 -- 172.21.15.114:0/2071120763 shutdown_connections 2024-09-21T07:00:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.827+0000 7f6f39600640 1 -- 172.21.15.114:0/2071120763 wait complete. 2024-09-21T07:00:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.828+0000 7f6f39600640 1 Processor -- start 2024-09-21T07:00:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.830+0000 7f6f39600640 1 -- start start 2024-09-21T07:00:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.831+0000 7f6f39600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 0x7f6f341a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.831+0000 7f6f39600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f3410aef0 0x7f6f341a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.832+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 0x7f6f341a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.832+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 0x7f6f341a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41442/0 (socket says 172.21.15.114:41442) 2024-09-21T07:00:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.832+0000 7f6f32a00640 1 -- 172.21.15.114:0/353523247 learned_addr learned my addr 172.21.15.114:0/353523247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.832+0000 7f6f32000640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f3410aef0 0x7f6f341a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.833+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f341165a0 0x7f6f341a8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.833+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3411c200 con 0x7f6f341165a0 2024-09-21T07:00:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.833+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3411c080 con 0x7f6f3410a5a0 2024-09-21T07:00:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.833+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3411c380 con 0x7f6f3410aef0 2024-09-21T07:00:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.834+0000 7f6f32a00640 1 -- 172.21.15.114:0/353523247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f3410aef0 msgr2=0x7f6f341a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.834+0000 7f6f32a00640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f3410aef0 0x7f6f341a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.834+0000 7f6f32a00640 1 -- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f341165a0 msgr2=0x7f6f341a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.834+0000 7f6f32a00640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f341165a0 0x7f6f341a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.834+0000 7f6f32a00640 1 -- 172.21.15.114:0/353523247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f341a8c70 con 0x7f6f3410a5a0 2024-09-21T07:00:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.835+0000 7f6f32000640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f3410aef0 0x7f6f341a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.836+0000 7f6f32a00640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 0x7f6f341a16d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400e9c0 tx=0x7f6f2400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.837+0000 7f6f2b400640 1 -- 172.21.15.114:0/353523247 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400ccc0 con 0x7f6f3410a5a0 2024-09-21T07:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.837+0000 7f6f2b400640 1 -- 172.21.15.114:0/353523247 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f240054d0 con 0x7f6f3410a5a0 2024-09-21T07:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.837+0000 7f6f2b400640 1 -- 172.21.15.114:0/353523247 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24010430 con 0x7f6f3410a5a0 2024-09-21T07:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.837+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341a8f60 con 0x7f6f3410a5a0 2024-09-21T07:00:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.838+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a9400 con 0x7f6f3410a5a0 2024-09-21T07:00:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.844+0000 7f6f405ea740 1 Processor -- start 2024-09-21T07:00:59.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.846+0000 7f6f405ea740 1 -- start start 2024-09-21T07:00:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.846+0000 7f6f2b400640 1 -- 172.21.15.114:0/353523247 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6f24004f80 con 0x7f6f3410a5a0 2024-09-21T07:00:59.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.848+0000 7f6f2b400640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6f180b6800 0x7f6f180b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.849+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 0x55c45ea0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.849+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea0e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.849+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea0e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.849+0000 7f6f405ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c45e9836b0 con 0x55c45e6402f0 2024-09-21T07:00:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.849+0000 7f6f405ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c45e972ae0 con 0x55c45e6484d0 2024-09-21T07:00:59.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.849+0000 7f6f405ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c45e9768f0 con 0x55c45e6691b0 2024-09-21T07:00:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.850+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 0x55c45ea0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.850+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 0x55c45ea0dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40246/0 (socket says 172.21.15.114:40246) 2024-09-21T07:00:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.850+0000 7f6f32a00640 1 -- 172.21.15.114:0/889753576 learned_addr learned my addr 172.21.15.114:0/889753576 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32000640 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea0e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32a00640 1 -- 172.21.15.114:0/889753576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 msgr2=0x55c45ea0e6b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:00:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32a00640 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea0e6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32a00640 1 -- 172.21.15.114:0/889753576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 msgr2=0x55c45ea0e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32a00640 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea0e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32a00640 1 -- 172.21.15.114:0/889753576 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45e93f780 con 0x55c45e6691b0 2024-09-21T07:00:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.851+0000 7f6f32a00640 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 0x55c45ea0dc30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f240e5270 tx=0x7f6f240f05f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f98c81c10b9b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:59.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.852+0000 7f6f1f400640 1 -- 172.21.15.114:0/889753576 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240fb070 con 0x55c45e6691b0 2024-09-21T07:00:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.852+0000 7f6f1f400640 1 -- 172.21.15.114:0/889753576 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f240f68b0 con 0x55c45e6691b0 2024-09-21T07:00:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.852+0000 7f6f1f400640 1 -- 172.21.15.114:0/889753576 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240e88e0 con 0x55c45e6691b0 2024-09-21T07:00:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.853+0000 7f6f33400640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6f180b6800 0x7f6f180b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.853+0000 7f6f2b400640 1 -- 172.21.15.114:0/353523247 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6f240e7f50 con 0x7f6f3410a5a0 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 -- 172.21.15.114:0/889753576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 msgr2=0x55c45ea0dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 0x55c45ea0dc30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6f240e5270 tx=0x7f6f240f05f0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 -- 172.21.15.114:0/889753576 shutdown_connections 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea0e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea0e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 --2- 172.21.15.114:0/889753576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6691b0 0x55c45ea0dc30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f405ea740 1 -- 172.21.15.114:0/889753576 >> 172.21.15.114:0/889753576 conn(0x55c45e65d300 msgr2=0x55c45ea0bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.855+0000 7f6f33400640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6f180b6800 0x7f6f180b8d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005c40 tx=0x7f6f2c005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.856+0000 7f6f405ea740 1 -- 172.21.15.114:0/889753576 shutdown_connections 2024-09-21T07:00:59.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.856+0000 7f6f405ea740 1 -- 172.21.15.114:0/889753576 wait complete. 2024-09-21T07:00:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.856+0000 7f6f405ea740 1 -- start start 2024-09-21T07:00:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.857+0000 7f6f405ea740 1 Processor -- start 2024-09-21T07:00:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.857+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea192c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.857+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f405ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c45e8aaab0 con 0x55c45e6402f0 2024-09-21T07:00:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f32000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea192c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f32000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea19800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40250/0 (socket says 172.21.15.114:40250) 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f32000640 1 -- 172.21.15.114:0/2734097868 learned_addr learned my addr 172.21.15.114:0/2734097868 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c45e919570 con 0x55c45e627d80 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.861+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c45ea1c3b0 con 0x55c45e6484d0 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.862+0000 7f6f405ea740 10 client.0 _ll_register_callbacks cb 0x55c45e928100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.862+0000 7f6f33400640 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.862+0000 7f6f32000640 1 -- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 msgr2=0x55c45ea256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.862+0000 7f6f32000640 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.863+0000 7f6f32000640 1 -- 172.21.15.114:0/2734097868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 msgr2=0x55c45ea192c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.863+0000 7f6f32000640 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea192c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.863+0000 7f6f32000640 1 -- 172.21.15.114:0/2734097868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45e6d4790 con 0x55c45e6484d0 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.863+0000 7f6f32000640 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea19800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009780 tx=0x7f6f20004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d57d8852f31a950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.863+0000 7f6f13e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.863+0000 7f6f13e00640 1 -- 172.21.15.114:0/2734097868 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f20037040 con 0x55c45e6484d0 2024-09-21T07:00:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.864+0000 7f6f13e00640 1 -- 172.21.15.114:0/2734097868 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6f20030450 con 0x55c45e6484d0 2024-09-21T07:00:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.865+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f240f68b0 con 0x55c45e6484d0 2024-09-21T07:00:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.865+0000 7f6f13e00640 1 -- 172.21.15.114:0/2734097868 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6f20030650 con 0x55c45e6484d0 2024-09-21T07:00:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.865+0000 7f6f405ea740 10 client.27325 fetch_fsmap learned FSMap version 43 2024-09-21T07:00:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.865+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c45e74d620 con 0x55c45e6484d0 2024-09-21T07:00:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.868+0000 7f6f13e00640 1 -- 172.21.15.114:0/2734097868 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f240f68b0 con 0x55c45e6484d0 2024-09-21T07:00:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.868+0000 7f6f13e00640 1 -- 172.21.15.114:0/2734097868 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f6f20004960 con 0x55c45e6484d0 2024-09-21T07:00:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.868+0000 7f6f405ea740 10 client.27325 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:00:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.868+0000 7f6f405ea740 10 client.27325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.868+0000 7f6f405ea740 20 client.27325 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.868+0000 7f6f405ea740 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45e5fd1c0 0x55c45ea2b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.869+0000 7f6f405ea740 4 client.27325 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:00:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.869+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c45e985600 con 0x55c45e5fd1c0 2024-09-21T07:00:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.869+0000 7f6f32a00640 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45e5fd1c0 0x55c45ea2b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.873+0000 7f6f32a00640 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45e5fd1c0 0x55c45ea2b3d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f4c822712b8ec36 server_cookie=907acea91b429798 in_seq=0 out_seq=0 2024-09-21T07:00:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.873+0000 7f6f13e00640 10 client.27325 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.874+0000 7f6f13e00640 1 -- 172.21.15.114:0/2734097868 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c45e985600 con 0x55c45e5fd1c0 2024-09-21T07:00:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.874+0000 7f6f13e00640 10 client.27325 handle_command_reply: tid=0 2024-09-21T07:00:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.874+0000 7f6f405ea740 1 client.27325 shutdown 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 msgr2=0x55c45ea19800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea19800 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009780 tx=0x7f6f20004250 comp rx=0 tx=0).stop 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 shutdown_connections 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45e5fd1c0 0x55c45ea2b3d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea19800 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 --2- 172.21.15.114:0/2734097868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea192c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f13e00640 0 client.27325 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 >> 172.21.15.114:0/2734097868 conn(0x55c45e65d300 msgr2=0x55c45ea0ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 shutdown_connections 2024-09-21T07:00:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.875+0000 7f6f405ea740 1 -- 172.21.15.114:0/2734097868 wait complete. 2024-09-21T07:00:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.876+0000 7f6f405ea740 20 client.27325 trim_cache size 0 max 16384 2024-09-21T07:00:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.906+0000 7f6f405ea740 1 Processor -- start 2024-09-21T07:00:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.906+0000 7f6f405ea740 1 -- start start 2024-09-21T07:00:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.906+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea0f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.906+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea16490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.906+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f405ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341c8660 con 0x55c45e6402f0 2024-09-21T07:00:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f405ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341ca200 con 0x55c45e627d80 2024-09-21T07:00:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f405ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341c8d30 con 0x55c45e6484d0 2024-09-21T07:00:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea0f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea0f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41466/0 (socket says 172.21.15.114:41466) 2024-09-21T07:00:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 -- 172.21.15.114:0/3862926911 learned_addr learned my addr 172.21.15.114:0/3862926911 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f33400640 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32000640 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea16490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 -- 172.21.15.114:0/3862926911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 msgr2=0x55c45ea169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 -- 172.21.15.114:0/3862926911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 msgr2=0x55c45ea16490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea16490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f32a00640 1 -- 172.21.15.114:0/3862926911 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c45ea107b0 con 0x55c45e627d80 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.907+0000 7f6f33400640 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.908+0000 7f6f32a00640 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea0f370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000c00 tx=0x7f6f24009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ef9599e6d2d9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.908+0000 7f6f1ea00640 1 -- 172.21.15.114:0/3862926911 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240a98d0 con 0x55c45e627d80 2024-09-21T07:00:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.908+0000 7f6f1ea00640 1 -- 172.21.15.114:0/3862926911 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f240ad540 con 0x55c45e627d80 2024-09-21T07:00:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.908+0000 7f6f1ea00640 1 -- 172.21.15.114:0/3862926911 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f240a91a0 con 0x55c45e627d80 2024-09-21T07:00:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 -- 172.21.15.114:0/3862926911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 msgr2=0x55c45ea0f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea0f370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000c00 tx=0x7f6f24009f90 comp rx=0 tx=0).stop 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 -- 172.21.15.114:0/3862926911 shutdown_connections 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6484d0 0x55c45ea169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e6402f0 0x55c45ea16490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 --2- 172.21.15.114:0/3862926911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e627d80 0x55c45ea0f370 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 -- 172.21.15.114:0/3862926911 >> 172.21.15.114:0/3862926911 conn(0x55c45e65d300 msgr2=0x55c45ea0ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 -- 172.21.15.114:0/3862926911 shutdown_connections 2024-09-21T07:00:59.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.910+0000 7f6f405ea740 1 -- 172.21.15.114:0/3862926911 wait complete. 2024-09-21T07:00:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.911+0000 7f6f405ea740 1 -- start start 2024-09-21T07:00:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.911+0000 7f6f405ea740 1 Processor -- start 2024-09-21T07:00:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e627d80 0x55c45ea1f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6402f0 0x55c45ea1fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea20150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341c9b00 con 0x55c45e627d80 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341c9400 con 0x55c45e6484d0 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341c6600 con 0x55c45e6402f0 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.913+0000 7f6f405ea740 10 client.0 _ll_register_callbacks cb 0x7f6f20035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.914+0000 7f6f33400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea20150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.914+0000 7f6f33400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea20150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41472/0 (socket says 172.21.15.114:41472) 2024-09-21T07:00:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.914+0000 7f6f33400640 1 -- 172.21.15.114:0/337534763 learned_addr learned my addr 172.21.15.114:0/337534763 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:00:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.914+0000 7f6f32a00640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e627d80 0x55c45ea1f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.914+0000 7f6f32000640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6402f0 0x55c45ea1fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.914+0000 7f6f33400640 1 -- 172.21.15.114:0/337534763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6402f0 msgr2=0x55c45ea1fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f33400640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6402f0 0x55c45ea1fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f33400640 1 -- 172.21.15.114:0/337534763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e627d80 msgr2=0x55c45ea1f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f33400640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e627d80 0x55c45ea1f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f33400640 1 -- 172.21.15.114:0/337534763 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c45ea94e90 con 0x55c45e6484d0 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f32a00640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e627d80 0x55c45ea1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f32000640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6402f0 0x55c45ea1fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f33400640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea20150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0108b0 tx=0x7f6f2c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e183154f43d11ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f13e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f13e00640 1 -- 172.21.15.114:0/337534763 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c01fb80 con 0x55c45e6484d0 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.915+0000 7f6f13e00640 1 -- 172.21.15.114:0/337534763 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f6f2c01fd20 con 0x55c45e6484d0 2024-09-21T07:00:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.916+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f240ad540 con 0x55c45e6484d0 2024-09-21T07:00:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.916+0000 7f6f13e00640 1 -- 172.21.15.114:0/337534763 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c010e30 con 0x55c45e6484d0 2024-09-21T07:00:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.916+0000 7f6f405ea740 10 client.27407 fetch_fsmap learned FSMap version 43 2024-09-21T07:00:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.916+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c45ea961b0 con 0x55c45e6484d0 2024-09-21T07:00:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.916+0000 7f6f13e00640 1 -- 172.21.15.114:0/337534763 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f240ad540 con 0x55c45e6484d0 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f13e00640 1 -- 172.21.15.114:0/337534763 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f6f2c026070 con 0x55c45e6484d0 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f405ea740 10 client.27407 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f405ea740 10 client.27407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f405ea740 20 client.27407 populate_metadata read hostname 'smithi114' 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f405ea740 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45ea969c0 0x55c45ea98de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f405ea740 4 client.27407 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c45e9810b0 con 0x55c45ea969c0 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.917+0000 7f6f32a00640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45ea969c0 0x55c45ea98de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.918+0000 7f6f13e00640 10 client.27407 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.918+0000 7f6f32a00640 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45ea969c0 0x55c45ea98de0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b4a7b4b80a713d server_cookie=d6181f03badb6f41 in_seq=0 out_seq=0 2024-09-21T07:00:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f13e00640 1 -- 172.21.15.114:0/337534763 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c45e9810b0 con 0x55c45ea969c0 2024-09-21T07:00:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f13e00640 10 client.27407 handle_command_reply: tid=0 2024-09-21T07:00:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f405ea740 1 client.27407 shutdown 2024-09-21T07:00:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 msgr2=0x55c45ea20150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f405ea740 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea20150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0108b0 tx=0x7f6f2c004a40 comp rx=0 tx=0).stop 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 shutdown_connections 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.930+0000 7f6f405ea740 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c45e6484d0 0x55c45ea20150 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f405ea740 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c45ea969c0 0x55c45ea98de0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f405ea740 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c45e6402f0 0x55c45ea1fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f405ea740 1 --2- 172.21.15.114:0/337534763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c45e627d80 0x55c45ea1f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 >> 172.21.15.114:0/337534763 conn(0x55c45e65d300 msgr2=0x55c45ea1a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f13e00640 0 client.27407 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 shutdown_connections 2024-09-21T07:00:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.931+0000 7f6f405ea740 1 -- 172.21.15.114:0/337534763 wait complete. 2024-09-21T07:00:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.932+0000 7f6f405ea740 20 client.27407 trim_cache size 0 max 16384 2024-09-21T07:00:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:00:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:00:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:00:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6f180b6800 msgr2=0x7f6f180b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6f180b6800 0x7f6f180b8d10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005c40 tx=0x7f6f2c005bd0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 msgr2=0x7f6f341a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 0x7f6f341a16d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400e9c0 tx=0x7f6f2400ee80 comp rx=0 tx=0).stop 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 shutdown_connections 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6f341165a0 0x7f6f341a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f3410aef0 0x7f6f341a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6f180b6800 0x7f6f180b8d10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 --2- 172.21.15.114:0/353523247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f3410a5a0 0x7f6f341a16d0 secure :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400e9c0 tx=0x7f6f2400ee80 comp rx=0 tx=0).stop 2024-09-21T07:00:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.934+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 >> 172.21.15.114:0/353523247 conn(0x7f6f34070dd0 msgr2=0x7f6f3410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:00:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.935+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 shutdown_connections 2024-09-21T07:00:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:00:59.935+0000 7f6f39600640 1 -- 172.21.15.114:0/353523247 wait complete. 2024-09-21T07:01:00.032 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:01:01.188 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:00 smithi114 ceph-mon[30654]: pgmap v602: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 494 op/s 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 -- 172.21.15.114:0/3817957127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc116510 msgr2=0x7f9bdc118990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 --2- 172.21.15.114:0/3817957127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc116510 0x7f9bdc118990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009be0 tx=0x7f9bd402a6e0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 -- 172.21.15.114:0/3817957127 shutdown_connections 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 --2- 172.21.15.114:0/3817957127 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc116510 0x7f9bdc118990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 --2- 172.21.15.114:0/3817957127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc10af90 0x7f9bdc115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 --2- 172.21.15.114:0/3817957127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10a640 0x7f9bdc10aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.185+0000 7f9be2a00640 1 -- 172.21.15.114:0/3817957127 >> 172.21.15.114:0/3817957127 conn(0x7f9bdc06e9f0 msgr2=0x7f9bdc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.186+0000 7f9be2a00640 1 -- 172.21.15.114:0/3817957127 shutdown_connections 2024-09-21T07:01:01.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.186+0000 7f9be2a00640 1 -- 172.21.15.114:0/3817957127 wait complete. 2024-09-21T07:01:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.187+0000 7f9be2a00640 1 Processor -- start 2024-09-21T07:01:01.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.188+0000 7f9be2a00640 1 -- start start 2024-09-21T07:01:01.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.190+0000 7f9be2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc10a640 0x7f9bdc1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.190+0000 7f9be2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 0x7f9bdc1a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc116510 0x7f9bdc1a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc10a640 0x7f9bdc1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc10a640 0x7f9bdc1a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:45452/0 (socket says 172.21.15.114:45452) 2024-09-21T07:01:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 0x7f9bdc1a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 0x7f9bdc1a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40298/0 (socket says 172.21.15.114:40298) 2024-09-21T07:01:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc11c0b0 con 0x7f9bdc10a640 2024-09-21T07:01:01.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be1600640 1 -- 172.21.15.114:0/1732680928 learned_addr learned my addr 172.21.15.114:0/1732680928 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.191+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc11bf30 con 0x7f9bdc116510 2024-09-21T07:01:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.192+0000 7f9be2000640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc116510 0x7f9bdc1a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.192+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc11c230 con 0x7f9bdc10af90 2024-09-21T07:01:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.193+0000 7f9be0c00640 1 -- 172.21.15.114:0/1732680928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc116510 msgr2=0x7f9bdc1a85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.193+0000 7f9be0c00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc116510 0x7f9bdc1a85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.193+0000 7f9be0c00640 1 -- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc10a640 msgr2=0x7f9bdc1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.193+0000 7f9be0c00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc10a640 0x7f9bdc1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.193+0000 7f9be0c00640 1 -- 172.21.15.114:0/1732680928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bdc1a2390 con 0x7f9bdc10af90 2024-09-21T07:01:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.193+0000 7f9be2000640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc116510 0x7f9bdc1a85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:01.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.194+0000 7f9be0c00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 0x7f9bdc1a1ac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc06c910 tx=0x7f9bc800db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.195+0000 7f9bda000640 1 -- 172.21.15.114:0/1732680928 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8017070 con 0x7f9bdc10af90 2024-09-21T07:01:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.195+0000 7f9bda000640 1 -- 172.21.15.114:0/1732680928 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc8009820 con 0x7f9bdc10af90 2024-09-21T07:01:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.195+0000 7f9bda000640 1 -- 172.21.15.114:0/1732680928 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8004b90 con 0x7f9bdc10af90 2024-09-21T07:01:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.195+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc1a8c40 con 0x7f9bdc10af90 2024-09-21T07:01:01.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.195+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc1a9150 con 0x7f9bdc10af90 2024-09-21T07:01:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.204+0000 7f9bda000640 1 -- 172.21.15.114:0/1732680928 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9bc8004d30 con 0x7f9bdc10af90 2024-09-21T07:01:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.204+0000 7f9beac43740 1 Processor -- start 2024-09-21T07:01:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.205+0000 7f9beac43740 1 -- start start 2024-09-21T07:01:01.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.205+0000 7f9beac43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9be2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9be2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e8ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40302/0 (socket says 172.21.15.114:40302) 2024-09-21T07:01:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9be2000640 1 -- 172.21.15.114:0/2483230811 learned_addr learned my addr 172.21.15.114:0/2483230811 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:01.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9beac43740 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6abd490 0x5601a6e8e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9beac43740 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ac5670 0x5601a6e8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601a6e03e90 con 0x5601a6abd490 2024-09-21T07:01:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5601a6df32c0 con 0x5601a6ac5670 2024-09-21T07:01:01.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.206+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5601a6df70d0 con 0x5601a6ae6350 2024-09-21T07:01:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.207+0000 7f9be1600640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6abd490 0x5601a6e8e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.208+0000 7f9be0c00640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ac5670 0x5601a6e8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.210+0000 7f9bda000640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bc40b6850 0x7f9bc40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.210+0000 7f9be2000640 1 -- 172.21.15.114:0/2483230811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ac5670 msgr2=0x5601a6e8e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.210+0000 7f9bda000640 1 -- 172.21.15.114:0/1732680928 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bc80e7280 con 0x7f9bdc10af90 2024-09-21T07:01:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.210+0000 7f9be2000640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ac5670 0x5601a6e8e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.210+0000 7f9be2000640 1 -- 172.21.15.114:0/2483230811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6abd490 msgr2=0x5601a6e8e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.212+0000 7f9be2000640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6abd490 0x5601a6e8e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.212+0000 7f9be2000640 1 -- 172.21.15.114:0/2483230811 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a6b51930 con 0x5601a6ae6350 2024-09-21T07:01:01.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.213+0000 7f9be0c00640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ac5670 0x5601a6e8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:01.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.213+0000 7f9be2000640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e8ded0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009be0 tx=0x7f9bd40307e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d92f6e500d34fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:01.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.213+0000 7f9be1600640 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6abd490 0x5601a6e8e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:01.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.213+0000 7f9bce600640 1 -- 172.21.15.114:0/2483230811 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4043070 con 0x5601a6ae6350 2024-09-21T07:01:01.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.214+0000 7f9bce600640 1 -- 172.21.15.114:0/2483230811 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd4030bf0 con 0x5601a6ae6350 2024-09-21T07:01:01.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.214+0000 7f9bce600640 1 -- 172.21.15.114:0/2483230811 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4036040 con 0x5601a6ae6350 2024-09-21T07:01:01.216 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.214+0000 7f9be2000640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bc40b6850 0x7f9bc40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.215+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 msgr2=0x5601a6e8ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.217 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.215+0000 7f9beac43740 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e8ded0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009be0 tx=0x7f9bd40307e0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.217+0000 7f9be2000640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bc40b6850 0x7f9bc40b8d60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9bd403dc40 tx=0x7f9bd404a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:01.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.217+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 shutdown_connections 2024-09-21T07:01:01.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.217+0000 7f9beac43740 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ac5670 0x5601a6e8e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.217+0000 7f9beac43740 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6abd490 0x5601a6e8e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.217+0000 7f9beac43740 1 --2- 172.21.15.114:0/2483230811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e8ded0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.217+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 >> 172.21.15.114:0/2483230811 conn(0x5601a6ada740 msgr2=0x5601a6e8bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:01.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.218+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 shutdown_connections 2024-09-21T07:01:01.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.218+0000 7f9beac43740 1 -- 172.21.15.114:0/2483230811 wait complete. 2024-09-21T07:01:01.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.219+0000 7f9beac43740 1 -- start start 2024-09-21T07:01:01.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.219+0000 7f9beac43740 1 Processor -- start 2024-09-21T07:01:01.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.222+0000 7f9beac43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 0x5601a6e99a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.222+0000 7f9beac43740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e99f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.223+0000 7f9be1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 0x5601a6e99a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.223+0000 7f9be1600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 0x5601a6e99a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40306/0 (socket says 172.21.15.114:40306) 2024-09-21T07:01:01.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.223+0000 7f9be1600640 1 -- 172.21.15.114:0/3165896505 learned_addr learned my addr 172.21.15.114:0/3165896505 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:01.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.224+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ae6350 0x5601a6ea5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.224+0000 7f9be0c00640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e99f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.224+0000 7f9be2000640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ae6350 0x5601a6ea5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.224+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5601a6c54890 con 0x5601a6ac5670 2024-09-21T07:01:01.226 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.225+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5601a6d8e160 con 0x5601a6ae6350 2024-09-21T07:01:01.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.225+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5601a6e9c650 con 0x5601a6abd490 2024-09-21T07:01:01.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.227+0000 7f9beac43740 10 client.0 _ll_register_callbacks cb 0x5601a6daaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:01.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.228+0000 7f9be0c00640 1 -- 172.21.15.114:0/3165896505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 msgr2=0x5601a6e99a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.228+0000 7f9be0c00640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 0x5601a6e99a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.229+0000 7f9be0c00640 1 -- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ae6350 msgr2=0x5601a6ea5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.229+0000 7f9be0c00640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ae6350 0x5601a6ea5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.229+0000 7f9be0c00640 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a6ea5eb0 con 0x5601a6ac5670 2024-09-21T07:01:01.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.230+0000 7f9be2000640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ae6350 0x5601a6ea5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:01.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.230+0000 7f9be1600640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 0x5601a6e99a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:01.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.231+0000 7f9be0c00640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e99f60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80e4600 tx=0x7f9bc80a92a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e28b21d8ae93743 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:01.232 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.231+0000 7f9bc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:01:01.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.231+0000 7f9bc2a00640 1 -- 172.21.15.114:0/3165896505 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8020040 con 0x5601a6ac5670 2024-09-21T07:01:01.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.231+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd4030bf0 con 0x5601a6ac5670 2024-09-21T07:01:01.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.231+0000 7f9bc2a00640 1 -- 172.21.15.114:0/3165896505 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bc80d02c0 con 0x5601a6ac5670 2024-09-21T07:01:01.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.233+0000 7f9bc2a00640 1 -- 172.21.15.114:0/3165896505 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9bc80a9400 con 0x5601a6ac5670 2024-09-21T07:01:01.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.233+0000 7f9beac43740 10 client.17055 fetch_fsmap learned FSMap version 43 2024-09-21T07:01:01.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.233+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5601a6ea86f0 con 0x5601a6ac5670 2024-09-21T07:01:01.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.234+0000 7f9bc2a00640 1 -- 172.21.15.114:0/3165896505 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd4030bf0 con 0x5601a6ac5670 2024-09-21T07:01:01.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9bc2a00640 1 -- 172.21.15.114:0/3165896505 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9bc80d40d0 con 0x5601a6ac5670 2024-09-21T07:01:01.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9beac43740 10 client.17055 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:01:01.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9beac43740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:01.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9beac43740 20 client.17055 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6aa4cd0 0x5601a6eabbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9beac43740 4 client.17055 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601a6e05de0 con 0x5601a6aa4cd0 2024-09-21T07:01:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.236+0000 7f9be1600640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6aa4cd0 0x5601a6eabbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.237+0000 7f9be1600640 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6aa4cd0 0x5601a6eabbc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8f8fe06221d525 server_cookie=a080a97bbbede206 in_seq=0 out_seq=0 2024-09-21T07:01:01.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.238+0000 7f9bc2a00640 10 client.17055 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:01.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.239+0000 7f9bc2a00640 1 -- 172.21.15.114:0/3165896505 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601a6e05de0 con 0x5601a6aa4cd0 2024-09-21T07:01:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.239+0000 7f9bc2a00640 10 client.17055 handle_command_reply: tid=0 2024-09-21T07:01:01.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.239+0000 7f9beac43740 1 client.17055 shutdown 2024-09-21T07:01:01.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 msgr2=0x5601a6e99f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e99f60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80e4600 tx=0x7f9bc80a92a0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 shutdown_connections 2024-09-21T07:01:01.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6ae6350 0x5601a6ea5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e99f60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6aa4cd0 0x5601a6eabbc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 --2- 172.21.15.114:0/3165896505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6abd490 0x5601a6e99a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.240+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 >> 172.21.15.114:0/3165896505 conn(0x5601a6ada740 msgr2=0x5601a6e8ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:01.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.241+0000 7f9bc2a00640 0 client.17055 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.241+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 shutdown_connections 2024-09-21T07:01:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.243+0000 7f9beac43740 1 -- 172.21.15.114:0/3165896505 wait complete. 2024-09-21T07:01:01.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.243+0000 7f9beac43740 20 client.17055 trim_cache size 0 max 16384 2024-09-21T07:01:01.253 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:00 smithi183 ceph-mon[34219]: pgmap v602: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 494 op/s 2024-09-21T07:01:01.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.279+0000 7f9beac43740 1 Processor -- start 2024-09-21T07:01:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 -- start start 2024-09-21T07:01:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6ea5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc1c7fc0 con 0x5601a6ac5670 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc1c9b60 con 0x5601a6abd490 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.280+0000 7f9beac43740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc1c8690 con 0x5601a6ae6350 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.281+0000 7f9be2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.281+0000 7f9be2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e9b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40322/0 (socket says 172.21.15.114:40322) 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.281+0000 7f9be2000640 1 -- 172.21.15.114:0/3547239366 learned_addr learned my addr 172.21.15.114:0/3547239366 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.281+0000 7f9be1600640 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6ea5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.281+0000 7f9be0c00640 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.282+0000 7f9be2000640 1 -- 172.21.15.114:0/3547239366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 msgr2=0x5601a6ea5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.282+0000 7f9be2000640 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6ea5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.282+0000 7f9be2000640 1 -- 172.21.15.114:0/3547239366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 msgr2=0x5601a6e990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.282+0000 7f9be2000640 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.282+0000 7f9be2000640 1 -- 172.21.15.114:0/3547239366 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a6e8bd00 con 0x5601a6ae6350 2024-09-21T07:01:01.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.286+0000 7f9be2000640 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e9b450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4034040 tx=0x7f9bd4052000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f091d0b4287ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:01.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.286+0000 7f9bcdc00640 1 -- 172.21.15.114:0/3547239366 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4043070 con 0x5601a6ae6350 2024-09-21T07:01:01.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.287+0000 7f9bcdc00640 1 -- 172.21.15.114:0/3547239366 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bd4003e20 con 0x5601a6ae6350 2024-09-21T07:01:01.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.287+0000 7f9bcdc00640 1 -- 172.21.15.114:0/3547239366 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4036040 con 0x5601a6ae6350 2024-09-21T07:01:01.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 -- 172.21.15.114:0/3547239366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 msgr2=0x5601a6e9b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e9b450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4034040 tx=0x7f9bd4052000 comp rx=0 tx=0).stop 2024-09-21T07:01:01.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 -- 172.21.15.114:0/3547239366 shutdown_connections 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ae6350 0x5601a6e9b450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ac5670 0x5601a6e990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 --2- 172.21.15.114:0/3547239366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6ea5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 -- 172.21.15.114:0/3547239366 >> 172.21.15.114:0/3547239366 conn(0x5601a6ada740 msgr2=0x5601a6e9ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 -- 172.21.15.114:0/3547239366 shutdown_connections 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.291+0000 7f9beac43740 1 -- 172.21.15.114:0/3547239366 wait complete. 2024-09-21T07:01:01.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.292+0000 7f9beac43740 1 -- start start 2024-09-21T07:01:01.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.292+0000 7f9beac43740 1 Processor -- start 2024-09-21T07:01:01.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.293+0000 7f9beac43740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6e9da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.293+0000 7f9beac43740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 0x5601a6e9dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9beac43740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ae6350 0x5601a6e9e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9be1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6e9da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 0x5601a6e9dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9be1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6e9da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41532/0 (socket says 172.21.15.114:41532) 2024-09-21T07:01:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 0x5601a6e9dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40326/0 (socket says 172.21.15.114:40326) 2024-09-21T07:01:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9be0c00640 1 -- 172.21.15.114:0/4256466198 learned_addr learned my addr 172.21.15.114:0/4256466198 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:01.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc1c9460 con 0x5601a6ae6350 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc1c8d60 con 0x5601a6abd490 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc1c6050 con 0x5601a6ac5670 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.294+0000 7f9be2000640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ae6350 0x5601a6e9e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be0c00640 1 -- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 msgr2=0x5601a6e9da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be0c00640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6e9da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be0c00640 1 -- 172.21.15.114:0/4256466198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ae6350 msgr2=0x5601a6e9e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:01.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be0c00640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ae6350 0x5601a6e9e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be0c00640 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a6f15780 con 0x5601a6ac5670 2024-09-21T07:01:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be2000640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ae6350 0x5601a6e9e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9beac43740 10 client.0 _ll_register_callbacks cb 0x7f9bc80057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.295+0000 7f9be0c00640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 0x5601a6e9dfb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80acd40 tx=0x7f9bc80acbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca5b8ca557a1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.296+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd4003e20 con 0x5601a6ac5670 2024-09-21T07:01:01.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.296+0000 7f9bc2a00640 10 client.27346 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.296+0000 7f9bc2a00640 1 -- 172.21.15.114:0/4256466198 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8020070 con 0x5601a6ac5670 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.296+0000 7f9bc2a00640 1 -- 172.21.15.114:0/4256466198 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f9bc80d00b0 con 0x5601a6ac5670 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.296+0000 7f9bc2a00640 1 -- 172.21.15.114:0/4256466198 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9bc80d02b0 con 0x5601a6ac5670 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.297+0000 7f9beac43740 10 client.27346 fetch_fsmap learned FSMap version 43 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.297+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5601a6f168d0 con 0x5601a6ac5670 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.297+0000 7f9bc2a00640 1 -- 172.21.15.114:0/4256466198 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f9bc8005220 con 0x5601a6ac5670 2024-09-21T07:01:01.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.298+0000 7f9beac43740 10 client.27346 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:01:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.298+0000 7f9beac43740 10 client.27346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.298+0000 7f9beac43740 20 client.27346 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.298+0000 7f9beac43740 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6a7a080 0x5601a6f18e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:01.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.298+0000 7f9be1600640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6a7a080 0x5601a6f18e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.300+0000 7f9beac43740 4 client.27346 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:01:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.300+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5601a6e01890 con 0x5601a6a7a080 2024-09-21T07:01:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.300+0000 7f9be1600640 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6a7a080 0x5601a6f18e20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7078e43e5ff484d server_cookie=9f6a68a2084fd58a in_seq=0 out_seq=0 2024-09-21T07:01:01.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.300+0000 7f9bc2a00640 10 client.27346 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9bc2a00640 1 -- 172.21.15.114:0/4256466198 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5601a6e01890 con 0x5601a6a7a080 2024-09-21T07:01:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9bc2a00640 10 client.27346 handle_command_reply: tid=0 2024-09-21T07:01:01.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 client.27346 shutdown 2024-09-21T07:01:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 msgr2=0x5601a6e9dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 0x5601a6e9dfb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80acd40 tx=0x7f9bc80acbf0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 shutdown_connections 2024-09-21T07:01:01.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5601a6ae6350 0x5601a6e9e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5601a6a7a080 0x5601a6f18e20 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5601a6ac5670 0x5601a6e9dfb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 --2- 172.21.15.114:0/4256466198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5601a6abd490 0x5601a6e9da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9bc2a00640 0 client.27346 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.303+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 >> 172.21.15.114:0/4256466198 conn(0x5601a6ada740 msgr2=0x5601a6e9acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.304+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 shutdown_connections 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.304+0000 7f9beac43740 1 -- 172.21.15.114:0/4256466198 wait complete. 2024-09-21T07:01:01.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.304+0000 7f9beac43740 20 client.27346 trim_cache size 0 max 16384 2024-09-21T07:01:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:01:01.307 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:01:01.307 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "c60e65cc-74ed-40b1-b950-c790a95d6c3e", 2024-09-21T07:01:01.307 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:01:01.307 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:01:01.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.305+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bc40b6850 msgr2=0x7f9bc40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.305+0000 7f9be2a00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bc40b6850 0x7f9bc40b8d60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9bd403dc40 tx=0x7f9bd404a040 comp rx=0 tx=0).stop 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.305+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 msgr2=0x7f9bdc1a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.305+0000 7f9be2a00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 0x7f9bdc1a1ac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc06c910 tx=0x7f9bc800db30 comp rx=0 tx=0).stop 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.306+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 shutdown_connections 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.306+0000 7f9be2a00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9bc40b6850 0x7f9bc40b8d60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.306+0000 7f9be2a00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bdc116510 0x7f9bdc1a85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.306+0000 7f9be2a00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9bdc10af90 0x7f9bdc1a1ac0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.306+0000 7f9be2a00640 1 --2- 172.21.15.114:0/1732680928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9bdc10a640 0x7f9bdc1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:01.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.306+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 >> 172.21.15.114:0/1732680928 conn(0x7f9bdc06e9f0 msgr2=0x7f9bdc073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.309+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 shutdown_connections 2024-09-21T07:01:01.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:01.309+0000 7f9be2a00640 1 -- 172.21.15.114:0/1732680928 wait complete. 2024-09-21T07:01:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:00 smithi193 ceph-mon[34601]: pgmap v602: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 10 MiB/s wr, 494 op/s 2024-09-21T07:01:02.249 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:01 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:01:02.250 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:02.250 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:01:02.250 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:01 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:02.250 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:01:02.314 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:01:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:01 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:01:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:01:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:01 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:02.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:01:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:01 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:01:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:01:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:01 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:01:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:01:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 385.0 s, 548.4 tps, lat 183.139 ms stddev 247.277 2024-09-21T07:01:03.058 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:01:03.233 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: pgmap v603: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 509 op/s 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.234 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:01:03.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: pgmap v603: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 509 op/s 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: pgmap v603: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 509 op/s 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:04.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:03 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:04.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:01:04.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:03 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:01:04.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:03 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:01:04.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:03 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:04.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:01:04.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:03 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:01:04.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:03 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:01:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:03 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:01:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:01:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:03 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:01:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:03 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:01:05.440 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:05 smithi114 ceph-mon[30654]: pgmap v604: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 821 op/s 2024-09-21T07:01:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:05 smithi183 ceph-mon[34219]: pgmap v604: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 821 op/s 2024-09-21T07:01:05.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:05 smithi193 ceph-mon[34601]: pgmap v604: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 14 MiB/s wr, 821 op/s 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.694+0000 7f502a400640 1 -- 172.21.15.114:0/930634742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 msgr2=0x7f50241085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.694+0000 7f502a400640 1 --2- 172.21.15.114:0/930634742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f50241085c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f501000b9f0 tx=0x7f501002c0b0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 -- 172.21.15.114:0/930634742 shutdown_connections 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 --2- 172.21.15.114:0/930634742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5024108b00 0x7f502410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 --2- 172.21.15.114:0/930634742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f50241085c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 --2- 172.21.15.114:0/930634742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 0x7f5024102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 -- 172.21.15.114:0/930634742 >> 172.21.15.114:0/930634742 conn(0x7f50240fe2c0 msgr2=0x7f5024100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:05.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 -- 172.21.15.114:0/930634742 shutdown_connections 2024-09-21T07:01:05.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.695+0000 7f502a400640 1 -- 172.21.15.114:0/930634742 wait complete. 2024-09-21T07:01:05.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.699+0000 7f502a400640 1 Processor -- start 2024-09-21T07:01:05.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.700+0000 7f502a400640 1 -- start start 2024-09-21T07:01:05.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.701+0000 7f502a400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 0x7f50241980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.701+0000 7f502a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f5024198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f502a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5024108b00 0x7f502419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f502a400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5024111700 con 0x7f5024102870 2024-09-21T07:01:05.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f5023e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f5024198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f5023e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f5024198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52724/0 (socket says 172.21.15.114:52724) 2024-09-21T07:01:05.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f5023e00640 1 -- 172.21.15.114:0/2688504437 learned_addr learned my addr 172.21.15.114:0/2688504437 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:05.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f5029a00640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5024108b00 0x7f502419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5024111580 con 0x7f50241081b0 2024-09-21T07:01:05.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.702+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5024111880 con 0x7f5024108b00 2024-09-21T07:01:05.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.703+0000 7f5029000640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 0x7f50241980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.704+0000 7f5029000640 1 -- 172.21.15.114:0/2688504437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5024108b00 msgr2=0x7f502419d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.704+0000 7f5029000640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5024108b00 0x7f502419d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.704+0000 7f5029000640 1 -- 172.21.15.114:0/2688504437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 msgr2=0x7f5024198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.704+0000 7f5029000640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f5024198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.704+0000 7f5029000640 1 -- 172.21.15.114:0/2688504437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502419e060 con 0x7f5024102870 2024-09-21T07:01:05.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.706+0000 7f5029000640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 0x7f50241980f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f501400be10 tx=0x7f501400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:05.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.707+0000 7f5021600640 1 -- 172.21.15.114:0/2688504437 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014018070 con 0x7f5024102870 2024-09-21T07:01:05.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.709+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502419e350 con 0x7f5024102870 2024-09-21T07:01:05.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.709+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a4ab0 con 0x7f5024102870 2024-09-21T07:01:05.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.709+0000 7f5021600640 1 -- 172.21.15.114:0/2688504437 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5014009c30 con 0x7f5024102870 2024-09-21T07:01:05.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.709+0000 7f5021600640 1 -- 172.21.15.114:0/2688504437 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014012e30 con 0x7f5024102870 2024-09-21T07:01:05.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.720+0000 7f5032668740 1 Processor -- start 2024-09-21T07:01:05.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.720+0000 7f5032668740 1 -- start start 2024-09-21T07:01:05.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.722+0000 7f5032668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b182414740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.722+0000 7f5032668740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b182414c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.722+0000 7f5032668740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 0x55b1824151c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.722+0000 7f5032668740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b182382f70 con 0x55b18203c1b0 2024-09-21T07:01:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.722+0000 7f5032668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b1823723a0 con 0x55b182065070 2024-09-21T07:01:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.722+0000 7f5032668740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b1823761b0 con 0x55b182044390 2024-09-21T07:01:05.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.725+0000 7f5029a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b182414740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.725+0000 7f5023e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 0x55b1824151c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.725+0000 7f5023e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 0x55b1824151c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54904/0 (socket says 172.21.15.114:54904) 2024-09-21T07:01:05.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.725+0000 7f5023e00640 1 -- 172.21.15.114:0/3825405601 learned_addr learned my addr 172.21.15.114:0/3825405601 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:05.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.726+0000 7f5023e00640 1 -- 172.21.15.114:0/3825405601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 msgr2=0x55b182414740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.726+0000 7f5023e00640 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b182414740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.726+0000 7f5023e00640 1 -- 172.21.15.114:0/3825405601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 msgr2=0x55b182414c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:01:05.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.726+0000 7f5021600640 1 -- 172.21.15.114:0/2688504437 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5014025440 con 0x7f5024102870 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.726+0000 7f5023e00640 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b182414c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.726+0000 7f5023e00640 1 -- 172.21.15.114:0/3825405601 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b18233cc90 con 0x55b182044390 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.727+0000 7f5029a00640 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b182414740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.727+0000 7f5023e00640 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 0x55b1824151c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5010000c00 tx=0x7f5010005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8f3967edc6fecbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.728+0000 7f501a000640 1 -- 172.21.15.114:0/3825405601 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010043030 con 0x55b182044390 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.728+0000 7f501a000640 1 -- 172.21.15.114:0/3825405601 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501002cc00 con 0x55b182044390 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.728+0000 7f501a000640 1 -- 172.21.15.114:0/3825405601 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010039040 con 0x55b182044390 2024-09-21T07:01:05.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.729+0000 7f5021600640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50080b6730 0x7f50080b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.730+0000 7f5021600640 1 -- 172.21.15.114:0/2688504437 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f50140e71d0 con 0x7f5024102870 2024-09-21T07:01:05.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.731+0000 7f5032668740 1 -- 172.21.15.114:0/3825405601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 msgr2=0x55b1824151c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.731+0000 7f5032668740 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 0x55b1824151c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5010000c00 tx=0x7f5010005e70 comp rx=0 tx=0).stop 2024-09-21T07:01:05.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.731+0000 7f5029a00640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50080b6730 0x7f50080b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.733+0000 7f5029a00640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50080b6730 0x7f50080b8c40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f501c005fd0 tx=0x7f501c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 -- 172.21.15.114:0/3825405601 shutdown_connections 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182044390 0x55b1824151c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b182414c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 --2- 172.21.15.114:0/3825405601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b182414740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 -- 172.21.15.114:0/3825405601 >> 172.21.15.114:0/3825405601 conn(0x55b182059460 msgr2=0x55b182412630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 -- 172.21.15.114:0/3825405601 shutdown_connections 2024-09-21T07:01:05.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.734+0000 7f5032668740 1 -- 172.21.15.114:0/3825405601 wait complete. 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.735+0000 7f5032668740 1 -- start start 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.735+0000 7f5032668740 1 Processor -- start 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.737+0000 7f5032668740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b18241ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.737+0000 7f5032668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b1824204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.737+0000 7f5032668740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182065070 0x55b18242c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5023e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b1824204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5032668740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b1821d3700 con 0x55b18203c1b0 2024-09-21T07:01:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5032668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b18230d540 con 0x55b182044390 2024-09-21T07:01:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5029a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182065070 0x55b18242c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5029a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182065070 0x55b18242c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54914/0 (socket says 172.21.15.114:54914) 2024-09-21T07:01:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5029a00640 1 -- 172.21.15.114:0/918064662 learned_addr learned my addr 172.21.15.114:0/918064662 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b182422ec0 con 0x55b182065070 2024-09-21T07:01:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.738+0000 7f5032668740 10 client.0 _ll_register_callbacks cb 0x55b18231d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:05.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.739+0000 7f5029000640 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b18241ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.739+0000 7f5023e00640 1 -- 172.21.15.114:0/918064662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182065070 msgr2=0x55b18242c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.739+0000 7f5023e00640 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182065070 0x55b18242c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.739+0000 7f5023e00640 1 -- 172.21.15.114:0/918064662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 msgr2=0x55b18241ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.739+0000 7f5023e00640 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b18241ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.739+0000 7f5023e00640 1 -- 172.21.15.114:0/918064662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1820d0650 con 0x55b182044390 2024-09-21T07:01:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.740+0000 7f5023e00640 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b1824204b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f501000e090 tx=0x7f501002cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9312e645af4e82fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.740+0000 7f500ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:01:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.740+0000 7f500ea00640 1 -- 172.21.15.114:0/918064662 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010039030 con 0x55b182044390 2024-09-21T07:01:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.740+0000 7f500ea00640 1 -- 172.21.15.114:0/918064662 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f5010004040 con 0x55b182044390 2024-09-21T07:01:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.740+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b18230fb50 con 0x55b182044390 2024-09-21T07:01:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.741+0000 7f500ea00640 1 -- 172.21.15.114:0/918064662 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5010004240 con 0x55b182044390 2024-09-21T07:01:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.741+0000 7f5032668740 10 client.27431 fetch_fsmap learned FSMap version 43 2024-09-21T07:01:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.741+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b18232a910 con 0x55b182044390 2024-09-21T07:01:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.741+0000 7f500ea00640 1 -- 172.21.15.114:0/918064662 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b18230fb50 con 0x55b182044390 2024-09-21T07:01:05.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f500ea00640 1 -- 172.21.15.114:0/918064662 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f5010051030 con 0x55b182044390 2024-09-21T07:01:05.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f5032668740 10 client.27431 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:01:05.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f5032668740 10 client.27431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:05.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f5032668740 20 client.27431 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:05.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f5032668740 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b1820239f0 0x55b182431eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f5032668740 4 client.27431 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:05.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.743+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b182384ec0 con 0x55b1820239f0 2024-09-21T07:01:05.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.744+0000 7f5029000640 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b1820239f0 0x55b182431eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.744+0000 7f500ea00640 10 client.27431 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:05.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.744+0000 7f5029000640 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b1820239f0 0x55b182431eb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ade2fefd6e7274 server_cookie=a6a0cc94f32e1a2e in_seq=0 out_seq=0 2024-09-21T07:01:05.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.745+0000 7f500ea00640 1 -- 172.21.15.114:0/918064662 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b182384ec0 con 0x55b1820239f0 2024-09-21T07:01:05.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.745+0000 7f500ea00640 10 client.27431 handle_command_reply: tid=0 2024-09-21T07:01:05.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.745+0000 7f5032668740 1 client.27431 shutdown 2024-09-21T07:01:05.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 msgr2=0x55b1824204b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b1824204b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f501000e090 tx=0x7f501002cf30 comp rx=0 tx=0).stop 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 shutdown_connections 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b1820239f0 0x55b182431eb0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b182065070 0x55b18242c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b1824204b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 --2- 172.21.15.114:0/918064662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b18203c1b0 0x55b18241ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f500ea00640 0 client.27431 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.746+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 >> 172.21.15.114:0/918064662 conn(0x55b182059460 msgr2=0x55b1824085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.747+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 shutdown_connections 2024-09-21T07:01:05.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.747+0000 7f5032668740 1 -- 172.21.15.114:0/918064662 wait complete. 2024-09-21T07:01:05.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.747+0000 7f5032668740 20 client.27431 trim_cache size 0 max 16384 2024-09-21T07:01:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.784+0000 7f5032668740 1 Processor -- start 2024-09-21T07:01:05.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.784+0000 7f5032668740 1 -- start start 2024-09-21T07:01:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.785+0000 7f5032668740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182420390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.785+0000 7f5032668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b182412130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.785+0000 7f5032668740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182065070 0x55b182421e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.785+0000 7f5032668740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f50241ae900 con 0x55b182065070 2024-09-21T07:01:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.785+0000 7f5032668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50241b04a0 con 0x55b182044390 2024-09-21T07:01:05.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.785+0000 7f5032668740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50241aefd0 con 0x55b18203c1b0 2024-09-21T07:01:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.786+0000 7f5029000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182420390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.786+0000 7f5029000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182420390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54922/0 (socket says 172.21.15.114:54922) 2024-09-21T07:01:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.786+0000 7f5029000640 1 -- 172.21.15.114:0/2561416163 learned_addr learned my addr 172.21.15.114:0/2561416163 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.786+0000 7f5023e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b182412130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.787+0000 7f5029000640 1 -- 172.21.15.114:0/2561416163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 msgr2=0x55b182412130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.787+0000 7f5029000640 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b182412130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.787+0000 7f5029000640 1 -- 172.21.15.114:0/2561416163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182065070 msgr2=0x55b182421e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.787+0000 7f5029000640 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182065070 0x55b182421e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.787+0000 7f5029000640 1 -- 172.21.15.114:0/2561416163 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b182421230 con 0x55b18203c1b0 2024-09-21T07:01:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.787+0000 7f5023e00640 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b182412130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:05.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.788+0000 7f5029000640 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182420390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f501400dfd0 tx=0x7f50140e7420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9125d65c51a3acab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:05.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.790+0000 7f5019600640 1 -- 172.21.15.114:0/2561416163 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140ad350 con 0x55b18203c1b0 2024-09-21T07:01:05.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.793+0000 7f5019600640 1 -- 172.21.15.114:0/2561416163 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50140b1730 con 0x55b18203c1b0 2024-09-21T07:01:05.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.793+0000 7f5019600640 1 -- 172.21.15.114:0/2561416163 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140e49d0 con 0x55b18203c1b0 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 -- 172.21.15.114:0/2561416163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 msgr2=0x55b182420390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182420390 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f501400dfd0 tx=0x7f50140e7420 comp rx=0 tx=0).stop 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 -- 172.21.15.114:0/2561416163 shutdown_connections 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182065070 0x55b182421e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182044390 0x55b182412130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 --2- 172.21.15.114:0/2561416163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182420390 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.794+0000 7f5032668740 1 -- 172.21.15.114:0/2561416163 >> 172.21.15.114:0/2561416163 conn(0x55b182059460 msgr2=0x55b1824085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:05.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.795+0000 7f5032668740 1 -- 172.21.15.114:0/2561416163 shutdown_connections 2024-09-21T07:01:05.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.795+0000 7f5032668740 1 -- 172.21.15.114:0/2561416163 wait complete. 2024-09-21T07:01:05.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.796+0000 7f5032668740 1 -- start start 2024-09-21T07:01:05.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.796+0000 7f5032668740 1 Processor -- start 2024-09-21T07:01:05.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5032668740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182431a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5032668740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182044390 0x55b182432040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5029000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182431a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5029000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182431a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54938/0 (socket says 172.21.15.114:54938) 2024-09-21T07:01:05.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5029000640 1 -- 172.21.15.114:0/2307123664 learned_addr learned my addr 172.21.15.114:0/2307123664 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:05.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b1824957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f50241afda0 con 0x55b182044390 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50241af6a0 con 0x55b182065070 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.798+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f50241ac990 con 0x55b18203c1b0 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5032668740 10 client.0 _ll_register_callbacks cb 0x7f5010005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029000640 1 -- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 msgr2=0x55b1824957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029a00640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b1824957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029000640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b1824957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029000640 1 -- 172.21.15.114:0/2307123664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182044390 msgr2=0x55b182432040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029000640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182044390 0x55b182432040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029000640 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b182495e10 con 0x55b18203c1b0 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.799+0000 7f5029a00640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b1824957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.800+0000 7f5029000640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182431a80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f50140a9410 tx=0x7f50140a90f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7d3847a402f3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.800+0000 7f500ea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.800+0000 7f500ea00640 1 -- 172.21.15.114:0/2307123664 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140b6670 con 0x55b18203c1b0 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.800+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50140b1730 con 0x55b18203c1b0 2024-09-21T07:01:05.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.800+0000 7f500ea00640 1 -- 172.21.15.114:0/2307123664 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9402+0+0 (secure 0 0 0) 0x7f50140d00b0 con 0x55b18203c1b0 2024-09-21T07:01:05.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.800+0000 7f500ea00640 1 -- 172.21.15.114:0/2307123664 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f50140e49b0 con 0x55b18203c1b0 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f5032668740 10 client.27364 fetch_fsmap learned FSMap version 43 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b18249d210 con 0x55b18203c1b0 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f500ea00640 1 -- 172.21.15.114:0/2307123664 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50140b1730 con 0x55b18203c1b0 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f500ea00640 1 -- 172.21.15.114:0/2307123664 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 43) ==== 8378+0+0 (secure 0 0 0) 0x7f50140b1250 con 0x55b18203c1b0 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f5032668740 10 client.27364 fetch_fsmap finished waiting for FSMap version 43 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f5032668740 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f5032668740 20 client.27364 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.801+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b181ff8da0 0x55b18249f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:05.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.802+0000 7f5032668740 4 client.27364 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:01:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.802+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b182380970 con 0x55b181ff8da0 2024-09-21T07:01:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.802+0000 7f5023e00640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b181ff8da0 0x55b18249f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.803+0000 7f500ea00640 10 client.27364 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.803+0000 7f5023e00640 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b181ff8da0 0x55b18249f820 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0e783419b3b536 server_cookie=25fd44e9957a1243 in_seq=0 out_seq=0 2024-09-21T07:01:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.803+0000 7f500ea00640 1 -- 172.21.15.114:0/2307123664 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55b182380970 con 0x55b181ff8da0 2024-09-21T07:01:05.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.803+0000 7f500ea00640 10 client.27364 handle_command_reply: tid=0 2024-09-21T07:01:05.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.804+0000 7f5032668740 1 client.27364 shutdown 2024-09-21T07:01:05.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 msgr2=0x55b182431a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182431a80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f50140a9410 tx=0x7f50140a90f0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 shutdown_connections 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b182065070 0x55b1824957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b182044390 0x55b182432040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b181ff8da0 0x55b18249f820 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 --2- 172.21.15.114:0/2307123664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b18203c1b0 0x55b182431a80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 >> 172.21.15.114:0/2307123664 conn(0x55b182059460 msgr2=0x55b182421b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:05.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.805+0000 7f500ea00640 0 client.27364 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:05.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.807+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 shutdown_connections 2024-09-21T07:01:05.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.807+0000 7f5032668740 1 -- 172.21.15.114:0/2307123664 wait complete. 2024-09-21T07:01:05.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.808+0000 7f5032668740 20 client.27364 trim_cache size 0 max 16384 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "c60e65cc-74ed-40b1-b950-c790a95d6c3e", 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:01:05.812 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.813 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:05.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50080b6730 msgr2=0x7f50080b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50080b6730 0x7f50080b8c40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f501c005fd0 tx=0x7f501c009500 comp rx=0 tx=0).stop 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 msgr2=0x7f50241980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 0x7f50241980f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f501400be10 tx=0x7f501400dd20 comp rx=0 tx=0).stop 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 shutdown_connections 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5024108b00 0x7f502419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f50080b6730 0x7f50080b8c40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50241081b0 0x7f5024198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 --2- 172.21.15.114:0/2688504437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5024102870 0x7f50241980f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.809+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 >> 172.21.15.114:0/2688504437 conn(0x7f50240fe2c0 msgr2=0x7f50241006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.811+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 shutdown_connections 2024-09-21T07:01:05.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:05.811+0000 7f502a400640 1 -- 172.21.15.114:0/2688504437 wait complete. 2024-09-21T07:01:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:07 smithi183 ceph-mon[34219]: pgmap v605: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 725 op/s 2024-09-21T07:01:07.618 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:07 smithi114 ceph-mon[30654]: pgmap v605: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 725 op/s 2024-09-21T07:01:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:07 smithi193 ceph-mon[34601]: pgmap v605: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 10 MiB/s wr, 725 op/s 2024-09-21T07:01:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 390.0 s, 444.6 tps, lat 216.888 ms stddev 275.458 2024-09-21T07:01:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:09 smithi114 ceph-mon[30654]: pgmap v606: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 886 op/s 2024-09-21T07:01:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:09 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:09 smithi183 ceph-mon[34219]: pgmap v606: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 886 op/s 2024-09-21T07:01:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:09 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:09 smithi193 ceph-mon[34601]: pgmap v606: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 13 MiB/s wr, 886 op/s 2024-09-21T07:01:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:09 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:10.305 INFO:tasks.cephfs.filesystem:scrub status for tag:c60e65cc-74ed-40b1-b950-c790a95d6c3e - {'path': '/', 'tag': 'c60e65cc-74ed-40b1-b950-c790a95d6c3e', 'options': 'recursive,force'} 2024-09-21T07:01:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:11 smithi114 ceph-mon[30654]: pgmap v607: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 10 MiB/s wr, 754 op/s 2024-09-21T07:01:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:11 smithi183 ceph-mon[34219]: pgmap v607: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 10 MiB/s wr, 754 op/s 2024-09-21T07:01:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:11 smithi193 ceph-mon[34601]: pgmap v607: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 10 MiB/s wr, 754 op/s 2024-09-21T07:01:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:12 smithi114 ceph-mon[30654]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T07:01:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:12 smithi183 ceph-mon[34219]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T07:01:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:12 smithi193 ceph-mon[34601]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T07:01:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 395.0 s, 726.4 tps, lat 144.500 ms stddev 200.484 2024-09-21T07:01:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.530+0000 7f15a5e00640 1 -- 172.21.15.114:0/2663941207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 msgr2=0x7f15a0102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.530+0000 7f15a5e00640 1 --2- 172.21.15.114:0/2663941207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 0x7f15a0102c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1594009860 tx=0x7f159402a340 comp rx=0 tx=0).stop 2024-09-21T07:01:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 -- 172.21.15.114:0/2663941207 shutdown_connections 2024-09-21T07:01:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 --2- 172.21.15.114:0/2663941207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 --2- 172.21.15.114:0/2663941207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15a0108120 0x7f15a0108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 --2- 172.21.15.114:0/2663941207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 0x7f15a0102c00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 -- 172.21.15.114:0/2663941207 >> 172.21.15.114:0/2663941207 conn(0x7f15a00fe230 msgr2=0x7f15a0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 -- 172.21.15.114:0/2663941207 shutdown_connections 2024-09-21T07:01:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.532+0000 7f15a5e00640 1 -- 172.21.15.114:0/2663941207 wait complete. 2024-09-21T07:01:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.534+0000 7f15a5e00640 1 Processor -- start 2024-09-21T07:01:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.535+0000 7f15a5e00640 1 -- start start 2024-09-21T07:01:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 0x7f15a019c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15a0108120 0x7f15a019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a01a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0111630 con 0x7f15a0108120 2024-09-21T07:01:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01114b0 con 0x7f15a0108a70 2024-09-21T07:01:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01117b0 con 0x7f15a01027f0 2024-09-21T07:01:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a01a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a01a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52784/0 (socket says 172.21.15.114:52784) 2024-09-21T07:01:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a5400640 1 -- 172.21.15.114:0/956459575 learned_addr learned my addr 172.21.15.114:0/956459575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.536+0000 7f15a4a00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 0x7f15a019c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.537+0000 7f15a5400640 1 -- 172.21.15.114:0/956459575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 msgr2=0x7f15a019c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.537+0000 7f15a5400640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 0x7f15a019c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.537+0000 7f15a5400640 1 -- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15a0108120 msgr2=0x7f15a019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.537+0000 7f15a5400640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15a0108120 0x7f15a019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.537+0000 7f15a5400640 1 -- 172.21.15.114:0/956459575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a01a23c0 con 0x7f15a0108a70 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.537+0000 7f15a5400640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a01a1cc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1598009fd0 tx=0x7f159800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.538+0000 7f159d600640 1 -- 172.21.15.114:0/956459575 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598004b20 con 0x7f15a0108a70 2024-09-21T07:01:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.538+0000 7f159d600640 1 -- 172.21.15.114:0/956459575 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1598003e20 con 0x7f15a0108a70 2024-09-21T07:01:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.538+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a01a26b0 con 0x7f15a0108a70 2024-09-21T07:01:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.538+0000 7f159d600640 1 -- 172.21.15.114:0/956459575 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598004820 con 0x7f15a0108a70 2024-09-21T07:01:13.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.538+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a01a8e40 con 0x7f15a0108a70 2024-09-21T07:01:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.545+0000 7f159d600640 1 -- 172.21.15.114:0/956459575 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f159801d020 con 0x7f15a0108a70 2024-09-21T07:01:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.548+0000 7f159d600640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15880b66f0 0x7f15880b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.549+0000 7f15a4a00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15880b66f0 0x7f15880b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.553+0000 7f15a4a00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15880b66f0 0x7f15880b8c00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f159402a930 tx=0x7f15940058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.556+0000 7f159d600640 1 -- 172.21.15.114:0/956459575 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9578+0+0 (secure 0 0 0) 0x7f15980a97f0 con 0x7f15a0108a70 2024-09-21T07:01:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:13 smithi114 ceph-mon[30654]: pgmap v609: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 10 MiB/s wr, 846 op/s 2024-09-21T07:01:13.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.567+0000 7f15ae62b740 1 Processor -- start 2024-09-21T07:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.572+0000 7f15ae62b740 1 -- start start 2024-09-21T07:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7cec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15ae62b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c9d7c616b0 con 0x55c9d791e2f0 2024-09-21T07:01:13.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15ae62b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9d7c50ae0 con 0x55c9d79471b0 2024-09-21T07:01:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15ae62b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9d7c548f0 con 0x55c9d79264d0 2024-09-21T07:01:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.573+0000 7f15a4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cebe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52800/0 (socket says 172.21.15.114:52800) 2024-09-21T07:01:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 -- 172.21.15.114:0/2201973689 learned_addr learned my addr 172.21.15.114:0/2201973689 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 -- 172.21.15.114:0/2201973689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 msgr2=0x55c9d7cec890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:01:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cec890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 -- 172.21.15.114:0/2201973689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 msgr2=0x55c9d7cec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7cec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.575+0000 7f15a4a00640 1 -- 172.21.15.114:0/2201973689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d7c1d780 con 0x55c9d79471b0 2024-09-21T07:01:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.576+0000 7f15a4a00640 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cebe10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1594034000 tx=0x7f159403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=458f310c617e9f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.577+0000 7f1591c00640 1 -- 172.21.15.114:0/2201973689 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594037040 con 0x55c9d79471b0 2024-09-21T07:01:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.577+0000 7f1591c00640 1 -- 172.21.15.114:0/2201973689 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159404d070 con 0x55c9d79471b0 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.577+0000 7f1591c00640 1 -- 172.21.15.114:0/2201973689 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594037040 con 0x55c9d79471b0 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.578+0000 7f15ae62b740 1 -- 172.21.15.114:0/2201973689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 msgr2=0x55c9d7cebe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.578+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cebe10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1594034000 tx=0x7f159403c000 comp rx=0 tx=0).stop 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.580+0000 7f15ae62b740 1 -- 172.21.15.114:0/2201973689 shutdown_connections 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.580+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.580+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7cec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.580+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2201973689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cebe10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.580+0000 7f15ae62b740 1 -- 172.21.15.114:0/2201973689 >> 172.21.15.114:0/2201973689 conn(0x55c9d793b300 msgr2=0x55c9d7ce9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.581+0000 7f15ae62b740 1 -- 172.21.15.114:0/2201973689 shutdown_connections 2024-09-21T07:01:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.581+0000 7f15ae62b740 1 -- 172.21.15.114:0/2201973689 wait complete. 2024-09-21T07:01:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.582+0000 7f15ae62b740 1 -- start start 2024-09-21T07:01:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.583+0000 7f15ae62b740 1 Processor -- start 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d791e2f0 0x55c9d7cf7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 0x55c9d7cf7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c9d7b88ab0 con 0x55c9d79264d0 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c9d7bf7570 con 0x55c9d79471b0 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c9d7d0b540 con 0x55c9d791e2f0 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f15ae62b740 10 client.0 _ll_register_callbacks cb 0x55c9d7c06100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f159fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 0x55c9d7cf7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.585+0000 7f159fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 0x55c9d7cf7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40274/0 (socket says 172.21.15.114:40274) 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f159fe00640 1 -- 172.21.15.114:0/350805169 learned_addr learned my addr 172.21.15.114:0/350805169 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f15a4a00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d791e2f0 0x55c9d7cf7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f159fe00640 1 -- 172.21.15.114:0/350805169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d791e2f0 msgr2=0x55c9d7cf7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f15a5400640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f159fe00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d791e2f0 0x55c9d7cf7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f159fe00640 1 -- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 msgr2=0x55c9d7cf7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f159fe00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.586+0000 7f159fe00640 1 -- 172.21.15.114:0/350805169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d79b2790 con 0x55c9d79264d0 2024-09-21T07:01:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.587+0000 7f15a4a00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d791e2f0 0x55c9d7cf7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.587+0000 7f15a5400640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.588+0000 7f159fe00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 0x55c9d7cf7980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f158c00afc0 tx=0x7f158c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3716ff6cb15b5851 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.588+0000 7f1586000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.588+0000 7f1586000640 1 -- 172.21.15.114:0/350805169 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c00f860 con 0x55c9d79264d0 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.588+0000 7f1586000640 1 -- 172.21.15.114:0/350805169 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9578+0+0 (secure 0 0 0) 0x7f158c00fa00 con 0x55c9d79264d0 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.588+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159404d070 con 0x55c9d79264d0 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.589+0000 7f1586000640 1 -- 172.21.15.114:0/350805169 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f158c00fc00 con 0x55c9d79264d0 2024-09-21T07:01:13.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.589+0000 7f15ae62b740 10 client.17073 fetch_fsmap learned FSMap version 45 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.589+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c9d7a2b620 con 0x55c9d79264d0 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.589+0000 7f1586000640 1 -- 172.21.15.114:0/350805169 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159404d070 con 0x55c9d79264d0 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f1586000640 1 -- 172.21.15.114:0/350805169 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f158c018400 con 0x55c9d79264d0 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f15ae62b740 10 client.17073 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f15ae62b740 10 client.17073 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f15ae62b740 20 client.17073 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f15ae62b740 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d7905d80 0x55c9d7d1a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f15ae62b740 4 client.17073 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:13.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.590+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d7c63600 con 0x55c9d7905d80 2024-09-21T07:01:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.591+0000 7f15a4a00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d7905d80 0x55c9d7d1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.593+0000 7f15a4a00640 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d7905d80 0x55c9d7d1a680 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e7324bf1932a06 server_cookie=ee52efd454b6d786 in_seq=0 out_seq=0 2024-09-21T07:01:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.593+0000 7f1586000640 10 client.17073 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.594+0000 7f1586000640 1 -- 172.21.15.114:0/350805169 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c9d7c63600 con 0x55c9d7905d80 2024-09-21T07:01:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.594+0000 7f1586000640 10 client.17073 handle_command_reply: tid=0 2024-09-21T07:01:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.594+0000 7f15ae62b740 1 client.17073 shutdown 2024-09-21T07:01:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.595+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 msgr2=0x55c9d7cf7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.595+0000 7f15ae62b740 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 0x55c9d7cf7980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f158c00afc0 tx=0x7f158c00c760 comp rx=0 tx=0).stop 2024-09-21T07:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.597+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 shutdown_connections 2024-09-21T07:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.597+0000 7f15ae62b740 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.597+0000 7f15ae62b740 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d79264d0 0x55c9d7cf7980 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.597+0000 7f15ae62b740 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d7905d80 0x55c9d7d1a680 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.597+0000 7f15ae62b740 1 --2- 172.21.15.114:0/350805169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d791e2f0 0x55c9d7cf7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.597+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 >> 172.21.15.114:0/350805169 conn(0x55c9d793b300 msgr2=0x55c9d7cece40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.598+0000 7f1586000640 0 client.17073 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.598+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 shutdown_connections 2024-09-21T07:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.598+0000 7f15ae62b740 1 -- 172.21.15.114:0/350805169 wait complete. 2024-09-21T07:01:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.598+0000 7f15ae62b740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T07:01:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:13 smithi183 ceph-mon[34219]: pgmap v609: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 10 MiB/s wr, 846 op/s 2024-09-21T07:01:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.642+0000 7f15ae62b740 1 Processor -- start 2024-09-21T07:01:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.642+0000 7f15ae62b740 1 -- start start 2024-09-21T07:01:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.642+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d15090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.642+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cf4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.643+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.643+0000 7f15ae62b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01aec40 con 0x55c9d791e2f0 2024-09-21T07:01:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.643+0000 7f15ae62b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01b0840 con 0x55c9d79471b0 2024-09-21T07:01:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.643+0000 7f15ae62b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01af310 con 0x55c9d79264d0 2024-09-21T07:01:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.643+0000 7f15a4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d15090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d15090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40282/0 (socket says 172.21.15.114:40282) 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 -- 172.21.15.114:0/2563826742 learned_addr learned my addr 172.21.15.114:0/2563826742 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f159fe00640 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cf4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a5400640 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 -- 172.21.15.114:0/2563826742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 msgr2=0x55c9d7cf4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cf4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 -- 172.21.15.114:0/2563826742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 msgr2=0x55c9d7cf4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.644+0000 7f15a4a00640 1 -- 172.21.15.114:0/2563826742 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d7cee140 con 0x55c9d791e2f0 2024-09-21T07:01:13.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.645+0000 7f15a4a00640 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d15090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1594005940 tx=0x7f1594003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7205c522698d60ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.646+0000 7f1591200640 1 -- 172.21.15.114:0/2563826742 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594056030 con 0x55c9d791e2f0 2024-09-21T07:01:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.647+0000 7f1591200640 1 -- 172.21.15.114:0/2563826742 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159404d070 con 0x55c9d791e2f0 2024-09-21T07:01:13.649 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.647+0000 7f1591200640 1 -- 172.21.15.114:0/2563826742 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15940486c0 con 0x55c9d791e2f0 2024-09-21T07:01:13.650 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.648+0000 7f15ae62b740 1 -- 172.21.15.114:0/2563826742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 msgr2=0x55c9d7d15090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.648+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d15090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1594005940 tx=0x7f1594003d00 comp rx=0 tx=0).stop 2024-09-21T07:01:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.649+0000 7f15ae62b740 1 -- 172.21.15.114:0/2563826742 shutdown_connections 2024-09-21T07:01:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.649+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79471b0 0x55c9d7cf4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.651 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.649+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79264d0 0x55c9d7cf4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.649+0000 7f15ae62b740 1 --2- 172.21.15.114:0/2563826742 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d15090 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.649+0000 7f15ae62b740 1 -- 172.21.15.114:0/2563826742 >> 172.21.15.114:0/2563826742 conn(0x55c9d793b300 msgr2=0x55c9d7d14980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:13.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.652+0000 7f15ae62b740 1 -- 172.21.15.114:0/2563826742 shutdown_connections 2024-09-21T07:01:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.652+0000 7f15ae62b740 1 -- 172.21.15.114:0/2563826742 wait complete. 2024-09-21T07:01:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.652+0000 7f15ae62b740 1 -- start start 2024-09-21T07:01:13.655 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.653+0000 7f15ae62b740 1 Processor -- start 2024-09-21T07:01:13.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.654+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.654+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 0x55c9d7d0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15a4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d0df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15ae62b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79471b0 0x55c9d7d83040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f159fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 0x55c9d7d0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15a4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d0df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40298/0 (socket says 172.21.15.114:40298) 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f159fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 0x55c9d7d0e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52816/0 (socket says 172.21.15.114:52816) 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15ae62b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01b0140 con 0x55c9d791e2f0 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15ae62b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01af9e0 con 0x55c9d79264d0 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15ae62b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01acc40 con 0x55c9d79471b0 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15ae62b740 10 client.0 _ll_register_callbacks cb 0x7f158c005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15a4a00640 1 -- 172.21.15.114:0/4167150396 learned_addr learned my addr 172.21.15.114:0/4167150396 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:13.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.655+0000 7f15a5400640 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79471b0 0x55c9d7d83040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.656+0000 7f159fe00640 1 -- 172.21.15.114:0/4167150396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79471b0 msgr2=0x55c9d7d83040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.656+0000 7f159fe00640 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79471b0 0x55c9d7d83040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.656+0000 7f159fe00640 1 -- 172.21.15.114:0/4167150396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 msgr2=0x55c9d7d0df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.656+0000 7f159fe00640 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d0df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.656+0000 7f159fe00640 1 -- 172.21.15.114:0/4167150396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d7d83600 con 0x55c9d79264d0 2024-09-21T07:01:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.658+0000 7f159fe00640 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 0x55c9d7d0e4d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f158c018850 tx=0x7f158c00f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5de84d8d050ee01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.658+0000 7f1586000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:01:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.658+0000 7f1586000640 1 -- 172.21.15.114:0/4167150396 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c030060 con 0x55c9d79264d0 2024-09-21T07:01:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.658+0000 7f1586000640 1 -- 172.21.15.114:0/4167150396 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9578+0+0 (secure 0 0 0) 0x7f158c022420 con 0x55c9d79264d0 2024-09-21T07:01:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.658+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159404d070 con 0x55c9d79264d0 2024-09-21T07:01:13.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:13 smithi193 ceph-mon[34601]: pgmap v609: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 10 MiB/s wr, 846 op/s 2024-09-21T07:01:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.659+0000 7f1586000640 1 -- 172.21.15.114:0/4167150396 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f158c028070 con 0x55c9d79264d0 2024-09-21T07:01:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.659+0000 7f15ae62b740 10 client.27446 fetch_fsmap learned FSMap version 45 2024-09-21T07:01:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.659+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c9d7d84870 con 0x55c9d79264d0 2024-09-21T07:01:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.659+0000 7f1586000640 1 -- 172.21.15.114:0/4167150396 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159404d070 con 0x55c9d79264d0 2024-09-21T07:01:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.659+0000 7f1586000640 1 -- 172.21.15.114:0/4167150396 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 45) ==== 8386+0+0 (secure 0 0 0) 0x7f158c02a350 con 0x55c9d79264d0 2024-09-21T07:01:13.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.660+0000 7f15ae62b740 10 client.27446 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T07:01:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.660+0000 7f15ae62b740 10 client.27446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.660+0000 7f15ae62b740 20 client.27446 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.660+0000 7f15ae62b740 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d78db1c0 0x55c9d7d87000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.660+0000 7f15ae62b740 4 client.27446 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:01:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.660+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9d7c5f0b0 con 0x55c9d78db1c0 2024-09-21T07:01:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.661+0000 7f15a4a00640 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d78db1c0 0x55c9d7d87000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.661+0000 7f15a4a00640 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d78db1c0 0x55c9d7d87000 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4238e52f41ba971c server_cookie=a65df8102d1c9c77 in_seq=0 out_seq=0 2024-09-21T07:01:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.661+0000 7f1586000640 10 client.27446 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f1586000640 1 -- 172.21.15.114:0/4167150396 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9d7c5f0b0 con 0x55c9d78db1c0 2024-09-21T07:01:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f1586000640 10 client.27446 handle_command_reply: tid=0 2024-09-21T07:01:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 client.27446 shutdown 2024-09-21T07:01:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 msgr2=0x55c9d7d0e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 0x55c9d7d0e4d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f158c018850 tx=0x7f158c00f8f0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 shutdown_connections 2024-09-21T07:01:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c9d78db1c0 0x55c9d7d87000 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c9d79471b0 0x55c9d7d83040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c9d79264d0 0x55c9d7d0e4d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 --2- 172.21.15.114:0/4167150396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c9d791e2f0 0x55c9d7d0df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 >> 172.21.15.114:0/4167150396 conn(0x55c9d793b300 msgr2=0x55c9d7ce8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f1586000640 0 client.27446 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 shutdown_connections 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.675+0000 7f15ae62b740 1 -- 172.21.15.114:0/4167150396 wait complete. 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.676+0000 7f15ae62b740 20 client.27446 trim_cache size 0 max 16384 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:01:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:01:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:01:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.676+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15880b66f0 msgr2=0x7f15880b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.676+0000 7f15a5e00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15880b66f0 0x7f15880b8c00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f159402a930 tx=0x7f15940058d0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 msgr2=0x7f15a01a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a01a1cc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1598009fd0 tx=0x7f159800c800 comp rx=0 tx=0).stop 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 shutdown_connections 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f15880b66f0 0x7f15880b8c00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a0108a70 0x7f15a01a1cc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f15a0108120 0x7f15a019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 --2- 172.21.15.114:0/956459575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15a01027f0 0x7f15a019c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 >> 172.21.15.114:0/956459575 conn(0x7f15a00fe230 msgr2=0x7f15a00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.677+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 shutdown_connections 2024-09-21T07:01:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:13.678+0000 7f15a5e00640 1 -- 172.21.15.114:0/956459575 wait complete. 2024-09-21T07:01:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:14 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:14.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:14 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:14 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:15 smithi114 ceph-mon[30654]: pgmap v610: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 8.7 MiB/s wr, 631 op/s 2024-09-21T07:01:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:15 smithi183 ceph-mon[34219]: pgmap v610: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 8.7 MiB/s wr, 631 op/s 2024-09-21T07:01:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:15 smithi193 ceph-mon[34601]: pgmap v610: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 8.7 MiB/s wr, 631 op/s 2024-09-21T07:01:16.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:01:16.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:01:16.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:01:16.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:16 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:01:16.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:01:16.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:01:16.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:01:16.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:16 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:01:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:01:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:01:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:01:16.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:16 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:01:17.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:17 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:01:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:17 smithi114 ceph-mon[30654]: pgmap v611: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-09-21T07:01:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:17 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:17 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:01:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:17 smithi183 ceph-mon[34219]: pgmap v611: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-09-21T07:01:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:17 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:17 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:01:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:17 smithi193 ceph-mon[34601]: pgmap v611: 129 pgs: 1 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 123 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-09-21T07:01:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:17 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:01:17.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 400.0 s, 753.9 tps, lat 126.906 ms stddev 151.353 2024-09-21T07:01:19.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:19 smithi114 ceph-mon[30654]: pgmap v612: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 10 MiB/s wr, 640 op/s 2024-09-21T07:01:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:19 smithi183 ceph-mon[34219]: pgmap v612: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 10 MiB/s wr, 640 op/s 2024-09-21T07:01:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:19 smithi193 ceph-mon[34601]: pgmap v612: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 10 MiB/s wr, 640 op/s 2024-09-21T07:01:21.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:20 smithi114 ceph-mon[30654]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T07:01:21.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:20 smithi114 ceph-mon[30654]: pgmap v614: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 689 op/s 2024-09-21T07:01:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:20 smithi183 ceph-mon[34219]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T07:01:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:20 smithi183 ceph-mon[34219]: pgmap v614: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 689 op/s 2024-09-21T07:01:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:20 smithi193 ceph-mon[34601]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T07:01:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:20 smithi193 ceph-mon[34601]: pgmap v614: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 11 MiB/s wr, 689 op/s 2024-09-21T07:01:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 405.0 s, 545.8 tps, lat 188.794 ms stddev 232.401 2024-09-21T07:01:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:22 smithi183 ceph-mon[34219]: pgmap v615: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-09-21T07:01:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:22 smithi193 ceph-mon[34601]: pgmap v615: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-09-21T07:01:23.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:22 smithi114 ceph-mon[30654]: pgmap v615: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 12 MiB/s wr, 687 op/s 2024-09-21T07:01:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:24 smithi183 ceph-mon[34219]: pgmap v616: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-21T07:01:25.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:24 smithi193 ceph-mon[34601]: pgmap v616: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-21T07:01:25.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:24 smithi114 ceph-mon[30654]: pgmap v616: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 724 op/s 2024-09-21T07:01:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:26 smithi193 ceph-mon[34601]: pgmap v617: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 736 op/s 2024-09-21T07:01:27.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:26 smithi114 ceph-mon[30654]: pgmap v617: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 736 op/s 2024-09-21T07:01:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.329+0000 7f5612c00640 1 -- 172.21.15.114:0/2443654599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109410 msgr2=0x7f560c109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.329+0000 7f5612c00640 1 --2- 172.21.15.114:0/2443654599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109410 0x7f560c109820 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009900 tx=0x7f55fc02a410 comp rx=0 tx=0).stop 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.330+0000 7f5612c00640 1 -- 172.21.15.114:0/2443654599 shutdown_connections 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.330+0000 7f5612c00640 1 --2- 172.21.15.114:0/2443654599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109d60 0x7f560c110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.330+0000 7f5612c00640 1 --2- 172.21.15.114:0/2443654599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109410 0x7f560c109820 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.330+0000 7f5612c00640 1 --2- 172.21.15.114:0/2443654599 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c103ae0 0x7f560c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.330+0000 7f5612c00640 1 -- 172.21.15.114:0/2443654599 >> 172.21.15.114:0/2443654599 conn(0x7f560c0ff520 msgr2=0x7f560c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.331+0000 7f5612c00640 1 -- 172.21.15.114:0/2443654599 shutdown_connections 2024-09-21T07:01:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.331+0000 7f5612c00640 1 -- 172.21.15.114:0/2443654599 wait complete. 2024-09-21T07:01:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.332+0000 7f5612c00640 1 Processor -- start 2024-09-21T07:01:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 -- start start 2024-09-21T07:01:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c103ae0 0x7f560c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 0x7f560c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109d60 0x7f560c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f560c113a60 con 0x7f560c103ae0 2024-09-21T07:01:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1138e0 con 0x7f560c109410 2024-09-21T07:01:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.333+0000 7f5612c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f560c113be0 con 0x7f560c109d60 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 0x7f560c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 0x7f560c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46276/0 (socket says 172.21.15.114:46276) 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 -- 172.21.15.114:0/2837969373 learned_addr learned my addr 172.21.15.114:0/2837969373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f5610c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109d60 0x7f560c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560be00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c103ae0 0x7f560c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 -- 172.21.15.114:0/2837969373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109d60 msgr2=0x7f560c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109d60 0x7f560c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 -- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c103ae0 msgr2=0x7f560c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c103ae0 0x7f560c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f560b400640 1 -- 172.21.15.114:0/2837969373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1a87b0 con 0x7f560c109410 2024-09-21T07:01:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.334+0000 7f5610c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109d60 0x7f560c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.335+0000 7f560b400640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 0x7f560c075c50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005ab0 tx=0x7f55fc004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.335+0000 7f5608c00640 1 -- 172.21.15.114:0/2837969373 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc03f030 con 0x7f560c109410 2024-09-21T07:01:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.336+0000 7f5608c00640 1 -- 172.21.15.114:0/2837969373 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc004460 con 0x7f560c109410 2024-09-21T07:01:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.336+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1a8a40 con 0x7f560c109410 2024-09-21T07:01:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.336+0000 7f5608c00640 1 -- 172.21.15.114:0/2837969373 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc03f030 con 0x7f560c109410 2024-09-21T07:01:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.336+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1a8ef0 con 0x7f560c109410 2024-09-21T07:01:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.342+0000 7f5608c00640 1 -- 172.21.15.114:0/2837969373 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f55fc042030 con 0x7f560c109410 2024-09-21T07:01:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.344+0000 7f5608c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6930 0x7f55f00b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.345+0000 7f5608c00640 1 -- 172.21.15.114:0/2837969373 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f55fc0c9790 con 0x7f560c109410 2024-09-21T07:01:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.347+0000 7f560be00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6930 0x7f55f00b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.350+0000 7f560be00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6930 0x7f55f00b8e40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f560c077160 tx=0x7f55f8008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.360+0000 7f5619ca0740 1 Processor -- start 2024-09-21T07:01:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.362+0000 7f5619ca0740 1 -- start start 2024-09-21T07:01:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.362+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.362+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9e1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:26 smithi183 ceph-mon[34219]: pgmap v617: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 12 MiB/s wr, 736 op/s 2024-09-21T07:01:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.363+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d61b4d0 0x55850d9e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.363+0000 7f560b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.363+0000 7f560b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9e1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46280/0 (socket says 172.21.15.114:46280) 2024-09-21T07:01:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.363+0000 7f560b400640 1 -- 172.21.15.114:0/211699953 learned_addr learned my addr 172.21.15.114:0/211699953 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.363+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55850d9566b0 con 0x55850d63c1b0 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.363+0000 7f5610c00640 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d61b4d0 0x55850d9e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.364+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55850d945ae0 con 0x55850d6132f0 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.364+0000 7f560be00640 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.364+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55850d9498f0 con 0x55850d61b4d0 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.364+0000 7f560b400640 1 -- 172.21.15.114:0/211699953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d61b4d0 msgr2=0x55850d9e16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.364+0000 7f560b400640 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d61b4d0 0x55850d9e16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.364+0000 7f560b400640 1 -- 172.21.15.114:0/211699953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 msgr2=0x55850d9e0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.367+0000 7f560b400640 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.367+0000 7f560b400640 1 -- 172.21.15.114:0/211699953 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850d912780 con 0x55850d6132f0 2024-09-21T07:01:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.368+0000 7f560be00640 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.368+0000 7f560b400640 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9e1170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0ca3c0 tx=0x7f55fc0d5310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0c949a1fb1786b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.368+0000 7f5601200640 1 -- 172.21.15.114:0/211699953 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0c9d10 con 0x55850d6132f0 2024-09-21T07:01:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.369+0000 7f5601200640 1 -- 172.21.15.114:0/211699953 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc0cd030 con 0x55850d6132f0 2024-09-21T07:01:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.369+0000 7f5601200640 1 -- 172.21.15.114:0/211699953 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0c9eb0 con 0x55850d6132f0 2024-09-21T07:01:27.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.370+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 msgr2=0x55850d9e1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.370+0000 7f5619ca0740 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9e1170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0ca3c0 tx=0x7f55fc0d5310 comp rx=0 tx=0).stop 2024-09-21T07:01:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.371+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 shutdown_connections 2024-09-21T07:01:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.371+0000 7f5619ca0740 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d61b4d0 0x55850d9e16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.371+0000 7f5619ca0740 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9e1170 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.371+0000 7f5619ca0740 1 --2- 172.21.15.114:0/211699953 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.371+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 >> 172.21.15.114:0/211699953 conn(0x55850d630300 msgr2=0x55850d9deb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.372+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 shutdown_connections 2024-09-21T07:01:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.372+0000 7f5619ca0740 1 -- 172.21.15.114:0/211699953 wait complete. 2024-09-21T07:01:27.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.373+0000 7f5619ca0740 1 -- start start 2024-09-21T07:01:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.374+0000 7f5619ca0740 1 Processor -- start 2024-09-21T07:01:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.377+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.377+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.378+0000 7f560be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.378+0000 7f560be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46284/0 (socket says 172.21.15.114:46284) 2024-09-21T07:01:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.378+0000 7f560be00640 1 -- 172.21.15.114:0/1420789055 learned_addr learned my addr 172.21.15.114:0/1420789055 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.378+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d63c1b0 0x55850d9ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.378+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55850d87dab0 con 0x55850d61b4d0 2024-09-21T07:01:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.379+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55850d8ec570 con 0x55850d6132f0 2024-09-21T07:01:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.379+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55850d945ae0 con 0x55850d63c1b0 2024-09-21T07:01:27.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.379+0000 7f5619ca0740 10 client.0 _ll_register_callbacks cb 0x55850d8fb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:27.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.379+0000 7f5610c00640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d63c1b0 0x55850d9ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.379+0000 7f560b400640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f560be00640 1 -- 172.21.15.114:0/1420789055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d63c1b0 msgr2=0x55850d9ecd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f560be00640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d63c1b0 0x55850d9ecd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f560be00640 1 -- 172.21.15.114:0/1420789055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 msgr2=0x55850d9ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f560be00640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9ec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f560be00640 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850d6a7790 con 0x55850d6132f0 2024-09-21T07:01:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f55f5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:01:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.380+0000 7f560be00640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9ec290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55f800dee0 tx=0x7f55f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=469a46715aa57eae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.381+0000 7f55f5600640 1 -- 172.21.15.114:0/1420789055 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8014e40 con 0x55850d6132f0 2024-09-21T07:01:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.381+0000 7f55f5600640 1 -- 172.21.15.114:0/1420789055 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f55f80125f0 con 0x55850d6132f0 2024-09-21T07:01:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.381+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55fc0cd030 con 0x55850d6132f0 2024-09-21T07:01:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.381+0000 7f55f5600640 1 -- 172.21.15.114:0/1420789055 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f55f80059c0 con 0x55850d6132f0 2024-09-21T07:01:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.382+0000 7f5619ca0740 10 client.27458 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.382+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55850d720620 con 0x55850d6132f0 2024-09-21T07:01:27.384 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.382+0000 7f55f5600640 1 -- 172.21.15.114:0/1420789055 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc0cd030 con 0x55850d6132f0 2024-09-21T07:01:27.388 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.383+0000 7f55f5600640 1 -- 172.21.15.114:0/1420789055 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f55f8013070 con 0x55850d6132f0 2024-09-21T07:01:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.387+0000 7f5619ca0740 10 client.27458 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:27.389 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.387+0000 7f5619ca0740 10 client.27458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.387+0000 7f5619ca0740 20 client.27458 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.387+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5fad80 0x55850d9fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.388+0000 7f5619ca0740 4 client.27458 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.388+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55850d958600 con 0x55850d5fad80 2024-09-21T07:01:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.388+0000 7f560b400640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5fad80 0x55850d9fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.390+0000 7f55f5600640 10 client.27458 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:27.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.390+0000 7f560b400640 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5fad80 0x55850d9fdfd0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8775eb987fa77a13 server_cookie=bc55edff69a358bb in_seq=0 out_seq=0 2024-09-21T07:01:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.391+0000 7f55f5600640 1 -- 172.21.15.114:0/1420789055 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55850d958600 con 0x55850d5fad80 2024-09-21T07:01:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.391+0000 7f55f5600640 10 client.27458 handle_command_reply: tid=0 2024-09-21T07:01:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.391+0000 7f5619ca0740 1 client.27458 shutdown 2024-09-21T07:01:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.392+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 msgr2=0x55850d9ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.392+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9ec290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f55f800dee0 tx=0x7f55f80046c0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 shutdown_connections 2024-09-21T07:01:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5fad80 0x55850d9fdfd0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d63c1b0 0x55850d9ecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1420789055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d6132f0 0x55850d9ec290 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 >> 172.21.15.114:0/1420789055 conn(0x55850d630300 msgr2=0x55850d9e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.393+0000 7f55f5600640 0 client.27458 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.396+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 shutdown_connections 2024-09-21T07:01:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.396+0000 7f5619ca0740 1 -- 172.21.15.114:0/1420789055 wait complete. 2024-09-21T07:01:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.396+0000 7f5619ca0740 20 client.27458 trim_cache size 0 max 16384 2024-09-21T07:01:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.442+0000 7f5619ca0740 1 Processor -- start 2024-09-21T07:01:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.442+0000 7f5619ca0740 1 -- start start 2024-09-21T07:01:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9e2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d61b4d0 0x55850d9e9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f560be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f5619ca0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1aee10 con 0x55850d63c1b0 2024-09-21T07:01:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f5619ca0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1b09e0 con 0x55850d61b4d0 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f5619ca0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1af4e0 con 0x55850d6132f0 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f560be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9e2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33054/0 (socket says 172.21.15.114:33054) 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.443+0000 7f560be00640 1 -- 172.21.15.114:0/2449891987 learned_addr learned my addr 172.21.15.114:0/2449891987 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.444+0000 7f560b400640 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d61b4d0 0x55850d9e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.444+0000 7f560be00640 1 -- 172.21.15.114:0/2449891987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d61b4d0 msgr2=0x55850d9e9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.444+0000 7f560be00640 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d61b4d0 0x55850d9e9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.444+0000 7f560be00640 1 -- 172.21.15.114:0/2449891987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 msgr2=0x55850d9e99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.444+0000 7f560be00640 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.444+0000 7f560be00640 1 -- 172.21.15.114:0/2449891987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850d9df420 con 0x55850d6132f0 2024-09-21T07:01:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.445+0000 7f560be00640 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9e2370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55f80061c0 tx=0x7f55f800db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52ac7f272aef11ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:27.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.445+0000 7f55f7e00640 1 -- 172.21.15.114:0/2449891987 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8036070 con 0x55850d6132f0 2024-09-21T07:01:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.446+0000 7f55f7e00640 1 -- 172.21.15.114:0/2449891987 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f8024070 con 0x55850d6132f0 2024-09-21T07:01:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.446+0000 7f55f7e00640 1 -- 172.21.15.114:0/2449891987 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8012af0 con 0x55850d6132f0 2024-09-21T07:01:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 -- 172.21.15.114:0/2449891987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 msgr2=0x55850d9e2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9e2370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55f80061c0 tx=0x7f55f800db70 comp rx=0 tx=0).stop 2024-09-21T07:01:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 -- 172.21.15.114:0/2449891987 shutdown_connections 2024-09-21T07:01:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d63c1b0 0x55850d9e99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d61b4d0 0x55850d9e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 --2- 172.21.15.114:0/2449891987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9e2370 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.447+0000 7f5619ca0740 1 -- 172.21.15.114:0/2449891987 >> 172.21.15.114:0/2449891987 conn(0x55850d630300 msgr2=0x55850d9e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.448+0000 7f5619ca0740 1 -- 172.21.15.114:0/2449891987 shutdown_connections 2024-09-21T07:01:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.448+0000 7f5619ca0740 1 -- 172.21.15.114:0/2449891987 wait complete. 2024-09-21T07:01:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.449+0000 7f5619ca0740 1 -- start start 2024-09-21T07:01:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.449+0000 7f5619ca0740 1 Processor -- start 2024-09-21T07:01:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9f2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d63c1b0 0x55850da609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1b0310 con 0x55850d61b4d0 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1afbb0 con 0x55850d63c1b0 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f560c1ace10 con 0x55850d6132f0 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f5619ca0740 10 client.0 _ll_register_callbacks cb 0x7f55f8009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.451+0000 7f560be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33066/0 (socket says 172.21.15.114:33066) 2024-09-21T07:01:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 -- 172.21.15.114:0/1028629951 learned_addr learned my addr 172.21.15.114:0/1028629951 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f5610c00640 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d63c1b0 0x55850da609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 -- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d63c1b0 msgr2=0x55850da609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d63c1b0 0x55850da609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 -- 172.21.15.114:0/1028629951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 msgr2=0x55850d9f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.452+0000 7f560be00640 1 -- 172.21.15.114:0/1028629951 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850da61050 con 0x55850d6132f0 2024-09-21T07:01:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.453+0000 7f560be00640 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9f1aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009f20 tx=0x7f55f8004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41c6b37bb17be31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.453+0000 7f55f5600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:01:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.453+0000 7f55f5600640 1 -- 172.21.15.114:0/1028629951 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f803f060 con 0x55850d6132f0 2024-09-21T07:01:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.454+0000 7f55f5600640 1 -- 172.21.15.114:0/1028629951 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f55f8012850 con 0x55850d6132f0 2024-09-21T07:01:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.454+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f8024070 con 0x55850d6132f0 2024-09-21T07:01:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.454+0000 7f55f5600640 1 -- 172.21.15.114:0/1028629951 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f55f8036040 con 0x55850d6132f0 2024-09-21T07:01:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.454+0000 7f5619ca0740 10 client.27394 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.454+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55850da68370 con 0x55850d6132f0 2024-09-21T07:01:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.454+0000 7f55f5600640 1 -- 172.21.15.114:0/1028629951 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f8024070 con 0x55850d6132f0 2024-09-21T07:01:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f55f5600640 1 -- 172.21.15.114:0/1028629951 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f55f8039070 con 0x55850d6132f0 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f5619ca0740 10 client.27394 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f5619ca0740 10 client.27394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f5619ca0740 20 client.27394 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5d01c0 0x55850da6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f5619ca0740 4 client.27394 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.455+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55850d9540b0 con 0x55850d5d01c0 2024-09-21T07:01:27.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.456+0000 7f560b400640 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5d01c0 0x55850da6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.460+0000 7f55f5600640 10 client.27394 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.460+0000 7f560b400640 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5d01c0 0x55850da6a9c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b213f674b0e59f07 server_cookie=23904ea71ff6d90d in_seq=0 out_seq=0 2024-09-21T07:01:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.575+0000 7f55f5600640 1 -- 172.21.15.114:0/1028629951 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55850d9540b0 con 0x55850d5d01c0 2024-09-21T07:01:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.575+0000 7f55f5600640 10 client.27394 handle_command_reply: tid=0 2024-09-21T07:01:27.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 client.27394 shutdown 2024-09-21T07:01:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 msgr2=0x55850d9f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9f1aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009f20 tx=0x7f55f8004660 comp rx=0 tx=0).stop 2024-09-21T07:01:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 shutdown_connections 2024-09-21T07:01:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55850d63c1b0 0x55850da609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55850d61b4d0 0x55850d9f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55850d5d01c0 0x55850da6a9c0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f5619ca0740 1 --2- 172.21.15.114:0/1028629951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55850d6132f0 0x55850d9f1aa0 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009f20 tx=0x7f55f8004660 comp rx=0 tx=0).stop 2024-09-21T07:01:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.576+0000 7f55f5600640 0 client.27394 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.577+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 >> 172.21.15.114:0/1028629951 conn(0x55850d630300 msgr2=0x55850d9e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.577+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 shutdown_connections 2024-09-21T07:01:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.577+0000 7f5619ca0740 1 -- 172.21.15.114:0/1028629951 wait complete. 2024-09-21T07:01:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.577+0000 7f5619ca0740 20 client.27394 trim_cache size 0 max 16384 2024-09-21T07:01:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:01:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:01:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:01:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:01:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6930 msgr2=0x7f55f00b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6930 0x7f55f00b8e40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f560c077160 tx=0x7f55f8008040 comp rx=0 tx=0).stop 2024-09-21T07:01:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 msgr2=0x7f560c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 0x7f560c075c50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005ab0 tx=0x7f55fc004060 comp rx=0 tx=0).stop 2024-09-21T07:01:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 shutdown_connections 2024-09-21T07:01:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c109d60 0x7f560c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6930 0x7f55f00b8e40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c109410 0x7f560c075c50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 --2- 172.21.15.114:0/2837969373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c103ae0 0x7f560c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 >> 172.21.15.114:0/2837969373 conn(0x7f560c0ff520 msgr2=0x7f560c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 shutdown_connections 2024-09-21T07:01:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:27.579+0000 7f5612c00640 1 -- 172.21.15.114:0/2837969373 wait complete. 2024-09-21T07:01:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 410.0 s, 726.8 tps, lat 131.963 ms stddev 156.830 2024-09-21T07:01:29.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:28 smithi193 ceph-mon[34601]: pgmap v618: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 642 op/s 2024-09-21T07:01:29.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:28 smithi114 ceph-mon[30654]: pgmap v618: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 642 op/s 2024-09-21T07:01:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:28 smithi183 ceph-mon[34219]: pgmap v618: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 642 op/s 2024-09-21T07:01:31.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:30 smithi114 ceph-mon[30654]: pgmap v619: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 628 op/s 2024-09-21T07:01:31.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:30 smithi183 ceph-mon[34219]: pgmap v619: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 628 op/s 2024-09-21T07:01:31.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:30 smithi193 ceph-mon[34601]: pgmap v619: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 10 MiB/s wr, 628 op/s 2024-09-21T07:01:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 415.0 s, 580.8 tps, lat 172.974 ms stddev 275.933 2024-09-21T07:01:33.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:32 smithi183 ceph-mon[34219]: pgmap v620: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.4 MiB/s wr, 579 op/s 2024-09-21T07:01:33.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:32 smithi193 ceph-mon[34601]: pgmap v620: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.4 MiB/s wr, 579 op/s 2024-09-21T07:01:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:32 smithi114 ceph-mon[30654]: pgmap v620: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.4 MiB/s wr, 579 op/s 2024-09-21T07:01:35.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:34 smithi183 ceph-mon[34219]: pgmap v621: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 659 op/s 2024-09-21T07:01:35.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:34 smithi193 ceph-mon[34601]: pgmap v621: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 659 op/s 2024-09-21T07:01:35.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:34 smithi114 ceph-mon[30654]: pgmap v621: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 11 MiB/s wr, 659 op/s 2024-09-21T07:01:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:36 smithi193 ceph-mon[34601]: pgmap v622: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.1 MiB/s wr, 508 op/s 2024-09-21T07:01:37.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:36 smithi114 ceph-mon[30654]: pgmap v622: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.1 MiB/s wr, 508 op/s 2024-09-21T07:01:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:36 smithi183 ceph-mon[34219]: pgmap v622: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.1 MiB/s wr, 508 op/s 2024-09-21T07:01:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 420.0 s, 622.2 tps, lat 168.050 ms stddev 257.951 2024-09-21T07:01:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:38 smithi193 ceph-mon[34601]: pgmap v623: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-09-21T07:01:39.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:38 smithi114 ceph-mon[30654]: pgmap v623: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-09-21T07:01:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:38 smithi183 ceph-mon[34219]: pgmap v623: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 711 op/s 2024-09-21T07:01:40.306 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:01:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:40 smithi193 ceph-mon[34601]: pgmap v624: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.7 MiB/s wr, 585 op/s 2024-09-21T07:01:41.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:40 smithi114 ceph-mon[30654]: pgmap v624: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.7 MiB/s wr, 585 op/s 2024-09-21T07:01:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:40 smithi183 ceph-mon[34219]: pgmap v624: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 8.7 MiB/s wr, 585 op/s 2024-09-21T07:01:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 425.0 s, 631.4 tps, lat 153.080 ms stddev 256.989 2024-09-21T07:01:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:42 smithi193 ceph-mon[34601]: pgmap v625: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 11 MiB/s wr, 725 op/s 2024-09-21T07:01:43.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:42 smithi114 ceph-mon[30654]: pgmap v625: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 11 MiB/s wr, 725 op/s 2024-09-21T07:01:43.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:42 smithi183 ceph-mon[34219]: pgmap v625: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 11 MiB/s wr, 725 op/s 2024-09-21T07:01:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 -- 172.21.15.114:0/38296052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109b90 msgr2=0x7fa9001103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 --2- 172.21.15.114:0/38296052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109b90 0x7fa9001103c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009900 tx=0x7fa8e802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 -- 172.21.15.114:0/38296052 shutdown_connections 2024-09-21T07:01:44.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 --2- 172.21.15.114:0/38296052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109b90 0x7fa9001103c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 --2- 172.21.15.114:0/38296052 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa900109240 0x7fa900109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 --2- 172.21.15.114:0/38296052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.010+0000 7fa904c00640 1 -- 172.21.15.114:0/38296052 >> 172.21.15.114:0/38296052 conn(0x7fa9000ff330 msgr2=0x7fa900101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.011+0000 7fa904c00640 1 -- 172.21.15.114:0/38296052 shutdown_connections 2024-09-21T07:01:44.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.011+0000 7fa904c00640 1 -- 172.21.15.114:0/38296052 wait complete. 2024-09-21T07:01:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.013+0000 7fa904c00640 1 Processor -- start 2024-09-21T07:01:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.014+0000 7fa904c00640 1 -- start start 2024-09-21T07:01:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.016+0000 7fa904c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900074e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.016+0000 7fa904c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109240 0x7fa90006e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900074e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109240 0x7fa90006e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa904c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa900109b90 0x7fa90006e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa904c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa900113850 con 0x7fa900109b90 2024-09-21T07:01:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa904c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9001136d0 con 0x7fa900103910 2024-09-21T07:01:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa904c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9001139d0 con 0x7fa900109240 2024-09-21T07:01:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900074e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43068/0 (socket says 172.21.15.114:43068) 2024-09-21T07:01:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.017+0000 7fa8ff400640 1 -- 172.21.15.114:0/4101285427 learned_addr learned my addr 172.21.15.114:0/4101285427 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.018+0000 7fa8ff400640 1 -- 172.21.15.114:0/4101285427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109240 msgr2=0x7fa90006e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.018+0000 7fa8ff400640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109240 0x7fa90006e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.018+0000 7fa8ff400640 1 -- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa900109b90 msgr2=0x7fa90006e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.018+0000 7fa8ff400640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa900109b90 0x7fa90006e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.018+0000 7fa8ff400640 1 -- 172.21.15.114:0/4101285427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90006eec0 con 0x7fa900103910 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.018+0000 7fa8fea00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109240 0x7fa90006e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.019+0000 7fa8ff400640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900074e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800eab0 tx=0x7fa8f800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.019+0000 7fa8f7e00640 1 -- 172.21.15.114:0/4101285427 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8020070 con 0x7fa900103910 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.020+0000 7fa8f7e00640 1 -- 172.21.15.114:0/4101285427 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f800cde0 con 0x7fa900103910 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.020+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa900073b80 con 0x7fa900103910 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.020+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9000740d0 con 0x7fa900103910 2024-09-21T07:01:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.020+0000 7fa8f7e00640 1 -- 172.21.15.114:0/4101285427 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f801bb80 con 0x7fa900103910 2024-09-21T07:01:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.034+0000 7fa90d3c0740 1 Processor -- start 2024-09-21T07:01:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.035+0000 7fa8f7e00640 1 -- 172.21.15.114:0/4101285427 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa8f8004a40 con 0x7fa900103910 2024-09-21T07:01:44.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa8f7e00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8e40bf010 0x7fa8e40c1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 -- start start 2024-09-21T07:01:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa8f7e00640 1 -- 172.21.15.114:0/4101285427 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7fa8f80e6f70 con 0x7fa900103910 2024-09-21T07:01:44.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa8ffe00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8e40bf010 0x7fa8e40c1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff73890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562d3fee86b0 con 0x562d3fbad4d0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d3fed7ae0 con 0x562d3fba52f0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.038+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d3fedb8f0 con 0x562d3fbce1b0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.039+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.039+0000 7fa8ffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff73890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.039+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.039+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff72e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37946/0 (socket says 172.21.15.114:37946) 2024-09-21T07:01:44.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.039+0000 7fa8ff400640 1 -- 172.21.15.114:0/2314111640 learned_addr learned my addr 172.21.15.114:0/2314111640 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.040+0000 7fa8ff400640 1 -- 172.21.15.114:0/2314111640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 msgr2=0x562d3ff73350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.041+0000 7fa8ff400640 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff73350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.041+0000 7fa8ff400640 1 -- 172.21.15.114:0/2314111640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 msgr2=0x562d3ff73890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.041+0000 7fa8ff400640 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff73890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.041+0000 7fa8ff400640 1 -- 172.21.15.114:0/2314111640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d3fea4780 con 0x562d3fbce1b0 2024-09-21T07:01:44.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.042+0000 7fa8ff400640 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff72e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80e4350 tx=0x7fa8f80ac160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7233ba59fb51a967 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.042+0000 7fa8efe00640 1 -- 172.21.15.114:0/2314111640 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f801b440 con 0x562d3fbce1b0 2024-09-21T07:01:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.042+0000 7fa8efe00640 1 -- 172.21.15.114:0/2314111640 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f80ac500 con 0x562d3fbce1b0 2024-09-21T07:01:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.042+0000 7fa8efe00640 1 -- 172.21.15.114:0/2314111640 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80b12d0 con 0x562d3fbce1b0 2024-09-21T07:01:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.043+0000 7fa8fea00640 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff73350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.043+0000 7fa8ffe00640 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff73890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.044+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2314111640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 msgr2=0x562d3ff72e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.044+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff72e10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80e4350 tx=0x7fa8f80ac160 comp rx=0 tx=0).stop 2024-09-21T07:01:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.048+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2314111640 shutdown_connections 2024-09-21T07:01:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.048+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff73890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.048+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff73350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.048+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2314111640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff72e10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.048+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2314111640 >> 172.21.15.114:0/2314111640 conn(0x562d3fbc2300 msgr2=0x562d3ff70d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.052+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2314111640 shutdown_connections 2024-09-21T07:01:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.052+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2314111640 wait complete. 2024-09-21T07:01:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.053+0000 7fa8ffe00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8e40bf010 0x7fa8e40c1520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009850 tx=0x7fa8e8005eb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.054+0000 7fa90d3c0740 1 -- start start 2024-09-21T07:01:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.055+0000 7fa90d3c0740 1 Processor -- start 2024-09-21T07:01:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.056+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fba52f0 0x562d3ff7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.056+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.056+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.056+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562d3fe0fab0 con 0x562d3fbad4d0 2024-09-21T07:01:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d3fe7e570 con 0x562d3fbce1b0 2024-09-21T07:01:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562d3ff81630 con 0x562d3fba52f0 2024-09-21T07:01:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa8ffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff7e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44444/0 (socket says 172.21.15.114:44444) 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa8fea00640 1 -- 172.21.15.114:0/1905217104 learned_addr learned my addr 172.21.15.114:0/1905217104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.057+0000 7fa90d3c0740 10 client.0 _ll_register_callbacks cb 0x562d3fe8d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.058+0000 7fa8fea00640 1 -- 172.21.15.114:0/1905217104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fba52f0 msgr2=0x562d3ff7e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.058+0000 7fa8fea00640 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fba52f0 0x562d3ff7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.058+0000 7fa8fea00640 1 -- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 msgr2=0x562d3ff7edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.058+0000 7fa8fea00640 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.058+0000 7fa8fea00640 1 -- 172.21.15.114:0/1905217104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d3fc39790 con 0x562d3fbad4d0 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.058+0000 7fa8ffe00640 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.061+0000 7fa8fea00640 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff7e890 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0011af0 tx=0x7fa8f0011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680b3175cd8d1f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.061+0000 7fa8ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.061+0000 7fa8ecc00640 1 -- 172.21.15.114:0/1905217104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f001f060 con 0x562d3fbad4d0 2024-09-21T07:01:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.062+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f80ac500 con 0x562d3fbad4d0 2024-09-21T07:01:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.062+0000 7fa8ecc00640 1 -- 172.21.15.114:0/1905217104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7fa8f0012450 con 0x562d3fbad4d0 2024-09-21T07:01:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.067+0000 7fa8ecc00640 1 -- 172.21.15.114:0/1905217104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa8f0012650 con 0x562d3fbad4d0 2024-09-21T07:01:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.067+0000 7fa90d3c0740 10 client.17091 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.067+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562d3fcb2620 con 0x562d3fbad4d0 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.068+0000 7fa8ecc00640 1 -- 172.21.15.114:0/1905217104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f0012ca0 con 0x562d3fbad4d0 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa8ecc00640 1 -- 172.21.15.114:0/1905217104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fa8f001aa60 con 0x562d3fbad4d0 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa90d3c0740 10 client.17091 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa90d3c0740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa90d3c0740 20 client.17091 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb8cd80 0x562d3ff90610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa90d3c0740 4 client.17091 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.069+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa8f80ac500 con 0x562d3fb8cd80 2024-09-21T07:01:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.070+0000 7fa8ff400640 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb8cd80 0x562d3ff90610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.074+0000 7fa8ff400640 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb8cd80 0x562d3ff90610 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d794a55761b99e0 server_cookie=ef22cfb5705db265 in_seq=0 out_seq=0 2024-09-21T07:01:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.074+0000 7fa8ecc00640 10 client.17091 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.078+0000 7fa8ecc00640 1 -- 172.21.15.114:0/1905217104 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa8f80ac500 con 0x562d3fb8cd80 2024-09-21T07:01:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.078+0000 7fa8ecc00640 10 client.17091 handle_command_reply: tid=0 2024-09-21T07:01:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.078+0000 7fa90d3c0740 1 client.17091 shutdown 2024-09-21T07:01:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 msgr2=0x562d3ff7e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff7e890 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0011af0 tx=0x7fa8f0011ef0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 shutdown_connections 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff7e890 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb8cd80 0x562d3ff90610 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/1905217104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fba52f0 0x562d3ff7e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 >> 172.21.15.114:0/1905217104 conn(0x562d3fbc2300 msgr2=0x562d3ff73e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.082+0000 7fa8ecc00640 0 client.17091 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.083+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 shutdown_connections 2024-09-21T07:01:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.084+0000 7fa90d3c0740 1 -- 172.21.15.114:0/1905217104 wait complete. 2024-09-21T07:01:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.084+0000 7fa90d3c0740 20 client.17091 trim_cache size 0 max 16384 2024-09-21T07:01:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.141+0000 7fa90d3c0740 1 Processor -- start 2024-09-21T07:01:44.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.141+0000 7fa90d3c0740 1 -- start start 2024-09-21T07:01:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.143+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fba52f0 0x562d3ff7ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 0x562d3ff7b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc00fe70 con 0x562d3fba52f0 2024-09-21T07:01:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc011a40 con 0x562d3fbce1b0 2024-09-21T07:01:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc010540 con 0x562d3fbad4d0 2024-09-21T07:01:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 0x562d3ff7b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa8fea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 0x562d3ff7b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37962/0 (socket says 172.21.15.114:37962) 2024-09-21T07:01:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa8fea00640 1 -- 172.21.15.114:0/971934268 learned_addr learned my addr 172.21.15.114:0/971934268 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa8ffe00640 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.144+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fba52f0 0x562d3ff7ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.145+0000 7fa8fea00640 1 -- 172.21.15.114:0/971934268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 msgr2=0x562d3ff7ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.145+0000 7fa8fea00640 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.145+0000 7fa8fea00640 1 -- 172.21.15.114:0/971934268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fba52f0 msgr2=0x562d3ff7ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.148+0000 7fa8fea00640 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fba52f0 0x562d3ff7ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.148+0000 7fa8fea00640 1 -- 172.21.15.114:0/971934268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d3ff75c20 con 0x562d3fbad4d0 2024-09-21T07:01:44.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.148+0000 7fa8ff400640 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fba52f0 0x562d3ff7ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:01:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.148+0000 7fa8fea00640 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 0x562d3ff7b520 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000a520 tx=0x7fa8f000f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b076dae8d463448d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.151+0000 7fa8ef400640 1 -- 172.21.15.114:0/971934268 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0032070 con 0x562d3fbad4d0 2024-09-21T07:01:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.151+0000 7fa8ef400640 1 -- 172.21.15.114:0/971934268 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f0012ca0 con 0x562d3fbad4d0 2024-09-21T07:01:44.153 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.151+0000 7fa8ef400640 1 -- 172.21.15.114:0/971934268 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0029080 con 0x562d3fbad4d0 2024-09-21T07:01:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.153+0000 7fa90d3c0740 1 -- 172.21.15.114:0/971934268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 msgr2=0x562d3ff7b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.153+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 0x562d3ff7b520 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000a520 tx=0x7fa8f000f7f0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.153+0000 7fa90d3c0740 1 -- 172.21.15.114:0/971934268 shutdown_connections 2024-09-21T07:01:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.153+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fbce1b0 0x562d3ff7ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.153+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbad4d0 0x562d3ff7b520 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.153+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/971934268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fba52f0 0x562d3ff7ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.154+0000 7fa90d3c0740 1 -- 172.21.15.114:0/971934268 >> 172.21.15.114:0/971934268 conn(0x562d3fbc2300 msgr2=0x562d3ff7f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.154+0000 7fa90d3c0740 1 -- 172.21.15.114:0/971934268 shutdown_connections 2024-09-21T07:01:44.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.154+0000 7fa90d3c0740 1 -- 172.21.15.114:0/971934268 wait complete. 2024-09-21T07:01:44.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.157+0000 7fa90d3c0740 1 -- start start 2024-09-21T07:01:44.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.157+0000 7fa90d3c0740 1 Processor -- start 2024-09-21T07:01:44.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.158+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.159+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff84fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.159+0000 7fa90d3c0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff854f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.159+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc011370 con 0x562d3fbad4d0 2024-09-21T07:01:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.159+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc010c10 con 0x562d3fba52f0 2024-09-21T07:01:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.159+0000 7fa90d3c0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa8dc00df00 con 0x562d3fbce1b0 2024-09-21T07:01:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.159+0000 7fa90d3c0740 10 client.0 _ll_register_callbacks cb 0x7fa8f000a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.160+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.161+0000 7fa8ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff84a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43098/0 (socket says 172.21.15.114:43098) 2024-09-21T07:01:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.161+0000 7fa8ff400640 1 -- 172.21.15.114:0/2605591583 learned_addr learned my addr 172.21.15.114:0/2605591583 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:44.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.161+0000 7fa8ffe00640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff854f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.161+0000 7fa8fea00640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff84fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ff400640 1 -- 172.21.15.114:0/2605591583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 msgr2=0x562d3ff854f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ff400640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff854f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ff400640 1 -- 172.21.15.114:0/2605591583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 msgr2=0x562d3ff84fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ff400640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff84fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ff400640 1 -- 172.21.15.114:0/2605591583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d3fffa220 con 0x562d3fba52f0 2024-09-21T07:01:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ff400640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff84a70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ac980 tx=0x7fa8f80ac290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e4cc07728d9976 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.162+0000 7fa8ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:01:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.163+0000 7fa8ecc00640 1 -- 172.21.15.114:0/2605591583 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80c7060 con 0x562d3fba52f0 2024-09-21T07:01:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.163+0000 7fa8ecc00640 1 -- 172.21.15.114:0/2605591583 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7fa8f80b10d0 con 0x562d3fba52f0 2024-09-21T07:01:44.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.163+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f0012ca0 con 0x562d3fba52f0 2024-09-21T07:01:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.163+0000 7fa8ecc00640 1 -- 172.21.15.114:0/2605591583 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa8f80a8890 con 0x562d3fba52f0 2024-09-21T07:01:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.163+0000 7fa90d3c0740 10 client.27476 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.163+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562d3fffb510 con 0x562d3fba52f0 2024-09-21T07:01:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.164+0000 7fa8ecc00640 1 -- 172.21.15.114:0/2605591583 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8f0012ca0 con 0x562d3fba52f0 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.164+0000 7fa8ecc00640 1 -- 172.21.15.114:0/2605591583 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7fa8f800f040 con 0x562d3fba52f0 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa90d3c0740 10 client.27476 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa90d3c0740 10 client.27476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa90d3c0740 20 client.27476 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb621c0 0x562d3fffd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa90d3c0740 4 client.27476 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:01:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d3feea600 con 0x562d3fb621c0 2024-09-21T07:01:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.165+0000 7fa8fea00640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb621c0 0x562d3fffd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:44.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.166+0000 7fa8ecc00640 10 client.27476 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.167+0000 7fa8fea00640 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb621c0 0x562d3fffd860 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3b4622de0783e9 server_cookie=5e8a7c7c31bddf in_seq=0 out_seq=0 2024-09-21T07:01:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.390+0000 7fa8ecc00640 1 -- 172.21.15.114:0/2605591583 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d3feea600 con 0x562d3fb621c0 2024-09-21T07:01:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.390+0000 7fa8ecc00640 10 client.27476 handle_command_reply: tid=0 2024-09-21T07:01:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.390+0000 7fa90d3c0740 1 client.27476 shutdown 2024-09-21T07:01:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 msgr2=0x562d3ff84a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff84a70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ac980 tx=0x7fa8f80ac290 comp rx=0 tx=0).stop 2024-09-21T07:01:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 shutdown_connections 2024-09-21T07:01:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562d3fb621c0 0x562d3fffd860 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d3fbce1b0 0x562d3ff854f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562d3fbad4d0 0x562d3ff84fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 --2- 172.21.15.114:0/2605591583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d3fba52f0 0x562d3ff84a70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.391+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 >> 172.21.15.114:0/2605591583 conn(0x562d3fbc2300 msgr2=0x562d3ff73e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.392+0000 7fa8ecc00640 0 client.27476 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.392+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 shutdown_connections 2024-09-21T07:01:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.392+0000 7fa90d3c0740 1 -- 172.21.15.114:0/2605591583 wait complete. 2024-09-21T07:01:44.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.392+0000 7fa90d3c0740 20 client.27476 trim_cache size 0 max 16384 2024-09-21T07:01:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:01:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:01:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:01:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:01:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.394+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8e40bf010 msgr2=0x7fa8e40c1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.394+0000 7fa904c00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8e40bf010 0x7fa8e40c1520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009850 tx=0x7fa8e8005eb0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.394+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 msgr2=0x7fa900074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.394+0000 7fa904c00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900074e60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800eab0 tx=0x7fa8f800ef70 comp rx=0 tx=0).stop 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 shutdown_connections 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa900109b90 0x7fa90006e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa900109240 0x7fa90006e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa8e40bf010 0x7fa8e40c1520 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 --2- 172.21.15.114:0/4101285427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa900103910 0x7fa900074e60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 >> 172.21.15.114:0/4101285427 conn(0x7fa9000ff330 msgr2=0x7fa900101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 shutdown_connections 2024-09-21T07:01:44.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:44.395+0000 7fa904c00640 1 -- 172.21.15.114:0/4101285427 wait complete. 2024-09-21T07:01:44.529 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:01:45.041 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:44 smithi114 ceph-mon[30654]: pgmap v626: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 905 op/s 2024-09-21T07:01:45.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:44 smithi183 ceph-mon[34219]: pgmap v626: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 905 op/s 2024-09-21T07:01:45.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:44 smithi193 ceph-mon[34601]: pgmap v626: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 14 MiB/s wr, 905 op/s 2024-09-21T07:01:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.679+0000 7f520b200640 1 -- 172.21.15.114:0/2629045445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520410dbf0 msgr2=0x7f520410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.679+0000 7f520b200640 1 --2- 172.21.15.114:0/2629045445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520410dbf0 0x7f520410e000 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009860 tx=0x7f51f402a340 comp rx=0 tx=0).stop 2024-09-21T07:01:45.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.683+0000 7f520b200640 1 -- 172.21.15.114:0/2629045445 shutdown_connections 2024-09-21T07:01:45.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.683+0000 7f520b200640 1 --2- 172.21.15.114:0/2629045445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520411cba0 0x7f520411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.684+0000 7f520b200640 1 --2- 172.21.15.114:0/2629045445 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f520410dbf0 0x7f520410e000 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.684+0000 7f520b200640 1 --2- 172.21.15.114:0/2629045445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5204111cd0 0x7f520410d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.684+0000 7f520b200640 1 -- 172.21.15.114:0/2629045445 >> 172.21.15.114:0/2629045445 conn(0x7f520406e9f0 msgr2=0x7f520406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.684+0000 7f520b200640 1 -- 172.21.15.114:0/2629045445 shutdown_connections 2024-09-21T07:01:45.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.684+0000 7f520b200640 1 -- 172.21.15.114:0/2629045445 wait complete. 2024-09-21T07:01:45.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.687+0000 7f520b200640 1 Processor -- start 2024-09-21T07:01:45.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.688+0000 7f520b200640 1 -- start start 2024-09-21T07:01:45.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.689+0000 7f520b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 0x7f52041a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f520b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5204111cd0 0x7f52041a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f520b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520411cba0 0x7f52041ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f520b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52041226d0 con 0x7f5204111cd0 2024-09-21T07:01:45.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f520b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5204122550 con 0x7f520411cba0 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f520b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5204122850 con 0x7f520410dbf0 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f5209e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 0x7f52041a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f5209e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 0x7f52041a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48378/0 (socket says 172.21.15.114:48378) 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f5209e00640 1 -- 172.21.15.114:0/430634198 learned_addr learned my addr 172.21.15.114:0/430634198 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.690+0000 7f5209400640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5204111cd0 0x7f52041a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.692+0000 7f5209e00640 1 -- 172.21.15.114:0/430634198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520411cba0 msgr2=0x7f52041ae9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.692+0000 7f5209e00640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520411cba0 0x7f52041ae9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.692+0000 7f5209e00640 1 -- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5204111cd0 msgr2=0x7f52041a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.692+0000 7f5209e00640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5204111cd0 0x7f52041a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.692+0000 7f5209e00640 1 -- 172.21.15.114:0/430634198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52041af010 con 0x7f520410dbf0 2024-09-21T07:01:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.692+0000 7f5209e00640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 0x7f52041a9160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51f800c970 tx=0x7f51f800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.693+0000 7f51fea00640 1 -- 172.21.15.114:0/430634198 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8007c10 con 0x7f520410dbf0 2024-09-21T07:01:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.693+0000 7f51fea00640 1 -- 172.21.15.114:0/430634198 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f8007db0 con 0x7f520410dbf0 2024-09-21T07:01:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.693+0000 7f51fea00640 1 -- 172.21.15.114:0/430634198 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f800f840 con 0x7f520410dbf0 2024-09-21T07:01:45.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.693+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52041af300 con 0x7f520410dbf0 2024-09-21T07:01:45.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.698+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041b5b20 con 0x7f520410dbf0 2024-09-21T07:01:45.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.708+0000 7f51fea00640 1 -- 172.21.15.114:0/430634198 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f51f8018b70 con 0x7f520410dbf0 2024-09-21T07:01:45.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.709+0000 7f5213406740 1 Processor -- start 2024-09-21T07:01:45.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.710+0000 7f5213406740 1 -- start start 2024-09-21T07:01:45.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.711+0000 7f5213406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c21cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.711+0000 7f5213406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.712+0000 7f520a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c21cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.712+0000 7f520a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c21cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48384/0 (socket says 172.21.15.114:48384) 2024-09-21T07:01:45.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.712+0000 7f5213406740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c22770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.712+0000 7f5213406740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5648e7b98f70 con 0x5648e785a390 2024-09-21T07:01:45.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.712+0000 7f5213406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648e7b883a0 con 0x5648e78521b0 2024-09-21T07:01:45.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.712+0000 7f5213406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5648e7b8c1b0 con 0x5648e787b070 2024-09-21T07:01:45.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.713+0000 7f520a800640 1 -- 172.21.15.114:0/3736779501 learned_addr learned my addr 172.21.15.114:0/3736779501 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.715+0000 7f520a800640 1 -- 172.21.15.114:0/3736779501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 msgr2=0x5648e7c22230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:01:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.715+0000 7f520a800640 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c22230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.715+0000 7f520a800640 1 -- 172.21.15.114:0/3736779501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 msgr2=0x5648e7c22770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.715+0000 7f520a800640 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c22770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.715+0000 7f520a800640 1 -- 172.21.15.114:0/3736779501 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e7b52c90 con 0x5648e787b070 2024-09-21T07:01:45.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.716+0000 7f51fea00640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51ec0b6760 0x7f51ec0b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.720+0000 7f520a800640 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c21cf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5200007bd0 tx=0x7f520000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2b9e1cea87ef5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:45.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.722+0000 7f5209e00640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51ec0b6760 0x7f51ec0b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.722+0000 7f51fea00640 1 -- 172.21.15.114:0/430634198 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f51f80e7770 con 0x7f520410dbf0 2024-09-21T07:01:45.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.724+0000 7f51f3000640 1 -- 172.21.15.114:0/3736779501 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200014070 con 0x5648e787b070 2024-09-21T07:01:45.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.724+0000 7f51f3000640 1 -- 172.21.15.114:0/3736779501 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f520000fb00 con 0x5648e787b070 2024-09-21T07:01:45.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.724+0000 7f51f3000640 1 -- 172.21.15.114:0/3736779501 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200014070 con 0x5648e787b070 2024-09-21T07:01:45.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.725+0000 7f5213406740 1 -- 172.21.15.114:0/3736779501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 msgr2=0x5648e7c21cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.725+0000 7f5213406740 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c21cf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5200007bd0 tx=0x7f520000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.725+0000 7f5213406740 1 -- 172.21.15.114:0/3736779501 shutdown_connections 2024-09-21T07:01:45.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.725+0000 7f5213406740 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c22770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.725+0000 7f5213406740 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c22230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.725+0000 7f5213406740 1 --2- 172.21.15.114:0/3736779501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c21cf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.726+0000 7f5213406740 1 -- 172.21.15.114:0/3736779501 >> 172.21.15.114:0/3736779501 conn(0x5648e786f460 msgr2=0x5648e7c1fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.726+0000 7f5213406740 1 -- 172.21.15.114:0/3736779501 shutdown_connections 2024-09-21T07:01:45.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.726+0000 7f5213406740 1 -- 172.21.15.114:0/3736779501 wait complete. 2024-09-21T07:01:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.726+0000 7f5213406740 1 -- start start 2024-09-21T07:01:45.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.727+0000 7f5213406740 1 Processor -- start 2024-09-21T07:01:45.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.727+0000 7f5209e00640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51ec0b6760 0x7f51ec0b8c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x5648e7c20970 tx=0x7f51f8004700 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.727+0000 7f5213406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c2d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5213406740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c2dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5213406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5213406740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5648e79e9700 con 0x5648e785a390 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5213406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648e7b23540 con 0x5648e78521b0 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5213406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5648e7c304d0 con 0x5648e787b070 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5213406740 10 client.0 _ll_register_callbacks cb 0x5648e7b33430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f520a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c2d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f520a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c2d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48062/0 (socket says 172.21.15.114:48062) 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f520a800640 1 -- 172.21.15.114:0/3297811684 learned_addr learned my addr 172.21.15.114:0/3297811684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.728+0000 7f5209400640 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c2dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.729+0000 7f5209400640 1 -- 172.21.15.114:0/3297811684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 msgr2=0x5648e7c397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.729+0000 7f5209400640 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.729+0000 7f5209400640 1 -- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 msgr2=0x5648e7c2d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.729+0000 7f5209400640 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c2d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.729+0000 7f5209400640 1 -- 172.21.15.114:0/3297811684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e78e6650 con 0x5648e785a390 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.730+0000 7f5209400640 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c2dd90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009780 tx=0x7f51f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80be8e17b9cd68e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.730+0000 7f51e7400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.731+0000 7f51e7400640 1 -- 172.21.15.114:0/3297811684 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4041020 con 0x5648e785a390 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.731+0000 7f51e7400640 1 -- 172.21.15.114:0/3297811684 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f51f4030650 con 0x5648e785a390 2024-09-21T07:01:45.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.731+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5648e7b25b50 con 0x5648e785a390 2024-09-21T07:01:45.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.736+0000 7f51e7400640 1 -- 172.21.15.114:0/3297811684 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f51f4037040 con 0x5648e785a390 2024-09-21T07:01:45.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.737+0000 7f5213406740 10 client.17106 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:45.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.737+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5648e7b40910 con 0x5648e785a390 2024-09-21T07:01:45.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.740+0000 7f51e7400640 1 -- 172.21.15.114:0/3297811684 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5648e7b25b50 con 0x5648e785a390 2024-09-21T07:01:45.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.740+0000 7f51e7400640 1 -- 172.21.15.114:0/3297811684 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f51f403d480 con 0x5648e785a390 2024-09-21T07:01:45.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.744+0000 7f5213406740 10 client.17106 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:45.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.744+0000 7f5213406740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:45.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.744+0000 7f5213406740 20 client.17106 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:45.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.744+0000 7f5213406740 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e78399f0 0x5648e7c3f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.744+0000 7f5213406740 4 client.17106 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:45.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.744+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648e7b9aec0 con 0x5648e78399f0 2024-09-21T07:01:45.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.745+0000 7f520a800640 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e78399f0 0x5648e7c3f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.746+0000 7f520a800640 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e78399f0 0x5648e7c3f820 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ad1392e30e4170 server_cookie=4475f90d288cf194 in_seq=0 out_seq=0 2024-09-21T07:01:45.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.747+0000 7f51e7400640 10 client.17106 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:45.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.748+0000 7f51e7400640 1 -- 172.21.15.114:0/3297811684 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5648e7b9aec0 con 0x5648e78399f0 2024-09-21T07:01:45.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.749+0000 7f51e7400640 10 client.17106 handle_command_reply: tid=0 2024-09-21T07:01:45.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.749+0000 7f5213406740 1 client.17106 shutdown 2024-09-21T07:01:45.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.749+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 msgr2=0x5648e7c2dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.749+0000 7f5213406740 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c2dd90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009780 tx=0x7f51f4004270 comp rx=0 tx=0).stop 2024-09-21T07:01:45.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 shutdown_connections 2024-09-21T07:01:45.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e78399f0 0x5648e7c3f820 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e787b070 0x5648e7c397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e785a390 0x5648e7c2dd90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 --2- 172.21.15.114:0/3297811684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c2d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 >> 172.21.15.114:0/3297811684 conn(0x5648e786f460 msgr2=0x5648e7c22d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:45.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f51e7400640 0 client.17106 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:45.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.750+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 shutdown_connections 2024-09-21T07:01:45.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.752+0000 7f5213406740 1 -- 172.21.15.114:0/3297811684 wait complete. 2024-09-21T07:01:45.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.752+0000 7f5213406740 20 client.17106 trim_cache size 0 max 16384 2024-09-21T07:01:45.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.811+0000 7f5213406740 1 Processor -- start 2024-09-21T07:01:45.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.818+0000 7f5213406740 1 -- start start 2024-09-21T07:01:45.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.819+0000 7f5213406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c27e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.819+0000 7f5213406740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c2c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.819+0000 7f5213406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c2a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.819+0000 7f5213406740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52041bfc00 con 0x5648e787b070 2024-09-21T07:01:45.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.819+0000 7f5213406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c17d0 con 0x5648e78521b0 2024-09-21T07:01:45.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.819+0000 7f5213406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c02d0 con 0x5648e780eda0 2024-09-21T07:01:45.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.822+0000 7f520a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c27e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.822+0000 7f520a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c27e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48072/0 (socket says 172.21.15.114:48072) 2024-09-21T07:01:45.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.822+0000 7f520a800640 1 -- 172.21.15.114:0/2563344198 learned_addr learned my addr 172.21.15.114:0/2563344198 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.822+0000 7f520a800640 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c2a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f5209400640 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c2c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f520a800640 1 -- 172.21.15.114:0/2563344198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 msgr2=0x5648e7c2a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f520a800640 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c2a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f520a800640 1 -- 172.21.15.114:0/2563344198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 msgr2=0x5648e7c2c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f520a800640 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c2c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f520a800640 1 -- 172.21.15.114:0/2563344198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e7c2e910 con 0x5648e78521b0 2024-09-21T07:01:45.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.823+0000 7f520a800640 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c27e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5200000f80 tx=0x7f520000e4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4fc913e0f5bd8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:45.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.824+0000 7f51f2600640 1 -- 172.21.15.114:0/2563344198 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200014040 con 0x5648e78521b0 2024-09-21T07:01:45.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.824+0000 7f51f2600640 1 -- 172.21.15.114:0/2563344198 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200003e20 con 0x5648e78521b0 2024-09-21T07:01:45.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.824+0000 7f51f2600640 1 -- 172.21.15.114:0/2563344198 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200004020 con 0x5648e78521b0 2024-09-21T07:01:45.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.826+0000 7f5213406740 1 -- 172.21.15.114:0/2563344198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 msgr2=0x5648e7c27e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.826+0000 7f5213406740 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c27e10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5200000f80 tx=0x7f520000e4e0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.830+0000 7f5213406740 1 -- 172.21.15.114:0/2563344198 shutdown_connections 2024-09-21T07:01:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.830+0000 7f5213406740 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c2a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.830+0000 7f5213406740 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c2c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.830+0000 7f5213406740 1 --2- 172.21.15.114:0/2563344198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c27e10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.830+0000 7f5213406740 1 -- 172.21.15.114:0/2563344198 >> 172.21.15.114:0/2563344198 conn(0x5648e786f460 msgr2=0x5648e7c2bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.831+0000 7f5213406740 1 -- 172.21.15.114:0/2563344198 shutdown_connections 2024-09-21T07:01:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.831+0000 7f5213406740 1 -- 172.21.15.114:0/2563344198 wait complete. 2024-09-21T07:01:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.832+0000 7f5213406740 1 -- start start 2024-09-21T07:01:45.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.832+0000 7f5213406740 1 Processor -- start 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.836+0000 7f5213406740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.836+0000 7f5213406740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c35c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.836+0000 7f5213406740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c29830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.836+0000 7f5213406740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c1100 con 0x5648e787b070 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.837+0000 7f5213406740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c09a0 con 0x5648e78521b0 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.837+0000 7f5213406740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52041bdc00 con 0x5648e780eda0 2024-09-21T07:01:45.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.837+0000 7f5213406740 10 client.0 _ll_register_callbacks cb 0x7f51f4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:45.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.838+0000 7f520a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.838+0000 7f520a800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c35750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48416/0 (socket says 172.21.15.114:48416) 2024-09-21T07:01:45.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.838+0000 7f520a800640 1 -- 172.21.15.114:0/151410108 learned_addr learned my addr 172.21.15.114:0/151410108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:45.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.838+0000 7f5209400640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c35c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.840+0000 7f520a800640 1 -- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 msgr2=0x5648e7c35c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.840+0000 7f520a800640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c35c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.840+0000 7f520a800640 1 -- 172.21.15.114:0/151410108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 msgr2=0x5648e7c29830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:45.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.840+0000 7f520a800640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c29830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.841+0000 7f520a800640 1 -- 172.21.15.114:0/151410108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e7c29d70 con 0x5648e780eda0 2024-09-21T07:01:45.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.841+0000 7f5209400640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c35c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:45.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.842+0000 7f520a800640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c35750 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f520000cc20 tx=0x7f52000072e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e6ae3ba1da8a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:45.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.842+0000 7f51e7400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:01:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.843+0000 7f51e7400640 1 -- 172.21.15.114:0/151410108 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200014050 con 0x5648e780eda0 2024-09-21T07:01:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.843+0000 7f51e7400640 1 -- 172.21.15.114:0/151410108 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f5200004b20 con 0x5648e780eda0 2024-09-21T07:01:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.844+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5200003e20 con 0x5648e780eda0 2024-09-21T07:01:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.844+0000 7f51e7400640 1 -- 172.21.15.114:0/151410108 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5200034040 con 0x5648e780eda0 2024-09-21T07:01:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.844+0000 7f5213406740 10 client.27421 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:45.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.844+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5648e7cbb840 con 0x5648e780eda0 2024-09-21T07:01:45.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.845+0000 7f51e7400640 1 -- 172.21.15.114:0/151410108 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200003e20 con 0x5648e780eda0 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.846+0000 7f51e7400640 1 -- 172.21.15.114:0/151410108 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f5200004190 con 0x5648e780eda0 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.846+0000 7f5213406740 10 client.27421 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.846+0000 7f5213406740 10 client.27421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.846+0000 7f5213406740 20 client.27421 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.846+0000 7f5213406740 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e7cbbd90 0x5648e7cbe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.847+0000 7f5213406740 4 client.27421 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:01:45.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.847+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648e7b96970 con 0x5648e7cbbd90 2024-09-21T07:01:45.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.847+0000 7f5209400640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e7cbbd90 0x5648e7cbe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:45.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.851+0000 7f5209400640 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e7cbbd90 0x5648e7cbe1b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379d583a97a22cc1 server_cookie=68c9e99c1cea0ed3 in_seq=0 out_seq=0 2024-09-21T07:01:45.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.851+0000 7f51e7400640 10 client.27421 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:45.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.852+0000 7f51e7400640 1 -- 172.21.15.114:0/151410108 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x5648e7b96970 con 0x5648e7cbbd90 2024-09-21T07:01:45.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.852+0000 7f51e7400640 10 client.27421 handle_command_reply: tid=0 2024-09-21T07:01:45.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.852+0000 7f5213406740 1 client.27421 shutdown 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 msgr2=0x5648e7c35750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c35750 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f520000cc20 tx=0x7f52000072e0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 shutdown_connections 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5648e787b070 0x5648e7c29830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648e78521b0 0x5648e7c35c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5648e7cbbd90 0x5648e7cbe1b0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 --2- 172.21.15.114:0/151410108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5648e780eda0 0x5648e7c35750 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.853+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 >> 172.21.15.114:0/151410108 conn(0x5648e786f460 msgr2=0x5648e7c1e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:45.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.854+0000 7f51e7400640 0 client.27421 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:45.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.854+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 shutdown_connections 2024-09-21T07:01:45.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.854+0000 7f5213406740 1 -- 172.21.15.114:0/151410108 wait complete. 2024-09-21T07:01:45.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.854+0000 7f5213406740 20 client.27421 trim_cache size 0 max 16384 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.858 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:01:45.859 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.860 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.861 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:01:45.862 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:01:45.863 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:01:45.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:01:45.863 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:01:45.863 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.855+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51ec0b6760 msgr2=0x7f51ec0b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.855+0000 7f520b200640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51ec0b6760 0x7f51ec0b8c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x5648e7c20970 tx=0x7f51f8004700 comp rx=0 tx=0).stop 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 msgr2=0x7f52041a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 0x7f52041a9160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51f800c970 tx=0x7f51f800ce30 comp rx=0 tx=0).stop 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 shutdown_connections 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f51ec0b6760 0x7f51ec0b8c70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f520411cba0 0x7f52041ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5204111cd0 0x7f52041a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 --2- 172.21.15.114:0/430634198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f520410dbf0 0x7f52041a9160 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:45.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.856+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 >> 172.21.15.114:0/430634198 conn(0x7f520406e9f0 msgr2=0x7f5204110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:45.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.858+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 shutdown_connections 2024-09-21T07:01:45.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:45.858+0000 7f520b200640 1 -- 172.21.15.114:0/430634198 wait complete. 2024-09-21T07:01:47.184 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:46 smithi114 ceph-mon[30654]: pgmap v627: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 12 MiB/s wr, 784 op/s 2024-09-21T07:01:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:46 smithi183 ceph-mon[34219]: pgmap v627: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 12 MiB/s wr, 784 op/s 2024-09-21T07:01:47.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:46 smithi193 ceph-mon[34601]: pgmap v627: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 12 MiB/s wr, 784 op/s 2024-09-21T07:01:47.710 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:01:47.711 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:01:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 430.0 s, 377.2 tps, lat 141.658 ms stddev 187.841 2024-09-21T07:01:48.407 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:01:49.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:48 smithi114 ceph-mon[30654]: pgmap v628: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-21T07:01:49.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.319+0000 7f2e23800640 1 -- 172.21.15.114:0/2437195668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 msgr2=0x7f2e1c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.320+0000 7f2e23800640 1 --2- 172.21.15.114:0/2437195668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c10d0e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009210 tx=0x7f2e1002b3e0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.321+0000 7f2e23800640 1 -- 172.21.15.114:0/2437195668 shutdown_connections 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.321+0000 7f2e23800640 1 --2- 172.21.15.114:0/2437195668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10d620 0x7f2e1c117090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.321+0000 7f2e23800640 1 --2- 172.21.15.114:0/2437195668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c10d0e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.321+0000 7f2e23800640 1 --2- 172.21.15.114:0/2437195668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10a650 0x7f2e1c10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.321+0000 7f2e23800640 1 -- 172.21.15.114:0/2437195668 >> 172.21.15.114:0/2437195668 conn(0x7f2e1c070ec0 msgr2=0x7f2e1c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:49.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.322+0000 7f2e23800640 1 -- 172.21.15.114:0/2437195668 shutdown_connections 2024-09-21T07:01:49.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.322+0000 7f2e23800640 1 -- 172.21.15.114:0/2437195668 wait complete. 2024-09-21T07:01:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.323+0000 7f2e23800640 1 Processor -- start 2024-09-21T07:01:49.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.324+0000 7f2e23800640 1 -- start start 2024-09-21T07:01:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.325+0000 7f2e23800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10a650 0x7f2e1c1146c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.325+0000 7f2e23800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c114c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:49.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.325+0000 7f2e23800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 0x7f2e1c125bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:49.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.325+0000 7f2e20e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10a650 0x7f2e1c1146c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:49.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.325+0000 7f2e20e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10a650 0x7f2e1c1146c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48108/0 (socket says 172.21.15.114:48108) 2024-09-21T07:01:49.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.326+0000 7f2e1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c114c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:49.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.326+0000 7f2e1be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c114c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44022/0 (socket says 172.21.15.114:44022) 2024-09-21T07:01:49.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.326+0000 7f2e21800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 0x7f2e1c125bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:49.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.326+0000 7f2e21800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 0x7f2e1c125bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48430/0 (socket says 172.21.15.114:48430) 2024-09-21T07:01:49.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.327+0000 7f2e20e00640 1 -- 172.21.15.114:0/1465103492 learned_addr learned my addr 172.21.15.114:0/1465103492 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:49.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.328+0000 7f2e23800640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c119e30 con 0x7f2e1c10ccd0 2024-09-21T07:01:49.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.328+0000 7f2e23800640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c119cb0 con 0x7f2e1c10a650 2024-09-21T07:01:49.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.328+0000 7f2e23800640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1c119fb0 con 0x7f2e1c10d620 2024-09-21T07:01:49.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.330+0000 7f2e21800640 1 -- 172.21.15.114:0/1465103492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10a650 msgr2=0x7f2e1c1146c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:49.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.330+0000 7f2e21800640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10a650 0x7f2e1c1146c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.330+0000 7f2e21800640 1 -- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 msgr2=0x7f2e1c114c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:49.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.330+0000 7f2e21800640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c114c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.330+0000 7f2e21800640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e1c1154a0 con 0x7f2e1c10d620 2024-09-21T07:01:49.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.330+0000 7f2e21800640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 0x7f2e1c125bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400be20 tx=0x7f2e1400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:49.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.332+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14019070 con 0x7f2e1c10d620 2024-09-21T07:01:49.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.332+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e140148f0 con 0x7f2e1c10d620 2024-09-21T07:01:49.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.332+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14012900 con 0x7f2e1c10d620 2024-09-21T07:01:49.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.332+0000 7f2e23800640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e1c1261f0 con 0x7f2e1c10d620 2024-09-21T07:01:49.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.333+0000 7f2e23800640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e1c126730 con 0x7f2e1c10d620 2024-09-21T07:01:49.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.334+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e1c10b870 con 0x7f2e1c10d620 2024-09-21T07:01:49.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.339+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f2e1401c070 con 0x7f2e1c10d620 2024-09-21T07:01:49.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.340+0000 7f2e19600640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2e040b6700 0x7f2e040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:49.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.340+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f2e140e71b0 con 0x7f2e1c10d620 2024-09-21T07:01:49.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.341+0000 7f2e20e00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2e040b6700 0x7f2e040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:49.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.342+0000 7f2e20e00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2e040b6700 0x7f2e040b8c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005fd0 tx=0x7f2e0800b480 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:49.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.344+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2e140b1710 con 0x7f2e1c10d620 2024-09-21T07:01:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:48 smithi183 ceph-mon[34219]: pgmap v628: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-21T07:01:49.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:48 smithi193 ceph-mon[34601]: pgmap v628: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-21T07:01:49.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.795+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e1c1159c0 con 0x7f2e1c10d620 2024-09-21T07:01:49.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.798+0000 7f2e19600640 1 -- 172.21.15.114:0/1465103492 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11994 (secure 0 0 0) 0x7f2e140b6650 con 0x7f2e1c10d620 2024-09-21T07:01:49.803 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:01:49.803 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":46,"btime":"2024-09-21T07:01:16:269196+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:01:15.619367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:01:49.803 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 46 2024-09-21T07:01:49.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2e040b6700 msgr2=0x7f2e040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:49.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2e040b6700 0x7f2e040b8c10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2e08005fd0 tx=0x7f2e0800b480 comp rx=0 tx=0).stop 2024-09-21T07:01:49.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 msgr2=0x7f2e1c125bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 0x7f2e1c125bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400be20 tx=0x7f2e1400bf10 comp rx=0 tx=0).stop 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 shutdown_connections 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2e1c10d620 0x7f2e1c125bf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2e1c10ccd0 0x7f2e1c114c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2e040b6700 0x7f2e040b8c10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 --2- 172.21.15.114:0/1465103492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e1c10a650 0x7f2e1c1146c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:49.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.803+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 >> 172.21.15.114:0/1465103492 conn(0x7f2e1c070ec0 msgr2=0x7f2e1c10dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:49.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.804+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 shutdown_connections 2024-09-21T07:01:49.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:49.805+0000 7f2e0ea00640 1 -- 172.21.15.114:0/1465103492 wait complete. 2024-09-21T07:01:50.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1465103492' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:01:50.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1465103492' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:01:50.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1465103492' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:01:51.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:50 smithi114 ceph-mon[30654]: pgmap v629: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-21T07:01:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:50 smithi183 ceph-mon[34219]: pgmap v629: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-21T07:01:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:50 smithi193 ceph-mon[34601]: pgmap v629: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 19 MiB/s wr, 1.24k op/s 2024-09-21T07:01:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 435.0 s, 442.4 tps, lat 330.328 ms stddev 700.299 2024-09-21T07:01:53.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:52 smithi114 ceph-mon[30654]: pgmap v630: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-21T07:01:53.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:52 smithi183 ceph-mon[34219]: pgmap v630: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-21T07:01:53.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:52 smithi193 ceph-mon[34601]: pgmap v630: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 21 MiB/s wr, 1.41k op/s 2024-09-21T07:01:54.540 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:01:54.959 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:01:54.959 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T07:01:54.959 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:01:55.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:54 smithi114 ceph-mon[30654]: pgmap v631: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-09-21T07:01:55.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:54 smithi183 ceph-mon[34219]: pgmap v631: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-09-21T07:01:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:54 smithi193 ceph-mon[34601]: pgmap v631: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 22 MiB/s wr, 1.39k op/s 2024-09-21T07:01:55.461 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:01:57.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:56 smithi114 ceph-mon[30654]: pgmap v632: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-09-21T07:01:57.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:56 smithi183 ceph-mon[34219]: pgmap v632: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-09-21T07:01:57.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:56 smithi193 ceph-mon[34601]: pgmap v632: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 18 MiB/s wr, 1.20k op/s 2024-09-21T07:01:57.426 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:01:57.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.530+0000 7f5450800640 1 -- 172.21.15.114:0/1410323065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448114440 msgr2=0x7f5448116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.530+0000 7f5450800640 1 --2- 172.21.15.114:0/1410323065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448114440 0x7f5448116910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f544401acb0 tx=0x7f544403b900 comp rx=0 tx=0).stop 2024-09-21T07:01:57.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.532+0000 7f5450800640 1 -- 172.21.15.114:0/1410323065 shutdown_connections 2024-09-21T07:01:57.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.532+0000 7f5450800640 1 --2- 172.21.15.114:0/1410323065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448114440 0x7f5448116910 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.532+0000 7f5450800640 1 --2- 172.21.15.114:0/1410323065 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448075dd0 0x7f54480761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.532+0000 7f5450800640 1 --2- 172.21.15.114:0/1410323065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448077380 0x7f54480757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.532+0000 7f5450800640 1 -- 172.21.15.114:0/1410323065 >> 172.21.15.114:0/1410323065 conn(0x7f54480ff5e0 msgr2=0x7f5448101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:57.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.544+0000 7f5450800640 1 -- 172.21.15.114:0/1410323065 shutdown_connections 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.545+0000 7f5450800640 1 -- 172.21.15.114:0/1410323065 wait complete. 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.547+0000 7f5450800640 1 Processor -- start 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.547+0000 7f5450800640 1 -- start start 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.548+0000 7f5450800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448075dd0 0x7f5448113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.548+0000 7f5450800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448077380 0x7f544810fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.548+0000 7f5450800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448114440 0x7f54481103c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.548+0000 7f5450800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f544810a420 con 0x7f5448114440 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.548+0000 7f5450800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f544810a2a0 con 0x7f5448077380 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.548+0000 7f5450800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f544810a5a0 con 0x7f5448075dd0 2024-09-21T07:01:57.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.549+0000 7f544e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448114440 0x7f54481103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.550+0000 7f544de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448075dd0 0x7f5448113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.550+0000 7f544de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448075dd0 0x7f5448113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46726/0 (socket says 172.21.15.114:46726) 2024-09-21T07:01:57.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.550+0000 7f544de00640 1 -- 172.21.15.114:0/2422284016 learned_addr learned my addr 172.21.15.114:0/2422284016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:57.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.550+0000 7f544d400640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448077380 0x7f544810fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.551+0000 7f544e800640 1 -- 172.21.15.114:0/2422284016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448075dd0 msgr2=0x7f5448113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.551+0000 7f544e800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448075dd0 0x7f5448113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.551+0000 7f544e800640 1 -- 172.21.15.114:0/2422284016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448077380 msgr2=0x7f544810fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.551+0000 7f544e800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448077380 0x7f544810fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.551+0000 7f544e800640 1 -- 172.21.15.114:0/2422284016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5448110b50 con 0x7f5448114440 2024-09-21T07:01:57.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.553+0000 7f544e800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448114440 0x7f54481103c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f544403be20 tx=0x7f5444003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:57.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.553+0000 7f5442a00640 1 -- 172.21.15.114:0/2422284016 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444011070 con 0x7f5448114440 2024-09-21T07:01:57.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.554+0000 7f5442a00640 1 -- 172.21.15.114:0/2422284016 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5444040eb0 con 0x7f5448114440 2024-09-21T07:01:57.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.554+0000 7f5442a00640 1 -- 172.21.15.114:0/2422284016 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444045070 con 0x7f5448114440 2024-09-21T07:01:57.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.554+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54481c65b0 con 0x7f5448114440 2024-09-21T07:01:57.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.554+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481c69b0 con 0x7f5448114440 2024-09-21T07:01:57.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.565+0000 7f545740b740 1 Processor -- start 2024-09-21T07:01:57.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.568+0000 7f5442a00640 1 -- 172.21.15.114:0/2422284016 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5444014070 con 0x7f5448114440 2024-09-21T07:01:57.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.570+0000 7f5442a00640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54300b6780 0x7f54300b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.571+0000 7f5442a00640 1 -- 172.21.15.114:0/2422284016 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f5444108880 con 0x7f5448114440 2024-09-21T07:01:57.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.572+0000 7f544d400640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54300b6780 0x7f54300b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.573+0000 7f544d400640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54300b6780 0x7f54300b8c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f54481162f0 tx=0x7f54380090e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:57.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.575+0000 7f545740b740 1 -- start start 2024-09-21T07:01:57.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.576+0000 7f545740b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x557847016ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.576+0000 7f545740b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c471b0 0x5578470170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.576+0000 7f545740b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 0x557847017620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.576+0000 7f545740b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557846f8dfb0 con 0x557846c471b0 2024-09-21T07:01:57.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.576+0000 7f545740b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557846f7d3e0 con 0x557846c70070 2024-09-21T07:01:57.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.576+0000 7f545740b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557846f811f0 con 0x557846c4f390 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 0x557847017620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 0x557847017620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46730/0 (socket says 172.21.15.114:46730) 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 -- 172.21.15.114:0/1588416975 learned_addr learned my addr 172.21.15.114:0/1588416975 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 -- 172.21.15.114:0/1588416975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 msgr2=0x557847016ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x557847016ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 -- 172.21.15.114:0/1588416975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c471b0 msgr2=0x5578470170e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.577+0000 7f544e800640 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c471b0 0x5578470170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.578+0000 7f544e800640 1 -- 172.21.15.114:0/1588416975 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557846f47cd0 con 0x557846c4f390 2024-09-21T07:01:57.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.578+0000 7f544e800640 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 0x557847017620 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f54440ce050 tx=0x7f54440cddc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4319474354ff153 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:57.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.578+0000 7f5437000640 1 -- 172.21.15.114:0/1588416975 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54440d7cc0 con 0x557846c4f390 2024-09-21T07:01:57.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.579+0000 7f5437000640 1 -- 172.21.15.114:0/1588416975 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5444004160 con 0x557846c4f390 2024-09-21T07:01:57.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.579+0000 7f5437000640 1 -- 172.21.15.114:0/1588416975 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54440c9ca0 con 0x557846c4f390 2024-09-21T07:01:57.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.580+0000 7f545740b740 1 -- 172.21.15.114:0/1588416975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 msgr2=0x557847017620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.580+0000 7f545740b740 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 0x557847017620 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f54440ce050 tx=0x7f54440cddc0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.582+0000 7f545740b740 1 -- 172.21.15.114:0/1588416975 shutdown_connections 2024-09-21T07:01:57.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.582+0000 7f545740b740 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c4f390 0x557847017620 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.582+0000 7f545740b740 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c471b0 0x5578470170e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.582+0000 7f545740b740 1 --2- 172.21.15.114:0/1588416975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x557847016ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.582+0000 7f545740b740 1 -- 172.21.15.114:0/1588416975 >> 172.21.15.114:0/1588416975 conn(0x557846c64460 msgr2=0x557847014a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:57.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.583+0000 7f545740b740 1 -- 172.21.15.114:0/1588416975 shutdown_connections 2024-09-21T07:01:57.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.583+0000 7f545740b740 1 -- 172.21.15.114:0/1588416975 wait complete. 2024-09-21T07:01:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.585+0000 7f545740b740 1 -- start start 2024-09-21T07:01:57.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.585+0000 7f545740b740 1 Processor -- start 2024-09-21T07:01:57.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.587+0000 7f545740b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 0x5578470227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.587+0000 7f545740b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c4f390 0x557847022cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.587+0000 7f544de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 0x5578470227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.587+0000 7f544de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 0x5578470227b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46736/0 (socket says 172.21.15.114:46736) 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.587+0000 7f544de00640 1 -- 172.21.15.114:0/3953206467 learned_addr learned my addr 172.21.15.114:0/3953206467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.587+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c70070 0x55784702e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.588+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557846dde700 con 0x557846c70070 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.588+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557846f18580 con 0x557846c4f390 2024-09-21T07:01:57.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.588+0000 7f544e800640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c70070 0x55784702e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.588+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557847025140 con 0x557846c471b0 2024-09-21T07:01:57.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.589+0000 7f545740b740 10 client.0 _ll_register_callbacks cb 0x557846f28470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.590+0000 7f544de00640 1 -- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c4f390 msgr2=0x557847022cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.590+0000 7f544de00640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c4f390 0x557847022cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.590+0000 7f544de00640 1 -- 172.21.15.114:0/3953206467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c70070 msgr2=0x55784702e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.590+0000 7f544de00640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c70070 0x55784702e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.590+0000 7f544de00640 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557846cdb650 con 0x557846c471b0 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.591+0000 7f544e800640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c70070 0x55784702e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.591+0000 7f542b400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:01:57.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.591+0000 7f544de00640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 0x5578470227b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f543c00f9e0 tx=0x7f543c00fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcce94fd89f83cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.592+0000 7f542b400640 1 -- 172.21.15.114:0/3953206467 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c01d060 con 0x557846c471b0 2024-09-21T07:01:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.592+0000 7f542b400640 1 -- 172.21.15.114:0/3953206467 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f543c00dc80 con 0x557846c471b0 2024-09-21T07:01:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.592+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557846f1ab90 con 0x557846c471b0 2024-09-21T07:01:57.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.593+0000 7f542b400640 1 -- 172.21.15.114:0/3953206467 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f543c00de80 con 0x557846c471b0 2024-09-21T07:01:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.593+0000 7f545740b740 10 client.27436 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.593+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557846f35950 con 0x557846c471b0 2024-09-21T07:01:57.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.593+0000 7f542b400640 1 -- 172.21.15.114:0/3953206467 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557846f1ab90 con 0x557846c471b0 2024-09-21T07:01:57.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f542b400640 1 -- 172.21.15.114:0/3953206467 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f543c0105a0 con 0x557846c471b0 2024-09-21T07:01:57.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f545740b740 10 client.27436 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:57.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f545740b740 10 client.27436 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:01:57.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f545740b740 20 client.27436 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:57.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557846c2e9f0 0x557847034920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f545740b740 4 client.27436 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.595+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557846f8ff00 con 0x557846c2e9f0 2024-09-21T07:01:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.596+0000 7f544d400640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557846c2e9f0 0x557847034920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.596+0000 7f544d400640 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557846c2e9f0 0x557847034920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd42c210de9d9b3d server_cookie=7f17ec4dfad80328 in_seq=0 out_seq=0 2024-09-21T07:01:57.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.596+0000 7f542b400640 10 client.27436 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.599+0000 7f542b400640 1 -- 172.21.15.114:0/3953206467 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557846f8ff00 con 0x557846c2e9f0 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.599+0000 7f542b400640 10 client.27436 handle_command_reply: tid=0 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.599+0000 7f545740b740 1 client.27436 shutdown 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 msgr2=0x5578470227b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 0x5578470227b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f543c00f9e0 tx=0x7f543c00fea0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 shutdown_connections 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c70070 0x55784702e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x557846c2e9f0 0x557847034920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c4f390 0x557847022cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 --2- 172.21.15.114:0/3953206467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c471b0 0x5578470227b0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f543c00f9e0 tx=0x7f543c00fea0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 >> 172.21.15.114:0/3953206467 conn(0x557846c64460 msgr2=0x557847019d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:57.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.600+0000 7f542b400640 0 client.27436 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:01:57.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.603+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 shutdown_connections 2024-09-21T07:01:57.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.603+0000 7f545740b740 1 -- 172.21.15.114:0/3953206467 wait complete. 2024-09-21T07:01:57.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.603+0000 7f545740b740 20 client.27436 trim_cache size 0 max 16384 2024-09-21T07:01:57.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.631+0000 7f545740b740 1 Processor -- start 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 -- start start 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x55784701bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x55784701c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 0x557847014040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f54481d0ad0 con 0x557846c4f390 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54481d26a0 con 0x557846c70070 2024-09-21T07:01:57.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.632+0000 7f545740b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54481d11a0 con 0x557846c03da0 2024-09-21T07:01:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.633+0000 7f544d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x55784701c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.633+0000 7f544e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 0x557847014040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.633+0000 7f544e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 0x557847014040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46746/0 (socket says 172.21.15.114:46746) 2024-09-21T07:01:57.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.633+0000 7f544e800640 1 -- 172.21.15.114:0/239098335 learned_addr learned my addr 172.21.15.114:0/239098335 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f544e800640 1 -- 172.21.15.114:0/239098335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 msgr2=0x55784701c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f544e800640 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x55784701c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f544e800640 1 -- 172.21.15.114:0/239098335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 msgr2=0x55784701bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f544e800640 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x55784701bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f544e800640 1 -- 172.21.15.114:0/239098335 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557847018e90 con 0x557846c03da0 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f544e800640 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 0x557847014040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f54440ce2e0 tx=0x7f544403b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44020dca35a7a6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f5436600640 1 -- 172.21.15.114:0/239098335 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54440e9070 con 0x557846c03da0 2024-09-21T07:01:57.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f5436600640 1 -- 172.21.15.114:0/239098335 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54440d7cc0 con 0x557846c03da0 2024-09-21T07:01:57.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.634+0000 7f5436600640 1 -- 172.21.15.114:0/239098335 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54440e9070 con 0x557846c03da0 2024-09-21T07:01:57.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.636+0000 7f545740b740 1 -- 172.21.15.114:0/239098335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 msgr2=0x557847014040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.636+0000 7f545740b740 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 0x557847014040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f54440ce2e0 tx=0x7f544403b900 comp rx=0 tx=0).stop 2024-09-21T07:01:57.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.636+0000 7f545740b740 1 -- 172.21.15.114:0/239098335 shutdown_connections 2024-09-21T07:01:57.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.637+0000 7f545740b740 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c03da0 0x557847014040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.637+0000 7f545740b740 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c70070 0x55784701c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.637+0000 7f545740b740 1 --2- 172.21.15.114:0/239098335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x55784701bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.637+0000 7f545740b740 1 -- 172.21.15.114:0/239098335 >> 172.21.15.114:0/239098335 conn(0x557846c64460 msgr2=0x557847020e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:57.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.637+0000 7f545740b740 1 -- 172.21.15.114:0/239098335 shutdown_connections 2024-09-21T07:01:57.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.637+0000 7f545740b740 1 -- 172.21.15.114:0/239098335 wait complete. 2024-09-21T07:01:57.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.639+0000 7f545740b740 1 -- start start 2024-09-21T07:01:57.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.640+0000 7f545740b740 1 Processor -- start 2024-09-21T07:01:57.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.640+0000 7f545740b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c03da0 0x557847035120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.640+0000 7f545740b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x5578470356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.640+0000 7f545740b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c70070 0x557847039f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.641+0000 7f544de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c03da0 0x557847035120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.641+0000 7f544de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c03da0 0x557847035120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57172/0 (socket says 172.21.15.114:57172) 2024-09-21T07:01:57.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.641+0000 7f544e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c70070 0x557847039f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.642+0000 7f544e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c70070 0x557847039f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46754/0 (socket says 172.21.15.114:46754) 2024-09-21T07:01:57.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.642+0000 7f544de00640 1 -- 172.21.15.114:0/4075747382 learned_addr learned my addr 172.21.15.114:0/4075747382 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:57.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.642+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f54481d1fd0 con 0x557846c4f390 2024-09-21T07:01:57.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.642+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54481d1870 con 0x557846c03da0 2024-09-21T07:01:57.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.642+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54481ceb60 con 0x557846c70070 2024-09-21T07:01:57.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.642+0000 7f545740b740 10 client.0 _ll_register_callbacks cb 0x7f543c00f5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:57.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.643+0000 7f544d400640 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x5578470356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.644+0000 7f544d400640 1 -- 172.21.15.114:0/4075747382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c70070 msgr2=0x557847039f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.644+0000 7f544d400640 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c70070 0x557847039f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.644+0000 7f544d400640 1 -- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c03da0 msgr2=0x557847035120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:57.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.644+0000 7f544d400640 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c03da0 0x557847035120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.644+0000 7f544d400640 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55784703a5c0 con 0x557846c4f390 2024-09-21T07:01:57.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.646+0000 7f544d400640 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x5578470356c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x557847032860 tx=0x7f5438004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e42abdc1d384667 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:57.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.646+0000 7f542b400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:01:57.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.647+0000 7f542b400640 1 -- 172.21.15.114:0/4075747382 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543802d060 con 0x557846c4f390 2024-09-21T07:01:57.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.647+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54440d7cc0 con 0x557846c4f390 2024-09-21T07:01:57.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.648+0000 7f542b400640 1 -- 172.21.15.114:0/4075747382 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f5438028460 con 0x557846c4f390 2024-09-21T07:01:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.649+0000 7f542b400640 1 -- 172.21.15.114:0/4075747382 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5438034050 con 0x557846c4f390 2024-09-21T07:01:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.649+0000 7f545740b740 10 client.17121 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.649+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5578470b0ca0 con 0x557846c4f390 2024-09-21T07:01:57.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f542b400640 1 -- 172.21.15.114:0/4075747382 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54440d7cc0 con 0x557846c4f390 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f542b400640 1 -- 172.21.15.114:0/4075747382 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f54380137d0 con 0x557846c4f390 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f545740b740 10 client.17121 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f545740b740 10 client.17121 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f545740b740 20 client.17121 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f545740b740 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5578470b13d0 0x5578470b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f545740b740 4 client.17121 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.650+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557846f8b9b0 con 0x5578470b13d0 2024-09-21T07:01:57.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.651+0000 7f544de00640 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5578470b13d0 0x5578470b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.652+0000 7f542b400640 10 client.17121 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:01:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.652+0000 7f544de00640 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5578470b13d0 0x5578470b37f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12db46662868e07a server_cookie=5321f49a67feeba in_seq=0 out_seq=0 2024-09-21T07:01:57.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.653+0000 7f542b400640 1 -- 172.21.15.114:0/4075747382 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557846f8b9b0 con 0x5578470b13d0 2024-09-21T07:01:57.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.653+0000 7f542b400640 10 client.17121 handle_command_reply: tid=0 2024-09-21T07:01:57.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.653+0000 7f545740b740 1 client.17121 shutdown 2024-09-21T07:01:57.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.653+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 msgr2=0x5578470356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.653+0000 7f545740b740 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x5578470356c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x557847032860 tx=0x7f5438004770 comp rx=0 tx=0).stop 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 shutdown_connections 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557846c70070 0x557847039f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557846c4f390 0x5578470356c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5578470b13d0 0x5578470b37f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 --2- 172.21.15.114:0/4075747382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557846c03da0 0x557847035120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 >> 172.21.15.114:0/4075747382 conn(0x557846c64460 msgr2=0x55784701bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:57.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f542b400640 0 client.17121 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:01:57.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 shutdown_connections 2024-09-21T07:01:57.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.654+0000 7f545740b740 1 -- 172.21.15.114:0/4075747382 wait complete. 2024-09-21T07:01:57.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.655+0000 7f545740b740 20 client.17121 trim_cache size 0 max 16384 2024-09-21T07:01:57.658 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:01:57.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54300b6780 msgr2=0x7f54300b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54300b6780 0x7f54300b8c90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f54481162f0 tx=0x7f54380090e0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448114440 msgr2=0x7f54481103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448114440 0x7f54481103c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f544403be20 tx=0x7f5444003ba0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 shutdown_connections 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5448114440 0x7f54481103c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54300b6780 0x7f54300b8c90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5448077380 0x7f544810fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 --2- 172.21.15.114:0/2422284016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5448075dd0 0x7f5448113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 >> 172.21.15.114:0/2422284016 conn(0x7f54480ff5e0 msgr2=0x7f54481035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 shutdown_connections 2024-09-21T07:01:57.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:57.656+0000 7f5450800640 1 -- 172.21.15.114:0/2422284016 wait complete. 2024-09-21T07:01:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 440.0 s, 479.2 tps, lat 210.279 ms stddev 250.318 2024-09-21T07:01:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 -- 172.21.15.114:0/385595368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 msgr2=0x7f7d58104b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 --2- 172.21.15.114:0/385595368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d58104b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009900 tx=0x7f7d5002a420 comp rx=0 tx=0).stop 2024-09-21T07:01:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 -- 172.21.15.114:0/385595368 shutdown_connections 2024-09-21T07:01:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 --2- 172.21.15.114:0/385595368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d5810dba0 0x7f7d581100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 --2- 172.21.15.114:0/385595368 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d581050d0 0x7f7d5810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 --2- 172.21.15.114:0/385595368 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d58104b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 -- 172.21.15.114:0/385595368 >> 172.21.15.114:0/385595368 conn(0x7f7d580fe230 msgr2=0x7f7d58100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 -- 172.21.15.114:0/385595368 shutdown_connections 2024-09-21T07:01:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.589+0000 7f7d5de00640 1 -- 172.21.15.114:0/385595368 wait complete. 2024-09-21T07:01:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.591+0000 7f7d5de00640 1 Processor -- start 2024-09-21T07:01:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.591+0000 7f7d5de00640 1 -- start start 2024-09-21T07:01:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d5819a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d581050d0 0x7f7d5819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d5810dba0 0x7f7d5819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5de00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581138d0 con 0x7f7d5810dba0 2024-09-21T07:01:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58113750 con 0x7f7d58104780 2024-09-21T07:01:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5de00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58113a50 con 0x7f7d581050d0 2024-09-21T07:01:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.592+0000 7f7d5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d5819a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d5819a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57184/0 (socket says 172.21.15.114:57184) 2024-09-21T07:01:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 -- 172.21.15.114:0/1938650071 learned_addr learned my addr 172.21.15.114:0/1938650071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d57e00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d581050d0 0x7f7d5819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 -- 172.21.15.114:0/1938650071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d581050d0 msgr2=0x7f7d5819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d581050d0 0x7f7d5819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 -- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d5810dba0 msgr2=0x7f7d5819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d5810dba0 0x7f7d5819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.593+0000 7f7d5ca00640 1 -- 172.21.15.114:0/1938650071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d581a0280 con 0x7f7d58104780 2024-09-21T07:01:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.594+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d5819a350 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005ab0 tx=0x7f7d50003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.596+0000 7f7d55600640 1 -- 172.21.15.114:0/1938650071 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5003f070 con 0x7f7d58104780 2024-09-21T07:01:58.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.596+0000 7f7d55600640 1 -- 172.21.15.114:0/1938650071 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d50031d60 con 0x7f7d58104780 2024-09-21T07:01:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.596+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d581a0510 con 0x7f7d58104780 2024-09-21T07:01:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.596+0000 7f7d55600640 1 -- 172.21.15.114:0/1938650071 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50035030 con 0x7f7d58104780 2024-09-21T07:01:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.596+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d58102390 con 0x7f7d58104780 2024-09-21T07:01:58.604 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.601+0000 7f7d665c7740 1 Processor -- start 2024-09-21T07:01:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.602+0000 7f7d55600640 1 -- 172.21.15.114:0/1938650071 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7d50042070 con 0x7f7d58104780 2024-09-21T07:01:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.602+0000 7f7d665c7740 1 -- start start 2024-09-21T07:01:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.603+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fc86e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.603+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc87350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.603+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8c14d0 0x55e55fc87890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.603+0000 7f7d665c7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e55fbfc6b0 con 0x55e55f8e21b0 2024-09-21T07:01:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.603+0000 7f7d665c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e55fbebae0 con 0x55e55f8c14d0 2024-09-21T07:01:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.603+0000 7f7d665c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e55fbef8f0 con 0x55e55f8b92f0 2024-09-21T07:01:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.604+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fc86e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.604+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fc86e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48390/0 (socket says 172.21.15.114:48390) 2024-09-21T07:01:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d5d400640 1 -- 172.21.15.114:0/2652316093 learned_addr learned my addr 172.21.15.114:0/2652316093 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d57e00640 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8c14d0 0x55e55fc87890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc87350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d55600640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7d3c0b67d0 0x7f7d3c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d57e00640 1 -- 172.21.15.114:0/2652316093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 msgr2=0x55e55fc87350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d57e00640 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc87350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d57e00640 1 -- 172.21.15.114:0/2652316093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 msgr2=0x55e55fc86e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d57e00640 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fc86e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.605+0000 7f7d57e00640 1 -- 172.21.15.114:0/2652316093 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55fbb8780 con 0x55e55f8c14d0 2024-09-21T07:01:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.606+0000 7f7d55600640 1 -- 172.21.15.114:0/1938650071 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f7d50108490 con 0x7f7d58104780 2024-09-21T07:01:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.606+0000 7f7d57e00640 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8c14d0 0x55e55fc87890 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d5806c500 tx=0x7f7d480096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62dd6d88e310309c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.606+0000 7f7d4dc00640 1 -- 172.21.15.114:0/2652316093 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48013790 con 0x55e55f8c14d0 2024-09-21T07:01:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.606+0000 7f7d4dc00640 1 -- 172.21.15.114:0/2652316093 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d48019070 con 0x55e55f8c14d0 2024-09-21T07:01:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.606+0000 7f7d4dc00640 1 -- 172.21.15.114:0/2652316093 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48014c80 con 0x55e55f8c14d0 2024-09-21T07:01:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.607+0000 7f7d5d400640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7d3c0b67d0 0x7f7d3c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.608+0000 7f7d665c7740 1 -- 172.21.15.114:0/2652316093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8c14d0 msgr2=0x55e55fc87890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.608+0000 7f7d665c7740 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8c14d0 0x55e55fc87890 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d5806c500 tx=0x7f7d480096d0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.610+0000 7f7d5d400640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7d3c0b67d0 0x7f7d3c0b8ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000acb0 tx=0x7f7d4000a7e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 -- 172.21.15.114:0/2652316093 shutdown_connections 2024-09-21T07:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8c14d0 0x55e55fc87890 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc87350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 --2- 172.21.15.114:0/2652316093 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fc86e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 -- 172.21.15.114:0/2652316093 >> 172.21.15.114:0/2652316093 conn(0x55e55f8d6300 msgr2=0x55e55fc84c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 -- 172.21.15.114:0/2652316093 shutdown_connections 2024-09-21T07:01:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.615+0000 7f7d665c7740 1 -- 172.21.15.114:0/2652316093 wait complete. 2024-09-21T07:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.619+0000 7f7d665c7740 1 -- start start 2024-09-21T07:01:58.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.619+0000 7f7d665c7740 1 Processor -- start 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.621+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc92d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.621+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.621+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 0x55e55fc98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.621+0000 7f7d665c7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e55fb23ab0 con 0x55e55f8c14d0 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.622+0000 7f7d665c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e55fb92570 con 0x55e55f8e21b0 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.622+0000 7f7d665c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e55fc8b4e0 con 0x55e55f8b92f0 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.622+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 0x55e55fc98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.622+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 0x55e55fc98300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57200/0 (socket says 172.21.15.114:57200) 2024-09-21T07:01:58.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.622+0000 7f7d5d400640 1 -- 172.21.15.114:0/3704622881 learned_addr learned my addr 172.21.15.114:0/3704622881 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d5d400640 1 -- 172.21.15.114:0/3704622881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 msgr2=0x55e55fc92d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:01:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d5d400640 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc92d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d5d400640 1 -- 172.21.15.114:0/3704622881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 msgr2=0x55e55fc932d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:01:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d5d400640 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc932d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d5d400640 1 -- 172.21.15.114:0/3704622881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55f94d790 con 0x55e55f8e21b0 2024-09-21T07:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d665c7740 10 client.0 _ll_register_callbacks cb 0x55e55fba1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d5d400640 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 0x55e55fc98300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d400126b0 tx=0x7f7d4000a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60790247cf26ea5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.623+0000 7f7d46000640 10 client.27512 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.624+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d48019070 con 0x55e55f8e21b0 2024-09-21T07:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.624+0000 7f7d46000640 1 -- 172.21.15.114:0/3704622881 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4001d9c0 con 0x55e55f8e21b0 2024-09-21T07:01:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.624+0000 7f7d46000640 1 -- 172.21.15.114:0/3704622881 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f7d4001db60 con 0x55e55f8e21b0 2024-09-21T07:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.625+0000 7f7d46000640 1 -- 172.21.15.114:0/3704622881 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7d4001ddc0 con 0x55e55f8e21b0 2024-09-21T07:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.625+0000 7f7d665c7740 10 client.27512 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:58.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.625+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=46}) -- 0x55e55f9c6620 con 0x55e55f8e21b0 2024-09-21T07:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.626+0000 7f7d46000640 1 -- 172.21.15.114:0/3704622881 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f7d40024070 con 0x55e55f8e21b0 2024-09-21T07:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.626+0000 7f7d665c7740 10 client.27512 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.626+0000 7f7d665c7740 10 client.27512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.626+0000 7f7d665c7740 20 client.27512 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:58.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.626+0000 7f7d665c7740 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8a0d80 0x55e55fcc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.630 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.627+0000 7f7d665c7740 4 client.27512 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.627+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e55fbfe600 con 0x55e55f8a0d80 2024-09-21T07:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.627+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8a0d80 0x55e55fcc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.629+0000 7f7d46000640 10 client.27512 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:58.631 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.629+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8a0d80 0x55e55fcc6580 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25c6376359ddc6f server_cookie=bb98d2b96e04a3bd in_seq=0 out_seq=0 2024-09-21T07:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d46000640 1 -- 172.21.15.114:0/3704622881 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e55fbfe600 con 0x55e55f8a0d80 2024-09-21T07:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d46000640 10 client.27512 handle_command_reply: tid=0 2024-09-21T07:01:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 client.27512 shutdown 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 msgr2=0x55e55fc98300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 0x55e55fc98300 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d400126b0 tx=0x7f7d4000a2d0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 shutdown_connections 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8e21b0 0x55e55fc98300 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8a0d80 0x55e55fcc6580 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 --2- 172.21.15.114:0/3704622881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8b92f0 0x55e55fc92d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.633 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 >> 172.21.15.114:0/3704622881 conn(0x55e55f8d6300 msgr2=0x55e55fc8a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.630+0000 7f7d46000640 0 client.27512 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.631+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 shutdown_connections 2024-09-21T07:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.631+0000 7f7d665c7740 1 -- 172.21.15.114:0/3704622881 wait complete. 2024-09-21T07:01:58.634 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.631+0000 7f7d665c7740 20 client.27512 trim_cache size 0 max 16384 2024-09-21T07:01:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 Processor -- start 2024-09-21T07:01:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 -- start start 2024-09-21T07:01:58.667 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fc96ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8e21b0 0x55e55fc8de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58074c30 con 0x55e55f8c14d0 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a7f10 con 0x55e55f8b92f0 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.664+0000 7f7d665c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58075300 con 0x55e55f8e21b0 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.665+0000 7f7d5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fc96ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.665+0000 7f7d5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fc96ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57210/0 (socket says 172.21.15.114:57210) 2024-09-21T07:01:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.665+0000 7f7d57e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.665+0000 7f7d5ca00640 1 -- 172.21.15.114:0/162060514 learned_addr learned my addr 172.21.15.114:0/162060514 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.665+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8e21b0 0x55e55fc8de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.666+0000 7f7d5ca00640 1 -- 172.21.15.114:0/162060514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8e21b0 msgr2=0x55e55fc8de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.666+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8e21b0 0x55e55fc8de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.666+0000 7f7d5ca00640 1 -- 172.21.15.114:0/162060514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 msgr2=0x55e55fc83360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.666+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc83360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.666+0000 7f7d5ca00640 1 -- 172.21.15.114:0/162060514 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e55fc85690 con 0x55e55f8b92f0 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.667+0000 7f7d5d400640 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8e21b0 0x55e55fc8de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.667+0000 7f7d57e00640 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc83360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:01:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.668+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fc96ad0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7d500ca4e0 tx=0x7f7d50031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5a387886fb5246 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.668+0000 7f7d4d200640 1 -- 172.21.15.114:0/162060514 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50049040 con 0x55e55f8b92f0 2024-09-21T07:01:58.674 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.669+0000 7f7d4d200640 1 -- 172.21.15.114:0/162060514 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d500ce610 con 0x55e55f8b92f0 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.669+0000 7f7d4d200640 1 -- 172.21.15.114:0/162060514 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5003b560 con 0x55e55f8b92f0 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.670+0000 7f7d665c7740 1 -- 172.21.15.114:0/162060514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 msgr2=0x55e55fc96ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.670+0000 7f7d665c7740 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fc96ad0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7d500ca4e0 tx=0x7f7d50031400 comp rx=0 tx=0).stop 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.670+0000 7f7d665c7740 1 -- 172.21.15.114:0/162060514 shutdown_connections 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.671+0000 7f7d665c7740 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8e21b0 0x55e55fc8de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.671+0000 7f7d665c7740 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8c14d0 0x55e55fc83360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.671+0000 7f7d665c7740 1 --2- 172.21.15.114:0/162060514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fc96ad0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.671+0000 7f7d665c7740 1 -- 172.21.15.114:0/162060514 >> 172.21.15.114:0/162060514 conn(0x55e55f8d6300 msgr2=0x55e55fc8d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.672+0000 7f7d665c7740 1 -- 172.21.15.114:0/162060514 shutdown_connections 2024-09-21T07:01:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.672+0000 7f7d665c7740 1 -- 172.21.15.114:0/162060514 wait complete. 2024-09-21T07:01:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.673+0000 7f7d665c7740 1 -- start start 2024-09-21T07:01:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.673+0000 7f7d665c7740 1 Processor -- start 2024-09-21T07:01:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fcd8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8c14d0 0x55e55fcd39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fcd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a7840 con 0x55e55f8e21b0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a70e0 con 0x55e55f8b92f0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58072c30 con 0x55e55f8c14d0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d665c7740 10 client.0 _ll_register_callbacks cb 0x7f7d4000af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fcd8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fcd8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57218/0 (socket says 172.21.15.114:57218) 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 -- 172.21.15.114:0/302307191 learned_addr learned my addr 172.21.15.114:0/302307191 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d57e00640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8c14d0 0x55e55fcd39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5d400640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fcd3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 -- 172.21.15.114:0/302307191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8c14d0 msgr2=0x55e55fcd39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8c14d0 0x55e55fcd39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 -- 172.21.15.114:0/302307191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 msgr2=0x55e55fcd3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:01:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.674+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fcd3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.675+0000 7f7d5ca00640 1 -- 172.21.15.114:0/302307191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e55fcd46b0 con 0x55e55f8b92f0 2024-09-21T07:01:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.675+0000 7f7d5ca00640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fcd8500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7d500cdea0 tx=0x7f7d5003b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63517f676a1bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.675+0000 7f7d46000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:01:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.675+0000 7f7d46000640 1 -- 172.21.15.114:0/302307191 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50049030 con 0x55e55f8b92f0 2024-09-21T07:01:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.675+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d500ce610 con 0x55e55f8b92f0 2024-09-21T07:01:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.675+0000 7f7d46000640 1 -- 172.21.15.114:0/302307191 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f7d5003b560 con 0x55e55f8b92f0 2024-09-21T07:01:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.677+0000 7f7d46000640 1 -- 172.21.15.114:0/302307191 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7d500d29f0 con 0x55e55f8b92f0 2024-09-21T07:01:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.677+0000 7f7d665c7740 10 client.27518 fetch_fsmap learned FSMap version 46 2024-09-21T07:01:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.677+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e55fd30ea0 con 0x55e55f8b92f0 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.677+0000 7f7d46000640 1 -- 172.21.15.114:0/302307191 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d500ce610 con 0x55e55f8b92f0 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d46000640 1 -- 172.21.15.114:0/302307191 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f7d50108150 con 0x55e55f8b92f0 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d665c7740 10 client.27518 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d665c7740 10 client.27518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d665c7740 20 client.27518 populate_metadata read hostname 'smithi114' 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d665c7740 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8761c0 0x55e55fd33500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d665c7740 4 client.27518 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:01:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.678+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e55fbfa0b0 con 0x55e55f8761c0 2024-09-21T07:01:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.684+0000 7f7d57e00640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8761c0 0x55e55fd33500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.685+0000 7f7d46000640 10 client.27518 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.685+0000 7f7d57e00640 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8761c0 0x55e55fd33500 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672dade83333af24 server_cookie=7558447e16920708 in_seq=0 out_seq=0 2024-09-21T07:01:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.704+0000 7f7d46000640 1 -- 172.21.15.114:0/302307191 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e55fbfa0b0 con 0x55e55f8761c0 2024-09-21T07:01:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.704+0000 7f7d46000640 10 client.27518 handle_command_reply: tid=0 2024-09-21T07:01:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.704+0000 7f7d665c7740 1 client.27518 shutdown 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 msgr2=0x55e55fcd8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fcd8500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7d500cdea0 tx=0x7f7d5003b900 comp rx=0 tx=0).stop 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 shutdown_connections 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e55f8e21b0 0x55e55fcd3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e55f8761c0 0x55e55fd33500 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e55f8c14d0 0x55e55fcd39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 --2- 172.21.15.114:0/302307191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e55f8b92f0 0x55e55fcd8500 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d46000640 0 client.27518 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:01:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 >> 172.21.15.114:0/302307191 conn(0x55e55f8d6300 msgr2=0x55e55fc968a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 shutdown_connections 2024-09-21T07:01:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.705+0000 7f7d665c7740 1 -- 172.21.15.114:0/302307191 wait complete. 2024-09-21T07:01:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.706+0000 7f7d665c7740 20 client.27518 trim_cache size 0 max 16384 2024-09-21T07:01:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:01:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:01:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:01:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:01:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7d3c0b67d0 msgr2=0x7f7d3c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7d3c0b67d0 0x7f7d3c0b8ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000acb0 tx=0x7f7d4000a7e0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 msgr2=0x7f7d5819a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d5819a350 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7d50005ab0 tx=0x7f7d50003d50 comp rx=0 tx=0).stop 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 shutdown_connections 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7d5810dba0 0x7f7d5819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7d581050d0 0x7f7d5819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7d3c0b67d0 0x7f7d3c0b8ce0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 --2- 172.21.15.114:0/1938650071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d58104780 0x7f7d5819a350 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.708+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 >> 172.21.15.114:0/1938650071 conn(0x7f7d580fe230 msgr2=0x7f7d581076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.712+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 shutdown_connections 2024-09-21T07:01:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:01:58.712+0000 7f7d5de00640 1 -- 172.21.15.114:0/1938650071 wait complete. 2024-09-21T07:01:59.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:58 smithi114 ceph-mon[30654]: pgmap v633: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-09-21T07:01:59.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.280+0000 7ff7f3400640 1 -- 172.21.15.114:0/2245054764 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00318a0 con 0x7ff7f4076100 2024-09-21T07:01:59.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.281+0000 7ff7fb400640 1 -- 172.21.15.114:0/2245054764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 msgr2=0x7ff7f410fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:59.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.281+0000 7ff7fb400640 1 --2- 172.21.15.114:0/2245054764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 0x7ff7f410fb90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009900 tx=0x7ff7e002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 -- 172.21.15.114:0/2245054764 shutdown_connections 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 --2- 172.21.15.114:0/2245054764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f41100d0 0x7ff7f4112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 --2- 172.21.15.114:0/2245054764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 0x7ff7f410fb90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 --2- 172.21.15.114:0/2245054764 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f40757b0 0x7ff7f4075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 -- 172.21.15.114:0/2245054764 >> 172.21.15.114:0/2245054764 conn(0x7ff7f40ff5e0 msgr2=0x7ff7f4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 -- 172.21.15.114:0/2245054764 shutdown_connections 2024-09-21T07:01:59.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.282+0000 7ff7fb400640 1 -- 172.21.15.114:0/2245054764 wait complete. 2024-09-21T07:01:59.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.284+0000 7ff7fb400640 1 Processor -- start 2024-09-21T07:01:59.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.286+0000 7ff7fb400640 1 -- start start 2024-09-21T07:01:59.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.288+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 0x7ff7f410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:59.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.288+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 0x7ff7f410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:59.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.288+0000 7ff7fb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f41100d0 0x7ff7f410ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:59.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.288+0000 7ff7fb400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4104130 con 0x7ff7f41100d0 2024-09-21T07:01:59.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.288+0000 7ff7fb400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f4103fb0 con 0x7ff7f40757b0 2024-09-21T07:01:59.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.288+0000 7ff7fb400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f41042b0 con 0x7ff7f4076100 2024-09-21T07:01:59.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.289+0000 7ff7f8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 0x7ff7f410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:59.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.289+0000 7ff7f8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 0x7ff7f410e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57236/0 (socket says 172.21.15.114:57236) 2024-09-21T07:01:59.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.289+0000 7ff7f8a00640 1 -- 172.21.15.114:0/3740188653 learned_addr learned my addr 172.21.15.114:0/3740188653 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.289+0000 7ff7f8a00640 1 -- 172.21.15.114:0/3740188653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 msgr2=0x7ff7f410e6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.289+0000 7ff7f8a00640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 0x7ff7f410e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.289+0000 7ff7f8a00640 1 -- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f41100d0 msgr2=0x7ff7f410ec20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.290+0000 7ff7f9400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f41100d0 0x7ff7f410ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.290+0000 7ff7f8a00640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f41100d0 0x7ff7f410ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.290+0000 7ff7f8a00640 1 -- 172.21.15.114:0/3740188653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f41acb10 con 0x7ff7f40757b0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.290+0000 7ff7f9400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f41100d0 0x7ff7f410ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.290+0000 7ff7f8a00640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 0x7ff7f410e180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800d950 tx=0x7ff7e800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.291+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8017030 con 0x7ff7f40757b0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.291+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e8004020 con 0x7ff7f40757b0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.291+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8003c50 con 0x7ff7f40757b0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.291+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f41ace00 con 0x7ff7f40757b0 2024-09-21T07:01:59.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.291+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f41ad260 con 0x7ff7f40757b0 2024-09-21T07:01:59.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.292+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f4075bc0 con 0x7ff7f40757b0 2024-09-21T07:01:59.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.295+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff7e801a070 con 0x7ff7f40757b0 2024-09-21T07:01:59.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.297+0000 7ff7f1600640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7dc0b6700 0x7ff7dc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:01:59.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.297+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7ff7e80e7b00 con 0x7ff7f40757b0 2024-09-21T07:01:59.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.298+0000 7ff7f3e00640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7dc0b6700 0x7ff7dc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:01:59.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.299+0000 7ff7f3e00640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7dc0b6700 0x7ff7dc0b8c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009230 tx=0x7ff7e0005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:01:59.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.301+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff7e80add00 con 0x7ff7f40757b0 2024-09-21T07:01:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:58 smithi183 ceph-mon[34219]: pgmap v633: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-09-21T07:01:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:58 smithi193 ceph-mon[34601]: pgmap v633: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 19 MiB/s wr, 1.31k op/s 2024-09-21T07:01:59.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.701+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f410f480 con 0x7ff7f40757b0 2024-09-21T07:01:59.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.705+0000 7ff7f1600640 1 -- 172.21.15.114:0/3740188653 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11994 (secure 0 0 0) 0x7ff7e80b20e0 con 0x7ff7f40757b0 2024-09-21T07:01:59.710 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:01:59.711 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":46,"btime":"2024-09-21T07:01:16:269196+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:01:15.619367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:01:59.711 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 46 2024-09-21T07:01:59.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.712+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7dc0b6700 msgr2=0x7ff7dc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:59.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.712+0000 7ff7fb400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7dc0b6700 0x7ff7dc0b8c10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009230 tx=0x7ff7e0005fd0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.712+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 msgr2=0x7ff7f410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:01:59.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.712+0000 7ff7fb400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 0x7ff7f410e180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff7e800d950 tx=0x7ff7e800de10 comp rx=0 tx=0).stop 2024-09-21T07:01:59.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.713+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 shutdown_connections 2024-09-21T07:01:59.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.713+0000 7ff7fb400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff7f41100d0 0x7ff7f410ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.713+0000 7ff7fb400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7f4076100 0x7ff7f410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.713+0000 7ff7fb400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff7dc0b6700 0x7ff7dc0b8c10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.713+0000 7ff7fb400640 1 --2- 172.21.15.114:0/3740188653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7f40757b0 0x7ff7f410e180 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:01:59.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.713+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 >> 172.21.15.114:0/3740188653 conn(0x7ff7f40ff5e0 msgr2=0x7ff7f4106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:01:59.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.715+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 shutdown_connections 2024-09-21T07:01:59.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:01:59.715+0000 7ff7fb400640 1 -- 172.21.15.114:0/3740188653 wait complete. 2024-09-21T07:01:59.801 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:02:00.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:01:59 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3740188653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:00.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:01:59 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3740188653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:00.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:01:59 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3740188653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:00.619 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T07:02:00.620 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T07:02:00.658 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:01.126 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:00 smithi114 ceph-mon[30654]: pgmap v634: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.4 MiB/s wr, 653 op/s 2024-09-21T07:02:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:00 smithi183 ceph-mon[34219]: pgmap v634: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.4 MiB/s wr, 653 op/s 2024-09-21T07:02:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:00 smithi193 ceph-mon[34601]: pgmap v634: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.4 MiB/s wr, 653 op/s 2024-09-21T07:02:01.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 -- 172.21.15.114:0/636369359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad8111f40 msgr2=0x7f6ad810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 --2- 172.21.15.114:0/636369359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad8111f40 0x7f6ad810a3c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac402a400 comp rx=0 tx=0).stop 2024-09-21T07:02:01.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 -- 172.21.15.114:0/636369359 shutdown_connections 2024-09-21T07:02:01.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 --2- 172.21.15.114:0/636369359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 0x7f6ad811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 --2- 172.21.15.114:0/636369359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad810a9e0 0x7f6ad810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 --2- 172.21.15.114:0/636369359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad8111f40 0x7f6ad810a3c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 -- 172.21.15.114:0/636369359 >> 172.21.15.114:0/636369359 conn(0x7f6ad8070ec0 msgr2=0x7f6ad80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:01.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 -- 172.21.15.114:0/636369359 shutdown_connections 2024-09-21T07:02:01.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.536+0000 7f6ade400640 1 -- 172.21.15.114:0/636369359 wait complete. 2024-09-21T07:02:01.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.538+0000 7f6ade400640 1 Processor -- start 2024-09-21T07:02:01.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.539+0000 7f6ade400640 1 -- start start 2024-09-21T07:02:01.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ade400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad810a9e0 0x7f6ad811beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ade400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad8111f40 0x7f6ad811c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ade400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 0x7f6ad8116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ade400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad8112830 con 0x7f6ad811cc30 2024-09-21T07:02:01.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ade400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81126b0 con 0x7f6ad810a9e0 2024-09-21T07:02:01.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ade400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81129b0 con 0x7f6ad8111f40 2024-09-21T07:02:01.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad8111f40 0x7f6ad811c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad8111f40 0x7f6ad811c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46822/0 (socket says 172.21.15.114:46822) 2024-09-21T07:02:01.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ad6a00640 1 -- 172.21.15.114:0/3740473855 learned_addr learned my addr 172.21.15.114:0/3740473855 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.540+0000 7f6ad7e00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 0x7f6ad8116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.541+0000 7f6ad7400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad810a9e0 0x7f6ad811beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.541+0000 7f6ad7e00640 1 -- 172.21.15.114:0/3740473855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad8111f40 msgr2=0x7f6ad811c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.541+0000 7f6ad7e00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad8111f40 0x7f6ad811c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.541+0000 7f6ad7e00640 1 -- 172.21.15.114:0/3740473855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad810a9e0 msgr2=0x7f6ad811beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.541+0000 7f6ad7e00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad810a9e0 0x7f6ad811beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.541+0000 7f6ad7e00640 1 -- 172.21.15.114:0/3740473855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad8116900 con 0x7f6ad811cc30 2024-09-21T07:02:01.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.542+0000 7f6ad7e00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 0x7f6ad8116170 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000da10 tx=0x7f6ad000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:01.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.542+0000 7f6acfe00640 1 -- 172.21.15.114:0/3740473855 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000bd70 con 0x7f6ad811cc30 2024-09-21T07:02:01.547 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:01.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.543+0000 7f6acfe00640 1 -- 172.21.15.114:0/3740473855 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ad0003e20 con 0x7f6ad811cc30 2024-09-21T07:02:01.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.543+0000 7f6acfe00640 1 -- 172.21.15.114:0/3740473855 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00106c0 con 0x7f6ad811cc30 2024-09-21T07:02:01.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.544+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad8116bf0 con 0x7f6ad811cc30 2024-09-21T07:02:01.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.544+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad81be1a0 con 0x7f6ad811cc30 2024-09-21T07:02:01.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.553+0000 7f6ae50a3740 1 Processor -- start 2024-09-21T07:02:01.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.553+0000 7f6acfe00640 1 -- 172.21.15.114:0/3740473855 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6ad0004270 con 0x7f6ad811cc30 2024-09-21T07:02:01.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 -- start start 2024-09-21T07:02:01.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560849e9dfb0 con 0x560849b80070 2024-09-21T07:02:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560849e8d3e0 con 0x560849b571b0 2024-09-21T07:02:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.554+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560849e911f0 con 0x560849b5f390 2024-09-21T07:02:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.555+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f27820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.555+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f27820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46824/0 (socket says 172.21.15.114:46824) 2024-09-21T07:02:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.556+0000 7f6ad6a00640 1 -- 172.21.15.114:0/1143520396 learned_addr learned my addr 172.21.15.114:0/1143520396 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:01.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.557+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.557+0000 7f6acfe00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6abc0b6780 0x7f6abc0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.558+0000 7f6ad6a00640 1 -- 172.21.15.114:0/1143520396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 msgr2=0x560849f272e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.558+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f272e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.558+0000 7f6ad6a00640 1 -- 172.21.15.114:0/1143520396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 msgr2=0x560849f26da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.558+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f26da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.558+0000 7f6ad6a00640 1 -- 172.21.15.114:0/1143520396 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560849e57cd0 con 0x560849b5f390 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.559+0000 7f6acfe00640 1 -- 172.21.15.114:0/3740473855 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f6ad00e7e80 con 0x7f6ad811cc30 2024-09-21T07:02:01.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.559+0000 7f6ad7e00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6abc0b6780 0x7f6abc0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.564+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f27820 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8011840 tx=0x7f6ac800a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54e56cb5f12b506 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:01.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.565+0000 7f6ad7e00640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6abc0b6780 0x7f6abc0b8c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00a9e70 tx=0x7f6ad0022000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.566+0000 7f6accc00640 1 -- 172.21.15.114:0/1143520396 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8004030 con 0x560849b5f390 2024-09-21T07:02:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.566+0000 7f6accc00640 1 -- 172.21.15.114:0/1143520396 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac8005020 con 0x560849b5f390 2024-09-21T07:02:01.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.566+0000 7f6accc00640 1 -- 172.21.15.114:0/1143520396 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8004030 con 0x560849b5f390 2024-09-21T07:02:01.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.568+0000 7f6ae50a3740 1 -- 172.21.15.114:0/1143520396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 msgr2=0x560849f27820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.568+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f27820 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8011840 tx=0x7f6ac800a7c0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 -- 172.21.15.114:0/1143520396 shutdown_connections 2024-09-21T07:02:01.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f27820 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/1143520396 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 -- 172.21.15.114:0/1143520396 >> 172.21.15.114:0/1143520396 conn(0x560849b74460 msgr2=0x560849f24c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 -- 172.21.15.114:0/1143520396 shutdown_connections 2024-09-21T07:02:01.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.569+0000 7f6ae50a3740 1 -- 172.21.15.114:0/1143520396 wait complete. 2024-09-21T07:02:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.570+0000 7f6ae50a3740 1 -- start start 2024-09-21T07:02:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.570+0000 7f6ae50a3740 1 Processor -- start 2024-09-21T07:02:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.571+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f325d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.571+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 0x560849f32b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.571+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b80070 0x560849f3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.571+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560849cee700 con 0x560849b5f390 2024-09-21T07:02:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.571+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560849e28580 con 0x560849b571b0 2024-09-21T07:02:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.571+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560849f35580 con 0x560849b80070 2024-09-21T07:02:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.572+0000 7f6ae50a3740 10 client.0 _ll_register_callbacks cb 0x560849e38470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.572+0000 7f6ad7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 0x560849f32b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.572+0000 7f6ad7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 0x560849f32b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48462/0 (socket says 172.21.15.114:48462) 2024-09-21T07:02:01.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.572+0000 7f6ad7400640 1 -- 172.21.15.114:0/3650977253 learned_addr learned my addr 172.21.15.114:0/3650977253 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:01.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.572+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b80070 0x560849f3e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ad7400640 1 -- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b80070 msgr2=0x560849f3e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ad7400640 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b80070 0x560849f3e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ad7400640 1 -- 172.21.15.114:0/3650977253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 msgr2=0x560849f325d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ad7400640 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f325d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ad7400640 1 -- 172.21.15.114:0/3650977253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560849beb650 con 0x560849b5f390 2024-09-21T07:02:01.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ad7400640 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 0x560849f32b10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4000c00 tx=0x7f6ac4032920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90c04bc03e84cef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:01.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.573+0000 7f6ac1600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:01.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.576+0000 7f6ac1600640 1 -- 172.21.15.114:0/3650977253 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4035040 con 0x560849b5f390 2024-09-21T07:02:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.576+0000 7f6ac1600640 1 -- 172.21.15.114:0/3650977253 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f6ac4004030 con 0x560849b5f390 2024-09-21T07:02:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.576+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560849e2ab90 con 0x560849b5f390 2024-09-21T07:02:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.576+0000 7f6ac1600640 1 -- 172.21.15.114:0/3650977253 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6ac404c050 con 0x560849b5f390 2024-09-21T07:02:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.576+0000 7f6ae50a3740 10 client.17142 fetch_fsmap learned FSMap version 46 2024-09-21T07:02:01.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.576+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560849e45950 con 0x560849b5f390 2024-09-21T07:02:01.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.577+0000 7f6ac1600640 1 -- 172.21.15.114:0/3650977253 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560849e2ab90 con 0x560849b5f390 2024-09-21T07:02:01.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ac1600640 1 -- 172.21.15.114:0/3650977253 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f6ac4004850 con 0x560849b5f390 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ae50a3740 10 client.17142 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ae50a3740 10 client.17142 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ae50a3740 20 client.17142 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b3e9f0 0x560849f44780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ae50a3740 4 client.17142 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.578+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560849e9ff00 con 0x560849b3e9f0 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.579+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b3e9f0 0x560849f44780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.579+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b3e9f0 0x560849f44780 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e0af97cecd8ce41 server_cookie=3e56e7637d879081 in_seq=0 out_seq=0 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.579+0000 7f6ac1600640 10 client.17142 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.580+0000 7f6ac1600640 1 -- 172.21.15.114:0/3650977253 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560849e9ff00 con 0x560849b3e9f0 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.580+0000 7f6ac1600640 10 client.17142 handle_command_reply: tid=0 2024-09-21T07:02:01.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.581+0000 7f6ae50a3740 1 client.17142 shutdown 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 msgr2=0x560849f32b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 0x560849f32b10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4000c00 tx=0x7f6ac4032920 comp rx=0 tx=0).stop 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 shutdown_connections 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b3e9f0 0x560849f44780 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b80070 0x560849f3e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b5f390 0x560849f32b10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/3650977253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f325d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ac1600640 0 client.17142 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:02:01.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.582+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 >> 172.21.15.114:0/3650977253 conn(0x560849b74460 msgr2=0x560849f29f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:01.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.583+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 shutdown_connections 2024-09-21T07:02:01.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.583+0000 7f6ae50a3740 1 -- 172.21.15.114:0/3650977253 wait complete. 2024-09-21T07:02:01.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.583+0000 7f6ae50a3740 20 client.17142 trim_cache size 0 max 16384 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.618+0000 7f6ae50a3740 1 Processor -- start 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.618+0000 7f6ae50a3740 1 -- start start 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.619+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f34200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.619+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f2f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.619+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.619+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81c8620 con 0x560849b80070 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.619+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81ca1f0 con 0x560849b571b0 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.619+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81c8cf0 con 0x560849b5f390 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.620+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f34200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.620+0000 7f6ad7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f2f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.620+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f34200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57292/0 (socket says 172.21.15.114:57292) 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.620+0000 7f6ad6a00640 1 -- 172.21.15.114:0/2723318075 learned_addr learned my addr 172.21.15.114:0/2723318075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:01.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.620+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.621+0000 7f6ad7400640 1 -- 172.21.15.114:0/2723318075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 msgr2=0x560849f34200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.621+0000 7f6ad7400640 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f34200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.621+0000 7f6ad7400640 1 -- 172.21.15.114:0/2723318075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 msgr2=0x560849f2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.621+0000 7f6ad7400640 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.621+0000 7f6ad7400640 1 -- 172.21.15.114:0/2723318075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560849f25a80 con 0x560849b5f390 2024-09-21T07:02:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.622+0000 7f6ad7400640 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f2f7c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004c70 tx=0x7f6ac4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef2e4849d294091 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.623+0000 7f6ac3e00640 1 -- 172.21.15.114:0/2723318075 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4054070 con 0x560849b5f390 2024-09-21T07:02:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.623+0000 7f6ac3e00640 1 -- 172.21.15.114:0/2723318075 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac4049040 con 0x560849b5f390 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.624+0000 7f6ae50a3740 1 -- 172.21.15.114:0/2723318075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 msgr2=0x560849f2f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.624+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f2f7c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004c70 tx=0x7f6ac4005fd0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 -- 172.21.15.114:0/2723318075 shutdown_connections 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f2f7c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/2723318075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f34200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 -- 172.21.15.114:0/2723318075 >> 172.21.15.114:0/2723318075 conn(0x560849b74460 msgr2=0x560849f33af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 -- 172.21.15.114:0/2723318075 shutdown_connections 2024-09-21T07:02:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.625+0000 7f6ae50a3740 1 -- 172.21.15.114:0/2723318075 wait complete. 2024-09-21T07:02:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.626+0000 7f6ae50a3740 1 -- start start 2024-09-21T07:02:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.626+0000 7f6ae50a3740 1 Processor -- start 2024-09-21T07:02:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f36830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f36d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ae50a3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f36830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81c9b20 con 0x560849b80070 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ae50a3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81c93c0 con 0x560849b571b0 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ad6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f36830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57302/0 (socket says 172.21.15.114:57302) 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad81c6620 con 0x560849b5f390 2024-09-21T07:02:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.630+0000 7f6ad6a00640 1 -- 172.21.15.114:0/452464943 learned_addr learned my addr 172.21.15.114:0/452464943 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ad7400640 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f36d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ae50a3740 10 client.0 _ll_register_callbacks cb 0x7f6ac4032620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ad7400640 1 -- 172.21.15.114:0/452464943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 msgr2=0x560849f36830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ad7400640 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f36830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ad7400640 1 -- 172.21.15.114:0/452464943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 msgr2=0x560849f372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ad7400640 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.631+0000 7f6ad7400640 1 -- 172.21.15.114:0/452464943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560849fae1c0 con 0x560849b5f390 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.632+0000 7f6ad7400640 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f36d70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004cf0 tx=0x7f6ac40056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e8d97f469e8a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.632+0000 7f6ac1600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.633+0000 7f6ac1600640 1 -- 172.21.15.114:0/452464943 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4035050 con 0x560849b5f390 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.633+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ac4049040 con 0x560849b5f390 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.633+0000 7f6ac1600640 1 -- 172.21.15.114:0/452464943 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f6ac4004950 con 0x560849b5f390 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.633+0000 7f6ac1600640 1 -- 172.21.15.114:0/452464943 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6ac4068050 con 0x560849b5f390 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.633+0000 7f6ae50a3740 10 client.27472 fetch_fsmap learned FSMap version 46 2024-09-21T07:02:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.633+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x560849faf420 con 0x560849b5f390 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ac1600640 1 -- 172.21.15.114:0/452464943 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac4049040 con 0x560849b5f390 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ac1600640 1 -- 172.21.15.114:0/452464943 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f6ac4003830 con 0x560849b5f390 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ae50a3740 10 client.27472 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ae50a3740 10 client.27472 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ae50a3740 20 client.27472 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b13da0 0x560849fb1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ae50a3740 4 client.27472 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.635+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560849e9b9b0 con 0x560849b13da0 2024-09-21T07:02:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.636+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b13da0 0x560849fb1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ad6a00640 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b13da0 0x560849fb1af0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8474a5f37aa5e64c server_cookie=8313cc25053243b1 in_seq=0 out_seq=0 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ac1600640 10 client.27472 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ac1600640 1 -- 172.21.15.114:0/452464943 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560849e9b9b0 con 0x560849b13da0 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ac1600640 10 client.27472 handle_command_reply: tid=0 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ae50a3740 1 client.27472 shutdown 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 msgr2=0x560849f36d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.640+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f36d70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4004cf0 tx=0x7f6ac40056d0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 shutdown_connections 2024-09-21T07:02:01.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x560849b13da0 0x560849fb1af0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560849b80070 0x560849f372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560849b5f390 0x560849f36d70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 --2- 172.21.15.114:0/452464943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560849b571b0 0x560849f36830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 >> 172.21.15.114:0/452464943 conn(0x560849b74460 msgr2=0x560849f31810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ac1600640 0 client.27472 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 shutdown_connections 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 1 -- 172.21.15.114:0/452464943 wait complete. 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.642+0000 7f6ae50a3740 20 client.27472 trim_cache size 0 max 16384 2024-09-21T07:02:01.645 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:02:01.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6abc0b6780 msgr2=0x7f6abc0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6abc0b6780 0x7f6abc0b8c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6ad00a9e70 tx=0x7f6ad0022000 comp rx=0 tx=0).stop 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 msgr2=0x7f6ad8116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 0x7f6ad8116170 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000da10 tx=0x7f6ad000ded0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 shutdown_connections 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6ad811cc30 0x7f6ad8116170 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ad8111f40 0x7f6ad811c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6abc0b6780 0x7f6abc0b8c90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 --2- 172.21.15.114:0/3740473855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ad810a9e0 0x7f6ad811beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 >> 172.21.15.114:0/3740473855 conn(0x7f6ad8070ec0 msgr2=0x7f6ad810ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 shutdown_connections 2024-09-21T07:02:01.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:01.646+0000 7f6ade400640 1 -- 172.21.15.114:0/3740473855 wait complete. 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 -- 172.21.15.114:0/3551945414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30103bf0 msgr2=0x7f1c30104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3551945414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30103bf0 0x7f1c30104000 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009900 tx=0x7f1c2002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 -- 172.21.15.114:0/3551945414 shutdown_connections 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3551945414 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30109e80 0x7f1c301106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3551945414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 0x7f1c30109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3551945414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30103bf0 0x7f1c30104000 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 -- 172.21.15.114:0/3551945414 >> 172.21.15.114:0/3551945414 conn(0x7f1c3007b8e0 msgr2=0x7f1c3007bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:02.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 -- 172.21.15.114:0/3551945414 shutdown_connections 2024-09-21T07:02:02.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.723+0000 7f1c34e00640 1 -- 172.21.15.114:0/3551945414 wait complete. 2024-09-21T07:02:02.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.724+0000 7f1c34e00640 1 Processor -- start 2024-09-21T07:02:02.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.724+0000 7f1c34e00640 1 -- start start 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c34e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 0x7f1c30077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c34e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 0x7f1c30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c34e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30109e80 0x7f1c30075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c34e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c30113b40 con 0x7f1c30103bf0 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c34e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301139c0 con 0x7f1c30109530 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c34e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c30113cc0 con 0x7f1c30109e80 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 0x7f1c30077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 0x7f1c30077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48490/0 (socket says 172.21.15.114:48490) 2024-09-21T07:02:02.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c2de00640 1 -- 172.21.15.114:0/3646050826 learned_addr learned my addr 172.21.15.114:0/3646050826 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c2d400640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 0x7f1c30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.725+0000 7f1c2e800640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30109e80 0x7f1c30075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2d400640 1 -- 172.21.15.114:0/3646050826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30109e80 msgr2=0x7f1c30075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2d400640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30109e80 0x7f1c30075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2d400640 1 -- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 msgr2=0x7f1c30077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2d400640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 0x7f1c30077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2d400640 1 -- 172.21.15.114:0/3646050826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c30076470 con 0x7f1c30109530 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2de00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 0x7f1c30077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c2d400640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 0x7f1c30075710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1c3006cb80 tx=0x7f1c1c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c26a00640 1 -- 172.21.15.114:0/3646050826 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00cc40 con 0x7f1c30109530 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c26a00640 1 -- 172.21.15.114:0/3646050826 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1c00cde0 con 0x7f1c30109530 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c301ad070 con 0x7f1c30109530 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.726+0000 7f1c26a00640 1 -- 172.21.15.114:0/3646050826 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c004a00 con 0x7f1c30109530 2024-09-21T07:02:02.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.727+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c301ad560 con 0x7f1c30109530 2024-09-21T07:02:02.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.729+0000 7f1c3bad9740 1 Processor -- start 2024-09-21T07:02:02.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 -- start start 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8eb59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8eb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8eb6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5605f8e2ba90 con 0x5605f8b0e850 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605f8e1aec0 con 0x5605f8ae5990 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605f8e1ecd0 con 0x5605f8aedb70 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c26a00640 1 -- 172.21.15.114:0/3646050826 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1c1c004ba0 con 0x7f1c30109530 2024-09-21T07:02:02.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8eb59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8eb59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48506/0 (socket says 172.21.15.114:48506) 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.730+0000 7f1c2de00640 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8eb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2e800640 1 -- 172.21.15.114:0/3863980637 learned_addr learned my addr 172.21.15.114:0/3863980637 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2d400640 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8eb6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2de00640 1 -- 172.21.15.114:0/3863980637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 msgr2=0x5605f8eb6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2de00640 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8eb6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2de00640 1 -- 172.21.15.114:0/3863980637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 msgr2=0x5605f8eb59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2de00640 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8eb59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2de00640 1 -- 172.21.15.114:0/3863980637 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f8dc09d0 con 0x5605f8ae5990 2024-09-21T07:02:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2d400640 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8eb6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c2de00640 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8eb5ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004770 tx=0x7f1c20005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb8333f1f7853b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.731+0000 7f1c1b400640 1 -- 172.21.15.114:0/3863980637 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20035040 con 0x5605f8ae5990 2024-09-21T07:02:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c1b400640 1 -- 172.21.15.114:0/3863980637 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c20031820 con 0x5605f8ae5990 2024-09-21T07:02:02.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c1b400640 1 -- 172.21.15.114:0/3863980637 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20004a00 con 0x5605f8ae5990 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c26a00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c140b6680 0x7f1c140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c3bad9740 1 -- 172.21.15.114:0/3863980637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 msgr2=0x5605f8eb5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8eb5ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004770 tx=0x7f1c20005ed0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c26a00640 1 -- 172.21.15.114:0/3646050826 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f1c1c0e7a70 con 0x7f1c30109530 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.732+0000 7f1c2e800640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c140b6680 0x7f1c140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 -- 172.21.15.114:0/3863980637 shutdown_connections 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8eb6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8eb5ef0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/3863980637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8eb59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 -- 172.21.15.114:0/3863980637 >> 172.21.15.114:0/3863980637 conn(0x5605f8b02c40 msgr2=0x5605f8eb38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:02.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 -- 172.21.15.114:0/3863980637 shutdown_connections 2024-09-21T07:02:02.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 -- 172.21.15.114:0/3863980637 wait complete. 2024-09-21T07:02:02.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c2e800640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c140b6680 0x7f1c140b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1c28005fd0 tx=0x7f1c28009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:02.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.733+0000 7f1c3bad9740 1 -- start start 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 Processor -- start 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8ec14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ec1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5605f8c7c7b0 con 0x5605f8b0e850 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605f8dbf650 con 0x5605f8ae5990 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5605f8ec4130 con 0x5605f8aedb70 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.734+0000 7f1c3bad9740 10 client.0 _ll_register_callbacks cb 0x5605f8b260a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8ec14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8ec14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57330/0 (socket says 172.21.15.114:57330) 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 -- 172.21.15.114:0/4051265437 learned_addr learned my addr 172.21.15.114:0/4051265437 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:02.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2d400640 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ec1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 -- 172.21.15.114:0/4051265437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 msgr2=0x5605f8ec1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ec1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 -- 172.21.15.114:0/4051265437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 msgr2=0x5605f8ecd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 -- 172.21.15.114:0/4051265437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f8b88ba0 con 0x5605f8ae5990 2024-09-21T07:02:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c2de00640 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8ec14c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004270 tx=0x7f1c20031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a11a8851256f8cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.735+0000 7f1c0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c0fe00640 1 -- 172.21.15.114:0/4051265437 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20035040 con 0x5605f8ae5990 2024-09-21T07:02:02.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c20031820 con 0x5605f8ae5990 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c0fe00640 1 -- 172.21.15.114:0/4051265437 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f1c200035f0 con 0x5605f8ae5990 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c0fe00640 1 -- 172.21.15.114:0/4051265437 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1c20058050 con 0x5605f8ae5990 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c3bad9740 10 client.27548 fetch_fsmap learned FSMap version 46 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5605f8ed0000 con 0x5605f8ae5990 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.736+0000 7f1c0fe00640 1 -- 172.21.15.114:0/4051265437 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c20031820 con 0x5605f8ae5990 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c0fe00640 1 -- 172.21.15.114:0/4051265437 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f1c200467b0 con 0x5605f8ae5990 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c3bad9740 10 client.27548 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c3bad9740 10 client.27548 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c3bad9740 20 client.27548 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8acd1d0 0x5605f8ed3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c3bad9740 4 client.27548 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f8e2d9e0 con 0x5605f8acd1d0 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c2d400640 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8acd1d0 0x5605f8ed3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c0fe00640 10 client.27548 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:02.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.737+0000 7f1c2d400640 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8acd1d0 0x5605f8ed3360 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cd1e9077eecf45a server_cookie=465e7cf2b1418460 in_seq=0 out_seq=0 2024-09-21T07:02:02.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.738+0000 7f1c0fe00640 1 -- 172.21.15.114:0/4051265437 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5605f8e2d9e0 con 0x5605f8acd1d0 2024-09-21T07:02:02.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.738+0000 7f1c0fe00640 10 client.27548 handle_command_reply: tid=0 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.738+0000 7f1c3bad9740 1 client.27548 shutdown 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.738+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 msgr2=0x5605f8ec14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.738+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8ec14c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1c20004270 tx=0x7f1c20031400 comp rx=0 tx=0).stop 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 shutdown_connections 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8acd1d0 0x5605f8ed3360 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ec1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/4051265437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8ae5990 0x5605f8ec14c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c0fe00640 0 client.27548 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 >> 172.21.15.114:0/4051265437 conn(0x5605f8b02c40 msgr2=0x5605f8eb8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 shutdown_connections 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 1 -- 172.21.15.114:0/4051265437 wait complete. 2024-09-21T07:02:02.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.739+0000 7f1c3bad9740 20 client.27548 trim_cache size 0 max 16384 2024-09-21T07:02:02.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.775+0000 7f1c3bad9740 1 Processor -- start 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 -- start start 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8ae5990 0x5605f8ebca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 0x5605f8ec2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b7560 con 0x5605f8b0e850 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b9130 con 0x5605f8aedb70 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b7c30 con 0x5605f8ae5990 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.776+0000 7f1c2d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 0x5605f8ec2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 0x5605f8ec2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57344/0 (socket says 172.21.15.114:57344) 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 -- 172.21.15.114:0/1572606435 learned_addr learned my addr 172.21.15.114:0/1572606435 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:02.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2de00640 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8ae5990 0x5605f8ebca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 -- 172.21.15.114:0/1572606435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8ae5990 msgr2=0x5605f8ebca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8ae5990 0x5605f8ebca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 -- 172.21.15.114:0/1572606435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 msgr2=0x5605f8ecd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2e800640 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2d400640 1 -- 172.21.15.114:0/1572606435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f8eb8810 con 0x5605f8aedb70 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.777+0000 7f1c2de00640 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8ae5990 0x5605f8ebca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.778+0000 7f1c2d400640 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 0x5605f8ec2ef0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c004770 tx=0x7f1c1c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f37861aefec0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.778+0000 7f1c1aa00640 1 -- 172.21.15.114:0/1572606435 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c01c030 con 0x5605f8aedb70 2024-09-21T07:02:02.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.778+0000 7f1c1aa00640 1 -- 172.21.15.114:0/1572606435 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1c0ad000 con 0x5605f8aedb70 2024-09-21T07:02:02.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.778+0000 7f1c1aa00640 1 -- 172.21.15.114:0/1572606435 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0e5070 con 0x5605f8aedb70 2024-09-21T07:02:02.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 -- 172.21.15.114:0/1572606435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 msgr2=0x5605f8ec2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:02.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 0x5605f8ec2ef0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c004770 tx=0x7f1c1c009f60 comp rx=0 tx=0).stop 2024-09-21T07:02:02.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 -- 172.21.15.114:0/1572606435 shutdown_connections 2024-09-21T07:02:02.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8b0e850 0x5605f8ecd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8aedb70 0x5605f8ec2ef0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/1572606435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8ae5990 0x5605f8ebca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.779+0000 7f1c3bad9740 1 -- 172.21.15.114:0/1572606435 >> 172.21.15.114:0/1572606435 conn(0x5605f8b02c40 msgr2=0x5605f8ec27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:02.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.780+0000 7f1c3bad9740 1 -- 172.21.15.114:0/1572606435 shutdown_connections 2024-09-21T07:02:02.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.780+0000 7f1c3bad9740 1 -- 172.21.15.114:0/1572606435 wait complete. 2024-09-21T07:02:02.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.780+0000 7f1c3bad9740 1 -- start start 2024-09-21T07:02:02.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.781+0000 7f1c3bad9740 1 Processor -- start 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8ae5990 0x5605f8ed26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ed2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 0x5605f8f4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b8a60 con 0x5605f8ae5990 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b8300 con 0x5605f8b0e850 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c301b5560 con 0x5605f8aedb70 2024-09-21T07:02:02.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.783+0000 7f1c3bad9740 10 client.0 _ll_register_callbacks cb 0x7f1c20040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 0x5605f8f4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 0x5605f8f4dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:57352/0 (socket says 172.21.15.114:57352) 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 -- 172.21.15.114:0/389313861 learned_addr learned my addr 172.21.15.114:0/389313861 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2d400640 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ed2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 -- 172.21.15.114:0/389313861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 msgr2=0x5605f8ed2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ed2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 -- 172.21.15.114:0/389313861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8ae5990 msgr2=0x5605f8ed26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:02.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8ae5990 0x5605f8ed26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:02.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.784+0000 7f1c2e800640 1 -- 172.21.15.114:0/389313861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f8f4e020 con 0x5605f8b0e850 2024-09-21T07:02:02.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.785+0000 7f1c2e800640 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 0x5605f8f4dae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c28004170 tx=0x7f1c28004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ded6bef24dccbeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:02.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.785+0000 7f1c0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:02.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.785+0000 7f1c0fe00640 1 -- 172.21.15.114:0/389313861 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2802d060 con 0x5605f8b0e850 2024-09-21T07:02:02.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.786+0000 7f1c0fe00640 1 -- 172.21.15.114:0/389313861 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f1c2800f970 con 0x5605f8b0e850 2024-09-21T07:02:02.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.786+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c0ad000 con 0x5605f8b0e850 2024-09-21T07:02:02.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.787+0000 7f1c0fe00640 1 -- 172.21.15.114:0/389313861 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1c2800fb70 con 0x5605f8b0e850 2024-09-21T07:02:02.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.787+0000 7f1c3bad9740 10 client.27554 fetch_fsmap learned FSMap version 46 2024-09-21T07:02:02.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.787+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5605f8f4f130 con 0x5605f8b0e850 2024-09-21T07:02:02.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.787+0000 7f1c0fe00640 1 -- 172.21.15.114:0/389313861 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c1c0ad000 con 0x5605f8b0e850 2024-09-21T07:02:02.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.788+0000 7f1c0fe00640 1 -- 172.21.15.114:0/389313861 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 46) ==== 8382+0+0 (secure 0 0 0) 0x7f1c28027070 con 0x5605f8b0e850 2024-09-21T07:02:02.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.788+0000 7f1c3bad9740 10 client.27554 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:02:02.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.790+0000 7f1c3bad9740 10 client.27554 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:02.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.790+0000 7f1c3bad9740 20 client.27554 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:02.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.790+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8aa2580 0x5605f8f51970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:02.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.790+0000 7f1c3bad9740 4 client.27554 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T07:02:02.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.790+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x5605f8e29490 con 0x5605f8aa2580 2024-09-21T07:02:02.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.790+0000 7f1c2de00640 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8aa2580 0x5605f8f51970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:02.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.791+0000 7f1c0fe00640 10 client.27554 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:02.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:02.792+0000 7f1c2de00640 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8aa2580 0x5605f8f51970 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161752d552218a87 server_cookie=d9ff890e470379cb in_seq=0 out_seq=0 2024-09-21T07:02:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 445.0 s, 502.5 tps, lat 176.975 ms stddev 265.685 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.128+0000 7f1c0fe00640 1 -- 172.21.15.114:0/389313861 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5605f8e29490 con 0x5605f8aa2580 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.128+0000 7f1c0fe00640 10 client.27554 handle_command_reply: tid=0 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.128+0000 7f1c3bad9740 1 client.27554 shutdown 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 msgr2=0x5605f8f4dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 0x5605f8f4dae0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1c28004170 tx=0x7f1c28004ab0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 shutdown_connections 2024-09-21T07:02:03.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f8b0e850 0x5605f8f4dae0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5605f8aa2580 0x5605f8f51970 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5605f8aedb70 0x5605f8ed2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 --2- 172.21.15.114:0/389313861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5605f8ae5990 0x5605f8ed26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 >> 172.21.15.114:0/389313861 conn(0x5605f8b02c40 msgr2=0x5605f8eb69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c0fe00640 0 client.27554 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 shutdown_connections 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 1 -- 172.21.15.114:0/389313861 wait complete. 2024-09-21T07:02:03.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.129+0000 7f1c3bad9740 20 client.27554 trim_cache size 0 max 16384 2024-09-21T07:02:03.897 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:02 smithi193 ceph-mon[34601]: pgmap v635: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.5 MiB/s wr, 678 op/s 2024-09-21T07:02:03.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:02 smithi183 ceph-mon[34219]: pgmap v635: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.5 MiB/s wr, 678 op/s 2024-09-21T07:02:03.897 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:03.897 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 47, 2024-09-21T07:02:03.897 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:02:03.897 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 30, 2024-09-21T07:02:03.897 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:02:03.897 INFO:teuthology.orchestra.run.smithi114.stdout: "c14c5ce": { 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 30, 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 141.1, 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:03.898 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:03.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:03.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:03.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:03.899 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:03.899 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:03.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c140b6680 msgr2=0x7f1c140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c140b6680 0x7f1c140b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1c28005fd0 tx=0x7f1c28009500 comp rx=0 tx=0).stop 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 msgr2=0x7f1c30075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 0x7f1c30075710 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1c3006cb80 tx=0x7f1c1c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 shutdown_connections 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c30109e80 0x7f1c30075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1c140b6680 0x7f1c140b8b90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c30109530 0x7f1c30075710 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 --2- 172.21.15.114:0/3646050826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c30103bf0 0x7f1c30077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.130+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 >> 172.21.15.114:0/3646050826 conn(0x7f1c3007b8e0 msgr2=0x7f1c30107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.131+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 shutdown_connections 2024-09-21T07:02:03.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:03.131+0000 7f1c34e00640 1 -- 172.21.15.114:0/3646050826 wait complete. 2024-09-21T07:02:03.901 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:02 smithi114 ceph-mon[30654]: pgmap v635: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 9.5 MiB/s wr, 678 op/s 2024-09-21T07:02:04.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:04 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:02:04.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:04 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:04.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:04 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:02:04.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:04 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:04 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:02:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:04 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:02:04.628 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:02:05.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:05 smithi114 ceph-mon[30654]: pgmap v636: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.9 MiB/s wr, 568 op/s 2024-09-21T07:02:05.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:05 smithi183 ceph-mon[34219]: pgmap v636: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.9 MiB/s wr, 568 op/s 2024-09-21T07:02:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:05 smithi193 ceph-mon[34601]: pgmap v636: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.9 MiB/s wr, 568 op/s 2024-09-21T07:02:05.558 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c14c5ce', quiesce took 10.6 seconds. Will release after: 10.450000000000001 2024-09-21T07:02:05.678 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:06.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:06.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:06.158 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:06 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 -- 172.21.15.114:0/165382673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 msgr2=0x7fbed01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 --2- 172.21.15.114:0/165382673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed01098d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8009900 tx=0x7fbeb802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 -- 172.21.15.114:0/165382673 shutdown_connections 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 --2- 172.21.15.114:0/165382673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbed0109e10 0x7fbed0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 --2- 172.21.15.114:0/165382673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed01098d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 --2- 172.21.15.114:0/165382673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 0x7fbed0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 -- 172.21.15.114:0/165382673 >> 172.21.15.114:0/165382673 conn(0x7fbed00ff5d0 msgr2=0x7fbed0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:06.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 -- 172.21.15.114:0/165382673 shutdown_connections 2024-09-21T07:02:06.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.340+0000 7fbed6e00640 1 -- 172.21.15.114:0/165382673 wait complete. 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.341+0000 7fbed6e00640 1 Processor -- start 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.341+0000 7fbed6e00640 1 -- start start 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed6e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 0x7fbed019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed6e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed6e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbed0109e10 0x7fbed019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed6e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113ad0 con 0x7fbed01094c0 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed6e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113950 con 0x7fbed0109e10 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed6e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113c50 con 0x7fbed0103b80 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 0x7fbed019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbec7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 0x7fbed019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50784/0 (socket says 172.21.15.114:50784) 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbecfe00640 1 -- 172.21.15.114:0/2495452244 learned_addr learned my addr 172.21.15.114:0/2495452244 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:06.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbed0109e10 0x7fbed019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbec7400640 1 -- 172.21.15.114:0/2495452244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 msgr2=0x7fbed019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbec7400640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 0x7fbed019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbec7400640 1 -- 172.21.15.114:0/2495452244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbed0109e10 msgr2=0x7fbed019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbec7400640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbed0109e10 0x7fbed019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.342+0000 7fbec7400640 1 -- 172.21.15.114:0/2495452244 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed01a03f0 con 0x7fbed01094c0 2024-09-21T07:02:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.343+0000 7fbec7400640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed019aa20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbeb802a910 tx=0x7fbeb8004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.343+0000 7fbecd600640 1 -- 172.21.15.114:0/2495452244 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb803f070 con 0x7fbed01094c0 2024-09-21T07:02:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.343+0000 7fbecd600640 1 -- 172.21.15.114:0/2495452244 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb8003ca0 con 0x7fbed01094c0 2024-09-21T07:02:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.343+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed01a0680 con 0x7fbed01094c0 2024-09-21T07:02:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.343+0000 7fbecd600640 1 -- 172.21.15.114:0/2495452244 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8035070 con 0x7fbed01094c0 2024-09-21T07:02:06.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.343+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a6e40 con 0x7fbed01094c0 2024-09-21T07:02:06.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.346+0000 7fbeddad9740 1 Processor -- start 2024-09-21T07:02:06.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 -- start start 2024-09-21T07:02:06.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbecd600640 1 -- 172.21.15.114:0/2495452244 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fbeb803b460 con 0x7fbed01094c0 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748d9bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55748d933fb0 con 0x55748d5f5390 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55748d9233e0 con 0x55748d5ed1b0 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbeddad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55748d9271f0 con 0x55748d616070 2024-09-21T07:02:06.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748d9bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbec7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748d9bcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50786/0 (socket says 172.21.15.114:50786) 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.347+0000 7fbed4e00640 1 -- 172.21.15.114:0/2677833213 learned_addr learned my addr 172.21.15.114:0/2677833213 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbecfe00640 1 -- 172.21.15.114:0/2677833213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 msgr2=0x55748d9bcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbecfe00640 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748d9bcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbecfe00640 1 -- 172.21.15.114:0/2677833213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 msgr2=0x55748d9bd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbecfe00640 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9bd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbecfe00640 1 -- 172.21.15.114:0/2677833213 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748d8edcd0 con 0x55748d5ed1b0 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748d9bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:06.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbecfe00640 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9bd2e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbec00104d0 tx=0x7fbec0010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a1cf5cf9e4eadc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:06.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.348+0000 7fbec5600640 1 -- 172.21.15.114:0/2677833213 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003b70 con 0x55748d5ed1b0 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.349+0000 7fbec5600640 1 -- 172.21.15.114:0/2677833213 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec0003d10 con 0x55748d5ed1b0 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.349+0000 7fbec5600640 1 -- 172.21.15.114:0/2677833213 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0011a60 con 0x55748d5ed1b0 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.349+0000 7fbecd600640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbeb40b6700 0x7fbeb40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.349+0000 7fbecd600640 1 -- 172.21.15.114:0/2495452244 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7fbeb8107f90 con 0x7fbed01094c0 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.349+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbeb40b6700 0x7fbeb40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.349+0000 7fbeddad9740 1 -- 172.21.15.114:0/2677833213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 msgr2=0x55748d9bd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9bd2e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbec00104d0 tx=0x7fbec0010990 comp rx=0 tx=0).stop 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 -- 172.21.15.114:0/2677833213 shutdown_connections 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9bd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9bd2e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2677833213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748d9bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 -- 172.21.15.114:0/2677833213 >> 172.21.15.114:0/2677833213 conn(0x55748d60a460 msgr2=0x55748d9bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:06.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbeb40b6700 0x7fbeb40b8c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbec80124a0 tx=0x7fbec8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:06.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 -- 172.21.15.114:0/2677833213 shutdown_connections 2024-09-21T07:02:06.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.350+0000 7fbeddad9740 1 -- 172.21.15.114:0/2677833213 wait complete. 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 -- start start 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 Processor -- start 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 0x55748d9cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9d8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55748d784700 con 0x55748d616070 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55748d8be580 con 0x55748d5ed1b0 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55748d9cf8b0 con 0x55748d5f5390 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.351+0000 7fbeddad9740 10 client.0 _ll_register_callbacks cb 0x55748d8ce470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 0x55748d9cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9d8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 0x55748d9cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50794/0 (socket says 172.21.15.114:50794) 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 -- 172.21.15.114:0/3063621818 learned_addr learned my addr 172.21.15.114:0/3063621818 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9d8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44482/0 (socket says 172.21.15.114:44482) 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbecfe00640 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 -- 172.21.15.114:0/3063621818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 msgr2=0x55748d9cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 -- 172.21.15.114:0/3063621818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 msgr2=0x55748d9d8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9d8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 -- 172.21.15.114:0/3063621818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748d681650 con 0x55748d5f5390 2024-09-21T07:02:06.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbec7400640 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 0x55748d9cce00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80cacd0 tx=0x7fbeb8105740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=240b7997ca8a5490 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:06.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.352+0000 7fbebe000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbebe000640 1 -- 172.21.15.114:0/3063621818 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8048040 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55748d8c0b90 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbebe000640 1 -- 172.21.15.114:0/3063621818 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7fbeb80037f0 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbebe000640 1 -- 172.21.15.114:0/3063621818 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbeb80ca0b0 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbeddad9740 10 client.27499 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55748d8db950 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbebe000640 1 -- 172.21.15.114:0/3063621818 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55748d8c0b90 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbebe000640 1 -- 172.21.15.114:0/3063621818 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fbeb80cece0 con 0x55748d5f5390 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbeddad9740 10 client.27499 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbeddad9740 10 client.27499 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.353+0000 7fbeddad9740 20 client.27499 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.354+0000 7fbeddad9740 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5d49f0 0x55748d9de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.354+0000 7fbeddad9740 4 client.27499 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:06.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.354+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55748d935f00 con 0x55748d5d49f0 2024-09-21T07:02:06.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.354+0000 7fbecfe00640 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5d49f0 0x55748d9de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.354+0000 7fbecfe00640 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5d49f0 0x55748d9de8e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b987dac226407bc server_cookie=8d1f88fe6e3a315e in_seq=0 out_seq=0 2024-09-21T07:02:06.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.354+0000 7fbebe000640 10 client.27499 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.355+0000 7fbebe000640 1 -- 172.21.15.114:0/3063621818 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55748d935f00 con 0x55748d5d49f0 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.355+0000 7fbebe000640 10 client.27499 handle_command_reply: tid=0 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.355+0000 7fbeddad9740 1 client.27499 shutdown 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 msgr2=0x55748d9cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 0x55748d9cce00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80cacd0 tx=0x7fbeb8105740 comp rx=0 tx=0).stop 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 shutdown_connections 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5d49f0 0x55748d9de8e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9d8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5f5390 0x55748d9cce00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 --2- 172.21.15.114:0/3063621818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 >> 172.21.15.114:0/3063621818 conn(0x55748d60a460 msgr2=0x55748d9bfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:06.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbebe000640 0 client.27499 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:02:06.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 shutdown_connections 2024-09-21T07:02:06.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 1 -- 172.21.15.114:0/3063621818 wait complete. 2024-09-21T07:02:06.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.356+0000 7fbeddad9740 20 client.27499 trim_cache size 0 max 16384 2024-09-21T07:02:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:06 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:02:06.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 Processor -- start 2024-09-21T07:02:06.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 -- start start 2024-09-21T07:02:06.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5ed1b0 0x55748d9c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5f5390 0x55748d9d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9ba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01aecd0 con 0x55748d616070 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01b0870 con 0x55748d5f5390 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbeddad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01af3a0 con 0x55748d5ed1b0 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.370+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9ba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5f5390 0x55748d9d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9ba5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44494/0 (socket says 172.21.15.114:44494) 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbed4e00640 1 -- 172.21.15.114:0/1683494670 learned_addr learned my addr 172.21.15.114:0/1683494670 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:06.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbecfe00640 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5ed1b0 0x55748d9c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 -- 172.21.15.114:0/1683494670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5ed1b0 msgr2=0x55748d9c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5ed1b0 0x55748d9c0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 -- 172.21.15.114:0/1683494670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 msgr2=0x55748d9ba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9ba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 -- 172.21.15.114:0/1683494670 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748d9b62e0 con 0x55748d5f5390 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec7400640 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5f5390 0x55748d9d8ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80cd9f0 tx=0x7fbeb80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=303ccfd30fedc0c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.371+0000 7fbec4c00640 1 -- 172.21.15.114:0/1683494670 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8048070 con 0x55748d5f5390 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbec4c00640 1 -- 172.21.15.114:0/1683494670 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb80cd570 con 0x55748d5f5390 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbec4c00640 1 -- 172.21.15.114:0/1683494670 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80c91d0 con 0x55748d5f5390 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 -- 172.21.15.114:0/1683494670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5f5390 msgr2=0x55748d9d8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5f5390 0x55748d9d8ae0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbeb80cd9f0 tx=0x7fbeb80057c0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 -- 172.21.15.114:0/1683494670 shutdown_connections 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d616070 0x55748d9ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5f5390 0x55748d9d8ae0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 --2- 172.21.15.114:0/1683494670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d5ed1b0 0x55748d9c0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 -- 172.21.15.114:0/1683494670 >> 172.21.15.114:0/1683494670 conn(0x55748d60a460 msgr2=0x55748d9bfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 -- 172.21.15.114:0/1683494670 shutdown_connections 2024-09-21T07:02:06.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.372+0000 7fbeddad9740 1 -- 172.21.15.114:0/1683494670 wait complete. 2024-09-21T07:02:06.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.373+0000 7fbeddad9740 1 -- start start 2024-09-21T07:02:06.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.373+0000 7fbeddad9740 1 Processor -- start 2024-09-21T07:02:06.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748da422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01b01a0 con 0x55748d5f5390 2024-09-21T07:02:06.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01afa70 con 0x55748d5ed1b0 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01accd0 con 0x55748d616070 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbeddad9740 10 client.0 _ll_register_callbacks cb 0x7fbeb80c9f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748da422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbec7400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbecfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748da422e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50804/0 (socket says 172.21.15.114:50804) 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 -- 172.21.15.114:0/2110800155 learned_addr learned my addr 172.21.15.114:0/2110800155 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 -- 172.21.15.114:0/2110800155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 msgr2=0x55748d9d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 -- 172.21.15.114:0/2110800155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 msgr2=0x55748d9d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbed4e00640 1 -- 172.21.15.114:0/2110800155 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748da42950 con 0x55748d616070 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.374+0000 7fbec7400640 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:06.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.375+0000 7fbed4e00640 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748da422e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009e10 tx=0x7fbec80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7127f114261267a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.375+0000 7fbebe000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.375+0000 7fbebe000640 1 -- 172.21.15.114:0/2110800155 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec802f060 con 0x55748d616070 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.375+0000 7fbebe000640 1 -- 172.21.15.114:0/2110800155 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7fbec801f700 con 0x55748d616070 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.375+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeb80cd570 con 0x55748d616070 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbebe000640 1 -- 172.21.15.114:0/2110800155 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbec8036050 con 0x55748d616070 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 10 client.27505 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:06.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55748da49cd0 con 0x55748d616070 2024-09-21T07:02:06.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbebe000640 1 -- 172.21.15.114:0/2110800155 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb80cd570 con 0x55748d616070 2024-09-21T07:02:06.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbebe000640 1 -- 172.21.15.114:0/2110800155 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fbec801eb30 con 0x55748d616070 2024-09-21T07:02:06.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 10 client.27505 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:06.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 10 client.27505 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 20 client.27505 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5a9da0 0x55748da4c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 4 client.27505 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.376+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55748d9319b0 con 0x55748d5a9da0 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbecfe00640 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5a9da0 0x55748da4c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbecfe00640 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5a9da0 0x55748da4c4c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67bf68ff7010cb05 server_cookie=d04850ce36442bae in_seq=0 out_seq=0 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbebe000640 10 client.27505 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbebe000640 1 -- 172.21.15.114:0/2110800155 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55748d9319b0 con 0x55748d5a9da0 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbebe000640 10 client.27505 handle_command_reply: tid=0 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbeddad9740 1 client.27505 shutdown 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 msgr2=0x55748da422e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.377+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748da422e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009e10 tx=0x7fbec80046c0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 shutdown_connections 2024-09-21T07:02:06.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55748d5a9da0 0x55748da4c4c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55748d616070 0x55748da422e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55748d5f5390 0x55748d9d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 --2- 172.21.15.114:0/2110800155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55748d5ed1b0 0x55748d9d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 >> 172.21.15.114:0/2110800155 conn(0x55748d60a460 msgr2=0x55748d9cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbebe000640 0 client.27505 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 shutdown_connections 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 1 -- 172.21.15.114:0/2110800155 wait complete. 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.378+0000 7fbeddad9740 20 client.27505 trim_cache size 0 max 16384 2024-09-21T07:02:06.382 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbeb40b6700 msgr2=0x7fbeb40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbeb40b6700 0x7fbeb40b8c10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbec80124a0 tx=0x7fbec8005f90 comp rx=0 tx=0).stop 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 msgr2=0x7fbed019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed019aa20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbeb802a910 tx=0x7fbeb8004060 comp rx=0 tx=0).stop 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 shutdown_connections 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbeb40b6700 0x7fbeb40b8c10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbed0109e10 0x7fbed019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbed01094c0 0x7fbed019aa20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 --2- 172.21.15.114:0/2495452244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbed0103b80 0x7fbed019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:06.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 >> 172.21.15.114:0/2495452244 conn(0x7fbed00ff5d0 msgr2=0x7fbed01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:06.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 shutdown_connections 2024-09-21T07:02:06.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:06.379+0000 7fbed6e00640 1 -- 172.21.15.114:0/2495452244 wait complete. 2024-09-21T07:02:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:02:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:02:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:06 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:02:07.168 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:02:07.545 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:07.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:07 smithi114 ceph-mon[30654]: pgmap v637: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.8 MiB/s wr, 460 op/s 2024-09-21T07:02:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:07 smithi183 ceph-mon[34219]: pgmap v637: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.8 MiB/s wr, 460 op/s 2024-09-21T07:02:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:07 smithi193 ceph-mon[34601]: pgmap v637: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.8 MiB/s wr, 460 op/s 2024-09-21T07:02:07.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 450.0 s, 2.0 tps, lat 687.546 ms stddev 263.821 2024-09-21T07:02:08.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 -- 172.21.15.114:0/1106767912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0109df0 msgr2=0x7f87b0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 --2- 172.21.15.114:0/1106767912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0109df0 0x7f87b0110620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009ca0 tx=0x7f87a802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 -- 172.21.15.114:0/1106767912 shutdown_connections 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 --2- 172.21.15.114:0/1106767912 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0109df0 0x7f87b0110620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 --2- 172.21.15.114:0/1106767912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b01094a0 0x7f87b01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 --2- 172.21.15.114:0/1106767912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0103b60 0x7f87b0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 -- 172.21.15.114:0/1106767912 >> 172.21.15.114:0/1106767912 conn(0x7f87b007b850 msgr2=0x7f87b007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 -- 172.21.15.114:0/1106767912 shutdown_connections 2024-09-21T07:02:08.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.434+0000 7f87b5400640 1 -- 172.21.15.114:0/1106767912 wait complete. 2024-09-21T07:02:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.435+0000 7f87b5400640 1 Processor -- start 2024-09-21T07:02:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.435+0000 7f87b5400640 1 -- start start 2024-09-21T07:02:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87b5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 0x7f87b019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87b5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b01094a0 0x7f87b019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87b5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0109df0 0x7f87b019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87b5400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87b0113ab0 con 0x7f87b0103b60 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87b5400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87b0113930 con 0x7f87b0109df0 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87b5400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87b0113c30 con 0x7f87b01094a0 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 0x7f87b019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 0x7f87b019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44520/0 (socket says 172.21.15.114:44520) 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 -- 172.21.15.114:0/718060776 learned_addr learned my addr 172.21.15.114:0/718060776 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87af200640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0109df0 0x7f87b019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ade00640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b01094a0 0x7f87b019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 -- 172.21.15.114:0/718060776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b01094a0 msgr2=0x7f87b019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b01094a0 0x7f87b019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 -- 172.21.15.114:0/718060776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0109df0 msgr2=0x7f87b019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0109df0 0x7f87b019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87ae800640 1 -- 172.21.15.114:0/718060776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b01a03b0 con 0x7f87b0103b60 2024-09-21T07:02:08.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.436+0000 7f87af200640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0109df0 0x7f87b019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.437+0000 7f87ae800640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 0x7f87b019a440 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f87a000e9f0 tx=0x7f87a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.437+0000 7f87a7400640 1 -- 172.21.15.114:0/718060776 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a000cd80 con 0x7f87b0103b60 2024-09-21T07:02:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.437+0000 7f87a7400640 1 -- 172.21.15.114:0/718060776 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a00108e0 con 0x7f87b0103b60 2024-09-21T07:02:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.437+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b01a06a0 con 0x7f87b0103b60 2024-09-21T07:02:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.437+0000 7f87a7400640 1 -- 172.21.15.114:0/718060776 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0018980 con 0x7f87b0103b60 2024-09-21T07:02:08.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.437+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b01a6e00 con 0x7f87b0103b60 2024-09-21T07:02:08.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.441+0000 7f87bc1bd740 1 Processor -- start 2024-09-21T07:02:08.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.441+0000 7f87bc1bd740 1 -- start start 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac782da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac7832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac783820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645ac6f9fb0 con 0x5645ac3b31b0 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645ac6e93e0 con 0x5645ac3bb390 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5645ac6ed1f0 con 0x5645ac3dc070 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87af200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac782da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87a7400640 1 -- 172.21.15.114:0/718060776 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f87a0010430 con 0x7f87b0103b60 2024-09-21T07:02:08.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87af200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac782da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50828/0 (socket says 172.21.15.114:50828) 2024-09-21T07:02:08.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.442+0000 7f87af200640 1 -- 172.21.15.114:0/603928386 learned_addr learned my addr 172.21.15.114:0/603928386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:08.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87ade00640 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac783820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87ae800640 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac7832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87af200640 1 -- 172.21.15.114:0/603928386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 msgr2=0x5645ac783820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87af200640 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac783820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87af200640 1 -- 172.21.15.114:0/603928386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 msgr2=0x5645ac7832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87af200640 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac7832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87af200640 1 -- 172.21.15.114:0/603928386 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ac6b3cd0 con 0x5645ac3dc070 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87ae800640 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac7832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f87af200640 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac782da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f87a8033040 tx=0x7f87a802f310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c30b87e747e3d654 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.443+0000 7f879b400640 1 -- 172.21.15.114:0/603928386 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8031650 con 0x5645ac3dc070 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.444+0000 7f879b400640 1 -- 172.21.15.114:0/603928386 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8036040 con 0x5645ac3dc070 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.444+0000 7f879b400640 1 -- 172.21.15.114:0/603928386 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a80044c0 con 0x5645ac3dc070 2024-09-21T07:02:08.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.444+0000 7f87a7400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87940b6680 0x7f87940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.444+0000 7f87bc1bd740 1 -- 172.21.15.114:0/603928386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 msgr2=0x5645ac782da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.444+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac782da0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f87a8033040 tx=0x7f87a802f310 comp rx=0 tx=0).stop 2024-09-21T07:02:08.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.444+0000 7f87a7400640 1 -- 172.21.15.114:0/718060776 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f87a00e7e80 con 0x7f87b0103b60 2024-09-21T07:02:08.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87ae800640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87940b6680 0x7f87940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 -- 172.21.15.114:0/603928386 shutdown_connections 2024-09-21T07:02:08.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac783820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac7832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/603928386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac782da0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 -- 172.21.15.114:0/603928386 >> 172.21.15.114:0/603928386 conn(0x5645ac3d0460 msgr2=0x5645ac780c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 -- 172.21.15.114:0/603928386 shutdown_connections 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 -- 172.21.15.114:0/603928386 wait complete. 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87ae800640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87940b6680 0x7f87940b8b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5645ac781a20 tx=0x7f87a0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.445+0000 7f87bc1bd740 1 -- start start 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 Processor -- start 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac78e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac78eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac79a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5645ac54a700 con 0x5645ac3dc070 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645ac684580 con 0x5645ac3b31b0 2024-09-21T07:02:08.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5645ac7913a0 con 0x5645ac3bb390 2024-09-21T07:02:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.446+0000 7f87bc1bd740 10 client.0 _ll_register_callbacks cb 0x5645ac694470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac78e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac78eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac78e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45476/0 (socket says 172.21.15.114:45476) 2024-09-21T07:02:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 -- 172.21.15.114:0/1902259265 learned_addr learned my addr 172.21.15.114:0/1902259265 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:08.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac79a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 -- 172.21.15.114:0/1902259265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 msgr2=0x5645ac78eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac78eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 -- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 msgr2=0x5645ac79a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac79a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 -- 172.21.15.114:0/1902259265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ac447650 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87ade00640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac78eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f87af200640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac78e5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f87a802ade0 tx=0x7f87a80040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a740e5d80e62b130 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.447+0000 7f878fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f878fe00640 1 -- 172.21.15.114:0/1902259265 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a804b040 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5645ac686b90 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f878fe00640 1 -- 172.21.15.114:0/1902259265 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f87a8004460 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f878fe00640 1 -- 172.21.15.114:0/1902259265 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f87a8004660 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f87bc1bd740 10 client.27584 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5645ac6a1950 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f878fe00640 1 -- 172.21.15.114:0/1902259265 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5645ac686b90 con 0x5645ac3b31b0 2024-09-21T07:02:08.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.448+0000 7f878fe00640 1 -- 172.21.15.114:0/1902259265 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f87a8051040 con 0x5645ac3b31b0 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87bc1bd740 10 client.27584 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87bc1bd740 10 client.27584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87bc1bd740 20 client.27584 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac39a9f0 0x5645ac7a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87bc1bd740 4 client.27584 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ac6fbf00 con 0x5645ac39a9f0 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.449+0000 7f87ade00640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac39a9f0 0x5645ac7a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.450+0000 7f878fe00640 10 client.27584 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:02:08.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.450+0000 7f87ade00640 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac39a9f0 0x5645ac7a0300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c1e51e4a4ac39b1 server_cookie=7f5dc1374afb0b6f in_seq=0 out_seq=0 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.450+0000 7f878fe00640 1 -- 172.21.15.114:0/1902259265 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5645ac6fbf00 con 0x5645ac39a9f0 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.450+0000 7f878fe00640 10 client.27584 handle_command_reply: tid=0 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.450+0000 7f87bc1bd740 1 client.27584 shutdown 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 msgr2=0x5645ac78e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac78e5c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f87a802ade0 tx=0x7f87a80040e0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 shutdown_connections 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac39a9f0 0x5645ac7a0300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac79a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac78eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f878fe00640 0 client.27584 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/1902259265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac78e5c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 >> 172.21.15.114:0/1902259265 conn(0x5645ac3d0460 msgr2=0x5645ac785f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 shutdown_connections 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 1 -- 172.21.15.114:0/1902259265 wait complete. 2024-09-21T07:02:08.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.451+0000 7f87bc1bd740 20 client.27584 trim_cache size 0 max 16384 2024-09-21T07:02:08.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 Processor -- start 2024-09-21T07:02:08.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 -- start start 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac780d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac79a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac79ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01b0e80 con 0x5645ac3b31b0 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01b2a20 con 0x5645ac3bb390 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01b1550 con 0x5645ac3dc070 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.465+0000 7f87af200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac780d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac780d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44562/0 (socket says 172.21.15.114:44562) 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 -- 172.21.15.114:0/2208395303 learned_addr learned my addr 172.21.15.114:0/2208395303 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:08.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87ade00640 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac79a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac79ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 -- 172.21.15.114:0/2208395303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 msgr2=0x5645ac79ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac79ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 -- 172.21.15.114:0/2208395303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 msgr2=0x5645ac79a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac79a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 -- 172.21.15.114:0/2208395303 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ac7852c0 con 0x5645ac3b31b0 2024-09-21T07:02:08.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.466+0000 7f87af200640 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac780d00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87a8002230 tx=0x7f87a8009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a0e9f3003ecb6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:08.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f879aa00640 1 -- 172.21.15.114:0/2208395303 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8003e70 con 0x5645ac3b31b0 2024-09-21T07:02:08.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f879aa00640 1 -- 172.21.15.114:0/2208395303 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8036030 con 0x5645ac3b31b0 2024-09-21T07:02:08.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f879aa00640 1 -- 172.21.15.114:0/2208395303 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8004220 con 0x5645ac3b31b0 2024-09-21T07:02:08.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2208395303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 msgr2=0x5645ac780d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac780d00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87a8002230 tx=0x7f87a8009450 comp rx=0 tx=0).stop 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2208395303 shutdown_connections 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3dc070 0x5645ac79ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3bb390 0x5645ac79a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2208395303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3b31b0 0x5645ac780d00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.467+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2208395303 >> 172.21.15.114:0/2208395303 conn(0x5645ac3d0460 msgr2=0x5645ac7805f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.468+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2208395303 shutdown_connections 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.468+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2208395303 wait complete. 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.468+0000 7f87bc1bd740 1 -- start start 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.468+0000 7f87bc1bd740 1 Processor -- start 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac7926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac792bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac793130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01b2350 con 0x5645ac3dc070 2024-09-21T07:02:08.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01b1c20 con 0x5645ac3b31b0 2024-09-21T07:02:08.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01aee80 con 0x5645ac3bb390 2024-09-21T07:02:08.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87bc1bd740 10 client.0 _ll_register_callbacks cb 0x7f87a8032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac792bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87af200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac7926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac792bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50856/0 (socket says 172.21.15.114:50856) 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 -- 172.21.15.114:0/2651238353 learned_addr learned my addr 172.21.15.114:0/2651238353 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87af200640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac793130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 -- 172.21.15.114:0/2651238353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 msgr2=0x5645ac7926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac7926b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 -- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 msgr2=0x5645ac793130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac793130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87ade00640 1 -- 172.21.15.114:0/2651238353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ac80e0a0 con 0x5645ac3bb390 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.469+0000 7f87af200640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac7926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f87ade00640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac792bf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87b0106fc0 tx=0x7f879c012ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95eadc8384bd6dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f878fe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f878fe00640 1 -- 172.21.15.114:0/2651238353 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c02a060 con 0x5645ac3bb390 2024-09-21T07:02:08.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f878fe00640 1 -- 172.21.15.114:0/2651238353 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f879c0035f0 con 0x5645ac3bb390 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a8036030 con 0x5645ac3bb390 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f878fe00640 1 -- 172.21.15.114:0/2651238353 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f879c0037f0 con 0x5645ac3bb390 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f87bc1bd740 10 client.27523 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.470+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5645ac80f380 con 0x5645ac3bb390 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f878fe00640 1 -- 172.21.15.114:0/2651238353 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8036030 con 0x5645ac3bb390 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f878fe00640 1 -- 172.21.15.114:0/2651238353 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f879c010a20 con 0x5645ac3bb390 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87bc1bd740 10 client.27523 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87bc1bd740 10 client.27523 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87bc1bd740 20 client.27523 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac36fda0 0x5645ac811ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87bc1bd740 4 client.27523 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645ac6f79b0 con 0x5645ac36fda0 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.471+0000 7f87af200640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac36fda0 0x5645ac811ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f87af200640 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac36fda0 0x5645ac811ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45a32ba0207eca47 server_cookie=b4fed718b9bebf13 in_seq=0 out_seq=0 2024-09-21T07:02:08.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f878fe00640 10 client.27523 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:02:08.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f878fe00640 1 -- 172.21.15.114:0/2651238353 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645ac6f79b0 con 0x5645ac36fda0 2024-09-21T07:02:08.476 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f878fe00640 10 client.27523 handle_command_reply: tid=0 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f87bc1bd740 1 client.27523 shutdown 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 msgr2=0x5645ac792bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac792bf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87b0106fc0 tx=0x7f879c012ae0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.472+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 shutdown_connections 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5645ac36fda0 0x5645ac811ba0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5645ac3dc070 0x5645ac793130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f878fe00640 0 client.27523 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5645ac3bb390 0x5645ac792bf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 --2- 172.21.15.114:0/2651238353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645ac3b31b0 0x5645ac7926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 >> 172.21.15.114:0/2651238353 conn(0x5645ac3d0460 msgr2=0x5645ac785e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 shutdown_connections 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 1 -- 172.21.15.114:0/2651238353 wait complete. 2024-09-21T07:02:08.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87bc1bd740 20 client.27523 trim_cache size 0 max 16384 2024-09-21T07:02:08.478 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:02:08.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87940b6680 msgr2=0x7f87940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.473+0000 7f87b5400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87940b6680 0x7f87940b8b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x5645ac781a20 tx=0x7f87a0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 msgr2=0x7f87b019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 0x7f87b019a440 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f87a000e9f0 tx=0x7f87a000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 shutdown_connections 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f87940b6680 0x7f87940b8b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87b0109df0 0x7f87b019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b01094a0 0x7f87b019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 --2- 172.21.15.114:0/718060776 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f87b0103b60 0x7f87b019a440 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 >> 172.21.15.114:0/718060776 conn(0x7f87b007b850 msgr2=0x7f87b0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 shutdown_connections 2024-09-21T07:02:08.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:08.474+0000 7f87b5400640 1 -- 172.21.15.114:0/718060776 wait complete. 2024-09-21T07:02:09.380 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:02:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:09 smithi114 ceph-mon[30654]: pgmap v638: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-09-21T07:02:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:09 smithi183 ceph-mon[34219]: pgmap v638: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-09-21T07:02:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:09 smithi193 ceph-mon[34601]: pgmap v638: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-09-21T07:02:09.749 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 -- 172.21.15.114:0/1079172677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04105d80 msgr2=0x7f9c04106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1079172677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04105d80 0x7f9c04106190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf402a410 comp rx=0 tx=0).stop 2024-09-21T07:02:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 -- 172.21.15.114:0/1079172677 shutdown_connections 2024-09-21T07:02:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1079172677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c0410c010 0x7f9c04112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1079172677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1079172677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c04105d80 0x7f9c04106190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.357+0000 7f9c0a600640 1 -- 172.21.15.114:0/1079172677 >> 172.21.15.114:0/1079172677 conn(0x7f9c04101810 msgr2=0x7f9c04103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.358+0000 7f9c0a600640 1 -- 172.21.15.114:0/1079172677 shutdown_connections 2024-09-21T07:02:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.358+0000 7f9c0a600640 1 -- 172.21.15.114:0/1079172677 wait complete. 2024-09-21T07:02:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.358+0000 7f9c0a600640 1 Processor -- start 2024-09-21T07:02:10.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 -- start start 2024-09-21T07:02:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04105d80 0x7f9c0419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0410c010 0x7f9c041a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04115cd0 con 0x7f9c0410c010 2024-09-21T07:02:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04115b50 con 0x7f9c0410b6c0 2024-09-21T07:02:10.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c0a600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04115e50 con 0x7f9c04105d80 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c02a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0410c010 0x7f9c041a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c02a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45524/0 (socket says 172.21.15.114:45524) 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c02a00640 1 -- 172.21.15.114:0/1568064685 learned_addr learned my addr 172.21.15.114:0/1568064685 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.359+0000 7f9c03400640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04105d80 0x7f9c0419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c02a00640 1 -- 172.21.15.114:0/1568064685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04105d80 msgr2=0x7f9c0419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04105d80 0x7f9c0419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c02a00640 1 -- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0410c010 msgr2=0x7f9c041a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0410c010 0x7f9c041a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c02a00640 1 -- 172.21.15.114:0/1568064685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c041a2630 con 0x7f9c0410b6c0 2024-09-21T07:02:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c03e00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0410c010 0x7f9c041a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:10.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.360+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0419cc60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e9f0 tx=0x7f9bec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.361+0000 7f9bfbe00640 1 -- 172.21.15.114:0/1568064685 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00ccc0 con 0x7f9c0410b6c0 2024-09-21T07:02:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.361+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c041a2920 con 0x7f9c0410b6c0 2024-09-21T07:02:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.361+0000 7f9bfbe00640 1 -- 172.21.15.114:0/1568064685 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0108e0 con 0x7f9c0410b6c0 2024-09-21T07:02:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.361+0000 7f9bfbe00640 1 -- 172.21.15.114:0/1568064685 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec018860 con 0x7f9c0410b6c0 2024-09-21T07:02:10.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.361+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041a9080 con 0x7f9c0410b6c0 2024-09-21T07:02:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 Processor -- start 2024-09-21T07:02:10.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 -- start start 2024-09-21T07:02:10.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcb9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a9bcb16fb0 con 0x55a9bc7f9070 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9bcb063e0 con 0x55a9bc7d8390 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c111fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a9bcb0a1f0 con 0x55a9bc7d01b0 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c03400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.364+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcb9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50878/0 (socket says 172.21.15.114:50878) 2024-09-21T07:02:10.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 -- 172.21.15.114:0/4037928413 learned_addr learned my addr 172.21.15.114:0/4037928413 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:10.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9bfbe00640 1 -- 172.21.15.114:0/1568064685 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9bec010430 con 0x7f9c0410b6c0 2024-09-21T07:02:10.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 -- 172.21.15.114:0/4037928413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 msgr2=0x55a9bcba0820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:02:10.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba0820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 -- 172.21.15.114:0/4037928413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 msgr2=0x55a9bcb9fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcb9fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 -- 172.21.15.114:0/4037928413 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9bcad0cd0 con 0x55a9bc7d01b0 2024-09-21T07:02:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.365+0000 7f9c03400640 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba02e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf4032b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5896b48839f36a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.366+0000 7f9bf8c00640 1 -- 172.21.15.114:0/4037928413 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4040070 con 0x55a9bc7d01b0 2024-09-21T07:02:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.366+0000 7f9bf8c00640 1 -- 172.21.15.114:0/4037928413 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf4004030 con 0x55a9bc7d01b0 2024-09-21T07:02:10.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.366+0000 7f9bf8c00640 1 -- 172.21.15.114:0/4037928413 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4035040 con 0x55a9bc7d01b0 2024-09-21T07:02:10.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.366+0000 7f9bfbe00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9be80b6680 0x7f9be80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9bfbe00640 1 -- 172.21.15.114:0/1568064685 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f9bec0e7d70 con 0x7f9c0410b6c0 2024-09-21T07:02:10.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 -- 172.21.15.114:0/4037928413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 msgr2=0x55a9bcba02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba02e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf4032b90 comp rx=0 tx=0).stop 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c03e00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9be80b6680 0x7f9be80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 -- 172.21.15.114:0/4037928413 shutdown_connections 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba02e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 --2- 172.21.15.114:0/4037928413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcb9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 -- 172.21.15.114:0/4037928413 >> 172.21.15.114:0/4037928413 conn(0x55a9bc7ed460 msgr2=0x55a9bcb9dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 -- 172.21.15.114:0/4037928413 shutdown_connections 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.367+0000 7f9c111fa740 1 -- 172.21.15.114:0/4037928413 wait complete. 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.368+0000 7f9c03e00640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9be80b6680 0x7f9be80b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009fd0 tx=0x7f9bfc009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.368+0000 7f9c111fa740 1 -- start start 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.368+0000 7f9c111fa740 1 Processor -- start 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcbaba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcbb7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a9bc967700 con 0x55a9bc7f9070 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9bcaa1580 con 0x55a9bc7d8390 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a9bcbae520 con 0x55a9bc7d01b0 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c111fa740 10 client.0 _ll_register_callbacks cb 0x55a9bcab1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcbb7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbab520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50882/0 (socket says 172.21.15.114:50882) 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 -- 172.21.15.114:0/3088253993 learned_addr learned my addr 172.21.15.114:0/3088253993 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c02a00640 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcbaba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 -- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 msgr2=0x55a9bcbaba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcbaba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 -- 172.21.15.114:0/3088253993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 msgr2=0x55a9bcbb7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcbb7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.369+0000 7f9c03400640 1 -- 172.21.15.114:0/3088253993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9bc864650 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9c03400640 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbab520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4031440 tx=0x7f9bf402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e845bca6d664be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9bf1600640 1 -- 172.21.15.114:0/3088253993 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf404c060 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9bcaa3b90 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9bf1600640 1 -- 172.21.15.114:0/3088253993 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f9bf40314a0 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9bf1600640 1 -- 172.21.15.114:0/3088253993 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9bf4053050 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.370+0000 7f9c111fa740 10 client.27535 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a9bcabe950 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9bf1600640 1 -- 172.21.15.114:0/3088253993 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a9bcaa3b90 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9bf1600640 1 -- 172.21.15.114:0/3088253993 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f9bf4004ab0 con 0x55a9bc7d01b0 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 10 client.27535 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:10.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 10 client.27535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:10.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 20 client.27535 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:10.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc7b79f0 0x55a9bcbbd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 4 client.27535 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:10.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.371+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9bcb18f00 con 0x55a9bc7b79f0 2024-09-21T07:02:10.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.372+0000 7f9c02a00640 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc7b79f0 0x55a9bcbbd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.372+0000 7f9c02a00640 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc7b79f0 0x55a9bcbbd490 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db08a3f6bc8e226 server_cookie=2b5d440d42c90289 in_seq=0 out_seq=0 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.372+0000 7f9bf1600640 10 client.27535 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9bf1600640 1 -- 172.21.15.114:0/3088253993 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9bcb18f00 con 0x55a9bc7b79f0 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9bf1600640 10 client.27535 handle_command_reply: tid=0 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 client.27535 shutdown 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 msgr2=0x55a9bcbab520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbab520 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4031440 tx=0x7f9bf402a410 comp rx=0 tx=0).stop 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 shutdown_connections 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcbb7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcbaba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc7b79f0 0x55a9bcbbd490 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 --2- 172.21.15.114:0/3088253993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbab520 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9bf1600640 0 client.27535 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.373+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 >> 172.21.15.114:0/3088253993 conn(0x55a9bc7ed460 msgr2=0x55a9bcba2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.374+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 shutdown_connections 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.374+0000 7f9c111fa740 1 -- 172.21.15.114:0/3088253993 wait complete. 2024-09-21T07:02:10.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.374+0000 7f9c111fa740 20 client.27535 trim_cache size 0 max 16384 2024-09-21T07:02:10.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.387+0000 7f9c111fa740 1 Processor -- start 2024-09-21T07:02:10.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.387+0000 7f9c111fa740 1 -- start start 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcba8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c111fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041b3150 con 0x55a9bc7f9070 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c111fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041b4cf0 con 0x55a9bc7d8390 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c111fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041b3820 con 0x55a9bc7d01b0 2024-09-21T07:02:10.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcba8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c03400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcba8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44596/0 (socket says 172.21.15.114:44596) 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c03400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50890/0 (socket says 172.21.15.114:50890) 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c03e00640 1 -- 172.21.15.114:0/1459019440 learned_addr learned my addr 172.21.15.114:0/1459019440 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.388+0000 7f9c03400640 1 -- 172.21.15.114:0/1459019440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 msgr2=0x55a9bcba8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9c03400640 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9c03400640 1 -- 172.21.15.114:0/1459019440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 msgr2=0x55a9bcba8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9c03400640 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcba8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9c03400640 1 -- 172.21.15.114:0/1459019440 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9bcba3af0 con 0x55a9bc7d01b0 2024-09-21T07:02:10.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9c03400640 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba35b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf402ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38fbeabe61f21451 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9bf3e00640 1 -- 172.21.15.114:0/1459019440 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf404c070 con 0x55a9bc7d01b0 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9bf3e00640 1 -- 172.21.15.114:0/1459019440 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf4005630 con 0x55a9bc7d01b0 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.389+0000 7f9bf3e00640 1 -- 172.21.15.114:0/1459019440 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4035030 con 0x55a9bc7d01b0 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 -- 172.21.15.114:0/1459019440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 msgr2=0x55a9bcba35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba35b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009900 tx=0x7f9bf402ab10 comp rx=0 tx=0).stop 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 -- 172.21.15.114:0/1459019440 shutdown_connections 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7f9070 0x55a9bcba8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7d8390 0x55a9bcba8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1459019440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d01b0 0x55a9bcba35b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 -- 172.21.15.114:0/1459019440 >> 172.21.15.114:0/1459019440 conn(0x55a9bc7ed460 msgr2=0x55a9bcba2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:10.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 -- 172.21.15.114:0/1459019440 shutdown_connections 2024-09-21T07:02:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.390+0000 7f9c111fa740 1 -- 172.21.15.114:0/1459019440 wait complete. 2024-09-21T07:02:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.391+0000 7f9c111fa740 1 -- start start 2024-09-21T07:02:10.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.391+0000 7f9c111fa740 1 Processor -- start 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.391+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.391+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d8390 0x55a9bcbbebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c111fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7f9070 0x55a9bcbbf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c111fa740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041b4620 con 0x55a9bc7d01b0 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c111fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041b3ef0 con 0x55a9bc7f9070 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c111fa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041b1150 con 0x55a9bc7d8390 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c111fa740 10 client.0 _ll_register_callbacks cb 0x7f9bf4005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c03400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c02a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d8390 0x55a9bcbbebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c03400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbbe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44606/0 (socket says 172.21.15.114:44606) 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c03400640 1 -- 172.21.15.114:0/1157077736 learned_addr learned my addr 172.21.15.114:0/1157077736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c03e00640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7f9070 0x55a9bcbbf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c02a00640 1 -- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7f9070 msgr2=0x55a9bcbbf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7f9070 0x55a9bcbbf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c02a00640 1 -- 172.21.15.114:0/1157077736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 msgr2=0x55a9bcbbe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbbe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c02a00640 1 -- 172.21.15.114:0/1157077736 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9bcc26e10 con 0x55a9bc7d8390 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.392+0000 7f9c03400640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9c02a00640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d8390 0x55a9bcbbebc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0e50f0 tx=0x7f9bec0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=194db0ce11eee127 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9bf1600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9bf1600640 1 -- 172.21.15.114:0/1157077736 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0d2070 con 0x55a9bc7d8390 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9bcb149b0 con 0x55a9bc7d8390 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9bf1600640 1 -- 172.21.15.114:0/1157077736 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9490+0+0 (secure 0 0 0) 0x7f9bec0aeb20 con 0x55a9bc7d8390 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9bf1600640 1 -- 172.21.15.114:0/1157077736 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9bec010720 con 0x55a9bc7d8390 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9c111fa740 10 client.27541 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a9bcc280b0 con 0x55a9bc7d8390 2024-09-21T07:02:10.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.393+0000 7f9bf1600640 1 -- 172.21.15.114:0/1157077736 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a9bcb149b0 con 0x55a9bc7d8390 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9bf1600640 1 -- 172.21.15.114:0/1157077736 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f9bec0ac570 con 0x55a9bc7d8390 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c111fa740 10 client.27541 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c111fa740 10 client.27541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c111fa740 20 client.27541 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc78cda0 0x55a9bcc2a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c111fa740 4 client.27541 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9bcb11de0 con 0x55a9bc78cda0 2024-09-21T07:02:10.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.394+0000 7f9c03400640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc78cda0 0x55a9bcc2a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c03400640 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc78cda0 0x55a9bcc2a820 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e305e59fa45c1d2 server_cookie=86caf51ed21bcc1f in_seq=0 out_seq=0 2024-09-21T07:02:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9bf1600640 10 client.27541 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9bf1600640 1 -- 172.21.15.114:0/1157077736 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9bcb11de0 con 0x55a9bc78cda0 2024-09-21T07:02:10.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9bf1600640 10 client.27541 handle_command_reply: tid=0 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 client.27541 shutdown 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d8390 msgr2=0x55a9bcbbebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d8390 0x55a9bcbbebc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0e50f0 tx=0x7f9bec0096a0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 shutdown_connections 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9bc7f9070 0x55a9bcbbf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a9bc78cda0 0x55a9bcc2a820 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a9bc7d8390 0x55a9bcbbebc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 --2- 172.21.15.114:0/1157077736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a9bc7d01b0 0x55a9bcbbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9bf1600640 0 client.27541 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 >> 172.21.15.114:0/1157077736 conn(0x55a9bc7ed460 msgr2=0x55a9bcba0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 shutdown_connections 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 1 -- 172.21.15.114:0/1157077736 wait complete. 2024-09-21T07:02:10.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.395+0000 7f9c111fa740 20 client.27541 trim_cache size 0 max 16384 2024-09-21T07:02:10.401 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:02:10.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9be80b6680 msgr2=0x7f9be80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9be80b6680 0x7f9be80b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009fd0 tx=0x7f9bfc009500 comp rx=0 tx=0).stop 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 msgr2=0x7f9c0419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0419cc60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00e9f0 tx=0x7f9bec00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 shutdown_connections 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c0410c010 0x7f9c041a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9be80b6680 0x7f9be80b8b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c0410b6c0 0x7f9c0419cc60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 --2- 172.21.15.114:0/1568064685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c04105d80 0x7f9c0419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 >> 172.21.15.114:0/1568064685 conn(0x7f9c04101810 msgr2=0x7f9c04103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 shutdown_connections 2024-09-21T07:02:10.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:10.396+0000 7f9c0a600640 1 -- 172.21.15.114:0/1568064685 wait complete. 2024-09-21T07:02:11.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:11 smithi114 ceph-mon[30654]: pgmap v639: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-09-21T07:02:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:11 smithi183 ceph-mon[34219]: pgmap v639: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-09-21T07:02:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:11 smithi193 ceph-mon[34601]: pgmap v639: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 -- 172.21.15.114:0/1598269425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8580105b10 msgr2=0x7f8580105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 --2- 172.21.15.114:0/1598269425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8580105b10 0x7f8580105f20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8570009860 tx=0x7f857002a340 comp rx=0 tx=0).stop 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 -- 172.21.15.114:0/1598269425 shutdown_connections 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 --2- 172.21.15.114:0/1598269425 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010bd90 0x7f85801125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 --2- 172.21.15.114:0/1598269425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f858010b440 0x7f858010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 --2- 172.21.15.114:0/1598269425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8580105b10 0x7f8580105f20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 -- 172.21.15.114:0/1598269425 >> 172.21.15.114:0/1598269425 conn(0x7f8580101550 msgr2=0x7f85801039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 -- 172.21.15.114:0/1598269425 shutdown_connections 2024-09-21T07:02:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.933+0000 7f857f200640 1 -- 172.21.15.114:0/1598269425 wait complete. 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 Processor -- start 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 -- start start 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 0x7f858019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010b440 0x7f858019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f858010bd90 0x7f85801a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8580115a50 con 0x7f858010b440 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85801158d0 con 0x7f858010bd90 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.934+0000 7f857f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8580115bd0 con 0x7f8580105b10 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 0x7f858019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f8575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010b440 0x7f858019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 0x7f858019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50902/0 (socket says 172.21.15.114:50902) 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 -- 172.21.15.114:0/115038306 learned_addr learned my addr 172.21.15.114:0/115038306 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857e800640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f858010bd90 0x7f85801a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 -- 172.21.15.114:0/115038306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f858010bd90 msgr2=0x7f85801a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f858010bd90 0x7f85801a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 -- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010b440 msgr2=0x7f858019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010b440 0x7f858019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 -- 172.21.15.114:0/115038306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85801a2400 con 0x7f8580105b10 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f8575400640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010b440 0x7f858019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.935+0000 7f857de00640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 0x7f858019c4d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85700303f0 tx=0x7f8570032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.936+0000 7f8577400640 1 -- 172.21.15.114:0/115038306 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570040020 con 0x7f8580105b10 2024-09-21T07:02:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.936+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a2690 con 0x7f8580105b10 2024-09-21T07:02:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.936+0000 7f8577400640 1 -- 172.21.15.114:0/115038306 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8570030dd0 con 0x7f8580105b10 2024-09-21T07:02:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.936+0000 7f8577400640 1 -- 172.21.15.114:0/115038306 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570037040 con 0x7f8580105b10 2024-09-21T07:02:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.936+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801a8e80 con 0x7f8580105b10 2024-09-21T07:02:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.939+0000 7f858c240740 1 Processor -- start 2024-09-21T07:02:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.939+0000 7f858c240740 1 -- start start 2024-09-21T07:02:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f8577400640 1 -- 172.21.15.114:0/115038306 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8570043020 con 0x7f8580105b10 2024-09-21T07:02:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f858c240740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e3be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f858c240740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f858c240740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a764d0 0x55f497e3c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f858c240740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f497db16b0 con 0x55f497a764d0 2024-09-21T07:02:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f858c240740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f497da0ae0 con 0x55f497a971b0 2024-09-21T07:02:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f858c240740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f497da48f0 con 0x55f497a6e2f0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e3be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f8575400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a764d0 0x55f497e3c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e3be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45566/0 (socket says 172.21.15.114:45566) 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 -- 172.21.15.114:0/548189518 learned_addr learned my addr 172.21.15.114:0/548189518 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 -- 172.21.15.114:0/548189518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 msgr2=0x55f497e3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e3c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 -- 172.21.15.114:0/548189518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a764d0 msgr2=0x55f497e3c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a764d0 0x55f497e3c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.940+0000 7f857e800640 1 -- 172.21.15.114:0/548189518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f497d6d780 con 0x55f497a971b0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.941+0000 7f857e800640 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e3be10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f857800e480 tx=0x7f85780095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6992c6529feb3fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.941+0000 7f856f400640 1 -- 172.21.15.114:0/548189518 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857800d910 con 0x55f497a971b0 2024-09-21T07:02:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.941+0000 7f856f400640 1 -- 172.21.15.114:0/548189518 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857801c070 con 0x55f497a971b0 2024-09-21T07:02:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.941+0000 7f856f400640 1 -- 172.21.15.114:0/548189518 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578017d90 con 0x55f497a971b0 2024-09-21T07:02:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.941+0000 7f8577400640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85640b6680 0x7f85640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f8577400640 1 -- 172.21.15.114:0/115038306 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f85701077a0 con 0x7f8580105b10 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f857de00640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85640b6680 0x7f85640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 -- 172.21.15.114:0/548189518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 msgr2=0x55f497e3be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e3be10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f857800e480 tx=0x7f85780095c0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 -- 172.21.15.114:0/548189518 shutdown_connections 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a764d0 0x55f497e3c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 --2- 172.21.15.114:0/548189518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e3be10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 -- 172.21.15.114:0/548189518 >> 172.21.15.114:0/548189518 conn(0x55f497a8b300 msgr2=0x55f497e39d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 -- 172.21.15.114:0/548189518 shutdown_connections 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f858c240740 1 -- 172.21.15.114:0/548189518 wait complete. 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.942+0000 7f857de00640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85640b6680 0x7f85640b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55f497e3aa90 tx=0x7f857010d530 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 -- start start 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 Processor -- start 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a6e2f0 0x55f497e47490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 0x55f497e479d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e47f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f497cd8ab0 con 0x55f497a6e2f0 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f497d47570 con 0x55f497a971b0 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f497e4a590 con 0x55f497a764d0 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.943+0000 7f858c240740 10 client.0 _ll_register_callbacks cb 0x55f497d56100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 0x55f497e479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f857e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a6e2f0 0x55f497e47490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 0x55f497e479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50932/0 (socket says 172.21.15.114:50932) 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 -- 172.21.15.114:0/1622415819 learned_addr learned my addr 172.21.15.114:0/1622415819 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f857e800640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e47f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 -- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 msgr2=0x55f497e47f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e47f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 -- 172.21.15.114:0/1622415819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a6e2f0 msgr2=0x55f497e47490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a6e2f0 0x55f497e47490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 -- 172.21.15.114:0/1622415819 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f497b02790 con 0x55f497a764d0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f857e800640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e47f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8575400640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 0x55f497e479d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8568012d20 tx=0x7f856800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580faeb17aeeb4bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8563e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.944+0000 7f8563e00640 1 -- 172.21.15.114:0/1622415819 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568012110 con 0x55f497a764d0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857801c070 con 0x55f497a764d0 2024-09-21T07:02:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f8563e00640 1 -- 172.21.15.114:0/1622415819 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f8568004050 con 0x55f497a764d0 2024-09-21T07:02:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f8563e00640 1 -- 172.21.15.114:0/1622415819 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f85680122b0 con 0x55f497a764d0 2024-09-21T07:02:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f858c240740 10 client.27553 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f497b7b620 con 0x55f497a764d0 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f8563e00640 1 -- 172.21.15.114:0/1622415819 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857801c070 con 0x55f497a764d0 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f8563e00640 1 -- 172.21.15.114:0/1622415819 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f8568010430 con 0x55f497a764d0 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f858c240740 10 client.27553 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f858c240740 10 client.27553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.945+0000 7f858c240740 20 client.27553 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.946+0000 7f858c240740 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a55d80 0x55f497e59630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.946+0000 7f858c240740 4 client.27553 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.946+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f497db3600 con 0x55f497a55d80 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.946+0000 7f857e800640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a55d80 0x55f497e59630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.946+0000 7f857e800640 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a55d80 0x55f497e59630 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab171ebe498ca7 server_cookie=839df66b60c44e7a in_seq=0 out_seq=0 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.946+0000 7f8563e00640 10 client.27553 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f8563e00640 1 -- 172.21.15.114:0/1622415819 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f497db3600 con 0x55f497a55d80 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f8563e00640 10 client.27553 handle_command_reply: tid=0 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 client.27553 shutdown 2024-09-21T07:02:11.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 msgr2=0x55f497e479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 0x55f497e479d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8568012d20 tx=0x7f856800f790 comp rx=0 tx=0).stop 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 shutdown_connections 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a971b0 0x55f497e47f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a55d80 0x55f497e59630 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a764d0 0x55f497e479d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f8563e00640 0 client.27553 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 --2- 172.21.15.114:0/1622415819 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a6e2f0 0x55f497e47490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 >> 172.21.15.114:0/1622415819 conn(0x55f497a8b300 msgr2=0x55f497e3ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 shutdown_connections 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.947+0000 7f858c240740 1 -- 172.21.15.114:0/1622415819 wait complete. 2024-09-21T07:02:11.952 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.948+0000 7f858c240740 20 client.27553 trim_cache size 0 max 16384 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 Processor -- start 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 -- start start 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e44660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b30c0 con 0x55f497a971b0 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b4c60 con 0x55f497a764d0 2024-09-21T07:02:11.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f858c240740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b3790 con 0x55f497a6e2f0 2024-09-21T07:02:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.963+0000 7f857e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e44ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44662/0 (socket says 172.21.15.114:44662) 2024-09-21T07:02:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 -- 172.21.15.114:0/1625425001 learned_addr learned my addr 172.21.15.114:0/1625425001 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f8575400640 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e44660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 -- 172.21.15.114:0/1625425001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 msgr2=0x55f497e38a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e38a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 -- 172.21.15.114:0/1625425001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 msgr2=0x55f497e44660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e44660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 -- 172.21.15.114:0/1625425001 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f497e49fd0 con 0x55f497a971b0 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f857e800640 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e44ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f857800e450 tx=0x7f8578017990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e6f6b16700db52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f856ea00640 1 -- 172.21.15.114:0/1625425001 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578034070 con 0x55f497a971b0 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f856ea00640 1 -- 172.21.15.114:0/1625425001 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857801c070 con 0x55f497a971b0 2024-09-21T07:02:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.964+0000 7f856ea00640 1 -- 172.21.15.114:0/1625425001 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578005370 con 0x55f497a971b0 2024-09-21T07:02:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 -- 172.21.15.114:0/1625425001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 msgr2=0x55f497e44ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e44ba0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f857800e450 tx=0x7f8578017990 comp rx=0 tx=0).stop 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 -- 172.21.15.114:0/1625425001 shutdown_connections 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e44ba0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e44660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 --2- 172.21.15.114:0/1625425001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 -- 172.21.15.114:0/1625425001 >> 172.21.15.114:0/1625425001 conn(0x55f497a8b300 msgr2=0x55f497e38360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 -- 172.21.15.114:0/1625425001 shutdown_connections 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.965+0000 7f858c240740 1 -- 172.21.15.114:0/1625425001 wait complete. 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 -- start start 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 Processor -- start 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e4e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b4590 con 0x55f497a971b0 2024-09-21T07:02:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b3e60 con 0x55f497a764d0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f85801b10c0 con 0x55f497a6e2f0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.966+0000 7f858c240740 10 client.0 _ll_register_callbacks cb 0x7f8568009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e4e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e4e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44670/0 (socket says 172.21.15.114:44670) 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 -- 172.21.15.114:0/2685186441 learned_addr learned my addr 172.21.15.114:0/2685186441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f8575400640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 -- 172.21.15.114:0/2685186441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 msgr2=0x55f497e4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 -- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 msgr2=0x55f497e4df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e4df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 -- 172.21.15.114:0/2685186441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f497e53780 con 0x55f497a971b0 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f8575400640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f857e800640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e4e480 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8578017680 tx=0x7f8578005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec0b933c27efc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.967+0000 7f8563e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f8563e00640 1 -- 172.21.15.114:0/2685186441 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578034040 con 0x55f497a971b0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f8563e00640 1 -- 172.21.15.114:0/2685186441 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f85780054c0 con 0x55f497a971b0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857801c070 con 0x55f497a971b0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f8563e00640 1 -- 172.21.15.114:0/2685186441 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8578018e30 con 0x55f497a971b0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 10 client.17226 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f497ec42f0 con 0x55f497a971b0 2024-09-21T07:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f8563e00640 1 -- 172.21.15.114:0/2685186441 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f857801c070 con 0x55f497a971b0 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f8563e00640 1 -- 172.21.15.114:0/2685186441 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f857803a070 con 0x55f497a971b0 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 10 client.17226 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 10 client.17226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 20 client.17226 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a2b1c0 0x55f497ec6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 4 client.17226 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.968+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f497daf0b0 con 0x55f497a2b1c0 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.969+0000 7f8575400640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a2b1c0 0x55f497ec6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.969+0000 7f8575400640 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a2b1c0 0x55f497ec6800 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0d41e2a48f02b4 server_cookie=eee8f8084647d328 in_seq=0 out_seq=0 2024-09-21T07:02:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.969+0000 7f8563e00640 10 client.17226 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f8563e00640 1 -- 172.21.15.114:0/2685186441 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f497daf0b0 con 0x55f497a2b1c0 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f8563e00640 10 client.17226 handle_command_reply: tid=0 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 client.17226 shutdown 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 msgr2=0x55f497e4e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e4e480 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8578017680 tx=0x7f8578005bd0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 shutdown_connections 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f497a971b0 0x55f497e4e480 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f497a764d0 0x55f497e4df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f497a2b1c0 0x55f497ec6800 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 --2- 172.21.15.114:0/2685186441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f497a6e2f0 0x55f497e4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 >> 172.21.15.114:0/2685186441 conn(0x55f497a8b300 msgr2=0x55f497e38360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f8563e00640 0 client.17226 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 shutdown_connections 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 1 -- 172.21.15.114:0/2685186441 wait complete. 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.977+0000 7f858c240740 20 client.17226 trim_cache size 0 max 16384 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:02:11.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85640b6680 msgr2=0x7f85640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85640b6680 0x7f85640b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55f497e3aa90 tx=0x7f857010d530 comp rx=0 tx=0).stop 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 msgr2=0x7f858019c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 0x7f858019c4d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85700303f0 tx=0x7f8570032da0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 shutdown_connections 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f85640b6680 0x7f85640b8b90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f858010bd90 0x7f85801a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f858010b440 0x7f858019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 --2- 172.21.15.114:0/115038306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8580105b10 0x7f858019c4d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 >> 172.21.15.114:0/115038306 conn(0x7f8580101550 msgr2=0x7f8580103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 shutdown_connections 2024-09-21T07:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:11.978+0000 7f857f200640 1 -- 172.21.15.114:0/115038306 wait complete. 2024-09-21T07:02:12.156 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:02:12.156 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:02:12.527 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:12 smithi114 ceph-mon[30654]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T07:02:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:12 smithi183 ceph-mon[34219]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T07:02:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:12 smithi193 ceph-mon[34601]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T07:02:12.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 455.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:02:13.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 -- 172.21.15.114:0/547677082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c81092f0 msgr2=0x7ff0c8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 --2- 172.21.15.114:0/547677082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c81092f0 0x7ff0c8109700 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009900 tx=0x7ff0b402a410 comp rx=0 tx=0).stop 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 -- 172.21.15.114:0/547677082 shutdown_connections 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 --2- 172.21.15.114:0/547677082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c8109c40 0x7ff0c8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 --2- 172.21.15.114:0/547677082 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c81092f0 0x7ff0c8109700 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 --2- 172.21.15.114:0/547677082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0c81039b0 0x7ff0c8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 -- 172.21.15.114:0/547677082 >> 172.21.15.114:0/547677082 conn(0x7ff0c80ff440 msgr2=0x7ff0c8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 -- 172.21.15.114:0/547677082 shutdown_connections 2024-09-21T07:02:13.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.097+0000 7ff0cea00640 1 -- 172.21.15.114:0/547677082 wait complete. 2024-09-21T07:02:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 Processor -- start 2024-09-21T07:02:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 -- start start 2024-09-21T07:02:13.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 0x7ff0c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0c81092f0 0x7ff0c8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c8109c40 0x7ff0c8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8113950 con 0x7ff0c8109c40 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81137d0 con 0x7ff0c81092f0 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.099+0000 7ff0cea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8113ad0 con 0x7ff0c81039b0 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 0x7ff0c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 0x7ff0c8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50956/0 (socket says 172.21.15.114:50956) 2024-09-21T07:02:13.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 -- 172.21.15.114:0/3204550241 learned_addr learned my addr 172.21.15.114:0/3204550241 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c8109c40 0x7ff0c8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0c81092f0 0x7ff0c8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 -- 172.21.15.114:0/3204550241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0c81092f0 msgr2=0x7ff0c8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0c81092f0 0x7ff0c8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 -- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c8109c40 msgr2=0x7ff0c8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c8109c40 0x7ff0c8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 -- 172.21.15.114:0/3204550241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c81a85a0 con 0x7ff0c81039b0 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c8109c40 0x7ff0c8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.100+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 0x7ff0c8075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00e9f0 tx=0x7ff0bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.101+0000 7ff0c6000640 1 -- 172.21.15.114:0/3204550241 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc00cd80 con 0x7ff0c81039b0 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.101+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c81a8890 con 0x7ff0c81039b0 2024-09-21T07:02:13.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.101+0000 7ff0c6000640 1 -- 172.21.15.114:0/3204550241 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0bc0108e0 con 0x7ff0c81039b0 2024-09-21T07:02:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.101+0000 7ff0c6000640 1 -- 172.21.15.114:0/3204550241 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc018980 con 0x7ff0c81039b0 2024-09-21T07:02:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.101+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c81a8df0 con 0x7ff0c81039b0 2024-09-21T07:02:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 Processor -- start 2024-09-21T07:02:13.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 -- start start 2024-09-21T07:02:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df884f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4b4490 0x5564df885440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df885980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564df7fae90 con 0x5564df4dd350 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5564df7ea2c0 con 0x5564df4bc670 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.104+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564df7ee0d0 con 0x5564df4b4490 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ce000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df884f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ce000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df884f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44704/0 (socket says 172.21.15.114:44704) 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ce000640 1 -- 172.21.15.114:0/3295758619 learned_addr learned my addr 172.21.15.114:0/3295758619 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df885980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0c6000640 1 -- 172.21.15.114:0/3204550241 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff0bc010430 con 0x7ff0c81039b0 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3295758619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4b4490 msgr2=0x5564df885440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4b4490 0x5564df885440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3295758619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 msgr2=0x5564df884f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df884f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3295758619 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564df548930 con 0x5564df4bc670 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.105+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df884f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.106+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df885980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009230 tx=0x7ff0b4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10d3b0353084a41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.106+0000 7ff0ba600640 1 -- 172.21.15.114:0/3295758619 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4035040 con 0x5564df4bc670 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.106+0000 7ff0ba600640 1 -- 172.21.15.114:0/3295758619 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0b40054f0 con 0x5564df4bc670 2024-09-21T07:02:13.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.106+0000 7ff0ba600640 1 -- 172.21.15.114:0/3295758619 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40035f0 con 0x5564df4bc670 2024-09-21T07:02:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3295758619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 msgr2=0x5564df885980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df885980 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009230 tx=0x7ff0b4004040 comp rx=0 tx=0).stop 2024-09-21T07:02:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0c6000640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff0b00b6700 0x7ff0b00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0c6000640 1 -- 172.21.15.114:0/3204550241 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7ff0bc0e7f00 con 0x7ff0c81039b0 2024-09-21T07:02:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3295758619 shutdown_connections 2024-09-21T07:02:13.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df885980 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4b4490 0x5564df885440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3295758619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df884f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3295758619 >> 172.21.15.114:0/3295758619 conn(0x5564df4d1740 msgr2=0x5564df882df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:13.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.107+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff0b00b6700 0x7ff0b00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.108+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3295758619 shutdown_connections 2024-09-21T07:02:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.108+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3295758619 wait complete. 2024-09-21T07:02:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.108+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff0b00b6700 0x7ff0b00b8c10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005c40 tx=0x7ff0c0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 -- start start 2024-09-21T07:02:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 Processor -- start 2024-09-21T07:02:13.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4b4490 0x5564df890b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df891100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df89c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5564df64b890 con 0x5564df4dd350 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5564df785160 con 0x5564df4b4490 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5564df893680 con 0x5564df4bc670 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.109+0000 7ff0d6dc5740 10 client.0 _ll_register_callbacks cb 0x5564df7a1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:13.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ce000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df89c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df891100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0cd600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4b4490 0x5564df890b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df891100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50962/0 (socket says 172.21.15.114:50962) 2024-09-21T07:02:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3453321749 learned_addr learned my addr 172.21.15.114:0/3453321749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:13.113 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4b4490 msgr2=0x5564df890b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4b4490 0x5564df890b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3453321749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 msgr2=0x5564df89c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df89c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ccc00640 1 -- 172.21.15.114:0/3453321749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564df89cfe0 con 0x5564df4bc670 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4b4490 0x5564df890b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.110+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df89c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df891100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40092a0 tx=0x7ff0b4005710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56cbe1d6eb81acc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0aea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0aea00640 1 -- 172.21.15.114:0/3453321749 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4053060 con 0x5564df4bc670 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b40054f0 con 0x5564df4bc670 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0aea00640 1 -- 172.21.15.114:0/3453321749 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7ff0b4004e30 con 0x5564df4bc670 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0aea00640 1 -- 172.21.15.114:0/3453321749 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff0b405a050 con 0x5564df4bc670 2024-09-21T07:02:13.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0d6dc5740 10 client.27568 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:13.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.111+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5564df89f8f0 con 0x5564df4bc670 2024-09-21T07:02:13.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0aea00640 1 -- 172.21.15.114:0/3453321749 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0b40054f0 con 0x5564df4bc670 2024-09-21T07:02:13.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0aea00640 1 -- 172.21.15.114:0/3453321749 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7ff0b4003d00 con 0x5564df4bc670 2024-09-21T07:02:13.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0d6dc5740 10 client.27568 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:13.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0d6dc5740 10 client.27568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:13.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0d6dc5740 20 client.27568 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df49bcd0 0x5564df8a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0d6dc5740 4 client.27568 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.112+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564df7fcde0 con 0x5564df49bcd0 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.113+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df49bcd0 0x5564df8a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.113+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df49bcd0 0x5564df8a2c50 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b06f2e25e4c7a59 server_cookie=10ab3999eff8c744 in_seq=0 out_seq=0 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.113+0000 7ff0aea00640 10 client.27568 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0aea00640 1 -- 172.21.15.114:0/3453321749 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5564df7fcde0 con 0x5564df49bcd0 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0aea00640 10 client.27568 handle_command_reply: tid=0 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 client.27568 shutdown 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 msgr2=0x5564df891100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df891100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40092a0 tx=0x7ff0b4005710 comp rx=0 tx=0).stop 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 shutdown_connections 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4dd350 0x5564df89c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df49bcd0 0x5564df8a2c50 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df891100 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3453321749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4b4490 0x5564df890b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0aea00640 0 client.27568 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.114+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 >> 172.21.15.114:0/3453321749 conn(0x5564df4d1740 msgr2=0x5564df8880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.115+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 shutdown_connections 2024-09-21T07:02:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.115+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3453321749 wait complete. 2024-09-21T07:02:13.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.115+0000 7ff0d6dc5740 20 client.27568 trim_cache size 0 max 16384 2024-09-21T07:02:13.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 Processor -- start 2024-09-21T07:02:13.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 -- start start 2024-09-21T07:02:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df89c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df890b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4dd350 0x5564df891060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81b3070 con 0x5564df4b4490 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81b4c10 con 0x5564df4dd350 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.133+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81b3740 con 0x5564df4bc670 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df89c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df89c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44720/0 (socket says 172.21.15.114:44720) 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 -- 172.21.15.114:0/3188158783 learned_addr learned my addr 172.21.15.114:0/3188158783 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df890b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4dd350 0x5564df891060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 -- 172.21.15.114:0/3188158783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 msgr2=0x5564df890b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df890b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 -- 172.21.15.114:0/3188158783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4dd350 msgr2=0x5564df891060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4dd350 0x5564df891060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 -- 172.21.15.114:0/3188158783 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564df887d60 con 0x5564df4b4490 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0ce000640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4dd350 0x5564df891060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df890b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.134+0000 7ff0cd600640 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df89c8e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0a7fd0 tx=0x7ff0bc0d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138006fea69696b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0b9c00640 1 -- 172.21.15.114:0/3188158783 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ad3e0 con 0x5564df4b4490 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0b9c00640 1 -- 172.21.15.114:0/3188158783 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0bc0adfd0 con 0x5564df4b4490 2024-09-21T07:02:13.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0b9c00640 1 -- 172.21.15.114:0/3188158783 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0a9220 con 0x5564df4b4490 2024-09-21T07:02:13.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3188158783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 msgr2=0x5564df89c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df89c8e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0a7fd0 tx=0x7ff0bc0d1040 comp rx=0 tx=0).stop 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3188158783 shutdown_connections 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4dd350 0x5564df891060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4bc670 0x5564df890b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.135+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/3188158783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df89c8e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.136+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3188158783 >> 172.21.15.114:0/3188158783 conn(0x5564df4d1740 msgr2=0x5564df88f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.136+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3188158783 shutdown_connections 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.136+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/3188158783 wait complete. 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.136+0000 7ff0d6dc5740 1 -- start start 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.136+0000 7ff0d6dc5740 1 Processor -- start 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df8a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df8a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4dd350 0x5564df8a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81b4540 con 0x5564df4b4490 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81b3e10 con 0x5564df4bc670 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c81b1070 con 0x5564df4dd350 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0d6dc5740 10 client.0 _ll_register_callbacks cb 0x7ff0c0009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df8a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df8a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45644/0 (socket says 172.21.15.114:45644) 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 -- 172.21.15.114:0/4040676892 learned_addr learned my addr 172.21.15.114:0/4040676892 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0cd600640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df8a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ce000640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4dd350 0x5564df8a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 -- 172.21.15.114:0/4040676892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4dd350 msgr2=0x5564df8a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4dd350 0x5564df8a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 -- 172.21.15.114:0/4040676892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 msgr2=0x5564df8a62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df8a62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.137+0000 7ff0ccc00640 1 -- 172.21.15.114:0/4040676892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564df8a42c0 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0ccc00640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df8a3790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff0b402a450 tx=0x7ff0b4031860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b27ad48e7495a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0aea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0aea00640 1 -- 172.21.15.114:0/4040676892 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4063060 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0bc0adfd0 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0aea00640 1 -- 172.21.15.114:0/4040676892 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7ff0b4004b90 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0aea00640 1 -- 172.21.15.114:0/4040676892 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff0b4053070 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0d6dc5740 10 client.27638 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.138+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5564df90d120 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0aea00640 1 -- 172.21.15.114:0/4040676892 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0bc0adfd0 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0aea00640 1 -- 172.21.15.114:0/4040676892 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7ff0b404e460 con 0x5564df4bc670 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0d6dc5740 10 client.27638 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0d6dc5740 10 client.27638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0d6dc5740 20 client.27638 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df471080 0x5564df90f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:13.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0d6dc5740 4 client.27638 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:02:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5564df7f8890 con 0x5564df471080 2024-09-21T07:02:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.139+0000 7ff0cd600640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df471080 0x5564df90f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.140+0000 7ff0cd600640 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df471080 0x5564df90f7e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1439aa13dab3bc2e server_cookie=55e1787633d2d472 in_seq=0 out_seq=0 2024-09-21T07:02:13.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.140+0000 7ff0aea00640 10 client.27638 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.141+0000 7ff0aea00640 1 -- 172.21.15.114:0/4040676892 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564df7f8890 con 0x5564df471080 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.141+0000 7ff0aea00640 10 client.27638 handle_command_reply: tid=0 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.141+0000 7ff0d6dc5740 1 client.27638 shutdown 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 msgr2=0x5564df8a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df8a3790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff0b402a450 tx=0x7ff0b4031860 comp rx=0 tx=0).stop 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 shutdown_connections 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5564df471080 0x5564df90f7e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5564df4dd350 0x5564df8a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5564df4bc670 0x5564df8a3790 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 --2- 172.21.15.114:0/4040676892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5564df4b4490 0x5564df8a62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0aea00640 0 client.27638 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 >> 172.21.15.114:0/4040676892 conn(0x5564df4d1740 msgr2=0x5564df88a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 shutdown_connections 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 1 -- 172.21.15.114:0/4040676892 wait complete. 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.142+0000 7ff0d6dc5740 20 client.27638 trim_cache size 0 max 16384 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506", 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:02:13.147 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff0b00b6700 msgr2=0x7ff0b00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff0b00b6700 0x7ff0b00b8c10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005c40 tx=0x7ff0c0009500 comp rx=0 tx=0).stop 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 msgr2=0x7ff0c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 0x7ff0c8075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00e9f0 tx=0x7ff0bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 shutdown_connections 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff0c8109c40 0x7ff0c8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff0b00b6700 0x7ff0b00b8c10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0c81092f0 0x7ff0c8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 --2- 172.21.15.114:0/3204550241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff0c81039b0 0x7ff0c8075710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:13.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 >> 172.21.15.114:0/3204550241 conn(0x7ff0c80ff440 msgr2=0x7ff0c8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:13.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 shutdown_connections 2024-09-21T07:02:13.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:13.143+0000 7ff0cea00640 1 -- 172.21.15.114:0/3204550241 wait complete. 2024-09-21T07:02:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:13 smithi114 ceph-mon[30654]: pgmap v641: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-09-21T07:02:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:13 smithi183 ceph-mon[34219]: pgmap v641: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-09-21T07:02:13.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:13 smithi193 ceph-mon[34601]: pgmap v641: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-09-21T07:02:14.259 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:14 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:14 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.628 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:14.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:14 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:02:15.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.263+0000 7f71c2600640 1 -- 172.21.15.114:0/4208286405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc109c40 msgr2=0x7f71bc110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.263+0000 7f71c2600640 1 --2- 172.21.15.114:0/4208286405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc109c40 0x7f71bc110470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009be0 tx=0x7f71b402a6c0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 -- 172.21.15.114:0/4208286405 shutdown_connections 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 --2- 172.21.15.114:0/4208286405 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc109c40 0x7f71bc110470 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 --2- 172.21.15.114:0/4208286405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc1092f0 0x7f71bc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 --2- 172.21.15.114:0/4208286405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71bc1039b0 0x7f71bc103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 -- 172.21.15.114:0/4208286405 >> 172.21.15.114:0/4208286405 conn(0x7f71bc0ff400 msgr2=0x7f71bc101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 -- 172.21.15.114:0/4208286405 shutdown_connections 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.264+0000 7f71c2600640 1 -- 172.21.15.114:0/4208286405 wait complete. 2024-09-21T07:02:15.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 Processor -- start 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 -- start start 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc1039b0 0x7f71bc19a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71bc1092f0 0x7f71bc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc109c40 0x7f71bc19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc113900 con 0x7f71bc1039b0 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc113780 con 0x7f71bc109c40 2024-09-21T07:02:15.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.265+0000 7f71c2600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc113a80 con 0x7f71bc1092f0 2024-09-21T07:02:15.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc1039b0 0x7f71bc19a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71bc1092f0 0x7f71bc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc1039b0 0x7f71bc19a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53646/0 (socket says 172.21.15.114:53646) 2024-09-21T07:02:15.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc109c40 0x7f71bc19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1200640 1 -- 172.21.15.114:0/1299062809 learned_addr learned my addr 172.21.15.114:0/1299062809 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 -- 172.21.15.114:0/1299062809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71bc1092f0 msgr2=0x7f71bc19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71bc1092f0 0x7f71bc19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 -- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc1039b0 msgr2=0x7f71bc19a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc1039b0 0x7f71bc19a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 -- 172.21.15.114:0/1299062809 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc1a0200 con 0x7f71bc109c40 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.266+0000 7f71c1c00640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc109c40 0x7f71bc19fad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009b30 tx=0x7f71b4003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.267+0000 7f71b9600640 1 -- 172.21.15.114:0/1299062809 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4041020 con 0x7f71bc109c40 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.267+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc1a0490 con 0x7f71bc109c40 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.267+0000 7f71b9600640 1 -- 172.21.15.114:0/1299062809 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b40318f0 con 0x7f71bc109c40 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.267+0000 7f71b9600640 1 -- 172.21.15.114:0/1299062809 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4036040 con 0x7f71bc109c40 2024-09-21T07:02:15.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.267+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1a6c50 con 0x7f71bc109c40 2024-09-21T07:02:15.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.270+0000 7f71caa1f740 1 Processor -- start 2024-09-21T07:02:15.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71b9600640 1 -- 172.21.15.114:0/1299062809 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f71b4044020 con 0x7f71bc109c40 2024-09-21T07:02:15.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 -- start start 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x5644225eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x5644225f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564422566f70 con 0x564422249070 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644225563a0 con 0x5644222201b0 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.271+0000 7f71caa1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56442255a1b0 con 0x564422228390 2024-09-21T07:02:15.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x5644225eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x5644225eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53658/0 (socket says 172.21.15.114:53658) 2024-09-21T07:02:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71bbe00640 1 -- 172.21.15.114:0/1627070966 learned_addr learned my addr 172.21.15.114:0/1627070966 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71bbe00640 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x5644225f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 -- 172.21.15.114:0/1627070966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 msgr2=0x5644225f09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x5644225f09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 -- 172.21.15.114:0/1627070966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 msgr2=0x5644225eff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x5644225eff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 -- 172.21.15.114:0/1627070966 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564422520c90 con 0x5644222201b0 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.272+0000 7f71c1200640 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225f0470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009cf0 tx=0x7f71b0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3eeb7348ac8676 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.273+0000 7f71b9600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71a40b6680 0x7f71a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.273+0000 7f71ae000640 1 -- 172.21.15.114:0/1627070966 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0005220 con 0x5644222201b0 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.273+0000 7f71ae000640 1 -- 172.21.15.114:0/1627070966 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0004c50 con 0x5644222201b0 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.273+0000 7f71ae000640 1 -- 172.21.15.114:0/1627070966 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0019670 con 0x5644222201b0 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.273+0000 7f71b9600640 1 -- 172.21.15.114:0/1299062809 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f71b4108660 con 0x7f71bc109c40 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.273+0000 7f71bbe00640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71a40b6680 0x7f71a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 -- 172.21.15.114:0/1627070966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 msgr2=0x5644225f0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71bbe00640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71a40b6680 0x7f71a40b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a8015040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225f0470 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009cf0 tx=0x7f71b0011810 comp rx=0 tx=0).stop 2024-09-21T07:02:15.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 -- 172.21.15.114:0/1627070966 shutdown_connections 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x5644225f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225f0470 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1627070966 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x5644225eff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 -- 172.21.15.114:0/1627070966 >> 172.21.15.114:0/1627070966 conn(0x56442223d460 msgr2=0x5644225ede20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 -- 172.21.15.114:0/1627070966 shutdown_connections 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.274+0000 7f71caa1f740 1 -- 172.21.15.114:0/1627070966 wait complete. 2024-09-21T07:02:15.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 -- start start 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 Processor -- start 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x564422600320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x564422600900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x564422605920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5644223b7700 con 0x564422249070 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644224f1540 con 0x5644222201b0 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644225fe710 con 0x564422228390 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.275+0000 7f71caa1f740 10 client.0 _ll_register_callbacks cb 0x564422501430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x564422605920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x564422600900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x564422605920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53660/0 (socket says 172.21.15.114:53660) 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 -- 172.21.15.114:0/118356358 learned_addr learned my addr 172.21.15.114:0/118356358 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:15.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1200640 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x564422600320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 -- 172.21.15.114:0/118356358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 msgr2=0x564422600900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x564422600900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 -- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 msgr2=0x564422600320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x564422600320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 -- 172.21.15.114:0/118356358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644222b4650 con 0x564422249070 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71c1c00640 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x564422605920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034040 tx=0x7f71b4030e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb26239cbd42c19b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.276+0000 7f71a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71a2a00640 1 -- 172.21.15.114:0/118356358 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b40cd980 con 0x564422249070 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71a2a00640 1 -- 172.21.15.114:0/118356358 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f71b41083a0 con 0x564422249070 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644224f3b50 con 0x564422249070 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71a2a00640 1 -- 172.21.15.114:0/118356358 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f71b40cd7a0 con 0x564422249070 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71caa1f740 10 client.17256 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56442250e910 con 0x564422249070 2024-09-21T07:02:15.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71a2a00640 1 -- 172.21.15.114:0/118356358 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5644224f3b50 con 0x564422249070 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.277+0000 7f71a2a00640 1 -- 172.21.15.114:0/118356358 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f71b40cd290 con 0x564422249070 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71caa1f740 10 client.17256 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71caa1f740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71caa1f740 20 client.17256 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71caa1f740 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644222079f0 0x564422611b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71caa1f740 4 client.17256 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564422568ec0 con 0x5644222079f0 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71c1200640 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644222079f0 0x564422611b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71c1200640 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644222079f0 0x564422611b00 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a949ede2c41c65cf server_cookie=86ef420b6ff7d32 in_seq=0 out_seq=0 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.278+0000 7f71a2a00640 10 client.17256 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.279+0000 7f71a2a00640 1 -- 172.21.15.114:0/118356358 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564422568ec0 con 0x5644222079f0 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.279+0000 7f71a2a00640 10 client.17256 handle_command_reply: tid=0 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 client.17256 shutdown 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 msgr2=0x564422605920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x564422605920 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034040 tx=0x7f71b4030e20 comp rx=0 tx=0).stop 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 shutdown_connections 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422249070 0x564422605920 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644222079f0 0x564422611b00 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422228390 0x564422600900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 --2- 172.21.15.114:0/118356358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x564422600320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 >> 172.21.15.114:0/118356358 conn(0x56442223d460 msgr2=0x5644225f3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71a2a00640 0 client.17256 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 shutdown_connections 2024-09-21T07:02:15.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 1 -- 172.21.15.114:0/118356358 wait complete. 2024-09-21T07:02:15.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.280+0000 7f71caa1f740 20 client.17256 trim_cache size 0 max 16384 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 Processor -- start 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 -- start start 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 0x564422604e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564422249070 0x5644225f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1aeac0 con 0x564422228390 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1b0660 con 0x564422249070 2024-09-21T07:02:15.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71caa1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1af190 con 0x5644222201b0 2024-09-21T07:02:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71c1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 0x564422604e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.299+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 0x564422604e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47212/0 (socket says 172.21.15.114:47212) 2024-09-21T07:02:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 -- 172.21.15.114:0/520537352 learned_addr learned my addr 172.21.15.114:0/520537352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1c00640 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564422249070 0x5644225f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 -- 172.21.15.114:0/520537352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564422249070 msgr2=0x5644225f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564422249070 0x5644225f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 -- 172.21.15.114:0/520537352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 msgr2=0x5644225f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 -- 172.21.15.114:0/520537352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644225f4040 con 0x5644222201b0 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.300+0000 7f71c1200640 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 0x564422604e90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e59cfe77b5b5944 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.301+0000 7f71ad600640 1 -- 172.21.15.114:0/520537352 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0019430 con 0x5644222201b0 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.301+0000 7f71ad600640 1 -- 172.21.15.114:0/520537352 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0019a80 con 0x5644222201b0 2024-09-21T07:02:15.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.301+0000 7f71ad600640 1 -- 172.21.15.114:0/520537352 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00237f0 con 0x5644222201b0 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.301+0000 7f71caa1f740 1 -- 172.21.15.114:0/520537352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 msgr2=0x564422604e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.301+0000 7f71caa1f740 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 0x564422604e90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fd0 tx=0x7f71b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 -- 172.21.15.114:0/520537352 shutdown_connections 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564422249070 0x5644225f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 --2- 172.21.15.114:0/520537352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644222201b0 0x564422604e90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 -- 172.21.15.114:0/520537352 >> 172.21.15.114:0/520537352 conn(0x56442223d460 msgr2=0x564422604780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 -- 172.21.15.114:0/520537352 shutdown_connections 2024-09-21T07:02:15.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 -- 172.21.15.114:0/520537352 wait complete. 2024-09-21T07:02:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 -- start start 2024-09-21T07:02:15.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.302+0000 7f71caa1f740 1 Processor -- start 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422249070 0x564422608960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1aff90 con 0x564422228390 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1af860 con 0x5644222201b0 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1acac0 con 0x564422249070 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71caa1f740 10 client.0 _ll_register_callbacks cb 0x7f71b40ca4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71c1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71c1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225feb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34708/0 (socket says 172.21.15.114:34708) 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71c1200640 1 -- 172.21.15.114:0/1609840153 learned_addr learned my addr 172.21.15.114:0/1609840153 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.303+0000 7f71c1c00640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422249070 0x564422608960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71c1200640 1 -- 172.21.15.114:0/1609840153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422249070 msgr2=0x564422608960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71c1200640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422249070 0x564422608960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71c1200640 1 -- 172.21.15.114:0/1609840153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 msgr2=0x5644225ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71c1200640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71c1200640 1 -- 172.21.15.114:0/1609840153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564422608f20 con 0x5644222201b0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71bbe00640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71c1200640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225feb50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fa0 tx=0x7f71b00053b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445e1c756bc5658b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71a2a00640 1 -- 172.21.15.114:0/1609840153 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0019e30 con 0x5644222201b0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b0019a80 con 0x5644222201b0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.304+0000 7f71a2a00640 1 -- 172.21.15.114:0/1609840153 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f71b0019430 con 0x5644222201b0 2024-09-21T07:02:15.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71a2a00640 1 -- 172.21.15.114:0/1609840153 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f71b004a050 con 0x5644222201b0 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 10 client.27656 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56442267ccf0 con 0x5644222201b0 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71a2a00640 1 -- 172.21.15.114:0/1609840153 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b0019a80 con 0x5644222201b0 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71a2a00640 1 -- 172.21.15.114:0/1609840153 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f71b0037bd0 con 0x5644222201b0 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 10 client.27656 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 10 client.27656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 20 client.27656 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644221dcda0 0x56442267f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 4 client.27656 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564422564970 con 0x5644221dcda0 2024-09-21T07:02:15.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.305+0000 7f71bbe00640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644221dcda0 0x56442267f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:15.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71bbe00640 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644221dcda0 0x56442267f230 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef577d34f49b54c server_cookie=dbaac7fd63b7f997 in_seq=0 out_seq=0 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71a2a00640 10 client.27656 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71a2a00640 1 -- 172.21.15.114:0/1609840153 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x564422564970 con 0x5644221dcda0 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71a2a00640 10 client.27656 handle_command_reply: tid=0 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 client.27656 shutdown 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 msgr2=0x5644225feb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225feb50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f71b0009fa0 tx=0x7f71b00053b0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 shutdown_connections 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644221dcda0 0x56442267f230 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564422249070 0x564422608960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.307+0000 7f71a2a00640 0 client.27656 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.306+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564422228390 0x5644225ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.307+0000 7f71caa1f740 1 --2- 172.21.15.114:0/1609840153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644222201b0 0x5644225feb50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.307+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 >> 172.21.15.114:0/1609840153 conn(0x56442223d460 msgr2=0x564422604780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:15.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.307+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 shutdown_connections 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.307+0000 7f71caa1f740 1 -- 172.21.15.114:0/1609840153 wait complete. 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.307+0000 7f71caa1f740 20 client.27656 trim_cache size 0 max 16384 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:02:15.312 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506", 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.313 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.314 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.315 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.316 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:15.317 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71a40b6680 msgr2=0x7f71a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71a40b6680 0x7f71a40b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f71a8005fd0 tx=0x7f71a8015040 comp rx=0 tx=0).stop 2024-09-21T07:02:15.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc109c40 msgr2=0x7f71bc19fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc109c40 0x7f71bc19fad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009b30 tx=0x7f71b4003b80 comp rx=0 tx=0).stop 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 shutdown_connections 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71a40b6680 0x7f71a40b8b90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71bc109c40 0x7f71bc19fad0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71bc1092f0 0x7f71bc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 --2- 172.21.15.114:0/1299062809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f71bc1039b0 0x7f71bc19a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 >> 172.21.15.114:0/1299062809 conn(0x7f71bc0ff400 msgr2=0x7f71bc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 shutdown_connections 2024-09-21T07:02:15.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:15.308+0000 7f71c2600640 1 -- 172.21.15.114:0/1299062809 wait complete. 2024-09-21T07:02:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:15 smithi114 ceph-mon[30654]: pgmap v642: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 992 KiB/s wr, 98 op/s 2024-09-21T07:02:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:15 smithi183 ceph-mon[34219]: pgmap v642: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 992 KiB/s wr, 98 op/s 2024-09-21T07:02:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:15 smithi193 ceph-mon[34601]: pgmap v642: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 992 KiB/s wr, 98 op/s 2024-09-21T07:02:16.008 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:02:16.156 INFO:tasks.cephfs.filesystem:scrub status for tag:4b612c18-5d84-41a3-bbbb-9a7ba6a4d506 - {'path': '/', 'tag': '4b612c18-5d84-41a3-bbbb-9a7ba6a4d506', 'options': 'recursive,force'} 2024-09-21T07:02:16.381 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 -- 172.21.15.114:0/2650450697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 msgr2=0x7f5038103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 --2- 172.21.15.114:0/2650450697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038103f70 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f502802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 -- 172.21.15.114:0/2650450697 shutdown_connections 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 --2- 172.21.15.114:0/2650450697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5038109df0 0x7f5038110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 --2- 172.21.15.114:0/2650450697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50381094a0 0x7f50381098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 --2- 172.21.15.114:0/2650450697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038103f70 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 -- 172.21.15.114:0/2650450697 >> 172.21.15.114:0/2650450697 conn(0x7f503807b850 msgr2=0x7f503807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:17.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 -- 172.21.15.114:0/2650450697 shutdown_connections 2024-09-21T07:02:17.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.034+0000 7f503cc00640 1 -- 172.21.15.114:0/2650450697 wait complete. 2024-09-21T07:02:17.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.035+0000 7f503cc00640 1 Processor -- start 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.035+0000 7f503cc00640 1 -- start start 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f503cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f503cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50381094a0 0x7f5038075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f503cc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5038109df0 0x7f5038076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f503cc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5038113ab0 con 0x7f5038103b60 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f503cc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5038113930 con 0x7f50381094a0 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f503cc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5038113c30 con 0x7f5038109df0 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53698/0 (socket says 172.21.15.114:53698) 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 -- 172.21.15.114:0/3028329930 learned_addr learned my addr 172.21.15.114:0/3028329930 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5036800640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5038109df0 0x7f5038076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:17.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f502d400640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50381094a0 0x7f5038075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:17.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 -- 172.21.15.114:0/3028329930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5038109df0 msgr2=0x7f5038076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:17.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5038109df0 0x7f5038076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 -- 172.21.15.114:0/3028329930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50381094a0 msgr2=0x7f5038075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50381094a0 0x7f5038075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.036+0000 7f5035e00640 1 -- 172.21.15.114:0/3028329930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50381a87a0 con 0x7f5038103b60 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.037+0000 7f5035e00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038075710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5028005940 tx=0x7f5028004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.037+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502803f030 con 0x7f5038103b60 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.037+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5028003aa0 con 0x7f5038103b60 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.037+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50381a8a30 con 0x7f5038103b60 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.037+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028035070 con 0x7f5038103b60 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.037+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a8f10 con 0x7f5038103b60 2024-09-21T07:02:17.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.038+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5038103f70 con 0x7f5038103b60 2024-09-21T07:02:17.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.042+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5028042030 con 0x7f5038103b60 2024-09-21T07:02:17.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.044+0000 7f502f400640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f501c0b6680 0x7f501c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:17.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.044+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f50281084b0 con 0x7f5038103b60 2024-09-21T07:02:17.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.044+0000 7f502d400640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f501c0b6680 0x7f501c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:17.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.045+0000 7f502d400640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f501c0b6680 0x7f501c0b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50381090b0 tx=0x7f502000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:17.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.046+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50280d29c0 con 0x7f5038103b60 2024-09-21T07:02:17.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.273+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5038076a10 con 0x7f5038103b60 2024-09-21T07:02:17.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.274+0000 7f502f400640 1 -- 172.21.15.114:0/3028329930 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11992 (secure 0 0 0) 0x7f5038076a10 con 0x7f5038103b60 2024-09-21T07:02:17.276 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:02:17.276 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"btime":"2024-09-21T07:02:02:992961+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:02:02.920775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:02:17.276 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-09-21T07:02:17.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.276+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f501c0b6680 msgr2=0x7f501c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:17.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.276+0000 7f503cc00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f501c0b6680 0x7f501c0b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f50381090b0 tx=0x7f502000b390 comp rx=0 tx=0).stop 2024-09-21T07:02:17.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 msgr2=0x7f5038075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:17.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038075710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5028005940 tx=0x7f5028004270 comp rx=0 tx=0).stop 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 shutdown_connections 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5038109df0 0x7f5038076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f501c0b6680 0x7f501c0b8b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50381094a0 0x7f5038075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 --2- 172.21.15.114:0/3028329930 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5038103b60 0x7f5038075710 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 >> 172.21.15.114:0/3028329930 conn(0x7f503807b850 msgr2=0x7f5038107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 shutdown_connections 2024-09-21T07:02:17.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:17.277+0000 7f503cc00640 1 -- 172.21.15.114:0/3028329930 wait complete. 2024-09-21T07:02:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:17 smithi114 ceph-mon[30654]: pgmap v643: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 426 KiB/s wr, 80 op/s 2024-09-21T07:02:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:17 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3028329930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:17 smithi183 ceph-mon[34219]: pgmap v643: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 426 KiB/s wr, 80 op/s 2024-09-21T07:02:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:17 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3028329930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:17 smithi193 ceph-mon[34601]: pgmap v643: 129 pgs: 5 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 102 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 426 KiB/s wr, 80 op/s 2024-09-21T07:02:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:17 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3028329930' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 460.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:02:18.021 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id c14c5ce --release --await' 2024-09-21T07:02:18.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id c14c5ce --release --await 2024-09-21T07:02:18.393 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:19.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 -- 172.21.15.114:0/4132038630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc1049d0 msgr2=0x7f96dc104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 --2- 172.21.15.114:0/4132038630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc1049d0 0x7f96dc104de0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009900 tx=0x7f96d002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 -- 172.21.15.114:0/4132038630 shutdown_connections 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 --2- 172.21.15.114:0/4132038630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc10de30 0x7f96dc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 --2- 172.21.15.114:0/4132038630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc105320 0x7f96dc10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 --2- 172.21.15.114:0/4132038630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc1049d0 0x7f96dc104de0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.049+0000 7f96e3c00640 1 -- 172.21.15.114:0/4132038630 >> 172.21.15.114:0/4132038630 conn(0x7f96dc079740 msgr2=0x7f96dc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.050+0000 7f96e3c00640 1 -- 172.21.15.114:0/4132038630 shutdown_connections 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.050+0000 7f96e3c00640 1 -- 172.21.15.114:0/4132038630 wait complete. 2024-09-21T07:02:19.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.051+0000 7f96e3c00640 1 Processor -- start 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.051+0000 7f96e3c00640 1 -- start start 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e3c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc1049d0 0x7f96dc102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e3c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc105320 0x7f96dc102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e3c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc10de30 0x7f96dc103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e3c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc113ab0 con 0x7f96dc105320 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e3c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc113930 con 0x7f96dc1049d0 2024-09-21T07:02:19.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e3c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc113c30 con 0x7f96dc10de30 2024-09-21T07:02:19.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc1049d0 0x7f96dc102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc10de30 0x7f96dc103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc1049d0 0x7f96dc102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34752/0 (socket says 172.21.15.114:34752) 2024-09-21T07:02:19.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96e1200640 1 -- 172.21.15.114:0/164621214 learned_addr learned my addr 172.21.15.114:0/164621214 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:19.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96dbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc105320 0x7f96dc102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96dbe00640 1 -- 172.21.15.114:0/164621214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc10de30 msgr2=0x7f96dc103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96dbe00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc10de30 0x7f96dc103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96dbe00640 1 -- 172.21.15.114:0/164621214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc1049d0 msgr2=0x7f96dc102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96dbe00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc1049d0 0x7f96dc102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.052+0000 7f96dbe00640 1 -- 172.21.15.114:0/164621214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1a87a0 con 0x7f96dc105320 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96e1c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc10de30 0x7f96dc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:19.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96dbe00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc105320 0x7f96dc102c50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f96dc06b950 tx=0x7f96c800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:19.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96d9600640 1 -- 172.21.15.114:0/164621214 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c800cc40 con 0x7f96dc105320 2024-09-21T07:02:19.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc1a8a90 con 0x7f96dc105320 2024-09-21T07:02:19.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96d9600640 1 -- 172.21.15.114:0/164621214 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96c800cde0 con 0x7f96dc105320 2024-09-21T07:02:19.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96d9600640 1 -- 172.21.15.114:0/164621214 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8004a00 con 0x7f96dc105320 2024-09-21T07:02:19.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.053+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1a8fd0 con 0x7f96dc105320 2024-09-21T07:02:19.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.056+0000 7f96eaa0f740 1 Processor -- start 2024-09-21T07:02:19.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.056+0000 7f96eaa0f740 1 -- start start 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608a9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608a9b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608a9bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561608a12290 con 0x5616086cb660 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561608a016c0 con 0x5616086d3840 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561608a054d0 con 0x5616086f4520 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96d9600640 1 -- 172.21.15.114:0/164621214 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f96c8004ba0 con 0x7f96dc105320 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608a9b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96e1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608a9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608a9b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53722/0 (socket says 172.21.15.114:53722) 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96e1200640 1 -- 172.21.15.114:0/4041971204 learned_addr learned my addr 172.21.15.114:0/4041971204 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:19.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.057+0000 7f96e1c00640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608a9b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47268/0 (socket says 172.21.15.114:47268) 2024-09-21T07:02:19.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1c00640 1 -- 172.21.15.114:0/4041971204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 msgr2=0x561608a9bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96dbe00640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608a9bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1c00640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608a9bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1c00640 1 -- 172.21.15.114:0/4041971204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 msgr2=0x561608a9b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1c00640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608a9b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1c00640 1 -- 172.21.15.114:0/4041971204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56160898e460 con 0x5616086f4520 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96dbe00640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608a9bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1200640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608a9b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96e1c00640 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608a9b1c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005fd0 tx=0x7f96d4011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a95c64fd3c8ad3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96ce000640 1 -- 172.21.15.114:0/4041971204 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4004d20 con 0x5616086f4520 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.058+0000 7f96ce000640 1 -- 172.21.15.114:0/4041971204 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d400f9a0 con 0x5616086f4520 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.059+0000 7f96ce000640 1 -- 172.21.15.114:0/4041971204 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4004410 con 0x5616086f4520 2024-09-21T07:02:19.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.059+0000 7f96d9600640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96c40b6700 0x7f96c40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.059+0000 7f96d9600640 1 -- 172.21.15.114:0/164621214 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f96c80e7a60 con 0x7f96dc105320 2024-09-21T07:02:19.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.059+0000 7f96eaa0f740 1 -- 172.21.15.114:0/4041971204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 msgr2=0x561608a9b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.059+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608a9b1c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005fd0 tx=0x7f96d4011bf0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.059+0000 7f96e1200640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96c40b6700 0x7f96c40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 -- 172.21.15.114:0/4041971204 shutdown_connections 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608a9bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608a9b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/4041971204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608a9b1c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 -- 172.21.15.114:0/4041971204 >> 172.21.15.114:0/4041971204 conn(0x5616086e8910 msgr2=0x561608a990b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 -- 172.21.15.114:0/4041971204 shutdown_connections 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 -- 172.21.15.114:0/4041971204 wait complete. 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96e1200640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96c40b6700 0x7f96c40b8c10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x561608a99e40 tx=0x7f96d0005890 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.060+0000 7f96eaa0f740 1 -- start start 2024-09-21T07:02:19.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 Processor -- start 2024-09-21T07:02:19.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608ab2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x561608862bb0 con 0x5616086f4520 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56160899c6c0 con 0x5616086d3840 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561608aa99a0 con 0x5616086cb660 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96eaa0f740 10 client.0 _ll_register_callbacks cb 0x5616089b7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608ab2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608ab2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53728/0 (socket says 172.21.15.114:53728) 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96e1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.061+0000 7f96e1c00640 1 -- 172.21.15.114:0/3414086337 learned_addr learned my addr 172.21.15.114:0/3414086337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96dbe00640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1c00640 1 -- 172.21.15.114:0/3414086337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 msgr2=0x561608aa6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1c00640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1c00640 1 -- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 msgr2=0x561608aa7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1c00640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1c00640 1 -- 172.21.15.114:0/3414086337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56160875fb00 con 0x5616086f4520 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1200640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96e1c00640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608ab2c90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f96d4004090 tx=0x7f96d4004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b416dad5c16acb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.062+0000 7f96c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96c2a00640 1 -- 172.21.15.114:0/3414086337 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4028020 con 0x5616086f4520 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96c2a00640 1 -- 172.21.15.114:0/3414086337 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f96d40041f0 con 0x5616086f4520 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d400f9a0 con 0x5616086f4520 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96c2a00640 1 -- 172.21.15.114:0/3414086337 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f96d4033050 con 0x5616086f4520 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96eaa0f740 10 client.17280 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:19.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5616087d6960 con 0x5616086f4520 2024-09-21T07:02:19.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96c2a00640 1 -- 172.21.15.114:0/3414086337 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d400f9a0 con 0x5616086f4520 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96c2a00640 1 -- 172.21.15.114:0/3414086337 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f96d4018070 con 0x5616086f4520 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.063+0000 7f96eaa0f740 10 client.17280 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96eaa0f740 10 client.17280 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96eaa0f740 20 client.17280 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616086b2ea0 0x561608ab8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96eaa0f740 4 client.17280 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561608a141e0 con 0x5616086b2ea0 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96e1200640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616086b2ea0 0x561608ab8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.065+0000 7f96c2a00640 10 client.17280 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.064+0000 7f96e1200640 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616086b2ea0 0x561608ab8b90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d8dcf0ddbf8ac7 server_cookie=c935e74ebef56712 in_seq=0 out_seq=0 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.065+0000 7f96c2a00640 1 -- 172.21.15.114:0/3414086337 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561608a141e0 con 0x5616086b2ea0 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.065+0000 7f96c2a00640 10 client.17280 handle_command_reply: tid=0 2024-09-21T07:02:19.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 client.17280 shutdown 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 msgr2=0x561608ab2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608ab2c90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f96d4004090 tx=0x7f96d4004d60 comp rx=0 tx=0).stop 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 shutdown_connections 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608ab2c90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616086b2ea0 0x561608ab8b90 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3414086337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 >> 172.21.15.114:0/3414086337 conn(0x5616086e8910 msgr2=0x561608a9e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96c2a00640 0 client.17280 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 shutdown_connections 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3414086337 wait complete. 2024-09-21T07:02:19.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.066+0000 7f96eaa0f740 20 client.17280 trim_cache size 0 max 16384 2024-09-21T07:02:19.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 Processor -- start 2024-09-21T07:02:19.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 -- start start 2024-09-21T07:02:19.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608aa3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b3290 con 0x5616086f4520 2024-09-21T07:02:19.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b4e30 con 0x5616086d3840 2024-09-21T07:02:19.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b3960 con 0x5616086cb660 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608aa3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608aa3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:53738/0 (socket says 172.21.15.114:53738) 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96e1c00640 1 -- 172.21.15.114:0/3394872377 learned_addr learned my addr 172.21.15.114:0/3394872377 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96e1200640 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.084+0000 7f96dbe00640 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96e1c00640 1 -- 172.21.15.114:0/3394872377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 msgr2=0x561608aa12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96e1c00640 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96e1c00640 1 -- 172.21.15.114:0/3394872377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 msgr2=0x561608aa8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96e1c00640 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96e1c00640 1 -- 172.21.15.114:0/3394872377 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561608a973b0 con 0x5616086f4520 2024-09-21T07:02:19.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96e1c00640 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608aa3d30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005720 tx=0x7f96d4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d774aa4f7ae356f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:19.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96cd600640 1 -- 172.21.15.114:0/3394872377 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4028040 con 0x5616086f4520 2024-09-21T07:02:19.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96cd600640 1 -- 172.21.15.114:0/3394872377 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d401f040 con 0x5616086f4520 2024-09-21T07:02:19.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.085+0000 7f96cd600640 1 -- 172.21.15.114:0/3394872377 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4003cc0 con 0x5616086f4520 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3394872377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 msgr2=0x561608aa3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608aa3d30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005720 tx=0x7f96d4005d30 comp rx=0 tx=0).stop 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3394872377 shutdown_connections 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086f4520 0x561608aa3d30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aa8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/3394872377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086cb660 0x561608aa12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3394872377 >> 172.21.15.114:0/3394872377 conn(0x5616086e8910 msgr2=0x561608a9c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3394872377 shutdown_connections 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.086+0000 7f96eaa0f740 1 -- 172.21.15.114:0/3394872377 wait complete. 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.087+0000 7f96eaa0f740 1 -- start start 2024-09-21T07:02:19.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.087+0000 7f96eaa0f740 1 Processor -- start 2024-09-21T07:02:19.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608aab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aab990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608b22470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b4760 con 0x5616086cb660 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b4030 con 0x5616086d3840 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b1290 con 0x5616086f4520 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96eaa0f740 10 client.0 _ll_register_callbacks cb 0x7f96c8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608aab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aab990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34780/0 (socket says 172.21.15.114:34780) 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 -- 172.21.15.114:0/1932785705 learned_addr learned my addr 172.21.15.114:0/1932785705 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96e1c00640 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608b22470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 -- 172.21.15.114:0/1932785705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 msgr2=0x561608b22470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608b22470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 -- 172.21.15.114:0/1932785705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 msgr2=0x561608aab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608aab3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.088+0000 7f96dbe00640 1 -- 172.21.15.114:0/1932785705 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561608b22ab0 con 0x5616086d3840 2024-09-21T07:02:19.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96dbe00640 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aab990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96c80e5390 tx=0x7f96c8005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df09ffee16de7bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96c2a00640 1 -- 172.21.15.114:0/1932785705 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c801c040 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96c2a00640 1 -- 172.21.15.114:0/1932785705 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9578+0+0 (secure 0 0 0) 0x7f96c8094bd0 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d401f040 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96c2a00640 1 -- 172.21.15.114:0/1932785705 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f96c80acc00 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96eaa0f740 10 client.27677 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561608b23dd0 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.089+0000 7f96c2a00640 1 -- 172.21.15.114:0/1932785705 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d401f040 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96c2a00640 1 -- 172.21.15.114:0/1932785705 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f96c80b6040 con 0x5616086d3840 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96eaa0f740 10 client.27677 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96eaa0f740 10 client.27677 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96eaa0f740 20 client.27677 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561608688250 0x561608b26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96eaa0f740 4 client.27677 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c14c5ce", "release": true, "await": true}] 2024-09-21T07:02:19.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c14c5ce", "release": true, "await": true}) -- 0x561608a0fc90 con 0x561608688250 2024-09-21T07:02:19.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.090+0000 7f96e1200640 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561608688250 0x561608b26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:19.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.091+0000 7f96e1200640 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561608688250 0x561608b26590 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c1aca303561ad2 server_cookie=4ad6d80b12a9678b in_seq=0 out_seq=0 2024-09-21T07:02:19.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.091+0000 7f96c2a00640 10 client.27677 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:19.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96c2a00640 1 -- 172.21.15.114:0/1932785705 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561608a0fc90 con 0x561608688250 2024-09-21T07:02:19.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96c2a00640 10 client.27677 handle_command_reply: tid=0 2024-09-21T07:02:19.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96eaa0f740 1 client.27677 shutdown 2024-09-21T07:02:19.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 msgr2=0x561608aab990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aab990 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f96c80e5390 tx=0x7f96c8005230 comp rx=0 tx=0).stop 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 shutdown_connections 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.305+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x561608688250 0x561608b26590 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616086f4520 0x561608b22470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616086d3840 0x561608aab990 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 1 --2- 172.21.15.114:0/1932785705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616086cb660 0x561608aab3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96c2a00640 0 client.27677 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 >> 172.21.15.114:0/1932785705 conn(0x5616086e8910 msgr2=0x561608aa7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:19.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 shutdown_connections 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 1 -- 172.21.15.114:0/1932785705 wait complete. 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96eaa0f740 20 client.27677 trim_cache size 0 max 16384 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 47, 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 32, 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "c14c5ce": { 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 32, 2024-09-21T07:02:19.309 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 141.1, 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:19.310 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.306+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96c40b6700 msgr2=0x7f96c40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96c40b6700 0x7f96c40b8c10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x561608a99e40 tx=0x7f96d0005890 comp rx=0 tx=0).stop 2024-09-21T07:02:19.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc105320 msgr2=0x7f96dc102c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc105320 0x7f96dc102c50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f96dc06b950 tx=0x7f96c800ee50 comp rx=0 tx=0).stop 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 shutdown_connections 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96dc10de30 0x7f96dc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f96dc105320 0x7f96dc102c50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f96c40b6700 0x7f96c40b8c10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 --2- 172.21.15.114:0/164621214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96dc1049d0 0x7f96dc102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 >> 172.21.15.114:0/164621214 conn(0x7f96dc079740 msgr2=0x7f96dc1007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 shutdown_connections 2024-09-21T07:02:19.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:19.307+0000 7f96e3c00640 1 -- 172.21.15.114:0/164621214 wait complete. 2024-09-21T07:02:19.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:19 smithi114 ceph-mon[30654]: pgmap v644: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 35 KiB/s wr, 111 op/s 2024-09-21T07:02:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:19 smithi183 ceph-mon[34219]: pgmap v644: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 35 KiB/s wr, 111 op/s 2024-09-21T07:02:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:19 smithi193 ceph-mon[34601]: pgmap v644: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 35 KiB/s wr, 111 op/s 2024-09-21T07:02:20.235 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c14c5ce', total seconds elapsed: 25.3 2024-09-21T07:02:20.655 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-09-21T07:02:20.655 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:02:21.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:20 smithi114 ceph-mon[30654]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T07:02:21.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:20 smithi114 ceph-mon[30654]: pgmap v646: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 37 KiB/s wr, 133 op/s 2024-09-21T07:02:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:20 smithi183 ceph-mon[34219]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T07:02:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:20 smithi183 ceph-mon[34219]: pgmap v646: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 37 KiB/s wr, 133 op/s 2024-09-21T07:02:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:20 smithi193 ceph-mon[34601]: osdmap e93: 12 total, 12 up, 12 in 2024-09-21T07:02:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:20 smithi193 ceph-mon[34601]: pgmap v646: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 37 KiB/s wr, 133 op/s 2024-09-21T07:02:21.388 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:22.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 -- 172.21.15.114:0/1744386398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81092f0 msgr2=0x7f14c8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:22.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 --2- 172.21.15.114:0/1744386398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81092f0 0x7f14c8109700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009900 tx=0x7f14b402a440 comp rx=0 tx=0).stop 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 -- 172.21.15.114:0/1744386398 shutdown_connections 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 --2- 172.21.15.114:0/1744386398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c8109c40 0x7f14c8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 --2- 172.21.15.114:0/1744386398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81092f0 0x7f14c8109700 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 --2- 172.21.15.114:0/1744386398 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c81039b0 0x7f14c8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 -- 172.21.15.114:0/1744386398 >> 172.21.15.114:0/1744386398 conn(0x7f14c80ff400 msgr2=0x7f14c8101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 -- 172.21.15.114:0/1744386398 shutdown_connections 2024-09-21T07:02:22.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.245+0000 7f14ce600640 1 -- 172.21.15.114:0/1744386398 wait complete. 2024-09-21T07:02:22.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.247+0000 7f14ce600640 1 Processor -- start 2024-09-21T07:02:22.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.252+0000 7f14ce600640 1 -- start start 2024-09-21T07:02:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14ce600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81039b0 0x7f14c8074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14ce600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 0x7f14c806e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:22.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14ce600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c8109c40 0x7f14c806e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14cd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81039b0 0x7f14c8074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 0x7f14c806e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14cd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81039b0 0x7f14c8074e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47314/0 (socket says 172.21.15.114:47314) 2024-09-21T07:02:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14c7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 0x7f14c806e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34800/0 (socket says 172.21.15.114:34800) 2024-09-21T07:02:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14cd200640 1 -- 172.21.15.114:0/2204622821 learned_addr learned my addr 172.21.15.114:0/2204622821 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:22.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8113900 con 0x7f14c8109c40 2024-09-21T07:02:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.253+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8113780 con 0x7f14c81092f0 2024-09-21T07:02:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.254+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8113a80 con 0x7f14c81039b0 2024-09-21T07:02:22.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.254+0000 7f14cdc00640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c8109c40 0x7f14c806e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.254+0000 7f14c7e00640 1 -- 172.21.15.114:0/2204622821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81039b0 msgr2=0x7f14c8074e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.254+0000 7f14c7e00640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81039b0 0x7f14c8074e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.254+0000 7f14c7e00640 1 -- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c8109c40 msgr2=0x7f14c806e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:22.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.254+0000 7f14c7e00640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c8109c40 0x7f14c806e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.255+0000 7f14c7e00640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c806ef10 con 0x7f14c81092f0 2024-09-21T07:02:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.255+0000 7f14cdc00640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c8109c40 0x7f14c806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:22.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.257+0000 7f14c7e00640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 0x7f14c806e150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f14b402a960 tx=0x7f14b40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.258+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4035070 con 0x7f14c81092f0 2024-09-21T07:02:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.258+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c8073b50 con 0x7f14c81092f0 2024-09-21T07:02:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.258+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b40044b0 con 0x7f14c81092f0 2024-09-21T07:02:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.258+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4003d60 con 0x7f14c81092f0 2024-09-21T07:02:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.258+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c8074050 con 0x7f14c81092f0 2024-09-21T07:02:22.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.260+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c8103dc0 con 0x7f14c81092f0 2024-09-21T07:02:22.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.264+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f14b403d760 con 0x7f14c81092f0 2024-09-21T07:02:22.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.269+0000 7f14c5600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14ac0b6680 0x7f14ac0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:22.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.269+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f14b410ca70 con 0x7f14c81092f0 2024-09-21T07:02:22.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.269+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14b413d770 con 0x7f14c81092f0 2024-09-21T07:02:22.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.270+0000 7f14cd200640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14ac0b6680 0x7f14ac0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:22.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.272+0000 7f14cd200640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14ac0b6680 0x7f14ac0b8b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009f30 tx=0x7f14c00094c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.762+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14c810b480 con 0x7f14c81092f0 2024-09-21T07:02:22.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.763+0000 7f14c5600640 1 -- 172.21.15.114:0/2204622821 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11992 (secure 0 0 0) 0x7f14b40d2bf0 con 0x7f14c81092f0 2024-09-21T07:02:22.765 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:02:22.766 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-09-21T07:02:22.766 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"btime":"2024-09-21T07:02:02:992961+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:02:02.920775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:02:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.767+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14ac0b6680 msgr2=0x7f14ac0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.767+0000 7f14ce600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14ac0b6680 0x7f14ac0b8b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009f30 tx=0x7f14c00094c0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.767+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 msgr2=0x7f14c806e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:22.769 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.767+0000 7f14ce600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 0x7f14c806e150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f14b402a960 tx=0x7f14b40040c0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 shutdown_connections 2024-09-21T07:02:22.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f14c8109c40 0x7f14c806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f14ac0b6680 0x7f14ac0b8b90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14c81092f0 0x7f14c806e150 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 --2- 172.21.15.114:0/2204622821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14c81039b0 0x7f14c8074e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 >> 172.21.15.114:0/2204622821 conn(0x7f14c80ff400 msgr2=0x7f14c8101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.772+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 shutdown_connections 2024-09-21T07:02:22.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:22.773+0000 7f14ce600640 1 -- 172.21.15.114:0/2204622821 wait complete. 2024-09-21T07:02:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 465.0 s, 459.1 tps, lat 867.243 ms stddev 3468.143 2024-09-21T07:02:23.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:22 smithi114 ceph-mon[30654]: pgmap v647: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 697 KiB/s wr, 157 op/s 2024-09-21T07:02:23.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:22 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2204622821' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:22 smithi183 ceph-mon[34219]: pgmap v647: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 697 KiB/s wr, 157 op/s 2024-09-21T07:02:23.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:22 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2204622821' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:22 smithi193 ceph-mon[34601]: pgmap v647: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 697 KiB/s wr, 157 op/s 2024-09-21T07:02:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:22 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2204622821' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:24.213 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0' 2024-09-21T07:02:24.213 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 141.05 --await-for 45.0 2024-09-21T07:02:25.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:24 smithi183 ceph-mon[34219]: pgmap v648: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-09-21T07:02:25.161 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:24 smithi193 ceph-mon[34601]: pgmap v648: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-09-21T07:02:25.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:24 smithi114 ceph-mon[30654]: pgmap v648: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.7 MiB/s wr, 250 op/s 2024-09-21T07:02:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:26.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:25 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:02:26.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:25 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:02:26.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:25 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 -- 172.21.15.114:0/1157081840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08103910 msgr2=0x7f1c08103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 --2- 172.21.15.114:0/1157081840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08103910 0x7f1c08103d20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009860 tx=0x7f1bf802a340 comp rx=0 tx=0).stop 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 -- 172.21.15.114:0/1157081840 shutdown_connections 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 --2- 172.21.15.114:0/1157081840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c08109b90 0x7f1c081103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 --2- 172.21.15.114:0/1157081840 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c08109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 --2- 172.21.15.114:0/1157081840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08103910 0x7f1c08103d20 secure :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009860 tx=0x7f1bf802a340 comp rx=0 tx=0).stop 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 -- 172.21.15.114:0/1157081840 >> 172.21.15.114:0/1157081840 conn(0x7f1c080ff330 msgr2=0x7f1c08101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.508+0000 7f1c07200640 1 -- 172.21.15.114:0/1157081840 shutdown_connections 2024-09-21T07:02:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.509+0000 7f1c07200640 1 -- 172.21.15.114:0/1157081840 wait complete. 2024-09-21T07:02:26.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.510+0000 7f1c07200640 1 Processor -- start 2024-09-21T07:02:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.511+0000 7f1c07200640 1 -- start start 2024-09-21T07:02:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.512+0000 7f1c07200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c0810dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.512+0000 7f1c07200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08109b90 0x7f1c0810e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.512+0000 7f1c07200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c0810e6a0 0x7f1c081ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.512+0000 7f1c07200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08077160 con 0x7f1c08109240 2024-09-21T07:02:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.512+0000 7f1c07200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08076fe0 con 0x7f1c08109b90 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.512+0000 7f1c07200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080772e0 con 0x7f1c0810e6a0 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.513+0000 7f1c05e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c0810dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.513+0000 7f1c05e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c0810dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35840/0 (socket says 172.21.15.114:35840) 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.513+0000 7f1c05e00640 1 -- 172.21.15.114:0/3739188386 learned_addr learned my addr 172.21.15.114:0/3739188386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.514+0000 7f1c05e00640 1 -- 172.21.15.114:0/3739188386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c0810e6a0 msgr2=0x7f1c081ac720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.514+0000 7f1c05e00640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c0810e6a0 0x7f1c081ac720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.514+0000 7f1c05e00640 1 -- 172.21.15.114:0/3739188386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08109b90 msgr2=0x7f1c0810e020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.514+0000 7f1c05e00640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08109b90 0x7f1c0810e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.514+0000 7f1c05e00640 1 -- 172.21.15.114:0/3739188386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c081acf40 con 0x7f1c08109240 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.515+0000 7f1c05e00640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c0810dae0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80057c0 tx=0x7f1bf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.515+0000 7f1bfea00640 1 -- 172.21.15.114:0/3739188386 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8040020 con 0x7f1c08109240 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.515+0000 7f1bfea00640 1 -- 172.21.15.114:0/3739188386 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf802ada0 con 0x7f1c08109240 2024-09-21T07:02:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.516+0000 7f1bfea00640 1 -- 172.21.15.114:0/3739188386 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8037040 con 0x7f1c08109240 2024-09-21T07:02:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.518+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081ad1d0 con 0x7f1c08109240 2024-09-21T07:02:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.518+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c081ad670 con 0x7f1c08109240 2024-09-21T07:02:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.528+0000 7f1c14203740 1 Processor -- start 2024-09-21T07:02:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.528+0000 7f1bfea00640 1 -- 172.21.15.114:0/3739188386 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1bf8043020 con 0x7f1c08109240 2024-09-21T07:02:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.528+0000 7f1c14203740 1 -- start start 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c14203740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c14203740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c14203740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c14203740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e52d6396b0 con 0x55e52d2f62f0 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c14203740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e52d628ae0 con 0x55e52d31f1b0 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c14203740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e52d62c8f0 con 0x55e52d2fe4d0 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46740/0 (socket says 172.21.15.114:46740) 2024-09-21T07:02:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.529+0000 7f1c05400640 1 -- 172.21.15.114:0/3631488754 learned_addr learned my addr 172.21.15.114:0/3631488754 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.530+0000 7f1c06800640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.530+0000 7f1c05e00640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.530+0000 7f1c05400640 1 -- 172.21.15.114:0/3631488754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 msgr2=0x55e52d6c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1c05400640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1c05400640 1 -- 172.21.15.114:0/3631488754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 msgr2=0x55e52d6c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1c05400640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1c05400640 1 -- 172.21.15.114:0/3631488754 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52d5f5780 con 0x55e52d2fe4d0 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1bfea00640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1bec0b6780 0x7f1bec0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1c05e00640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1bfea00640 1 -- 172.21.15.114:0/3739188386 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1bf8107200 con 0x7f1c08109240 2024-09-21T07:02:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.531+0000 7f1c06800640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.532+0000 7f1c06800640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1bec0b6780 0x7f1bec0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.532+0000 7f1c05400640 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c4890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000ba20 tx=0x7f1bf000bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6da1dced8500187 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.535+0000 7f1bf7400640 1 -- 172.21.15.114:0/3631488754 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0016070 con 0x55e52d2fe4d0 2024-09-21T07:02:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.535+0000 7f1bf7400640 1 -- 172.21.15.114:0/3631488754 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf000dc40 con 0x55e52d2fe4d0 2024-09-21T07:02:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.535+0000 7f1bf7400640 1 -- 172.21.15.114:0/3631488754 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf000f600 con 0x55e52d2fe4d0 2024-09-21T07:02:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.536+0000 7f1c14203740 1 -- 172.21.15.114:0/3631488754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 msgr2=0x55e52d6c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.537+0000 7f1c14203740 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c4890 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000ba20 tx=0x7f1bf000bee0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.537+0000 7f1c14203740 1 -- 172.21.15.114:0/3631488754 shutdown_connections 2024-09-21T07:02:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.537+0000 7f1c14203740 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c4890 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.538+0000 7f1c14203740 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.538+0000 7f1c14203740 1 --2- 172.21.15.114:0/3631488754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.538+0000 7f1c14203740 1 -- 172.21.15.114:0/3631488754 >> 172.21.15.114:0/3631488754 conn(0x55e52d313300 msgr2=0x55e52d6c1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.539+0000 7f1c06800640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1bec0b6780 0x7f1bec0b8c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c00010040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.539+0000 7f1c14203740 1 -- 172.21.15.114:0/3631488754 shutdown_connections 2024-09-21T07:02:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.539+0000 7f1c14203740 1 -- 172.21.15.114:0/3631488754 wait complete. 2024-09-21T07:02:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.540+0000 7f1c14203740 1 -- start start 2024-09-21T07:02:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.544+0000 7f1c14203740 1 Processor -- start 2024-09-21T07:02:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.546+0000 7f1c14203740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.546+0000 7f1c14203740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.547+0000 7f1c14203740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.547+0000 7f1c05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.547+0000 7f1c14203740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e52d560ab0 con 0x55e52d2f62f0 2024-09-21T07:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.547+0000 7f1c14203740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e52d5cf570 con 0x55e52d31f1b0 2024-09-21T07:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.547+0000 7f1c14203740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e52d6e35a0 con 0x55e52d2fe4d0 2024-09-21T07:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.548+0000 7f1c14203740 10 client.0 _ll_register_callbacks cb 0x55e52d5de100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.548+0000 7f1c06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.548+0000 7f1c06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42870/0 (socket says 172.21.15.114:42870) 2024-09-21T07:02:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.548+0000 7f1c06800640 1 -- 172.21.15.114:0/1604156961 learned_addr learned my addr 172.21.15.114:0/1604156961 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.549+0000 7f1c05400640 1 -- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 msgr2=0x55e52d6cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.549+0000 7f1c05400640 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.549+0000 7f1c05400640 1 -- 172.21.15.114:0/1604156961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 msgr2=0x55e52d6cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.549+0000 7f1c05400640 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.549+0000 7f1c05400640 1 -- 172.21.15.114:0/1604156961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52d38a790 con 0x55e52d2fe4d0 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.549+0000 7f1c05400640 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6cf940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000d770 tx=0x7f1bf0010ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b544335e4c02fbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.550+0000 7f1bebe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.550+0000 7f1bebe00640 1 -- 172.21.15.114:0/1604156961 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0016070 con 0x55e52d2fe4d0 2024-09-21T07:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.550+0000 7f1bebe00640 1 -- 172.21.15.114:0/1604156961 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1bf000da40 con 0x55e52d2fe4d0 2024-09-21T07:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.550+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf000dc40 con 0x55e52d2fe4d0 2024-09-21T07:02:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.551+0000 7f1bebe00640 1 -- 172.21.15.114:0/1604156961 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1bf0016070 con 0x55e52d2fe4d0 2024-09-21T07:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.551+0000 7f1c14203740 10 client.27631 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.551+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e52d403620 con 0x55e52d2fe4d0 2024-09-21T07:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1bebe00640 1 -- 172.21.15.114:0/1604156961 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bf000dc40 con 0x55e52d2fe4d0 2024-09-21T07:02:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1bebe00640 1 -- 172.21.15.114:0/1604156961 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1bf0005040 con 0x55e52d2fe4d0 2024-09-21T07:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1c14203740 10 client.27631 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1c14203740 10 client.27631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1c14203740 20 client.27631 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1c14203740 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2ddd80 0x55e52d6f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1c14203740 4 client.27631 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.552+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e52d63b600 con 0x55e52d2ddd80 2024-09-21T07:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.553+0000 7f1c05e00640 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2ddd80 0x55e52d6f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.553+0000 7f1c05e00640 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2ddd80 0x55e52d6f23b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=296251e24652ff13 server_cookie=531d65451152f0cc in_seq=0 out_seq=0 2024-09-21T07:02:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.553+0000 7f1bebe00640 10 client.27631 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.554+0000 7f1bebe00640 1 -- 172.21.15.114:0/1604156961 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e52d63b600 con 0x55e52d2ddd80 2024-09-21T07:02:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.554+0000 7f1bebe00640 10 client.27631 handle_command_reply: tid=0 2024-09-21T07:02:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.554+0000 7f1c14203740 1 client.27631 shutdown 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 msgr2=0x55e52d6cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6cf940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000d770 tx=0x7f1bf0010ed0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 shutdown_connections 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d31f1b0 0x55e52d6cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2ddd80 0x55e52d6f23b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d2fe4d0 0x55e52d6cf940 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 --2- 172.21.15.114:0/1604156961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 >> 172.21.15.114:0/1604156961 conn(0x55e52d313300 msgr2=0x55e52d6c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1bebe00640 0 client.27631 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 shutdown_connections 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 1 -- 172.21.15.114:0/1604156961 wait complete. 2024-09-21T07:02:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.555+0000 7f1c14203740 20 client.27631 trim_cache size 0 max 16384 2024-09-21T07:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.570+0000 7f1c14203740 1 Processor -- start 2024-09-21T07:02:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 -- start start 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d6cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081af0e0 con 0x55e52d2f62f0 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081b0c80 con 0x55e52d2fe4d0 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.571+0000 7f1c14203740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081af7b0 con 0x55e52d31f1b0 2024-09-21T07:02:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.572+0000 7f1c05400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.572+0000 7f1c05400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42884/0 (socket says 172.21.15.114:42884) 2024-09-21T07:02:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.572+0000 7f1c05400640 1 -- 172.21.15.114:0/168734451 learned_addr learned my addr 172.21.15.114:0/168734451 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.572+0000 7f1c06800640 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d6cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1c06800640 1 -- 172.21.15.114:0/168734451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 msgr2=0x55e52d6c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1c06800640 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1c06800640 1 -- 172.21.15.114:0/168734451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 msgr2=0x55e52d6c0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1c06800640 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1c06800640 1 -- 172.21.15.114:0/168734451 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52d2ddfb0 con 0x55e52d31f1b0 2024-09-21T07:02:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1c06800640 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d6cc5d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003cc0 tx=0x7f1c00014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fef33dcc2b0cff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.573+0000 7f1bf6a00640 1 -- 172.21.15.114:0/168734451 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004b20 con 0x55e52d31f1b0 2024-09-21T07:02:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.574+0000 7f1bf6a00640 1 -- 172.21.15.114:0/168734451 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0000db60 con 0x55e52d31f1b0 2024-09-21T07:02:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.574+0000 7f1bf6a00640 1 -- 172.21.15.114:0/168734451 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00004b20 con 0x55e52d31f1b0 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.575+0000 7f1c14203740 1 -- 172.21.15.114:0/168734451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 msgr2=0x55e52d6cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.575+0000 7f1c14203740 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d6cc5d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003cc0 tx=0x7f1c00014000 comp rx=0 tx=0).stop 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.575+0000 7f1c14203740 1 -- 172.21.15.114:0/168734451 shutdown_connections 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.575+0000 7f1c14203740 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d6cc5d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.575+0000 7f1c14203740 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.576+0000 7f1c14203740 1 --2- 172.21.15.114:0/168734451 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6c0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.576+0000 7f1c14203740 1 -- 172.21.15.114:0/168734451 >> 172.21.15.114:0/168734451 conn(0x55e52d313300 msgr2=0x55e52d6c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.576+0000 7f1c14203740 1 -- 172.21.15.114:0/168734451 shutdown_connections 2024-09-21T07:02:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.576+0000 7f1c14203740 1 -- 172.21.15.114:0/168734451 wait complete. 2024-09-21T07:02:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.577+0000 7f1c14203740 1 -- start start 2024-09-21T07:02:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.577+0000 7f1c14203740 1 Processor -- start 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d754be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081b05b0 con 0x55e52d2f62f0 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081afe80 con 0x55e52d2fe4d0 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08074e00 con 0x55e52d31f1b0 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c14203740 10 client.0 _ll_register_callbacks cb 0x7f1c00009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c06800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d754be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c06800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d754be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46758/0 (socket says 172.21.15.114:46758) 2024-09-21T07:02:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.578+0000 7f1c06800640 1 -- 172.21.15.114:0/2985835192 learned_addr learned my addr 172.21.15.114:0/2985835192 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1c06800640 1 -- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 msgr2=0x55e52d6e6780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1c06800640 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6e6780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1c06800640 1 -- 172.21.15.114:0/2985835192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 msgr2=0x55e52d6e61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1c06800640 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1c06800640 1 -- 172.21.15.114:0/2985835192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52d755280 con 0x55e52d31f1b0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1c06800640 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d754be0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009950 tx=0x7f1c000196b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8012d4c72855a570 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.579+0000 7f1bebe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1bebe00640 1 -- 172.21.15.114:0/2985835192 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00032060 con 0x55e52d31f1b0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1bebe00640 1 -- 172.21.15.114:0/2985835192 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1c00004a40 con 0x55e52d31f1b0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c0000db60 con 0x55e52d31f1b0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1bebe00640 1 -- 172.21.15.114:0/2985835192 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1c00039050 con 0x55e52d31f1b0 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1c14203740 10 client.27637 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:26.582 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e52d75c6a0 con 0x55e52d31f1b0 2024-09-21T07:02:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.580+0000 7f1bebe00640 1 -- 172.21.15.114:0/2985835192 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c0000db60 con 0x55e52d31f1b0 2024-09-21T07:02:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1bebe00640 1 -- 172.21.15.114:0/2985835192 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1c00024070 con 0x55e52d31f1b0 2024-09-21T07:02:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c14203740 10 client.27637 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c14203740 10 client.27637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c14203740 20 client.27637 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c14203740 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2b31c0 0x55e52d75ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c14203740 4 client.27637 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e52d6370b0 con 0x55e52d2b31c0 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.581+0000 7f1c05e00640 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2b31c0 0x55e52d75ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.582+0000 7f1bebe00640 10 client.27637 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.582+0000 7f1c05e00640 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2b31c0 0x55e52d75ede0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe60d43694c1d87 server_cookie=3884dd6653b43067 in_seq=0 out_seq=0 2024-09-21T07:02:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.592+0000 7f1bebe00640 1 -- 172.21.15.114:0/2985835192 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e52d6370b0 con 0x55e52d2b31c0 2024-09-21T07:02:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.592+0000 7f1bebe00640 10 client.27637 handle_command_reply: tid=0 2024-09-21T07:02:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.592+0000 7f1c14203740 1 client.27637 shutdown 2024-09-21T07:02:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.593+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 msgr2=0x55e52d754be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.593+0000 7f1c14203740 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d754be0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009950 tx=0x7f1c000196b0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.593+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 shutdown_connections 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e52d2b31c0 0x55e52d75ede0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e52d31f1b0 0x55e52d754be0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e52d2fe4d0 0x55e52d6e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 --2- 172.21.15.114:0/2985835192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e52d2f62f0 0x55e52d6e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1bebe00640 0 client.27637 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 >> 172.21.15.114:0/2985835192 conn(0x55e52d313300 msgr2=0x55e52d6c0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 shutdown_connections 2024-09-21T07:02:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.594+0000 7f1c14203740 1 -- 172.21.15.114:0/2985835192 wait complete. 2024-09-21T07:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.595+0000 7f1c14203740 20 client.27637 trim_cache size 0 max 16384 2024-09-21T07:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:02:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.595+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1bec0b6780 msgr2=0x7f1bec0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.595+0000 7f1c07200640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1bec0b6780 0x7f1bec0b8c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c00010040 comp rx=0 tx=0).stop 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.595+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 msgr2=0x7f1c0810dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.595+0000 7f1c07200640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c0810dae0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80057c0 tx=0x7f1bf80042f0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 shutdown_connections 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c0810e6a0 0x7f1c081ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1bec0b6780 0x7f1bec0b8c90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c08109b90 0x7f1c0810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 --2- 172.21.15.114:0/3739188386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1c08109240 0x7f1c0810dae0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 >> 172.21.15.114:0/3739188386 conn(0x7f1c080ff330 msgr2=0x7f1c08101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 shutdown_connections 2024-09-21T07:02:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:26.596+0000 7f1c07200640 1 -- 172.21.15.114:0/3739188386 wait complete. 2024-09-21T07:02:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:02:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:02:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:02:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:02:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:26 smithi183 ceph-mon[34219]: pgmap v649: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-09-21T07:02:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:02:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:02:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:02:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:02:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:26 smithi193 ceph-mon[34601]: pgmap v649: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-09-21T07:02:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:02:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:27 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:02:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:27 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:02:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:27 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:02:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:27 smithi114 ceph-mon[30654]: pgmap v649: 129 pgs: 129 active+clean; 2.7 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.2 MiB/s wr, 286 op/s 2024-09-21T07:02:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 470.0 s, 609.4 tps, lat 177.580 ms stddev 294.867 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.940+0000 7f639f200640 1 -- 172.21.15.114:0/3291187255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a010dc30 msgr2=0x7f63a0110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.940+0000 7f639f200640 1 --2- 172.21.15.114:0/3291187255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a010dc30 0x7f63a0110140 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6398009be0 tx=0x7f639802a6d0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.941+0000 7f639f200640 1 -- 172.21.15.114:0/3291187255 shutdown_connections 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.941+0000 7f639f200640 1 --2- 172.21.15.114:0/3291187255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a010dc30 0x7f63a0110140 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.941+0000 7f639f200640 1 --2- 172.21.15.114:0/3291187255 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63a01050b0 0x7f63a010d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.941+0000 7f639f200640 1 --2- 172.21.15.114:0/3291187255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a0104680 0x7f63a0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.941+0000 7f639f200640 1 -- 172.21.15.114:0/3291187255 >> 172.21.15.114:0/3291187255 conn(0x7f63a00fe330 msgr2=0x7f63a0100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.942+0000 7f639f200640 1 -- 172.21.15.114:0/3291187255 shutdown_connections 2024-09-21T07:02:28.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.942+0000 7f639f200640 1 -- 172.21.15.114:0/3291187255 wait complete. 2024-09-21T07:02:28.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.945+0000 7f639f200640 1 Processor -- start 2024-09-21T07:02:28.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.945+0000 7f639f200640 1 -- start start 2024-09-21T07:02:28.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63a0104680 0x7f63a010bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a01050b0 0x7f63a010c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 0x7f63a01ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01116a0 con 0x7f63a0104680 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0111520 con 0x7f63a01050b0 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63a0111820 con 0x7f63a010dc30 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 0x7f63a01ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 0x7f63a01ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46790/0 (socket says 172.21.15.114:46790) 2024-09-21T07:02:28.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639e800640 1 -- 172.21.15.114:0/372844844 learned_addr learned my addr 172.21.15.114:0/372844844 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.946+0000 7f639d400640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a01050b0 0x7f63a010c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.947+0000 7f639de00640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63a0104680 0x7f63a010bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.947+0000 7f639e800640 1 -- 172.21.15.114:0/372844844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a01050b0 msgr2=0x7f63a010c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.947+0000 7f639e800640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a01050b0 0x7f63a010c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.947+0000 7f639e800640 1 -- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63a0104680 msgr2=0x7f63a010bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.947+0000 7f639e800640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63a0104680 0x7f63a010bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.947+0000 7f639e800640 1 -- 172.21.15.114:0/372844844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a010cb80 con 0x7f63a010dc30 2024-09-21T07:02:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.948+0000 7f639e800640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 0x7f63a01ac580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6398009b30 tx=0x7f6398003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.948+0000 7f6396a00640 1 -- 172.21.15.114:0/372844844 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398040020 con 0x7f63a010dc30 2024-09-21T07:02:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.948+0000 7f6396a00640 1 -- 172.21.15.114:0/372844844 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63980318f0 con 0x7f63a010dc30 2024-09-21T07:02:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.949+0000 7f6396a00640 1 -- 172.21.15.114:0/372844844 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398036040 con 0x7f63a010dc30 2024-09-21T07:02:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.949+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a01acb80 con 0x7f63a010dc30 2024-09-21T07:02:28.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.949+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a01acfe0 con 0x7f63a010dc30 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.961+0000 7f6396a00640 1 -- 172.21.15.114:0/372844844 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6398043020 con 0x7f63a010dc30 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.962+0000 7f63aba53740 1 Processor -- start 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.963+0000 7f63aba53740 1 -- start start 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.963+0000 7f63aba53740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e66744b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.963+0000 7f63aba53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 0x562e667450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f63aba53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e66745610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f63aba53740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e666baa90 con 0x562e6639d850 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f63aba53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e666a9ec0 con 0x562e6637cb70 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f63aba53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e666adcd0 con 0x562e66374990 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f639de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 0x562e667450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f639de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 0x562e667450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46802/0 (socket says 172.21.15.114:46802) 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.964+0000 7f639de00640 1 -- 172.21.15.114:0/1172272347 learned_addr learned my addr 172.21.15.114:0/1172272347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.965+0000 7f639d400640 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e66745610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.965+0000 7f639de00640 1 -- 172.21.15.114:0/1172272347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 msgr2=0x562e66745610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:28.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.965+0000 7f639de00640 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e66745610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.967+0000 7f6396a00640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63840b67c0 0x7f63840b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.967+0000 7f6396a00640 1 -- 172.21.15.114:0/372844844 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f6398108700 con 0x7f63a010dc30 2024-09-21T07:02:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.967+0000 7f639d400640 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e66744b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.966+0000 7f639de00640 1 -- 172.21.15.114:0/1172272347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 msgr2=0x562e66744b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:28.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.969+0000 7f639de00640 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e66744b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.970+0000 7f639de00640 1 -- 172.21.15.114:0/1172272347 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6664f9d0 con 0x562e66374990 2024-09-21T07:02:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.971+0000 7f639d400640 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e66744b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:28.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.971+0000 7f639d400640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63840b67c0 0x7f63840b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.974+0000 7f639de00640 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 0x562e667450d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6390009c50 tx=0x7f6390010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=864a643762cb7230 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:28.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.975+0000 7f638b400640 1 -- 172.21.15.114:0/1172272347 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390005020 con 0x562e66374990 2024-09-21T07:02:28.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.976+0000 7f638b400640 1 -- 172.21.15.114:0/1172272347 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63900047a0 con 0x562e66374990 2024-09-21T07:02:28.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.976+0000 7f638b400640 1 -- 172.21.15.114:0/1172272347 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900126b0 con 0x562e66374990 2024-09-21T07:02:28.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.977+0000 7f63aba53740 1 -- 172.21.15.114:0/1172272347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 msgr2=0x562e667450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:28.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.977+0000 7f63aba53740 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 0x562e667450d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6390009c50 tx=0x7f6390010740 comp rx=0 tx=0).stop 2024-09-21T07:02:28.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.978+0000 7f639d400640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63840b67c0 0x7f63840b8cd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f638c0058c0 tx=0x7f638c0046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.979+0000 7f63aba53740 1 -- 172.21.15.114:0/1172272347 shutdown_connections 2024-09-21T07:02:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.979+0000 7f63aba53740 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e66745610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.979+0000 7f63aba53740 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e66374990 0x562e667450d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.979+0000 7f63aba53740 1 --2- 172.21.15.114:0/1172272347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e66744b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.979+0000 7f63aba53740 1 -- 172.21.15.114:0/1172272347 >> 172.21.15.114:0/1172272347 conn(0x562e66391c40 msgr2=0x562e66742a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:28.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.980+0000 7f63aba53740 1 -- 172.21.15.114:0/1172272347 shutdown_connections 2024-09-21T07:02:28.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.980+0000 7f63aba53740 1 -- 172.21.15.114:0/1172272347 wait complete. 2024-09-21T07:02:28.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.980+0000 7f63aba53740 1 -- start start 2024-09-21T07:02:28.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.981+0000 7f63aba53740 1 Processor -- start 2024-09-21T07:02:28.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f63aba53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66750310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f63aba53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6637cb70 0x562e66750850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f63aba53740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e6676d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f639de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66750310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f639de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66750310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42914/0 (socket says 172.21.15.114:42914) 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f639e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e6676d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f639de00640 1 -- 172.21.15.114:0/4174820202 learned_addr learned my addr 172.21.15.114:0/4174820202 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562e6650b7b0 con 0x562e6639d850 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e6664e650 con 0x562e66374990 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.982+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e667643d0 con 0x562e6637cb70 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f639d400640 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6637cb70 0x562e66750850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f63aba53740 10 client.0 _ll_register_callbacks cb 0x562e663b50a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f639de00640 1 -- 172.21.15.114:0/4174820202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6637cb70 msgr2=0x562e66750850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f639de00640 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6637cb70 0x562e66750850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f639de00640 1 -- 172.21.15.114:0/4174820202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 msgr2=0x562e6676d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f639de00640 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e6676d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.983+0000 7f639de00640 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e66417ba0 con 0x562e66374990 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.984+0000 7f639de00640 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66750310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6390009fa0 tx=0x7f6390005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03e95a158ab5519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:28.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.984+0000 7f637fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.984+0000 7f637fe00640 1 -- 172.21.15.114:0/4174820202 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639002a060 con 0x562e66374990 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.985+0000 7f637fe00640 1 -- 172.21.15.114:0/4174820202 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f6390005330 con 0x562e66374990 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.985+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63900047a0 con 0x562e66374990 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.985+0000 7f637fe00640 1 -- 172.21.15.114:0/4174820202 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f6390031050 con 0x562e66374990 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.985+0000 7f63aba53740 10 client.27707 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.985+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562e66770140 con 0x562e66374990 2024-09-21T07:02:28.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.986+0000 7f637fe00640 1 -- 172.21.15.114:0/4174820202 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63900047a0 con 0x562e66374990 2024-09-21T07:02:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.986+0000 7f637fe00640 1 -- 172.21.15.114:0/4174820202 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f639000f040 con 0x562e66374990 2024-09-21T07:02:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.986+0000 7f63aba53740 10 client.27707 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.986+0000 7f63aba53740 10 client.27707 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.987+0000 7f63aba53740 20 client.27707 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.987+0000 7f63aba53740 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e6635c1d0 0x562e66773550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:28.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.987+0000 7f63aba53740 4 client.27707 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:28.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.987+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e666bc9e0 con 0x562e6635c1d0 2024-09-21T07:02:28.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.989+0000 7f639d400640 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e6635c1d0 0x562e66773550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.991+0000 7f639d400640 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e6635c1d0 0x562e66773550 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d7a5d87e3683d9 server_cookie=ad12770afa43c8e6 in_seq=0 out_seq=0 2024-09-21T07:02:28.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.991+0000 7f637fe00640 10 client.27707 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.992+0000 7f637fe00640 1 -- 172.21.15.114:0/4174820202 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e666bc9e0 con 0x562e6635c1d0 2024-09-21T07:02:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.992+0000 7f637fe00640 10 client.27707 handle_command_reply: tid=0 2024-09-21T07:02:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.992+0000 7f63aba53740 1 client.27707 shutdown 2024-09-21T07:02:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.992+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 msgr2=0x562e66750310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:28.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.992+0000 7f63aba53740 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66750310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6390009fa0 tx=0x7f6390005db0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 shutdown_connections 2024-09-21T07:02:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f63aba53740 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6639d850 0x562e6676d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f63aba53740 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e6635c1d0 0x562e66773550 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f637fe00640 0 client.27707 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:28.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f63aba53740 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6637cb70 0x562e66750850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f63aba53740 1 --2- 172.21.15.114:0/4174820202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66750310 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:28.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.994+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 >> 172.21.15.114:0/4174820202 conn(0x562e66391c40 msgr2=0x562e66747d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.998+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 shutdown_connections 2024-09-21T07:02:28.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.998+0000 7f63aba53740 1 -- 172.21.15.114:0/4174820202 wait complete. 2024-09-21T07:02:29.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:28.998+0000 7f63aba53740 20 client.27707 trim_cache size 0 max 16384 2024-09-21T07:02:29.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.024+0000 7f63aba53740 1 Processor -- start 2024-09-21T07:02:29.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.024+0000 7f63aba53740 1 -- start start 2024-09-21T07:02:29.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f63aba53740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e66374990 0x562e6674bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f63aba53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e667462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f63aba53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e6676d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f63aba53740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01aeb50 con 0x562e66374990 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f63aba53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01b06f0 con 0x562e6637cb70 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f63aba53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01af220 con 0x562e6639d850 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e667462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e667462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42930/0 (socket says 172.21.15.114:42930) 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 -- 172.21.15.114:0/2103234707 learned_addr learned my addr 172.21.15.114:0/2103234707 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639e800640 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e6676d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 -- 172.21.15.114:0/2103234707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 msgr2=0x562e6676d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e6676d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 -- 172.21.15.114:0/2103234707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e66374990 msgr2=0x562e6674bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e66374990 0x562e6674bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.025+0000 7f639d400640 1 -- 172.21.15.114:0/2103234707 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e667515d0 con 0x562e6637cb70 2024-09-21T07:02:29.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.026+0000 7f639d400640 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e667462d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f638c01d6d0 tx=0x7f638c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead7aad9bec87265 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:29.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.026+0000 7f638aa00640 1 -- 172.21.15.114:0/2103234707 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c035070 con 0x562e6637cb70 2024-09-21T07:02:29.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.026+0000 7f638aa00640 1 -- 172.21.15.114:0/2103234707 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f638c030420 con 0x562e6637cb70 2024-09-21T07:02:29.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.026+0000 7f638aa00640 1 -- 172.21.15.114:0/2103234707 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c02a680 con 0x562e6637cb70 2024-09-21T07:02:29.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.027+0000 7f63aba53740 1 -- 172.21.15.114:0/2103234707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 msgr2=0x562e667462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:29.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.027+0000 7f63aba53740 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e667462d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f638c01d6d0 tx=0x7f638c018000 comp rx=0 tx=0).stop 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 -- 172.21.15.114:0/2103234707 shutdown_connections 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e6676d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e6637cb70 0x562e667462d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 --2- 172.21.15.114:0/2103234707 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e66374990 0x562e6674bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 -- 172.21.15.114:0/2103234707 >> 172.21.15.114:0/2103234707 conn(0x562e66391c40 msgr2=0x562e66745bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 -- 172.21.15.114:0/2103234707 shutdown_connections 2024-09-21T07:02:29.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.028+0000 7f63aba53740 1 -- 172.21.15.114:0/2103234707 wait complete. 2024-09-21T07:02:29.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.029+0000 7f63aba53740 1 -- start start 2024-09-21T07:02:29.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.029+0000 7f63aba53740 1 Processor -- start 2024-09-21T07:02:29.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66765e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6637cb70 0x562e66766390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e667dce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01b0020 con 0x562e6637cb70 2024-09-21T07:02:29.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01af8f0 con 0x562e66374990 2024-09-21T07:02:29.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63a010d070 con 0x562e6639d850 2024-09-21T07:02:29.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.030+0000 7f63aba53740 10 client.0 _ll_register_callbacks cb 0x7f638c0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.031+0000 7f639de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66765e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.031+0000 7f639de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66765e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42946/0 (socket says 172.21.15.114:42946) 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.031+0000 7f639de00640 1 -- 172.21.15.114:0/2751312560 learned_addr learned my addr 172.21.15.114:0/2751312560 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.031+0000 7f639e800640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e667dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.031+0000 7f639d400640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6637cb70 0x562e66766390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.032+0000 7f639d400640 1 -- 172.21.15.114:0/2751312560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 msgr2=0x562e667dce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.032+0000 7f639d400640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e667dce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.032+0000 7f639d400640 1 -- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 msgr2=0x562e66765e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:29.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.032+0000 7f639d400640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66765e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.032+0000 7f639d400640 1 -- 172.21.15.114:0/2751312560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e667dd420 con 0x562e6637cb70 2024-09-21T07:02:29.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.032+0000 7f639e800640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e667dce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:29.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.033+0000 7f639d400640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6637cb70 0x562e66766390 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a52515e1b9c4f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:29.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.033+0000 7f637fe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:29.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.033+0000 7f637fe00640 1 -- 172.21.15.114:0/2751312560 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c035040 con 0x562e6637cb70 2024-09-21T07:02:29.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.033+0000 7f637fe00640 1 -- 172.21.15.114:0/2751312560 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f638c005040 con 0x562e6637cb70 2024-09-21T07:02:29.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.033+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f638c030420 con 0x562e6637cb70 2024-09-21T07:02:29.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.034+0000 7f637fe00640 1 -- 172.21.15.114:0/2751312560 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f638c005240 con 0x562e6637cb70 2024-09-21T07:02:29.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.034+0000 7f63aba53740 10 client.17307 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:29.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.034+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562e667de680 con 0x562e6637cb70 2024-09-21T07:02:29.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.034+0000 7f637fe00640 1 -- 172.21.15.114:0/2751312560 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f638c030420 con 0x562e6637cb70 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.034+0000 7f637fe00640 1 -- 172.21.15.114:0/2751312560 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f638c038030 con 0x562e6637cb70 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f63aba53740 10 client.17307 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f63aba53740 10 client.17307 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f63aba53740 20 client.17307 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f63aba53740 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e66331580 0x562e667e0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f63aba53740 4 client.17307 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}] 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.05, "await_for": 45.0}) -- 0x562e666b8490 con 0x562e66331580 2024-09-21T07:02:29.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.035+0000 7f639de00640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e66331580 0x562e667e0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:29.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.036+0000 7f639de00640 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e66331580 0x562e667e0dd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed47b07146069930 server_cookie=db243318dea00180 in_seq=0 out_seq=0 2024-09-21T07:02:29.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.036+0000 7f637fe00640 10 client.17307 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:29.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:28 smithi193 ceph-mon[34601]: pgmap v650: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-09-21T07:02:29.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.248+0000 7f637fe00640 1 -- 172.21.15.114:0/2751312560 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562e666b8490 con 0x562e66331580 2024-09-21T07:02:29.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.248+0000 7f637fe00640 10 client.17307 handle_command_reply: tid=0 2024-09-21T07:02:29.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.248+0000 7f63aba53740 1 client.17307 shutdown 2024-09-21T07:02:29.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6637cb70 msgr2=0x562e66766390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:29.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6637cb70 0x562e66766390 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c031de0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 shutdown_connections 2024-09-21T07:02:29.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562e66331580 0x562e667e0dd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.251 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:28 smithi114 ceph-mon[30654]: pgmap v650: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e6639d850 0x562e667dce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562e6637cb70 0x562e66766390 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 --2- 172.21.15.114:0/2751312560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e66374990 0x562e66765e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 >> 172.21.15.114:0/2751312560 conn(0x562e66391c40 msgr2=0x562e66741060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f637fe00640 0 client.17307 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 shutdown_connections 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 1 -- 172.21.15.114:0/2751312560 wait complete. 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.249+0000 7f63aba53740 20 client.17307 trim_cache size 0 max 16384 2024-09-21T07:02:29.252 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 47, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 34, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "2d786875": { 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 34, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:02:29.253 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 141.1, 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:29.254 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:29.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63840b67c0 msgr2=0x7f63840b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:29.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63840b67c0 0x7f63840b8cd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f638c0058c0 tx=0x7f638c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 msgr2=0x7f63a01ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:29.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 0x7f63a01ac580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6398009b30 tx=0x7f6398003c10 comp rx=0 tx=0).stop 2024-09-21T07:02:29.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 shutdown_connections 2024-09-21T07:02:29.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a010dc30 0x7f63a01ac580 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f63840b67c0 0x7f63840b8cd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63a01050b0 0x7f63a010c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 --2- 172.21.15.114:0/372844844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f63a0104680 0x7f63a010bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:29.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 >> 172.21.15.114:0/372844844 conn(0x7f63a00fe330 msgr2=0x7f63a0100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:29.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 shutdown_connections 2024-09-21T07:02:29.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:29.250+0000 7f639f200640 1 -- 172.21.15.114:0/372844844 wait complete. 2024-09-21T07:02:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:28 smithi183 ceph-mon[34219]: pgmap v650: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-09-21T07:02:30.178 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2d786875', quiesce took 9.5 seconds. Will release after: 11.55 2024-09-21T07:02:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:30 smithi193 ceph-mon[34601]: pgmap v651: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 488 op/s 2024-09-21T07:02:31.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:30 smithi114 ceph-mon[30654]: pgmap v651: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 488 op/s 2024-09-21T07:02:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:30 smithi183 ceph-mon[34219]: pgmap v651: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 488 op/s 2024-09-21T07:02:32.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 475.0 s, 121.7 tps, lat 228.239 ms stddev 240.000 2024-09-21T07:02:33.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:32 smithi193 ceph-mon[34601]: pgmap v652: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 462 op/s 2024-09-21T07:02:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:32 smithi114 ceph-mon[30654]: pgmap v652: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 462 op/s 2024-09-21T07:02:33.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:32 smithi183 ceph-mon[34219]: pgmap v652: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 4.6 MiB/s wr, 462 op/s 2024-09-21T07:02:35.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:34 smithi193 ceph-mon[34601]: pgmap v653: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-21T07:02:35.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:34 smithi114 ceph-mon[30654]: pgmap v653: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-21T07:02:35.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:34 smithi183 ceph-mon[34219]: pgmap v653: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-21T07:02:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:36 smithi193 ceph-mon[34601]: pgmap v654: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-09-21T07:02:37.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:36 smithi114 ceph-mon[30654]: pgmap v654: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-09-21T07:02:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:36 smithi183 ceph-mon[34219]: pgmap v654: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 3.7 MiB/s wr, 390 op/s 2024-09-21T07:02:37.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 480.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:02:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:38 smithi193 ceph-mon[34601]: pgmap v655: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 347 op/s 2024-09-21T07:02:39.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:38 smithi114 ceph-mon[30654]: pgmap v655: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 347 op/s 2024-09-21T07:02:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:38 smithi183 ceph-mon[34219]: pgmap v655: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.4 MiB/s wr, 347 op/s 2024-09-21T07:02:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 -- 172.21.15.114:0/2561469325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f98105cc0 msgr2=0x7f0f981060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2561469325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f98105cc0 0x7f0f981060d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f8802a410 comp rx=0 tx=0).stop 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 -- 172.21.15.114:0/2561469325 shutdown_connections 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2561469325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f9810bf40 0x7f0f98112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2561469325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f9810b5f0 0x7f0f9810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2561469325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f98105cc0 0x7f0f981060d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.836+0000 7f0f9e600640 1 -- 172.21.15.114:0/2561469325 >> 172.21.15.114:0/2561469325 conn(0x7f0f981016e0 msgr2=0x7f0f98103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.837+0000 7f0f9e600640 1 -- 172.21.15.114:0/2561469325 shutdown_connections 2024-09-21T07:02:39.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.837+0000 7f0f9e600640 1 -- 172.21.15.114:0/2561469325 wait complete. 2024-09-21T07:02:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 Processor -- start 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 -- start start 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98105cc0 0x7f0f9819c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f9810b5f0 0x7f0f9819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 0x7f0f981a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98115c00 con 0x7f0f9810bf40 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98115a80 con 0x7f0f9810b5f0 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f9e600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f98115d80 con 0x7f0f98105cc0 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f97e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 0x7f0f981a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f97e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 0x7f0f981a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52692/0 (socket says 172.21.15.114:52692) 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f97e00640 1 -- 172.21.15.114:0/2358170347 learned_addr learned my addr 172.21.15.114:0/2358170347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.838+0000 7f0f97400640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98105cc0 0x7f0f9819c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f9810b5f0 0x7f0f9819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f97e00640 1 -- 172.21.15.114:0/2358170347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98105cc0 msgr2=0x7f0f9819c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f97e00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98105cc0 0x7f0f9819c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f97e00640 1 -- 172.21.15.114:0/2358170347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f9810b5f0 msgr2=0x7f0f9819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f97e00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f9810b5f0 0x7f0f9819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f97e00640 1 -- 172.21.15.114:0/2358170347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f981a2570 con 0x7f0f9810bf40 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f97e00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 0x7f0f981a1e70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000eb10 tx=0x7f0f9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f94c00640 1 -- 172.21.15.114:0/2358170347 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90004b20 con 0x7f0f9810bf40 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f94c00640 1 -- 172.21.15.114:0/2358170347 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f90004cc0 con 0x7f0f9810bf40 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f981a2860 con 0x7f0f9810bf40 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.839+0000 7f0f94c00640 1 -- 172.21.15.114:0/2358170347 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90010690 con 0x7f0f9810bf40 2024-09-21T07:02:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.840+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f981a8ff0 con 0x7f0f9810bf40 2024-09-21T07:02:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.842+0000 7f0fa5847740 1 Processor -- start 2024-09-21T07:02:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.843+0000 7f0f94c00640 1 -- 172.21.15.114:0/2358170347 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0f90005020 con 0x7f0f9810bf40 2024-09-21T07:02:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.843+0000 7f0fa5847740 1 -- start start 2024-09-21T07:02:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310d9cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310d9d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0fa5847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564310d126b0 con 0x5643109cf2f0 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0fa5847740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564310d01ae0 con 0x5643109d74d0 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0fa5847740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564310d058f0 con 0x5643109f81b0 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d9d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52696/0 (socket says 172.21.15.114:52696) 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f8ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310d9cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 -- 172.21.15.114:0/3834213232 learned_addr learned my addr 172.21.15.114:0/3834213232 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310d9d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 -- 172.21.15.114:0/3834213232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 msgr2=0x564310d9cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310d9cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 -- 172.21.15.114:0/3834213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 msgr2=0x564310d9d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310d9d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.844+0000 7f0f97400640 1 -- 172.21.15.114:0/3834213232 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564310cce780 con 0x5643109cf2f0 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f94c00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0f7c0b6680 0x7f0f7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f97400640 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d9d170 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f88032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff265f01e3b9546f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f94c00640 1 -- 172.21.15.114:0/2358170347 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f0f900e80e0 con 0x7f0f9810bf40 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0f7c0b6680 0x7f0f7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f8cc00640 1 -- 172.21.15.114:0/3834213232 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88040070 con 0x5643109cf2f0 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f8cc00640 1 -- 172.21.15.114:0/3834213232 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f88004030 con 0x5643109cf2f0 2024-09-21T07:02:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.845+0000 7f0f8cc00640 1 -- 172.21.15.114:0/3834213232 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88035040 con 0x5643109cf2f0 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 -- 172.21.15.114:0/3834213232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 msgr2=0x564310d9d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d9d170 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009900 tx=0x7f0f88032c40 comp rx=0 tx=0).stop 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0f7c0b6680 0x7f0f7c0b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f9806b100 tx=0x7f0f800046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 -- 172.21.15.114:0/3834213232 shutdown_connections 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310d9d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d9d170 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3834213232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310d9cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 -- 172.21.15.114:0/3834213232 >> 172.21.15.114:0/3834213232 conn(0x5643109ec300 msgr2=0x564310d9ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 -- 172.21.15.114:0/3834213232 shutdown_connections 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.846+0000 7f0fa5847740 1 -- 172.21.15.114:0/3834213232 wait complete. 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.847+0000 7f0fa5847740 1 -- start start 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.847+0000 7f0fa5847740 1 Processor -- start 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310da81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310da8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310da8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564310c39ab0 con 0x5643109cf2f0 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564310ca8570 con 0x5643109d74d0 2024-09-21T07:02:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564310dab350 con 0x5643109f81b0 2024-09-21T07:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0fa5847740 10 client.0 _ll_register_callbacks cb 0x564310cb7100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310da81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310da81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52710/0 (socket says 172.21.15.114:52710) 2024-09-21T07:02:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f8ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310da8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 -- 172.21.15.114:0/564796192 learned_addr learned my addr 172.21.15.114:0/564796192 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97e00640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310da8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 -- 172.21.15.114:0/564796192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 msgr2=0x564310da8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310da8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 -- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 msgr2=0x564310da8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310da8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f97400640 1 -- 172.21.15.114:0/564796192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564310a63790 con 0x5643109cf2f0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.848+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310da8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0f97400640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310da81d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0f8802a930 tx=0x7f0f880050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55563571c870357 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0f85600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0f85600640 1 -- 172.21.15.114:0/564796192 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88040070 con 0x5643109cf2f0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0f85600640 1 -- 172.21.15.114:0/564796192 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f0f88005300 con 0x5643109cf2f0 2024-09-21T07:02:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f88004030 con 0x5643109cf2f0 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0f85600640 1 -- 172.21.15.114:0/564796192 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0f88056050 con 0x5643109cf2f0 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0fa5847740 10 client.17319 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564310adc620 con 0x5643109cf2f0 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.849+0000 7f0f85600640 1 -- 172.21.15.114:0/564796192 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f88004030 con 0x5643109cf2f0 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0f85600640 1 -- 172.21.15.114:0/564796192 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f0f88004bf0 con 0x5643109cf2f0 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0fa5847740 10 client.17319 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0fa5847740 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0fa5847740 20 client.17319 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0fa5847740 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5643109b6d80 0x564310dba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0fa5847740 4 client.17319 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564310d14600 con 0x5643109b6d80 2024-09-21T07:02:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.850+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5643109b6d80 0x564310dba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.851+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5643109b6d80 0x564310dba110 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390ed60595cf6bea server_cookie=f8cafba5c9768152 in_seq=0 out_seq=0 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.851+0000 7f0f85600640 10 client.17319 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0f85600640 1 -- 172.21.15.114:0/564796192 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564310d14600 con 0x5643109b6d80 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0f85600640 10 client.17319 handle_command_reply: tid=0 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 client.17319 shutdown 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 msgr2=0x564310da81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310da81d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0f8802a930 tx=0x7f0f880050e0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 shutdown_connections 2024-09-21T07:02:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5643109b6d80 0x564310dba110 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109f81b0 0x564310da8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109d74d0 0x564310da8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0f85600640 0 client.17319 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 --2- 172.21.15.114:0/564796192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310da81d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 >> 172.21.15.114:0/564796192 conn(0x5643109ec300 msgr2=0x564310d9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 shutdown_connections 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.852+0000 7f0fa5847740 1 -- 172.21.15.114:0/564796192 wait complete. 2024-09-21T07:02:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.853+0000 7f0fa5847740 20 client.17319 trim_cache size 0 max 16384 2024-09-21T07:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 Processor -- start 2024-09-21T07:02:39.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 -- start start 2024-09-21T07:02:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d99aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109d74d0 0x564310db3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310da53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b3140 con 0x5643109cf2f0 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b4ce0 con 0x5643109f81b0 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0fa5847740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b3810 con 0x5643109d74d0 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0f97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d99aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0f97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d99aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52716/0 (socket says 172.21.15.114:52716) 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.868+0000 7f0f97400640 1 -- 172.21.15.114:0/1782926715 learned_addr learned my addr 172.21.15.114:0/1782926715 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97e00640 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310da53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109d74d0 0x564310db3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97400640 1 -- 172.21.15.114:0/1782926715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109d74d0 msgr2=0x564310db3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97400640 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109d74d0 0x564310db3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97400640 1 -- 172.21.15.114:0/1782926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 msgr2=0x564310da53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97400640 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310da53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97400640 1 -- 172.21.15.114:0/1782926715 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564310daaf80 con 0x5643109cf2f0 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f97400640 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d99aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0f880050b0 tx=0x7f0f8802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12780bf2c505b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f87e00640 1 -- 172.21.15.114:0/1782926715 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88040070 con 0x5643109cf2f0 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f87e00640 1 -- 172.21.15.114:0/1782926715 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8804d070 con 0x5643109cf2f0 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.869+0000 7f0f87e00640 1 -- 172.21.15.114:0/1782926715 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88035040 con 0x5643109cf2f0 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 -- 172.21.15.114:0/1782926715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 msgr2=0x564310d99aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d99aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0f880050b0 tx=0x7f0f8802ab10 comp rx=0 tx=0).stop 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 -- 172.21.15.114:0/1782926715 shutdown_connections 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310da53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109d74d0 0x564310db3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 --2- 172.21.15.114:0/1782926715 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109cf2f0 0x564310d99aa0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 -- 172.21.15.114:0/1782926715 >> 172.21.15.114:0/1782926715 conn(0x5643109ec300 msgr2=0x564310db35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 -- 172.21.15.114:0/1782926715 shutdown_connections 2024-09-21T07:02:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.870+0000 7f0fa5847740 1 -- 172.21.15.114:0/1782926715 wait complete. 2024-09-21T07:02:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.871+0000 7f0fa5847740 1 -- start start 2024-09-21T07:02:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.871+0000 7f0fa5847740 1 Processor -- start 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.871+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109cf2f0 0x564310dbd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.871+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 0x564310dbd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.871+0000 7f0fa5847740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310e25670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0fa5847740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b4610 con 0x5643109d74d0 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0fa5847740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b3ee0 con 0x5643109f81b0 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0fa5847740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f981b1140 con 0x5643109cf2f0 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0fa5847740 10 client.0 _ll_register_callbacks cb 0x7f0f8000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 0x564310dbd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 0x564310dbd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52724/0 (socket says 172.21.15.114:52724) 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 -- 172.21.15.114:0/3588211567 learned_addr learned my addr 172.21.15.114:0/3588211567 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:39.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f97e00640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310e25670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f97400640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109cf2f0 0x564310dbd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 -- 172.21.15.114:0/3588211567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109cf2f0 msgr2=0x564310dbd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109cf2f0 0x564310dbd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 -- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 msgr2=0x564310e25670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310e25670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 -- 172.21.15.114:0/3588211567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564310e25d10 con 0x5643109d74d0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f8ea00640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 0x564310dbd720 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0f9806c7b0 tx=0x7f0f8000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7e0b187c1f8d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.872+0000 7f0f85600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0f85600640 1 -- 172.21.15.114:0/3588211567 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80004020 con 0x5643109d74d0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0f85600640 1 -- 172.21.15.114:0/3588211567 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f0f800041c0 con 0x5643109d74d0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8804d070 con 0x5643109d74d0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0f85600640 1 -- 172.21.15.114:0/3588211567 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0f8003d050 con 0x5643109d74d0 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0fa5847740 10 client.17325 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:39.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564310e2d150 con 0x5643109d74d0 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.873+0000 7f0f85600640 1 -- 172.21.15.114:0/3588211567 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8804d070 con 0x5643109d74d0 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0f85600640 1 -- 172.21.15.114:0/3588211567 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f0f80029050 con 0x5643109d74d0 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0fa5847740 10 client.17325 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0fa5847740 10 client.17325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0fa5847740 20 client.17325 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56431098c1c0 0x564310e2f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0fa5847740 4 client.17325 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564310d100b0 con 0x56431098c1c0 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.874+0000 7f0f97400640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56431098c1c0 0x564310e2f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.875+0000 7f0f97400640 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56431098c1c0 0x564310e2f940 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f25979e7d75b9fb server_cookie=f8bc4d4441fc3494 in_seq=0 out_seq=0 2024-09-21T07:02:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.875+0000 7f0f85600640 10 client.17325 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0f85600640 1 -- 172.21.15.114:0/3588211567 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564310d100b0 con 0x56431098c1c0 2024-09-21T07:02:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0f85600640 10 client.17325 handle_command_reply: tid=0 2024-09-21T07:02:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 client.17325 shutdown 2024-09-21T07:02:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 msgr2=0x564310dbd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 0x564310dbd720 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0f9806c7b0 tx=0x7f0f8000c4d0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 shutdown_connections 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643109f81b0 0x564310e25670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643109d74d0 0x564310dbd720 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56431098c1c0 0x564310e2f940 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 --2- 172.21.15.114:0/3588211567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643109cf2f0 0x564310dbd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0f85600640 0 client.17325 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.883+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 >> 172.21.15.114:0/3588211567 conn(0x5643109ec300 msgr2=0x564310d99870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 shutdown_connections 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0fa5847740 1 -- 172.21.15.114:0/3588211567 wait complete. 2024-09-21T07:02:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0fa5847740 20 client.17325 trim_cache size 0 max 16384 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0f7c0b6680 msgr2=0x7f0f7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0f7c0b6680 0x7f0f7c0b8b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f9806b100 tx=0x7f0f800046c0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 msgr2=0x7f0f981a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 0x7f0f981a1e70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000eb10 tx=0x7f0f9000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 shutdown_connections 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f9810bf40 0x7f0f981a1e70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0f7c0b6680 0x7f0f7c0b8b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f9810b5f0 0x7f0f9819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 --2- 172.21.15.114:0/2358170347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f98105cc0 0x7f0f9819c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.884+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 >> 172.21.15.114:0/2358170347 conn(0x7f0f981016e0 msgr2=0x7f0f98103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.885+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 shutdown_connections 2024-09-21T07:02:39.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:39.885+0000 7f0f9e600640 1 -- 172.21.15.114:0/2358170347 wait complete. 2024-09-21T07:02:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:40 smithi193 ceph-mon[34601]: pgmap v656: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-09-21T07:02:41.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:40 smithi114 ceph-mon[30654]: pgmap v656: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-09-21T07:02:41.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:40 smithi183 ceph-mon[34219]: pgmap v656: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-09-21T07:02:41.730 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:02:42.114 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 -- 172.21.15.114:0/2109803946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 msgr2=0x7f1a7810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 --2- 172.21.15.114:0/2109803946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 0x7f1a7810b900 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a6002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 -- 172.21.15.114:0/2109803946 shutdown_connections 2024-09-21T07:02:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 --2- 172.21.15.114:0/2109803946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a78112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 --2- 172.21.15.114:0/2109803946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 0x7f1a7810b900 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 --2- 172.21.15.114:0/2109803946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 0x7f1a78105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 -- 172.21.15.114:0/2109803946 >> 172.21.15.114:0/2109803946 conn(0x7f1a78101680 msgr2=0x7f1a78103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:42.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 -- 172.21.15.114:0/2109803946 shutdown_connections 2024-09-21T07:02:42.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.750+0000 7f1a77200640 1 -- 172.21.15.114:0/2109803946 wait complete. 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.751+0000 7f1a77200640 1 Processor -- start 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.751+0000 7f1a77200640 1 -- start start 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a77200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 0x7f1a7819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a77200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 0x7f1a7819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a77200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a781a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a77200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78115b00 con 0x7f1a7810b4f0 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a77200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78115980 con 0x7f1a78105bb0 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a77200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a78115c80 con 0x7f1a7810be40 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a76800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a781a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a75e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 0x7f1a7819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a75400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 0x7f1a7819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:42.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a76800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a781a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33198/0 (socket says 172.21.15.114:33198) 2024-09-21T07:02:42.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.752+0000 7f1a76800640 1 -- 172.21.15.114:0/1733465938 learned_addr learned my addr 172.21.15.114:0/1733465938 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:42.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a76800640 1 -- 172.21.15.114:0/1733465938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 msgr2=0x7f1a7819c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:42.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a76800640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 0x7f1a7819c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a76800640 1 -- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 msgr2=0x7f1a7819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a76800640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 0x7f1a7819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a76800640 1 -- 172.21.15.114:0/1733465938 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a781a24e0 con 0x7f1a7810be40 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a75e00640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 0x7f1a7819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a76800640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a781a1db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000e970 tx=0x7f1a7000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7000ccc0 con 0x7f1a7810be40 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a781a27d0 con 0x7f1a7810be40 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7000ce60 con 0x7f1a7810be40 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.753+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a70010430 con 0x7f1a7810be40 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.754+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a781a8f30 con 0x7f1a7810be40 2024-09-21T07:02:42.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.754+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a78105fc0 con 0x7f1a7810be40 2024-09-21T07:02:42.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.757+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1a70004f70 con 0x7f1a7810be40 2024-09-21T07:02:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.759+0000 7f1a6ea00640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b6680 0x7f1a5c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.759+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1a700e7ff0 con 0x7f1a7810be40 2024-09-21T07:02:42.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.759+0000 7f1a75e00640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b6680 0x7f1a5c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:42.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.760+0000 7f1a75e00640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b6680 0x7f1a5c0b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009fd0 tx=0x7f1a68009760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:42.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.761+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1a70056710 con 0x7f1a7810be40 2024-09-21T07:02:42.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 485.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:02:42.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.986+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a7819d6a0 con 0x7f1a7810be40 2024-09-21T07:02:42.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.987+0000 7f1a6ea00640 1 -- 172.21.15.114:0/1733465938 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11992 (secure 0 0 0) 0x7f1a7819d6a0 con 0x7f1a7810be40 2024-09-21T07:02:42.990 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:02:42.990 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"btime":"2024-09-21T07:02:02:992961+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:02:02.920775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:02:42.990 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-09-21T07:02:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b6680 msgr2=0x7f1a5c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b6680 0x7f1a5c0b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009fd0 tx=0x7f1a68009760 comp rx=0 tx=0).stop 2024-09-21T07:02:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 msgr2=0x7f1a781a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a781a1db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000e970 tx=0x7f1a7000ed70 comp rx=0 tx=0).stop 2024-09-21T07:02:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 shutdown_connections 2024-09-21T07:02:42.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a7810be40 0x7f1a781a1db0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1a7810b4f0 0x7f1a7819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1a5c0b6680 0x7f1a5c0b8b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 --2- 172.21.15.114:0/1733465938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a78105bb0 0x7f1a7819c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:42.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 >> 172.21.15.114:0/1733465938 conn(0x7f1a78101680 msgr2=0x7f1a78103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:42.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 shutdown_connections 2024-09-21T07:02:42.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:42.989+0000 7f1a77200640 1 -- 172.21.15.114:0/1733465938 wait complete. 2024-09-21T07:02:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:42 smithi193 ceph-mon[34601]: pgmap v657: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 936 KiB/s wr, 111 op/s 2024-09-21T07:02:43.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:42 smithi114 ceph-mon[30654]: pgmap v657: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 936 KiB/s wr, 111 op/s 2024-09-21T07:02:43.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:42 smithi183 ceph-mon[34219]: pgmap v657: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 936 KiB/s wr, 111 op/s 2024-09-21T07:02:43.898 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 2d786875 --release --await' 2024-09-21T07:02:43.899 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 2d786875 --release --await 2024-09-21T07:02:44.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:43 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1733465938' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:44.270 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:44.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:43 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1733465938' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:44.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:43 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1733465938' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 -- 172.21.15.114:0/2091883812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010c020 msgr2=0x7f24d0112850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 --2- 172.21.15.114:0/2091883812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010c020 0x7f24d0112850 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009ca0 tx=0x7f24c802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 -- 172.21.15.114:0/2091883812 shutdown_connections 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 --2- 172.21.15.114:0/2091883812 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010c020 0x7f24d0112850 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 --2- 172.21.15.114:0/2091883812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d010b6d0 0x7f24d010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 --2- 172.21.15.114:0/2091883812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d0105d90 0x7f24d01061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 -- 172.21.15.114:0/2091883812 >> 172.21.15.114:0/2091883812 conn(0x7f24d0101860 msgr2=0x7f24d0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 -- 172.21.15.114:0/2091883812 shutdown_connections 2024-09-21T07:02:44.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.947+0000 7f24d5200640 1 -- 172.21.15.114:0/2091883812 wait complete. 2024-09-21T07:02:44.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 Processor -- start 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 -- start start 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 0x7f24d019c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010b6d0 0x7f24d019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d010c020 0x7f24d01a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0115ce0 con 0x7f24d010b6d0 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0115b60 con 0x7f24d010c020 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24d5200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24d0115e60 con 0x7f24d0105d90 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24cd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010b6d0 0x7f24d019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.949+0000 7f24cde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 0x7f24d019c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 0x7f24d019c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39020/0 (socket says 172.21.15.114:39020) 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010b6d0 0x7f24d019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59760/0 (socket says 172.21.15.114:59760) 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 -- 172.21.15.114:0/3063931922 learned_addr learned my addr 172.21.15.114:0/3063931922 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:44.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24ce800640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d010c020 0x7f24d01a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 -- 172.21.15.114:0/3063931922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d010c020 msgr2=0x7f24d01a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:44.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d010c020 0x7f24d01a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 -- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010b6d0 msgr2=0x7f24d019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:44.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010b6d0 0x7f24d019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 -- 172.21.15.114:0/3063931922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d01a2700 con 0x7f24d0105d90 2024-09-21T07:02:44.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.950+0000 7f24cde00640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 0x7f24d019c790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:44.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.951+0000 7f24c6a00640 1 -- 172.21.15.114:0/3063931922 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c0017030 con 0x7f24d0105d90 2024-09-21T07:02:44.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.951+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d01a29f0 con 0x7f24d0105d90 2024-09-21T07:02:44.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.951+0000 7f24c6a00640 1 -- 172.21.15.114:0/3063931922 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c0004020 con 0x7f24d0105d90 2024-09-21T07:02:44.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.951+0000 7f24c6a00640 1 -- 172.21.15.114:0/3063931922 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00047a0 con 0x7f24d0105d90 2024-09-21T07:02:44.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.951+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d01a9150 con 0x7f24d0105d90 2024-09-21T07:02:44.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.955+0000 7f24dbfd7740 1 Processor -- start 2024-09-21T07:02:44.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.955+0000 7f24dbfd7740 1 -- start start 2024-09-21T07:02:44.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731309700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731309c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560731280290 con 0x560730f62520 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56073126f6c0 con 0x560730f41840 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607312734d0 con 0x560730f39660 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59770/0 (socket says 172.21.15.114:59770) 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 -- 172.21.15.114:0/3580805552 learned_addr learned my addr 172.21.15.114:0/3580805552 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24cd400640 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731309c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 -- 172.21.15.114:0/3580805552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 msgr2=0x560731309700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731309700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 -- 172.21.15.114:0/3580805552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 msgr2=0x560731309c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24c6a00640 1 -- 172.21.15.114:0/3063931922 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f24c0009a30 con 0x7f24d0105d90 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731309c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24ce800640 1 -- 172.21.15.114:0/3580805552 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607311fc460 con 0x560730f62520 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.956+0000 7f24cd400640 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731309c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:44.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.957+0000 7f24ce800640 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313091c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f24c802f4c0 tx=0x7f24c8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8327c2a544b47b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:44.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.957+0000 7f24bb400640 1 -- 172.21.15.114:0/3580805552 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8031cf0 con 0x560730f62520 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.957+0000 7f24bb400640 1 -- 172.21.15.114:0/3580805552 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036030 con 0x560730f62520 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.957+0000 7f24bb400640 1 -- 172.21.15.114:0/3580805552 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c80042a0 con 0x560730f62520 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 -- 172.21.15.114:0/3580805552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 msgr2=0x5607313091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313091c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f24c802f4c0 tx=0x7f24c8009800 comp rx=0 tx=0).stop 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 -- 172.21.15.114:0/3580805552 shutdown_connections 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24c6a00640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24b40b6680 0x7f24b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731309c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731309700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/3580805552 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313091c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 -- 172.21.15.114:0/3580805552 >> 172.21.15.114:0/3580805552 conn(0x560730f56910 msgr2=0x5607313070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24c6a00640 1 -- 172.21.15.114:0/3063931922 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f24c00e7550 con 0x7f24d0105d90 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24dbfd7740 1 -- 172.21.15.114:0/3580805552 shutdown_connections 2024-09-21T07:02:44.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.958+0000 7f24ce800640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24b40b6680 0x7f24b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.959+0000 7f24dbfd7740 1 -- 172.21.15.114:0/3580805552 wait complete. 2024-09-21T07:02:44.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.959+0000 7f24dbfd7740 1 -- start start 2024-09-21T07:02:44.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.959+0000 7f24ce800640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24b40b6680 0x7f24b40b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c802a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.959+0000 7f24dbfd7740 1 Processor -- start 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731314e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731315410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x560731320ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5607310d0bb0 con 0x560730f62520 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56073120a6c0 con 0x560730f41840 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607313177c0 con 0x560730f39660 2024-09-21T07:02:44.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24dbfd7740 10 client.0 _ll_register_callbacks cb 0x560731225490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24ce800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x560731320ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24ce800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x560731320ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59780/0 (socket says 172.21.15.114:59780) 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24ce800640 1 -- 172.21.15.114:0/2798368496 learned_addr learned my addr 172.21.15.114:0/2798368496 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24cde00640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731314e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.960+0000 7f24cd400640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731315410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24ce800640 1 -- 172.21.15.114:0/2798368496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 msgr2=0x560731314e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24ce800640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731314e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24ce800640 1 -- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 msgr2=0x560731315410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24ce800640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731315410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24ce800640 1 -- 172.21.15.114:0/2798368496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560730fcdb00 con 0x560730f62520 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24cd400640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731315410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:44.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24ce800640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x560731320ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f24c8045000 tx=0x7f24c8003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9305e91985ffd039 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24afe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.961+0000 7f24afe00640 1 -- 172.21.15.114:0/2798368496 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c8060060 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c8036030 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24afe00640 1 -- 172.21.15.114:0/2798368496 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f24c802f870 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24afe00640 1 -- 172.21.15.114:0/2798368496 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f24c8067050 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 10 client.17343 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560731044960 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24afe00640 1 -- 172.21.15.114:0/2798368496 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c8036030 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24afe00640 1 -- 172.21.15.114:0/2798368496 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f24c804d030 con 0x560730f62520 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 10 client.17343 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 10 client.17343 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 20 client.17343 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:44.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730f20ea0 0x560731326a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 4 client.17343 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:44.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.962+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607312821e0 con 0x560730f20ea0 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.963+0000 7f24cde00640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730f20ea0 0x560731326a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.963+0000 7f24cde00640 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730f20ea0 0x560731326a10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316ec10d76ce6c68 server_cookie=d76362c59e490b03 in_seq=0 out_seq=0 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.963+0000 7f24afe00640 10 client.17343 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.964+0000 7f24afe00640 1 -- 172.21.15.114:0/2798368496 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607312821e0 con 0x560730f20ea0 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.964+0000 7f24afe00640 10 client.17343 handle_command_reply: tid=0 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.964+0000 7f24dbfd7740 1 client.17343 shutdown 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 msgr2=0x560731320ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x560731320ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f24c8045000 tx=0x7f24c8003d70 comp rx=0 tx=0).stop 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 shutdown_connections 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x560731320ab0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x560731315410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730f20ea0 0x560731326a10 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/2798368496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x560731314e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 >> 172.21.15.114:0/2798368496 conn(0x560730f56910 msgr2=0x56073130c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:44.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24afe00640 0 client.17343 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:44.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 shutdown_connections 2024-09-21T07:02:44.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 1 -- 172.21.15.114:0/2798368496 wait complete. 2024-09-21T07:02:44.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.965+0000 7f24dbfd7740 20 client.17343 trim_cache size 0 max 16384 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 Processor -- start 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 -- start start 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f39660 0x5607313071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 0x56073130e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f62520 0x56073130e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b3180 con 0x560730f41840 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b4d20 con 0x560730f39660 2024-09-21T07:02:44.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b3850 con 0x560730f62520 2024-09-21T07:02:44.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24cd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 0x56073130e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24cde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f39660 0x5607313071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24cd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 0x56073130e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59786/0 (socket says 172.21.15.114:59786) 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.983+0000 7f24cd400640 1 -- 172.21.15.114:0/1613347131 learned_addr learned my addr 172.21.15.114:0/1613347131 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24ce800640 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f62520 0x56073130e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cde00640 1 -- 172.21.15.114:0/1613347131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f62520 msgr2=0x56073130e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cde00640 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f62520 0x56073130e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cde00640 1 -- 172.21.15.114:0/1613347131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 msgr2=0x56073130e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cde00640 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 0x56073130e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cde00640 1 -- 172.21.15.114:0/1613347131 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073130f940 con 0x560730f39660 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cd400640 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 0x56073130e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.984+0000 7f24cde00640 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f39660 0x5607313071b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24c00e4870 tx=0x7f24c00a9690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6170c372ded5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24baa00640 1 -- 172.21.15.114:0/1613347131 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00ad670 con 0x560730f39660 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24baa00640 1 -- 172.21.15.114:0/1613347131 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00b1a50 con 0x560730f39660 2024-09-21T07:02:44.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24baa00640 1 -- 172.21.15.114:0/1613347131 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00a91e0 con 0x560730f39660 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 -- 172.21.15.114:0/1613347131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f39660 msgr2=0x5607313071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f39660 0x5607313071b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f24c00e4870 tx=0x7f24c00a9690 comp rx=0 tx=0).stop 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 -- 172.21.15.114:0/1613347131 shutdown_connections 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f62520 0x56073130e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f41840 0x56073130e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/1613347131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f39660 0x5607313071b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.985+0000 7f24dbfd7740 1 -- 172.21.15.114:0/1613347131 >> 172.21.15.114:0/1613347131 conn(0x560730f56910 msgr2=0x560731306aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.986+0000 7f24dbfd7740 1 -- 172.21.15.114:0/1613347131 shutdown_connections 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.986+0000 7f24dbfd7740 1 -- 172.21.15.114:0/1613347131 wait complete. 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.986+0000 7f24dbfd7740 1 -- start start 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.986+0000 7f24dbfd7740 1 Processor -- start 2024-09-21T07:02:44.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x56073131cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x56073131d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b4650 con 0x560730f62520 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b3f20 con 0x560730f41840 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24d01b1180 con 0x560730f39660 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24dbfd7740 10 client.0 _ll_register_callbacks cb 0x7f24bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24cd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x56073131d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24ce800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24cd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x56073131d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41010/0 (socket says 172.21.15.114:41010) 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24cde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x56073131cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.987+0000 7f24cd400640 1 -- 172.21.15.114:0/853855933 learned_addr learned my addr 172.21.15.114:0/853855933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24cd400640 1 -- 172.21.15.114:0/853855933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 msgr2=0x56073131cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24cd400640 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x56073131cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24cd400640 1 -- 172.21.15.114:0/853855933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 msgr2=0x5607313179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24cd400640 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24cd400640 1 -- 172.21.15.114:0/853855933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560731318070 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24ce800640 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24cd400640 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x56073131d180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f24bc010820 tx=0x7f24bc010ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5c144bb700d9eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24afe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24afe00640 1 -- 172.21.15.114:0/853855933 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc022060 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24afe00640 1 -- 172.21.15.114:0/853855933 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f24bc00fd30 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.988+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24c00b1a50 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24afe00640 1 -- 172.21.15.114:0/853855933 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f24bc029050 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 10 client.27755 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560731391c20 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24afe00640 1 -- 172.21.15.114:0/853855933 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24c00b1a50 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24afe00640 1 -- 172.21.15.114:0/853855933 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f24bc00e3b0 con 0x560730f41840 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 10 client.27755 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 10 client.27755 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:02:44.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 20 client.27755 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730ef6250 0x560731394440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 4 client.27755 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2d786875", "release": true, "await": true}] 2024-09-21T07:02:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.989+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2d786875", "release": true, "await": true}) -- 0x56073127dc90 con 0x560730ef6250 2024-09-21T07:02:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.990+0000 7f24cde00640 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730ef6250 0x560731394440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.990+0000 7f24cde00640 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730ef6250 0x560731394440 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b895f05a172d18 server_cookie=8b1fe8ad0b4c2342 in_seq=0 out_seq=0 2024-09-21T07:02:44.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:44.990+0000 7f24afe00640 10 client.27755 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:45.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.021+0000 7f24afe00640 1 -- 172.21.15.114:0/853855933 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56073127dc90 con 0x560730ef6250 2024-09-21T07:02:45.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.021+0000 7f24afe00640 10 client.27755 handle_command_reply: tid=0 2024-09-21T07:02:45.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.021+0000 7f24dbfd7740 1 client.27755 shutdown 2024-09-21T07:02:45.024 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:44 smithi114 ceph-mon[30654]: pgmap v658: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 569 KiB/s wr, 62 op/s 2024-09-21T07:02:45.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.022+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 msgr2=0x56073131d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:45.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.022+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x56073131d180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f24bc010820 tx=0x7f24bc010ce0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 shutdown_connections 2024-09-21T07:02:45.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x560730f62520 0x5607313179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560730f41840 0x56073131d180 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560730ef6250 0x560731394440 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 --2- 172.21.15.114:0/853855933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560730f39660 0x56073131cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24afe00640 0 client.27755 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:45.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 >> 172.21.15.114:0/853855933 conn(0x560730f56910 msgr2=0x56073130e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:45.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.023+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 shutdown_connections 2024-09-21T07:02:45.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.024+0000 7f24dbfd7740 1 -- 172.21.15.114:0/853855933 wait complete. 2024-09-21T07:02:45.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.024+0000 7f24dbfd7740 20 client.27755 trim_cache size 0 max 16384 2024-09-21T07:02:45.027 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:45.027 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 47, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 36, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "2d786875": { 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 36, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 141.1, 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:02:45.028 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:45.029 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24b40b6680 msgr2=0x7f24b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24b40b6680 0x7f24b40b8b90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 msgr2=0x7f24d019c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 0x7f24d019c790 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).stop 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 shutdown_connections 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f24b40b6680 0x7f24b40b8b90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d010c020 0x7f24d01a1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f24d010b6d0 0x7f24d019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 --2- 172.21.15.114:0/3063931922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d0105d90 0x7f24d019c790 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:45.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.025+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 >> 172.21.15.114:0/3063931922 conn(0x7f24d0101860 msgr2=0x7f24d010ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:45.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.026+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 shutdown_connections 2024-09-21T07:02:45.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:45.026+0000 7f24d5200640 1 -- 172.21.15.114:0/3063931922 wait complete. 2024-09-21T07:02:45.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:44 smithi193 ceph-mon[34601]: pgmap v658: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 569 KiB/s wr, 62 op/s 2024-09-21T07:02:45.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:44 smithi183 ceph-mon[34219]: pgmap v658: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 569 KiB/s wr, 62 op/s 2024-09-21T07:02:45.991 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2d786875', total seconds elapsed: 25.3 2024-09-21T07:02:46.157 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:02:46.879 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:47.278 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:46 smithi114 ceph-mon[30654]: pgmap v659: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-21T07:02:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:46 smithi183 ceph-mon[34219]: pgmap v659: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-21T07:02:47.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:46 smithi193 ceph-mon[34601]: pgmap v659: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-21T07:02:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 -- 172.21.15.114:0/1617334343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c101f70 msgr2=0x7f3a0c102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 --2- 172.21.15.114:0/1617334343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c101f70 0x7f3a0c102380 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009900 tx=0x7f3a0002a410 comp rx=0 tx=0).stop 2024-09-21T07:02:47.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 -- 172.21.15.114:0/1617334343 shutdown_connections 2024-09-21T07:02:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 --2- 172.21.15.114:0/1617334343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c109e00 0x7f3a0c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 --2- 172.21.15.114:0/1617334343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 --2- 172.21.15.114:0/1617334343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c101f70 0x7f3a0c102380 secure :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009900 tx=0x7f3a0002a410 comp rx=0 tx=0).stop 2024-09-21T07:02:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.772+0000 7f3a13a00640 1 -- 172.21.15.114:0/1617334343 >> 172.21.15.114:0/1617334343 conn(0x7f3a0c077c60 msgr2=0x7f3a0c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.774+0000 7f3a13a00640 1 -- 172.21.15.114:0/1617334343 shutdown_connections 2024-09-21T07:02:47.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.774+0000 7f3a13a00640 1 -- 172.21.15.114:0/1617334343 wait complete. 2024-09-21T07:02:47.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.777+0000 7f3a13a00640 1 Processor -- start 2024-09-21T07:02:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.777+0000 7f3a13a00640 1 -- start start 2024-09-21T07:02:47.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.777+0000 7f3a13a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.778+0000 7f3a13a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109e00 0x7f3a0c103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.778+0000 7f3a13a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c103870 0x7f3a0c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.778+0000 7f3a13a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c113b10 con 0x7f3a0c1094b0 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.778+0000 7f3a13a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c113990 con 0x7f3a0c103870 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.778+0000 7f3a13a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c113c90 con 0x7f3a0c109e00 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c102df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59808/0 (socket says 172.21.15.114:59808) 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 -- 172.21.15.114:0/3782004058 learned_addr learned my addr 172.21.15.114:0/3782004058 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 -- 172.21.15.114:0/3782004058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109e00 msgr2=0x7f3a0c103330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109e00 0x7f3a0c103330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 -- 172.21.15.114:0/3782004058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c103870 msgr2=0x7f3a0c0741e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c103870 0x7f3a0c0741e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.779+0000 7f3a11000640 1 -- 172.21.15.114:0/3782004058 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c074720 con 0x7f3a0c1094b0 2024-09-21T07:02:47.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.780+0000 7f3a11000640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c102df0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a00005370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:47.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.783+0000 7f3a09600640 1 -- 172.21.15.114:0/3782004058 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00040030 con 0x7f3a0c1094b0 2024-09-21T07:02:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.786+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c0749b0 con 0x7f3a0c1094b0 2024-09-21T07:02:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.786+0000 7f3a09600640 1 -- 172.21.15.114:0/3782004058 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0003c8f0 con 0x7f3a0c1094b0 2024-09-21T07:02:47.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.786+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c074e10 con 0x7f3a0c1094b0 2024-09-21T07:02:47.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.787+0000 7f3a09600640 1 -- 172.21.15.114:0/3782004058 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00031860 con 0x7f3a0c1094b0 2024-09-21T07:02:47.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.792+0000 7f3a1a637740 1 Processor -- start 2024-09-21T07:02:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.792+0000 7f3a1a637740 1 -- start start 2024-09-21T07:02:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.792+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 0x5572f0da0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d11b0 0x5572f0da10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a11a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 0x5572f0da0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 0x5572f0da15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a1a637740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5572f0d17f70 con 0x5572f09d9390 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a1a637740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572f0d073a0 con 0x5572f09d11b0 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a1a637740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572f0d0b1b0 con 0x5572f09fa070 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a11a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 0x5572f0da0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39078/0 (socket says 172.21.15.114:39078) 2024-09-21T07:02:47.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.793+0000 7f3a11a00640 1 -- 172.21.15.114:0/1876362993 learned_addr learned my addr 172.21.15.114:0/1876362993 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.794+0000 7f3a11a00640 1 -- 172.21.15.114:0/1876362993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d11b0 msgr2=0x5572f0da10a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:02:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.794+0000 7f3a11a00640 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d11b0 0x5572f0da10a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.794+0000 7f3a11a00640 1 -- 172.21.15.114:0/1876362993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 msgr2=0x5572f0da15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.794+0000 7f3a11a00640 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 0x5572f0da15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.794+0000 7f3a11a00640 1 -- 172.21.15.114:0/1876362993 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f0cd1c90 con 0x5572f09fa070 2024-09-21T07:02:47.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.795+0000 7f3a11a00640 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 0x5572f0da0b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e30 tx=0x7f3a0400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cac151f388d4ab5a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:47.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.796+0000 7f39fe000640 1 -- 172.21.15.114:0/1876362993 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04004d20 con 0x5572f09fa070 2024-09-21T07:02:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.797+0000 7f39fe000640 1 -- 172.21.15.114:0/1876362993 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0400ee70 con 0x5572f09fa070 2024-09-21T07:02:47.800 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.797+0000 7f39fe000640 1 -- 172.21.15.114:0/1876362993 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040107b0 con 0x5572f09fa070 2024-09-21T07:02:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.799+0000 7f3a1a637740 1 -- 172.21.15.114:0/1876362993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 msgr2=0x5572f0da0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.799+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 0x5572f0da0b60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009e30 tx=0x7f3a0400e720 comp rx=0 tx=0).stop 2024-09-21T07:02:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.799+0000 7f3a1a637740 1 -- 172.21.15.114:0/1876362993 shutdown_connections 2024-09-21T07:02:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.799+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 0x5572f0da15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.799+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d11b0 0x5572f0da10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.799+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1876362993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09fa070 0x5572f0da0b60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.800+0000 7f3a1a637740 1 -- 172.21.15.114:0/1876362993 >> 172.21.15.114:0/1876362993 conn(0x5572f09ee460 msgr2=0x5572f0d9ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:47.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.806+0000 7f3a09600640 1 -- 172.21.15.114:0/3782004058 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f3a00043030 con 0x7f3a0c1094b0 2024-09-21T07:02:47.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.810+0000 7f3a09600640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b67d0 0x7f39f40b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.811+0000 7f3a09600640 1 -- 172.21.15.114:0/3782004058 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f3a001082f0 con 0x7f3a0c1094b0 2024-09-21T07:02:47.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.811+0000 7f3a0be00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b67d0 0x7f39f40b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.812+0000 7f3a1a637740 1 -- 172.21.15.114:0/1876362993 shutdown_connections 2024-09-21T07:02:47.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.812+0000 7f3a1a637740 1 -- 172.21.15.114:0/1876362993 wait complete. 2024-09-21T07:02:47.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.813+0000 7f3a1a637740 1 -- start start 2024-09-21T07:02:47.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.813+0000 7f3a1a637740 1 Processor -- start 2024-09-21T07:02:47.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.815+0000 7f3a0be00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b67d0 0x7f39f40b8ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c109230 tx=0x7f39f8005740 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:47.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.817+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 0x5572f0dac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.817+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09fa070 0x5572f0dacc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0db86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a1a637740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5572f0b68700 con 0x5572f09d9390 2024-09-21T07:02:47.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a1a637740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572f0ca2540 con 0x5572f09fa070 2024-09-21T07:02:47.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09fa070 0x5572f0dacc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a1a637740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572f0daf340 con 0x5572f09b89f0 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a11a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0db86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a11a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0db86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:39086/0 (socket says 172.21.15.114:39086) 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a11a00640 1 -- 172.21.15.114:0/745112287 learned_addr learned my addr 172.21.15.114:0/745112287 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.818+0000 7f3a1a637740 10 client.0 _ll_register_callbacks cb 0x5572f0cb2430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f3a11a00640 1 -- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09fa070 msgr2=0x5572f0dacc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f3a11a00640 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09fa070 0x5572f0dacc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f3a11a00640 1 -- 172.21.15.114:0/745112287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 msgr2=0x5572f0dac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f3a11a00640 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 0x5572f0dac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f3a11a00640 1 -- 172.21.15.114:0/745112287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f0a65650 con 0x5572f09b89f0 2024-09-21T07:02:47.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f39f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:47.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.819+0000 7f3a11a00640 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0db86c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009240 tx=0x7f3a04005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0dccafd5bb89eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:47.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.820+0000 7f39f2a00640 1 -- 172.21.15.114:0/745112287 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04026020 con 0x5572f09b89f0 2024-09-21T07:02:47.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.820+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572f0ca4b50 con 0x5572f09b89f0 2024-09-21T07:02:47.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.820+0000 7f39f2a00640 1 -- 172.21.15.114:0/745112287 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f3a040053c0 con 0x5572f09b89f0 2024-09-21T07:02:47.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.821+0000 7f39f2a00640 1 -- 172.21.15.114:0/745112287 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3a04026020 con 0x5572f09b89f0 2024-09-21T07:02:47.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.821+0000 7f3a1a637740 10 client.27703 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:47.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.821+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5572f0cbf910 con 0x5572f09b89f0 2024-09-21T07:02:47.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.821+0000 7f39f2a00640 1 -- 172.21.15.114:0/745112287 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5572f0ca4b50 con 0x5572f09b89f0 2024-09-21T07:02:47.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.821+0000 7f39f2a00640 1 -- 172.21.15.114:0/745112287 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f3a04010ca0 con 0x5572f09b89f0 2024-09-21T07:02:47.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a1a637740 10 client.27703 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:47.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a1a637740 10 client.27703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:47.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a1a637740 20 client.27703 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a1a637740 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f098dda0 0x5572f0dbe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a1a637740 4 client.27703 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f0d19ec0 con 0x5572f098dda0 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.822+0000 7f3a11000640 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f098dda0 0x5572f0dbe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.823+0000 7f3a11000640 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f098dda0 0x5572f0dbe500 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1624246ae5492bb server_cookie=4dade59c70c4cb04 in_seq=0 out_seq=0 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.823+0000 7f39f2a00640 10 client.27703 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.824+0000 7f39f2a00640 1 -- 172.21.15.114:0/745112287 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572f0d19ec0 con 0x5572f098dda0 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.824+0000 7f39f2a00640 10 client.27703 handle_command_reply: tid=0 2024-09-21T07:02:47.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.824+0000 7f3a1a637740 1 client.27703 shutdown 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 msgr2=0x5572f0db86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0db86c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009240 tx=0x7f3a04005250 comp rx=0 tx=0).stop 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 shutdown_connections 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f098dda0 0x5572f0dbe500 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0db86c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09fa070 0x5572f0dacc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 --2- 172.21.15.114:0/745112287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09d9390 0x5572f0dac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 >> 172.21.15.114:0/745112287 conn(0x5572f09ee460 msgr2=0x5572f0da3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:47.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.825+0000 7f39f2a00640 0 client.27703 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:47.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.829+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 shutdown_connections 2024-09-21T07:02:47.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.829+0000 7f3a1a637740 1 -- 172.21.15.114:0/745112287 wait complete. 2024-09-21T07:02:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.829+0000 7f3a1a637740 20 client.27703 trim_cache size 0 max 16384 2024-09-21T07:02:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 490.0 s, 385.7 tps, lat 975.847 ms stddev 3578.665 2024-09-21T07:02:47.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.860+0000 7f3a1a637740 1 Processor -- start 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 -- start start 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0dab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 0x5572f0dace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec0100e0 con 0x5572f09fa070 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec011c80 con 0x5572f09d9390 2024-09-21T07:02:47.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a1a637740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec0107b0 con 0x5572f09b89f0 2024-09-21T07:02:47.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.861+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 0x5572f0dace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.862+0000 7f3a0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 0x5572f0dace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41050/0 (socket says 172.21.15.114:41050) 2024-09-21T07:02:47.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.862+0000 7f3a0be00640 1 -- 172.21.15.114:0/1910541407 learned_addr learned my addr 172.21.15.114:0/1910541407 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:47.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.862+0000 7f3a0be00640 1 -- 172.21.15.114:0/1910541407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 msgr2=0x5572f0dab440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:02:47.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.862+0000 7f3a11000640 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0dab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.863+0000 7f3a0be00640 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0dab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.863+0000 7f3a0be00640 1 -- 172.21.15.114:0/1910541407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 msgr2=0x5572f0da13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:47.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.863+0000 7f3a0be00640 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.863+0000 7f3a0be00640 1 -- 172.21.15.114:0/1910541407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f0db96b0 con 0x5572f09d9390 2024-09-21T07:02:47.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.863+0000 7f3a11000640 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0dab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:47.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.864+0000 7f3a0be00640 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 0x5572f0dace20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f39f800eda0 tx=0x7f39f8011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185d017a22a4778e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:47.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.864+0000 7f39fd600640 1 -- 172.21.15.114:0/1910541407 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f802a070 con 0x5572f09d9390 2024-09-21T07:02:47.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.864+0000 7f39fd600640 1 -- 172.21.15.114:0/1910541407 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f80108e0 con 0x5572f09d9390 2024-09-21T07:02:47.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.864+0000 7f39fd600640 1 -- 172.21.15.114:0/1910541407 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8025940 con 0x5572f09d9390 2024-09-21T07:02:47.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 -- 172.21.15.114:0/1910541407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 msgr2=0x5572f0dace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 0x5572f0dace20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f39f800eda0 tx=0x7f39f8011000 comp rx=0 tx=0).stop 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 -- 172.21.15.114:0/1910541407 shutdown_connections 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09d9390 0x5572f0dace20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1910541407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09b89f0 0x5572f0dab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 -- 172.21.15.114:0/1910541407 >> 172.21.15.114:0/1910541407 conn(0x5572f09ee460 msgr2=0x5572f0dac710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:47.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.865+0000 7f3a1a637740 1 -- 172.21.15.114:0/1910541407 shutdown_connections 2024-09-21T07:02:47.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.866+0000 7f3a1a637740 1 -- 172.21.15.114:0/1910541407 wait complete. 2024-09-21T07:02:47.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.866+0000 7f3a1a637740 1 -- start start 2024-09-21T07:02:47.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.866+0000 7f3a1a637740 1 Processor -- start 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 0x5572f0db4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09d9390 0x5572f0da94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec0115b0 con 0x5572f09fa070 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec010e80 con 0x5572f09b89f0 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f39ec00e080 con 0x5572f09d9390 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a1a637740 10 client.0 _ll_register_callbacks cb 0x7f3a04004ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:47.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a11000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 0x5572f0db4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a11000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 0x5572f0db4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41062/0 (socket says 172.21.15.114:41062) 2024-09-21T07:02:47.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a11000640 1 -- 172.21.15.114:0/1373483653 learned_addr learned my addr 172.21.15.114:0/1373483653 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:47.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a0be00640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09d9390 0x5572f0da94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.867+0000 7f3a11a00640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11000640 1 -- 172.21.15.114:0/1373483653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09d9390 msgr2=0x5572f0da94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11000640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09d9390 0x5572f0da94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11000640 1 -- 172.21.15.114:0/1373483653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 msgr2=0x5572f0da9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11000640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11000640 1 -- 172.21.15.114:0/1373483653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f0daa1d0 con 0x5572f09b89f0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11a00640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a11000640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 0x5572f0db4e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3a000cda70 tx=0x7f3a000054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b54ef9d9d2ce46d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f39f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f39f2a00640 1 -- 172.21.15.114:0/1373483653 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0004a040 con 0x5572f09b89f0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f39f2a00640 1 -- 172.21.15.114:0/1373483653 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f3a000d5dd0 con 0x5572f09b89f0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.868+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39f80108e0 con 0x5572f09b89f0 2024-09-21T07:02:47.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f39f2a00640 1 -- 172.21.15.114:0/1373483653 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3a000cd880 con 0x5572f09b89f0 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 10 client.27767 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5572f0e28d00 con 0x5572f09b89f0 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f39f2a00640 1 -- 172.21.15.114:0/1373483653 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f80108e0 con 0x5572f09b89f0 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f39f2a00640 1 -- 172.21.15.114:0/1373483653 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f3a000f5120 con 0x5572f09b89f0 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 10 client.27767 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 10 client.27767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 20 client.27767 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f0e29330 0x5572f0e2b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 4 client.27767 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.869+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572f0d15970 con 0x5572f0e29330 2024-09-21T07:02:47.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.870+0000 7f3a0be00640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f0e29330 0x5572f0e2b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.870+0000 7f3a0be00640 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f0e29330 0x5572f0e2b750 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be238c65a72197b6 server_cookie=fdad850d9f427665 in_seq=0 out_seq=0 2024-09-21T07:02:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.870+0000 7f39f2a00640 10 client.27767 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.871+0000 7f39f2a00640 1 -- 172.21.15.114:0/1373483653 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5572f0d15970 con 0x5572f0e29330 2024-09-21T07:02:47.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.871+0000 7f39f2a00640 10 client.27767 handle_command_reply: tid=0 2024-09-21T07:02:47.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.872+0000 7f3a1a637740 1 client.27767 shutdown 2024-09-21T07:02:47.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.872+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 msgr2=0x5572f0db4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.872+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 0x5572f0db4e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3a000cda70 tx=0x7f3a000054e0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 shutdown_connections 2024-09-21T07:02:47.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5572f09fa070 0x5572f0da9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5572f0e29330 0x5572f0e2b750 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572f09d9390 0x5572f0da94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f3a1a637740 1 --2- 172.21.15.114:0/1373483653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572f09b89f0 0x5572f0db4e30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 >> 172.21.15.114:0/1373483653 conn(0x5572f09ee460 msgr2=0x5572f0d9e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:47.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.873+0000 7f39f2a00640 0 client.27767 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.874+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 shutdown_connections 2024-09-21T07:02:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.874+0000 7f3a1a637740 1 -- 172.21.15.114:0/1373483653 wait complete. 2024-09-21T07:02:47.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.874+0000 7f3a1a637740 20 client.27767 trim_cache size 0 max 16384 2024-09-21T07:02:47.878 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:02:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:02:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:02:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:02:47.878 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.880 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.881 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.882 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:02:47.883 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b67d0 msgr2=0x7f39f40b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b67d0 0x7f39f40b8ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c109230 tx=0x7f39f8005740 comp rx=0 tx=0).stop 2024-09-21T07:02:47.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 msgr2=0x7f3a0c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c102df0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006010 tx=0x7f3a00005370 comp rx=0 tx=0).stop 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 shutdown_connections 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f39f40b67d0 0x7f39f40b8ce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a0c103870 0x7f3a0c0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a0c109e00 0x7f3a0c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 --2- 172.21.15.114:0/3782004058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3a0c1094b0 0x7f3a0c102df0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.875+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 >> 172.21.15.114:0/3782004058 conn(0x7f3a0c077c60 msgr2=0x7f3a0c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.876+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 shutdown_connections 2024-09-21T07:02:47.885 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:47.876+0000 7f3a13a00640 1 -- 172.21.15.114:0/3782004058 wait complete. 2024-09-21T07:02:48.887 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:02:48.888 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:02:49.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:48 smithi114 ceph-mon[30654]: pgmap v660: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 854 KiB/s wr, 86 op/s 2024-09-21T07:02:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:48 smithi183 ceph-mon[34219]: pgmap v660: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 854 KiB/s wr, 86 op/s 2024-09-21T07:02:49.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:48 smithi193 ceph-mon[34601]: pgmap v660: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 854 KiB/s wr, 86 op/s 2024-09-21T07:02:49.532 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:51.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 -- 172.21.15.114:0/2524056187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10a640 msgr2=0x7fdd0c10aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:51.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 --2- 172.21.15.114:0/2524056187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10a640 0x7fdd0c10aa50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ca0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 -- 172.21.15.114:0/2524056187 shutdown_connections 2024-09-21T07:02:51.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 --2- 172.21.15.114:0/2524056187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd0c116510 0x7fdd0c118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 --2- 172.21.15.114:0/2524056187 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c10af90 0x7fdd0c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 --2- 172.21.15.114:0/2524056187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10a640 0x7fdd0c10aa50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 -- 172.21.15.114:0/2524056187 >> 172.21.15.114:0/2524056187 conn(0x7fdd0c06e9f0 msgr2=0x7fdd0c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:51.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 -- 172.21.15.114:0/2524056187 shutdown_connections 2024-09-21T07:02:51.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.026+0000 7fdd12400640 1 -- 172.21.15.114:0/2524056187 wait complete. 2024-09-21T07:02:51.032 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:50 smithi114 ceph-mon[30654]: pgmap v661: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1005 KiB/s wr, 102 op/s 2024-09-21T07:02:51.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.031+0000 7fdd12400640 1 Processor -- start 2024-09-21T07:02:51.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.031+0000 7fdd12400640 1 -- start start 2024-09-21T07:02:51.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd12400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd0c10a640 0x7fdd0c115f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:51.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd12400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10af90 0x7fdd0c1131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:51.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd12400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c116510 0x7fdd0c1138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:51.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd12400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10d590 con 0x7fdd0c116510 2024-09-21T07:02:51.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd12400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10d410 con 0x7fdd0c10af90 2024-09-21T07:02:51.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd12400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10d710 con 0x7fdd0c10a640 2024-09-21T07:02:51.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.032+0000 7fdd11a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c116510 0x7fdd0c1138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:51.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd11000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd0c10a640 0x7fdd0c115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:51.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd11a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c116510 0x7fdd0c1138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59854/0 (socket says 172.21.15.114:59854) 2024-09-21T07:02:51.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd11a00640 1 -- 172.21.15.114:0/1799650424 learned_addr learned my addr 172.21.15.114:0/1799650424 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:51.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10af90 0x7fdd0c1131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:51.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd0be00640 1 -- 172.21.15.114:0/1799650424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd0c10a640 msgr2=0x7fdd0c115f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:51.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd0be00640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd0c10a640 0x7fdd0c115f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.033+0000 7fdd0be00640 1 -- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c116510 msgr2=0x7fdd0c1138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:51.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.034+0000 7fdd0be00640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c116510 0x7fdd0c1138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.034+0000 7fdd0be00640 1 -- 172.21.15.114:0/1799650424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c1b5280 con 0x7fdd0c10af90 2024-09-21T07:02:51.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.034+0000 7fdd0be00640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10af90 0x7fdd0c1131f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c06c910 tx=0x7fdcf800db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:51.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.035+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8017070 con 0x7fdd0c10af90 2024-09-21T07:02:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.035+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf8009820 con 0x7fdd0c10af90 2024-09-21T07:02:51.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.035+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8004b90 con 0x7fdd0c10af90 2024-09-21T07:02:51.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.039+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1b54b0 con 0x7fdd0c10af90 2024-09-21T07:02:51.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.039+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c1b5970 con 0x7fdd0c10af90 2024-09-21T07:02:51.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.041+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c10aa50 con 0x7fdd0c10af90 2024-09-21T07:02:51.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.047+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fdcf8004d30 con 0x7fdd0c10af90 2024-09-21T07:02:51.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.052+0000 7fdd09600640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdcf00b6730 0x7fdcf00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:51.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.053+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fdcf80e79e0 con 0x7fdd0c10af90 2024-09-21T07:02:51.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.053+0000 7fdd11000640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdcf00b6730 0x7fdcf00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:51.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.053+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdcf8020020 con 0x7fdd0c10af90 2024-09-21T07:02:51.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.079+0000 7fdd11000640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdcf00b6730 0x7fdcf00b8c40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c115c50 tx=0x7fdd04005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:50 smithi183 ceph-mon[34219]: pgmap v661: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1005 KiB/s wr, 102 op/s 2024-09-21T07:02:51.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:50 smithi193 ceph-mon[34601]: pgmap v661: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 1005 KiB/s wr, 102 op/s 2024-09-21T07:02:51.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.504+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd0c1b5cb0 con 0x7fdd0c10af90 2024-09-21T07:02:51.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.506+0000 7fdd09600640 1 -- 172.21.15.114:0/1799650424 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11992 (secure 0 0 0) 0x7fdcf80099d0 con 0x7fdd0c10af90 2024-09-21T07:02:51.508 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:02:51.509 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":47,"btime":"2024-09-21T07:02:02:992961+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:02:02.920775+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:02:51.509 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 47 2024-09-21T07:02:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdcf00b6730 msgr2=0x7fdcf00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdcf00b6730 0x7fdcf00b8c40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c115c50 tx=0x7fdd04005fd0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10af90 msgr2=0x7fdd0c1131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:51.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10af90 0x7fdd0c1131f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c06c910 tx=0x7fdcf800db30 comp rx=0 tx=0).stop 2024-09-21T07:02:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 shutdown_connections 2024-09-21T07:02:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdd0c116510 0x7fdd0c1138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdcf00b6730 0x7fdcf00b8c40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd0c10af90 0x7fdd0c1131f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 --2- 172.21.15.114:0/1799650424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd0c10a640 0x7fdd0c115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:51.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.510+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 >> 172.21.15.114:0/1799650424 conn(0x7fdd0c06e9f0 msgr2=0x7fdd0c072cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:51.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.511+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 shutdown_connections 2024-09-21T07:02:51.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:51.511+0000 7fdd12400640 1 -- 172.21.15.114:0/1799650424 wait complete. 2024-09-21T07:02:52.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:51 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1799650424' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:52.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:51 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1799650424' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:52.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:51 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1799650424' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:02:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 495.0 s, 482.2 tps, lat 209.120 ms stddev 260.711 2024-09-21T07:02:53.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:52 smithi114 ceph-mon[30654]: pgmap v662: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 -- 172.21.15.114:0/2544298025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e841100a0 msgr2=0x7f1e841124c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 --2- 172.21.15.114:0/2544298025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e841100a0 0x7f1e841124c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 -- 172.21.15.114:0/2544298025 shutdown_connections 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 --2- 172.21.15.114:0/2544298025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e841100a0 0x7f1e841124c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 --2- 172.21.15.114:0/2544298025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e84076080 0x7f1e8410fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 --2- 172.21.15.114:0/2544298025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e84075730 0x7f1e84075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 -- 172.21.15.114:0/2544298025 >> 172.21.15.114:0/2544298025 conn(0x7f1e840ff510 msgr2=0x7f1e84101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 -- 172.21.15.114:0/2544298025 shutdown_connections 2024-09-21T07:02:53.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.342+0000 7f1e8a400640 1 -- 172.21.15.114:0/2544298025 wait complete. 2024-09-21T07:02:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.343+0000 7f1e8a400640 1 Processor -- start 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.343+0000 7f1e8a400640 1 -- start start 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e8a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 0x7f1e84107540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e8a400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e84076080 0x7f1e8410fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e8a400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e841100a0 0x7f1e84103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e8a400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8406d6a0 con 0x7f1e841100a0 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e8a400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8406d520 con 0x7f1e84075730 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e8a400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8406d820 con 0x7f1e84076080 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e83400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 0x7f1e84107540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e83400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 0x7f1e84107540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41102/0 (socket says 172.21.15.114:41102) 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e841100a0 0x7f1e84103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.346 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e83400640 1 -- 172.21.15.114:0/3449356689 learned_addr learned my addr 172.21.15.114:0/3449356689 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:53.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e82a00640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e84076080 0x7f1e8410fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.344+0000 7f1e83400640 1 -- 172.21.15.114:0/3449356689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e84076080 msgr2=0x7f1e8410fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.345+0000 7f1e83400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e84076080 0x7f1e8410fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.345+0000 7f1e83400640 1 -- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e841100a0 msgr2=0x7f1e84103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.345+0000 7f1e83400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e841100a0 0x7f1e84103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.345+0000 7f1e83400640 1 -- 172.21.15.114:0/3449356689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e841040f0 con 0x7f1e84075730 2024-09-21T07:02:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.345+0000 7f1e83400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 0x7f1e84107540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000c920 tx=0x7f1e7000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.346+0000 7f1e7be00640 1 -- 172.21.15.114:0/3449356689 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70007c70 con 0x7f1e84075730 2024-09-21T07:02:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.346+0000 7f1e7be00640 1 -- 172.21.15.114:0/3449356689 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e70007e10 con 0x7f1e84075730 2024-09-21T07:02:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.346+0000 7f1e7be00640 1 -- 172.21.15.114:0/3449356689 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70018820 con 0x7f1e84075730 2024-09-21T07:02:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.346+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e841043e0 con 0x7f1e84075730 2024-09-21T07:02:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.346+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841b5a90 con 0x7f1e84075730 2024-09-21T07:02:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.349+0000 7f1e91442740 1 Processor -- start 2024-09-21T07:02:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 -- start start 2024-09-21T07:02:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe9132c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d652f0 0x555fe9133170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d6d4d0 0x555fe91336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555fe90a86b0 con 0x555fe8d6d4d0 2024-09-21T07:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555fe9097ae0 con 0x555fe8d8e1b0 2024-09-21T07:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e91442740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555fe909b8f0 con 0x555fe8d652f0 2024-09-21T07:02:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e7be00640 1 -- 172.21.15.114:0/3449356689 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1e700189c0 con 0x7f1e84075730 2024-09-21T07:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe9132c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe9132c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41108/0 (socket says 172.21.15.114:41108) 2024-09-21T07:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.350+0000 7f1e83e00640 1 -- 172.21.15.114:0/1525957815 learned_addr learned my addr 172.21.15.114:0/1525957815 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83400640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d652f0 0x555fe9133170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83e00640 1 -- 172.21.15.114:0/1525957815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d652f0 msgr2=0x555fe9133170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e82a00640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d6d4d0 0x555fe91336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d652f0 0x555fe9133170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83e00640 1 -- 172.21.15.114:0/1525957815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d6d4d0 msgr2=0x555fe91336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d6d4d0 0x555fe91336b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83e00640 1 -- 172.21.15.114:0/1525957815 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe9064780 con 0x555fe8d8e1b0 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83400640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d652f0 0x555fe9133170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e82a00640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d6d4d0 0x555fe91336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.351+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe9132c30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c035000 tx=0x7f1e7c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34101a06c9f9d3ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.352+0000 7f1e78c00640 1 -- 172.21.15.114:0/1525957815 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c030a00 con 0x555fe8d8e1b0 2024-09-21T07:02:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.352+0000 7f1e78c00640 1 -- 172.21.15.114:0/1525957815 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c036070 con 0x555fe8d8e1b0 2024-09-21T07:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.352+0000 7f1e78c00640 1 -- 172.21.15.114:0/1525957815 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c0045a0 con 0x555fe8d8e1b0 2024-09-21T07:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.352+0000 7f1e7be00640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e680b6700 0x7f1e680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.353+0000 7f1e7be00640 1 -- 172.21.15.114:0/3449356689 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1e700e7cd0 con 0x7f1e84075730 2024-09-21T07:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.353+0000 7f1e83400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e680b6700 0x7f1e680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.354+0000 7f1e91442740 1 -- 172.21.15.114:0/1525957815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 msgr2=0x555fe9132c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.354+0000 7f1e91442740 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe9132c30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c035000 tx=0x7f1e7c0099e0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.356+0000 7f1e91442740 1 -- 172.21.15.114:0/1525957815 shutdown_connections 2024-09-21T07:02:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.356+0000 7f1e91442740 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d6d4d0 0x555fe91336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.356+0000 7f1e91442740 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d652f0 0x555fe9133170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.356+0000 7f1e91442740 1 --2- 172.21.15.114:0/1525957815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe9132c30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.356+0000 7f1e91442740 1 -- 172.21.15.114:0/1525957815 >> 172.21.15.114:0/1525957815 conn(0x555fe8d82300 msgr2=0x555fe9130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.358+0000 7f1e91442740 1 -- 172.21.15.114:0/1525957815 shutdown_connections 2024-09-21T07:02:53.359 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.358+0000 7f1e91442740 1 -- 172.21.15.114:0/1525957815 wait complete. 2024-09-21T07:02:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.360+0000 7f1e83400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e680b6700 0x7f1e680b8c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x555fe91318b0 tx=0x7f1e70015000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.360+0000 7f1e91442740 1 -- start start 2024-09-21T07:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.361+0000 7f1e91442740 1 Processor -- start 2024-09-21T07:02:53.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 0x555fe913e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d8e1b0 0x555fe914a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555fe8fcfab0 con 0x555fe8d8e1b0 2024-09-21T07:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555fe903e570 con 0x555fe8d652f0 2024-09-21T07:02:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555fe9141350 con 0x555fe8d6d4d0 2024-09-21T07:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.362+0000 7f1e91442740 10 client.0 _ll_register_callbacks cb 0x555fe904d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 0x555fe913e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 0x555fe913e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41110/0 (socket says 172.21.15.114:41110) 2024-09-21T07:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e83e00640 1 -- 172.21.15.114:0/4134919585 learned_addr learned my addr 172.21.15.114:0/4134919585 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e83e00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d8e1b0 0x555fe914a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:52 smithi183 ceph-mon[34219]: pgmap v662: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-21T07:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e82a00640 1 -- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 msgr2=0x555fe913e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 0x555fe913e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e82a00640 1 -- 172.21.15.114:0/4134919585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d8e1b0 msgr2=0x555fe914a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d8e1b0 0x555fe914a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e82a00640 1 -- 172.21.15.114:0/4134919585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe8df9790 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.363+0000 7f1e83e00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 0x555fe913e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913e7d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e8406c8b0 tx=0x7f1e74012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781e7e41d36e0889 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e6d600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e6d600640 1 -- 172.21.15.114:0/4134919585 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74010dc0 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e6d600640 1 -- 172.21.15.114:0/4134919585 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1e7401a430 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7c036070 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e6d600640 1 -- 172.21.15.114:0/4134919585 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1e74025050 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e91442740 10 client.27718 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.364+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555fe8e72620 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e6d600640 1 -- 172.21.15.114:0/4134919585 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7c036070 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e6d600640 1 -- 172.21.15.114:0/4134919585 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1e7400e6b0 con 0x555fe8d6d4d0 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e91442740 10 client.27718 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e91442740 10 client.27718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e91442740 20 client.27718 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e91442740 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d4cd80 0x555fe91503a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e91442740 4 client.27718 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fe90aa600 con 0x555fe8d4cd80 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.365+0000 7f1e83e00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d4cd80 0x555fe91503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.366+0000 7f1e83e00640 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d4cd80 0x555fe91503a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5df4035d8643387f server_cookie=e489bf898cd876ce in_seq=0 out_seq=0 2024-09-21T07:02:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.366+0000 7f1e6d600640 10 client.27718 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e6d600640 1 -- 172.21.15.114:0/4134919585 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fe90aa600 con 0x555fe8d4cd80 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e6d600640 10 client.27718 handle_command_reply: tid=0 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e91442740 1 client.27718 shutdown 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 msgr2=0x555fe913e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e91442740 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913e7d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1e8406c8b0 tx=0x7f1e74012b60 comp rx=0 tx=0).stop 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 shutdown_connections 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.367+0000 7f1e91442740 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d8e1b0 0x555fe914a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d4cd80 0x555fe91503a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913e7d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 1 --2- 172.21.15.114:0/4134919585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d652f0 0x555fe913e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e6d600640 0 client.27718 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 >> 172.21.15.114:0/4134919585 conn(0x555fe8d82300 msgr2=0x555fe9133c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 shutdown_connections 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 1 -- 172.21.15.114:0/4134919585 wait complete. 2024-09-21T07:02:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.368+0000 7f1e91442740 20 client.27718 trim_cache size 0 max 16384 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.394+0000 7f1e91442740 1 Processor -- start 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.394+0000 7f1e91442740 1 -- start start 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e91442740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe913e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e91442740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e91442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe914a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e91442740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841bfbb0 con 0x555fe8d652f0 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e91442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841c1750 con 0x555fe8d8e1b0 2024-09-21T07:02:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e91442740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841c0280 con 0x555fe8d6d4d0 2024-09-21T07:02:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe913e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe913e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59898/0 (socket says 172.21.15.114:59898) 2024-09-21T07:02:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e83e00640 1 -- 172.21.15.114:0/4208606960 learned_addr learned my addr 172.21.15.114:0/4208606960 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e83e00640 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe914a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e82a00640 1 -- 172.21.15.114:0/4208606960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 msgr2=0x555fe914a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.395+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe914a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e82a00640 1 -- 172.21.15.114:0/4208606960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 msgr2=0x555fe913e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe913e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e82a00640 1 -- 172.21.15.114:0/4208606960 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fe9135a40 con 0x555fe8d6d4d0 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e83e00640 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe914a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:02:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e82a00640 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913fec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e74000c00 tx=0x7f1e74010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4114d9c27f0875 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e6fe00640 1 -- 172.21.15.114:0/4208606960 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74010dc0 con 0x555fe8d6d4d0 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e6fe00640 1 -- 172.21.15.114:0/4208606960 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7400e3b0 con 0x555fe8d6d4d0 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.396+0000 7f1e6fe00640 1 -- 172.21.15.114:0/4208606960 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e740053b0 con 0x555fe8d6d4d0 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 -- 172.21.15.114:0/4208606960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 msgr2=0x555fe913fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913fec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1e74000c00 tx=0x7f1e74010880 comp rx=0 tx=0).stop 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 -- 172.21.15.114:0/4208606960 shutdown_connections 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d8e1b0 0x555fe914a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d6d4d0 0x555fe913fec0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 --2- 172.21.15.114:0/4208606960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe913e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 -- 172.21.15.114:0/4208606960 >> 172.21.15.114:0/4208606960 conn(0x555fe8d82300 msgr2=0x555fe913f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 -- 172.21.15.114:0/4208606960 shutdown_connections 2024-09-21T07:02:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.397+0000 7f1e91442740 1 -- 172.21.15.114:0/4208606960 wait complete. 2024-09-21T07:02:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.398+0000 7f1e91442740 1 -- start start 2024-09-21T07:02:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.398+0000 7f1e91442740 1 Processor -- start 2024-09-21T07:02:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe9146e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d6d4d0 0x555fe91422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d8e1b0 0x555fe91428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841c1080 con 0x555fe8d652f0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841c0950 con 0x555fe8d6d4d0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841bdbb0 con 0x555fe8d8e1b0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e91442740 10 client.0 _ll_register_callbacks cb 0x7f1e740094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe9146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e82a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d6d4d0 0x555fe91422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe9146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59914/0 (socket says 172.21.15.114:59914) 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 -- 172.21.15.114:0/1026714736 learned_addr learned my addr 172.21.15.114:0/1026714736 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d8e1b0 0x555fe91428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 -- 172.21.15.114:0/1026714736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d8e1b0 msgr2=0x555fe91428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d8e1b0 0x555fe91428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 -- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d6d4d0 msgr2=0x555fe91422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d6d4d0 0x555fe91422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.399+0000 7f1e83e00640 1 -- 172.21.15.114:0/1026714736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fe9143050 con 0x555fe8d652f0 2024-09-21T07:02:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.400+0000 7f1e83e00640 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe9146e50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c004af0 tx=0x7f1e7c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf45cad40120e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.400+0000 7f1e6d600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.400+0000 7f1e6d600640 1 -- 172.21.15.114:0/1026714736 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7c004150 con 0x555fe8d652f0 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.400+0000 7f1e6d600640 1 -- 172.21.15.114:0/1026714736 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f1e7c0042f0 con 0x555fe8d652f0 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.400+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e7400e3b0 con 0x555fe8d652f0 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e6d600640 1 -- 172.21.15.114:0/1026714736 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f1e7c0044f0 con 0x555fe8d652f0 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 10 client.17376 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555fe91bad40 con 0x555fe8d652f0 2024-09-21T07:02:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e6d600640 1 -- 172.21.15.114:0/1026714736 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e7400e3b0 con 0x555fe8d652f0 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e6d600640 1 -- 172.21.15.114:0/1026714736 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7f1e7c043020 con 0x555fe8d652f0 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 10 client.17376 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 20 client.17376 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d221c0 0x555fe91bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 4 client.17376 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.401+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fe90a60b0 con 0x555fe8d221c0 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.402+0000 7f1e82a00640 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d221c0 0x555fe91bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.402+0000 7f1e6d600640 10 client.17376 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.402+0000 7f1e82a00640 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d221c0 0x555fe91bcfd0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584a4a000958c529 server_cookie=15fb56fdf82c3ca in_seq=0 out_seq=0 2024-09-21T07:02:53.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:52 smithi193 ceph-mon[34601]: pgmap v662: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-21T07:02:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e6d600640 1 -- 172.21.15.114:0/1026714736 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fe90a60b0 con 0x555fe8d221c0 2024-09-21T07:02:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e6d600640 10 client.17376 handle_command_reply: tid=0 2024-09-21T07:02:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 client.17376 shutdown 2024-09-21T07:02:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 msgr2=0x555fe9146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe9146e50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c004af0 tx=0x7f1e7c005df0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 shutdown_connections 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555fe8d221c0 0x555fe91bcfd0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fe8d8e1b0 0x555fe91428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fe8d6d4d0 0x555fe91422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 --2- 172.21.15.114:0/1026714736 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555fe8d652f0 0x555fe9146e50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 >> 172.21.15.114:0/1026714736 conn(0x555fe8d82300 msgr2=0x555fe91343a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.417+0000 7f1e6d600640 0 client.17376 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:02:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.418+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 shutdown_connections 2024-09-21T07:02:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.418+0000 7f1e91442740 1 -- 172.21.15.114:0/1026714736 wait complete. 2024-09-21T07:02:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.418+0000 7f1e91442740 20 client.17376 trim_cache size 0 max 16384 2024-09-21T07:02:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:02:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:02:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:02:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e680b6700 msgr2=0x7f1e680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e680b6700 0x7f1e680b8c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x555fe91318b0 tx=0x7f1e70015000 comp rx=0 tx=0).stop 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 msgr2=0x7f1e84107540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 0x7f1e84107540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000c920 tx=0x7f1e7000cde0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 shutdown_connections 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1e841100a0 0x7f1e84103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e84076080 0x7f1e8410fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e680b6700 0x7f1e680b8c10 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 --2- 172.21.15.114:0/3449356689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e84075730 0x7f1e84107540 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 >> 172.21.15.114:0/3449356689 conn(0x7f1e840ff510 msgr2=0x7f1e841010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 shutdown_connections 2024-09-21T07:02:53.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:02:53.419+0000 7f1e8a400640 1 -- 172.21.15.114:0/3449356689 wait complete. 2024-09-21T07:02:53.469 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:02:54.126 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:55.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:54 smithi114 ceph-mon[30654]: pgmap v663: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.3 MiB/s wr, 324 op/s 2024-09-21T07:02:55.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:54 smithi183 ceph-mon[34219]: pgmap v663: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.3 MiB/s wr, 324 op/s 2024-09-21T07:02:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:54 smithi193 ceph-mon[34601]: pgmap v663: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.3 MiB/s wr, 324 op/s 2024-09-21T07:02:55.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.071+0000 7fe0c0e00640 1 -- 172.21.15.114:0/1309575213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc10b500 msgr2=0x7fe0bc10b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.071+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/1309575213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc10b500 0x7fe0bc10b910 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009900 tx=0x7fe0a802a410 comp rx=0 tx=0).stop 2024-09-21T07:02:55.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 -- 172.21.15.114:0/1309575213 shutdown_connections 2024-09-21T07:02:55.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/1309575213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10be50 0x7fe0bc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/1309575213 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc10b500 0x7fe0bc10b910 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/1309575213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 -- 172.21.15.114:0/1309575213 >> 172.21.15.114:0/1309575213 conn(0x7fe0bc101670 msgr2=0x7fe0bc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 -- 172.21.15.114:0/1309575213 shutdown_connections 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.074+0000 7fe0c0e00640 1 -- 172.21.15.114:0/1309575213 wait complete. 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.078+0000 7fe0c0e00640 1 Processor -- start 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.079+0000 7fe0c0e00640 1 -- start start 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.084+0000 7fe0c0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.084+0000 7fe0c0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10b500 0x7fe0bc19ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0c0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc10be50 0x7fe0bc1a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0c0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc115b60 con 0x7fe0bc105bc0 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0c0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc1159e0 con 0x7fe0bc10be50 2024-09-21T07:02:55.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0c0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc115ce0 con 0x7fe0bc10b500 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39196/0 (socket says 172.21.15.114:39196) 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0bb400640 1 -- 172.21.15.114:0/2050249353 learned_addr learned my addr 172.21.15.114:0/2050249353 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0baa00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10b500 0x7fe0bc19ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.085+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc10be50 0x7fe0bc1a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.087+0000 7fe0bb400640 1 -- 172.21.15.114:0/2050249353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10b500 msgr2=0x7fe0bc19ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.087+0000 7fe0bb400640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10b500 0x7fe0bc19ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.087+0000 7fe0bb400640 1 -- 172.21.15.114:0/2050249353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc10be50 msgr2=0x7fe0bc1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.087+0000 7fe0bb400640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc10be50 0x7fe0bc1a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.087+0000 7fe0bb400640 1 -- 172.21.15.114:0/2050249353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc1a2460 con 0x7fe0bc105bc0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.088+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc10be50 0x7fe0bc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.088+0000 7fe0baa00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10b500 0x7fe0bc19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.089+0000 7fe0bb400640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc19c4f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e9f0 tx=0x7fe0ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.090+0000 7fe0b3e00640 1 -- 172.21.15.114:0/2050249353 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00cd80 con 0x7fe0bc105bc0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.090+0000 7fe0b3e00640 1 -- 172.21.15.114:0/2050249353 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ac0108e0 con 0x7fe0bc105bc0 2024-09-21T07:02:55.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.090+0000 7fe0b3e00640 1 -- 172.21.15.114:0/2050249353 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac018980 con 0x7fe0bc105bc0 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.090+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc1a2750 con 0x7fe0bc105bc0 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.091+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc1a8eb0 con 0x7fe0bc105bc0 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.095+0000 7fe0c90db740 1 Processor -- start 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.095+0000 7fe0c90db740 1 -- start start 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.096+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 0x5647330d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.096+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.096+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.096+0000 7fe0c90db740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56473304cfb0 con 0x564732d0e390 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.096+0000 7fe0c90db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56473303c3e0 con 0x564732d061b0 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.096+0000 7fe0c90db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5647330401f0 con 0x564732d2f070 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 0x5647330d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0baa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 0x5647330d5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33142/0 (socket says 172.21.15.114:33142) 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 -- 172.21.15.114:0/441920042 learned_addr learned my addr 172.21.15.114:0/441920042 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:55.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 -- 172.21.15.114:0/441920042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 msgr2=0x5647330d64b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330d64b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 -- 172.21.15.114:0/441920042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 msgr2=0x5647330d69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330d69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.097+0000 7fe0bbe00640 1 -- 172.21.15.114:0/441920042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564733006cd0 con 0x564732d2f070 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.098+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 0x5647330d5f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40117e0 tx=0x7fe0b4011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84af11bc70b0e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.098+0000 7fe0b3e00640 1 -- 172.21.15.114:0/2050249353 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe0ac010430 con 0x7fe0bc105bc0 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.099+0000 7fe0b0c00640 1 -- 172.21.15.114:0/441920042 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4011310 con 0x564732d2f070 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.099+0000 7fe0b0c00640 1 -- 172.21.15.114:0/441920042 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b4003e20 con 0x564732d2f070 2024-09-21T07:02:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.099+0000 7fe0b0c00640 1 -- 172.21.15.114:0/441920042 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40044c0 con 0x564732d2f070 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.100+0000 7fe0c90db740 1 -- 172.21.15.114:0/441920042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 msgr2=0x5647330d5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.100+0000 7fe0c90db740 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 0x5647330d5f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe0b40117e0 tx=0x7fe0b4011ca0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 -- 172.21.15.114:0/441920042 shutdown_connections 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330d69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330d64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 --2- 172.21.15.114:0/441920042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d2f070 0x5647330d5f70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 -- 172.21.15.114:0/441920042 >> 172.21.15.114:0/441920042 conn(0x564732d23460 msgr2=0x5647330d3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0b3e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0a00b6800 0x7fe0a00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 -- 172.21.15.114:0/441920042 shutdown_connections 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0b3e00640 1 -- 172.21.15.114:0/2050249353 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fe0ac0e7d10 con 0x7fe0bc105bc0 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.101+0000 7fe0c90db740 1 -- 172.21.15.114:0/441920042 wait complete. 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.102+0000 7fe0c90db740 1 -- start start 2024-09-21T07:02:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.102+0000 7fe0baa00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0a00b6800 0x7fe0a00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.102+0000 7fe0c90db740 1 Processor -- start 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0baa00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0a00b6800 0x7fe0a00b8d10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe0a802aa00 tx=0x7fe0a8000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d061b0 0x5647330f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x564733107220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0c90db740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564732e9d700 con 0x564732d061b0 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0c90db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564732fd7580 con 0x564732d2f070 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.104+0000 7fe0c90db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5647330f56b0 con 0x564732d0e390 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d061b0 0x5647330f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x564733107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x564733107220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43258/0 (socket says 172.21.15.114:43258) 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bbe00640 1 -- 172.21.15.114:0/399905877 learned_addr learned my addr 172.21.15.114:0/399905877 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bb400640 1 -- 172.21.15.114:0/399905877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 msgr2=0x5647330f2cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:02:55.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bb400640 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330f2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bb400640 1 -- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 msgr2=0x564733107220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bb400640 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x564733107220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0bb400640 1 -- 172.21.15.114:0/399905877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564732d9a650 con 0x564732d061b0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0c90db740 10 client.0 _ll_register_callbacks cb 0x564732fe7470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.105+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564732fd9b90 con 0x564732d061b0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.106+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x564733107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0bb400640 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d061b0 0x5647330f2790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0e5090 tx=0x7fe0ac010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadaedb9eaf1a236 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0a5600640 10 client.17385 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0a5600640 1 -- 172.21.15.114:0/399905877 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac010720 con 0x564732d061b0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0a5600640 1 -- 172.21.15.114:0/399905877 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fe0ac0a9190 con 0x564732d061b0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0a5600640 1 -- 172.21.15.114:0/399905877 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac0a93b0 con 0x564732d061b0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0c90db740 10 client.17385 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.107+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({fsmap=47}) -- 0x564732ff4950 con 0x564732d061b0 2024-09-21T07:02:55.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0a5600640 1 -- 172.21.15.114:0/399905877 <== mon.0 v2:172.21.15.114:3300/0 4 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fe0ac0ac510 con 0x564732d061b0 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0c90db740 10 client.17385 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0c90db740 10 client.17385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0c90db740 20 client.17385 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0c90db740 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732ced9f0 0x56473310cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0c90db740 4 client.17385 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56473304ef00 con 0x564732ced9f0 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.108+0000 7fe0baa00640 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732ced9f0 0x56473310cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.109+0000 7fe0a5600640 10 client.17385 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.109+0000 7fe0baa00640 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732ced9f0 0x56473310cec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=60252369cbdce3dc server_cookie=365549dcd9742bea in_seq=0 out_seq=0 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.110+0000 7fe0a5600640 1 -- 172.21.15.114:0/399905877 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56473304ef00 con 0x564732ced9f0 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.110+0000 7fe0a5600640 10 client.17385 handle_command_reply: tid=0 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.110+0000 7fe0c90db740 1 client.17385 shutdown 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.110+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d061b0 msgr2=0x5647330f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.110+0000 7fe0c90db740 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d061b0 0x5647330f2790 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0e5090 tx=0x7fe0ac010cc0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 shutdown_connections 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732ced9f0 0x56473310cec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x564733107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 --2- 172.21.15.114:0/399905877 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d061b0 0x5647330f2790 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0a5600640 0 client.17385 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 >> 172.21.15.114:0/399905877 conn(0x564732d23460 msgr2=0x5647330d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 shutdown_connections 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 1 -- 172.21.15.114:0/399905877 wait complete. 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.111+0000 7fe0c90db740 20 client.17385 trim_cache size 0 max 16384 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 Processor -- start 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 -- start start 2024-09-21T07:02:55.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d2f070 0x5647330efee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc072920 con 0x564732d2f070 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc0744c0 con 0x564732d061b0 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.132+0000 7fe0c90db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc072ff0 con 0x564732d0e390 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43272/0 (socket says 172.21.15.114:43272) 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330ef9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33152/0 (socket says 172.21.15.114:33152) 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 -- 172.21.15.114:0/3231190649 learned_addr learned my addr 172.21.15.114:0/3231190649 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d2f070 0x5647330efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 -- 172.21.15.114:0/3231190649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 msgr2=0x5647330ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 -- 172.21.15.114:0/3231190649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d2f070 msgr2=0x5647330efee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:55.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d2f070 0x5647330efee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 -- 172.21.15.114:0/3231190649 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647330d8d30 con 0x564732d061b0 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0baa00640 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.133+0000 7fe0bb400640 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330f43c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e9c0 tx=0x7fe0ac022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695a92b9cd450a6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.134+0000 7fe0a7e00640 1 -- 172.21.15.114:0/3231190649 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0cd070 con 0x564732d061b0 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.134+0000 7fe0a7e00640 1 -- 172.21.15.114:0/3231190649 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564732fd9b90 con 0x564732d061b0 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.134+0000 7fe0a7e00640 1 -- 172.21.15.114:0/3231190649 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0b2270 con 0x564732d061b0 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 -- 172.21.15.114:0/3231190649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 msgr2=0x5647330f43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330f43c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e9c0 tx=0x7fe0ac022000 comp rx=0 tx=0).stop 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 -- 172.21.15.114:0/3231190649 shutdown_connections 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d2f070 0x5647330efee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d0e390 0x5647330ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 --2- 172.21.15.114:0/3231190649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d061b0 0x5647330f43c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 -- 172.21.15.114:0/3231190649 >> 172.21.15.114:0/3231190649 conn(0x564732d23460 msgr2=0x5647330f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 -- 172.21.15.114:0/3231190649 shutdown_connections 2024-09-21T07:02:55.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.135+0000 7fe0c90db740 1 -- 172.21.15.114:0/3231190649 wait complete. 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.136+0000 7fe0c90db740 1 -- start start 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.136+0000 7fe0c90db740 1 Processor -- start 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 0x5647331037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x5647331001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc073df0 con 0x564732d0e390 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc0736c0 con 0x564732d2f070 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc070920 con 0x564732d061b0 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.137+0000 7fe0c90db740 10 client.0 _ll_register_callbacks cb 0x7fe0ac0ad610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 0x5647331037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 0x5647331037e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33156/0 (socket says 172.21.15.114:33156) 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 -- 172.21.15.114:0/261963510 learned_addr learned my addr 172.21.15.114:0/261963510 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:02:55.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bbe00640 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x5647331001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 -- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 msgr2=0x5647331001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x5647331001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 -- 172.21.15.114:0/261963510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 msgr2=0x5647330ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.138+0000 7fe0bb400640 1 -- 172.21.15.114:0/261963510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647331007a0 con 0x564732d061b0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0bb400640 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 0x5647331037e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0aca30 tx=0x7fe0ac0e5140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b7915bb47597cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0a5600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0a5600640 1 -- 172.21.15.114:0/261963510 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0cd030 con 0x564732d061b0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0a5600640 1 -- 172.21.15.114:0/261963510 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fe0ac0aecd0 con 0x564732d061b0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564732fd9b90 con 0x564732d061b0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0a5600640 1 -- 172.21.15.114:0/261963510 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac010720 con 0x564732d061b0 2024-09-21T07:02:55.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0c90db740 10 client.27739 fetch_fsmap learned FSMap version 47 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5647331785c0 con 0x564732d061b0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.139+0000 7fe0a5600640 1 -- 172.21.15.114:0/261963510 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564732fd9b90 con 0x564732d061b0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0a5600640 1 -- 172.21.15.114:0/261963510 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fe0ac0b66f0 con 0x564732d061b0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0c90db740 10 client.27739 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0c90db740 10 client.27739 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0c90db740 20 client.27739 populate_metadata read hostname 'smithi114' 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0c90db740 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732cc2da0 0x56473317aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0c90db740 4 client.27739 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56473304a9b0 con 0x564732cc2da0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.140+0000 7fe0baa00640 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732cc2da0 0x56473317aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.141+0000 7fe0a5600640 10 client.27739 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.141+0000 7fe0baa00640 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732cc2da0 0x56473317aab0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36c2ea31922fc247 server_cookie=92acb3de45f93bd0 in_seq=0 out_seq=0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.141+0000 7fe0a5600640 1 -- 172.21.15.114:0/261963510 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56473304a9b0 con 0x564732cc2da0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.141+0000 7fe0a5600640 10 client.27739 handle_command_reply: tid=0 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.141+0000 7fe0c90db740 1 client.27739 shutdown 2024-09-21T07:02:55.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 msgr2=0x5647331037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 0x5647331037e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0aca30 tx=0x7fe0ac0e5140 comp rx=0 tx=0).stop 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 shutdown_connections 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x564732cc2da0 0x56473317aab0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0a5600640 0 client.27739 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564732d2f070 0x5647331001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564732d0e390 0x5647330ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 --2- 172.21.15.114:0/261963510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564732d061b0 0x5647331037e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 >> 172.21.15.114:0/261963510 conn(0x564732d23460 msgr2=0x5647330d6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 shutdown_connections 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 1 -- 172.21.15.114:0/261963510 wait complete. 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.142+0000 7fe0c90db740 20 client.27739 trim_cache size 0 max 16384 2024-09-21T07:02:55.561 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.143+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0a00b6800 msgr2=0x7fe0a00b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.143+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0a00b6800 0x7fe0a00b8d10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe0a802aa00 tx=0x7fe0a8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.143+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 msgr2=0x7fe0bc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.143+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc19c4f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e9f0 tx=0x7fe0ac00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 shutdown_connections 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe0a00b6800 0x7fe0a00b8d10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc10be50 0x7fe0bc1a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0bc10b500 0x7fe0bc19ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 --2- 172.21.15.114:0/2050249353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe0bc105bc0 0x7fe0bc19c4f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 >> 172.21.15.114:0/2050249353 conn(0x7fe0bc101670 msgr2=0x7fe0bc108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 shutdown_connections 2024-09-21T07:02:55.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:02:55.144+0000 7fe0c0e00640 1 -- 172.21.15.114:0/2050249353 wait complete. 2024-09-21T07:02:57.127 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:02:57.223 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:56 smithi114 ceph-mon[30654]: pgmap v664: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-09-21T07:02:57.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:56 smithi183 ceph-mon[34219]: pgmap v664: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-09-21T07:02:57.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:56 smithi193 ceph-mon[34601]: pgmap v664: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 337 op/s 2024-09-21T07:02:57.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 500.0 s, 465.0 tps, lat 211.402 ms stddev 254.761 2024-09-21T07:02:58.043 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:02:59.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:02:58 smithi114 ceph-mon[30654]: pgmap v665: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-09-21T07:02:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:02:58 smithi183 ceph-mon[34219]: pgmap v665: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-09-21T07:02:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:02:58 smithi193 ceph-mon[34601]: pgmap v665: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.8 MiB/s wr, 480 op/s 2024-09-21T07:03:00.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 -- 172.21.15.114:0/1738124361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d010de10 msgr2=0x7fb2d0110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 --2- 172.21.15.114:0/1738124361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d010de10 0x7fb2d0110300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009ca0 tx=0x7fb2c802a8d0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 -- 172.21.15.114:0/1738124361 shutdown_connections 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 --2- 172.21.15.114:0/1738124361 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d010de10 0x7fb2d0110300 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 --2- 172.21.15.114:0/1738124361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 0x7fb2d010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 --2- 172.21.15.114:0/1738124361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d01049b0 0x7fb2d0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.213+0000 7fb2d8600640 1 -- 172.21.15.114:0/1738124361 >> 172.21.15.114:0/1738124361 conn(0x7fb2d0079730 msgr2=0x7fb2d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.214+0000 7fb2d8600640 1 -- 172.21.15.114:0/1738124361 shutdown_connections 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.214+0000 7fb2d8600640 1 -- 172.21.15.114:0/1738124361 wait complete. 2024-09-21T07:03:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.215+0000 7fb2d8600640 1 Processor -- start 2024-09-21T07:03:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.215+0000 7fb2d8600640 1 -- start start 2024-09-21T07:03:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.216+0000 7fb2d8600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 0x7fb2d019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.216+0000 7fb2d8600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 0x7fb2d019d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.216+0000 7fb2d8600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 0x7fb2d010cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 0x7fb2d019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 0x7fb2d019d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 0x7fb2d019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39228/0 (socket says 172.21.15.114:39228) 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 0x7fb2d019d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33162/0 (socket says 172.21.15.114:33162) 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 0x7fb2d010cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 0x7fb2d010cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43300/0 (socket says 172.21.15.114:43300) 2024-09-21T07:03:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d6600640 1 -- 172.21.15.114:0/2641333456 learned_addr learned my addr 172.21.15.114:0/2641333456 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:00.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0113bc0 con 0x7fb2d01049b0 2024-09-21T07:03:00.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0113a40 con 0x7fb2d010de10 2024-09-21T07:03:00.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.217+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d0113d40 con 0x7fb2d0105300 2024-09-21T07:03:00.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d6600640 1 -- 172.21.15.114:0/2641333456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 msgr2=0x7fb2d019d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d6600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 0x7fb2d019d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d6600640 1 -- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 msgr2=0x7fb2d019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d6600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 0x7fb2d019cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d6600640 1 -- 172.21.15.114:0/2641333456 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d010d510 con 0x7fb2d010de10 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 0x7fb2d019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.218+0000 7fb2d6600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 0x7fb2d010cd40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8003cc0 tx=0x7fb2c8003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.219+0000 7fb2ce000640 1 -- 172.21.15.114:0/2641333456 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8004240 con 0x7fb2d010de10 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.219+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d010d7a0 con 0x7fb2d010de10 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.219+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d01a4b50 con 0x7fb2d010de10 2024-09-21T07:03:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.222+0000 7fb2ce000640 1 -- 172.21.15.114:0/2641333456 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c80305d0 con 0x7fb2d010de10 2024-09-21T07:03:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.222+0000 7fb2ce000640 1 -- 172.21.15.114:0/2641333456 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803b650 con 0x7fb2d010de10 2024-09-21T07:03:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.233+0000 7fb2df227740 1 Processor -- start 2024-09-21T07:03:00.238 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.237+0000 7fb2ce000640 1 -- 172.21.15.114:0/2641333456 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb2c803b8b0 con 0x7fb2d010de10 2024-09-21T07:03:00.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.239+0000 7fb2ce000640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2b80b6700 0x7fb2b80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.240+0000 7fb2ce000640 1 -- 172.21.15.114:0/2641333456 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fb2c8109070 con 0x7fb2d010de10 2024-09-21T07:03:00.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.240+0000 7fb2df227740 1 -- start start 2024-09-21T07:03:00.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2df227740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55753734a070 0x5575376f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2df227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373211b0 0x5575376f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2df227740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376f1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2df227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557537667fb0 con 0x5575373211b0 2024-09-21T07:03:00.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2df227740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575376573e0 con 0x557537329390 2024-09-21T07:03:00.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2df227740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55753765b1f0 con 0x55753734a070 2024-09-21T07:03:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43306/0 (socket says 172.21.15.114:43306) 2024-09-21T07:03:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2d6600640 1 -- 172.21.15.114:0/401967891 learned_addr learned my addr 172.21.15.114:0/401967891 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.241+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55753734a070 0x5575376f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.242+0000 7fb2d6600640 1 -- 172.21.15.114:0/401967891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55753734a070 msgr2=0x5575376f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.242+0000 7fb2d6600640 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55753734a070 0x5575376f0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.242+0000 7fb2d6600640 1 -- 172.21.15.114:0/401967891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373211b0 msgr2=0x5575376f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.242+0000 7fb2d6600640 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373211b0 0x5575376f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.246 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.242+0000 7fb2d6600640 1 -- 172.21.15.114:0/401967891 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557537621cd0 con 0x557537329390 2024-09-21T07:03:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.242+0000 7fb2d6600640 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376f1820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb2c802a910 tx=0x7fb2c80cf120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cdd65a10314d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.243+0000 7fb2c2a00640 1 -- 172.21.15.114:0/401967891 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80d00b0 con 0x557537329390 2024-09-21T07:03:00.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.243+0000 7fb2c2a00640 1 -- 172.21.15.114:0/401967891 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2c8036070 con 0x557537329390 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.243+0000 7fb2c2a00640 1 -- 172.21.15.114:0/401967891 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8106670 con 0x557537329390 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.244+0000 7fb2d5200640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2b80b6700 0x7fb2b80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 -- 172.21.15.114:0/401967891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 msgr2=0x5575376f1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376f1820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb2c802a910 tx=0x7fb2c80cf120 comp rx=0 tx=0).stop 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 -- 172.21.15.114:0/401967891 shutdown_connections 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376f1820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373211b0 0x5575376f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 --2- 172.21.15.114:0/401967891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55753734a070 0x5575376f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 -- 172.21.15.114:0/401967891 >> 172.21.15.114:0/401967891 conn(0x55753733e460 msgr2=0x5575376eec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.245+0000 7fb2df227740 1 -- 172.21.15.114:0/401967891 shutdown_connections 2024-09-21T07:03:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.249+0000 7fb2d5200640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2b80b6700 0x7fb2b80b8c10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc006e00 tx=0x7fb2bc008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.249+0000 7fb2df227740 1 -- 172.21.15.114:0/401967891 wait complete. 2024-09-21T07:03:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.250+0000 7fb2df227740 1 -- start start 2024-09-21T07:03:00.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.250+0000 7fb2df227740 1 Processor -- start 2024-09-21T07:03:00.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x5575376fc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55753734a070 0x5575377085a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575374b8700 con 0x55753734a070 2024-09-21T07:03:00.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575375f2580 con 0x557537329390 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575376ff340 con 0x5575373211b0 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.251+0000 7fb2df227740 10 client.0 _ll_register_callbacks cb 0x557537602470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376fcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43318/0 (socket says 172.21.15.114:43318) 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 -- 172.21.15.114:0/3760096733 learned_addr learned my addr 172.21.15.114:0/3760096733 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x5575376fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d6600640 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55753734a070 0x5575377085a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 -- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 msgr2=0x5575376fc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x5575376fc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 -- 172.21.15.114:0/3760096733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55753734a070 msgr2=0x5575377085a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55753734a070 0x5575377085a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.252+0000 7fb2d5200640 1 -- 172.21.15.114:0/3760096733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575373b5650 con 0x557537329390 2024-09-21T07:03:00.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.253+0000 7fb2d5200640 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376fcc10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb2d006aaa0 tx=0x7fb2bc006a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d739923aac7be350 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.253+0000 7fb2b7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:00.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.254+0000 7fb2b7400640 1 -- 172.21.15.114:0/3760096733 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc024060 con 0x557537329390 2024-09-21T07:03:00.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.254+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575375f4b90 con 0x557537329390 2024-09-21T07:03:00.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.255+0000 7fb2b7400640 1 -- 172.21.15.114:0/3760096733 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fb2bc00f040 con 0x557537329390 2024-09-21T07:03:00.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.257+0000 7fb2b7400640 1 -- 172.21.15.114:0/3760096733 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb2bc02b050 con 0x557537329390 2024-09-21T07:03:00.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.257+0000 7fb2df227740 10 client.27812 fetch_fsmap learned FSMap version 47 2024-09-21T07:03:00.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.257+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55753760f950 con 0x557537329390 2024-09-21T07:03:00.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.257+0000 7fb2b7400640 1 -- 172.21.15.114:0/3760096733 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575375f4b90 con 0x557537329390 2024-09-21T07:03:00.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.257+0000 7fb2b7400640 1 -- 172.21.15.114:0/3760096733 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fb2bc013070 con 0x557537329390 2024-09-21T07:03:00.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.261+0000 7fb2df227740 10 client.27812 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:03:00.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.261+0000 7fb2df227740 10 client.27812 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:03:00.262 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.261+0000 7fb2df227740 20 client.27812 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.261+0000 7fb2df227740 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5575373089f0 0x55753770e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.261+0000 7fb2df227740 4 client.27812 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:00.263 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.261+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557537669f00 con 0x5575373089f0 2024-09-21T07:03:00.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.264+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5575373089f0 0x55753770e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.266+0000 7fb2b7400640 10 client.27812 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:03:00.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.266+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5575373089f0 0x55753770e500 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=450c8b0160c2feb6 server_cookie=acf1b4bd2d5db438 in_seq=0 out_seq=0 2024-09-21T07:03:00.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.268+0000 7fb2b7400640 1 -- 172.21.15.114:0/3760096733 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557537669f00 con 0x5575373089f0 2024-09-21T07:03:00.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.268+0000 7fb2b7400640 10 client.27812 handle_command_reply: tid=0 2024-09-21T07:03:00.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.268+0000 7fb2df227740 1 client.27812 shutdown 2024-09-21T07:03:00.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.270+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 msgr2=0x5575376fcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.270+0000 7fb2df227740 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376fcc10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb2d006aaa0 tx=0x7fb2bc006a20 comp rx=0 tx=0).stop 2024-09-21T07:03:00.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 shutdown_connections 2024-09-21T07:03:00.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2df227740 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5575373089f0 0x55753770e500 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2df227740 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55753734a070 0x5575377085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2b7400640 0 client.27812 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:03:00.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2df227740 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557537329390 0x5575376fcc10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2df227740 1 --2- 172.21.15.114:0/3760096733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x5575376fc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.273+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 >> 172.21.15.114:0/3760096733 conn(0x55753733e460 msgr2=0x5575376f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.274+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 shutdown_connections 2024-09-21T07:03:00.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.274+0000 7fb2df227740 1 -- 172.21.15.114:0/3760096733 wait complete. 2024-09-21T07:03:00.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.274+0000 7fb2df227740 20 client.27812 trim_cache size 0 max 16384 2024-09-21T07:03:00.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.317+0000 7fb2df227740 1 Processor -- start 2024-09-21T07:03:00.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.318+0000 7fb2df227740 1 -- start start 2024-09-21T07:03:00.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2df227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373089f0 0x5575376eece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2df227740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373211b0 0x5575377084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2df227740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557537329390 0x5575377089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2df227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01aee70 con 0x5575373089f0 2024-09-21T07:03:00.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2df227740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01b0a10 con 0x5575373211b0 2024-09-21T07:03:00.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2df227740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01af540 con 0x557537329390 2024-09-21T07:03:00.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373211b0 0x5575377084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557537329390 0x5575377089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2d6600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557537329390 0x5575377089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33182/0 (socket says 172.21.15.114:33182) 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.319+0000 7fb2d6600640 1 -- 172.21.15.114:0/335309957 learned_addr learned my addr 172.21.15.114:0/335309957 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.320+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373089f0 0x5575376eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.320+0000 7fb2d5200640 1 -- 172.21.15.114:0/335309957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557537329390 msgr2=0x5575377089e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.320+0000 7fb2d5200640 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557537329390 0x5575377089e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.320+0000 7fb2d5200640 1 -- 172.21.15.114:0/335309957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373089f0 msgr2=0x5575376eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.320+0000 7fb2d5200640 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373089f0 0x5575376eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.320+0000 7fb2d5200640 1 -- 172.21.15.114:0/335309957 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575376f4e20 con 0x5575373211b0 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.322+0000 7fb2d5200640 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373211b0 0x5575377084a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc01e450 tx=0x7fb2bc014cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8824e08b3992605f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.322+0000 7fb2c2000640 1 -- 172.21.15.114:0/335309957 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc024060 con 0x5575373211b0 2024-09-21T07:03:00.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.322+0000 7fb2c2000640 1 -- 172.21.15.114:0/335309957 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2bc005e40 con 0x5575373211b0 2024-09-21T07:03:00.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.322+0000 7fb2c2000640 1 -- 172.21.15.114:0/335309957 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0038f0 con 0x5575373211b0 2024-09-21T07:03:00.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 -- 172.21.15.114:0/335309957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373211b0 msgr2=0x5575377084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373211b0 0x5575377084a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc01e450 tx=0x7fb2bc014cd0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 -- 172.21.15.114:0/335309957 shutdown_connections 2024-09-21T07:03:00.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557537329390 0x5575377089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373211b0 0x5575377084a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 --2- 172.21.15.114:0/335309957 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575373089f0 0x5575376eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.325+0000 7fb2df227740 1 -- 172.21.15.114:0/335309957 >> 172.21.15.114:0/335309957 conn(0x55753733e460 msgr2=0x5575376ee5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.326+0000 7fb2df227740 1 -- 172.21.15.114:0/335309957 shutdown_connections 2024-09-21T07:03:00.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.327+0000 7fb2df227740 1 -- 172.21.15.114:0/335309957 wait complete. 2024-09-21T07:03:00.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.327+0000 7fb2df227740 1 -- start start 2024-09-21T07:03:00.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.328+0000 7fb2df227740 1 Processor -- start 2024-09-21T07:03:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.329+0000 7fb2df227740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373089f0 0x5575377747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2df227740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x557537704470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2df227740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557537329390 0x557537704a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2df227740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01b0340 con 0x557537329390 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2df227740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01afc10 con 0x5575373089f0 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2df227740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d01993e0 con 0x5575373211b0 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2df227740 10 client.0 _ll_register_callbacks cb 0x7fb2bc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x557537704470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2d5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x557537704470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33194/0 (socket says 172.21.15.114:33194) 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2d5200640 1 -- 172.21.15.114:0/3254969377 learned_addr learned my addr 172.21.15.114:0/3254969377 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:00.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.330+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373089f0 0x5575377747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.331+0000 7fb2d5c00640 1 -- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 msgr2=0x557537704470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.331+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x557537704470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.331+0000 7fb2d5c00640 1 -- 172.21.15.114:0/3254969377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557537329390 msgr2=0x557537704a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.331+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557537329390 0x557537704a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.331+0000 7fb2d5c00640 1 -- 172.21.15.114:0/3254969377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575377051c0 con 0x5575373089f0 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.332+0000 7fb2d5200640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x557537704470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.332+0000 7fb2d5c00640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373089f0 0x5575377747d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55753770ec40 tx=0x7fb2c4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b94f71733b72ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:00.334 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.332+0000 7fb2b7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:00.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.333+0000 7fb2b7400640 1 -- 172.21.15.114:0/3254969377 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40223f0 con 0x5575373089f0 2024-09-21T07:03:00.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.334+0000 7fb2b7400640 1 -- 172.21.15.114:0/3254969377 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fb2c40213a0 con 0x5575373089f0 2024-09-21T07:03:00.335 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.334+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575376659b0 con 0x5575373089f0 2024-09-21T07:03:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.334+0000 7fb2b7400640 1 -- 172.21.15.114:0/3254969377 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb2c40215c0 con 0x5575373089f0 2024-09-21T07:03:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.334+0000 7fb2df227740 10 client.27818 fetch_fsmap learned FSMap version 47 2024-09-21T07:03:00.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.335+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557537789f70 con 0x5575373089f0 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2b7400640 1 -- 172.21.15.114:0/3254969377 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575376659b0 con 0x5575373089f0 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2b7400640 1 -- 172.21.15.114:0/3254969377 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 47) ==== 8378+0+0 (secure 0 0 0) 0x7fb2c4022940 con 0x5575373089f0 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2df227740 10 client.27818 fetch_fsmap finished waiting for FSMap version 47 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2df227740 10 client.27818 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2df227740 20 client.27818 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2df227740 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55753734a070 0x55753778c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2df227740 4 client.27818 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:03:00.338 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.336+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557537662de0 con 0x55753734a070 2024-09-21T07:03:00.340 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.339+0000 7fb2d5200640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55753734a070 0x55753778c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.339+0000 7fb2d5200640 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55753734a070 0x55753778c670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b60b35a25bcee76 server_cookie=56f8564237ee5688 in_seq=0 out_seq=0 2024-09-21T07:03:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.339+0000 7fb2b7400640 10 client.27818 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:03:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.340+0000 7fb2b7400640 1 -- 172.21.15.114:0/3254969377 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557537662de0 con 0x55753734a070 2024-09-21T07:03:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.340+0000 7fb2b7400640 10 client.27818 handle_command_reply: tid=0 2024-09-21T07:03:00.341 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.340+0000 7fb2df227740 1 client.27818 shutdown 2024-09-21T07:03:00.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.341+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373089f0 msgr2=0x5575377747d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.342 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.341+0000 7fb2df227740 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373089f0 0x5575377747d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55753770ec40 tx=0x7fb2c4008040 comp rx=0 tx=0).stop 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 shutdown_connections 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55753734a070 0x55753778c670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557537329390 0x557537704a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575373211b0 0x557537704470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 --2- 172.21.15.114:0/3254969377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575373089f0 0x5575377747d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 >> 172.21.15.114:0/3254969377 conn(0x55753733e460 msgr2=0x5575376f3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2b7400640 0 client.27818 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 shutdown_connections 2024-09-21T07:03:00.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 1 -- 172.21.15.114:0/3254969377 wait complete. 2024-09-21T07:03:00.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.342+0000 7fb2df227740 20 client.27818 trim_cache size 0 max 16384 2024-09-21T07:03:00.348 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:03:00.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.346+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2b80b6700 msgr2=0x7fb2b80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.346+0000 7fb2d8600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2b80b6700 0x7fb2b80b8c10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc006e00 tx=0x7fb2bc008040 comp rx=0 tx=0).stop 2024-09-21T07:03:00.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.346+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 msgr2=0x7fb2d010cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:00.348 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.346+0000 7fb2d8600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 0x7fb2d010cd40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8003cc0 tx=0x7fb2c8003cf0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 shutdown_connections 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2b80b6700 0x7fb2b80b8c10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2d010de10 0x7fb2d010cd40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2d0105300 0x7fb2d019d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 --2- 172.21.15.114:0/2641333456 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2d01049b0 0x7fb2d019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 >> 172.21.15.114:0/2641333456 conn(0x7fb2d0079730 msgr2=0x7fb2d0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:00.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 shutdown_connections 2024-09-21T07:03:00.350 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:00.347+0000 7fb2d8600640 1 -- 172.21.15.114:0/2641333456 wait complete. 2024-09-21T07:03:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:00 smithi114 ceph-mon[30654]: pgmap v666: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-09-21T07:03:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:00 smithi183 ceph-mon[34219]: pgmap v666: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-09-21T07:03:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:00 smithi193 ceph-mon[34601]: pgmap v666: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.4 MiB/s wr, 444 op/s 2024-09-21T07:03:02.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 505.0 s, 555.4 tps, lat 178.525 ms stddev 233.830 2024-09-21T07:03:02.856 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:03:03.065 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:02 smithi114 ceph-mon[30654]: pgmap v667: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-21T07:03:03.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:02 smithi183 ceph-mon[34219]: pgmap v667: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-21T07:03:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:02 smithi193 ceph-mon[34601]: pgmap v667: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-21T07:03:03.576 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:04.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:04 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:03:04.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:04 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:03:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:04 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:03:05.038 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:05 smithi114 ceph-mon[30654]: pgmap v668: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-09-21T07:03:05.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:05 smithi183 ceph-mon[34219]: pgmap v668: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-09-21T07:03:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:05 smithi193 ceph-mon[34601]: pgmap v668: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 565 op/s 2024-09-21T07:03:06.194 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:06 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:06.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:06 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:06.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:06 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:03:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 -- 172.21.15.114:0/973337593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010c040 msgr2=0x7fa6c0112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 --2- 172.21.15.114:0/973337593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010c040 0x7fa6c0112870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009860 tx=0x7fa6b802a340 comp rx=0 tx=0).stop 2024-09-21T07:03:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 -- 172.21.15.114:0/973337593 shutdown_connections 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 --2- 172.21.15.114:0/973337593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010c040 0x7fa6c0112870 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 --2- 172.21.15.114:0/973337593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c010b6f0 0x7fa6c010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 --2- 172.21.15.114:0/973337593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c0105db0 0x7fa6c01061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 -- 172.21.15.114:0/973337593 >> 172.21.15.114:0/973337593 conn(0x7fa6c01017c0 msgr2=0x7fa6c0103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.176+0000 7fa6c8200640 1 -- 172.21.15.114:0/973337593 shutdown_connections 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.177+0000 7fa6c8200640 1 -- 172.21.15.114:0/973337593 wait complete. 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.178+0000 7fa6c8200640 1 Processor -- start 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.178+0000 7fa6c8200640 1 -- start start 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c8200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 0x7fa6c010fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c8200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 0x7fa6c01102b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.545 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c8200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c010c040 0x7fa6c01107f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c8200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0115d00 con 0x7fa6c0105db0 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c8200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0115b80 con 0x7fa6c010c040 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c8200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0115e80 con 0x7fa6c010b6f0 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.179+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 0x7fa6c010fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 0x7fa6c01102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 0x7fa6c010fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40296/0 (socket says 172.21.15.114:40296) 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 -- 172.21.15.114:0/4262386992 learned_addr learned my addr 172.21.15.114:0/4262386992 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c6200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c010c040 0x7fa6c01107f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c4e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 0x7fa6c01102b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47366/0 (socket says 172.21.15.114:47366) 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 -- 172.21.15.114:0/4262386992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 msgr2=0x7fa6c01102b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 0x7fa6c01102b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 -- 172.21.15.114:0/4262386992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c010c040 msgr2=0x7fa6c01107f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c010c040 0x7fa6c01107f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c5800640 1 -- 172.21.15.114:0/4262386992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01acc30 con 0x7fa6c0105db0 2024-09-21T07:03:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 0x7fa6c01102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.180+0000 7fa6c6200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c010c040 0x7fa6c01107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.181+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 0x7fa6c010fd70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea80 tx=0x7fa6b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.181+0000 7fa6be000640 1 -- 172.21.15.114:0/4262386992 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b400cd80 con 0x7fa6c0105db0 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.181+0000 7fa6be000640 1 -- 172.21.15.114:0/4262386992 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b4005020 con 0x7fa6c0105db0 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.181+0000 7fa6be000640 1 -- 172.21.15.114:0/4262386992 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40116d0 con 0x7fa6c0105db0 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.181+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01acf20 con 0x7fa6c0105db0 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.181+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01ad380 con 0x7fa6c0105db0 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.187+0000 7fa6cee31740 1 Processor -- start 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.190+0000 7fa6cee31740 1 -- start start 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.190+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a88070 0x556738e2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.190+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a5f1b0 0x556738e2f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.190+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a67390 0x556738e2f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.190+0000 7fa6cee31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556738da5fb0 con 0x556738a67390 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.190+0000 7fa6cee31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556738d953e0 con 0x556738a5f1b0 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.191+0000 7fa6cee31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556738d991f0 con 0x556738a88070 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.191+0000 7fa6c6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a88070 0x556738e2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.191+0000 7fa6c6200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a88070 0x556738e2eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47370/0 (socket says 172.21.15.114:47370) 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.191+0000 7fa6c6200640 1 -- 172.21.15.114:0/3735192644 learned_addr learned my addr 172.21.15.114:0/3735192644 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.191+0000 7fa6be000640 1 -- 172.21.15.114:0/4262386992 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa6b4011870 con 0x7fa6c0105db0 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.192+0000 7fa6c5800640 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a5f1b0 0x556738e2f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.193+0000 7fa6c5800640 1 -- 172.21.15.114:0/3735192644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a88070 msgr2=0x556738e2eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.193+0000 7fa6c5800640 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a88070 0x556738e2eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.193+0000 7fa6c5800640 1 -- 172.21.15.114:0/3735192644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a67390 msgr2=0x556738e2f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.193+0000 7fa6c5800640 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a67390 0x556738e2f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.193+0000 7fa6c5800640 1 -- 172.21.15.114:0/3735192644 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556738d5fcd0 con 0x556738a5f1b0 2024-09-21T07:03:06.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.193+0000 7fa6c5800640 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a5f1b0 0x556738e2f2e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40e4fe0 tx=0x7fa6b40f0710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba14bce766abdd7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.194+0000 7fa6b2a00640 1 -- 172.21.15.114:0/3735192644 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40f2620 con 0x556738a5f1b0 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.194+0000 7fa6be000640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6a80b67b0 0x7fa6a80b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.194+0000 7fa6b2a00640 1 -- 172.21.15.114:0/3735192644 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b40f2c70 con 0x556738a5f1b0 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.195+0000 7fa6be000640 1 -- 172.21.15.114:0/4262386992 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fa6b40e7cc0 con 0x7fa6c0105db0 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.196+0000 7fa6cee31740 1 -- 172.21.15.114:0/3735192644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a5f1b0 msgr2=0x556738e2f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.196+0000 7fa6cee31740 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a5f1b0 0x556738e2f2e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40e4fe0 tx=0x7fa6b40f0710 comp rx=0 tx=0).stop 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6c6200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6a80b67b0 0x7fa6a80b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 -- 172.21.15.114:0/3735192644 shutdown_connections 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a67390 0x556738e2f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a5f1b0 0x556738e2f2e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 --2- 172.21.15.114:0/3735192644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a88070 0x556738e2eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 -- 172.21.15.114:0/3735192644 >> 172.21.15.114:0/3735192644 conn(0x556738a7c460 msgr2=0x556738e2cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 -- 172.21.15.114:0/3735192644 shutdown_connections 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.197+0000 7fa6cee31740 1 -- 172.21.15.114:0/3735192644 wait complete. 2024-09-21T07:03:06.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.199+0000 7fa6cee31740 1 -- start start 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.199+0000 7fa6c6200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6a80b67b0 0x7fa6a80b8cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa6b802a8b0 tx=0x7fa6b802c040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.199+0000 7fa6cee31740 1 Processor -- start 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.200+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.200+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738e3a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a88070 0x556738e579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6cee31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556738bf6700 con 0x556738a5f1b0 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6cee31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556738d30580 con 0x556738a88070 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6cee31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556738e4e5e0 con 0x556738a67390 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40322/0 (socket says 172.21.15.114:40322) 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 -- 172.21.15.114:0/4071754013 learned_addr learned my addr 172.21.15.114:0/4071754013 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6cee31740 10 client.0 _ll_register_callbacks cb 0x556738d40470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738e3a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c6200640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a88070 0x556738e579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.552 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 -- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 msgr2=0x556738e3a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738e3a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 -- 172.21.15.114:0/4071754013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a88070 msgr2=0x556738e579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a88070 0x556738e579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.201+0000 7fa6c5800640 1 -- 172.21.15.114:0/4071754013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556738af3650 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.202+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3a3f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400efd0 tx=0x7fa6b400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e157a34f7bea9cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.202+0000 7fa6a7400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.202+0000 7fa6a7400640 1 -- 172.21.15.114:0/4071754013 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b40ac1f0 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.202+0000 7fa6a7400640 1 -- 172.21.15.114:0/4071754013 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fa6b40ab5f0 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.202+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556738d32b90 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.203+0000 7fa6a7400640 1 -- 172.21.15.114:0/4071754013 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa6b401f020 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.203+0000 7fa6cee31740 10 client.17406 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.203+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556738d4d950 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.203+0000 7fa6a7400640 1 -- 172.21.15.114:0/4071754013 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b40eb9e0 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6a7400640 1 -- 172.21.15.114:0/4071754013 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7fa6b400f040 con 0x556738a5f1b0 2024-09-21T07:03:06.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6cee31740 10 client.17406 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6cee31740 10 client.17406 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6cee31740 20 client.17406 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a469f0 0x556738e5d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6cee31740 4 client.17406 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.204+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556738d32b90 con 0x556738a469f0 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.205+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a469f0 0x556738e5d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.206+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a469f0 0x556738e5d4f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf1221d2828f4f87 server_cookie=67d0641d25797a1d in_seq=0 out_seq=0 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.206+0000 7fa6a7400640 10 client.17406 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.207+0000 7fa6a7400640 1 -- 172.21.15.114:0/4071754013 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556738d32b90 con 0x556738a469f0 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.207+0000 7fa6a7400640 10 client.17406 handle_command_reply: tid=0 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.207+0000 7fa6cee31740 1 client.17406 shutdown 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.211+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 msgr2=0x556738e3a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.211+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3a3f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400efd0 tx=0x7fa6b400cf20 comp rx=0 tx=0).stop 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 shutdown_connections 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a469f0 0x556738e5d4f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6a7400640 0 client.17406 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a88070 0x556738e579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738e3a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4071754013 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3a3f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 >> 172.21.15.114:0/4071754013 conn(0x556738a7c460 msgr2=0x556738e31f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.212+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 shutdown_connections 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.213+0000 7fa6cee31740 1 -- 172.21.15.114:0/4071754013 wait complete. 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.213+0000 7fa6cee31740 20 client.17406 trim_cache size 0 max 16384 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 Processor -- start 2024-09-21T07:03:06.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 -- start start 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a469f0 0x556738e308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a67390 0x556738e57910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b10c0 con 0x556738a5f1b0 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01aee80 con 0x556738a67390 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.234+0000 7fa6cee31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b09f0 con 0x556738a469f0 2024-09-21T07:03:06.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a469f0 0x556738e308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c6200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a67390 0x556738e57910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a469f0 0x556738e308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47390/0 (socket says 172.21.15.114:47390) 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c5800640 1 -- 172.21.15.114:0/1770644604 learned_addr learned my addr 172.21.15.114:0/1770644604 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c6200640 1 -- 172.21.15.114:0/1770644604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a469f0 msgr2=0x556738e308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c6200640 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a469f0 0x556738e308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.235+0000 7fa6c6200640 1 -- 172.21.15.114:0/1770644604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 msgr2=0x556738e3c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.236+0000 7fa6c6200640 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.236+0000 7fa6c6200640 1 -- 172.21.15.114:0/1770644604 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556738e35f60 con 0x556738a67390 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.236+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:06.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.236+0000 7fa6c6200640 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a67390 0x556738e57910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005eb0 tx=0x7fa6b803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d92aeeadbea99fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.236+0000 7fa6b2000640 1 -- 172.21.15.114:0/1770644604 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8044070 con 0x556738a67390 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.237+0000 7fa6b2000640 1 -- 172.21.15.114:0/1770644604 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8050070 con 0x556738a67390 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.237+0000 7fa6b2000640 1 -- 172.21.15.114:0/1770644604 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8037040 con 0x556738a67390 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 -- 172.21.15.114:0/1770644604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a67390 msgr2=0x556738e57910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a67390 0x556738e57910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8005eb0 tx=0x7fa6b803e000 comp rx=0 tx=0).stop 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 -- 172.21.15.114:0/1770644604 shutdown_connections 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a67390 0x556738e57910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e3c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 --2- 172.21.15.114:0/1770644604 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a469f0 0x556738e308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 -- 172.21.15.114:0/1770644604 >> 172.21.15.114:0/1770644604 conn(0x556738a7c460 msgr2=0x556738e38c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.238+0000 7fa6cee31740 1 -- 172.21.15.114:0/1770644604 shutdown_connections 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.239+0000 7fa6cee31740 1 -- 172.21.15.114:0/1770644604 wait complete. 2024-09-21T07:03:06.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.240+0000 7fa6cee31740 1 -- start start 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.240+0000 7fa6cee31740 1 Processor -- start 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a469f0 0x556738e5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e5ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738ec6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01afc20 con 0x556738a5f1b0 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01af550 con 0x556738a469f0 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01b0320 con 0x556738a67390 2024-09-21T07:03:06.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6cee31740 10 client.0 _ll_register_callbacks cb 0x7fa6b40e5010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:06.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a469f0 0x556738e5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6c5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a469f0 0x556738e5e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:47960/0 (socket says 172.21.15.114:47960) 2024-09-21T07:03:06.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6c5800640 1 -- 172.21.15.114:0/4267278449 learned_addr learned my addr 172.21.15.114:0/4267278449 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:06.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6c6200640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738ec6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.241+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e5ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6c4e00640 1 -- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 msgr2=0x556738ec6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738ec6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6c4e00640 1 -- 172.21.15.114:0/4267278449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a469f0 msgr2=0x556738e5e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a469f0 0x556738e5e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6c4e00640 1 -- 172.21.15.114:0/4267278449 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556738ec71b0 con 0x556738a5f1b0 2024-09-21T07:03:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6c4e00640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e5ef30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004700 tx=0x7fa6ac012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bee59178ab7e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.242+0000 7fa6a7400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.243+0000 7fa6a7400640 1 -- 172.21.15.114:0/4267278449 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac02a060 con 0x556738a5f1b0 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.243+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b8050070 con 0x556738a5f1b0 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.243+0000 7fa6a7400640 1 -- 172.21.15.114:0/4267278449 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7fa6ac00f8f0 con 0x556738a5f1b0 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.243+0000 7fa6a7400640 1 -- 172.21.15.114:0/4267278449 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa6ac031050 con 0x556738a5f1b0 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.243+0000 7fa6cee31740 10 client.17412 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.243+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556738ec83a0 con 0x556738a5f1b0 2024-09-21T07:03:06.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6a7400640 1 -- 172.21.15.114:0/4267278449 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8050070 con 0x556738a5f1b0 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6a7400640 1 -- 172.21.15.114:0/4267278449 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7fa6ac01fa70 con 0x556738a5f1b0 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6cee31740 10 client.17412 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6cee31740 10 client.17412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6cee31740 20 client.17412 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a88070 0x556738ecaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6cee31740 4 client.17412 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.244+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556738da7f00 con 0x556738a88070 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.245+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a88070 0x556738ecaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.245+0000 7fa6c5800640 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a88070 0x556738ecaae0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebbe4a2904f82ecd server_cookie=f3dfcbe3840a268d in_seq=0 out_seq=0 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.245+0000 7fa6a7400640 10 client.17412 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.245+0000 7fa6a7400640 1 -- 172.21.15.114:0/4267278449 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556738da7f00 con 0x556738a88070 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.245+0000 7fa6a7400640 10 client.17412 handle_command_reply: tid=0 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 client.17412 shutdown 2024-09-21T07:03:06.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 msgr2=0x556738e5ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e5ef30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004700 tx=0x7fa6ac012dd0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 shutdown_connections 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x556738a88070 0x556738ecaae0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556738a67390 0x556738ec6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556738a5f1b0 0x556738e5ef30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 --2- 172.21.15.114:0/4267278449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556738a469f0 0x556738e5e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6a7400640 0 client.17412 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.246+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 >> 172.21.15.114:0/4267278449 conn(0x556738a7c460 msgr2=0x556738e3b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.247+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 shutdown_connections 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.247+0000 7fa6cee31740 1 -- 172.21.15.114:0/4267278449 wait complete. 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.247+0000 7fa6cee31740 20 client.17412 trim_cache size 0 max 16384 2024-09-21T07:03:06.564 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6a80b67b0 msgr2=0x7fa6a80b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6a80b67b0 0x7fa6a80b8cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa6b802a8b0 tx=0x7fa6b802c040 comp rx=0 tx=0).stop 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 msgr2=0x7fa6c010fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 0x7fa6c010fd70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa6b400ea80 tx=0x7fa6b400ee80 comp rx=0 tx=0).stop 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 shutdown_connections 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6a80b67b0 0x7fa6a80b8cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c010c040 0x7fa6c01107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6c010b6f0 0x7fa6c01102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 --2- 172.21.15.114:0/4262386992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6c0105db0 0x7fa6c010fd70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:06.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 >> 172.21.15.114:0/4262386992 conn(0x7fa6c01017c0 msgr2=0x7fa6c0109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:06.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 shutdown_connections 2024-09-21T07:03:06.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:06.249+0000 7fa6c8200640 1 -- 172.21.15.114:0/4262386992 wait complete. 2024-09-21T07:03:07.224 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:07 smithi193 ceph-mon[34601]: pgmap v669: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-09-21T07:03:07.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:07 smithi114 ceph-mon[30654]: pgmap v669: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-09-21T07:03:07.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:07 smithi183 ceph-mon[34219]: pgmap v669: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-09-21T07:03:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.426+0000 7f3706000640 1 -- 172.21.15.114:0/3776660441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 msgr2=0x7f370010faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.426+0000 7f3706000640 1 --2- 172.21.15.114:0/3776660441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010faf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009860 tx=0x7f36f002a340 comp rx=0 tx=0).stop 2024-09-21T07:03:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.428+0000 7f3706000640 1 -- 172.21.15.114:0/3776660441 shutdown_connections 2024-09-21T07:03:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.428+0000 7f3706000640 1 --2- 172.21.15.114:0/3776660441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3700110030 0x7f3700112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.428+0000 7f3706000640 1 --2- 172.21.15.114:0/3776660441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010faf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.428+0000 7f3706000640 1 --2- 172.21.15.114:0/3776660441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37000757b0 0x7f3700075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.428+0000 7f3706000640 1 -- 172.21.15.114:0/3776660441 >> 172.21.15.114:0/3776660441 conn(0x7f37000ff520 msgr2=0x7f3700101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.430+0000 7f3706000640 1 -- 172.21.15.114:0/3776660441 shutdown_connections 2024-09-21T07:03:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.430+0000 7f3706000640 1 -- 172.21.15.114:0/3776660441 wait complete. 2024-09-21T07:03:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.433+0000 7f3706000640 1 Processor -- start 2024-09-21T07:03:07.434 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.433+0000 7f3706000640 1 -- start start 2024-09-21T07:03:07.435 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f3706000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37000757b0 0x7f370010eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f3706000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f3706000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3700110030 0x7f3700115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f3706000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3700104820 con 0x7f3700110030 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f3706000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37001046a0 con 0x7f3700076100 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f3706000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37001049a0 con 0x7f37000757b0 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f36fea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f36fea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:47990/0 (socket says 172.21.15.114:47990) 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.434+0000 7f36fea00640 1 -- 172.21.15.114:0/423253997 learned_addr learned my addr 172.21.15.114:0/423253997 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36ff400640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37000757b0 0x7f370010eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36ffe00640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3700110030 0x7f3700115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36fea00640 1 -- 172.21.15.114:0/423253997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37000757b0 msgr2=0x7f370010eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36fea00640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37000757b0 0x7f370010eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36fea00640 1 -- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3700110030 msgr2=0x7f3700115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36fea00640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3700110030 0x7f3700115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.435+0000 7f36fea00640 1 -- 172.21.15.114:0/423253997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3700116230 con 0x7f3700076100 2024-09-21T07:03:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.436+0000 7f36fea00640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010f0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f36f0035040 tx=0x7f36f0032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.437+0000 7f36f7e00640 1 -- 172.21.15.114:0/423253997 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0037040 con 0x7f3700076100 2024-09-21T07:03:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.437+0000 7f36f7e00640 1 -- 172.21.15.114:0/423253997 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f0030c90 con 0x7f3700076100 2024-09-21T07:03:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.437+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37001164c0 con 0x7f3700076100 2024-09-21T07:03:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.437+0000 7f36f7e00640 1 -- 172.21.15.114:0/423253997 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f0003850 con 0x7f3700076100 2024-09-21T07:03:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.437+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3700116950 con 0x7f3700076100 2024-09-21T07:03:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.443+0000 7f370d0a0740 1 Processor -- start 2024-09-21T07:03:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.443+0000 7f370d0a0740 1 -- start start 2024-09-21T07:03:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.444+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e44c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.444+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.444+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e456b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.444+0000 7f370d0a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55af06dba6b0 con 0x55af06a772f0 2024-09-21T07:03:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.444+0000 7f370d0a0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af06da9ae0 con 0x55af06a7f4d0 2024-09-21T07:03:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.444+0000 7f370d0a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af06dad8f0 con 0x55af06aa01b0 2024-09-21T07:03:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.446+0000 7f36ffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e44c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.446+0000 7f36ffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e44c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47424/0 (socket says 172.21.15.114:47424) 2024-09-21T07:03:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.446+0000 7f36ffe00640 1 -- 172.21.15.114:0/877470282 learned_addr learned my addr 172.21.15.114:0/877470282 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.447+0000 7f36ffe00640 1 -- 172.21.15.114:0/877470282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 msgr2=0x55af06e456b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:03:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.447+0000 7f36ffe00640 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e456b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.447+0000 7f36ffe00640 1 -- 172.21.15.114:0/877470282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 msgr2=0x55af06e45170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.447+0000 7f36ffe00640 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.447+0000 7f36ffe00640 1 -- 172.21.15.114:0/877470282 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af06d76780 con 0x55af06aa01b0 2024-09-21T07:03:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.448+0000 7f36f7e00640 1 -- 172.21.15.114:0/423253997 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f36f0043030 con 0x7f3700076100 2024-09-21T07:03:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.448+0000 7f36ffe00640 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e44c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00bca0 tx=0x7f36ec00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa259ca98e2f2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.451+0000 7f36ebe00640 1 -- 172.21.15.114:0/877470282 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec015070 con 0x55af06aa01b0 2024-09-21T07:03:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.451+0000 7f36ebe00640 1 -- 172.21.15.114:0/877470282 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36ec01a910 con 0x55af06aa01b0 2024-09-21T07:03:07.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.451+0000 7f36ebe00640 1 -- 172.21.15.114:0/877470282 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec015070 con 0x55af06aa01b0 2024-09-21T07:03:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.456+0000 7f36f7e00640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e40b67d0 0x7f36e40b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.456+0000 7f36f7e00640 1 -- 172.21.15.114:0/423253997 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f36f0106f60 con 0x7f3700076100 2024-09-21T07:03:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 -- 172.21.15.114:0/877470282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 msgr2=0x55af06e44c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e44c30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00bca0 tx=0x7f36ec00f730 comp rx=0 tx=0).stop 2024-09-21T07:03:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 -- 172.21.15.114:0/877470282 shutdown_connections 2024-09-21T07:03:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 --2- 172.21.15.114:0/877470282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e44c30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.457+0000 7f370d0a0740 1 -- 172.21.15.114:0/877470282 >> 172.21.15.114:0/877470282 conn(0x55af06a94300 msgr2=0x55af06e42b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.460+0000 7f370d0a0740 1 -- 172.21.15.114:0/877470282 shutdown_connections 2024-09-21T07:03:07.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.460+0000 7f370d0a0740 1 -- 172.21.15.114:0/877470282 wait complete. 2024-09-21T07:03:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.461+0000 7f370d0a0740 1 -- start start 2024-09-21T07:03:07.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.462+0000 7f370d0a0740 1 Processor -- start 2024-09-21T07:03:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.462+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 0x55af06e503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.462+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e508f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.462+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06aa01b0 0x55af06e5c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.463+0000 7f36ff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 0x55af06e503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.463+0000 7f36fea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.463+0000 7f36fea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:47996/0 (socket says 172.21.15.114:47996) 2024-09-21T07:03:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.463+0000 7f36ff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 0x55af06e503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47434/0 (socket says 172.21.15.114:47434) 2024-09-21T07:03:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.464+0000 7f36fea00640 1 -- 172.21.15.114:0/3368762238 learned_addr learned my addr 172.21.15.114:0/3368762238 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.464+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55af06ce1ab0 con 0x55af06aa01b0 2024-09-21T07:03:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.464+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af06d50570 con 0x55af06a7f4d0 2024-09-21T07:03:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.464+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55af06e533b0 con 0x55af06a772f0 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.464+0000 7f370d0a0740 10 client.0 _ll_register_callbacks cb 0x55af06d5f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.464+0000 7f36ffe00640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06aa01b0 0x55af06e5c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.465+0000 7f36ffe00640 1 -- 172.21.15.114:0/3368762238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 msgr2=0x55af06e503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.466+0000 7f36ffe00640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 0x55af06e503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.466+0000 7f36ffe00640 1 -- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 msgr2=0x55af06e508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.466+0000 7f36ffe00640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.466+0000 7f36ffe00640 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af06b0b790 con 0x55af06aa01b0 2024-09-21T07:03:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.466+0000 7f36ff400640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 0x55af06e503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.468+0000 7f36ff400640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e40b67d0 0x7f36e40b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.470+0000 7f36ffe00640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06aa01b0 0x55af06e5c6d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00bca0 tx=0x7f36ec01a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30570616816d372 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.470+0000 7f36e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.470+0000 7f36e8c00640 1 -- 172.21.15.114:0/3368762238 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec028060 con 0x55af06aa01b0 2024-09-21T07:03:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.471+0000 7f36e8c00640 1 -- 172.21.15.114:0/3368762238 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f36ec005040 con 0x55af06aa01b0 2024-09-21T07:03:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.471+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36ec01a910 con 0x55af06aa01b0 2024-09-21T07:03:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.471+0000 7f36ff400640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e40b67d0 0x7f36e40b8ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55af06e52c00 tx=0x7f36f8012040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.472+0000 7f36e8c00640 1 -- 172.21.15.114:0/3368762238 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f36ec015040 con 0x55af06aa01b0 2024-09-21T07:03:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.475+0000 7f370d0a0740 10 client.17421 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.475+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55af06b84620 con 0x55af06aa01b0 2024-09-21T07:03:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.475+0000 7f36e8c00640 1 -- 172.21.15.114:0/3368762238 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36ec01a910 con 0x55af06aa01b0 2024-09-21T07:03:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.477+0000 7f36e8c00640 1 -- 172.21.15.114:0/3368762238 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f36ec022030 con 0x55af06aa01b0 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f370d0a0740 10 client.17421 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f370d0a0740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f370d0a0740 20 client.17421 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f370d0a0740 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06a5ed80 0x55af06e623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f370d0a0740 4 client.17421 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af06dbc600 con 0x55af06a5ed80 2024-09-21T07:03:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.478+0000 7f36ff400640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06a5ed80 0x55af06e623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.480+0000 7f36ff400640 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06a5ed80 0x55af06e623b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5411a6735575f88 server_cookie=ac21ccaf0d857557 in_seq=0 out_seq=0 2024-09-21T07:03:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.480+0000 7f36e8c00640 10 client.17421 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.483+0000 7f36e8c00640 1 -- 172.21.15.114:0/3368762238 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55af06dbc600 con 0x55af06a5ed80 2024-09-21T07:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.483+0000 7f36e8c00640 10 client.17421 handle_command_reply: tid=0 2024-09-21T07:03:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.483+0000 7f370d0a0740 1 client.17421 shutdown 2024-09-21T07:03:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.483+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06aa01b0 msgr2=0x55af06e5c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.483+0000 7f370d0a0740 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06aa01b0 0x55af06e5c6d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00bca0 tx=0x7f36ec01a690 comp rx=0 tx=0).stop 2024-09-21T07:03:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 shutdown_connections 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06aa01b0 0x55af06e5c6d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06e508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06a5ed80 0x55af06e623b0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 --2- 172.21.15.114:0/3368762238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a772f0 0x55af06e503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f36e8c00640 0 client.17421 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 >> 172.21.15.114:0/3368762238 conn(0x55af06a94300 msgr2=0x55af06e45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.484+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 shutdown_connections 2024-09-21T07:03:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.485+0000 7f370d0a0740 1 -- 172.21.15.114:0/3368762238 wait complete. 2024-09-21T07:03:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.485+0000 7f370d0a0740 20 client.17421 trim_cache size 0 max 16384 2024-09-21T07:03:07.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.528+0000 7f370d0a0740 1 Processor -- start 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 -- start start 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a7f4d0 0x55af06e5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06aa01b0 0x55af06e5cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37001bfd30 con 0x55af06a772f0 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37001c18a0 con 0x55af06aa01b0 2024-09-21T07:03:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.529+0000 7f370d0a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37001c0400 con 0x55af06a7f4d0 2024-09-21T07:03:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.530+0000 7f36ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.530+0000 7f36ff400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40390/0 (socket says 172.21.15.114:40390) 2024-09-21T07:03:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.530+0000 7f36ff400640 1 -- 172.21.15.114:0/1930360444 learned_addr learned my addr 172.21.15.114:0/1930360444 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.531+0000 7f36ff400640 1 -- 172.21.15.114:0/1930360444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a7f4d0 msgr2=0x55af06e5c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.531+0000 7f36ff400640 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a7f4d0 0x55af06e5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.531+0000 7f36ff400640 1 -- 172.21.15.114:0/1930360444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06aa01b0 msgr2=0x55af06e5cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.531+0000 7f36ff400640 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06aa01b0 0x55af06e5cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.531+0000 7f36ff400640 1 -- 172.21.15.114:0/1930360444 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af06e4e8b0 con 0x55af06a772f0 2024-09-21T07:03:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.531+0000 7f36ff400640 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45200 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3700119e20 tx=0x7f36f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5391219c6c28290e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.532+0000 7f36eb400640 1 -- 172.21.15.114:0/1930360444 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f801bc60 con 0x55af06a772f0 2024-09-21T07:03:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.532+0000 7f36eb400640 1 -- 172.21.15.114:0/1930360444 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f8032070 con 0x55af06a772f0 2024-09-21T07:03:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.532+0000 7f36eb400640 1 -- 172.21.15.114:0/1930360444 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8027650 con 0x55af06a772f0 2024-09-21T07:03:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 -- 172.21.15.114:0/1930360444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 msgr2=0x55af06e45200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45200 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3700119e20 tx=0x7f36f8004a40 comp rx=0 tx=0).stop 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 -- 172.21.15.114:0/1930360444 shutdown_connections 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06aa01b0 0x55af06e5cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06a7f4d0 0x55af06e5c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 --2- 172.21.15.114:0/1930360444 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a772f0 0x55af06e45200 secure :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3700119e20 tx=0x7f36f8004a40 comp rx=0 tx=0).stop 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.534+0000 7f370d0a0740 1 -- 172.21.15.114:0/1930360444 >> 172.21.15.114:0/1930360444 conn(0x55af06a94300 msgr2=0x55af06e410e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.535+0000 7f370d0a0740 1 -- 172.21.15.114:0/1930360444 shutdown_connections 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.535+0000 7f370d0a0740 1 -- 172.21.15.114:0/1930360444 wait complete. 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.536+0000 7f370d0a0740 1 -- start start 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.536+0000 7f370d0a0740 1 Processor -- start 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06eca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a341c0 0x55af06e58b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f36ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06eca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f36ff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06eca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48026/0 (socket says 172.21.15.114:48026) 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37001c11d0 con 0x55af06a341c0 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37001c0ad0 con 0x55af06a7f4d0 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37001bdd30 con 0x55af06aa01b0 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f36ff400640 1 -- 172.21.15.114:0/4191738058 learned_addr learned my addr 172.21.15.114:0/4191738058 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.538+0000 7f370d0a0740 10 client.0 _ll_register_callbacks cb 0x7f36ec00bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.539+0000 7f36fea00640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.539+0000 7f36ffe00640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a341c0 0x55af06e58b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.540+0000 7f36fea00640 1 -- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 msgr2=0x55af06eca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.540+0000 7f36fea00640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06eca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.540+0000 7f36fea00640 1 -- 172.21.15.114:0/4191738058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a341c0 msgr2=0x55af06e58b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.540+0000 7f36fea00640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a341c0 0x55af06e58b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.540+0000 7f36fea00640 1 -- 172.21.15.114:0/4191738058 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af06e590a0 con 0x55af06aa01b0 2024-09-21T07:03:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.540+0000 7f36ff400640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06eca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.542+0000 7f36fea00640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e58620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f36f00cc750 tx=0x7f36f0009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee47cabf9d9c7b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.542+0000 7f36e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.549+0000 7f36e8c00640 1 -- 172.21.15.114:0/4191738058 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f00d1020 con 0x55af06aa01b0 2024-09-21T07:03:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.550+0000 7f36e8c00640 1 -- 172.21.15.114:0/4191738058 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9490+0+0 (secure 0 0 0) 0x7f36f00d11c0 con 0x55af06aa01b0 2024-09-21T07:03:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.550+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36f8032070 con 0x55af06aa01b0 2024-09-21T07:03:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.550+0000 7f36e8c00640 1 -- 172.21.15.114:0/4191738058 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f36f00cc3a0 con 0x55af06aa01b0 2024-09-21T07:03:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.550+0000 7f370d0a0740 10 client.27787 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.551+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55af06e68150 con 0x55af06aa01b0 2024-09-21T07:03:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.551+0000 7f36e8c00640 1 -- 172.21.15.114:0/4191738058 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36f8032070 con 0x55af06aa01b0 2024-09-21T07:03:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.551+0000 7f36e8c00640 1 -- 172.21.15.114:0/4191738058 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f36f00d5f50 con 0x55af06aa01b0 2024-09-21T07:03:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.552+0000 7f370d0a0740 10 client.27787 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.552+0000 7f370d0a0740 10 client.27787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.552+0000 7f370d0a0740 20 client.27787 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.552+0000 7f370d0a0740 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06ede090 0x55af06ee04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.553+0000 7f370d0a0740 4 client.27787 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:03:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.553+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af06db80b0 con 0x55af06ede090 2024-09-21T07:03:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.553+0000 7f36ff400640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06ede090 0x55af06ee04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.555+0000 7f36ff400640 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06ede090 0x55af06ee04b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920675cd8428973b server_cookie=54342203847afc7b in_seq=0 out_seq=0 2024-09-21T07:03:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.555+0000 7f36e8c00640 10 client.27787 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.700+0000 7f36e8c00640 1 -- 172.21.15.114:0/4191738058 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af06db80b0 con 0x55af06ede090 2024-09-21T07:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.700+0000 7f36e8c00640 10 client.27787 handle_command_reply: tid=0 2024-09-21T07:03:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.700+0000 7f370d0a0740 1 client.27787 shutdown 2024-09-21T07:03:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 msgr2=0x55af06e58620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e58620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f36f00cc750 tx=0x7f36f0009550 comp rx=0 tx=0).stop 2024-09-21T07:03:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 shutdown_connections 2024-09-21T07:03:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55af06a341c0 0x55af06e58b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55af06ede090 0x55af06ee04b0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55af06aa01b0 0x55af06e58620 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 --2- 172.21.15.114:0/4191738058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af06a7f4d0 0x55af06eca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 >> 172.21.15.114:0/4191738058 conn(0x55af06a94300 msgr2=0x55af06e461a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f36e8c00640 0 client.27787 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.701+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 shutdown_connections 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.702+0000 7f370d0a0740 1 -- 172.21.15.114:0/4191738058 wait complete. 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.702+0000 7f370d0a0740 20 client.27787 trim_cache size 0 max 16384 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:03:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.702+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e40b67d0 msgr2=0x7f36e40b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.702+0000 7f3706000640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e40b67d0 0x7f36e40b8ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55af06e52c00 tx=0x7f36f8012040 comp rx=0 tx=0).stop 2024-09-21T07:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.702+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 msgr2=0x7f370010f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.702+0000 7f3706000640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010f0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f36f0035040 tx=0x7f36f0032dc0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 shutdown_connections 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3700110030 0x7f3700115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f36e40b67d0 0x7f36e40b8ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3700076100 0x7f370010f0b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 --2- 172.21.15.114:0/423253997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37000757b0 0x7f370010eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 >> 172.21.15.114:0/423253997 conn(0x7f37000ff520 msgr2=0x7f3700105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 shutdown_connections 2024-09-21T07:03:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:07.703+0000 7f3706000640 1 -- 172.21.15.114:0/423253997 wait complete. 2024-09-21T07:03:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 510.0 s, 494.2 tps, lat 205.727 ms stddev 297.964 2024-09-21T07:03:08.820 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: pgmap v670: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.6 MiB/s wr, 589 op/s 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:08.952 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:08 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: pgmap v670: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.6 MiB/s wr, 589 op/s 2024-09-21T07:03:09.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:08 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: pgmap v670: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 5.6 MiB/s wr, 589 op/s 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:08 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:10.854 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:11.280 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:10 smithi114 ceph-mon[30654]: pgmap v671: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-09-21T07:03:11.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:10 smithi183 ceph-mon[34219]: pgmap v671: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-09-21T07:03:11.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:10 smithi193 ceph-mon[34601]: pgmap v671: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.5 MiB/s wr, 481 op/s 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.967+0000 7f48d2e00640 1 -- 172.21.15.114:0/2835470874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 msgr2=0x7f48cc0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 --2- 172.21.15.114:0/2835470874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc0761e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f48b801a910 tx=0x7f48b803b470 comp rx=0 tx=0).stop 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 -- 172.21.15.114:0/2835470874 shutdown_connections 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 --2- 172.21.15.114:0/2835470874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 0x7f48cc116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 --2- 172.21.15.114:0/2835470874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc0761e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 --2- 172.21.15.114:0/2835470874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48cc077380 0x7f48cc0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 -- 172.21.15.114:0/2835470874 >> 172.21.15.114:0/2835470874 conn(0x7f48cc0ff5e0 msgr2=0x7f48cc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.968+0000 7f48d2e00640 1 -- 172.21.15.114:0/2835470874 shutdown_connections 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.970+0000 7f48d2e00640 1 -- 172.21.15.114:0/2835470874 wait complete. 2024-09-21T07:03:12.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.972+0000 7f48d2e00640 1 Processor -- start 2024-09-21T07:03:12.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.973+0000 7f48d2e00640 1 -- start start 2024-09-21T07:03:12.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.974+0000 7f48d2e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc113ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.975+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc113ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.975+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc113ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47460/0 (socket says 172.21.15.114:47460) 2024-09-21T07:03:12.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.975+0000 7f48cbe00640 1 -- 172.21.15.114:0/1599720610 learned_addr learned my addr 172.21.15.114:0/1599720610 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:12.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.975+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48cc077380 0x7f48cc10fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.976+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 0x7f48cc1102d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.977+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10a420 con 0x7f48cc077380 2024-09-21T07:03:12.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.977+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10a2a0 con 0x7f48cc114440 2024-09-21T07:03:12.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.977+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10a5a0 con 0x7f48cc075dd0 2024-09-21T07:03:12.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.977+0000 7f48d0e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 0x7f48cc1102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.978+0000 7f48cb400640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48cc077380 0x7f48cc10fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.979+0000 7f48cbe00640 1 -- 172.21.15.114:0/1599720610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 msgr2=0x7f48cc1102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.979+0000 7f48cbe00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 0x7f48cc1102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.979+0000 7f48cbe00640 1 -- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48cc077380 msgr2=0x7f48cc10fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.980+0000 7f48cbe00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48cc077380 0x7f48cc10fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.980+0000 7f48cbe00640 1 -- 172.21.15.114:0/1599720610 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc110a00 con 0x7f48cc075dd0 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.981+0000 7f48d0e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 0x7f48cc1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:12.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.982+0000 7f48cbe00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc113ab0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d950 tx=0x7f48bc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.983+0000 7f48c8c00640 1 -- 172.21.15.114:0/1599720610 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc017070 con 0x7f48cc075dd0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.983+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc1a45a0 con 0x7f48cc075dd0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.983+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1a4b00 con 0x7f48cc075dd0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.983+0000 7f48c8c00640 1 -- 172.21.15.114:0/1599720610 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48bc009c30 con 0x7f48cc075dd0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.983+0000 7f48c8c00640 1 -- 172.21.15.114:0/1599720610 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00f620 con 0x7f48cc075dd0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.991+0000 7f48c8c00640 1 -- 172.21.15.114:0/1599720610 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f48bc00f7c0 con 0x7f48cc075dd0 2024-09-21T07:03:12.136 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.994+0000 7f48c8c00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48b00b6700 0x7f48b00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.995+0000 7f48c8c00640 1 -- 172.21.15.114:0/1599720610 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9578+0+0 (secure 0 0 0) 0x7f48bc0eb170 con 0x7f48cc075dd0 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:11.996+0000 7f48cb400640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48b00b6700 0x7f48b00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.005+0000 7f48cb400640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48b00b6700 0x7f48b00b8c10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f48b801a860 tx=0x7f48b8005ee0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.008+0000 7f48d9a31740 1 Processor -- start 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.009+0000 7f48d9a31740 1 -- start start 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413ca8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413ca90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413ca9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48d9a31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560413c1ffb0 con 0x5604138e1390 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48d9a31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560413c0f3e0 con 0x560413902070 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48d9a31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560413c131f0 con 0x5604138d91b0 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.011+0000 7f48cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413ca90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.012+0000 7f48cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413ca90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47470/0 (socket says 172.21.15.114:47470) 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.012+0000 7f48cb400640 1 -- 172.21.15.114:0/4201253920 learned_addr learned my addr 172.21.15.114:0/4201253920 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:12.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.012+0000 7f48d0e00640 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413ca9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.012+0000 7f48cbe00640 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413ca8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.013+0000 7f48cb400640 1 -- 172.21.15.114:0/4201253920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 msgr2=0x560413ca8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.013+0000 7f48cb400640 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413ca8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.013+0000 7f48cb400640 1 -- 172.21.15.114:0/4201253920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 msgr2=0x560413ca9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.013+0000 7f48cb400640 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413ca9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.013+0000 7f48cb400640 1 -- 172.21.15.114:0/4201253920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560413bd9cd0 con 0x5604138d91b0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.014+0000 7f48cb400640 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413ca90e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48b8044000 tx=0x7f48b804c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2312a7c72f3ffbcc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.014+0000 7f48c1600640 1 -- 172.21.15.114:0/4201253920 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b805e070 con 0x5604138d91b0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.014+0000 7f48c1600640 1 -- 172.21.15.114:0/4201253920 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b8005540 con 0x5604138d91b0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.014+0000 7f48c1600640 1 -- 172.21.15.114:0/4201253920 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b805e070 con 0x5604138d91b0 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.017+0000 7f48d9a31740 1 -- 172.21.15.114:0/4201253920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 msgr2=0x560413ca90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.017+0000 7f48d9a31740 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413ca90e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48b8044000 tx=0x7f48b804c000 comp rx=0 tx=0).stop 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.018+0000 7f48d9a31740 1 -- 172.21.15.114:0/4201253920 shutdown_connections 2024-09-21T07:03:12.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.018+0000 7f48d9a31740 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413ca9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.018+0000 7f48d9a31740 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413ca90e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.018+0000 7f48d9a31740 1 --2- 172.21.15.114:0/4201253920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413ca8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.018+0000 7f48d9a31740 1 -- 172.21.15.114:0/4201253920 >> 172.21.15.114:0/4201253920 conn(0x5604138f6460 msgr2=0x560413ca6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.019+0000 7f48d9a31740 1 -- 172.21.15.114:0/4201253920 shutdown_connections 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.019+0000 7f48d9a31740 1 -- 172.21.15.114:0/4201253920 wait complete. 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.021+0000 7f48d9a31740 1 -- start start 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.021+0000 7f48d9a31740 1 Processor -- start 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.022+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 0x560413cb47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.023+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cb4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.023+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 0x560413cb47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.023+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 0x560413cb47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48062/0 (socket says 172.21.15.114:48062) 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.023+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560413902070 0x560413cc0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.023+0000 7f48cbe00640 1 -- 172.21.15.114:0/3340430359 learned_addr learned my addr 172.21.15.114:0/3340430359 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:12.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.025+0000 7f48d0e00640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560413902070 0x560413cc0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.028+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560413a70700 con 0x5604138e1390 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.028+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560413baa580 con 0x5604138d91b0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.028+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560413cb7320 con 0x560413902070 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.028+0000 7f48d9a31740 10 client.0 _ll_register_callbacks cb 0x560413bba470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.029+0000 7f48cb400640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cb4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.029+0000 7f48cbe00640 1 -- 172.21.15.114:0/3340430359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560413902070 msgr2=0x560413cc0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.029+0000 7f48cbe00640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560413902070 0x560413cc0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.029+0000 7f48cbe00640 1 -- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 msgr2=0x560413cb4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.029+0000 7f48cbe00640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cb4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.029+0000 7f48cbe00640 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56041396d650 con 0x5604138d91b0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.032+0000 7f48cbe00640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 0x560413cb47a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ad520 tx=0x7f48bc0a9ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6908c20ed7d737ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.032+0000 7f48b6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.032+0000 7f48b6000640 1 -- 172.21.15.114:0/3340430359 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0ec060 con 0x5604138d91b0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.032+0000 7f48b6000640 1 -- 172.21.15.114:0/3340430359 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9578+0+0 (secure 0 0 0) 0x7f48bc0e70d0 con 0x5604138d91b0 2024-09-21T07:03:12.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.032+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560413bacb90 con 0x5604138d91b0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48b6000640 1 -- 172.21.15.114:0/3340430359 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f48bc0adc90 con 0x5604138d91b0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 10 client.27854 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560413bc7950 con 0x5604138d91b0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48b6000640 1 -- 172.21.15.114:0/3340430359 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560413bacb90 con 0x5604138d91b0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48b6000640 1 -- 172.21.15.114:0/3340430359 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f48bc0ac260 con 0x5604138d91b0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 10 client.27854 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 10 client.27854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 20 client.27854 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5604138c09f0 0x560413cc6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 4 client.27854 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.033+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560413c21f00 con 0x5604138c09f0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.034+0000 7f48cb400640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5604138c09f0 0x560413cc6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.036+0000 7f48b6000640 10 client.27854 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.037+0000 7f48cb400640 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5604138c09f0 0x560413cc6750 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e02ea660aca4382 server_cookie=9309ada7f4ab8cee in_seq=0 out_seq=0 2024-09-21T07:03:12.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.039+0000 7f48b6000640 1 -- 172.21.15.114:0/3340430359 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560413c21f00 con 0x5604138c09f0 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.039+0000 7f48b6000640 10 client.27854 handle_command_reply: tid=0 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.039+0000 7f48d9a31740 1 client.27854 shutdown 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 msgr2=0x560413cb47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 0x560413cb47a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ad520 tx=0x7f48bc0a9ab0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 shutdown_connections 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5604138c09f0 0x560413cc6750 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560413902070 0x560413cc0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cb4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 --2- 172.21.15.114:0/3340430359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5604138d91b0 0x560413cb47a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.040+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 >> 172.21.15.114:0/3340430359 conn(0x5604138f6460 msgr2=0x560413cabd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.041+0000 7f48b6000640 0 client.27854 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.041+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 shutdown_connections 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.041+0000 7f48d9a31740 1 -- 172.21.15.114:0/3340430359 wait complete. 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.042+0000 7f48d9a31740 20 client.27854 trim_cache size 0 max 16384 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.069+0000 7f48d9a31740 1 Processor -- start 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 -- start start 2024-09-21T07:03:12.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138d91b0 0x560413cc0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 0x560413cadcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413cae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1aedb0 con 0x5604138d91b0 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b0920 con 0x560413902070 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48d9a31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1af480 con 0x5604138e1390 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.070+0000 7f48cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 0x560413cadcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.071+0000 7f48cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 0x560413cadcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47490/0 (socket says 172.21.15.114:47490) 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.071+0000 7f48cb400640 1 -- 172.21.15.114:0/2895206711 learned_addr learned my addr 172.21.15.114:0/2895206711 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.071+0000 7f48cbe00640 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138d91b0 0x560413cc0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.071+0000 7f48d0e00640 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413cae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cb400640 1 -- 172.21.15.114:0/2895206711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 msgr2=0x560413cae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cb400640 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413cae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cb400640 1 -- 172.21.15.114:0/2895206711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138d91b0 msgr2=0x560413cc0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cb400640 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138d91b0 0x560413cc0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cb400640 1 -- 172.21.15.114:0/2895206711 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560413cb5860 con 0x5604138e1390 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cbe00640 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138d91b0 0x560413cc0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48cb400640 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 0x560413cadcb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f48b801a910 tx=0x7f48b8005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e755b3f839e1ad1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48c0c00640 1 -- 172.21.15.114:0/2895206711 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8068070 con 0x5604138e1390 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48c0c00640 1 -- 172.21.15.114:0/2895206711 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b80409c0 con 0x5604138e1390 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.072+0000 7f48c0c00640 1 -- 172.21.15.114:0/2895206711 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b805e040 con 0x5604138e1390 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 -- 172.21.15.114:0/2895206711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 msgr2=0x560413cadcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 0x560413cadcb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f48b801a910 tx=0x7f48b8005750 comp rx=0 tx=0).stop 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 -- 172.21.15.114:0/2895206711 shutdown_connections 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413cae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138e1390 0x560413cadcb0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 --2- 172.21.15.114:0/2895206711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138d91b0 0x560413cc0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 -- 172.21.15.114:0/2895206711 >> 172.21.15.114:0/2895206711 conn(0x5604138f6460 msgr2=0x560413cb2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 -- 172.21.15.114:0/2895206711 shutdown_connections 2024-09-21T07:03:12.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.074+0000 7f48d9a31740 1 -- 172.21.15.114:0/2895206711 wait complete. 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.075+0000 7f48d9a31740 1 -- start start 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.075+0000 7f48d9a31740 1 Processor -- start 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.077+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413cbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.077+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cbcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.077+0000 7f48d9a31740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413d2a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.079+0000 7f48cb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cbcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.079+0000 7f48cb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cbcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40442/0 (socket says 172.21.15.114:40442) 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.079+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413cbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.079+0000 7f48cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413cbc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47498/0 (socket says 172.21.15.114:47498) 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413d2a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48d0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413d2a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48074/0 (socket says 172.21.15.114:48074) 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48d9a31740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1b0250 con 0x5604138e1390 2024-09-21T07:03:12.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48d9a31740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1afb50 con 0x560413902070 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48d9a31740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1acdb0 con 0x5604138d91b0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48cb400640 1 -- 172.21.15.114:0/231719800 learned_addr learned my addr 172.21.15.114:0/231719800 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.080+0000 7f48d9a31740 10 client.0 _ll_register_callbacks cb 0x7f48bc0a94e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48cbe00640 1 -- 172.21.15.114:0/231719800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 msgr2=0x560413d2a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48cbe00640 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413d2a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48cbe00640 1 -- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 msgr2=0x560413cbcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48cbe00640 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cbcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48cbe00640 1 -- 172.21.15.114:0/231719800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560413cbd150 con 0x5604138d91b0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48cbe00640 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413cbc570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ac600 tx=0x7f48bc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc05fa1928b8ca0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.081+0000 7f48b6000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.082+0000 7f48b6000640 1 -- 172.21.15.114:0/231719800 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0acd00 con 0x5604138d91b0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.082+0000 7f48b6000640 1 -- 172.21.15.114:0/231719800 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9578+0+0 (secure 0 0 0) 0x7f48bc0a8ee0 con 0x5604138d91b0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.082+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48b80409c0 con 0x5604138d91b0 2024-09-21T07:03:12.147 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.082+0000 7f48b6000640 1 -- 172.21.15.114:0/231719800 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f48bc009de0 con 0x5604138d91b0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.082+0000 7f48d9a31740 10 client.27805 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.082+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560413d42510 con 0x5604138d91b0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48b6000640 1 -- 172.21.15.114:0/231719800 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48b80409c0 con 0x5604138d91b0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48b6000640 1 -- 172.21.15.114:0/231719800 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f48bc0c8cb0 con 0x5604138d91b0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48d9a31740 10 client.27805 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48d9a31740 10 client.27805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48d9a31740 20 client.27805 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48d9a31740 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x560413895da0 0x560413d44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48d9a31740 4 client.27805 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.083+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560413c1d9b0 con 0x560413895da0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.084+0000 7f48cb400640 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x560413895da0 0x560413d44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.086+0000 7f48cb400640 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x560413895da0 0x560413d44c00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ad29fd708ad9650 server_cookie=48963103568004eb in_seq=0 out_seq=0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.086+0000 7f48b6000640 10 client.27805 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.086+0000 7f48b6000640 1 -- 172.21.15.114:0/231719800 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560413c1d9b0 con 0x560413895da0 2024-09-21T07:03:12.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48b6000640 10 client.27805 handle_command_reply: tid=0 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 client.27805 shutdown 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 msgr2=0x560413cbc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413cbc570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ac600 tx=0x7f48bc000bc0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 shutdown_connections 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x560413895da0 0x560413d44c00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560413902070 0x560413d2a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48b6000640 0 client.27805 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5604138e1390 0x560413cbcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 --2- 172.21.15.114:0/231719800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5604138d91b0 0x560413cbc570 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0ac600 tx=0x7f48bc000bc0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 >> 172.21.15.114:0/231719800 conn(0x5604138f6460 msgr2=0x560413cadcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.087+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 shutdown_connections 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.088+0000 7f48d9a31740 1 -- 172.21.15.114:0/231719800 wait complete. 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.088+0000 7f48d9a31740 20 client.27805 trim_cache size 0 max 16384 2024-09-21T07:03:12.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48b00b6700 msgr2=0x7f48b00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48b00b6700 0x7f48b00b8c10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f48b801a860 tx=0x7f48b8005ee0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 msgr2=0x7f48cc113ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc113ab0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00d950 tx=0x7f48bc00de10 comp rx=0 tx=0).stop 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 shutdown_connections 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48b00b6700 0x7f48b00b8c10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48cc114440 0x7f48cc1102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48cc077380 0x7f48cc10fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 --2- 172.21.15.114:0/1599720610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48cc075dd0 0x7f48cc113ab0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.089+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 >> 172.21.15.114:0/1599720610 conn(0x7f48cc0ff5e0 msgr2=0x7f48cc1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.090+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 shutdown_connections 2024-09-21T07:03:12.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:12.090+0000 7f48d2e00640 1 -- 172.21.15.114:0/1599720610 wait complete. 2024-09-21T07:03:12.312 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:11 smithi193 ceph-mon[34601]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T07:03:12.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:11 smithi114 ceph-mon[30654]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T07:03:12.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:11 smithi183 ceph-mon[34219]: osdmap e94: 12 total, 12 up, 12 in 2024-09-21T07:03:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 515.0 s, 413.0 tps, lat 236.811 ms stddev 345.035 2024-09-21T07:03:13.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:13 smithi114 ceph-mon[30654]: pgmap v673: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-09-21T07:03:13.326 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:13 smithi193 ceph-mon[34601]: pgmap v673: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-09-21T07:03:13.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:13 smithi183 ceph-mon[34219]: pgmap v673: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 4.9 MiB/s wr, 530 op/s 2024-09-21T07:03:14.764 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:03:15.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:15 smithi114 ceph-mon[30654]: pgmap v674: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.7 MiB/s wr, 492 op/s 2024-09-21T07:03:15.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:15 smithi183 ceph-mon[34219]: pgmap v674: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.7 MiB/s wr, 492 op/s 2024-09-21T07:03:15.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:15 smithi193 ceph-mon[34601]: pgmap v674: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.7 MiB/s wr, 492 op/s 2024-09-21T07:03:15.596 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:16.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.589+0000 7f11d7400640 1 -- 172.21.15.114:0/684216857 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0031d20 con 0x7f11d8109c40 2024-09-21T07:03:16.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.591+0000 7f11dde00640 1 -- 172.21.15.114:0/684216857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d8109c40 msgr2=0x7f11d8110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.591+0000 7f11dde00640 1 --2- 172.21.15.114:0/684216857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d8109c40 0x7f11d8110470 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009ca0 tx=0x7f11d002a8e0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 -- 172.21.15.114:0/684216857 shutdown_connections 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 --2- 172.21.15.114:0/684216857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d8109c40 0x7f11d8110470 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 --2- 172.21.15.114:0/684216857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81092f0 0x7f11d8109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 --2- 172.21.15.114:0/684216857 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11d81039b0 0x7f11d8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 -- 172.21.15.114:0/684216857 >> 172.21.15.114:0/684216857 conn(0x7f11d80ff440 msgr2=0x7f11d8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 -- 172.21.15.114:0/684216857 shutdown_connections 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.592+0000 7f11dde00640 1 -- 172.21.15.114:0/684216857 wait complete. 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.594+0000 7f11dde00640 1 Processor -- start 2024-09-21T07:03:16.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.594+0000 7f11dde00640 1 -- start start 2024-09-21T07:03:16.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.595+0000 7f11dde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81039b0 0x7f11d8110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.595+0000 7f11dde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d81092f0 0x7f11d810c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.595+0000 7f11dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81039b0 0x7f11d8110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.595+0000 7f11dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81039b0 0x7f11d8110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49148/0 (socket says 172.21.15.114:49148) 2024-09-21T07:03:16.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.595+0000 7f11dca00640 1 -- 172.21.15.114:0/2472024367 learned_addr learned my addr 172.21.15.114:0/2472024367 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:16.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.595+0000 7f11d7e00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d81092f0 0x7f11d810c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.598+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11d8109c40 0x7f11d810cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.598+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8075d10 con 0x7f11d8109c40 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.598+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8075b90 con 0x7f11d81039b0 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.598+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11d8075e90 con 0x7f11d81092f0 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.598+0000 7f11dd400640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11d8109c40 0x7f11d810cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.599+0000 7f11d7e00640 1 -- 172.21.15.114:0/2472024367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81039b0 msgr2=0x7f11d8110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.599+0000 7f11d7e00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81039b0 0x7f11d8110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.599+0000 7f11d7e00640 1 -- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11d8109c40 msgr2=0x7f11d810cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.599+0000 7f11d7e00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11d8109c40 0x7f11d810cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.599+0000 7f11d7e00640 1 -- 172.21.15.114:0/2472024367 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d810d2f0 con 0x7f11d81092f0 2024-09-21T07:03:16.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.599+0000 7f11d7e00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d81092f0 0x7f11d810c530 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f11d806b990 tx=0x7f11c400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.600+0000 7f11d5600640 1 -- 172.21.15.114:0/2472024367 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c400cc40 con 0x7f11d81092f0 2024-09-21T07:03:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.600+0000 7f11d5600640 1 -- 172.21.15.114:0/2472024367 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c400cde0 con 0x7f11d81092f0 2024-09-21T07:03:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.600+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d81acc20 con 0x7f11d81092f0 2024-09-21T07:03:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.601+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81ad1b0 con 0x7f11d81092f0 2024-09-21T07:03:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.601+0000 7f11d5600640 1 -- 172.21.15.114:0/2472024367 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4018780 con 0x7f11d81092f0 2024-09-21T07:03:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.618+0000 7f11d5600640 1 -- 172.21.15.114:0/2472024367 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f11c4010a90 con 0x7f11d81092f0 2024-09-21T07:03:16.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.618+0000 7f11e61ee740 1 Processor -- start 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.618+0000 7f11e61ee740 1 -- start start 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b461070 0x55852b807f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b8084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b8089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11e61ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55852b77efb0 con 0x55852b440390 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11e61ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55852b76e3e0 con 0x55852b4381b0 2024-09-21T07:03:16.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11e61ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55852b7721f0 con 0x55852b461070 2024-09-21T07:03:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.620+0000 7f11dd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b461070 0x55852b807f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.621+0000 7f11dd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b461070 0x55852b807f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41026/0 (socket says 172.21.15.114:41026) 2024-09-21T07:03:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.621+0000 7f11dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b8084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.621+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b8089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.621+0000 7f11dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b8084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49156/0 (socket says 172.21.15.114:49156) 2024-09-21T07:03:16.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.621+0000 7f11dca00640 1 -- 172.21.15.114:0/427815569 learned_addr learned my addr 172.21.15.114:0/427815569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:16.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.623+0000 7f11d5600640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f11c00b6700 0x7f11c00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.623+0000 7f11d5600640 1 -- 172.21.15.114:0/2472024367 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9578+0+0 (secure 0 0 0) 0x7f11c40e7db0 con 0x7f11d81092f0 2024-09-21T07:03:16.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.624+0000 7f11dca00640 1 -- 172.21.15.114:0/427815569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b461070 msgr2=0x55852b807f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.624+0000 7f11dca00640 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b461070 0x55852b807f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.624+0000 7f11dca00640 1 -- 172.21.15.114:0/427815569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 msgr2=0x55852b8089f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.624+0000 7f11dca00640 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b8089f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.624+0000 7f11dca00640 1 -- 172.21.15.114:0/427815569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852b738cd0 con 0x55852b4381b0 2024-09-21T07:03:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.624+0000 7f11d7e00640 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b8089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:03:16.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.625+0000 7f11dca00640 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b8084b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00aca0 tx=0x7f11cc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f76c46f7eeecd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.625+0000 7f11dd400640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f11c00b6700 0x7f11c00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.625+0000 7f11c9c00640 1 -- 172.21.15.114:0/427815569 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc01d070 con 0x55852b4381b0 2024-09-21T07:03:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.625+0000 7f11c9c00640 1 -- 172.21.15.114:0/427815569 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11cc018af0 con 0x55852b4381b0 2024-09-21T07:03:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.625+0000 7f11c9c00640 1 -- 172.21.15.114:0/427815569 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc010c50 con 0x55852b4381b0 2024-09-21T07:03:16.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.627+0000 7f11e61ee740 1 -- 172.21.15.114:0/427815569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 msgr2=0x55852b8084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.627+0000 7f11e61ee740 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b8084b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00aca0 tx=0x7f11cc00f790 comp rx=0 tx=0).stop 2024-09-21T07:03:16.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.628+0000 7f11dd400640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f11c00b6700 0x7f11c00b8c10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f11d8110980 tx=0x7f11d00095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 -- 172.21.15.114:0/427815569 shutdown_connections 2024-09-21T07:03:16.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b8089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b8084b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 --2- 172.21.15.114:0/427815569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b461070 0x55852b807f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 -- 172.21.15.114:0/427815569 >> 172.21.15.114:0/427815569 conn(0x55852b455460 msgr2=0x55852b805e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 -- 172.21.15.114:0/427815569 shutdown_connections 2024-09-21T07:03:16.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.629+0000 7f11e61ee740 1 -- 172.21.15.114:0/427815569 wait complete. 2024-09-21T07:03:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.630+0000 7f11e61ee740 1 -- start start 2024-09-21T07:03:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.630+0000 7f11e61ee740 1 Processor -- start 2024-09-21T07:03:16.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.631+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b824710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.631+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.631+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b461070 0x55852b8309e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.631+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.631+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41032/0 (socket says 172.21.15.114:41032) 2024-09-21T07:03:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b824710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11dca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b824710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49170/0 (socket says 172.21.15.114:49170) 2024-09-21T07:03:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11dd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b461070 0x55852b8309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11dd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b461070 0x55852b8309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34456/0 (socket says 172.21.15.114:34456) 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11d7e00640 1 -- 172.21.15.114:0/2197176696 learned_addr learned my addr 172.21.15.114:0/2197176696 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55852b5cf700 con 0x55852b461070 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55852b709580 con 0x55852b4381b0 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.633+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55852b8276b0 con 0x55852b440390 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.634+0000 7f11e61ee740 10 client.0 _ll_register_callbacks cb 0x55852b719470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.634+0000 7f11dca00640 1 -- 172.21.15.114:0/2197176696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 msgr2=0x55852b824c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.634+0000 7f11dca00640 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.634+0000 7f11dca00640 1 -- 172.21.15.114:0/2197176696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b461070 msgr2=0x55852b8309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.634+0000 7f11dca00640 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b461070 0x55852b8309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.634+0000 7f11dca00640 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852b4cc650 con 0x55852b4381b0 2024-09-21T07:03:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.635+0000 7f11dca00640 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b824710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc0058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750f120e0c9a225d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.635+0000 7f11be000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:16.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.635+0000 7f11be000640 1 -- 172.21.15.114:0/2197176696 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc01d040 con 0x55852b4381b0 2024-09-21T07:03:16.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.635+0000 7f11be000640 1 -- 172.21.15.114:0/2197176696 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9578+0+0 (secure 0 0 0) 0x7f11cc005230 con 0x55852b4381b0 2024-09-21T07:03:16.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.636+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55852b70bb90 con 0x55852b4381b0 2024-09-21T07:03:16.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.636+0000 7f11be000640 1 -- 172.21.15.114:0/2197176696 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f11cc005430 con 0x55852b4381b0 2024-09-21T07:03:16.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.636+0000 7f11e61ee740 10 client.27872 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:16.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.636+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55852b726950 con 0x55852b4381b0 2024-09-21T07:03:16.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.637+0000 7f11be000640 1 -- 172.21.15.114:0/2197176696 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55852b70bb90 con 0x55852b4381b0 2024-09-21T07:03:16.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.640+0000 7f11be000640 1 -- 172.21.15.114:0/2197176696 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f11cc023470 con 0x55852b4381b0 2024-09-21T07:03:16.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.640+0000 7f11e61ee740 10 client.27872 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:16.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.640+0000 7f11e61ee740 10 client.27872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:16.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.640+0000 7f11e61ee740 20 client.27872 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:16.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.640+0000 7f11e61ee740 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b41f9f0 0x55852b836ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.641+0000 7f11d7e00640 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b41f9f0 0x55852b836ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.641+0000 7f11e61ee740 4 client.27872 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:16.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.641+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852b780f00 con 0x55852b41f9f0 2024-09-21T07:03:16.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.643+0000 7f11be000640 10 client.27872 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:16.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.643+0000 7f11d7e00640 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b41f9f0 0x55852b836ac0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad73fed6fef26e42 server_cookie=c89af66548e44674 in_seq=0 out_seq=0 2024-09-21T07:03:16.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.644+0000 7f11be000640 1 -- 172.21.15.114:0/2197176696 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55852b780f00 con 0x55852b41f9f0 2024-09-21T07:03:16.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.644+0000 7f11be000640 10 client.27872 handle_command_reply: tid=0 2024-09-21T07:03:16.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.645+0000 7f11e61ee740 1 client.27872 shutdown 2024-09-21T07:03:16.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.645+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 msgr2=0x55852b824710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.645+0000 7f11e61ee740 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b824710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc0058c0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 shutdown_connections 2024-09-21T07:03:16.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b461070 0x55852b8309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b41f9f0 0x55852b836ac0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 --2- 172.21.15.114:0/2197176696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b4381b0 0x55852b824710 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 >> 172.21.15.114:0/2197176696 conn(0x55852b455460 msgr2=0x55852b809960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11be000640 0 client.27872 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 shutdown_connections 2024-09-21T07:03:16.649 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 1 -- 172.21.15.114:0/2197176696 wait complete. 2024-09-21T07:03:16.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.647+0000 7f11e61ee740 20 client.27872 trim_cache size 0 max 16384 2024-09-21T07:03:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.677+0000 7f11e61ee740 1 Processor -- start 2024-09-21T07:03:16.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.677+0000 7f11e61ee740 1 -- start start 2024-09-21T07:03:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.682+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b4381b0 0x55852b8244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b805760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11e61ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81aec60 con 0x55852b4381b0 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11e61ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81b07d0 con 0x55852b461070 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11e61ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81af330 con 0x55852b440390 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41040/0 (socket says 172.21.15.114:41040) 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11d7e00640 1 -- 172.21.15.114:0/24183498 learned_addr learned my addr 172.21.15.114:0/24183498 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:16.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.683+0000 7f11dd400640 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b805760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.684+0000 7f11dd400640 1 -- 172.21.15.114:0/24183498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 msgr2=0x55852b824a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.684+0000 7f11dd400640 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.684+0000 7f11dd400640 1 -- 172.21.15.114:0/24183498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b4381b0 msgr2=0x55852b8244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.684+0000 7f11dd400640 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b4381b0 0x55852b8244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.684+0000 7f11dd400640 1 -- 172.21.15.114:0/24183498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852b8259d0 con 0x55852b461070 2024-09-21T07:03:16.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.685+0000 7f11dd400640 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b805760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005cc0 tx=0x7f11d003e030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eb99f525e31b3b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.685+0000 7f11c9200640 1 -- 172.21.15.114:0/24183498 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0030d30 con 0x55852b461070 2024-09-21T07:03:16.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.686+0000 7f11c9200640 1 -- 172.21.15.114:0/24183498 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0036040 con 0x55852b461070 2024-09-21T07:03:16.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.686+0000 7f11c9200640 1 -- 172.21.15.114:0/24183498 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0004710 con 0x55852b461070 2024-09-21T07:03:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.687+0000 7f11e61ee740 1 -- 172.21.15.114:0/24183498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 msgr2=0x55852b805760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.687+0000 7f11e61ee740 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b805760 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f11d0005cc0 tx=0x7f11d003e030 comp rx=0 tx=0).stop 2024-09-21T07:03:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 -- 172.21.15.114:0/24183498 shutdown_connections 2024-09-21T07:03:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b805760 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b440390 0x55852b824a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 --2- 172.21.15.114:0/24183498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b4381b0 0x55852b8244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 -- 172.21.15.114:0/24183498 >> 172.21.15.114:0/24183498 conn(0x55852b455460 msgr2=0x55852b822f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 -- 172.21.15.114:0/24183498 shutdown_connections 2024-09-21T07:03:16.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.689+0000 7f11e61ee740 1 -- 172.21.15.114:0/24183498 wait complete. 2024-09-21T07:03:16.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.690+0000 7f11e61ee740 1 -- start start 2024-09-21T07:03:16.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.690+0000 7f11e61ee740 1 Processor -- start 2024-09-21T07:03:16.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.691+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b4381b0 0x55852b83a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b835680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11e61ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b835bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11e61ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81b0100 con 0x55852b440390 2024-09-21T07:03:16.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11e61ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11d81afa00 con 0x55852b461070 2024-09-21T07:03:16.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11e61ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11d810d830 con 0x55852b4381b0 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11e61ee740 10 client.0 _ll_register_callbacks cb 0x7f11cc00aaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b835680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.692+0000 7f11dca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b4381b0 0x55852b83a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11d7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b835680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34470/0 (socket says 172.21.15.114:34470) 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11d7e00640 1 -- 172.21.15.114:0/91241470 learned_addr learned my addr 172.21.15.114:0/91241470 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11dd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b835bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11dca00640 1 -- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 msgr2=0x55852b835bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11dca00640 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b835bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11dca00640 1 -- 172.21.15.114:0/91241470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 msgr2=0x55852b835680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:16.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11dca00640 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b835680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.693+0000 7f11dca00640 1 -- 172.21.15.114:0/91241470 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852b836300 con 0x55852b4381b0 2024-09-21T07:03:16.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.695+0000 7f11dca00640 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b4381b0 0x55852b83a1d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00abc0 tx=0x7f11cc024f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2cba9e526e737d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:16.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.695+0000 7f11be000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:16.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.696+0000 7f11be000640 1 -- 172.21.15.114:0/91241470 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc005da0 con 0x55852b4381b0 2024-09-21T07:03:16.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.696+0000 7f11be000640 1 -- 172.21.15.114:0/91241470 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9578+0+0 (secure 0 0 0) 0x7f11cc0104f0 con 0x55852b4381b0 2024-09-21T07:03:16.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.696+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d0036040 con 0x55852b4381b0 2024-09-21T07:03:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.711+0000 7f11be000640 1 -- 172.21.15.114:0/91241470 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f11cc023e60 con 0x55852b4381b0 2024-09-21T07:03:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.711+0000 7f11e61ee740 10 client.27823 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:16.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.711+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55852b8a11e0 con 0x55852b4381b0 2024-09-21T07:03:16.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.712+0000 7f11be000640 1 -- 172.21.15.114:0/91241470 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d0036040 con 0x55852b4381b0 2024-09-21T07:03:16.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.712+0000 7f11be000640 1 -- 172.21.15.114:0/91241470 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f11cc02c430 con 0x55852b4381b0 2024-09-21T07:03:16.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.712+0000 7f11e61ee740 10 client.27823 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:16.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.713+0000 7f11e61ee740 10 client.27823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:16.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.713+0000 7f11e61ee740 20 client.27823 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:16.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.713+0000 7f11e61ee740 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b3f4da0 0x55852b8a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:16.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.713+0000 7f11e61ee740 4 client.27823 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:03:16.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.713+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55852b77c9b0 con 0x55852b3f4da0 2024-09-21T07:03:16.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.714+0000 7f11d7e00640 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b3f4da0 0x55852b8a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:16.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.714+0000 7f11be000640 10 client.27823 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:16.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.714+0000 7f11d7e00640 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b3f4da0 0x55852b8a3990 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc242bdb84feaab server_cookie=e6442d13f2ca021d in_seq=0 out_seq=0 2024-09-21T07:03:16.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.717+0000 7f11be000640 1 -- 172.21.15.114:0/91241470 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55852b77c9b0 con 0x55852b3f4da0 2024-09-21T07:03:16.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.717+0000 7f11be000640 10 client.27823 handle_command_reply: tid=0 2024-09-21T07:03:16.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.717+0000 7f11e61ee740 1 client.27823 shutdown 2024-09-21T07:03:16.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b4381b0 msgr2=0x55852b83a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b4381b0 0x55852b83a1d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00abc0 tx=0x7f11cc024f40 comp rx=0 tx=0).stop 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 shutdown_connections 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55852b461070 0x55852b835bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55852b440390 0x55852b835680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55852b3f4da0 0x55852b8a3990 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 --2- 172.21.15.114:0/91241470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55852b4381b0 0x55852b83a1d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 >> 172.21.15.114:0/91241470 conn(0x55852b455460 msgr2=0x55852b805760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.719+0000 7f11be000640 0 client.27823 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:16.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.720+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 shutdown_connections 2024-09-21T07:03:16.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.720+0000 7f11e61ee740 1 -- 172.21.15.114:0/91241470 wait complete. 2024-09-21T07:03:16.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.720+0000 7f11e61ee740 20 client.27823 trim_cache size 0 max 16384 2024-09-21T07:03:16.724 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:03:16.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f11c00b6700 msgr2=0x7f11c00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f11c00b6700 0x7f11c00b8c10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f11d8110980 tx=0x7f11d00095c0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d81092f0 msgr2=0x7f11d810c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:16.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d81092f0 0x7f11d810c530 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f11d806b990 tx=0x7f11c400ee50 comp rx=0 tx=0).stop 2024-09-21T07:03:16.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 shutdown_connections 2024-09-21T07:03:16.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f11d8109c40 0x7f11d810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11d81092f0 0x7f11d810c530 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f11c00b6700 0x7f11c00b8c10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 --2- 172.21.15.114:0/2472024367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11d81039b0 0x7f11d8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:16.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.722+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 >> 172.21.15.114:0/2472024367 conn(0x7f11d80ff440 msgr2=0x7f11d8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:16.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.723+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 shutdown_connections 2024-09-21T07:03:16.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:16.723+0000 7f11dde00640 1 -- 172.21.15.114:0/2472024367 wait complete. 2024-09-21T07:03:16.965 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:16 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:16.965 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:16 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:16.965 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:16 smithi193 ceph-mon[34601]: pgmap v675: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-09-21T07:03:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:16 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:16 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:17.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:16 smithi114 ceph-mon[30654]: pgmap v675: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-09-21T07:03:17.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:16 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:17.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:16 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:17.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:16 smithi183 ceph-mon[34219]: pgmap v675: 129 pgs: 13 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 108 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-09-21T07:03:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 520.0 s, 484.0 tps, lat 208.967 ms stddev 293.665 2024-09-21T07:03:18.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:17 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:18.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:17 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:18.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:17 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:18.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:17 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:18.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:17 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:18.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:17 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:18.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:17 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:03:18.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:17 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:03:18.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:17 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:03:18.665 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:03:18.665 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:03:19.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:18 smithi114 ceph-mon[30654]: pgmap v676: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-09-21T07:03:19.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:18 smithi183 ceph-mon[34219]: pgmap v676: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-09-21T07:03:19.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:18 smithi193 ceph-mon[34601]: pgmap v676: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.5 MiB/s wr, 462 op/s 2024-09-21T07:03:19.503 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:20.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.394+0000 7f1301200640 1 -- 172.21.15.114:0/1267808804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc103b90 msgr2=0x7f12fc103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.394+0000 7f1301200640 1 --2- 172.21.15.114:0/1267808804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc103b90 0x7f12fc103fa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009900 tx=0x7f12ec02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 -- 172.21.15.114:0/1267808804 shutdown_connections 2024-09-21T07:03:20.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 --2- 172.21.15.114:0/1267808804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc109e20 0x7f12fc110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 --2- 172.21.15.114:0/1267808804 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc1094d0 0x7f12fc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 --2- 172.21.15.114:0/1267808804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc103b90 0x7f12fc103fa0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 -- 172.21.15.114:0/1267808804 >> 172.21.15.114:0/1267808804 conn(0x7f12fc0ff5e0 msgr2=0x7f12fc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 -- 172.21.15.114:0/1267808804 shutdown_connections 2024-09-21T07:03:20.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.395+0000 7f1301200640 1 -- 172.21.15.114:0/1267808804 wait complete. 2024-09-21T07:03:20.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.397+0000 7f1301200640 1 Processor -- start 2024-09-21T07:03:20.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.398+0000 7f1301200640 1 -- start start 2024-09-21T07:03:20.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.399+0000 7f1301200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 0x7f12fc1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.399+0000 7f1301200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc1094d0 0x7f12fc1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.399+0000 7f1301200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc109e20 0x7f12fc1ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.399+0000 7f1301200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc113ae0 con 0x7f12fc103b90 2024-09-21T07:03:20.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.399+0000 7f1301200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc113960 con 0x7f12fc109e20 2024-09-21T07:03:20.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.399+0000 7f1301200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc113c60 con 0x7f12fc1094d0 2024-09-21T07:03:20.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.400+0000 7f12f9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc1094d0 0x7f12fc1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.400+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 0x7f12fc1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.400+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 0x7f12fc1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34480/0 (socket says 172.21.15.114:34480) 2024-09-21T07:03:20.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.400+0000 7f12fa800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc109e20 0x7f12fc1ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.402 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.400+0000 7f12f9e00640 1 -- 172.21.15.114:0/2683592689 learned_addr learned my addr 172.21.15.114:0/2683592689 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9400640 1 -- 172.21.15.114:0/2683592689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc109e20 msgr2=0x7f12fc1ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9400640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc109e20 0x7f12fc1ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9400640 1 -- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 msgr2=0x7f12fc1a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9400640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 0x7f12fc1a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9400640 1 -- 172.21.15.114:0/2683592689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc1acd60 con 0x7f12fc1094d0 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9e00640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 0x7f12fc1a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:20.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f9400640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc1094d0 0x7f12fc1a1ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12fc06bb30 tx=0x7f12e800ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.401+0000 7f12f2a00640 1 -- 172.21.15.114:0/2683592689 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800ccc0 con 0x7f12fc1094d0 2024-09-21T07:03:20.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.402+0000 7f12f2a00640 1 -- 172.21.15.114:0/2683592689 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e80035f0 con 0x7f12fc1094d0 2024-09-21T07:03:20.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.402+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc1acf30 con 0x7f12fc1094d0 2024-09-21T07:03:20.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.402+0000 7f12f2a00640 1 -- 172.21.15.114:0/2683592689 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800ce60 con 0x7f12fc1094d0 2024-09-21T07:03:20.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.402+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc1ad320 con 0x7f12fc1094d0 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.406+0000 7f12f2a00640 1 -- 172.21.15.114:0/2683592689 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f12e8004000 con 0x7f12fc1094d0 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.407+0000 7f1307ea1740 1 Processor -- start 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.407+0000 7f1307ea1740 1 -- start start 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48471d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48472250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a9670 0x558e48472790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f1307ea1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558e483e7e90 con 0x558e480a9670 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f1307ea1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558e483d72c0 con 0x558e480ca350 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f1307ea1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558e483db0d0 con 0x558e480a1490 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48472250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f12fa800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48471d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48472250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41082/0 (socket says 172.21.15.114:41082) 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f12f9e00640 1 -- 172.21.15.114:0/427208210 learned_addr learned my addr 172.21.15.114:0/427208210 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.408+0000 7f12f9400640 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a9670 0x558e48472790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f9e00640 1 -- 172.21.15.114:0/427208210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 msgr2=0x558e48471d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f9e00640 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48471d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f9e00640 1 -- 172.21.15.114:0/427208210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a9670 msgr2=0x558e48472790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f9e00640 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a9670 0x558e48472790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f9e00640 1 -- 172.21.15.114:0/427208210 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e48135930 con 0x558e480a1490 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f2a00640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12e00b6680 0x7f12e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f2a00640 1 -- 172.21.15.114:0/2683592689 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f12e80e7d40 con 0x7f12fc1094d0 2024-09-21T07:03:20.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12fa800640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12e00b6680 0x7f12e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.409+0000 7f12f9e00640 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48472250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f12ec006010 tx=0x7f12ec005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8856552596befef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.410+0000 7f12e7400640 1 -- 172.21.15.114:0/427208210 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec035040 con 0x558e480a1490 2024-09-21T07:03:20.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.410+0000 7f12e7400640 1 -- 172.21.15.114:0/427208210 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec031820 con 0x558e480a1490 2024-09-21T07:03:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.410+0000 7f12e7400640 1 -- 172.21.15.114:0/427208210 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004a00 con 0x558e480a1490 2024-09-21T07:03:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.412+0000 7f1307ea1740 1 -- 172.21.15.114:0/427208210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 msgr2=0x558e48472250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.412+0000 7f1307ea1740 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48472250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f12ec006010 tx=0x7f12ec005ed0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.413+0000 7f1307ea1740 1 -- 172.21.15.114:0/427208210 shutdown_connections 2024-09-21T07:03:20.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.413+0000 7f1307ea1740 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a9670 0x558e48472790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.413+0000 7f1307ea1740 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48472250 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.413+0000 7f12fa800640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12e00b6680 0x7f12e00b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005fd0 tx=0x7f12f4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.414+0000 7f1307ea1740 1 --2- 172.21.15.114:0/427208210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48471d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.414+0000 7f1307ea1740 1 -- 172.21.15.114:0/427208210 >> 172.21.15.114:0/427208210 conn(0x558e480be740 msgr2=0x558e4846fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.414+0000 7f1307ea1740 1 -- 172.21.15.114:0/427208210 shutdown_connections 2024-09-21T07:03:20.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.414+0000 7f1307ea1740 1 -- 172.21.15.114:0/427208210 wait complete. 2024-09-21T07:03:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.415+0000 7f1307ea1740 1 -- start start 2024-09-21T07:03:20.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.415+0000 7f1307ea1740 1 Processor -- start 2024-09-21T07:03:20.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.417+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a1490 0x558e4847dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.417+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 0x558e4847e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.418+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48483310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.418+0000 7f1307ea1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558e48238890 con 0x558e480a1490 2024-09-21T07:03:20.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.418+0000 7f1307ea1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558e48372160 con 0x558e480ca350 2024-09-21T07:03:20.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.418+0000 7f1307ea1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558e48478220 con 0x558e480a9670 2024-09-21T07:03:20.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.419+0000 7f1307ea1740 10 client.0 _ll_register_callbacks cb 0x558e4838ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.420+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a1490 0x558e4847dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.420+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a1490 0x558e4847dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34502/0 (socket says 172.21.15.114:34502) 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.420+0000 7f12f9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 0x558e4847e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.420+0000 7f12f9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 0x558e4847e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41098/0 (socket says 172.21.15.114:41098) 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.420+0000 7f12f9400640 1 -- 172.21.15.114:0/3495634130 learned_addr learned my addr 172.21.15.114:0/3495634130 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.421+0000 7f12f9400640 1 -- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 msgr2=0x558e48483310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.421+0000 7f12f9400640 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48483310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.421+0000 7f12f9400640 1 -- 172.21.15.114:0/3495634130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a1490 msgr2=0x558e4847dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.421+0000 7f12f9400640 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a1490 0x558e4847dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.421+0000 7f12f9400640 1 -- 172.21.15.114:0/3495634130 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e48483980 con 0x558e480a9670 2024-09-21T07:03:20.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.427+0000 7f12f9400640 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 0x558e4847e2d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f12e80ad460 tx=0x7f12e80039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972c672aeba33a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.427+0000 7f12dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.427+0000 7f12dbe00640 1 -- 172.21.15.114:0/3495634130 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e80a7e60 con 0x558e480a9670 2024-09-21T07:03:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.427+0000 7f12dbe00640 1 -- 172.21.15.114:0/3495634130 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f12e80a90b0 con 0x558e480a9670 2024-09-21T07:03:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.427+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ec031820 con 0x558e480a9670 2024-09-21T07:03:20.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.428+0000 7f12dbe00640 1 -- 172.21.15.114:0/3495634130 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12e80ad530 con 0x558e480a9670 2024-09-21T07:03:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.428+0000 7f1307ea1740 10 client.27835 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.428+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558e4848c480 con 0x558e480a9670 2024-09-21T07:03:20.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f12dbe00640 1 -- 172.21.15.114:0/3495634130 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12e80e54e0 con 0x558e480a9670 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f12dbe00640 1 -- 172.21.15.114:0/3495634130 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f12e80a92b0 con 0x558e480a9670 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f1307ea1740 10 client.27835 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f1307ea1740 10 client.27835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f1307ea1740 20 client.27835 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e48088cd0 0x558e4848f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f1307ea1740 4 client.27835 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.429+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f12ec031820 con 0x558e48088cd0 2024-09-21T07:03:20.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.430+0000 7f12f9e00640 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e48088cd0 0x558e4848f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.431+0000 7f12f9e00640 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e48088cd0 0x558e4848f9a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f975d8796c8d6b server_cookie=b94f6ad6b4a3b26c in_seq=0 out_seq=0 2024-09-21T07:03:20.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.431+0000 7f12dbe00640 10 client.27835 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.433+0000 7f12dbe00640 1 -- 172.21.15.114:0/3495634130 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f12ec031820 con 0x558e48088cd0 2024-09-21T07:03:20.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.433+0000 7f12dbe00640 10 client.27835 handle_command_reply: tid=0 2024-09-21T07:03:20.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.433+0000 7f1307ea1740 1 client.27835 shutdown 2024-09-21T07:03:20.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.435+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 msgr2=0x558e4847e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.435+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 0x558e4847e2d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f12e80ad460 tx=0x7f12e80039d0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 shutdown_connections 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e48483310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e48088cd0 0x558e4848f9a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f12dbe00640 0 client.27835 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a9670 0x558e4847e2d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3495634130 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480a1490 0x558e4847dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 >> 172.21.15.114:0/3495634130 conn(0x558e480be740 msgr2=0x558e4846e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 shutdown_connections 2024-09-21T07:03:20.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.436+0000 7f1307ea1740 1 -- 172.21.15.114:0/3495634130 wait complete. 2024-09-21T07:03:20.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.437+0000 7f1307ea1740 20 client.27835 trim_cache size 0 max 16384 2024-09-21T07:03:20.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.481+0000 7f1307ea1740 1 Processor -- start 2024-09-21T07:03:20.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.481+0000 7f1307ea1740 1 -- start start 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e48088cd0 0x558e4847c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e4847c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e4846e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f1307ea1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc1aedb0 con 0x558e48088cd0 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f1307ea1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc1b0920 con 0x558e480ca350 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f1307ea1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc1af480 con 0x558e480a1490 2024-09-21T07:03:20.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f12f9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e4847c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f12f9400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e4847c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41106/0 (socket says 172.21.15.114:41106) 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f12f9400640 1 -- 172.21.15.114:0/3737748626 learned_addr learned my addr 172.21.15.114:0/3737748626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e48088cd0 0x558e4847c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.482+0000 7f12fa800640 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e4846e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.483+0000 7f12f9e00640 1 -- 172.21.15.114:0/3737748626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 msgr2=0x558e4847c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.483+0000 7f12f9e00640 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e4847c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.483+0000 7f12f9e00640 1 -- 172.21.15.114:0/3737748626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 msgr2=0x558e4846e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.483+0000 7f12f9e00640 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e4846e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.483+0000 7f12f9e00640 1 -- 172.21.15.114:0/3737748626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e48478160 con 0x558e48088cd0 2024-09-21T07:03:20.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.484+0000 7f12f9e00640 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e48088cd0 0x558e4847c390 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x558e484843a0 tx=0x7f12ec06b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de417961c0197ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.484+0000 7f12e6a00640 1 -- 172.21.15.114:0/3737748626 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec035040 con 0x558e48088cd0 2024-09-21T07:03:20.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.484+0000 7f12e6a00640 1 -- 172.21.15.114:0/3737748626 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec0748e0 con 0x558e48088cd0 2024-09-21T07:03:20.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.484+0000 7f12e6a00640 1 -- 172.21.15.114:0/3737748626 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec05e7d0 con 0x558e48088cd0 2024-09-21T07:03:20.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.486+0000 7f1307ea1740 1 -- 172.21.15.114:0/3737748626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e48088cd0 msgr2=0x558e4847c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.486+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e48088cd0 0x558e4847c390 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x558e484843a0 tx=0x7f12ec06b040 comp rx=0 tx=0).stop 2024-09-21T07:03:20.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.486+0000 7f1307ea1740 1 -- 172.21.15.114:0/3737748626 shutdown_connections 2024-09-21T07:03:20.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.487+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e480ca350 0x558e4846e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.487+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e4847c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.487+0000 7f1307ea1740 1 --2- 172.21.15.114:0/3737748626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e48088cd0 0x558e4847c390 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.487+0000 7f1307ea1740 1 -- 172.21.15.114:0/3737748626 >> 172.21.15.114:0/3737748626 conn(0x558e480be740 msgr2=0x558e48478c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.488+0000 7f1307ea1740 1 -- 172.21.15.114:0/3737748626 shutdown_connections 2024-09-21T07:03:20.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.488+0000 7f1307ea1740 1 -- 172.21.15.114:0/3737748626 wait complete. 2024-09-21T07:03:20.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.489+0000 7f1307ea1740 1 -- start start 2024-09-21T07:03:20.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.489+0000 7f1307ea1740 1 Processor -- start 2024-09-21T07:03:20.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.490+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 0x558e484965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.491+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48496bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.491+0000 7f1307ea1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480ca350 0x558e48472ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.491+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 0x558e484965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.491+0000 7f12f9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 0x558e484965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49274/0 (socket says 172.21.15.114:49274) 2024-09-21T07:03:20.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.492+0000 7f1307ea1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc1b0250 con 0x558e480ca350 2024-09-21T07:03:20.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.492+0000 7f1307ea1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc1afb50 con 0x558e48088cd0 2024-09-21T07:03:20.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.492+0000 7f1307ea1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc1a0f10 con 0x558e480a1490 2024-09-21T07:03:20.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.492+0000 7f12f9e00640 1 -- 172.21.15.114:0/1620731016 learned_addr learned my addr 172.21.15.114:0/1620731016 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:20.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.492+0000 7f12fa800640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480ca350 0x558e48472ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.493+0000 7f1307ea1740 10 client.0 _ll_register_callbacks cb 0x7f12e80eb4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:20.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.492+0000 7f12f9400640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48496bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.494+0000 7f12f9e00640 1 -- 172.21.15.114:0/1620731016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 msgr2=0x558e48496bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.494+0000 7f12f9e00640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48496bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.494+0000 7f12f9e00640 1 -- 172.21.15.114:0/1620731016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480ca350 msgr2=0x558e48472ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:20.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.494+0000 7f12f9e00640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480ca350 0x558e48472ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.494+0000 7f12f9e00640 1 -- 172.21.15.114:0/1620731016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e48473610 con 0x558e48088cd0 2024-09-21T07:03:20.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.496+0000 7f12f9400640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48496bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:20.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.497+0000 7f12f9e00640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 0x558e484965d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12ec05e470 tx=0x7f12ec005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8363a6faf94307df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:20.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.497+0000 7f12dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:20.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.498+0000 7f12dbe00640 1 -- 172.21.15.114:0/1620731016 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec035030 con 0x558e48088cd0 2024-09-21T07:03:20.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.498+0000 7f12dbe00640 1 -- 172.21.15.114:0/1620731016 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f12ec074430 con 0x558e48088cd0 2024-09-21T07:03:20.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.498+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ec0748e0 con 0x558e48088cd0 2024-09-21T07:03:20.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.499+0000 7f12dbe00640 1 -- 172.21.15.114:0/1620731016 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f12ec074630 con 0x558e48088cd0 2024-09-21T07:03:20.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.499+0000 7f1307ea1740 10 client.27893 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:20.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.499+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558e48494c70 con 0x558e48088cd0 2024-09-21T07:03:20.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.501+0000 7f12dbe00640 1 -- 172.21.15.114:0/1620731016 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12ec0748e0 con 0x558e48088cd0 2024-09-21T07:03:20.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f12dbe00640 1 -- 172.21.15.114:0/1620731016 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f12ec0035f0 con 0x558e48088cd0 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f1307ea1740 10 client.27893 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f1307ea1740 10 client.27893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f1307ea1740 20 client.27893 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f1307ea1740 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e4805e080 0x558e484fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f1307ea1740 4 client.27893 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.502+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e483e9de0 con 0x558e4805e080 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.503+0000 7f12f9400640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e4805e080 0x558e484fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:20.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.504+0000 7f12dbe00640 10 client.27893 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:20.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.504+0000 7f12f9400640 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e4805e080 0x558e484fd450 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccbf04586e73ed97 server_cookie=7eaf318c4501024e in_seq=0 out_seq=0 2024-09-21T07:03:20.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f12dbe00640 1 -- 172.21.15.114:0/1620731016 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558e483e9de0 con 0x558e4805e080 2024-09-21T07:03:20.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f12dbe00640 10 client.27893 handle_command_reply: tid=0 2024-09-21T07:03:20.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 client.27893 shutdown 2024-09-21T07:03:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 msgr2=0x558e484965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 0x558e484965d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12ec05e470 tx=0x7f12ec005850 comp rx=0 tx=0).stop 2024-09-21T07:03:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 shutdown_connections 2024-09-21T07:03:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558e480ca350 0x558e48472ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558e4805e080 0x558e484fd450 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558e480a1490 0x558e48496bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 --2- 172.21.15.114:0/1620731016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e48088cd0 0x558e484965d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 >> 172.21.15.114:0/1620731016 conn(0x558e480be740 msgr2=0x558e4846e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.506+0000 7f12dbe00640 0 client.27893 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.507+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 shutdown_connections 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.507+0000 7f1307ea1740 1 -- 172.21.15.114:0/1620731016 wait complete. 2024-09-21T07:03:20.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.507+0000 7f1307ea1740 20 client.27893 trim_cache size 0 max 16384 2024-09-21T07:03:20.511 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:03:20.511 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551", 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.510+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12e00b6680 msgr2=0x7f12e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.510+0000 7f1301200640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12e00b6680 0x7f12e00b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005fd0 tx=0x7f12f4009500 comp rx=0 tx=0).stop 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.510+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc1094d0 msgr2=0x7f12fc1a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:20.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.510+0000 7f1301200640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc1094d0 0x7f12fc1a1ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f12fc06bb30 tx=0x7f12e800ed90 comp rx=0 tx=0).stop 2024-09-21T07:03:20.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.511+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 shutdown_connections 2024-09-21T07:03:20.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.511+0000 7f1301200640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12e00b6680 0x7f12e00b8b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.511+0000 7f1301200640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12fc109e20 0x7f12fc1ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.511+0000 7f1301200640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12fc1094d0 0x7f12fc1a1ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.511+0000 7f1301200640 1 --2- 172.21.15.114:0/2683592689 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12fc103b90 0x7f12fc1a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:20.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.511+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 >> 172.21.15.114:0/2683592689 conn(0x7f12fc0ff5e0 msgr2=0x7f12fc108ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:20.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.512+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 shutdown_connections 2024-09-21T07:03:20.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:20.512+0000 7f1301200640 1 -- 172.21.15.114:0/2683592689 wait complete. 2024-09-21T07:03:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 -- 172.21.15.114:0/3365902482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109410 msgr2=0x7fd9cc109820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 --2- 172.21.15.114:0/3365902482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109410 0x7fd9cc109820 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009860 tx=0x7fd9b402a340 comp rx=0 tx=0).stop 2024-09-21T07:03:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 -- 172.21.15.114:0/3365902482 shutdown_connections 2024-09-21T07:03:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 --2- 172.21.15.114:0/3365902482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9cc109d60 0x7fd9cc110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 --2- 172.21.15.114:0/3365902482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109410 0x7fd9cc109820 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 --2- 172.21.15.114:0/3365902482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc103ae0 0x7fd9cc103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 -- 172.21.15.114:0/3365902482 >> 172.21.15.114:0/3365902482 conn(0x7fd9cc0ff520 msgr2=0x7fd9cc101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.051+0000 7fd9d2200640 1 -- 172.21.15.114:0/3365902482 shutdown_connections 2024-09-21T07:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.052+0000 7fd9d2200640 1 -- 172.21.15.114:0/3365902482 wait complete. 2024-09-21T07:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 Processor -- start 2024-09-21T07:03:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 -- start start 2024-09-21T07:03:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9cc103ae0 0x7fd9cc10dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc109410 0x7fd9cc10e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109d60 0x7fd9cc1ac750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc077250 con 0x7fd9cc109d60 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0770d0 con 0x7fd9cc103ae0 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.053+0000 7fd9d2200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc0773d0 con 0x7fd9cc109410 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc109410 0x7fd9cc10e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc109410 0x7fd9cc10e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41144/0 (socket says 172.21.15.114:41144) 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9c2a00640 1 -- 172.21.15.114:0/642069433 learned_addr learned my addr 172.21.15.114:0/642069433 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109d60 0x7fd9cc1ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cb400640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9cc103ae0 0x7fd9cc10dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cbe00640 1 -- 172.21.15.114:0/642069433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc109410 msgr2=0x7fd9cc10e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc109410 0x7fd9cc10e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cbe00640 1 -- 172.21.15.114:0/642069433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9cc103ae0 msgr2=0x7fd9cc10dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9cc103ae0 0x7fd9cc10dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.054+0000 7fd9cbe00640 1 -- 172.21.15.114:0/642069433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc1acc90 con 0x7fd9cc109d60 2024-09-21T07:03:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.055+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109d60 0x7fd9cc1ac750 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.055+0000 7fd9c8c00640 1 -- 172.21.15.114:0/642069433 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4004220 con 0x7fd9cc109d60 2024-09-21T07:03:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.055+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc1ace60 con 0x7fd9cc109d60 2024-09-21T07:03:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.055+0000 7fd9c8c00640 1 -- 172.21.15.114:0/642069433 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c40043c0 con 0x7fd9cc109d60 2024-09-21T07:03:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.055+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc1ad2d0 con 0x7fd9cc109d60 2024-09-21T07:03:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.055+0000 7fd9c8c00640 1 -- 172.21.15.114:0/642069433 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4005650 con 0x7fd9cc109d60 2024-09-21T07:03:21.060 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.059+0000 7fd9d9247740 1 Processor -- start 2024-09-21T07:03:21.062 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.061+0000 7fd9c8c00640 1 -- 172.21.15.114:0/642069433 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd9c4004a00 con 0x7fd9cc109d60 2024-09-21T07:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.061+0000 7fd9d9247740 1 -- start start 2024-09-21T07:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b376c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b377170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b3776b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9d9247740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a55b2ec6b0 con 0x55a55afa92f0 2024-09-21T07:03:21.064 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9d9247740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a55b2dbae0 con 0x55a55afb14d0 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9d9247740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a55b2df8f0 con 0x55a55afd21b0 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b376c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9c2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b376c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41158/0 (socket says 172.21.15.114:41158) 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9c2a00640 1 -- 172.21.15.114:0/2166878257 learned_addr learned my addr 172.21.15.114:0/2166878257 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b377170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b3776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.062+0000 7fd9cb400640 1 -- 172.21.15.114:0/2166878257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 msgr2=0x55a55b376c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b376c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9cb400640 1 -- 172.21.15.114:0/2166878257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 msgr2=0x55a55b3776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b3776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9cb400640 1 -- 172.21.15.114:0/2166878257 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55b2a8780 con 0x55a55afa92f0 2024-09-21T07:03:21.065 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b376c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b377170 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005fd0 tx=0x7fd9bc003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bcc3baaebbc5e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9c8c00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9b00b66b0 0x7fd9b00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9c8c00640 1 -- 172.21.15.114:0/642069433 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fd9c40e7990 con 0x7fd9cc109d60 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9b00b66b0 0x7fd9b00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9bbe00640 1 -- 172.21.15.114:0/2166878257 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc004b20 con 0x55a55afa92f0 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9bbe00640 1 -- 172.21.15.114:0/2166878257 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc004cc0 con 0x55a55afa92f0 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.063+0000 7fd9bbe00640 1 -- 172.21.15.114:0/2166878257 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc0126b0 con 0x55a55afa92f0 2024-09-21T07:03:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 -- 172.21.15.114:0/2166878257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 msgr2=0x55a55b377170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b377170 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005fd0 tx=0x7fd9bc003e00 comp rx=0 tx=0).stop 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9b00b66b0 0x7fd9b00b8bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40095f0 tx=0x7fd9b4030340 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 -- 172.21.15.114:0/2166878257 shutdown_connections 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b3776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b377170 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2166878257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b376c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 -- 172.21.15.114:0/2166878257 >> 172.21.15.114:0/2166878257 conn(0x55a55afc6300 msgr2=0x55a55b374b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 -- 172.21.15.114:0/2166878257 shutdown_connections 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.064+0000 7fd9d9247740 1 -- 172.21.15.114:0/2166878257 wait complete. 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.065+0000 7fd9d9247740 1 -- start start 2024-09-21T07:03:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.065+0000 7fd9d9247740 1 Processor -- start 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b382210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b382750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b382c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a55b213ab0 con 0x55a55afa92f0 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a55b282570 con 0x55a55afb14d0 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a55b3853b0 con 0x55a55afd21b0 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9d9247740 10 client.0 _ll_register_callbacks cb 0x55a55b291100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b382210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b382210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34570/0 (socket says 172.21.15.114:34570) 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 -- 172.21.15.114:0/4255977140 learned_addr learned my addr 172.21.15.114:0/4255977140 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b382750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b382c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 -- 172.21.15.114:0/4255977140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 msgr2=0x55a55b382c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b382c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 -- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 msgr2=0x55a55b382750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b382750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.066+0000 7fd9cb400640 1 -- 172.21.15.114:0/4255977140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55b03d790 con 0x55a55afa92f0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9cb400640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b382210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005940 tx=0x7fd9bc005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a552e78fb67814c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9b8c00640 1 -- 172.21.15.114:0/4255977140 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc005070 con 0x55a55afa92f0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9b8c00640 1 -- 172.21.15.114:0/4255977140 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fd9bc005210 con 0x55a55afa92f0 2024-09-21T07:03:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9bc004cc0 con 0x55a55afa92f0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9b8c00640 1 -- 172.21.15.114:0/4255977140 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc005430 con 0x55a55afa92f0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9d9247740 10 client.17472 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a55b0b6620 con 0x55a55afa92f0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9b8c00640 1 -- 172.21.15.114:0/4255977140 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc004cc0 con 0x55a55afa92f0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.067+0000 7fd9b8c00640 1 -- 172.21.15.114:0/4255977140 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7fd9bc00f040 con 0x55a55afa92f0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9d9247740 10 client.17472 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9d9247740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9d9247740 20 client.17472 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9d9247740 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af90d80 0x55a55b394250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9d9247740 4 client.17472 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a55b2ee600 con 0x55a55af90d80 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.068+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af90d80 0x55a55b394250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.069+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af90d80 0x55a55b394250 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a605e4cd81cc0364 server_cookie=84bf44e9bf8fbbb4 in_seq=0 out_seq=0 2024-09-21T07:03:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.069+0000 7fd9b8c00640 10 client.17472 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9b8c00640 1 -- 172.21.15.114:0/4255977140 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a55b2ee600 con 0x55a55af90d80 2024-09-21T07:03:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9b8c00640 10 client.17472 handle_command_reply: tid=0 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 client.17472 shutdown 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 msgr2=0x55a55b382210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b382210 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005940 tx=0x7fd9bc005c60 comp rx=0 tx=0).stop 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 shutdown_connections 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af90d80 0x55a55b394250 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b382c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afb14d0 0x55a55b382750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 --2- 172.21.15.114:0/4255977140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afa92f0 0x55a55b382210 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9b8c00640 0 client.17472 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 >> 172.21.15.114:0/4255977140 conn(0x55a55afc6300 msgr2=0x55a55b377c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 shutdown_connections 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.070+0000 7fd9d9247740 1 -- 172.21.15.114:0/4255977140 wait complete. 2024-09-21T07:03:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.071+0000 7fd9d9247740 20 client.17472 trim_cache size 0 max 16384 2024-09-21T07:03:21.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.097+0000 7fd9d9247740 1 Processor -- start 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 -- start start 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afa92f0 0x55a55b378370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 0x55a55b37f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b37f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b7550 con 0x55a55afb14d0 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b90c0 con 0x55a55afa92f0 2024-09-21T07:03:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.098+0000 7fd9d9247740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b7c20 con 0x55a55afd21b0 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.099+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b37f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.099+0000 7fd9cbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b37f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41166/0 (socket says 172.21.15.114:41166) 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.099+0000 7fd9cbe00640 1 -- 172.21.15.114:0/3940973478 learned_addr learned my addr 172.21.15.114:0/3940973478 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.100+0000 7fd9cbe00640 1 -- 172.21.15.114:0/3940973478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afa92f0 msgr2=0x55a55b378370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.100+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afa92f0 0x55a55b378370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.100+0000 7fd9cbe00640 1 -- 172.21.15.114:0/3940973478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 msgr2=0x55a55b37f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.100+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 0x55a55b37f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.100+0000 7fd9cbe00640 1 -- 172.21.15.114:0/3940973478 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55b3946b0 con 0x55a55afd21b0 2024-09-21T07:03:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.100+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b37f920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40e4d50 tx=0x7fd9c40acc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd7c0b7c5fbf6ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.101+0000 7fd9bb400640 1 -- 172.21.15.114:0/3940973478 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c40d4020 con 0x55a55afd21b0 2024-09-21T07:03:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.101+0000 7fd9bb400640 1 -- 172.21.15.114:0/3940973478 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c40b1ed0 con 0x55a55afd21b0 2024-09-21T07:03:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.101+0000 7fd9bb400640 1 -- 172.21.15.114:0/3940973478 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c401c070 con 0x55a55afd21b0 2024-09-21T07:03:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 -- 172.21.15.114:0/3940973478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 msgr2=0x55a55b37f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b37f920 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40e4d50 tx=0x7fd9c40acc20 comp rx=0 tx=0).stop 2024-09-21T07:03:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 -- 172.21.15.114:0/3940973478 shutdown_connections 2024-09-21T07:03:21.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afd21b0 0x55a55b37f920 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 0x55a55b37f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 --2- 172.21.15.114:0/3940973478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afa92f0 0x55a55b378370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.102+0000 7fd9d9247740 1 -- 172.21.15.114:0/3940973478 >> 172.21.15.114:0/3940973478 conn(0x55a55afc6300 msgr2=0x55a55b377c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.103+0000 7fd9d9247740 1 -- 172.21.15.114:0/3940973478 shutdown_connections 2024-09-21T07:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.103+0000 7fd9d9247740 1 -- 172.21.15.114:0/3940973478 wait complete. 2024-09-21T07:03:21.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.104+0000 7fd9d9247740 1 -- start start 2024-09-21T07:03:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.104+0000 7fd9d9247740 1 Processor -- start 2024-09-21T07:03:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 0x55a55b3f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 0x55a55b3f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afd21b0 0x55a55b3f8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b89f0 con 0x55a55afb14d0 2024-09-21T07:03:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b82f0 con 0x55a55afd21b0 2024-09-21T07:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc1b5550 con 0x55a55afa92f0 2024-09-21T07:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9d9247740 10 client.0 _ll_register_callbacks cb 0x7fd9c4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 0x55a55b3f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 0x55a55b3f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41182/0 (socket says 172.21.15.114:41182) 2024-09-21T07:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9cb400640 1 -- 172.21.15.114:0/2303474203 learned_addr learned my addr 172.21.15.114:0/2303474203 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:21.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.105+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afd21b0 0x55a55b3f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.106+0000 7fd9cb400640 1 -- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afd21b0 msgr2=0x55a55b3f8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.106+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afd21b0 0x55a55b3f8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.106+0000 7fd9cb400640 1 -- 172.21.15.114:0/2303474203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 msgr2=0x55a55b3f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.106+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 0x55a55b3f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.106+0000 7fd9cb400640 1 -- 172.21.15.114:0/2303474203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a55b379720 con 0x55a55afa92f0 2024-09-21T07:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.107+0000 7fd9cbe00640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afd21b0 0x55a55b3f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.108+0000 7fd9cb400640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 0x55a55b3f79d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005d40 tx=0x7fd9bc0098c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a08357a39d7eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.108+0000 7fd9b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.109+0000 7fd9b8c00640 1 -- 172.21.15.114:0/2303474203 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc01ac60 con 0x55a55afa92f0 2024-09-21T07:03:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.109+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c40b1ed0 con 0x55a55afa92f0 2024-09-21T07:03:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.109+0000 7fd9b8c00640 1 -- 172.21.15.114:0/2303474203 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fd9bc01ae00 con 0x55a55afa92f0 2024-09-21T07:03:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.113+0000 7fd9b8c00640 1 -- 172.21.15.114:0/2303474203 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc03f050 con 0x55a55afa92f0 2024-09-21T07:03:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:20 smithi183 ceph-mon[34219]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T07:03:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:20 smithi183 ceph-mon[34219]: pgmap v678: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-09-21T07:03:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:20 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:03:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:20 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:21.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:20 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.113+0000 7fd9d9247740 10 client.27859 fetch_fsmap learned FSMap version 48 2024-09-21T07:03:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.113+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55a55b40fb40 con 0x55a55afa92f0 2024-09-21T07:03:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.114+0000 7fd9b8c00640 1 -- 172.21.15.114:0/2303474203 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c40b1ed0 con 0x55a55afa92f0 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9b8c00640 1 -- 172.21.15.114:0/2303474203 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7fd9bc004c50 con 0x55a55afa92f0 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9d9247740 10 client.27859 fetch_fsmap finished waiting for FSMap version 48 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9d9247740 10 client.27859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9d9247740 20 client.27859 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af661c0 0x55a55b4121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9d9247740 4 client.27859 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a55b2ea0b0 con 0x55a55af661c0 2024-09-21T07:03:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.117+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af661c0 0x55a55b4121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.118+0000 7fd9c2a00640 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af661c0 0x55a55b4121e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff0dfcf5e293914 server_cookie=5229994b89794d33 in_seq=0 out_seq=0 2024-09-21T07:03:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.118+0000 7fd9b8c00640 10 client.27859 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:20 smithi193 ceph-mon[34601]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T07:03:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:20 smithi193 ceph-mon[34601]: pgmap v678: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-09-21T07:03:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:20 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:03:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:20 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:20 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:20 smithi114 ceph-mon[30654]: osdmap e95: 12 total, 12 up, 12 in 2024-09-21T07:03:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:20 smithi114 ceph-mon[30654]: pgmap v678: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.9 MiB/s wr, 499 op/s 2024-09-21T07:03:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:20 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:03:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:20 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:21.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:20 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9b8c00640 1 -- 172.21.15.114:0/2303474203 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a55b2ea0b0 con 0x55a55af661c0 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9b8c00640 10 client.27859 handle_command_reply: tid=0 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9d9247740 1 client.27859 shutdown 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 msgr2=0x55a55b3f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 0x55a55b3f79d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005d40 tx=0x7fd9bc0098c0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 shutdown_connections 2024-09-21T07:03:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a55afd21b0 0x55a55b3f8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.551+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a55afb14d0 0x55a55b3f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.552+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a55af661c0 0x55a55b4121e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.552+0000 7fd9d9247740 1 --2- 172.21.15.114:0/2303474203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a55afa92f0 0x55a55b3f79d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.552+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 >> 172.21.15.114:0/2303474203 conn(0x55a55afc6300 msgr2=0x55a55b374ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.552+0000 7fd9b8c00640 0 client.27859 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.552+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 shutdown_connections 2024-09-21T07:03:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.552+0000 7fd9d9247740 1 -- 172.21.15.114:0/2303474203 wait complete. 2024-09-21T07:03:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.553+0000 7fd9d9247740 20 client.27859 trim_cache size 0 max 16384 2024-09-21T07:03:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:03:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:03:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:03:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:03:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9b00b66b0 msgr2=0x7fd9b00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9b00b66b0 0x7fd9b00b8bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40095f0 tx=0x7fd9b4030340 comp rx=0 tx=0).stop 2024-09-21T07:03:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109d60 msgr2=0x7fd9cc1ac750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109d60 0x7fd9cc1ac750 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400eba0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 shutdown_connections 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9cc109d60 0x7fd9cc1ac750 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.554+0000 7fd9d2200640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9cc109410 0x7fd9cc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.555+0000 7fd9d2200640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9b00b66b0 0x7fd9b00b8bc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.555+0000 7fd9d2200640 1 --2- 172.21.15.114:0/642069433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9cc103ae0 0x7fd9cc10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.555+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 >> 172.21.15.114:0/642069433 conn(0x7fd9cc0ff520 msgr2=0x7fd9cc101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.555+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 shutdown_connections 2024-09-21T07:03:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:21.555+0000 7fd9d2200640 1 -- 172.21.15.114:0/642069433 wait complete. 2024-09-21T07:03:22.116 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:03:22.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:21 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:03:22.228 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:21 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:22.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:22.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:22.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:22.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:22.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:03:22.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:21 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:22.809 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 525.0 s, 664.2 tps, lat 148.116 ms stddev 210.626 2024-09-21T07:03:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:22 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:22 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:22 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:22 smithi193 ceph-mon[34601]: pgmap v679: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.1 MiB/s wr, 496 op/s 2024-09-21T07:03:23.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:22 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:22 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:22 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:22 smithi114 ceph-mon[30654]: pgmap v679: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.1 MiB/s wr, 496 op/s 2024-09-21T07:03:23.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:22 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:22 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:22 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:23.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:22 smithi183 ceph-mon[34219]: pgmap v679: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 5.1 MiB/s wr, 496 op/s 2024-09-21T07:03:24.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:23 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:24.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:23 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:24.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:23 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:03:24.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.901+0000 7f6c3d600640 1 -- 172.21.15.114:0/3385275001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 msgr2=0x7f6c38107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.901+0000 7f6c3d600640 1 --2- 172.21.15.114:0/3385275001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 0x7f6c38107620 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009900 tx=0x7f6c2402a400 comp rx=0 tx=0).stop 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.902+0000 7f6c3d600640 1 -- 172.21.15.114:0/3385275001 shutdown_connections 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.902+0000 7f6c3d600640 1 --2- 172.21.15.114:0/3385275001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c381080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.902+0000 7f6c3d600640 1 --2- 172.21.15.114:0/3385275001 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 0x7f6c38107620 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.902+0000 7f6c3d600640 1 --2- 172.21.15.114:0/3385275001 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 0x7f6c38102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.902+0000 7f6c3d600640 1 -- 172.21.15.114:0/3385275001 >> 172.21.15.114:0/3385275001 conn(0x7f6c38079730 msgr2=0x7f6c38079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.903+0000 7f6c3d600640 1 -- 172.21.15.114:0/3385275001 shutdown_connections 2024-09-21T07:03:24.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.903+0000 7f6c3d600640 1 -- 172.21.15.114:0/3385275001 wait complete. 2024-09-21T07:03:24.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.904+0000 7f6c3d600640 1 Processor -- start 2024-09-21T07:03:24.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.905+0000 7f6c3d600640 1 -- start start 2024-09-21T07:03:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.906+0000 7f6c3d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 0x7f6c3806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.907+0000 7f6c3d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 0x7f6c3806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.907+0000 7f6c3d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c38072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.907+0000 7f6c3d600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3811a090 con 0x7f6c381027a0 2024-09-21T07:03:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.907+0000 7f6c3d600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38119f10 con 0x7f6c38107be0 2024-09-21T07:03:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.907+0000 7f6c3d600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c3811a210 con 0x7f6c381030f0 2024-09-21T07:03:24.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.908+0000 7f6c37400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c38072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.908+0000 7f6c37400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c38072ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43876/0 (socket says 172.21.15.114:43876) 2024-09-21T07:03:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.908+0000 7f6c37400640 1 -- 172.21.15.114:0/2777433406 learned_addr learned my addr 172.21.15.114:0/2777433406 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.908+0000 7f6c36a00640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 0x7f6c3806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.908+0000 7f6c37400640 1 -- 172.21.15.114:0/2777433406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 msgr2=0x7f6c3806dc30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:03:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.910+0000 7f6c36000640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 0x7f6c3806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.910+0000 7f6c37400640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 0x7f6c3806dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.910+0000 7f6c37400640 1 -- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 msgr2=0x7f6c3806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.910+0000 7f6c37400640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 0x7f6c3806d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.910+0000 7f6c37400640 1 -- 172.21.15.114:0/2777433406 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c38073530 con 0x7f6c38107be0 2024-09-21T07:03:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.910+0000 7f6c36a00640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 0x7f6c3806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.911+0000 7f6c37400640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c38072ef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000efd0 tx=0x7f6c3000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.912+0000 7f6c2f400640 1 -- 172.21.15.114:0/2777433406 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30003b70 con 0x7f6c38107be0 2024-09-21T07:03:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.912+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c38073820 con 0x7f6c38107be0 2024-09-21T07:03:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.912+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c38074060 con 0x7f6c38107be0 2024-09-21T07:03:24.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.913+0000 7f6c2f400640 1 -- 172.21.15.114:0/2777433406 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c30009820 con 0x7f6c38107be0 2024-09-21T07:03:24.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.913+0000 7f6c2f400640 1 -- 172.21.15.114:0/2777433406 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c30010790 con 0x7f6c38107be0 2024-09-21T07:03:24.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.916+0000 7f6c441c3740 1 Processor -- start 2024-09-21T07:03:24.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.917+0000 7f6c2f400640 1 -- 172.21.15.114:0/2777433406 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6c30003fe0 con 0x7f6c38107be0 2024-09-21T07:03:24.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.917+0000 7f6c441c3740 1 -- start start 2024-09-21T07:03:24.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e76383d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e763842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e763847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c441c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555e762faf70 con 0x555e75fbc390 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c441c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e762ea3a0 con 0x555e75fb41b0 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c441c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e762ee1b0 con 0x555e75fdd070 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e76383d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e76383d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:38380/0 (socket says 172.21.15.114:38380) 2024-09-21T07:03:24.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.918+0000 7f6c36000640 1 -- 172.21.15.114:0/1337856190 learned_addr learned my addr 172.21.15.114:0/1337856190 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:24.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.919+0000 7f6c36000640 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e763847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.919+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e763842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.921+0000 7f6c2f400640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c1c0b6700 0x7f6c1c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.921+0000 7f6c36a00640 1 -- 172.21.15.114:0/1337856190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 msgr2=0x555e76383d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.921+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e76383d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.921+0000 7f6c36a00640 1 -- 172.21.15.114:0/1337856190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 msgr2=0x555e763847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e763847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c36a00640 1 -- 172.21.15.114:0/1337856190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e762b4c90 con 0x555e75fb41b0 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c2f400640 1 -- 172.21.15.114:0/2777433406 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f6c300e7eb0 con 0x7f6c38107be0 2024-09-21T07:03:24.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e763842a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6c3806bc60 tx=0x7f6c2800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1429a84d28b985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c23400640 1 -- 172.21.15.114:0/1337856190 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28014070 con 0x555e75fb41b0 2024-09-21T07:03:24.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c23400640 1 -- 172.21.15.114:0/1337856190 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c28004220 con 0x555e75fb41b0 2024-09-21T07:03:24.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.922+0000 7f6c23400640 1 -- 172.21.15.114:0/1337856190 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28010690 con 0x555e75fb41b0 2024-09-21T07:03:24.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.923+0000 7f6c37400640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c1c0b6700 0x7f6c1c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.924+0000 7f6c441c3740 1 -- 172.21.15.114:0/1337856190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 msgr2=0x555e763842a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.924+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e763842a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6c3806bc60 tx=0x7f6c2800e720 comp rx=0 tx=0).stop 2024-09-21T07:03:24.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.925+0000 7f6c37400640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c1c0b6700 0x7f6c1c0b8c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6c300ad860 tx=0x7f6c30004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 -- 172.21.15.114:0/1337856190 shutdown_connections 2024-09-21T07:03:24.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e763847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e763842a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1337856190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e76383d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 -- 172.21.15.114:0/1337856190 >> 172.21.15.114:0/1337856190 conn(0x555e75fd1460 msgr2=0x555e76381c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 -- 172.21.15.114:0/1337856190 shutdown_connections 2024-09-21T07:03:24.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.926+0000 7f6c441c3740 1 -- 172.21.15.114:0/1337856190 wait complete. 2024-09-21T07:03:24.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.927+0000 7f6c441c3740 1 -- start start 2024-09-21T07:03:24.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.927+0000 7f6c441c3740 1 Processor -- start 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e7638f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e7638fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7639b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555e7614b700 con 0x555e75fbc390 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e76285540 con 0x555e75fb41b0 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e76392540 con 0x555e75fdd070 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c441c3740 10 client.0 _ll_register_callbacks cb 0x555e76295430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:24.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c36000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e7638f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.929+0000 7f6c36000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e7638f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43896/0 (socket says 172.21.15.114:43896) 2024-09-21T07:03:24.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.930+0000 7f6c36000640 1 -- 172.21.15.114:0/1969446688 learned_addr learned my addr 172.21.15.114:0/1969446688 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:24.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.930+0000 7f6c36000640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7639b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.930+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e7638fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.931+0000 7f6c36a00640 1 -- 172.21.15.114:0/1969446688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 msgr2=0x555e7639b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.931+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7639b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.931+0000 7f6c36a00640 1 -- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 msgr2=0x555e7638f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.931+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e7638f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.931+0000 7f6c36a00640 1 -- 172.21.15.114:0/1969446688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e76048650 con 0x555e75fbc390 2024-09-21T07:03:24.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.932+0000 7f6c36a00640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e7638fa80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6c28003cd0 tx=0x7f6c28008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741cab1fbbdc9699 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.932+0000 7f6c17e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:24.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.933+0000 7f6c17e00640 1 -- 172.21.15.114:0/1969446688 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28014070 con 0x555e75fbc390 2024-09-21T07:03:24.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.933+0000 7f6c17e00640 1 -- 172.21.15.114:0/1969446688 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f6c28005450 con 0x555e75fbc390 2024-09-21T07:03:24.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.933+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e76287b50 con 0x555e75fbc390 2024-09-21T07:03:24.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.933+0000 7f6c17e00640 1 -- 172.21.15.114:0/1969446688 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6c280087d0 con 0x555e75fbc390 2024-09-21T07:03:24.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.933+0000 7f6c441c3740 10 client.17481 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:24.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.933+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555e762a2910 con 0x555e75fbc390 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c17e00640 1 -- 172.21.15.114:0/1969446688 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555e76287b50 con 0x555e75fbc390 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c17e00640 1 -- 172.21.15.114:0/1969446688 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f6c28004a70 con 0x555e75fbc390 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c441c3740 10 client.17481 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c441c3740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c441c3740 20 client.17481 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f9b9f0 0x555e763b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c441c3740 4 client.17481 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:24.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e762fcec0 con 0x555e75f9b9f0 2024-09-21T07:03:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.935+0000 7f6c36000640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f9b9f0 0x555e763b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.936+0000 7f6c36000640 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f9b9f0 0x555e763b2550 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708661e54af27882 server_cookie=7cbe9e149a003946 in_seq=0 out_seq=0 2024-09-21T07:03:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.936+0000 7f6c17e00640 10 client.17481 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.937+0000 7f6c17e00640 1 -- 172.21.15.114:0/1969446688 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e762fcec0 con 0x555e75f9b9f0 2024-09-21T07:03:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.937+0000 7f6c17e00640 10 client.17481 handle_command_reply: tid=0 2024-09-21T07:03:24.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.937+0000 7f6c441c3740 1 client.17481 shutdown 2024-09-21T07:03:24.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.937+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 msgr2=0x555e7638fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.937+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e7638fa80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6c28003cd0 tx=0x7f6c28008ec0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 shutdown_connections 2024-09-21T07:03:24.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f9b9f0 0x555e763b2550 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c17e00640 0 client.17481 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7639b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fbc390 0x555e7638fa80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 --2- 172.21.15.114:0/1969446688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fb41b0 0x555e7638f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 >> 172.21.15.114:0/1969446688 conn(0x555e75fd1460 msgr2=0x555e76386ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 shutdown_connections 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 1 -- 172.21.15.114:0/1969446688 wait complete. 2024-09-21T07:03:24.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.938+0000 7f6c441c3740 20 client.17481 trim_cache size 0 max 16384 2024-09-21T07:03:24.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.965+0000 7f6c441c3740 1 Processor -- start 2024-09-21T07:03:24.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.966+0000 7f6c441c3740 1 -- start start 2024-09-21T07:03:24.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.966+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 0x555e7638e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e7639b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7638a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c441c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1800ffe0 con 0x555e75fb41b0 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c441c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18011b20 con 0x555e75fbc390 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c441c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180106b0 con 0x555e75fdd070 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c36000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 0x555e7638e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c36000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 0x555e7638e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39554/0 (socket says 172.21.15.114:39554) 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c36000640 1 -- 172.21.15.114:0/3695755869 learned_addr learned my addr 172.21.15.114:0/3695755869 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:24.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c36000640 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7638a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.967+0000 7f6c36a00640 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e7639b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.968+0000 7f6c36000640 1 -- 172.21.15.114:0/3695755869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 msgr2=0x555e7638a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.968+0000 7f6c36000640 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7638a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.968+0000 7f6c36000640 1 -- 172.21.15.114:0/3695755869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 msgr2=0x555e7639b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.968+0000 7f6c36000640 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e7639b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.968+0000 7f6c36000640 1 -- 172.21.15.114:0/3695755869 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e76386bc0 con 0x555e75fb41b0 2024-09-21T07:03:24.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.968+0000 7f6c36000640 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 0x555e7638e430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x555e76392110 tx=0x7f6c24005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacd4a242c4b3aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.969+0000 7f6c22a00640 1 -- 172.21.15.114:0/3695755869 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24035040 con 0x555e75fb41b0 2024-09-21T07:03:24.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.969+0000 7f6c22a00640 1 -- 172.21.15.114:0/3695755869 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c24004c30 con 0x555e75fb41b0 2024-09-21T07:03:24.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.969+0000 7f6c22a00640 1 -- 172.21.15.114:0/3695755869 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24003a90 con 0x555e75fb41b0 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 -- 172.21.15.114:0/3695755869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 msgr2=0x555e7638e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 0x555e7638e430 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x555e76392110 tx=0x7f6c24005fd0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 -- 172.21.15.114:0/3695755869 shutdown_connections 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fdd070 0x555e7638a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e7639b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 --2- 172.21.15.114:0/3695755869 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fb41b0 0x555e7638e430 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 -- 172.21.15.114:0/3695755869 >> 172.21.15.114:0/3695755869 conn(0x555e75fd1460 msgr2=0x555e76381590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 -- 172.21.15.114:0/3695755869 shutdown_connections 2024-09-21T07:03:24.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.970+0000 7f6c441c3740 1 -- 172.21.15.114:0/3695755869 wait complete. 2024-09-21T07:03:24.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.971+0000 7f6c441c3740 1 -- start start 2024-09-21T07:03:24.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.971+0000 7f6c441c3740 1 Processor -- start 2024-09-21T07:03:24.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fb41b0 0x555e76393f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e76394490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fdd070 0x555e7641bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18011450 con 0x555e75fdd070 2024-09-21T07:03:24.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18010d80 con 0x555e75fbc390 2024-09-21T07:03:24.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1800dfe0 con 0x555e75fb41b0 2024-09-21T07:03:24.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.972+0000 7f6c441c3740 10 client.0 _ll_register_callbacks cb 0x7f6c28008a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:24.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e76394490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e76394490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43922/0 (socket says 172.21.15.114:43922) 2024-09-21T07:03:24.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36a00640 1 -- 172.21.15.114:0/4043956633 learned_addr learned my addr 172.21.15.114:0/4043956633 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:24.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36000640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fb41b0 0x555e76393f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36000640 1 -- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 msgr2=0x555e76394490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36000640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e76394490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36000640 1 -- 172.21.15.114:0/4043956633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fdd070 msgr2=0x555e7641bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36000640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fdd070 0x555e7641bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36000640 1 -- 172.21.15.114:0/4043956633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7641c290 con 0x555e75fb41b0 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.973+0000 7f6c36a00640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e76394490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c36000640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fb41b0 0x555e76393f50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009900 tx=0x7f6c24005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5a4d6979b78f9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:24.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c17e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c17e00640 1 -- 172.21.15.114:0/4043956633 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2405c060 con 0x555e75fb41b0 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c17e00640 1 -- 172.21.15.114:0/4043956633 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f6c240037f0 con 0x555e75fb41b0 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c24004c30 con 0x555e75fb41b0 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c17e00640 1 -- 172.21.15.114:0/4043956633 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6c24035070 con 0x555e75fb41b0 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c441c3740 10 client.27874 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.974+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555e7641d520 con 0x555e75fb41b0 2024-09-21T07:03:24.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.975+0000 7f6c17e00640 1 -- 172.21.15.114:0/4043956633 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c24004c30 con 0x555e75fb41b0 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.975+0000 7f6c17e00640 1 -- 172.21.15.114:0/4043956633 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f6c2405f050 con 0x555e75fb41b0 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c441c3740 10 client.27874 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c441c3740 10 client.27874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c441c3740 20 client.27874 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c441c3740 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f70da0 0x555e7641faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c441c3740 4 client.27874 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e762f8970 con 0x555e75f70da0 2024-09-21T07:03:24.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.976+0000 7f6c36a00640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f70da0 0x555e7641faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c36a00640 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f70da0 0x555e7641faf0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825c912d8d0e90ea server_cookie=40da43756f9da7f0 in_seq=0 out_seq=0 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c17e00640 10 client.27874 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c17e00640 1 -- 172.21.15.114:0/4043956633 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x555e762f8970 con 0x555e75f70da0 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c17e00640 10 client.27874 handle_command_reply: tid=0 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c441c3740 1 client.27874 shutdown 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fb41b0 msgr2=0x555e76393f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.977+0000 7f6c441c3740 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fb41b0 0x555e76393f50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009900 tx=0x7f6c24005ec0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 shutdown_connections 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555e75fdd070 0x555e7641bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e75fbc390 0x555e76394490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x555e75f70da0 0x555e7641faf0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 --2- 172.21.15.114:0/4043956633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e75fb41b0 0x555e76393f50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 >> 172.21.15.114:0/4043956633 conn(0x555e75fd1460 msgr2=0x555e76381510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c17e00640 0 client.27874 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:24.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 shutdown_connections 2024-09-21T07:03:24.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 1 -- 172.21.15.114:0/4043956633 wait complete. 2024-09-21T07:03:24.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.978+0000 7f6c441c3740 20 client.27874 trim_cache size 0 max 16384 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551", 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:03:24.983 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.985 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.986 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.987 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:24.988 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c1c0b6700 msgr2=0x7f6c1c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c1c0b6700 0x7f6c1c0b8c10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6c300ad860 tx=0x7f6c30004a40 comp rx=0 tx=0).stop 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 msgr2=0x7f6c38072ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:24.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c38072ef0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000efd0 tx=0x7f6c3000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 shutdown_connections 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c1c0b6700 0x7f6c1c0b8c10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c38107be0 0x7f6c38072ef0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c381030f0 0x7f6c3806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.979+0000 7f6c3d600640 1 --2- 172.21.15.114:0/2777433406 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c381027a0 0x7f6c3806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.980+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 >> 172.21.15.114:0/2777433406 conn(0x7f6c38079730 msgr2=0x7f6c381055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.981+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 shutdown_connections 2024-09-21T07:03:24.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:24.981+0000 7f6c3d600640 1 -- 172.21.15.114:0/2777433406 wait complete. 2024-09-21T07:03:25.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:24 smithi193 ceph-mon[34601]: pgmap v680: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.0 MiB/s wr, 583 op/s 2024-09-21T07:03:25.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:24 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:03:25.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:24 smithi114 ceph-mon[30654]: pgmap v680: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.0 MiB/s wr, 583 op/s 2024-09-21T07:03:25.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:24 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:03:25.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:24 smithi183 ceph-mon[34219]: pgmap v680: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 5.0 MiB/s wr, 583 op/s 2024-09-21T07:03:25.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:24 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:03:26.874 INFO:tasks.cephfs.filesystem:scrub status for tag:427a4e34-6fa9-4f02-b1be-7f0b6f8ee551 - {'path': '/', 'tag': '427a4e34-6fa9-4f02-b1be-7f0b6f8ee551', 'options': 'recursive,force'} 2024-09-21T07:03:27.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:26 smithi114 ceph-mon[30654]: pgmap v681: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 5.4 MiB/s wr, 658 op/s 2024-09-21T07:03:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:26 smithi193 ceph-mon[34601]: pgmap v681: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 5.4 MiB/s wr, 658 op/s 2024-09-21T07:03:27.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:26 smithi183 ceph-mon[34219]: pgmap v681: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 5.4 MiB/s wr, 658 op/s 2024-09-21T07:03:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 530.0 s, 501.0 tps, lat 198.096 ms stddev 248.530 2024-09-21T07:03:29.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:28 smithi114 ceph-mon[30654]: pgmap v682: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 5.4 MiB/s wr, 660 op/s 2024-09-21T07:03:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:28 smithi183 ceph-mon[34219]: pgmap v682: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 5.4 MiB/s wr, 660 op/s 2024-09-21T07:03:29.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:28 smithi193 ceph-mon[34601]: pgmap v682: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 5.4 MiB/s wr, 660 op/s 2024-09-21T07:03:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:30 smithi193 ceph-mon[34601]: pgmap v683: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 5.6 MiB/s wr, 684 op/s 2024-09-21T07:03:31.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:30 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:03:31.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:30 smithi114 ceph-mon[30654]: pgmap v683: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 5.6 MiB/s wr, 684 op/s 2024-09-21T07:03:31.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:30 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:03:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:30 smithi183 ceph-mon[34219]: pgmap v683: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 5.6 MiB/s wr, 684 op/s 2024-09-21T07:03:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:30 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:03:32.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:31 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:03:32.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:31 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:03:32.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:31 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:03:32.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:31 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:03:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:31 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:03:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:31 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:03:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:31 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:03:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:31 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:03:32.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:31 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:03:32.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:31 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:03:32.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:31 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:03:32.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:31 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:03:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 535.0 s, 514.2 tps, lat 200.776 ms stddev 253.812 2024-09-21T07:03:33.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:32 smithi193 ceph-mon[34601]: pgmap v684: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.5 MiB/s wr, 633 op/s 2024-09-21T07:03:33.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:32 smithi114 ceph-mon[30654]: pgmap v684: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.5 MiB/s wr, 633 op/s 2024-09-21T07:03:33.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:32 smithi183 ceph-mon[34219]: pgmap v684: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.5 MiB/s wr, 633 op/s 2024-09-21T07:03:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3791606321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 msgr2=0x7f8a18103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3791606321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a18103ee0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009900 tx=0x7f8a0802a410 comp rx=0 tx=0).stop 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3791606321 shutdown_connections 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3791606321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a18109d50 0x7f8a18110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3791606321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109400 0x7f8a18109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3791606321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a18103ee0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.845+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3791606321 >> 172.21.15.114:0/3791606321 conn(0x7f8a180ff510 msgr2=0x7f8a18101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.846+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3791606321 shutdown_connections 2024-09-21T07:03:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.846+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3791606321 wait complete. 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 Processor -- start 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 -- start start 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a1819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a18109400 0x7f8a1819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109d50 0x7f8a181a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18113a10 con 0x7f8a18103ad0 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18113890 con 0x7f8a18109400 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a1dc00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a18113b90 con 0x7f8a18109d50 2024-09-21T07:03:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a16a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a1819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a16a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a1819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49326/0 (socket says 172.21.15.114:49326) 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a16a00640 1 -- 172.21.15.114:0/3182248104 learned_addr learned my addr 172.21.15.114:0/3182248104 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a16000640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a18109400 0x7f8a1819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.847+0000 7f8a17400640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109d50 0x7f8a181a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a16a00640 1 -- 172.21.15.114:0/3182248104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109d50 msgr2=0x7f8a181a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a16a00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109d50 0x7f8a181a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a16a00640 1 -- 172.21.15.114:0/3182248104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a18109400 msgr2=0x7f8a1819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a16a00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a18109400 0x7f8a1819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a16a00640 1 -- 172.21.15.114:0/3182248104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a181a25c0 con 0x7f8a18103ad0 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a17400640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109d50 0x7f8a181a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.848+0000 7f8a16a00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a1819c690 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005940 tx=0x7f8a080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.849+0000 7f8a0f400640 1 -- 172.21.15.114:0/3182248104 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0803f030 con 0x7f8a18103ad0 2024-09-21T07:03:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.849+0000 7f8a0f400640 1 -- 172.21.15.114:0/3182248104 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a080044c0 con 0x7f8a18103ad0 2024-09-21T07:03:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.849+0000 7f8a0f400640 1 -- 172.21.15.114:0/3182248104 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08035070 con 0x7f8a18103ad0 2024-09-21T07:03:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.849+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181a2850 con 0x7f8a18103ad0 2024-09-21T07:03:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.849+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181a9040 con 0x7f8a18103ad0 2024-09-21T07:03:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.853+0000 7f8a24dd1740 1 Processor -- start 2024-09-21T07:03:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.853+0000 7f8a24dd1740 1 -- start start 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 0x564437601c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644372342f0 0x564437602170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x5644376026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5644375776b0 con 0x56443725d1b0 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564437566ae0 con 0x56443723c4d0 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56443756a8f0 con 0x5644372342f0 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a17400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 0x564437601c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a16000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x5644376026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a17400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 0x564437601c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49334/0 (socket says 172.21.15.114:49334) 2024-09-21T07:03:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a17400640 1 -- 172.21.15.114:0/440960477 learned_addr learned my addr 172.21.15.114:0/440960477 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a16000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x5644376026b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:53028/0 (socket says 172.21.15.114:53028) 2024-09-21T07:03:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.854+0000 7f8a0f400640 1 -- 172.21.15.114:0/3182248104 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8a08042030 con 0x7f8a18103ad0 2024-09-21T07:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a16a00640 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644372342f0 0x564437602170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a17400640 1 -- 172.21.15.114:0/440960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644372342f0 msgr2=0x564437602170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a17400640 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644372342f0 0x564437602170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a17400640 1 -- 172.21.15.114:0/440960477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 msgr2=0x5644376026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a17400640 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x5644376026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a17400640 1 -- 172.21.15.114:0/440960477 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564437533780 con 0x56443725d1b0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a16000640 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x5644376026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a17400640 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 0x564437601c30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8a100117e0 tx=0x7f8a10011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05faa862fed42f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.855+0000 7f8a03e00640 1 -- 172.21.15.114:0/440960477 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10011310 con 0x56443725d1b0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a03e00640 1 -- 172.21.15.114:0/440960477 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10003e20 con 0x56443725d1b0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a03e00640 1 -- 172.21.15.114:0/440960477 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a100044f0 con 0x56443725d1b0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a0f400640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6700 0x7f89fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a0f400640 1 -- 172.21.15.114:0/3182248104 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f8a08107e00 con 0x7f8a18103ad0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a16a00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6700 0x7f89fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a24dd1740 1 -- 172.21.15.114:0/440960477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 msgr2=0x564437601c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.856+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 0x564437601c30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8a100117e0 tx=0x7f8a10011ca0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 -- 172.21.15.114:0/440960477 shutdown_connections 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x5644376026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5644372342f0 0x564437602170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/440960477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443725d1b0 0x564437601c30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 -- 172.21.15.114:0/440960477 >> 172.21.15.114:0/440960477 conn(0x564437251300 msgr2=0x5644375ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 -- 172.21.15.114:0/440960477 shutdown_connections 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 -- 172.21.15.114:0/440960477 wait complete. 2024-09-21T07:03:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a16a00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6700 0x7f89fc0b8c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8a08105230 tx=0x7f8a0802a410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.857+0000 7f8a24dd1740 1 -- start start 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 Processor -- start 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x56443760d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x56443760dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56443749eab0 con 0x5644372342f0 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56443750d570 con 0x56443723c4d0 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5644376103b0 con 0x56443725d1b0 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a24dd1740 10 client.0 _ll_register_callbacks cb 0x56443751c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a17400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a17400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49338/0 (socket says 172.21.15.114:49338) 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a17400640 1 -- 172.21.15.114:0/1595166569 learned_addr learned my addr 172.21.15.114:0/1595166569 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.858+0000 7f8a16000640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x56443760d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a17400640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x56443760dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a16000640 1 -- 172.21.15.114:0/1595166569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 msgr2=0x56443760dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a16000640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x56443760dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a16000640 1 -- 172.21.15.114:0/1595166569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 msgr2=0x56443760d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a16000640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a16000640 1 -- 172.21.15.114:0/1595166569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644372c8790 con 0x56443723c4d0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a16000640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x56443760d7a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8a18107e00 tx=0x7f8a04013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5e16cb62de822a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a00c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.859+0000 7f8a00c00640 1 -- 172.21.15.114:0/1595166569 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04011990 con 0x56443723c4d0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a00c00640 1 -- 172.21.15.114:0/1595166569 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f8a04011b30 con 0x56443723c4d0 2024-09-21T07:03:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a10003e20 con 0x56443723c4d0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a00c00640 1 -- 172.21.15.114:0/1595166569 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8a04011d30 con 0x56443723c4d0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 10 client.27935 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564437341620 con 0x56443723c4d0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a00c00640 1 -- 172.21.15.114:0/1595166569 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a10003e20 con 0x56443723c4d0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a00c00640 1 -- 172.21.15.114:0/1595166569 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f8a0401a070 con 0x56443723c4d0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 10 client.27935 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 10 client.27935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 20 client.27935 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56443721bd80 0x56443761f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 4 client.27935 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.860+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564437579600 con 0x56443721bd80 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.861+0000 7f8a17400640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56443721bd80 0x56443761f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.861+0000 7f8a17400640 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56443721bd80 0x56443761f690 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d9b083f8444cfc server_cookie=a968e6132610f574 in_seq=0 out_seq=0 2024-09-21T07:03:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.861+0000 7f8a00c00640 10 client.27935 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a00c00640 1 -- 172.21.15.114:0/1595166569 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564437579600 con 0x56443721bd80 2024-09-21T07:03:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a00c00640 10 client.27935 handle_command_reply: tid=0 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 client.27935 shutdown 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 msgr2=0x56443760d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x56443760d7a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8a18107e00 tx=0x7f8a04013b70 comp rx=0 tx=0).stop 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 shutdown_connections 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56443721bd80 0x56443761f690 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x56443760dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443723c4d0 0x56443760d7a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/1595166569 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a00c00640 0 client.27935 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 >> 172.21.15.114:0/1595166569 conn(0x564437251300 msgr2=0x564437602c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 shutdown_connections 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 1 -- 172.21.15.114:0/1595166569 wait complete. 2024-09-21T07:03:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.862+0000 7f8a24dd1740 20 client.27935 trim_cache size 0 max 16384 2024-09-21T07:03:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 Processor -- start 2024-09-21T07:03:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 -- start start 2024-09-21T07:03:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 0x5644376225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443725d1b0 0x564437622b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b3150 con 0x5644372342f0 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b4c90 con 0x56443725d1b0 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b3820 con 0x56443723c4d0 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a17400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a16000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 0x5644376225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a17400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49350/0 (socket says 172.21.15.114:49350) 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a16000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 0x5644376225d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45554/0 (socket says 172.21.15.114:45554) 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a17400640 1 -- 172.21.15.114:0/122781815 learned_addr learned my addr 172.21.15.114:0/122781815 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.878+0000 7f8a17400640 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443725d1b0 0x564437622b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a16000640 1 -- 172.21.15.114:0/122781815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443725d1b0 msgr2=0x564437622b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a16000640 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443725d1b0 0x564437622b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a16000640 1 -- 172.21.15.114:0/122781815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 msgr2=0x56443760ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a16000640 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a16000640 1 -- 172.21.15.114:0/122781815 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564437619d90 con 0x56443723c4d0 2024-09-21T07:03:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a16000640 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 0x5644376225d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401b940 tx=0x7f8a04009d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d34889c93189677 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a03400640 1 -- 172.21.15.114:0/122781815 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402d020 con 0x56443723c4d0 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a03400640 1 -- 172.21.15.114:0/122781815 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a04005460 con 0x56443723c4d0 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.879+0000 7f8a03400640 1 -- 172.21.15.114:0/122781815 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003d00 con 0x56443723c4d0 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 -- 172.21.15.114:0/122781815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 msgr2=0x5644376225d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 0x5644376225d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401b940 tx=0x7f8a04009d70 comp rx=0 tx=0).stop 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 -- 172.21.15.114:0/122781815 shutdown_connections 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56443725d1b0 0x564437622b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443723c4d0 0x5644376225d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/122781815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5644372342f0 0x56443760ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 -- 172.21.15.114:0/122781815 >> 172.21.15.114:0/122781815 conn(0x564437251300 msgr2=0x56443760e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 -- 172.21.15.114:0/122781815 shutdown_connections 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.880+0000 7f8a24dd1740 1 -- 172.21.15.114:0/122781815 wait complete. 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 -- start start 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 Processor -- start 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644372342f0 0x564437621f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 0x564437622540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x564437681e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b45c0 con 0x56443723c4d0 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b3ef0 con 0x5644372342f0 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a181b1150 con 0x56443725d1b0 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a24dd1740 10 client.0 _ll_register_callbacks cb 0x7f8a040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a16000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 0x564437622540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.881+0000 7f8a16000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 0x564437622540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49362/0 (socket says 172.21.15.114:49362) 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 -- 172.21.15.114:0/749759626 learned_addr learned my addr 172.21.15.114:0/749759626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a17400640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x564437681e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a17400640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644372342f0 0x564437621f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 -- 172.21.15.114:0/749759626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 msgr2=0x564437681e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x564437681e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 -- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644372342f0 msgr2=0x564437621f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644372342f0 0x564437621f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 -- 172.21.15.114:0/749759626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644376824f0 con 0x56443723c4d0 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a17400640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644372342f0 0x564437621f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a16000640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 0x564437622540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8a04012010 tx=0x7f8a04005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967c0c617a33de96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a00c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a00c00640 1 -- 172.21.15.114:0/749759626 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003a00 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a00c00640 1 -- 172.21.15.114:0/749759626 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f8a04003ba0 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.882+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a04005460 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a00c00640 1 -- 172.21.15.114:0/749759626 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8a04003da0 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 10 client.17502 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564437689830 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a00c00640 1 -- 172.21.15.114:0/749759626 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a04005460 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a00c00640 1 -- 172.21.15.114:0/749759626 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f8a0401a070 con 0x56443723c4d0 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 10 client.17502 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 20 client.17502 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644371f11c0 0x56443768beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 4 client.17502 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:03:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.883+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644375750b0 con 0x5644371f11c0 2024-09-21T07:03:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.884+0000 7f8a17400640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644371f11c0 0x56443768beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.884+0000 7f8a17400640 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644371f11c0 0x56443768beb0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f400321aa1b5d794 server_cookie=aef542c91414e9c1 in_seq=0 out_seq=0 2024-09-21T07:03:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.884+0000 7f8a00c00640 10 client.17502 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a00c00640 1 -- 172.21.15.114:0/749759626 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644375750b0 con 0x5644371f11c0 2024-09-21T07:03:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a00c00640 10 client.17502 handle_command_reply: tid=0 2024-09-21T07:03:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 client.17502 shutdown 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 msgr2=0x564437622540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 0x564437622540 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8a04012010 tx=0x7f8a04005f30 comp rx=0 tx=0).stop 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 shutdown_connections 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5644371f11c0 0x56443768beb0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56443725d1b0 0x564437681e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56443723c4d0 0x564437622540 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 --2- 172.21.15.114:0/749759626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644372342f0 0x564437621f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 >> 172.21.15.114:0/749759626 conn(0x564437251300 msgr2=0x5644375ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.892+0000 7f8a00c00640 0 client.17502 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 shutdown_connections 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a24dd1740 1 -- 172.21.15.114:0/749759626 wait complete. 2024-09-21T07:03:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a24dd1740 20 client.17502 trim_cache size 0 max 16384 2024-09-21T07:03:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:03:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:03:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:03:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:03:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6700 msgr2=0x7f89fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6700 0x7f89fc0b8c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8a08105230 tx=0x7f8a0802a410 comp rx=0 tx=0).stop 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 msgr2=0x7f8a1819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.893+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a1819c690 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8a08005940 tx=0x7f8a080040c0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 shutdown_connections 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a18109d50 0x7f8a181a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89fc0b6700 0x7f89fc0b8c10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a18109400 0x7f8a1819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 --2- 172.21.15.114:0/3182248104 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a18103ad0 0x7f8a1819c690 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 >> 172.21.15.114:0/3182248104 conn(0x7f8a180ff510 msgr2=0x7f8a18108b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 shutdown_connections 2024-09-21T07:03:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:34.894+0000 7f8a1dc00640 1 -- 172.21.15.114:0/3182248104 wait complete. 2024-09-21T07:03:35.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:34 smithi193 ceph-mon[34601]: pgmap v685: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 5.8 MiB/s wr, 700 op/s 2024-09-21T07:03:35.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:34 smithi114 ceph-mon[30654]: pgmap v685: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 5.8 MiB/s wr, 700 op/s 2024-09-21T07:03:35.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:34 smithi183 ceph-mon[34219]: pgmap v685: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 5.8 MiB/s wr, 700 op/s 2024-09-21T07:03:37.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:36 smithi193 ceph-mon[34601]: pgmap v686: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-09-21T07:03:37.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:36 smithi114 ceph-mon[30654]: pgmap v686: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-09-21T07:03:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:36 smithi183 ceph-mon[34219]: pgmap v686: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.4 MiB/s wr, 503 op/s 2024-09-21T07:03:37.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 540.0 s, 61.4 tps, lat 213.020 ms stddev 204.671 2024-09-21T07:03:39.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:38 smithi193 ceph-mon[34601]: pgmap v687: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.6 MiB/s wr, 494 op/s 2024-09-21T07:03:39.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:38 smithi114 ceph-mon[30654]: pgmap v687: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.6 MiB/s wr, 494 op/s 2024-09-21T07:03:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:38 smithi183 ceph-mon[34219]: pgmap v687: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 4.6 MiB/s wr, 494 op/s 2024-09-21T07:03:41.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:40 smithi193 ceph-mon[34601]: pgmap v688: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-09-21T07:03:41.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:40 smithi114 ceph-mon[30654]: pgmap v688: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-09-21T07:03:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:40 smithi183 ceph-mon[34219]: pgmap v688: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 3.3 MiB/s wr, 367 op/s 2024-09-21T07:03:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 545.0 s, 296.3 tps, lat 639.585 ms stddev 1973.929 2024-09-21T07:03:43.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:42 smithi193 ceph-mon[34601]: pgmap v689: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-09-21T07:03:43.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:42 smithi114 ceph-mon[30654]: pgmap v689: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-09-21T07:03:43.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:42 smithi183 ceph-mon[34219]: pgmap v689: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-09-21T07:03:45.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:44 smithi114 ceph-mon[30654]: pgmap v690: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 2.6 MiB/s wr, 315 op/s 2024-09-21T07:03:45.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:44 smithi183 ceph-mon[34219]: pgmap v690: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 2.6 MiB/s wr, 315 op/s 2024-09-21T07:03:45.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:44 smithi193 ceph-mon[34601]: pgmap v690: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 2.6 MiB/s wr, 315 op/s 2024-09-21T07:03:47.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:46 smithi114 ceph-mon[30654]: pgmap v691: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-09-21T07:03:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:46 smithi183 ceph-mon[34219]: pgmap v691: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-09-21T07:03:47.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:46 smithi193 ceph-mon[34601]: pgmap v691: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-09-21T07:03:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 550.0 s, 510.2 tps, lat 194.875 ms stddev 244.323 2024-09-21T07:03:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.238+0000 7f8c59600640 1 -- 172.21.15.114:0/4092993512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410a0e0 msgr2=0x7f8c5410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.238+0000 7f8c59600640 1 --2- 172.21.15.114:0/4092993512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410a0e0 0x7f8c5410a4f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009860 tx=0x7f8c4002a340 comp rx=0 tx=0).stop 2024-09-21T07:03:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 -- 172.21.15.114:0/4092993512 shutdown_connections 2024-09-21T07:03:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 --2- 172.21.15.114:0/4092993512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c54110360 0x7f8c54116b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 --2- 172.21.15.114:0/4092993512 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c5410fa10 0x7f8c5410fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 --2- 172.21.15.114:0/4092993512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410a0e0 0x7f8c5410a4f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 -- 172.21.15.114:0/4092993512 >> 172.21.15.114:0/4092993512 conn(0x7f8c54079630 msgr2=0x7f8c541081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 -- 172.21.15.114:0/4092993512 shutdown_connections 2024-09-21T07:03:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.240+0000 7f8c59600640 1 -- 172.21.15.114:0/4092993512 wait complete. 2024-09-21T07:03:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.243+0000 7f8c59600640 1 Processor -- start 2024-09-21T07:03:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.243+0000 7f8c59600640 1 -- start start 2024-09-21T07:03:48.245 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c59600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 0x7f8c541a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c59600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410fa10 0x7f8c541a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c59600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c54110360 0x7f8c541a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c59600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54119fd0 con 0x7f8c54110360 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c59600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54119e50 con 0x7f8c5410fa10 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c59600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5411a150 con 0x7f8c5410a0e0 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.244+0000 7f8c52a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 0x7f8c541a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 0x7f8c541a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54114/0 (socket says 172.21.15.114:54114) 2024-09-21T07:03:48.246 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 -- 172.21.15.114:0/2453851678 learned_addr learned my addr 172.21.15.114:0/2453851678 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52000640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410fa10 0x7f8c541a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 -- 172.21.15.114:0/2453851678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410fa10 msgr2=0x7f8c541a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410fa10 0x7f8c541a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 -- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c54110360 msgr2=0x7f8c541a60e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c54110360 0x7f8c541a60e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 -- 172.21.15.114:0/2453851678 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c541a67e0 con 0x7f8c5410a0e0 2024-09-21T07:03:48.247 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.245+0000 7f8c52a00640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 0x7f8c541a08b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8c4002a930 tx=0x7f8c40005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.247+0000 7f8c4b400640 1 -- 172.21.15.114:0/2453851678 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40040020 con 0x7f8c5410a0e0 2024-09-21T07:03:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.247+0000 7f8c4b400640 1 -- 172.21.15.114:0/2453851678 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c40030450 con 0x7f8c5410a0e0 2024-09-21T07:03:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.247+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c541a6a70 con 0x7f8c5410a0e0 2024-09-21T07:03:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.247+0000 7f8c4b400640 1 -- 172.21.15.114:0/2453851678 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40037040 con 0x7f8c5410a0e0 2024-09-21T07:03:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.247+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541ad260 con 0x7f8c5410a0e0 2024-09-21T07:03:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.253+0000 7f8c4b400640 1 -- 172.21.15.114:0/2453851678 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8c40043020 con 0x7f8c5410a0e0 2024-09-21T07:03:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.261+0000 7f8c606ad740 1 Processor -- start 2024-09-21T07:03:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.261+0000 7f8c606ad740 1 -- start start 2024-09-21T07:03:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a99ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a99f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a99f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c606ad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55910a9146b0 con 0x55910a5fa1b0 2024-09-21T07:03:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c606ad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55910a903ae0 con 0x55910a5d94d0 2024-09-21T07:03:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c606ad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55910a9078f0 con 0x55910a5d12f0 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a99ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.262+0000 7f8c52000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a99f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a99f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41308/0 (socket says 172.21.15.114:41308) 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c53400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a99ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44326/0 (socket says 172.21.15.114:44326) 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 -- 172.21.15.114:0/4221531614 learned_addr learned my addr 172.21.15.114:0/4221531614 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52a00640 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a99f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c4b400640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c380b67d0 0x7f8c380b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 -- 172.21.15.114:0/4221531614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 msgr2=0x55910a99f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a99f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 -- 172.21.15.114:0/4221531614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 msgr2=0x55910a99ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a99ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.263+0000 7f8c52000640 1 -- 172.21.15.114:0/4221531614 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910a8d0780 con 0x55910a5d94d0 2024-09-21T07:03:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.264+0000 7f8c4b400640 1 -- 172.21.15.114:0/2453851678 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f8c40107790 con 0x7f8c5410a0e0 2024-09-21T07:03:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.265+0000 7f8c52000640 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a99f6b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c010400 tx=0x7f8c4c0108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3089b2727e4931b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.265+0000 7f8c3f400640 1 -- 172.21.15.114:0/4221531614 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c003b70 con 0x55910a5d94d0 2024-09-21T07:03:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.266+0000 7f8c3f400640 1 -- 172.21.15.114:0/4221531614 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c015070 con 0x55910a5d94d0 2024-09-21T07:03:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.266+0000 7f8c3f400640 1 -- 172.21.15.114:0/4221531614 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c011920 con 0x55910a5d94d0 2024-09-21T07:03:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.267+0000 7f8c606ad740 1 -- 172.21.15.114:0/4221531614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 msgr2=0x55910a99f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.267+0000 7f8c606ad740 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a99f6b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c010400 tx=0x7f8c4c0108c0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.271+0000 7f8c606ad740 1 -- 172.21.15.114:0/4221531614 shutdown_connections 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.271+0000 7f8c606ad740 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a99f6b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.271+0000 7f8c606ad740 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a99f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.271+0000 7f8c606ad740 1 --2- 172.21.15.114:0/4221531614 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a99ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.271+0000 7f8c606ad740 1 -- 172.21.15.114:0/4221531614 >> 172.21.15.114:0/4221531614 conn(0x55910a5ee300 msgr2=0x55910a99cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.271+0000 7f8c53400640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c380b67d0 0x7f8c380b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.272+0000 7f8c606ad740 1 -- 172.21.15.114:0/4221531614 shutdown_connections 2024-09-21T07:03:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.272+0000 7f8c606ad740 1 -- 172.21.15.114:0/4221531614 wait complete. 2024-09-21T07:03:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.273+0000 7f8c606ad740 1 -- start start 2024-09-21T07:03:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.273+0000 7f8c606ad740 1 Processor -- start 2024-09-21T07:03:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.274+0000 7f8c53400640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c380b67d0 0x7f8c380b8ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009510 tx=0x7f8c44004730 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.276+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.276+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.277+0000 7f8c52a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.277+0000 7f8c52a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54120/0 (socket says 172.21.15.114:54120) 2024-09-21T07:03:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.277+0000 7f8c52a00640 1 -- 172.21.15.114:0/136905075 learned_addr learned my addr 172.21.15.114:0/136905075 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.277+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a9aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.278+0000 7f8c53400640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a9aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.278+0000 7f8c52000640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.278+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55910a83bab0 con 0x55910a5fa1b0 2024-09-21T07:03:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.278+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55910a8aa570 con 0x55910a5d94d0 2024-09-21T07:03:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.278+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55910a9ad3b0 con 0x55910a5d12f0 2024-09-21T07:03:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.278+0000 7f8c606ad740 10 client.0 _ll_register_callbacks cb 0x55910a8b9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c52a00640 1 -- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 msgr2=0x55910a9aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c52a00640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c52a00640 1 -- 172.21.15.114:0/136905075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 msgr2=0x55910a9aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c52a00640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a9aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c52a00640 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910a665790 con 0x55910a5d12f0 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c015070 con 0x55910a5d12f0 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c52a00640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aa220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c40004650 tx=0x7f8c40009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26bd4c2810e700d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c33e00640 10 client.27901 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:03:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.279+0000 7f8c33e00640 1 -- 172.21.15.114:0/136905075 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400cca00 con 0x55910a5d12f0 2024-09-21T07:03:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.280+0000 7f8c33e00640 1 -- 172.21.15.114:0/136905075 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f8c40107360 con 0x55910a5d12f0 2024-09-21T07:03:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.280+0000 7f8c33e00640 1 -- 172.21.15.114:0/136905075 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8c40107560 con 0x55910a5d12f0 2024-09-21T07:03:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.280+0000 7f8c606ad740 10 client.27901 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.280+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=49}) -- 0x55910a6de620 con 0x55910a5d12f0 2024-09-21T07:03:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c33e00640 1 -- 172.21.15.114:0/136905075 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f8c400f0b90 con 0x55910a5d12f0 2024-09-21T07:03:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c606ad740 10 client.27901 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c606ad740 10 client.27901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c606ad740 20 client.27901 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:48.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5b8d80 0x55910a9bc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c606ad740 4 client.27901 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.281+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55910a916600 con 0x55910a5b8d80 2024-09-21T07:03:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.282+0000 7f8c52000640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5b8d80 0x55910a9bc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.284+0000 7f8c52000640 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5b8d80 0x55910a9bc270 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ff9b44265a6eca server_cookie=3b855cc5f306e396 in_seq=0 out_seq=0 2024-09-21T07:03:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.284+0000 7f8c33e00640 10 client.27901 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.285+0000 7f8c33e00640 1 -- 172.21.15.114:0/136905075 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55910a916600 con 0x55910a5b8d80 2024-09-21T07:03:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.285+0000 7f8c33e00640 10 client.27901 handle_command_reply: tid=0 2024-09-21T07:03:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.285+0000 7f8c606ad740 1 client.27901 shutdown 2024-09-21T07:03:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.286+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 msgr2=0x55910a9aa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.286+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aa220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c40004650 tx=0x7f8c40009580 comp rx=0 tx=0).stop 2024-09-21T07:03:48.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 shutdown_connections 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5fa1b0 0x55910a9aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5b8d80 0x55910a9bc270 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c33e00640 0 client.27901 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 --2- 172.21.15.114:0/136905075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aa220 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 >> 172.21.15.114:0/136905075 conn(0x55910a5ee300 msgr2=0x55910a9a1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:48.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.287+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 shutdown_connections 2024-09-21T07:03:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.288+0000 7f8c606ad740 1 -- 172.21.15.114:0/136905075 wait complete. 2024-09-21T07:03:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.288+0000 7f8c606ad740 20 client.27901 trim_cache size 0 max 16384 2024-09-21T07:03:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.323+0000 7f8c606ad740 1 Processor -- start 2024-09-21T07:03:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.323+0000 7f8c606ad740 1 -- start start 2024-09-21T07:03:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.324+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a99b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.324+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d12f0 0x55910a9aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.324+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 0x55910a9aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c606ad740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b74a0 con 0x55910a5b8d80 2024-09-21T07:03:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c606ad740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b8fe0 con 0x55910a5d12f0 2024-09-21T07:03:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c606ad740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b7b70 con 0x55910a5d94d0 2024-09-21T07:03:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c53400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 0x55910a9aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c53400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 0x55910a9aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54128/0 (socket says 172.21.15.114:54128) 2024-09-21T07:03:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c53400640 1 -- 172.21.15.114:0/3708780888 learned_addr learned my addr 172.21.15.114:0/3708780888 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.325+0000 7f8c52000640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d12f0 0x55910a9aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52a00640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a99b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52000640 1 -- 172.21.15.114:0/3708780888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 msgr2=0x55910a9aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52000640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 0x55910a9aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52000640 1 -- 172.21.15.114:0/3708780888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 msgr2=0x55910a99b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52000640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a99b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52000640 1 -- 172.21.15.114:0/3708780888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910a9a1a40 con 0x55910a5d12f0 2024-09-21T07:03:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c53400640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 0x55910a9aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.326+0000 7f8c52a00640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a99b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:03:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.327+0000 7f8c52000640 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d12f0 0x55910a9aa720 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c004da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ab4c9e8db9961e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.328+0000 7f8c3ea00640 1 -- 172.21.15.114:0/3708780888 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c003b70 con 0x55910a5d12f0 2024-09-21T07:03:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.328+0000 7f8c3ea00640 1 -- 172.21.15.114:0/3708780888 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c003d10 con 0x55910a5d12f0 2024-09-21T07:03:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.328+0000 7f8c3ea00640 1 -- 172.21.15.114:0/3708780888 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c011930 con 0x55910a5d12f0 2024-09-21T07:03:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.329+0000 7f8c606ad740 1 -- 172.21.15.114:0/3708780888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d12f0 msgr2=0x55910a9aa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.329+0000 7f8c606ad740 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d12f0 0x55910a9aa720 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006010 tx=0x7f8c4c004da0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 -- 172.21.15.114:0/3708780888 shutdown_connections 2024-09-21T07:03:48.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d94d0 0x55910a9aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d12f0 0x55910a9aa720 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 --2- 172.21.15.114:0/3708780888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a99b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 -- 172.21.15.114:0/3708780888 >> 172.21.15.114:0/3708780888 conn(0x55910a5ee300 msgr2=0x55910a99b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 -- 172.21.15.114:0/3708780888 shutdown_connections 2024-09-21T07:03:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.330+0000 7f8c606ad740 1 -- 172.21.15.114:0/3708780888 wait complete. 2024-09-21T07:03:48.332 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.331+0000 7f8c606ad740 1 -- start start 2024-09-21T07:03:48.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.331+0000 7f8c606ad740 1 Processor -- start 2024-09-21T07:03:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.332+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a9ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.332+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.332+0000 7f8c606ad740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c52000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c52000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54136/0 (socket says 172.21.15.114:54136) 2024-09-21T07:03:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c52000640 1 -- 172.21.15.114:0/1633714459 learned_addr learned my addr 172.21.15.114:0/1633714459 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c53400640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b8910 con 0x55910a5b8d80 2024-09-21T07:03:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b8240 con 0x55910a5d94d0 2024-09-21T07:03:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b54a0 con 0x55910a5d12f0 2024-09-21T07:03:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.333+0000 7f8c606ad740 10 client.0 _ll_register_callbacks cb 0x7f8c40005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c53400640 1 -- 172.21.15.114:0/1633714459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 msgr2=0x55910a9aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c53400640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c53400640 1 -- 172.21.15.114:0/1633714459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 msgr2=0x55910a9ae640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:03:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c52a00640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a9ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c53400640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a9ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c53400640 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55910aa25cf0 con 0x55910a5d94d0 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.334+0000 7f8c52a00640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a9ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.336+0000 7f8c53400640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9af0c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009600 tx=0x7f8c44014070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37afb062000bdebf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.336+0000 7f8c33e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.336+0000 7f8c33e00640 1 -- 172.21.15.114:0/1633714459 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4401d620 con 0x55910a5d94d0 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.336+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c003d10 con 0x55910a5d94d0 2024-09-21T07:03:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.336+0000 7f8c33e00640 1 -- 172.21.15.114:0/1633714459 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f8c4401d7c0 con 0x55910a5d94d0 2024-09-21T07:03:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.337+0000 7f8c33e00640 1 -- 172.21.15.114:0/1633714459 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8c44035050 con 0x55910a5d94d0 2024-09-21T07:03:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.337+0000 7f8c606ad740 10 client.27959 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.337+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55910aa26fa0 con 0x55910a5d94d0 2024-09-21T07:03:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.337+0000 7f8c33e00640 1 -- 172.21.15.114:0/1633714459 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c003d10 con 0x55910a5d94d0 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c33e00640 1 -- 172.21.15.114:0/1633714459 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f8c4401aa20 con 0x55910a5d94d0 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c606ad740 10 client.27959 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c606ad740 10 client.27959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c606ad740 20 client.27959 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c606ad740 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5fa1b0 0x55910aa29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c606ad740 4 client.27959 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55910a9120b0 con 0x55910a5fa1b0 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.338+0000 7f8c52a00640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5fa1b0 0x55910aa29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.339+0000 7f8c33e00640 10 client.27959 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.339+0000 7f8c52a00640 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5fa1b0 0x55910aa29760 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58272dca87c33d0e server_cookie=70c9d9f364e256d3 in_seq=0 out_seq=0 2024-09-21T07:03:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.419+0000 7f8c33e00640 1 -- 172.21.15.114:0/1633714459 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55910a9120b0 con 0x55910a5fa1b0 2024-09-21T07:03:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.419+0000 7f8c33e00640 10 client.27959 handle_command_reply: tid=0 2024-09-21T07:03:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.419+0000 7f8c606ad740 1 client.27959 shutdown 2024-09-21T07:03:48.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 msgr2=0x55910a9af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9af0c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009600 tx=0x7f8c44014070 comp rx=0 tx=0).stop 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 shutdown_connections 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55910a5d94d0 0x55910a9af0c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55910a5fa1b0 0x55910aa29760 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55910a5d12f0 0x55910a9aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 --2- 172.21.15.114:0/1633714459 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55910a5b8d80 0x55910a9ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c33e00640 0 client.27959 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.421+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 >> 172.21.15.114:0/1633714459 conn(0x55910a5ee300 msgr2=0x55910a99c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.422+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 shutdown_connections 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.422+0000 7f8c606ad740 1 -- 172.21.15.114:0/1633714459 wait complete. 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.422+0000 7f8c606ad740 20 client.27959 trim_cache size 0 max 16384 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:03:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:03:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.423+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c380b67d0 msgr2=0x7f8c380b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.423+0000 7f8c59600640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c380b67d0 0x7f8c380b8ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009510 tx=0x7f8c44004730 comp rx=0 tx=0).stop 2024-09-21T07:03:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.423+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 msgr2=0x7f8c541a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.423+0000 7f8c59600640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 0x7f8c541a08b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8c4002a930 tx=0x7f8c40005dc0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 shutdown_connections 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c54110360 0x7f8c541a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c380b67d0 0x7f8c380b8ce0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c5410fa10 0x7f8c541a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 --2- 172.21.15.114:0/2453851678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c5410a0e0 0x7f8c541a08b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 >> 172.21.15.114:0/2453851678 conn(0x7f8c54079630 msgr2=0x7f8c54110a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 shutdown_connections 2024-09-21T07:03:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:03:48.424+0000 7f8c59600640 1 -- 172.21.15.114:0/2453851678 wait complete. 2024-09-21T07:03:49.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:48 smithi114 ceph-mon[30654]: pgmap v692: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-09-21T07:03:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:48 smithi183 ceph-mon[34219]: pgmap v692: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-09-21T07:03:49.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:48 smithi193 ceph-mon[34601]: pgmap v692: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-09-21T07:03:51.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:50 smithi114 ceph-mon[30654]: pgmap v693: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-21T07:03:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:50 smithi183 ceph-mon[34219]: pgmap v693: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-21T07:03:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:50 smithi193 ceph-mon[34601]: pgmap v693: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-21T07:03:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 555.0 s, 464.0 tps, lat 205.288 ms stddev 230.577 2024-09-21T07:03:53.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:52 smithi114 ceph-mon[30654]: pgmap v694: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-09-21T07:03:53.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:52 smithi183 ceph-mon[34219]: pgmap v694: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-09-21T07:03:53.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:52 smithi193 ceph-mon[34601]: pgmap v694: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-09-21T07:03:55.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:54 smithi114 ceph-mon[30654]: pgmap v695: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-09-21T07:03:55.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:54 smithi183 ceph-mon[34219]: pgmap v695: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-09-21T07:03:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:54 smithi193 ceph-mon[34601]: pgmap v695: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 3.8 MiB/s wr, 426 op/s 2024-09-21T07:03:56.875 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:03:57.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:56 smithi114 ceph-mon[30654]: pgmap v696: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-21T07:03:57.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:56 smithi183 ceph-mon[34219]: pgmap v696: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-21T07:03:57.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:56 smithi193 ceph-mon[34601]: pgmap v696: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-09-21T07:03:57.666 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:03:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 560.0 s, 502.2 tps, lat 205.630 ms stddev 233.432 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 -- 172.21.15.114:0/856048565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd104105300 msgr2=0x7fd10410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 --2- 172.21.15.114:0/856048565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd104105300 0x7fd10410d850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009900 tx=0x7fd0f002a410 comp rx=0 tx=0).stop 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 -- 172.21.15.114:0/856048565 shutdown_connections 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 --2- 172.21.15.114:0/856048565 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 0x7fd104110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 --2- 172.21.15.114:0/856048565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd104105300 0x7fd10410d850 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 --2- 172.21.15.114:0/856048565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1041049b0 0x7fd104104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 -- 172.21.15.114:0/856048565 >> 172.21.15.114:0/856048565 conn(0x7fd104079730 msgr2=0x7fd104079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 -- 172.21.15.114:0/856048565 shutdown_connections 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.694+0000 7fd10ae00640 1 -- 172.21.15.114:0/856048565 wait complete. 2024-09-21T07:03:58.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.696+0000 7fd10ae00640 1 Processor -- start 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 -- start start 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 0x7fd10419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104105300 0x7fd10419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 0x7fd1041a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd104113b20 con 0x7fd10410de10 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041139a0 con 0x7fd104105300 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.697+0000 7fd10ae00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd104113ca0 con 0x7fd1041049b0 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.698+0000 7fd103e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 0x7fd10419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.698+0000 7fd103e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 0x7fd10419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44298/0 (socket says 172.21.15.114:44298) 2024-09-21T07:03:58.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.698+0000 7fd103e00640 1 -- 172.21.15.114:0/1584906083 learned_addr learned my addr 172.21.15.114:0/1584906083 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:58.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.699+0000 7fd108e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 0x7fd1041a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.699+0000 7fd103400640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104105300 0x7fd10419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.701+0000 7fd103e00640 1 -- 172.21.15.114:0/1584906083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104105300 msgr2=0x7fd10419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.701+0000 7fd103e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104105300 0x7fd10419cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.701+0000 7fd103e00640 1 -- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 msgr2=0x7fd1041a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.701+0000 7fd103e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 0x7fd1041a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.701+0000 7fd103e00640 1 -- 172.21.15.114:0/1584906083 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041a24e0 con 0x7fd1041049b0 2024-09-21T07:03:58.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.701+0000 7fd108e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 0x7fd1041a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:03:58.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.703+0000 7fd103e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 0x7fd10419c570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400b450 tx=0x7fd0f400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:58.705 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.703+0000 7fd100c00640 1 -- 172.21.15.114:0/1584906083 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4015020 con 0x7fd1041049b0 2024-09-21T07:03:58.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.704+0000 7fd100c00640 1 -- 172.21.15.114:0/1584906083 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f4011d40 con 0x7fd1041049b0 2024-09-21T07:03:58.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.708+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041a27d0 con 0x7fd1041049b0 2024-09-21T07:03:58.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.708+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041057f0 con 0x7fd1041049b0 2024-09-21T07:03:58.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.708+0000 7fd100c00640 1 -- 172.21.15.114:0/1584906083 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400f340 con 0x7fd1041049b0 2024-09-21T07:03:58.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.712+0000 7fd111be0740 1 Processor -- start 2024-09-21T07:03:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.714+0000 7fd111be0740 1 -- start start 2024-09-21T07:03:58.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.714+0000 7fd100c00640 1 -- 172.21.15.114:0/1584906083 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd0f4018020 con 0x7fd1041049b0 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.714+0000 7fd111be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 0x562264debb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.714+0000 7fd111be0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a1c1b0 0x562264dec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.714+0000 7fd111be0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a24390 0x562264dec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.714+0000 7fd111be0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562264d62f70 con 0x562264a24390 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd111be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562264d523a0 con 0x562264a45070 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd111be0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562264d561b0 con 0x562264a1c1b0 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 0x562264debb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 0x562264debb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39900/0 (socket says 172.21.15.114:39900) 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 -- 172.21.15.114:0/2929275791 learned_addr learned my addr 172.21.15.114:0/2929275791 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd103e00640 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a1c1b0 0x562264dec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 -- 172.21.15.114:0/2929275791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a1c1b0 msgr2=0x562264dec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a1c1b0 0x562264dec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 -- 172.21.15.114:0/2929275791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a24390 msgr2=0x562264dec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a24390 0x562264dec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.715+0000 7fd108e00640 1 -- 172.21.15.114:0/2929275791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562264d1cc90 con 0x562264a45070 2024-09-21T07:03:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.718+0000 7fd108e00640 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 0x562264debb60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc010750 tx=0x7fd0fc010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ad3f32336e5ab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:58.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.718+0000 7fd0f9600640 1 -- 172.21.15.114:0/2929275791 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc00ead0 con 0x562264a45070 2024-09-21T07:03:58.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.719+0000 7fd0f9600640 1 -- 172.21.15.114:0/2929275791 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0fc010e60 con 0x562264a45070 2024-09-21T07:03:58.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.719+0000 7fd0f9600640 1 -- 172.21.15.114:0/2929275791 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc011640 con 0x562264a45070 2024-09-21T07:03:58.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.723+0000 7fd111be0740 1 -- 172.21.15.114:0/2929275791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 msgr2=0x562264debb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.723+0000 7fd111be0740 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 0x562264debb60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc010750 tx=0x7fd0fc010c10 comp rx=0 tx=0).stop 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.725+0000 7fd100c00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e80b6780 0x7fd0e80b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.727+0000 7fd111be0740 1 -- 172.21.15.114:0/2929275791 shutdown_connections 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.727+0000 7fd111be0740 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a24390 0x562264dec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.727+0000 7fd111be0740 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a1c1b0 0x562264dec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.727+0000 7fd111be0740 1 --2- 172.21.15.114:0/2929275791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a45070 0x562264debb60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.727+0000 7fd111be0740 1 -- 172.21.15.114:0/2929275791 >> 172.21.15.114:0/2929275791 conn(0x562264a39460 msgr2=0x562264de9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.730+0000 7fd108e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e80b6780 0x7fd0e80b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.730+0000 7fd100c00640 1 -- 172.21.15.114:0/1584906083 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fd0f40e71e0 con 0x7fd1041049b0 2024-09-21T07:03:58.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.730+0000 7fd111be0740 1 -- 172.21.15.114:0/2929275791 shutdown_connections 2024-09-21T07:03:58.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.732+0000 7fd111be0740 1 -- 172.21.15.114:0/2929275791 wait complete. 2024-09-21T07:03:58.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.733+0000 7fd111be0740 1 -- start start 2024-09-21T07:03:58.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.733+0000 7fd111be0740 1 Processor -- start 2024-09-21T07:03:58.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.735+0000 7fd108e00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e80b6780 0x7fd0e80b8c90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009840 tx=0x7fd0fc012c60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:58.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.735+0000 7fd111be0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a1c1b0 0x562264df76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd111be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 0x562264df7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd111be0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a45070 0x562264e146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd111be0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562264bb3700 con 0x562264a1c1b0 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd111be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562264ced540 con 0x562264a24390 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd111be0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562264e0b350 con 0x562264a45070 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd111be0740 10 client.0 _ll_register_callbacks cb 0x562264cfd430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 0x562264df7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.738 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd108e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a45070 0x562264e146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 0x562264df7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39916/0 (socket says 172.21.15.114:39916) 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 -- 172.21.15.114:0/410416502 learned_addr learned my addr 172.21.15.114:0/410416502 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 -- 172.21.15.114:0/410416502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a45070 msgr2=0x562264e146d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a45070 0x562264e146d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 -- 172.21.15.114:0/410416502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a1c1b0 msgr2=0x562264df76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a1c1b0 0x562264df76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.736+0000 7fd103400640 1 -- 172.21.15.114:0/410416502 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562264ab0650 con 0x562264a24390 2024-09-21T07:03:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.740+0000 7fd103400640 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 0x562264df7c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009900 tx=0x7fd0f000b1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5eb4ba7ddbe52e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.740+0000 7fd0ee000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:03:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.740+0000 7fd0ee000640 1 -- 172.21.15.114:0/410416502 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0035040 con 0x562264a24390 2024-09-21T07:03:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.740+0000 7fd0ee000640 1 -- 172.21.15.114:0/410416502 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fd0f0004030 con 0x562264a24390 2024-09-21T07:03:58.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.741+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562264cefb50 con 0x562264a24390 2024-09-21T07:03:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.741+0000 7fd0ee000640 1 -- 172.21.15.114:0/410416502 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd0f004d050 con 0x562264a24390 2024-09-21T07:03:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.741+0000 7fd111be0740 10 client.27968 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:58.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.741+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562264d0a910 con 0x562264a24390 2024-09-21T07:03:58.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd0ee000640 1 -- 172.21.15.114:0/410416502 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562264cefb50 con 0x562264a24390 2024-09-21T07:03:58.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd0ee000640 1 -- 172.21.15.114:0/410416502 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fd0f003c440 con 0x562264a24390 2024-09-21T07:03:58.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd111be0740 10 client.27968 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd111be0740 10 client.27968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd111be0740 20 client.27968 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd111be0740 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562264a039f0 0x562264e1abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd111be0740 4 client.27968 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:03:58.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.744+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562264d64ec0 con 0x562264a039f0 2024-09-21T07:03:58.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.746+0000 7fd103e00640 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562264a039f0 0x562264e1abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.751+0000 7fd103e00640 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562264a039f0 0x562264e1abe0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78048af1846b81a9 server_cookie=2d9bc74de60538a1 in_seq=0 out_seq=0 2024-09-21T07:03:58.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.751+0000 7fd0ee000640 10 client.27968 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:58.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.753+0000 7fd0ee000640 1 -- 172.21.15.114:0/410416502 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562264d64ec0 con 0x562264a039f0 2024-09-21T07:03:58.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.753+0000 7fd0ee000640 10 client.27968 handle_command_reply: tid=0 2024-09-21T07:03:58.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.753+0000 7fd111be0740 1 client.27968 shutdown 2024-09-21T07:03:58.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 msgr2=0x562264df7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 0x562264df7c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009900 tx=0x7fd0f000b1f0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 shutdown_connections 2024-09-21T07:03:58.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562264a039f0 0x562264e1abe0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a45070 0x562264e146d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a24390 0x562264df7c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 --2- 172.21.15.114:0/410416502 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a1c1b0 0x562264df76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 >> 172.21.15.114:0/410416502 conn(0x562264a39460 msgr2=0x562264deecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:58.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.754+0000 7fd0ee000640 0 client.27968 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:58.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.756+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 shutdown_connections 2024-09-21T07:03:58.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.756+0000 7fd111be0740 1 -- 172.21.15.114:0/410416502 wait complete. 2024-09-21T07:03:58.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.756+0000 7fd111be0740 20 client.27968 trim_cache size 0 max 16384 2024-09-21T07:03:58.802 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.801+0000 7fd111be0740 1 Processor -- start 2024-09-21T07:03:58.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 -- start start 2024-09-21T07:03:58.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264de9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264de9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264df46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e4010040 con 0x562264a45070 2024-09-21T07:03:58.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e4011b80 con 0x562264a1c1b0 2024-09-21T07:03:58.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.802+0000 7fd111be0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e4010710 con 0x562264a24390 2024-09-21T07:03:58.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.803+0000 7fd103e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264de9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.803+0000 7fd103e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264de9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39922/0 (socket says 172.21.15.114:39922) 2024-09-21T07:03:58.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.803+0000 7fd103e00640 1 -- 172.21.15.114:0/3113511761 learned_addr learned my addr 172.21.15.114:0/3113511761 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:58.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.803+0000 7fd103400640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264de9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.803+0000 7fd108e00640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264df46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.804+0000 7fd103400640 1 -- 172.21.15.114:0/3113511761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 msgr2=0x562264de9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.805+0000 7fd103400640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264de9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.805+0000 7fd103400640 1 -- 172.21.15.114:0/3113511761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 msgr2=0x562264df46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.805+0000 7fd103400640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264df46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.805+0000 7fd103400640 1 -- 172.21.15.114:0/3113511761 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562264df8790 con 0x562264a24390 2024-09-21T07:03:58.807 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.805+0000 7fd103e00640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264de9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:58.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.806+0000 7fd108e00640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264df46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:03:58.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.806+0000 7fd103400640 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264de9540 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0f003c7e0 tx=0x7fd0f0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2aa39b4a0bebfb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:58.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.808+0000 7fd0f8c00640 1 -- 172.21.15.114:0/3113511761 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0056070 con 0x562264a24390 2024-09-21T07:03:58.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.808+0000 7fd0f8c00640 1 -- 172.21.15.114:0/3113511761 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0005520 con 0x562264a24390 2024-09-21T07:03:58.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.809+0000 7fd0f8c00640 1 -- 172.21.15.114:0/3113511761 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0035040 con 0x562264a24390 2024-09-21T07:03:58.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 -- 172.21.15.114:0/3113511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 msgr2=0x562264de9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264de9540 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0f003c7e0 tx=0x7fd0f0005fd0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 -- 172.21.15.114:0/3113511761 shutdown_connections 2024-09-21T07:03:58.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264df46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264de9540 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 --2- 172.21.15.114:0/3113511761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264de9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.811+0000 7fd111be0740 1 -- 172.21.15.114:0/3113511761 >> 172.21.15.114:0/3113511761 conn(0x562264a39460 msgr2=0x562264df5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:58.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.812+0000 7fd111be0740 1 -- 172.21.15.114:0/3113511761 shutdown_connections 2024-09-21T07:03:58.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.812+0000 7fd111be0740 1 -- 172.21.15.114:0/3113511761 wait complete. 2024-09-21T07:03:58.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.813+0000 7fd111be0740 1 -- start start 2024-09-21T07:03:58.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.813+0000 7fd111be0740 1 Processor -- start 2024-09-21T07:03:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264e0ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264e0d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264e845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40114b0 con 0x562264a45070 2024-09-21T07:03:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e4010de0 con 0x562264a1c1b0 2024-09-21T07:03:58.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e400e040 con 0x562264a24390 2024-09-21T07:03:58.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.814+0000 7fd111be0740 10 client.0 _ll_register_callbacks cb 0x7fd0f00326e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:03:58.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.816+0000 7fd108e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264e845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.817+0000 7fd108e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264e845a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40274/0 (socket says 172.21.15.114:40274) 2024-09-21T07:03:58.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.817+0000 7fd108e00640 1 -- 172.21.15.114:0/512701284 learned_addr learned my addr 172.21.15.114:0/512701284 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:03:58.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.817+0000 7fd103e00640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264e0ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.817+0000 7fd103400640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264e0d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.818+0000 7fd108e00640 1 -- 172.21.15.114:0/512701284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 msgr2=0x562264e0d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.818+0000 7fd108e00640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264e0d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.818+0000 7fd108e00640 1 -- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 msgr2=0x562264e0ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:03:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.818+0000 7fd108e00640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264e0ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.818+0000 7fd108e00640 1 -- 172.21.15.114:0/512701284 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562264e95bc0 con 0x562264a45070 2024-09-21T07:03:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.819+0000 7fd108e00640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264e845a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc010750 tx=0x7fd0fc009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90e3ff9a8deebd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:03:58.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.819+0000 7fd0ee000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:03:58.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.819+0000 7fd0ee000640 1 -- 172.21.15.114:0/512701284 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc039060 con 0x562264a45070 2024-09-21T07:03:58.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.819+0000 7fd0ee000640 1 -- 172.21.15.114:0/512701284 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fd0fc0113e0 con 0x562264a45070 2024-09-21T07:03:58.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.820+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f0005520 con 0x562264a45070 2024-09-21T07:03:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.820+0000 7fd0ee000640 1 -- 172.21.15.114:0/512701284 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd0fc012450 con 0x562264a45070 2024-09-21T07:03:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.820+0000 7fd111be0740 10 client.17523 fetch_fsmap learned FSMap version 49 2024-09-21T07:03:58.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.820+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562264e96e50 con 0x562264a45070 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd0ee000640 1 -- 172.21.15.114:0/512701284 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0005520 con 0x562264a45070 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd0ee000640 1 -- 172.21.15.114:0/512701284 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fd0fc00f040 con 0x562264a45070 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd111be0740 10 client.17523 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd111be0740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd111be0740 20 client.17523 populate_metadata read hostname 'smithi114' 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd111be0740 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622649d8da0 0x562264e99300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd111be0740 4 client.17523 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:03:58.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.821+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562264d60970 con 0x5622649d8da0 2024-09-21T07:03:58.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.823+0000 7fd103e00640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622649d8da0 0x562264e99300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:03:58.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.824+0000 7fd103e00640 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622649d8da0 0x562264e99300 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a480382f29f76aa server_cookie=4ee44dbda1c7c8cc in_seq=0 out_seq=0 2024-09-21T07:03:58.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.824+0000 7fd0ee000640 10 client.17523 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:58.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.824+0000 7fd0ee000640 1 -- 172.21.15.114:0/512701284 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x562264d60970 con 0x5622649d8da0 2024-09-21T07:03:58.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.824+0000 7fd0ee000640 10 client.17523 handle_command_reply: tid=0 2024-09-21T07:03:58.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.824+0000 7fd111be0740 1 client.17523 shutdown 2024-09-21T07:03:58.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 msgr2=0x562264e845a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264e845a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc010750 tx=0x7fd0fc009870 comp rx=0 tx=0).stop 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 shutdown_connections 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562264a45070 0x562264e845a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622649d8da0 0x562264e99300 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562264a24390 0x562264e0d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 --2- 172.21.15.114:0/512701284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562264a1c1b0 0x562264e0ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 >> 172.21.15.114:0/512701284 conn(0x562264a39460 msgr2=0x562264df5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd0ee000640 0 client.17523 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:03:58.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.825+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 shutdown_connections 2024-09-21T07:03:58.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.826+0000 7fd111be0740 1 -- 172.21.15.114:0/512701284 wait complete. 2024-09-21T07:03:58.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.826+0000 7fd111be0740 20 client.17523 trim_cache size 0 max 16384 2024-09-21T07:03:58.828 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:03:58.828 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.829 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:03:58.830 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.831 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.832 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.833 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:03:58.834 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.828+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e80b6780 msgr2=0x7fd0e80b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.828+0000 7fd10ae00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e80b6780 0x7fd0e80b8c90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc009840 tx=0x7fd0fc012c60 comp rx=0 tx=0).stop 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.828+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 msgr2=0x7fd10419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.828+0000 7fd10ae00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 0x7fd10419c570 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400b450 tx=0x7fd0f400b910 comp rx=0 tx=0).stop 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 shutdown_connections 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd10410de10 0x7fd1041a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd0e80b6780 0x7fd0e80b8c90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd104105300 0x7fd10419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 --2- 172.21.15.114:0/1584906083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1041049b0 0x7fd10419c570 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:03:58.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 >> 172.21.15.114:0/1584906083 conn(0x7fd104079730 msgr2=0x7fd10410caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:03:58.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 shutdown_connections 2024-09-21T07:03:58.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:03:58.829+0000 7fd10ae00640 1 -- 172.21.15.114:0/1584906083 wait complete. 2024-09-21T07:03:59.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:03:58 smithi114 ceph-mon[30654]: pgmap v697: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 518 op/s 2024-09-21T07:03:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:03:58 smithi183 ceph-mon[34219]: pgmap v697: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 518 op/s 2024-09-21T07:03:59.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:03:58 smithi193 ceph-mon[34601]: pgmap v697: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.0 MiB/s wr, 518 op/s 2024-09-21T07:04:00.553 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:04:00.553 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:04:01.192 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:01.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:00 smithi114 ceph-mon[30654]: pgmap v698: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-21T07:04:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:00 smithi183 ceph-mon[34219]: pgmap v698: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-21T07:04:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:00 smithi193 ceph-mon[34601]: pgmap v698: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.4 MiB/s wr, 477 op/s 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.700+0000 7f6876000640 1 -- 172.21.15.114:0/2898630902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6870111ca0 msgr2=0x7f687010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.700+0000 7f6876000640 1 --2- 172.21.15.114:0/2898630902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6870111ca0 0x7f687010a3c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f686401a910 tx=0x7f686403b420 comp rx=0 tx=0).stop 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 -- 172.21.15.114:0/2898630902 shutdown_connections 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 --2- 172.21.15.114:0/2898630902 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f687011cba0 0x7f687011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 --2- 172.21.15.114:0/2898630902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f687010a900 0x7f687010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 --2- 172.21.15.114:0/2898630902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6870111ca0 0x7f687010a3c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 -- 172.21.15.114:0/2898630902 >> 172.21.15.114:0/2898630902 conn(0x7f6870070c00 msgr2=0x7f6870071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 -- 172.21.15.114:0/2898630902 shutdown_connections 2024-09-21T07:04:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.703+0000 7f6876000640 1 -- 172.21.15.114:0/2898630902 wait complete. 2024-09-21T07:04:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 Processor -- start 2024-09-21T07:04:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 -- start start 2024-09-21T07:04:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f687010a900 0x7f687011c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 0x7f68701188a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687011cba0 0x7f6870118fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6870114c70 con 0x7f687010a900 2024-09-21T07:04:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6870114af0 con 0x7f687011cba0 2024-09-21T07:04:01.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.705+0000 7f6876000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6870114df0 con 0x7f6870111ca0 2024-09-21T07:04:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.706+0000 7f6874c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f687010a900 0x7f687011c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.706+0000 7f686fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 0x7f68701188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.706+0000 7f686fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 0x7f68701188a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44354/0 (socket says 172.21.15.114:44354) 2024-09-21T07:04:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.706+0000 7f686fe00640 1 -- 172.21.15.114:0/111057712 learned_addr learned my addr 172.21.15.114:0/111057712 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.708+0000 7f6875600640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687011cba0 0x7f6870118fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.708+0000 7f686fe00640 1 -- 172.21.15.114:0/111057712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687011cba0 msgr2=0x7f6870118fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.708+0000 7f686fe00640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687011cba0 0x7f6870118fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.708+0000 7f686fe00640 1 -- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f687010a900 msgr2=0x7f687011c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.708+0000 7f686fe00640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f687010a900 0x7f687011c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.708+0000 7f686fe00640 1 -- 172.21.15.114:0/111057712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68701acb60 con 0x7f6870111ca0 2024-09-21T07:04:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.709+0000 7f686fe00640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 0x7f68701188a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e9a0 tx=0x7f685c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.709+0000 7f686d600640 1 -- 172.21.15.114:0/111057712 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c00cd80 con 0x7f6870111ca0 2024-09-21T07:04:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.709+0000 7f686d600640 1 -- 172.21.15.114:0/111057712 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685c0035f0 con 0x7f6870111ca0 2024-09-21T07:04:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.709+0000 7f686d600640 1 -- 172.21.15.114:0/111057712 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c010690 con 0x7f6870111ca0 2024-09-21T07:04:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.709+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68701acd90 con 0x7f6870111ca0 2024-09-21T07:04:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.710+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68701ad1a0 con 0x7f6870111ca0 2024-09-21T07:04:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.718+0000 7f687e66e740 1 Processor -- start 2024-09-21T07:04:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 -- start start 2024-09-21T07:04:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523ed2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ed3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c523e486b0 con 0x55c523b2e1b0 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c523e37ae0 con 0x55c523b052f0 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.721+0000 7f687e66e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c523e3b8f0 con 0x55c523b0d4d0 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.722+0000 7f686d600640 1 -- 172.21.15.114:0/111057712 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f685c004000 con 0x7f6870111ca0 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.723+0000 7f6874c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ed3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.723+0000 7f6874c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ed3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39952/0 (socket says 172.21.15.114:39952) 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.723+0000 7f6874c00640 1 -- 172.21.15.114:0/2257954223 learned_addr learned my addr 172.21.15.114:0/2257954223 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.723+0000 7f686fe00640 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.724+0000 7f6874c00640 1 -- 172.21.15.114:0/2257954223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 msgr2=0x55c523ed3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.724+0000 7f6874c00640 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.724+0000 7f6874c00640 1 -- 172.21.15.114:0/2257954223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 msgr2=0x55c523ed2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.724+0000 7f6874c00640 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523ed2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.724+0000 7f6874c00640 1 -- 172.21.15.114:0/2257954223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c523e04780 con 0x55c523b052f0 2024-09-21T07:04:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.724+0000 7f686d600640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68580b6700 0x7f68580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.725+0000 7f686d600640 1 -- 172.21.15.114:0/111057712 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f685c0e7d80 con 0x7f6870111ca0 2024-09-21T07:04:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.725+0000 7f6874c00640 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ed3350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68700755c0 tx=0x7f6864004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c958b9379a704f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.725+0000 7f6861c00640 1 -- 172.21.15.114:0/2257954223 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864046030 con 0x55c523b052f0 2024-09-21T07:04:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.725+0000 7f6875600640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68580b6700 0x7f68580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.726+0000 7f6861c00640 1 -- 172.21.15.114:0/2257954223 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864004a20 con 0x55c523b052f0 2024-09-21T07:04:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.726+0000 7f6861c00640 1 -- 172.21.15.114:0/2257954223 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640035f0 con 0x55c523b052f0 2024-09-21T07:04:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 -- 172.21.15.114:0/2257954223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 msgr2=0x55c523ed3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ed3350 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f68700755c0 tx=0x7f6864004770 comp rx=0 tx=0).stop 2024-09-21T07:04:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 -- 172.21.15.114:0/2257954223 shutdown_connections 2024-09-21T07:04:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ed3350 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 --2- 172.21.15.114:0/2257954223 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523ed2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.728+0000 7f687e66e740 1 -- 172.21.15.114:0/2257954223 >> 172.21.15.114:0/2257954223 conn(0x55c523b22300 msgr2=0x55c523ed0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.730+0000 7f687e66e740 1 -- 172.21.15.114:0/2257954223 shutdown_connections 2024-09-21T07:04:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.731+0000 7f687e66e740 1 -- 172.21.15.114:0/2257954223 wait complete. 2024-09-21T07:04:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.731+0000 7f6875600640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68580b6700 0x7f68580b8c10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f68680070b0 tx=0x7f686800d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.732+0000 7f687e66e740 1 -- start start 2024-09-21T07:04:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.732+0000 7f687e66e740 1 Processor -- start 2024-09-21T07:04:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.736+0000 7f687e66e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b052f0 0x55c523ede4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.736+0000 7f687e66e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523edea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.736+0000 7f687e66e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 0x55c523eea820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.737+0000 7f6875600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 0x55c523eea820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.737+0000 7f6875600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 0x55c523eea820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44360/0 (socket says 172.21.15.114:44360) 2024-09-21T07:04:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.737+0000 7f686fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523edea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.737+0000 7f686fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523edea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39968/0 (socket says 172.21.15.114:39968) 2024-09-21T07:04:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.738+0000 7f6875600640 1 -- 172.21.15.114:0/1273621868 learned_addr learned my addr 172.21.15.114:0/1273621868 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.738+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c523d6fab0 con 0x55c523b052f0 2024-09-21T07:04:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.738+0000 7f6874c00640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b052f0 0x55c523ede4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.739+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c523dde570 con 0x55c523b0d4d0 2024-09-21T07:04:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.739+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c523ee1530 con 0x55c523b2e1b0 2024-09-21T07:04:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.739+0000 7f687e66e740 10 client.0 _ll_register_callbacks cb 0x55c523ded100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.741+0000 7f6875600640 1 -- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 msgr2=0x55c523edea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.741+0000 7f6875600640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523edea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.741+0000 7f6875600640 1 -- 172.21.15.114:0/1273621868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b052f0 msgr2=0x55c523ede4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.742+0000 7f6875600640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b052f0 0x55c523ede4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.742+0000 7f6875600640 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c523b99790 con 0x55c523b2e1b0 2024-09-21T07:04:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.742+0000 7f6875600640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 0x55c523eea820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f686800afd0 tx=0x7f6868004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b7a857da3d6e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.742+0000 7f6874c00640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b052f0 0x55c523ede4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.742+0000 7f6856000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.744+0000 7f6856000640 1 -- 172.21.15.114:0/1273621868 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868025060 con 0x55c523b2e1b0 2024-09-21T07:04:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.744+0000 7f6856000640 1 -- 172.21.15.114:0/1273621868 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f68680205f0 con 0x55c523b2e1b0 2024-09-21T07:04:01.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.744+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6864004a20 con 0x55c523b2e1b0 2024-09-21T07:04:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.748+0000 7f6856000640 1 -- 172.21.15.114:0/1273621868 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f68680207f0 con 0x55c523b2e1b0 2024-09-21T07:04:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.748+0000 7f687e66e740 10 client.27937 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.748+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c523c12620 con 0x55c523b2e1b0 2024-09-21T07:04:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.748+0000 7f6856000640 1 -- 172.21.15.114:0/1273621868 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864004a20 con 0x55c523b2e1b0 2024-09-21T07:04:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.749+0000 7f6856000640 1 -- 172.21.15.114:0/1273621868 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f6868028350 con 0x55c523b2e1b0 2024-09-21T07:04:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.749+0000 7f687e66e740 10 client.27937 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.749+0000 7f687e66e740 10 client.27937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.751+0000 7f687e66e740 20 client.27937 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.751+0000 7f687e66e740 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523aecd80 0x55c523ef05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.751+0000 7f687e66e740 4 client.27937 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.751+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c523e4a600 con 0x55c523aecd80 2024-09-21T07:04:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.751+0000 7f6874c00640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523aecd80 0x55c523ef05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.752+0000 7f6856000640 10 client.27937 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:01.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.752+0000 7f6874c00640 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523aecd80 0x55c523ef05c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8f6d126648cd74 server_cookie=ad32cd880fdec029 in_seq=0 out_seq=0 2024-09-21T07:04:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.753+0000 7f6856000640 1 -- 172.21.15.114:0/1273621868 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c523e4a600 con 0x55c523aecd80 2024-09-21T07:04:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.753+0000 7f6856000640 10 client.27937 handle_command_reply: tid=0 2024-09-21T07:04:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 client.27937 shutdown 2024-09-21T07:04:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 msgr2=0x55c523eea820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 0x55c523eea820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f686800afd0 tx=0x7f6868004a40 comp rx=0 tx=0).stop 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 shutdown_connections 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523aecd80 0x55c523ef05c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b2e1b0 0x55c523eea820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523edea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 --2- 172.21.15.114:0/1273621868 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b052f0 0x55c523ede4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 >> 172.21.15.114:0/1273621868 conn(0x55c523b22300 msgr2=0x55c523ed3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.754+0000 7f6856000640 0 client.27937 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.757+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 shutdown_connections 2024-09-21T07:04:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.757+0000 7f687e66e740 1 -- 172.21.15.114:0/1273621868 wait complete. 2024-09-21T07:04:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.757+0000 7f687e66e740 20 client.27937 trim_cache size 0 max 16384 2024-09-21T07:04:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.805+0000 7f687e66e740 1 Processor -- start 2024-09-21T07:04:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.805+0000 7f687e66e740 1 -- start start 2024-09-21T07:04:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f687e66e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ede9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f687e66e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f687e66e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523eea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f687e66e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b7520 con 0x55c523b2e1b0 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f687e66e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b9060 con 0x55c523b052f0 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f687e66e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b7bf0 con 0x55c523b0d4d0 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f6874c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ede9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f6874c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ede9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:39970/0 (socket says 172.21.15.114:39970) 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.806+0000 7f6874c00640 1 -- 172.21.15.114:0/2939318558 learned_addr learned my addr 172.21.15.114:0/2939318558 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.807+0000 7f6875600640 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523eea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.807+0000 7f686fe00640 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.808+0000 7f6874c00640 1 -- 172.21.15.114:0/2939318558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 msgr2=0x55c523ed4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.808+0000 7f6874c00640 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.808+0000 7f6874c00640 1 -- 172.21.15.114:0/2939318558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 msgr2=0x55c523eea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.808+0000 7f6874c00640 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523eea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.808+0000 7f6874c00640 1 -- 172.21.15.114:0/2939318558 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c523ed89a0 con 0x55c523b052f0 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.808+0000 7f6875600640 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523eea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.809+0000 7f6874c00640 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ede9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f686403ba10 tx=0x7f6864005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d011f48b55627fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.810+0000 7f6861200640 1 -- 172.21.15.114:0/2939318558 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864046040 con 0x55c523b052f0 2024-09-21T07:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.810+0000 7f6861200640 1 -- 172.21.15.114:0/2939318558 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864004e20 con 0x55c523b052f0 2024-09-21T07:04:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.810+0000 7f6861200640 1 -- 172.21.15.114:0/2939318558 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864003850 con 0x55c523b052f0 2024-09-21T07:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.813+0000 7f687e66e740 1 -- 172.21.15.114:0/2939318558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 msgr2=0x55c523ede9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.813+0000 7f687e66e740 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ede9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f686403ba10 tx=0x7f6864005900 comp rx=0 tx=0).stop 2024-09-21T07:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 -- 172.21.15.114:0/2939318558 shutdown_connections 2024-09-21T07:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523eea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b0d4d0 0x55c523ed4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 --2- 172.21.15.114:0/2939318558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b052f0 0x55c523ede9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 -- 172.21.15.114:0/2939318558 >> 172.21.15.114:0/2939318558 conn(0x55c523b22300 msgr2=0x55c523ed3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 -- 172.21.15.114:0/2939318558 shutdown_connections 2024-09-21T07:04:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.814+0000 7f687e66e740 1 -- 172.21.15.114:0/2939318558 wait complete. 2024-09-21T07:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.815+0000 7f687e66e740 1 -- start start 2024-09-21T07:04:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.816+0000 7f687e66e740 1 Processor -- start 2024-09-21T07:04:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.817+0000 7f687e66e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b052f0 0x55c523ee1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.817+0000 7f687e66e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523ee2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.818+0000 7f687e66e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523f58f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.818+0000 7f6875600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523f58f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.818+0000 7f6875600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523f58f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40346/0 (socket says 172.21.15.114:40346) 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f686fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523ee2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f6875600640 1 -- 172.21.15.114:0/1540498752 learned_addr learned my addr 172.21.15.114:0/1540498752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b8990 con 0x55c523b2e1b0 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b82c0 con 0x55c523b0d4d0 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f68701b5520 con 0x55c523b052f0 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f687e66e740 10 client.0 _ll_register_callbacks cb 0x7f6868009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.820+0000 7f6874c00640 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b052f0 0x55c523ee1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.821+0000 7f6874c00640 1 -- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 msgr2=0x55c523ee2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.821+0000 7f6874c00640 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523ee2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.821+0000 7f6874c00640 1 -- 172.21.15.114:0/1540498752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 msgr2=0x55c523f58f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.821+0000 7f6874c00640 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523f58f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.821+0000 7f6874c00640 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c523f595b0 con 0x55c523b052f0 2024-09-21T07:04:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.822+0000 7f6874c00640 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b052f0 0x55c523ee1f80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6864006010 tx=0x7f6864004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8840c7d188fc4e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.822+0000 7f6856000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.824+0000 7f6856000640 1 -- 172.21.15.114:0/1540498752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864046040 con 0x55c523b052f0 2024-09-21T07:04:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.824+0000 7f6856000640 1 -- 172.21.15.114:0/1540498752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7f6864019070 con 0x55c523b052f0 2024-09-21T07:04:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.824+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6864004e20 con 0x55c523b052f0 2024-09-21T07:04:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.825+0000 7f6856000640 1 -- 172.21.15.114:0/1540498752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6864074050 con 0x55c523b052f0 2024-09-21T07:04:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.825+0000 7f687e66e740 10 client.27943 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.825+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c523f5a820 con 0x55c523b052f0 2024-09-21T07:04:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.826+0000 7f6856000640 1 -- 172.21.15.114:0/1540498752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864004e20 con 0x55c523b052f0 2024-09-21T07:04:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.826+0000 7f6856000640 1 -- 172.21.15.114:0/1540498752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f6864060c00 con 0x55c523b052f0 2024-09-21T07:04:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.827+0000 7f687e66e740 10 client.27943 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.827+0000 7f687e66e740 10 client.27943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.827+0000 7f687e66e740 20 client.27943 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.827+0000 7f687e66e740 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523ac21c0 0x55c523f5cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.828+0000 7f687e66e740 4 client.27943 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:04:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.828+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c523e460b0 con 0x55c523ac21c0 2024-09-21T07:04:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.829+0000 7f686fe00640 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523ac21c0 0x55c523f5cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.830+0000 7f686fe00640 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523ac21c0 0x55c523f5cac0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9958537e864446e7 server_cookie=e8cdb80510a58161 in_seq=0 out_seq=0 2024-09-21T07:04:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:01.830+0000 7f6856000640 10 client.27943 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 565.0 s, 390.8 tps, lat 242.008 ms stddev 313.276 2024-09-21T07:04:03.266 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.264+0000 7fa94d600640 1 -- 172.21.15.114:0/1204739740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 msgr2=0x7fa94009bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:03.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.264+0000 7fa94d600640 1 --2- 172.21.15.114:0/1204739740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94009bea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa938009860 tx=0x7fa93802a340 comp rx=0 tx=0).stop 2024-09-21T07:04:03.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.265+0000 7fa94d600640 1 -- 172.21.15.114:0/1204739740 shutdown_connections 2024-09-21T07:04:03.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.265+0000 7fa94d600640 1 --2- 172.21.15.114:0/1204739740 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 0x7fa9400a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:03.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.265+0000 7fa94d600640 1 --2- 172.21.15.114:0/1204739740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94009bea0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:03.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.265+0000 7fa94d600640 1 --2- 172.21.15.114:0/1204739740 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 0x7fa940096560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:03.267 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.265+0000 7fa94d600640 1 -- 172.21.15.114:0/1204739740 >> 172.21.15.114:0/1204739740 conn(0x7fa94000de50 msgr2=0x7fa94000e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:03.267 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:02 smithi114 ceph-mon[30654]: pgmap v699: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-09-21T07:04:03.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.266+0000 7fa94d600640 1 -- 172.21.15.114:0/1204739740 shutdown_connections 2024-09-21T07:04:03.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.266+0000 7fa94d600640 1 -- 172.21.15.114:0/1204739740 wait complete. 2024-09-21T07:04:03.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.267+0000 7fa94d600640 1 Processor -- start 2024-09-21T07:04:03.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.267+0000 7fa94d600640 1 -- start start 2024-09-21T07:04:03.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa94d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 0x7fa94012c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:03.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa94d600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94012cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:03.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa94d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 0x7fa9401321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:03.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa946a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 0x7fa94012c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:03.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa946a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 0x7fa94012c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40008/0 (socket says 172.21.15.114:40008) 2024-09-21T07:04:03.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa946000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94012cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:03.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.268+0000 7fa946000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94012cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44390/0 (socket says 172.21.15.114:44390) 2024-09-21T07:04:03.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.269+0000 7fa947400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 0x7fa9401321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:03.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.269+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400a60b0 con 0x7fa94009c3e0 2024-09-21T07:04:03.271 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.269+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400a5f30 con 0x7fa940096150 2024-09-21T07:04:03.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.269+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9400a6230 con 0x7fa94009ba90 2024-09-21T07:04:03.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.269+0000 7fa946000640 1 -- 172.21.15.114:0/2760697362 learned_addr learned my addr 172.21.15.114:0/2760697362 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:03.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.270+0000 7fa946000640 1 -- 172.21.15.114:0/2760697362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 msgr2=0x7fa94012c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:03.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.270+0000 7fa946000640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 0x7fa94012c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:03.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.270+0000 7fa946000640 1 -- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 msgr2=0x7fa9401321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:03.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.270+0000 7fa946000640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 0x7fa9401321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:03.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.270+0000 7fa946000640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa940132910 con 0x7fa94009ba90 2024-09-21T07:04:03.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.271+0000 7fa947400640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 0x7fa9401321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:03.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.271+0000 7fa946000640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94012cf40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa938000c00 tx=0x7fa938033dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:03.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.272+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938037040 con 0x7fa94009ba90 2024-09-21T07:04:03.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.272+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa940132ba0 con 0x7fa94009ba90 2024-09-21T07:04:03.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.272+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa940007760 con 0x7fa94009ba90 2024-09-21T07:04:03.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.272+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa938030780 con 0x7fa94009ba90 2024-09-21T07:04:03.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.272+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938003850 con 0x7fa94009ba90 2024-09-21T07:04:03.277 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.275+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa928005290 con 0x7fa94009ba90 2024-09-21T07:04:03.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.277+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa938004100 con 0x7fa94009ba90 2024-09-21T07:04:03.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.280+0000 7fa93f400640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa92c0b6780 0x7fa92c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:03.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.281+0000 7fa946a00640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa92c0b6780 0x7fa92c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:03.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.281+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9490+0+0 (secure 0 0 0) 0x7fa938106fe0 con 0x7fa94009ba90 2024-09-21T07:04:03.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.282+0000 7fa946a00640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa92c0b6780 0x7fa92c0b8c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa9340045f0 tx=0x7fa93400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:03.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.284+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa9380cd100 con 0x7fa94009ba90 2024-09-21T07:04:03.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:02 smithi183 ceph-mon[34219]: pgmap v699: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-09-21T07:04:03.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:02 smithi193 ceph-mon[34601]: pgmap v699: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2024-09-21T07:04:03.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.680+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa928005850 con 0x7fa94009ba90 2024-09-21T07:04:03.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.681+0000 7fa93f400640 1 -- 172.21.15.114:0/2760697362 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7fa9380d14e0 con 0x7fa94009ba90 2024-09-21T07:04:03.684 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:04:03.685 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 49 2024-09-21T07:04:03.685 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":49,"btime":"2024-09-21T07:03:23:885749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:03:22.931940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:04:04.187 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:04:04.187 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T07:04:04.188 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.685+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa92c0b6780 msgr2=0x7fa92c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.685+0000 7fa94d600640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa92c0b6780 0x7fa92c0b8c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa9340045f0 tx=0x7fa93400b390 comp rx=0 tx=0).stop 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.685+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 msgr2=0x7fa94012cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.685+0000 7fa94d600640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94012cf40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa938000c00 tx=0x7fa938033dd0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 shutdown_connections 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa94009c3e0 0x7fa9401321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa94009ba90 0x7fa94012cf40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa92c0b6780 0x7fa92c0b8c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 --2- 172.21.15.114:0/2760697362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa940096150 0x7fa94012c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 >> 172.21.15.114:0/2760697362 conn(0x7fa94000de50 msgr2=0x7fa940099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 shutdown_connections 2024-09-21T07:04:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:03.687+0000 7fa94d600640 1 -- 172.21.15.114:0/2760697362 wait complete. 2024-09-21T07:04:04.307 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:04 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2760697362' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:04.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:04 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2760697362' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:04.415 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:04 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2760697362' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:05.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:05 smithi114 ceph-mon[30654]: pgmap v700: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-09-21T07:04:05.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:05 smithi183 ceph-mon[34219]: pgmap v700: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-09-21T07:04:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:05 smithi193 ceph-mon[34601]: pgmap v700: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 525 op/s 2024-09-21T07:04:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.702+0000 7f6856000640 1 -- 172.21.15.114:0/1540498752 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c523e460b0 con 0x55c523ac21c0 2024-09-21T07:04:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.702+0000 7f6856000640 10 client.27943 handle_command_reply: tid=0 2024-09-21T07:04:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.702+0000 7f687e66e740 1 client.27943 shutdown 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b052f0 msgr2=0x55c523ee1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b052f0 0x55c523ee1f80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6864006010 tx=0x7f6864004770 comp rx=0 tx=0).stop 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 shutdown_connections 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c523b2e1b0 0x55c523f58f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c523b0d4d0 0x55c523ee2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c523ac21c0 0x55c523f5cac0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 --2- 172.21.15.114:0/1540498752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c523b052f0 0x55c523ee1f80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 >> 172.21.15.114:0/1540498752 conn(0x55c523b22300 msgr2=0x55c523ecf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.703+0000 7f6856000640 0 client.27943 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.704+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 shutdown_connections 2024-09-21T07:04:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.704+0000 7f687e66e740 1 -- 172.21.15.114:0/1540498752 wait complete. 2024-09-21T07:04:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.704+0000 7f687e66e740 20 client.27943 trim_cache size 0 max 16384 2024-09-21T07:04:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:04:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:04:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:04:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.709+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68580b6700 msgr2=0x7f68580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.709+0000 7f6876000640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68580b6700 0x7f68580b8c10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f68680070b0 tx=0x7f686800d040 comp rx=0 tx=0).stop 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.709+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 msgr2=0x7f68701188a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.709+0000 7f6876000640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 0x7f68701188a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e9a0 tx=0x7f685c00ee60 comp rx=0 tx=0).stop 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.710+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 shutdown_connections 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.711+0000 7f6876000640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68580b6700 0x7f68580b8c10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.711+0000 7f6876000640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687011cba0 0x7f6870118fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.711+0000 7f6876000640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6870111ca0 0x7f68701188a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.711+0000 7f6876000640 1 --2- 172.21.15.114:0/111057712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f687010a900 0x7f687011c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.711+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 >> 172.21.15.114:0/111057712 conn(0x7f6870070c00 msgr2=0x7f687011cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.712+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 shutdown_connections 2024-09-21T07:04:05.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:05.712+0000 7f6876000640 1 -- 172.21.15.114:0/111057712 wait complete. 2024-09-21T07:04:06.335 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:04:06.419 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:07.320 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:07 smithi114 ceph-mon[30654]: pgmap v701: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.3 MiB/s wr, 442 op/s 2024-09-21T07:04:07.320 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:07 smithi114 ceph-mon[30654]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T07:04:07.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:07 smithi183 ceph-mon[34219]: pgmap v701: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.3 MiB/s wr, 442 op/s 2024-09-21T07:04:07.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:07 smithi183 ceph-mon[34219]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T07:04:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:07 smithi193 ceph-mon[34601]: pgmap v701: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.3 MiB/s wr, 442 op/s 2024-09-21T07:04:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:07 smithi193 ceph-mon[34601]: osdmap e96: 12 total, 12 up, 12 in 2024-09-21T07:04:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 570.0 s, 341.6 tps, lat 299.427 ms stddev 398.811 2024-09-21T07:04:08.155 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:08 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2346884671' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.417+0000 7f80a4a00640 1 -- 172.21.15.114:0/3561481835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 msgr2=0x7f80a0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.417+0000 7f80a4a00640 1 --2- 172.21.15.114:0/3561481835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a0110470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8098009be0 tx=0x7f809802a6c0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.418+0000 7f80a4a00640 1 -- 172.21.15.114:0/3561481835 shutdown_connections 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.418+0000 7f80a4a00640 1 --2- 172.21.15.114:0/3561481835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a0110470 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.418+0000 7f80a4a00640 1 --2- 172.21.15.114:0/3561481835 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80a01092f0 0x7f80a0109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.418+0000 7f80a4a00640 1 --2- 172.21.15.114:0/3561481835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80a01039b0 0x7f80a0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.418+0000 7f80a4a00640 1 -- 172.21.15.114:0/3561481835 >> 172.21.15.114:0/3561481835 conn(0x7f80a00ff440 msgr2=0x7f80a0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.419+0000 7f80a4a00640 1 -- 172.21.15.114:0/3561481835 shutdown_connections 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.419+0000 7f80a4a00640 1 -- 172.21.15.114:0/3561481835 wait complete. 2024-09-21T07:04:08.167 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.420+0000 7f80a4a00640 1 Processor -- start 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.421+0000 7f80a4a00640 1 -- start start 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f80a4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80a01039b0 0x7f80a010daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f80a4a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80a01092f0 0x7f80a010dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f80a4a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a01ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f80a4a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0077160 con 0x7f80a01092f0 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f80a4a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0076fe0 con 0x7f80a01039b0 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f80a4a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f80a00772e0 con 0x7f80a0109c40 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f809fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a01ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f809fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a01ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33706/0 (socket says 172.21.15.114:33706) 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.422+0000 7f809fe00640 1 -- 172.21.15.114:0/2346884671 learned_addr learned my addr 172.21.15.114:0/2346884671 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809ea00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80a01092f0 0x7f80a010dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809fe00640 1 -- 172.21.15.114:0/2346884671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80a01039b0 msgr2=0x7f80a010daa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809fe00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80a01039b0 0x7f80a010daa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.168 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809fe00640 1 -- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80a01092f0 msgr2=0x7f80a010dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809fe00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80a01092f0 0x7f80a010dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809fe00640 1 -- 172.21.15.114:0/2346884671 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a010e880 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.423+0000 7f809fe00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a01ac720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8098009b30 tx=0x7f8098030710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.424+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098041020 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.424+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8098030e20 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.424+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098036040 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.429+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a006ce40 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.429+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a006d2a0 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.432+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a0103dc0 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.440+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8098044020 con 0x7f80a0109c40 2024-09-21T07:04:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.448+0000 7f8097e00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80840b6700 0x7f80840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.448+0000 7f809f400640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80840b6700 0x7f80840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.449+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9578+0+0 (secure 0 0 0) 0x7f8098108a60 con 0x7f80a0109c40 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.449+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f809804b070 con 0x7f80a0109c40 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.451+0000 7f809f400640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80840b6700 0x7f80840b8c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f80a0110500 tx=0x7f8090005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.849+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80a010eda0 con 0x7f80a0109c40 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.850+0000 7f8097e00640 1 -- 172.21.15.114:0/2346884671 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7f80980ceb90 con 0x7f80a0109c40 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 49 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.853+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80840b6700 msgr2=0x7f80840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.853+0000 7f80a4a00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80840b6700 0x7f80840b8c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f80a0110500 tx=0x7f8090005f90 comp rx=0 tx=0).stop 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.853+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 msgr2=0x7f80a01ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:08.170 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.853+0000 7f80a4a00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a01ac720 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8098009b30 tx=0x7f8098030710 comp rx=0 tx=0).stop 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":49,"btime":"2024-09-21T07:03:23:885749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:03:22.931940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 shutdown_connections 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80a0109c40 0x7f80a01ac720 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f80a01092f0 0x7f80a010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f80840b6700 0x7f80840b8c10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 --2- 172.21.15.114:0/2346884671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f80a01039b0 0x7f80a010daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 >> 172.21.15.114:0/2346884671 conn(0x7f80a00ff440 msgr2=0x7f80a0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 shutdown_connections 2024-09-21T07:04:08.171 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:07.854+0000 7f80a4a00640 1 -- 172.21.15.114:0/2346884671 wait complete. 2024-09-21T07:04:08.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:08 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2346884671' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:08.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:08 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2346884671' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:09.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:09 smithi183 ceph-mon[34219]: pgmap v703: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-09-21T07:04:09.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:09 smithi193 ceph-mon[34601]: pgmap v703: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-09-21T07:04:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:09 smithi114 ceph-mon[30654]: pgmap v703: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.7 MiB/s wr, 423 op/s 2024-09-21T07:04:10.529 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:10.634 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T07:04:10.634 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T07:04:11.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:11 smithi183 ceph-mon[34219]: pgmap v704: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-21T07:04:11.398 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:11.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:11 smithi193 ceph-mon[34601]: pgmap v704: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-21T07:04:11.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.547+0000 7fe955200640 1 -- 172.21.15.114:0/453637355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950109fc0 msgr2=0x7fe95010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.547+0000 7fe955200640 1 --2- 172.21.15.114:0/453637355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950109fc0 0x7fe95010a3d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe94002a400 comp rx=0 tx=0).stop 2024-09-21T07:04:11.553 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.551+0000 7fe955200640 1 -- 172.21.15.114:0/453637355 shutdown_connections 2024-09-21T07:04:11.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.551+0000 7fe955200640 1 --2- 172.21.15.114:0/453637355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe950110250 0x7fe950116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.551+0000 7fe955200640 1 --2- 172.21.15.114:0/453637355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe95010f900 0x7fe95010fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.555 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:11 smithi114 ceph-mon[30654]: pgmap v704: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 3.9 MiB/s wr, 415 op/s 2024-09-21T07:04:11.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.551+0000 7fe955200640 1 --2- 172.21.15.114:0/453637355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950109fc0 0x7fe95010a3d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.551+0000 7fe955200640 1 -- 172.21.15.114:0/453637355 >> 172.21.15.114:0/453637355 conn(0x7fe9500795c0 msgr2=0x7fe9501081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:11.556 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.553+0000 7fe955200640 1 -- 172.21.15.114:0/453637355 shutdown_connections 2024-09-21T07:04:11.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.554+0000 7fe955200640 1 -- 172.21.15.114:0/453637355 wait complete. 2024-09-21T07:04:11.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.556+0000 7fe955200640 1 Processor -- start 2024-09-21T07:04:11.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.556+0000 7fe955200640 1 -- start start 2024-09-21T07:04:11.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.558+0000 7fe955200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe950109fc0 0x7fe9501a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.558+0000 7fe955200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 0x7fe9501a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.558+0000 7fe955200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950110250 0x7fe9501a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.559+0000 7fe94f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe950109fc0 0x7fe9501a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.559+0000 7fe94f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe950109fc0 0x7fe9501a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49866/0 (socket says 172.21.15.114:49866) 2024-09-21T07:04:11.563 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.561+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 0x7fe9501a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.561+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 0x7fe9501a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33734/0 (socket says 172.21.15.114:33734) 2024-09-21T07:04:11.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.561+0000 7fe94f400640 1 -- 172.21.15.114:0/4212522137 learned_addr learned my addr 172.21.15.114:0/4212522137 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:11.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.561+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe950119f40 con 0x7fe950110250 2024-09-21T07:04:11.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe950119dc0 con 0x7fe950109fc0 2024-09-21T07:04:11.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe94fe00640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950110250 0x7fe9501a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe95011a0c0 con 0x7fe95010f900 2024-09-21T07:04:11.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe94ea00640 1 -- 172.21.15.114:0/4212522137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe950109fc0 msgr2=0x7fe9501a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe94ea00640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe950109fc0 0x7fe9501a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe94ea00640 1 -- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950110250 msgr2=0x7fe9501a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe94ea00640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950110250 0x7fe9501a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.562+0000 7fe94ea00640 1 -- 172.21.15.114:0/4212522137 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9501a67a0 con 0x7fe95010f900 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.563+0000 7fe94fe00640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950110250 0x7fe9501a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.563+0000 7fe94ea00640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 0x7fe9501a0e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe93c005fd0 tx=0x7fe93c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.563+0000 7fe947e00640 1 -- 172.21.15.114:0/4212522137 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c009c00 con 0x7fe95010f900 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.563+0000 7fe947e00640 1 -- 172.21.15.114:0/4212522137 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93c009da0 con 0x7fe95010f900 2024-09-21T07:04:11.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.563+0000 7fe947e00640 1 -- 172.21.15.114:0/4212522137 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c003e50 con 0x7fe95010f900 2024-09-21T07:04:11.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.565+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9501a6a90 con 0x7fe95010f900 2024-09-21T07:04:11.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.565+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9501a6f50 con 0x7fe95010f900 2024-09-21T07:04:11.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.572+0000 7fe947e00640 1 -- 172.21.15.114:0/4212522137 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe93c0035f0 con 0x7fe95010f900 2024-09-21T07:04:11.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.575+0000 7fe947e00640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9340b6990 0x7fe9340b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.576+0000 7fe947e00640 1 -- 172.21.15.114:0/4212522137 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9578+0+0 (secure 0 0 0) 0x7fe93c0e76a0 con 0x7fe95010f900 2024-09-21T07:04:11.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.576+0000 7fe95d4c9740 1 Processor -- start 2024-09-21T07:04:11.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.576+0000 7fe94f400640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9340b6990 0x7fe9340b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 -- start start 2024-09-21T07:04:11.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x558915523d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 0x558915524270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55891549afb0 con 0x55891517d070 2024-09-21T07:04:11.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55891548a3e0 con 0x55891515c390 2024-09-21T07:04:11.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.577+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55891548e1f0 con 0x5589151541b0 2024-09-21T07:04:11.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.580+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 0x558915524270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.580+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 0x558915524270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33738/0 (socket says 172.21.15.114:33738) 2024-09-21T07:04:11.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.580+0000 7fe94ea00640 1 -- 172.21.15.114:0/1134612723 learned_addr learned my addr 172.21.15.114:0/1134612723 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:11.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.580+0000 7fe94f400640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x558915523d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.580+0000 7fe94fe00640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.581+0000 7fe94fe00640 1 -- 172.21.15.114:0/1134612723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 msgr2=0x558915524270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.581+0000 7fe94fe00640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 0x558915524270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.581+0000 7fe94fe00640 1 -- 172.21.15.114:0/1134612723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 msgr2=0x558915523d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.582+0000 7fe94fe00640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x558915523d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.582+0000 7fe94fe00640 1 -- 172.21.15.114:0/1134612723 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558915454cd0 con 0x55891515c390 2024-09-21T07:04:11.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.582+0000 7fe94ea00640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 0x558915524270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:11.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.582+0000 7fe94f400640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x558915523d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:04:11.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.583+0000 7fe94f400640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9340b6990 0x7fe9340b8ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe940005ab0 tx=0x7fe940031b50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:11.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.583+0000 7fe94fe00640 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155247b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe948007db0 tx=0x7fe94800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc9c3a43b6fd1670 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:11.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.584+0000 7fe944c00640 1 -- 172.21.15.114:0/1134612723 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948014070 con 0x55891515c390 2024-09-21T07:04:11.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.584+0000 7fe944c00640 1 -- 172.21.15.114:0/1134612723 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe948004d20 con 0x55891515c390 2024-09-21T07:04:11.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.584+0000 7fe944c00640 1 -- 172.21.15.114:0/1134612723 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9480107f0 con 0x55891515c390 2024-09-21T07:04:11.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1134612723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 msgr2=0x5589155247b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155247b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe948007db0 tx=0x7fe94800e720 comp rx=0 tx=0).stop 2024-09-21T07:04:11.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1134612723 shutdown_connections 2024-09-21T07:04:11.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155247b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589151541b0 0x558915524270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1134612723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x558915523d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.586+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1134612723 >> 172.21.15.114:0/1134612723 conn(0x558915171460 msgr2=0x558915521bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:11.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.587+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1134612723 shutdown_connections 2024-09-21T07:04:11.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.587+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1134612723 wait complete. 2024-09-21T07:04:11.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.590+0000 7fe95d4c9740 1 -- start start 2024-09-21T07:04:11.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.591+0000 7fe95d4c9740 1 Processor -- start 2024-09-21T07:04:11.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.592+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 0x55891552f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.593+0000 7fe94f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 0x55891552f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.594+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891515c390 0x55891552ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.593+0000 7fe94f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 0x55891552f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49894/0 (socket says 172.21.15.114:49894) 2024-09-21T07:04:11.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.594+0000 7fe94f400640 1 -- 172.21.15.114:0/1182242317 learned_addr learned my addr 172.21.15.114:0/1182242317 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:11.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.596+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x55891554c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.596+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5589152eb700 con 0x55891517d070 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.596+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558915425580 con 0x5589151541b0 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558915543520 con 0x55891515c390 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe95d4c9740 10 client.0 _ll_register_callbacks cb 0x558915435470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe94f400640 1 -- 172.21.15.114:0/1182242317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891515c390 msgr2=0x55891552ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe94f400640 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891515c390 0x55891552ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe94f400640 1 -- 172.21.15.114:0/1182242317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 msgr2=0x55891554c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe94f400640 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x55891554c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.597+0000 7fe94f400640 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589151e8650 con 0x5589151541b0 2024-09-21T07:04:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.601+0000 7fe94f400640 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 0x55891552f970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe940031870 tx=0x7fe94002a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd5439fb745486 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:11.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.601+0000 7fe939600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:11.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.603+0000 7fe939600640 1 -- 172.21.15.114:0/1182242317 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94004e030 con 0x5589151541b0 2024-09-21T07:04:11.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.603+0000 7fe939600640 1 -- 172.21.15.114:0/1182242317 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9578+0+0 (secure 0 0 0) 0x7fe940004a90 con 0x5589151541b0 2024-09-21T07:04:11.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.603+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558915427b90 con 0x5589151541b0 2024-09-21T07:04:11.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.605+0000 7fe939600640 1 -- 172.21.15.114:0/1182242317 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe940035030 con 0x5589151541b0 2024-09-21T07:04:11.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.605+0000 7fe95d4c9740 10 client.28013 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:11.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.606+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x558915442950 con 0x5589151541b0 2024-09-21T07:04:11.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.606+0000 7fe939600640 1 -- 172.21.15.114:0/1182242317 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558915427b90 con 0x5589151541b0 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.609+0000 7fe939600640 1 -- 172.21.15.114:0/1182242317 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fe94003a040 con 0x5589151541b0 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.609+0000 7fe95d4c9740 10 client.28013 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.609+0000 7fe95d4c9740 10 client.28013 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.609+0000 7fe95d4c9740 20 client.28013 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.609+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55891513b9f0 0x5589155529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.609+0000 7fe95d4c9740 4 client.28013 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:11.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.610+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891549cf00 con 0x55891513b9f0 2024-09-21T07:04:11.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.610+0000 7fe94ea00640 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55891513b9f0 0x5589155529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.612+0000 7fe94ea00640 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55891513b9f0 0x5589155529f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2ff1307a47c11a5 server_cookie=22a25275131e5ba5 in_seq=0 out_seq=0 2024-09-21T07:04:11.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.612+0000 7fe939600640 10 client.28013 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:04:11.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.613+0000 7fe939600640 1 -- 172.21.15.114:0/1182242317 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55891549cf00 con 0x55891513b9f0 2024-09-21T07:04:11.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.613+0000 7fe939600640 10 client.28013 handle_command_reply: tid=0 2024-09-21T07:04:11.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.613+0000 7fe95d4c9740 1 client.28013 shutdown 2024-09-21T07:04:11.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 msgr2=0x55891552f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 0x55891552f970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe940031870 tx=0x7fe94002a400 comp rx=0 tx=0).stop 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 shutdown_connections 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55891517d070 0x55891554c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891515c390 0x55891552ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55891513b9f0 0x5589155529f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1182242317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589151541b0 0x55891552f970 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 >> 172.21.15.114:0/1182242317 conn(0x558915171460 msgr2=0x558915526f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:11.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.614+0000 7fe939600640 0 client.28013 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:04:11.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.615+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 shutdown_connections 2024-09-21T07:04:11.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.615+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1182242317 wait complete. 2024-09-21T07:04:11.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.616+0000 7fe95d4c9740 20 client.28013 trim_cache size 0 max 16384 2024-09-21T07:04:11.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.648+0000 7fe95d4c9740 1 Processor -- start 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.648+0000 7fe95d4c9740 1 -- start start 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.649+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x55891552af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.649+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x558915521c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.649+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x55891552c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.649+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe9501aed10 con 0x5589151541b0 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.649+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9501b0850 con 0x55891515c390 2024-09-21T07:04:11.652 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.649+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9501af3e0 con 0x55891517d070 2024-09-21T07:04:11.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.650+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x558915521c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.650+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x558915521c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49900/0 (socket says 172.21.15.114:49900) 2024-09-21T07:04:11.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.650+0000 7fe94fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x55891552c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.650+0000 7fe94fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x55891552c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:33764/0 (socket says 172.21.15.114:33764) 2024-09-21T07:04:11.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.650+0000 7fe94fe00640 1 -- 172.21.15.114:0/3114885007 learned_addr learned my addr 172.21.15.114:0/3114885007 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94fe00640 1 -- 172.21.15.114:0/3114885007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 msgr2=0x558915521c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94fe00640 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x558915521c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94fe00640 1 -- 172.21.15.114:0/3114885007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 msgr2=0x55891552af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94fe00640 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x55891552af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94fe00640 1 -- 172.21.15.114:0/3114885007 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891551bdc0 con 0x55891517d070 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94fe00640 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x55891552c960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe948007d80 tx=0x7fe948007560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01f1d4a34ff0d6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:11.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.651+0000 7fe94ea00640 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x558915521c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:11.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.653+0000 7fe93be00640 1 -- 172.21.15.114:0/3114885007 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948014040 con 0x55891517d070 2024-09-21T07:04:11.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.653+0000 7fe93be00640 1 -- 172.21.15.114:0/3114885007 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe948003e20 con 0x55891517d070 2024-09-21T07:04:11.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.653+0000 7fe93be00640 1 -- 172.21.15.114:0/3114885007 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948005040 con 0x55891517d070 2024-09-21T07:04:11.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.655+0000 7fe95d4c9740 1 -- 172.21.15.114:0/3114885007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 msgr2=0x55891552c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.655+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x55891552c960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe948007d80 tx=0x7fe948007560 comp rx=0 tx=0).stop 2024-09-21T07:04:11.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.656+0000 7fe95d4c9740 1 -- 172.21.15.114:0/3114885007 shutdown_connections 2024-09-21T07:04:11.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.656+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x55891552c960 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.656+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x558915521c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.656+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/3114885007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x55891552af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.656+0000 7fe95d4c9740 1 -- 172.21.15.114:0/3114885007 >> 172.21.15.114:0/3114885007 conn(0x558915171460 msgr2=0x558915521530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:11.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.658+0000 7fe95d4c9740 1 -- 172.21.15.114:0/3114885007 shutdown_connections 2024-09-21T07:04:11.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.658+0000 7fe95d4c9740 1 -- 172.21.15.114:0/3114885007 wait complete. 2024-09-21T07:04:11.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.659+0000 7fe95d4c9740 1 -- start start 2024-09-21T07:04:11.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.659+0000 7fe95d4c9740 1 Processor -- start 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x558915522e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x558915546760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe9501b0180 con 0x5589151541b0 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9501afab0 con 0x55891515c390 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe950083d60 con 0x55891517d070 2024-09-21T07:04:11.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.660+0000 7fe95d4c9740 10 client.0 _ll_register_callbacks cb 0x7fe940031d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:11.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.661+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.661+0000 7fe94ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155233f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49902/0 (socket says 172.21.15.114:49902) 2024-09-21T07:04:11.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.662+0000 7fe94ea00640 1 -- 172.21.15.114:0/1214269039 learned_addr learned my addr 172.21.15.114:0/1214269039 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:11.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.662+0000 7fe94fe00640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x558915546760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.662+0000 7fe94f400640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x558915522e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.663+0000 7fe94ea00640 1 -- 172.21.15.114:0/1214269039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 msgr2=0x558915546760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.663+0000 7fe94ea00640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x558915546760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.663+0000 7fe94ea00640 1 -- 172.21.15.114:0/1214269039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 msgr2=0x558915522e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:11.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.663+0000 7fe94ea00640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x558915522e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.663+0000 7fe94ea00640 1 -- 172.21.15.114:0/1214269039 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558915546da0 con 0x55891515c390 2024-09-21T07:04:11.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.664+0000 7fe94ea00640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155233f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0a9080 tx=0x7fe93c0accd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71eebf741afd70a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:11.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.664+0000 7fe939600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:11.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.665+0000 7fe939600640 1 -- 172.21.15.114:0/1214269039 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c0d4020 con 0x55891515c390 2024-09-21T07:04:11.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.665+0000 7fe939600640 1 -- 172.21.15.114:0/1214269039 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9578+0+0 (secure 0 0 0) 0x7fe93c098350 con 0x55891515c390 2024-09-21T07:04:11.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.665+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe948003e20 con 0x55891515c390 2024-09-21T07:04:11.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.665+0000 7fe939600640 1 -- 172.21.15.114:0/1214269039 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe93c0e4e00 con 0x55891515c390 2024-09-21T07:04:11.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.665+0000 7fe95d4c9740 10 client.28019 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:11.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.665+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5589155480d0 con 0x55891515c390 2024-09-21T07:04:11.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.666+0000 7fe939600640 1 -- 172.21.15.114:0/1214269039 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe948003e20 con 0x55891515c390 2024-09-21T07:04:11.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.666+0000 7fe939600640 1 -- 172.21.15.114:0/1214269039 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fe93c0b6ce0 con 0x55891515c390 2024-09-21T07:04:11.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.666+0000 7fe95d4c9740 10 client.28019 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:11.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.666+0000 7fe95d4c9740 10 client.28019 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:04:11.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.666+0000 7fe95d4c9740 20 client.28019 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:11.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.666+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x558915110da0 0x5589155488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:11.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.667+0000 7fe95d4c9740 4 client.28019 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:04:11.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.667+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589155265e0 con 0x558915110da0 2024-09-21T07:04:11.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.667+0000 7fe94f400640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x558915110da0 0x5589155488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.668+0000 7fe939600640 10 client.28019 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.669+0000 7fe94f400640 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x558915110da0 0x5589155488b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4f6c17494554ffd server_cookie=9269e981051ce404 in_seq=0 out_seq=0 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.669+0000 7fe939600640 1 -- 172.21.15.114:0/1214269039 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589155265e0 con 0x558915110da0 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.669+0000 7fe939600640 10 client.28019 handle_command_reply: tid=0 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.669+0000 7fe95d4c9740 1 client.28019 shutdown 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.669+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 msgr2=0x5589155233f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.669+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155233f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe93c0a9080 tx=0x7fe93c0accd0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 shutdown_connections 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55891517d070 0x558915546760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x558915110da0 0x5589155488b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55891515c390 0x5589155233f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 --2- 172.21.15.114:0/1214269039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589151541b0 0x558915522e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 >> 172.21.15.114:0/1214269039 conn(0x558915171460 msgr2=0x558915524d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe939600640 0 client.28019 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:04:11.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 shutdown_connections 2024-09-21T07:04:11.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.670+0000 7fe95d4c9740 1 -- 172.21.15.114:0/1214269039 wait complete. 2024-09-21T07:04:11.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.671+0000 7fe95d4c9740 20 client.28019 trim_cache size 0 max 16384 2024-09-21T07:04:11.678 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:04:11.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.677+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9340b6990 msgr2=0x7fe9340b8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.677+0000 7fe955200640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9340b6990 0x7fe9340b8ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe940005ab0 tx=0x7fe940031b50 comp rx=0 tx=0).stop 2024-09-21T07:04:11.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.677+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 msgr2=0x7fe9501a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:11.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.677+0000 7fe955200640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 0x7fe9501a0e90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe93c005fd0 tx=0x7fe93c00dde0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.678+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 shutdown_connections 2024-09-21T07:04:11.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.678+0000 7fe955200640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe950110250 0x7fe9501a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.678+0000 7fe955200640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe95010f900 0x7fe9501a0e90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.678+0000 7fe955200640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9340b6990 0x7fe9340b8ea0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.678+0000 7fe955200640 1 --2- 172.21.15.114:0/4212522137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe950109fc0 0x7fe9501a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:11.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.678+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 >> 172.21.15.114:0/4212522137 conn(0x7fe9500795c0 msgr2=0x7fe950079d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:11.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.680+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 shutdown_connections 2024-09-21T07:04:11.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:11.680+0000 7fe955200640 1 -- 172.21.15.114:0/4212522137 wait complete. 2024-09-21T07:04:12.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 575.0 s, 435.1 tps, lat 232.456 ms stddev 266.440 2024-09-21T07:04:13.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:13 smithi183 ceph-mon[34219]: pgmap v705: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.7 MiB/s wr, 388 op/s 2024-09-21T07:04:13.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:13 smithi193 ceph-mon[34601]: pgmap v705: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.7 MiB/s wr, 388 op/s 2024-09-21T07:04:13.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:13 smithi114 ceph-mon[30654]: pgmap v705: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.7 MiB/s wr, 388 op/s 2024-09-21T07:04:15.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:15 smithi114 ceph-mon[30654]: pgmap v706: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-09-21T07:04:15.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:15 smithi114 ceph-mon[30654]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T07:04:15.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:15 smithi193 ceph-mon[34601]: pgmap v706: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-09-21T07:04:15.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:15 smithi193 ceph-mon[34601]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T07:04:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:15 smithi183 ceph-mon[34219]: pgmap v706: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-09-21T07:04:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:15 smithi183 ceph-mon[34219]: osdmap e97: 12 total, 12 up, 12 in 2024-09-21T07:04:15.751 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:04:16.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 -- 172.21.15.114:0/668378668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc0a2970 msgr2=0x7f21fc0a91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 --2- 172.21.15.114:0/668378668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc0a2970 0x7f21fc0a91c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2204069e80 tx=0x7f2204098110 comp rx=0 tx=0).stop 2024-09-21T07:04:16.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 -- 172.21.15.114:0/668378668 shutdown_connections 2024-09-21T07:04:16.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 --2- 172.21.15.114:0/668378668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc0a2970 0x7f21fc0a91c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 --2- 172.21.15.114:0/668378668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 0x7f21fc0a2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 --2- 172.21.15.114:0/668378668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21fc09c930 0x7f21fc09cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.353+0000 7f2209800640 1 -- 172.21.15.114:0/668378668 >> 172.21.15.114:0/668378668 conn(0x7f21fc00a6b0 msgr2=0x7f21fc00aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:16.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.354+0000 7f2209800640 1 -- 172.21.15.114:0/668378668 shutdown_connections 2024-09-21T07:04:16.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.354+0000 7f2209800640 1 -- 172.21.15.114:0/668378668 wait complete. 2024-09-21T07:04:16.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.356+0000 7f2209800640 1 Processor -- start 2024-09-21T07:04:16.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.356+0000 7f2209800640 1 -- start start 2024-09-21T07:04:16.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.359+0000 7f2209800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc09c930 0x7f21fc132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.359+0000 7f2209800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 0x7f21fc133580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.359+0000 7f2209800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21fc0a2970 0x7f21fc138820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.359+0000 7f2209800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0ac570 con 0x7f21fc0a2970 2024-09-21T07:04:16.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.359+0000 7f2209800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0ac3f0 con 0x7f21fc0a2020 2024-09-21T07:04:16.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.359+0000 7f2209800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0ac6f0 con 0x7f21fc09c930 2024-09-21T07:04:16.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.360+0000 7f2203e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc09c930 0x7f21fc132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.360+0000 7f2203e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc09c930 0x7f21fc132fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46046/0 (socket says 172.21.15.114:46046) 2024-09-21T07:04:16.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.360+0000 7f2203e00640 1 -- 172.21.15.114:0/4256268033 learned_addr learned my addr 172.21.15.114:0/4256268033 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:16.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.360+0000 7f2203400640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 0x7f21fc133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.361+0000 7f2208e00640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21fc0a2970 0x7f21fc138820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.361+0000 7f2203400640 1 -- 172.21.15.114:0/4256268033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc09c930 msgr2=0x7f21fc132fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.361+0000 7f2203400640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc09c930 0x7f21fc132fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.361+0000 7f2203400640 1 -- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21fc0a2970 msgr2=0x7f21fc138820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.361+0000 7f2203400640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21fc0a2970 0x7f21fc138820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.361+0000 7f2203400640 1 -- 172.21.15.114:0/4256268033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc138f50 con 0x7f21fc0a2020 2024-09-21T07:04:16.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.363+0000 7f2203400640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 0x7f21fc133580 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f21fc005ae0 tx=0x7f21f000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:16.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.364+0000 7f2200c00640 1 -- 172.21.15.114:0/4256268033 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f000cc40 con 0x7f21fc0a2020 2024-09-21T07:04:16.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.364+0000 7f2200c00640 1 -- 172.21.15.114:0/4256268033 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f000cde0 con 0x7f21fc0a2020 2024-09-21T07:04:16.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.364+0000 7f2200c00640 1 -- 172.21.15.114:0/4256268033 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0018780 con 0x7f21fc0a2020 2024-09-21T07:04:16.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.364+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc139240 con 0x7f21fc0a2020 2024-09-21T07:04:16.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.364+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc13f9a0 con 0x7f21fc0a2020 2024-09-21T07:04:16.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.371+0000 7f2211bc9740 1 Processor -- start 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.374+0000 7f2211bc9740 1 -- start start 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 0x55f7be13f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be13fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2200c00640 1 -- 172.21.15.114:0/4256268033 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f21f0010430 con 0x7f21fc0a2020 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd76b60 0x55f7be13ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2211bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f7be0b4ca0 con 0x55f7bdd76b60 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2211bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7be0a40d0 con 0x55f7bdd6e980 2024-09-21T07:04:16.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.375+0000 7f2211bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f7be0a7ee0 con 0x55f7bdd97840 2024-09-21T07:04:16.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.376+0000 7f2203e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be13fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.376+0000 7f2208e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 0x55f7be13f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.376+0000 7f2203e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be13fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41830/0 (socket says 172.21.15.114:41830) 2024-09-21T07:04:16.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.376+0000 7f2203e00640 1 -- 172.21.15.114:0/2941795946 learned_addr learned my addr 172.21.15.114:0/2941795946 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:16.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.376+0000 7f2203400640 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd76b60 0x55f7be13ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.376+0000 7f2208e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 0x55f7be13f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46052/0 (socket says 172.21.15.114:46052) 2024-09-21T07:04:16.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.377+0000 7f2208e00640 1 -- 172.21.15.114:0/2941795946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 msgr2=0x55f7be13fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.377+0000 7f2208e00640 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be13fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.377+0000 7f2208e00640 1 -- 172.21.15.114:0/2941795946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd76b60 msgr2=0x55f7be13ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.377+0000 7f2208e00640 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd76b60 0x55f7be13ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.377+0000 7f2208e00640 1 -- 172.21.15.114:0/2941795946 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7be056400 con 0x55f7bdd97840 2024-09-21T07:04:16.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.379+0000 7f2200c00640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f21ec0b6850 0x7f21ec0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.379+0000 7f2200c00640 1 -- 172.21.15.114:0/4256268033 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f21f00e7cf0 con 0x7f21fc0a2020 2024-09-21T07:04:16.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.380+0000 7f2203e00640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f21ec0b6850 0x7f21ec0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.380+0000 7f2208e00640 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 0x55f7be13f500 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f22040524b0 tx=0x7f2204069e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2f2ea0807ff96e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:16.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.381+0000 7f21f5600640 1 -- 172.21.15.114:0/2941795946 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040a1530 con 0x55f7bdd97840 2024-09-21T07:04:16.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.381+0000 7f21f5600640 1 -- 172.21.15.114:0/2941795946 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220409c040 con 0x55f7bdd97840 2024-09-21T07:04:16.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.381+0000 7f21f5600640 1 -- 172.21.15.114:0/2941795946 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204075280 con 0x55f7bdd97840 2024-09-21T07:04:16.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.382+0000 7f2211bc9740 1 -- 172.21.15.114:0/2941795946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 msgr2=0x55f7be13f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.382+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 0x55f7be13f500 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f22040524b0 tx=0x7f2204069e10 comp rx=0 tx=0).stop 2024-09-21T07:04:16.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.382+0000 7f2203e00640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f21ec0b6850 0x7f21ec0b8d60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55f7be13e180 tx=0x7f21f8009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:16.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 -- 172.21.15.114:0/2941795946 shutdown_connections 2024-09-21T07:04:16.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd76b60 0x55f7be13ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be13fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2941795946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd97840 0x55f7be13f500 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 -- 172.21.15.114:0/2941795946 >> 172.21.15.114:0/2941795946 conn(0x55f7bdd8b990 msgr2=0x55f7be13d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:16.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 -- 172.21.15.114:0/2941795946 shutdown_connections 2024-09-21T07:04:16.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.383+0000 7f2211bc9740 1 -- 172.21.15.114:0/2941795946 wait complete. 2024-09-21T07:04:16.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.384+0000 7f2211bc9740 1 -- start start 2024-09-21T07:04:16.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.385+0000 7f2211bc9740 1 Processor -- start 2024-09-21T07:04:16.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.385+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be14ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.385+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd76b60 0x55f7be14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.385+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2211bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f7bdf057a0 con 0x55f7bdd97840 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2211bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7be049a10 con 0x55f7bdd6e980 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2211bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f7be15ec90 con 0x55f7bdd76b60 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2211bc9740 10 client.0 _ll_register_callbacks cb 0x55f7be059be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2203e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be14ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2203e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be14ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41840/0 (socket says 172.21.15.114:41840) 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2203e00640 1 -- 172.21.15.114:0/2835321238 learned_addr learned my addr 172.21.15.114:0/2835321238 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:16.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2208e00640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2203400640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd76b60 0x55f7be14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.386+0000 7f2203e00640 1 -- 172.21.15.114:0/2835321238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd76b60 msgr2=0x55f7be14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f2203e00640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd76b60 0x55f7be14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f2203e00640 1 -- 172.21.15.114:0/2835321238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 msgr2=0x55f7be167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f2203e00640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be167fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f2203e00640 1 -- 172.21.15.114:0/2835321238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7bde11b90 con 0x55f7bdd6e980 2024-09-21T07:04:16.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f2208e00640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f2203e00640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be14ac60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f21f801d9c0 tx=0x7f21f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b688f29757bc6eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.387+0000 7f21ea000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.388+0000 7f21ea000640 1 -- 172.21.15.114:0/2835321238 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f802d060 con 0x55f7bdd6e980 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.388+0000 7f21ea000640 1 -- 172.21.15.114:0/2835321238 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f21f8018030 con 0x55f7bdd6e980 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.388+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220409c040 con 0x55f7bdd6e980 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.388+0000 7f21ea000640 1 -- 172.21.15.114:0/2835321238 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f21f8028420 con 0x55f7bdd6e980 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.389+0000 7f2211bc9740 10 client.28031 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:16.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.389+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f7be16aaf0 con 0x55f7bdd6e980 2024-09-21T07:04:16.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.390+0000 7f21ea000640 1 -- 172.21.15.114:0/2835321238 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220409c040 con 0x55f7bdd6e980 2024-09-21T07:04:16.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.390+0000 7f21ea000640 1 -- 172.21.15.114:0/2835321238 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f21f80315f0 con 0x55f7bdd6e980 2024-09-21T07:04:16.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.391+0000 7f2211bc9740 10 client.28031 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:16.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.391+0000 7f2211bc9740 10 client.28031 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:04:16.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.391+0000 7f2211bc9740 20 client.28031 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:16.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.391+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd561c0 0x55f7be16df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.391+0000 7f2211bc9740 4 client.28031 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:16.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.391+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7be0b6bf0 con 0x55f7bdd561c0 2024-09-21T07:04:16.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.392+0000 7f2203400640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd561c0 0x55f7be16df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.393+0000 7f21ea000640 10 client.28031 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:16.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.393+0000 7f2203400640 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd561c0 0x55f7be16df80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17f473c704256d7 server_cookie=bd71e9660d5fbaa5 in_seq=0 out_seq=0 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.394+0000 7f21ea000640 1 -- 172.21.15.114:0/2835321238 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7be0b6bf0 con 0x55f7bdd561c0 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.394+0000 7f21ea000640 10 client.28031 handle_command_reply: tid=0 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.394+0000 7f2211bc9740 1 client.28031 shutdown 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 msgr2=0x55f7be14ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be14ac60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f21f801d9c0 tx=0x7f21f8004730 comp rx=0 tx=0).stop 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 shutdown_connections 2024-09-21T07:04:16.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd561c0 0x55f7be16df80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd76b60 0x55f7be14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 --2- 172.21.15.114:0/2835321238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd6e980 0x55f7be14ac60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 >> 172.21.15.114:0/2835321238 conn(0x55f7bdd8b990 msgr2=0x55f7be13bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:16.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f21ea000640 0 client.28031 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:16.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 shutdown_connections 2024-09-21T07:04:16.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.395+0000 7f2211bc9740 1 -- 172.21.15.114:0/2835321238 wait complete. 2024-09-21T07:04:16.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.396+0000 7f2211bc9740 20 client.28031 trim_cache size 0 max 16384 2024-09-21T07:04:16.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.420+0000 7f2211bc9740 1 Processor -- start 2024-09-21T07:04:16.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.420+0000 7f2211bc9740 1 -- start start 2024-09-21T07:04:16.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.420+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be147e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2211bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc149a70 con 0x55f7bdd97840 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2211bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc14b5b0 con 0x55f7bdd76b60 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2211bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc14a140 con 0x55f7bdd6e980 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2203e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2208e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2203e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be14cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46058/0 (socket says 172.21.15.114:46058) 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2203e00640 1 -- 172.21.15.114:0/3217245376 learned_addr learned my addr 172.21.15.114:0/3217245376 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:16.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.421+0000 7f2203400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203400640 1 -- 172.21.15.114:0/3217245376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 msgr2=0x55f7be14cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203400640 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be14cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203400640 1 -- 172.21.15.114:0/3217245376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 msgr2=0x55f7be147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203400640 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203400640 1 -- 172.21.15.114:0/3217245376 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7be141c30 con 0x55f7bdd76b60 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203e00640 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.422+0000 7f2203400640 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be167ee0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f21f00e5010 tx=0x7f21f00a9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c5a9e9fd178a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:16.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.423+0000 7f21f4c00640 1 -- 172.21.15.114:0/3217245376 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f00b21f0 con 0x55f7bdd76b60 2024-09-21T07:04:16.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.423+0000 7f21f4c00640 1 -- 172.21.15.114:0/3217245376 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f00b7130 con 0x55f7bdd76b60 2024-09-21T07:04:16.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.423+0000 7f21f4c00640 1 -- 172.21.15.114:0/3217245376 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f00ac490 con 0x55f7bdd76b60 2024-09-21T07:04:16.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 -- 172.21.15.114:0/3217245376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 msgr2=0x55f7be167ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be167ee0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f21f00e5010 tx=0x7f21f00a9040 comp rx=0 tx=0).stop 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 -- 172.21.15.114:0/3217245376 shutdown_connections 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be167ee0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 --2- 172.21.15.114:0/3217245376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be14cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.424+0000 7f2211bc9740 1 -- 172.21.15.114:0/3217245376 >> 172.21.15.114:0/3217245376 conn(0x55f7bdd8b990 msgr2=0x55f7be13bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.425+0000 7f2211bc9740 1 -- 172.21.15.114:0/3217245376 shutdown_connections 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.425+0000 7f2211bc9740 1 -- 172.21.15.114:0/3217245376 wait complete. 2024-09-21T07:04:16.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.425+0000 7f2211bc9740 1 -- start start 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.425+0000 7f2211bc9740 1 Processor -- start 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be1d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc14aee0 con 0x55f7bdd97840 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc14a810 con 0x55f7bdd76b60 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc147a70 con 0x55f7bdd6e980 2024-09-21T07:04:16.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2211bc9740 10 client.0 _ll_register_callbacks cb 0x7f21f801d880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.426+0000 7f2208e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be1d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be1d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37454/0 (socket says 172.21.15.114:37454) 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 -- 172.21.15.114:0/1339436170 learned_addr learned my addr 172.21.15.114:0/1339436170 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2203400640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2203e00640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 -- 172.21.15.114:0/1339436170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 msgr2=0x55f7be160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 -- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 msgr2=0x55f7be160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:16.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 -- 172.21.15.114:0/1339436170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7be1d1f80 con 0x55f7bdd97840 2024-09-21T07:04:16.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.427+0000 7f2208e00640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be1d1910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2204052910 tx=0x7f220406ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244b4ea70704d9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:16.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f21ea000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f21ea000640 1 -- 172.21.15.114:0/1339436170 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040b0020 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f21ea000640 1 -- 172.21.15.114:0/1339436170 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f2204075040 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f00b7130 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f21ea000640 1 -- 172.21.15.114:0/1339436170 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f22040a1530 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f2211bc9740 10 client.17565 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.428+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f7be1d92a0 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.429+0000 7f21ea000640 1 -- 172.21.15.114:0/1339436170 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f00b7130 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.429+0000 7f21ea000640 1 -- 172.21.15.114:0/1339436170 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f22040a17b0 con 0x55f7bdd97840 2024-09-21T07:04:16.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.429+0000 7f2211bc9740 10 client.17565 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.429+0000 7f2211bc9740 10 client.17565 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:04:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.430+0000 7f2211bc9740 20 client.17565 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.430+0000 7f2211bc9740 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd2b570 0x55f7be1db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.430+0000 7f2211bc9740 4 client.17565 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T07:04:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.430+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55f7be0b26a0 con 0x55f7bdd2b570 2024-09-21T07:04:16.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.430+0000 7f2203e00640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd2b570 0x55f7be1db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:16.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.432+0000 7f21ea000640 10 client.17565 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:16.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.433+0000 7f2203e00640 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd2b570 0x55f7be1db5c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11f3ee070387564 server_cookie=38a93103f4fdf493 in_seq=0 out_seq=0 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.599+0000 7f21ea000640 1 -- 172.21.15.114:0/1339436170 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f7be0b26a0 con 0x55f7bdd2b570 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.599+0000 7f21ea000640 10 client.17565 handle_command_reply: tid=0 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.599+0000 7f2211bc9740 1 client.17565 shutdown 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 msgr2=0x55f7be1d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be1d1910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2204052910 tx=0x7f220406ad20 comp rx=0 tx=0).stop 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 shutdown_connections 2024-09-21T07:04:16.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f7bdd97840 0x55f7be1d1910 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7bdd76b60 0x55f7be160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f7bdd2b570 0x55f7be1db5c0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 --2- 172.21.15.114:0/1339436170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7bdd6e980 0x55f7be160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 >> 172.21.15.114:0/1339436170 conn(0x55f7bdd8b990 msgr2=0x55f7be14bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f21ea000640 0 client.17565 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 shutdown_connections 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 1 -- 172.21.15.114:0/1339436170 wait complete. 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.600+0000 7f2211bc9740 20 client.17565 trim_cache size 0 max 16384 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:04:16.604 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 49, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 38, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "230636ca": { 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 38, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 144.5, 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:04:16.605 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:16.606 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:04:16.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f21ec0b6850 msgr2=0x7f21ec0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f21ec0b6850 0x7f21ec0b8d60 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55f7be13e180 tx=0x7f21f8009260 comp rx=0 tx=0).stop 2024-09-21T07:04:16.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 msgr2=0x7f21fc133580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:16.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 0x7f21fc133580 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f21fc005ae0 tx=0x7f21f000ee50 comp rx=0 tx=0).stop 2024-09-21T07:04:16.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 shutdown_connections 2024-09-21T07:04:16.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f21fc0a2970 0x7f21fc138820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f21ec0b6850 0x7f21ec0b8d60 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21fc0a2020 0x7f21fc133580 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 --2- 172.21.15.114:0/4256268033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21fc09c930 0x7f21fc132fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:16.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.601+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 >> 172.21.15.114:0/4256268033 conn(0x7f21fc00a6b0 msgr2=0x7f21fc0a0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:16.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.602+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 shutdown_connections 2024-09-21T07:04:16.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:16.602+0000 7f2209800640 1 -- 172.21.15.114:0/4256268033 wait complete. 2024-09-21T07:04:16.615 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:17.394 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:17 smithi114 ceph-mon[30654]: pgmap v708: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-21T07:04:17.394 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:17 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:17.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:17 smithi193 ceph-mon[34601]: pgmap v708: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-21T07:04:17.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:17 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:17.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:17 smithi183 ceph-mon[34219]: pgmap v708: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-21T07:04:17.417 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:17 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:04:17.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 -- 172.21.15.114:0/386875993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 msgr2=0x7f1d6c116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 --2- 172.21.15.114:0/386875993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 0x7f1d6c116b60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ca0 tx=0x7f1d6402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 -- 172.21.15.114:0/386875993 shutdown_connections 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 --2- 172.21.15.114:0/386875993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 0x7f1d6c116b60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 --2- 172.21.15.114:0/386875993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 --2- 172.21.15.114:0/386875993 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.729+0000 7f1d73a00640 1 -- 172.21.15.114:0/386875993 >> 172.21.15.114:0/386875993 conn(0x7f1d6c101810 msgr2=0x7f1d6c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.730+0000 7f1d73a00640 1 -- 172.21.15.114:0/386875993 shutdown_connections 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.730+0000 7f1d73a00640 1 -- 172.21.15.114:0/386875993 wait complete. 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 Processor -- start 2024-09-21T07:04:17.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 -- start start 2024-09-21T07:04:17.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 0x7f1d6c1a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c11a100 con 0x7f1d6c077940 2024-09-21T07:04:17.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c119f80 con 0x7f1d6c078290 2024-09-21T07:04:17.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.731+0000 7f1d73a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c11a280 con 0x7f1d6c114740 2024-09-21T07:04:17.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c1a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37468/0 (socket says 172.21.15.114:37468) 2024-09-21T07:04:17.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 0x7f1d6c1a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 -- 172.21.15.114:0/1159298695 learned_addr learned my addr 172.21.15.114:0/1159298695 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:17.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c1a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41882/0 (socket says 172.21.15.114:41882) 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 -- 172.21.15.114:0/1159298695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 msgr2=0x7f1d6c1a6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 0x7f1d6c1a6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 -- 172.21.15.114:0/1159298695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 msgr2=0x7f1d6c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 -- 172.21.15.114:0/1159298695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c1a69a0 con 0x7f1d6c077940 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d63e00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.732+0000 7f1d71000640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c1a0a30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9f0 tx=0x7f1d5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.733+0000 7f1d6a000640 1 -- 172.21.15.114:0/1159298695 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00cd80 con 0x7f1d6c077940 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.733+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c1a6c90 con 0x7f1d6c077940 2024-09-21T07:04:17.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.733+0000 7f1d6a000640 1 -- 172.21.15.114:0/1159298695 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c0108e0 con 0x7f1d6c077940 2024-09-21T07:04:17.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.733+0000 7f1d6a000640 1 -- 172.21.15.114:0/1159298695 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c018980 con 0x7f1d6c077940 2024-09-21T07:04:17.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.733+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c1ad3f0 con 0x7f1d6c077940 2024-09-21T07:04:17.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.737+0000 7f1d6a000640 1 -- 172.21.15.114:0/1159298695 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1d5c010430 con 0x7f1d6c077940 2024-09-21T07:04:17.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.737+0000 7f1d7a635740 1 Processor -- start 2024-09-21T07:04:17.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.737+0000 7f1d7a635740 1 -- start start 2024-09-21T07:04:17.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 0x557dbe2d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d7a635740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557dbe247fb0 con 0x557dbdf2a070 2024-09-21T07:04:17.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d7a635740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557dbe2373e0 con 0x557dbdf09390 2024-09-21T07:04:17.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d7a635740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557dbe23b1f0 con 0x557dbdf011b0 2024-09-21T07:04:17.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d71000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d71a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 0x557dbe2d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d71a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 0x557dbe2d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37478/0 (socket says 172.21.15.114:37478) 2024-09-21T07:04:17.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d71000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46076/0 (socket says 172.21.15.114:46076) 2024-09-21T07:04:17.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d71a00640 1 -- 172.21.15.114:0/3621856600 learned_addr learned my addr 172.21.15.114:0/3621856600 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d63e00640 1 -- 172.21.15.114:0/3621856600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 msgr2=0x557dbe2d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d63e00640 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d63e00640 1 -- 172.21.15.114:0/3621856600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 msgr2=0x557dbe2d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d63e00640 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 0x557dbe2d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.738+0000 7f1d63e00640 1 -- 172.21.15.114:0/3621856600 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe201cd0 con 0x557dbdf09390 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d71a00640 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 0x557dbe2d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d6a000640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d500b6680 0x7f1d500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d6a000640 1 -- 172.21.15.114:0/1159298695 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f1d5c0e7e70 con 0x7f1d6c077940 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d63e00640 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2d1820 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c06c810 tx=0x7f1d540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0be98621f14f4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d71a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d500b6680 0x7f1d500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d62000640 1 -- 172.21.15.114:0/3621856600 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004030 con 0x557dbdf09390 2024-09-21T07:04:17.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d62000640 1 -- 172.21.15.114:0/3621856600 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d540054d0 con 0x557dbdf09390 2024-09-21T07:04:17.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.739+0000 7f1d62000640 1 -- 172.21.15.114:0/3621856600 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54004ce0 con 0x557dbdf09390 2024-09-21T07:04:17.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 -- 172.21.15.114:0/3621856600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 msgr2=0x557dbe2d1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d71a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d500b6680 0x7f1d500b8b90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1d6402aeb0 tx=0x7f1d64005d80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:17.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2d1820 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c06c810 tx=0x7f1d540118d0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 -- 172.21.15.114:0/3621856600 shutdown_connections 2024-09-21T07:04:17.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2d1820 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3621856600 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf2a070 0x557dbe2d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 -- 172.21.15.114:0/3621856600 >> 172.21.15.114:0/3621856600 conn(0x557dbdf1e460 msgr2=0x557dbe2cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 -- 172.21.15.114:0/3621856600 shutdown_connections 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.740+0000 7f1d7a635740 1 -- 172.21.15.114:0/3621856600 wait complete. 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.741+0000 7f1d7a635740 1 -- start start 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.741+0000 7f1d7a635740 1 Processor -- start 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2dc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2dca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf2a070 0x557dbe2e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557dbe098700 con 0x557dbdf011b0 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557dbe1d2580 con 0x557dbdf09390 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557dbe2df580 con 0x557dbdf2a070 2024-09-21T07:04:17.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d7a635740 10 client.0 _ll_register_callbacks cb 0x557dbe1e2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2dca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d71000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2dc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2dca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41892/0 (socket says 172.21.15.114:41892) 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d63e00640 1 -- 172.21.15.114:0/257577171 learned_addr learned my addr 172.21.15.114:0/257577171 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d71a00640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf2a070 0x557dbe2e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d63e00640 1 -- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf2a070 msgr2=0x557dbe2e8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d63e00640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf2a070 0x557dbe2e8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.742+0000 7f1d63e00640 1 -- 172.21.15.114:0/257577171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 msgr2=0x557dbe2dc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d63e00640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2dc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d63e00640 1 -- 172.21.15.114:0/257577171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbdf95650 con 0x557dbdf09390 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d71a00640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf2a070 0x557dbe2e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:17.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d71000640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d63e00640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2dca40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1d54000c00 tx=0x7f1d5400fbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c5140a703b3b98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d5aa00640 1 -- 172.21.15.114:0/257577171 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54028020 con 0x557dbdf09390 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dbe1d4b90 con 0x557dbdf09390 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d5aa00640 1 -- 172.21.15.114:0/257577171 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f1d54024480 con 0x557dbdf09390 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d5aa00640 1 -- 172.21.15.114:0/257577171 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1d54033050 con 0x557dbdf09390 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d7a635740 10 client.28049 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.743+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557dbe1ef950 con 0x557dbdf09390 2024-09-21T07:04:17.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d5aa00640 1 -- 172.21.15.114:0/257577171 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557dbe1d4b90 con 0x557dbdf09390 2024-09-21T07:04:17.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d5aa00640 1 -- 172.21.15.114:0/257577171 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f1d540194e0 con 0x557dbdf09390 2024-09-21T07:04:17.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d7a635740 10 client.28049 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:17.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d7a635740 10 client.28049 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:04:17.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d7a635740 20 client.28049 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:17.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d7a635740 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdee89f0 0x557dbe2ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d7a635740 4 client.28049 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dbe249f00 con 0x557dbdee89f0 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.744+0000 7f1d71000640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdee89f0 0x557dbe2ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.745+0000 7f1d71000640 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdee89f0 0x557dbe2ee580 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75f2ee35681ffd4 server_cookie=6f25b33d6ba00288 in_seq=0 out_seq=0 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.745+0000 7f1d5aa00640 10 client.28049 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.745+0000 7f1d5aa00640 1 -- 172.21.15.114:0/257577171 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557dbe249f00 con 0x557dbdee89f0 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.745+0000 7f1d5aa00640 10 client.28049 handle_command_reply: tid=0 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 client.28049 shutdown 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 msgr2=0x557dbe2dca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2dca40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1d54000c00 tx=0x7f1d5400fbc0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 shutdown_connections 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdee89f0 0x557dbe2ee580 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf2a070 0x557dbe2e8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf09390 0x557dbe2dca40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 --2- 172.21.15.114:0/257577171 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 >> 172.21.15.114:0/257577171 conn(0x557dbdf1e460 msgr2=0x557dbe2d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d5aa00640 0 client.28049 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 shutdown_connections 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 1 -- 172.21.15.114:0/257577171 wait complete. 2024-09-21T07:04:17.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.746+0000 7f1d7a635740 20 client.28049 trim_cache size 0 max 16384 2024-09-21T07:04:17.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.774+0000 7f1d7a635740 1 Processor -- start 2024-09-21T07:04:17.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 -- start start 2024-09-21T07:04:17.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf09390 0x557dbe2e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1b7620 con 0x557dbdf09390 2024-09-21T07:04:17.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1b9160 con 0x557dbdf2a070 2024-09-21T07:04:17.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.775+0000 7f1d7a635740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1b7cf0 con 0x557dbdf011b0 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46094/0 (socket says 172.21.15.114:46094) 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 -- 172.21.15.114:0/3233065753 learned_addr learned my addr 172.21.15.114:0/3233065753 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d63e00640 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf09390 0x557dbe2e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 -- 172.21.15.114:0/3233065753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 msgr2=0x557dbe2d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 -- 172.21.15.114:0/3233065753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf09390 msgr2=0x557dbe2e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf09390 0x557dbe2e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.776+0000 7f1d71000640 1 -- 172.21.15.114:0/3233065753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dbe2dd430 con 0x557dbdf011b0 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.777+0000 7f1d71000640 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d24e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0ad640 tx=0x7f1d5c0a9040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c992996848e5794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.777+0000 7f1d61600640 1 -- 172.21.15.114:0/3233065753 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0ad290 con 0x557dbdf011b0 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.777+0000 7f1d61600640 1 -- 172.21.15.114:0/3233065753 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c010720 con 0x557dbdf011b0 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.777+0000 7f1d61600640 1 -- 172.21.15.114:0/3233065753 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0a90b0 con 0x557dbdf011b0 2024-09-21T07:04:17.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.778+0000 7f1d7a635740 1 -- 172.21.15.114:0/3233065753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 msgr2=0x557dbe2d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.778+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d24e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0ad640 tx=0x7f1d5c0a9040 comp rx=0 tx=0).stop 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 -- 172.21.15.114:0/3233065753 shutdown_connections 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf09390 0x557dbe2e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 --2- 172.21.15.114:0/3233065753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf011b0 0x557dbe2d24e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 -- 172.21.15.114:0/3233065753 >> 172.21.15.114:0/3233065753 conn(0x557dbdf1e460 msgr2=0x557dbe2d1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 -- 172.21.15.114:0/3233065753 shutdown_connections 2024-09-21T07:04:17.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.779+0000 7f1d7a635740 1 -- 172.21.15.114:0/3233065753 wait complete. 2024-09-21T07:04:17.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.780+0000 7f1d7a635740 1 -- start start 2024-09-21T07:04:17.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.780+0000 7f1d7a635740 1 Processor -- start 2024-09-21T07:04:17.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf09390 0x557dbe2e0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1b8a90 con 0x557dbdf011b0 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1b83c0 con 0x557dbdf2a070 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c1b5620 con 0x557dbdf09390 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d7a635740 10 client.0 _ll_register_callbacks cb 0x7f1d54009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d71a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.781+0000 7f1d71a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2e0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41898/0 (socket says 172.21.15.114:41898) 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71a00640 1 -- 172.21.15.114:0/1468997917 learned_addr learned my addr 172.21.15.114:0/1468997917 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71000640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71a00640 1 -- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf09390 msgr2=0x557dbe2e0300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:04:17.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d63e00640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf09390 0x557dbe2e0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71a00640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf09390 0x557dbe2e0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71a00640 1 -- 172.21.15.114:0/1468997917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 msgr2=0x557dbe2e4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:17.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71a00640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2e4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.782+0000 7f1d71a00640 1 -- 172.21.15.114:0/1468997917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dbe2e0e00 con 0x557dbdf2a070 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.783+0000 7f1d71a00640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2e0840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ca0 tx=0x7f1d640095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac564803f7d2bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.783+0000 7f1d5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.783+0000 7f1d5aa00640 1 -- 172.21.15.114:0/1468997917 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64059060 con 0x557dbdf2a070 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.783+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dbe2459b0 con 0x557dbdf2a070 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.784+0000 7f1d5aa00640 1 -- 172.21.15.114:0/1468997917 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f1d64003dc0 con 0x557dbdf2a070 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.784+0000 7f1d5aa00640 1 -- 172.21.15.114:0/1468997917 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1d64060050 con 0x557dbdf2a070 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.784+0000 7f1d7a635740 10 client.28055 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:17.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.784+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557dbe358bd0 con 0x557dbdf2a070 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d5aa00640 1 -- 172.21.15.114:0/1468997917 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557dbe2459b0 con 0x557dbdf2a070 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d5aa00640 1 -- 172.21.15.114:0/1468997917 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f1d6402fa30 con 0x557dbdf2a070 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d7a635740 10 client.28055 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d7a635740 10 client.28055 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d7a635740 20 client.28055 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d7a635740 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdebdda0 0x557dbe35b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:17.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d7a635740 4 client.28055 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:04:17.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557dbe242de0 con 0x557dbdebdda0 2024-09-21T07:04:17.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.785+0000 7f1d71000640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdebdda0 0x557dbe35b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.786+0000 7f1d71000640 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdebdda0 0x557dbe35b3a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d90e05bceb00cec4 server_cookie=421fb5abd722907f in_seq=0 out_seq=0 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.786+0000 7f1d5aa00640 10 client.28055 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.786+0000 7f1d5aa00640 1 -- 172.21.15.114:0/1468997917 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557dbe242de0 con 0x557dbdebdda0 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.786+0000 7f1d5aa00640 10 client.28055 handle_command_reply: tid=0 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.786+0000 7f1d7a635740 1 client.28055 shutdown 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 msgr2=0x557dbe2e0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2e0840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009ca0 tx=0x7f1d640095c0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 shutdown_connections 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x557dbdebdda0 0x557dbe35b3a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557dbdf2a070 0x557dbe2e0840 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557dbdf09390 0x557dbe2e0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d5aa00640 0 client.28055 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:04:17.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 --2- 172.21.15.114:0/1468997917 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557dbdf011b0 0x557dbe2e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 >> 172.21.15.114:0/1468997917 conn(0x557dbdf1e460 msgr2=0x557dbe2dd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:17.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 shutdown_connections 2024-09-21T07:04:17.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 1 -- 172.21.15.114:0/1468997917 wait complete. 2024-09-21T07:04:17.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.787+0000 7f1d7a635740 20 client.28055 trim_cache size 0 max 16384 2024-09-21T07:04:17.792 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:04:17.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d500b6680 msgr2=0x7f1d500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d500b6680 0x7f1d500b8b90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1d6402aeb0 tx=0x7f1d64005d80 comp rx=0 tx=0).stop 2024-09-21T07:04:17.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 msgr2=0x7f1d6c1a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:17.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c1a0a30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e9f0 tx=0x7f1d5c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 shutdown_connections 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d6c114740 0x7f1d6c1a6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1d500b6680 0x7f1d500b8b90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6c078290 0x7f1d6c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 --2- 172.21.15.114:0/1159298695 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1d6c077940 0x7f1d6c1a0a30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.788+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 >> 172.21.15.114:0/1159298695 conn(0x7f1d6c101810 msgr2=0x7f1d6c116730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.789+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 shutdown_connections 2024-09-21T07:04:17.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:17.789+0000 7f1d73a00640 1 -- 172.21.15.114:0/1159298695 wait complete. 2024-09-21T07:04:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 580.0 s, 295.8 tps, lat 254.402 ms stddev 315.487 2024-09-21T07:04:18.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:18 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:18 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:18 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:18 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:18 smithi114 ceph-mon[30654]: pgmap v709: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.8 MiB/s wr, 385 op/s 2024-09-21T07:04:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:18 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:18 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:18 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:18 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:18 smithi183 ceph-mon[34219]: pgmap v709: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.8 MiB/s wr, 385 op/s 2024-09-21T07:04:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:18 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:18 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:18 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:18 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:18.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:18 smithi193 ceph-mon[34601]: pgmap v709: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 3.8 MiB/s wr, 385 op/s 2024-09-21T07:04:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 -- 172.21.15.114:0/2768311081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 msgr2=0x7f17101100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 --2- 172.21.15.114:0/2768311081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f17101100a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1708009ca0 tx=0x7f170802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 -- 172.21.15.114:0/2768311081 shutdown_connections 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 --2- 172.21.15.114:0/2768311081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f17101100a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 --2- 172.21.15.114:0/2768311081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17101050f0 0x7f171010d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 --2- 172.21.15.114:0/2768311081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101047a0 0x7f1710104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 -- 172.21.15.114:0/2768311081 >> 172.21.15.114:0/2768311081 conn(0x7f17100fe230 msgr2=0x7f1710100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 -- 172.21.15.114:0/2768311081 shutdown_connections 2024-09-21T07:04:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.928+0000 7f170fe00640 1 -- 172.21.15.114:0/2768311081 wait complete. 2024-09-21T07:04:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.929+0000 7f170fe00640 1 Processor -- start 2024-09-21T07:04:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.929+0000 7f170fe00640 1 -- start start 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17101047a0 0x7f171019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101050f0 0x7f171019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f171019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1710113840 con 0x7f171010dbc0 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17101136c0 con 0x7f17101050f0 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17101139c0 con 0x7f17101047a0 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f171019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f171019fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37544/0 (socket says 172.21.15.114:37544) 2024-09-21T07:04:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.930+0000 7f170f400640 1 -- 172.21.15.114:0/3686394528 learned_addr learned my addr 172.21.15.114:0/3686394528 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:18.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170e000640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101050f0 0x7f171019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170f400640 1 -- 172.21.15.114:0/3686394528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17101047a0 msgr2=0x7f171019a220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170f400640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17101047a0 0x7f171019a220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170f400640 1 -- 172.21.15.114:0/3686394528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101050f0 msgr2=0x7f171019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170f400640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101050f0 0x7f171019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170f400640 1 -- 172.21.15.114:0/3686394528 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17101a0150 con 0x7f171010dbc0 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170e000640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101050f0 0x7f171019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f170f400640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f171019fa50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1708005cc0 tx=0x7f1708031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.931+0000 7f1714a00640 1 -- 172.21.15.114:0/3686394528 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708003b70 con 0x7f171010dbc0 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.932+0000 7f1714a00640 1 -- 172.21.15.114:0/3686394528 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17080318f0 con 0x7f171010dbc0 2024-09-21T07:04:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.932+0000 7f1714a00640 1 -- 172.21.15.114:0/3686394528 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170802f950 con 0x7f171010dbc0 2024-09-21T07:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.932+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17101a03e0 con 0x7f171010dbc0 2024-09-21T07:04:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.932+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101a6bd0 con 0x7f171010dbc0 2024-09-21T07:04:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.935+0000 7f171ceb8740 1 Processor -- start 2024-09-21T07:04:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.936+0000 7f1714a00640 1 -- 172.21.15.114:0/3686394528 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f170802fb50 con 0x7f171010dbc0 2024-09-21T07:04:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.936+0000 7f171ceb8740 1 -- start start 2024-09-21T07:04:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 0x5563236d7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236d8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f171ceb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55632364d6b0 con 0x5563233331b0 2024-09-21T07:04:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f171ceb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55632363cae0 con 0x5563233124d0 2024-09-21T07:04:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f171ceb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563236408f0 con 0x55632330a2f0 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 0x5563236d7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 0x5563236d7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37554/0 (socket says 172.21.15.114:37554) 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 -- 172.21.15.114:0/2437749208 learned_addr learned my addr 172.21.15.114:0/2437749208 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170ea00640 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236d8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 -- 172.21.15.114:0/2437749208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 msgr2=0x5563236d8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236d8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 -- 172.21.15.114:0/2437749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 msgr2=0x5563236d8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236d8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.937+0000 7f170e000640 1 -- 172.21.15.114:0/2437749208 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556323609780 con 0x5563233331b0 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f170e000640 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 0x5563236d7e10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0139a0 tx=0x7f16fc009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b748514afc8a54c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f1714a00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b6680 0x7f16f40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f16fbe00640 1 -- 172.21.15.114:0/2437749208 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc004050 con 0x5563233331b0 2024-09-21T07:04:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f16fbe00640 1 -- 172.21.15.114:0/2437749208 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc00fd70 con 0x5563233331b0 2024-09-21T07:04:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f16fbe00640 1 -- 172.21.15.114:0/2437749208 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc005440 con 0x5563233331b0 2024-09-21T07:04:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f1714a00640 1 -- 172.21.15.114:0/3686394528 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f1708108cf0 con 0x7f171010dbc0 2024-09-21T07:04:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.938+0000 7f170ea00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b6680 0x7f16f40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 -- 172.21.15.114:0/2437749208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 msgr2=0x5563236d7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 0x5563236d7e10 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0139a0 tx=0x7f16fc009be0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f170ea00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b6680 0x7f16f40b8b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5563236d6a90 tx=0x7f1700005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 -- 172.21.15.114:0/2437749208 shutdown_connections 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236d8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 --2- 172.21.15.114:0/2437749208 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233331b0 0x5563236d7e10 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 -- 172.21.15.114:0/2437749208 >> 172.21.15.114:0/2437749208 conn(0x556323327300 msgr2=0x5563236d5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 -- 172.21.15.114:0/2437749208 shutdown_connections 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.939+0000 7f171ceb8740 1 -- 172.21.15.114:0/2437749208 wait complete. 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.940+0000 7f171ceb8740 1 -- start start 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.940+0000 7f171ceb8740 1 Processor -- start 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.940+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 0x5563236e3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.940+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236e3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.940+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f171ceb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556323574ab0 con 0x55632330a2f0 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f171ceb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563235e3570 con 0x5563233124d0 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f171ceb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563236e6590 con 0x5563233331b0 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f171ceb8740 10 client.0 _ll_register_callbacks cb 0x5563235f2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 0x5563236e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 0x5563236e3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37568/0 (socket says 172.21.15.114:37568) 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 -- 172.21.15.114:0/3070619290 learned_addr learned my addr 172.21.15.114:0/3070619290 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170e000640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170f400640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 -- 172.21.15.114:0/3070619290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 msgr2=0x5563236e3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 -- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 msgr2=0x5563236e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 -- 172.21.15.114:0/3070619290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632339e790 con 0x55632330a2f0 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170f400640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f170ea00640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 0x5563236e3430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1700003850 tx=0x7f1700004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd56676b11e9090b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.941+0000 7f16f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.942+0000 7f16f8c00640 1 -- 172.21.15.114:0/3070619290 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170000eab0 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.942+0000 7f16f8c00640 1 -- 172.21.15.114:0/3070619290 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f170000ec50 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.942+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc00fd70 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.942+0000 7f16f8c00640 1 -- 172.21.15.114:0/3070619290 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f170000ee50 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.942+0000 7f171ceb8740 10 client.17595 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.942+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556323417620 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f16f8c00640 1 -- 172.21.15.114:0/3070619290 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc00fd70 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f16f8c00640 1 -- 172.21.15.114:0/3070619290 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f1700026660 con 0x55632330a2f0 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f171ceb8740 10 client.17595 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f171ceb8740 10 client.17595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f171ceb8740 20 client.17595 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232f1d80 0x5563236f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f171ceb8740 4 client.17595 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.943+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55632364f600 con 0x5563232f1d80 2024-09-21T07:04:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.944+0000 7f170e000640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232f1d80 0x5563236f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.945+0000 7f170e000640 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232f1d80 0x5563236f9a50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2638046fb6b344a server_cookie=8c1a5b907c7bc486 in_seq=0 out_seq=0 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.945+0000 7f16f8c00640 10 client.17595 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f16f8c00640 1 -- 172.21.15.114:0/3070619290 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55632364f600 con 0x5563232f1d80 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f16f8c00640 10 client.17595 handle_command_reply: tid=0 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 client.17595 shutdown 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 msgr2=0x5563236e3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 0x5563236e3430 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1700003850 tx=0x7f1700004730 comp rx=0 tx=0).stop 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 shutdown_connections 2024-09-21T07:04:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232f1d80 0x5563236f9a50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233124d0 0x5563236e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f16f8c00640 0 client.17595 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3070619290 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55632330a2f0 0x5563236e3430 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 >> 172.21.15.114:0/3070619290 conn(0x556323327300 msgr2=0x5563236d8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 shutdown_connections 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 1 -- 172.21.15.114:0/3070619290 wait complete. 2024-09-21T07:04:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.946+0000 7f171ceb8740 20 client.17595 trim_cache size 0 max 16384 2024-09-21T07:04:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.961+0000 7f171ceb8740 1 Processor -- start 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.961+0000 7f171ceb8740 1 -- start start 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55632330a2f0 0x5563236d9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f171ceb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f17101aea90 con 0x5563233124d0 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f171ceb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17101b05d0 con 0x55632330a2f0 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f171ceb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17101af160 con 0x5563233331b0 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236e0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236e0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37576/0 (socket says 172.21.15.114:37576) 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 -- 172.21.15.114:0/3043793683 learned_addr learned my addr 172.21.15.114:0/3043793683 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170f400640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170ea00640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55632330a2f0 0x5563236d9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.966 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 -- 172.21.15.114:0/3043793683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 msgr2=0x5563236e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 -- 172.21.15.114:0/3043793683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55632330a2f0 msgr2=0x5563236d9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55632330a2f0 0x5563236d9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170e000640 1 -- 172.21.15.114:0/3043793683 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563236dac20 con 0x5563233124d0 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170f400640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.962+0000 7f170ea00640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55632330a2f0 0x5563236d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.963+0000 7f170e000640 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236e0600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0139a0 tx=0x7f16fc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddb1fc7e29c40be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.963+0000 7f16fb400640 1 -- 172.21.15.114:0/3043793683 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc02f070 con 0x5563233124d0 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.963+0000 7f16fb400640 1 -- 172.21.15.114:0/3043793683 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc00f790 con 0x5563233124d0 2024-09-21T07:04:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.963+0000 7f16fb400640 1 -- 172.21.15.114:0/3043793683 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0048e0 con 0x5563233124d0 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 -- 172.21.15.114:0/3043793683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 msgr2=0x5563236e0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236e0600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0139a0 tx=0x7f16fc004610 comp rx=0 tx=0).stop 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 -- 172.21.15.114:0/3043793683 shutdown_connections 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563233331b0 0x5563236e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236e0600 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3043793683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55632330a2f0 0x5563236d9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 -- 172.21.15.114:0/3043793683 >> 172.21.15.114:0/3043793683 conn(0x556323327300 msgr2=0x5563236d8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 -- 172.21.15.114:0/3043793683 shutdown_connections 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 -- 172.21.15.114:0/3043793683 wait complete. 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.964+0000 7f171ceb8740 1 -- start start 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 Processor -- start 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233331b0 0x5563236edbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f17101aff00 con 0x5563233124d0 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17101af830 con 0x5563233331b0 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f17101aca90 con 0x55632330a2f0 2024-09-21T07:04:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f171ceb8740 10 client.0 _ll_register_callbacks cb 0x7f17000098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f170e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f170e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37580/0 (socket says 172.21.15.114:37580) 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f170e000640 1 -- 172.21.15.114:0/3111138336 learned_addr learned my addr 172.21.15.114:0/3111138336 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.965+0000 7f170ea00640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170f400640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233331b0 0x5563236edbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170e000640 1 -- 172.21.15.114:0/3111138336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 msgr2=0x5563236f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170e000640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170e000640 1 -- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233331b0 msgr2=0x5563236edbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170e000640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233331b0 0x5563236edbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170e000640 1 -- 172.21.15.114:0/3111138336 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563236ee120 con 0x5563233124d0 2024-09-21T07:04:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f170e000640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236ed6a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00f940 tx=0x7f16fc00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291018319b09003a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f16f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f16f8c00640 1 -- 172.21.15.114:0/3111138336 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc02f040 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f16f8c00640 1 -- 172.21.15.114:0/3111138336 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f16fc005240 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.966+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc00f790 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f16f8c00640 1 -- 172.21.15.114:0/3111138336 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f16fc00fe60 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f171ceb8740 10 client.17601 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5563237640d0 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f16f8c00640 1 -- 172.21.15.114:0/3111138336 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc00f790 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f16f8c00640 1 -- 172.21.15.114:0/3111138336 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f16fc02a030 con 0x5563233124d0 2024-09-21T07:04:18.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f171ceb8740 10 client.17601 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.967+0000 7f171ceb8740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f171ceb8740 20 client.17601 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232c71c0 0x556323766610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f171ceb8740 4 client.17601 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55632364b0b0 con 0x5563232c71c0 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f170ea00640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232c71c0 0x556323766610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f170ea00640 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232c71c0 0x556323766610 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f30b378b016e6b server_cookie=1ed1b8c13a2acdb3 in_seq=0 out_seq=0 2024-09-21T07:04:18.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.968+0000 7f16f8c00640 10 client.17601 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.976+0000 7f16f8c00640 1 -- 172.21.15.114:0/3111138336 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55632364b0b0 con 0x5563232c71c0 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.976+0000 7f16f8c00640 10 client.17601 handle_command_reply: tid=0 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.976+0000 7f171ceb8740 1 client.17601 shutdown 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 msgr2=0x5563236ed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236ed6a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00f940 tx=0x7f16fc00f970 comp rx=0 tx=0).stop 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 shutdown_connections 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563233331b0 0x5563236edbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563233124d0 0x5563236ed6a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563232c71c0 0x556323766610 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 --2- 172.21.15.114:0/3111138336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55632330a2f0 0x5563236f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 >> 172.21.15.114:0/3111138336 conn(0x556323327300 msgr2=0x5563236e48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f16f8c00640 0 client.17601 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 shutdown_connections 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 1 -- 172.21.15.114:0/3111138336 wait complete. 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.977+0000 7f171ceb8740 20 client.17601 trim_cache size 0 max 16384 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:04:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:04:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:04:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b6680 msgr2=0x7f16f40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b6680 0x7f16f40b8b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5563236d6a90 tx=0x7f1700005f90 comp rx=0 tx=0).stop 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 msgr2=0x7f171019fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f171019fa50 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1708005cc0 tx=0x7f1708031700 comp rx=0 tx=0).stop 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 shutdown_connections 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f171010dbc0 0x7f171019fa50 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b6680 0x7f16f40b8b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17101050f0 0x7f171019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 --2- 172.21.15.114:0/3686394528 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17101047a0 0x7f171019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 >> 172.21.15.114:0/3686394528 conn(0x7f17100fe230 msgr2=0x7f1710107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 shutdown_connections 2024-09-21T07:04:18.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:18.978+0000 7f170fe00640 1 -- 172.21.15.114:0/3686394528 wait complete. 2024-09-21T07:04:19.668 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '230636ca', quiesce took 15.5 seconds. Will release after: 9.0 2024-09-21T07:04:20.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:19 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:20.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:19 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:20.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:19 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:20.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:19 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:20.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:19 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:20.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:19 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:20.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:19 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:04:20.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:19 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:04:20.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:19 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:04:20.194 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:04:20.565 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:21.051 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:20 smithi114 ceph-mon[30654]: pgmap v710: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-21T07:04:21.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:20 smithi183 ceph-mon[34219]: pgmap v710: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-21T07:04:21.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:20 smithi193 ceph-mon[34601]: pgmap v710: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 -- 172.21.15.114:0/3249114719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a01049d0 msgr2=0x7f92a0104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 --2- 172.21.15.114:0/3249114719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a01049d0 0x7f92a0104de0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9290009900 tx=0x7f929002a410 comp rx=0 tx=0).stop 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 -- 172.21.15.114:0/3249114719 shutdown_connections 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 --2- 172.21.15.114:0/3249114719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a010de30 0x7f92a0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 --2- 172.21.15.114:0/3249114719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 0x7f92a010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 --2- 172.21.15.114:0/3249114719 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a01049d0 0x7f92a0104de0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.237+0000 7f92a6e00640 1 -- 172.21.15.114:0/3249114719 >> 172.21.15.114:0/3249114719 conn(0x7f92a0079740 msgr2=0x7f92a0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.238+0000 7f92a6e00640 1 -- 172.21.15.114:0/3249114719 shutdown_connections 2024-09-21T07:04:21.241 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.238+0000 7f92a6e00640 1 -- 172.21.15.114:0/3249114719 wait complete. 2024-09-21T07:04:21.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.239+0000 7f92a6e00640 1 Processor -- start 2024-09-21T07:04:21.242 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.239+0000 7f92a6e00640 1 -- start start 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a6e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a01049d0 0x7f92a019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a6e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 0x7f92a019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a6e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a010de30 0x7f92a01a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a6e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0113ab0 con 0x7f92a010de30 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a6e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0113930 con 0x7f92a01049d0 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a6e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0113c30 con 0x7f92a0105320 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f929fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a01049d0 0x7f92a019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a010de30 0x7f92a01a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f929f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 0x7f92a019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f929fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a01049d0 0x7f92a019c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41956/0 (socket says 172.21.15.114:41956) 2024-09-21T07:04:21.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f929fe00640 1 -- 172.21.15.114:0/1738135891 learned_addr learned my addr 172.21.15.114:0/1738135891 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:21.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.240+0000 7f929f400640 1 -- 172.21.15.114:0/1738135891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a01049d0 msgr2=0x7f92a019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929f400640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a01049d0 0x7f92a019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929f400640 1 -- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a010de30 msgr2=0x7f92a01a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929f400640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a010de30 0x7f92a01a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929f400640 1 -- 172.21.15.114:0/1738135891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a01a26d0 con 0x7f92a0105320 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929f400640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 0x7f92a019cd00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f92a006b950 tx=0x7f928c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929cc00640 1 -- 172.21.15.114:0/1738135891 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c00cc40 con 0x7f92a0105320 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a01a29c0 con 0x7f92a0105320 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929cc00640 1 -- 172.21.15.114:0/1738135891 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928c00cde0 con 0x7f92a0105320 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f929cc00640 1 -- 172.21.15.114:0/1738135891 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c004a00 con 0x7f92a0105320 2024-09-21T07:04:21.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.241+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01a9120 con 0x7f92a0105320 2024-09-21T07:04:21.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.244+0000 7f92ad9df740 1 Processor -- start 2024-09-21T07:04:21.247 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 -- start start 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f4b3070 0x55749f859da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f85a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55749f7d0fb0 con 0x55749f48a1b0 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55749f7c03e0 con 0x55749f492390 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f92ad9df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55749f7c41f0 con 0x55749f4b3070 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f929cc00640 1 -- 172.21.15.114:0/1738135891 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f928c004ba0 con 0x7f92a0105320 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.245+0000 7f929fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.248 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37606/0 (socket says 172.21.15.114:37606) 2024-09-21T07:04:21.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 -- 172.21.15.114:0/157700915 learned_addr learned my addr 172.21.15.114:0/157700915 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:21.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929f400640 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f85a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 -- 172.21.15.114:0/157700915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f4b3070 msgr2=0x55749f859da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f4b3070 0x55749f859da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 -- 172.21.15.114:0/157700915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 msgr2=0x55749f85a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f85a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 -- 172.21.15.114:0/157700915 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749f78acd0 con 0x55749f48a1b0 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.246+0000 7f929fe00640 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85a2e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9290006010 tx=0x7f9290031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b981d61312720f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.247+0000 7f9295600640 1 -- 172.21.15.114:0/157700915 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290040070 con 0x55749f48a1b0 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.247+0000 7f9295600640 1 -- 172.21.15.114:0/157700915 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92900316a0 con 0x55749f48a1b0 2024-09-21T07:04:21.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.247+0000 7f9295600640 1 -- 172.21.15.114:0/157700915 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290035040 con 0x55749f48a1b0 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 -- 172.21.15.114:0/157700915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 msgr2=0x55749f85a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85a2e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9290006010 tx=0x7f9290031bc0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f929cc00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92840b6700 0x7f92840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 -- 172.21.15.114:0/157700915 shutdown_connections 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f85a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85a2e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 --2- 172.21.15.114:0/157700915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f4b3070 0x55749f859da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 -- 172.21.15.114:0/157700915 >> 172.21.15.114:0/157700915 conn(0x55749f4a7460 msgr2=0x55749f857c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f929cc00640 1 -- 172.21.15.114:0/1738135891 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f928c0e7a10 con 0x7f92a0105320 2024-09-21T07:04:21.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 -- 172.21.15.114:0/157700915 shutdown_connections 2024-09-21T07:04:21.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92a4e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92840b6700 0x7f92840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.252 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.248+0000 7f92ad9df740 1 -- 172.21.15.114:0/157700915 wait complete. 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.249+0000 7f92ad9df740 1 -- start start 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.249+0000 7f92a4e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92840b6700 0x7f92840b8c10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9298005fd0 tx=0x7f9298004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.249+0000 7f92ad9df740 1 Processor -- start 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f865510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f865a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f871810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55749f621700 con 0x55749f4b3070 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55749f75b580 con 0x55749f492390 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55749f868520 con 0x55749f48a1b0 2024-09-21T07:04:21.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92ad9df740 10 client.0 _ll_register_callbacks cb 0x55749f76b470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:21.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f871810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f929fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f865510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f929fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f865510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46194/0 (socket says 172.21.15.114:46194) 2024-09-21T07:04:21.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f871810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37610/0 (socket says 172.21.15.114:37610) 2024-09-21T07:04:21.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f929fe00640 1 -- 172.21.15.114:0/381856755 learned_addr learned my addr 172.21.15.114:0/381856755 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:21.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.250+0000 7f929f400640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f865a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929fe00640 1 -- 172.21.15.114:0/381856755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 msgr2=0x55749f865a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929fe00640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f865a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929fe00640 1 -- 172.21.15.114:0/381856755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 msgr2=0x55749f871810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929fe00640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f871810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929fe00640 1 -- 172.21.15.114:0/381856755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749f51e650 con 0x55749f48a1b0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929f400640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f865a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f929fe00640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f865510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f92900056c0 tx=0x7f929002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800a6b0c57cc6459 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f928a000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f928a000640 1 -- 172.21.15.114:0/381856755 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290040070 con 0x55749f48a1b0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55749f75db90 con 0x55749f48a1b0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.251+0000 7f928a000640 1 -- 172.21.15.114:0/381856755 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f92900318b0 con 0x55749f48a1b0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f928a000640 1 -- 172.21.15.114:0/381856755 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9290056050 con 0x55749f48a1b0 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 10 client.28021 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:21.256 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55749f778950 con 0x55749f48a1b0 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f928a000640 1 -- 172.21.15.114:0/381856755 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55749f75db90 con 0x55749f48a1b0 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f928a000640 1 -- 172.21.15.114:0/381856755 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f92900035f0 con 0x55749f48a1b0 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 10 client.28021 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 10 client.28021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 20 client.28021 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f4719f0 0x55749f877480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 4 client.28021 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.252+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55749f7d2f00 con 0x55749f4719f0 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.253+0000 7f929f400640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f4719f0 0x55749f877480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.253+0000 7f929f400640 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f4719f0 0x55749f877480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2321963e57ceb01 server_cookie=f18930b18596b1b3 in_seq=0 out_seq=0 2024-09-21T07:04:21.257 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.253+0000 7f928a000640 10 client.28021 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:04:21.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.254+0000 7f928a000640 1 -- 172.21.15.114:0/381856755 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55749f7d2f00 con 0x55749f4719f0 2024-09-21T07:04:21.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.254+0000 7f928a000640 10 client.28021 handle_command_reply: tid=0 2024-09-21T07:04:21.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.254+0000 7f92ad9df740 1 client.28021 shutdown 2024-09-21T07:04:21.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 msgr2=0x55749f865510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.258 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f865510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f92900056c0 tx=0x7f929002a410 comp rx=0 tx=0).stop 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 shutdown_connections 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f4719f0 0x55749f877480 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f871810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f865a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 --2- 172.21.15.114:0/381856755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f865510 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f928a000640 0 client.28021 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 >> 172.21.15.114:0/381856755 conn(0x55749f4a7460 msgr2=0x55749f85cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 shutdown_connections 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 1 -- 172.21.15.114:0/381856755 wait complete. 2024-09-21T07:04:21.259 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.255+0000 7f92ad9df740 20 client.28021 trim_cache size 0 max 16384 2024-09-21T07:04:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.269+0000 7f92ad9df740 1 Processor -- start 2024-09-21T07:04:21.272 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.269+0000 7f92ad9df740 1 -- start start 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f492390 0x55749f862720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f4b3070 0x55749f862c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92ad9df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b30a0 con 0x55749f48a1b0 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92ad9df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b4be0 con 0x55749f4b3070 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92ad9df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b3770 con 0x55749f492390 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f929fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f929fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37612/0 (socket says 172.21.15.114:37612) 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f929fe00640 1 -- 172.21.15.114:0/3422395237 learned_addr learned my addr 172.21.15.114:0/3422395237 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f929f400640 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f492390 0x55749f862720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.273 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.270+0000 7f92a4e00640 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f4b3070 0x55749f862c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929f400640 1 -- 172.21.15.114:0/3422395237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f4b3070 msgr2=0x55749f862c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929f400640 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f4b3070 0x55749f862c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929f400640 1 -- 172.21.15.114:0/3422395237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 msgr2=0x55749f85b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929f400640 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.274 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929f400640 1 -- 172.21.15.114:0/3422395237 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749f8667d0 con 0x55749f492390 2024-09-21T07:04:21.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929fe00640 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:21.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f929f400640 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f492390 0x55749f862720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f928c0e4d30 tx=0x7f928c00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a038ae46d193256a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:21.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f9294c00640 1 -- 172.21.15.114:0/3422395237 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c01c040 con 0x55749f492390 2024-09-21T07:04:21.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f9294c00640 1 -- 172.21.15.114:0/3422395237 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928c0acf40 con 0x55749f492390 2024-09-21T07:04:21.275 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.271+0000 7f9294c00640 1 -- 172.21.15.114:0/3422395237 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c0b69c0 con 0x55749f492390 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 -- 172.21.15.114:0/3422395237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f492390 msgr2=0x55749f862720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f492390 0x55749f862720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f928c0e4d30 tx=0x7f928c00cf90 comp rx=0 tx=0).stop 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 -- 172.21.15.114:0/3422395237 shutdown_connections 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f4b3070 0x55749f862c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f492390 0x55749f862720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 --2- 172.21.15.114:0/3422395237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f48a1b0 0x55749f85b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 -- 172.21.15.114:0/3422395237 >> 172.21.15.114:0/3422395237 conn(0x55749f4a7460 msgr2=0x55749f85add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 -- 172.21.15.114:0/3422395237 shutdown_connections 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.272+0000 7f92ad9df740 1 -- 172.21.15.114:0/3422395237 wait complete. 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.273+0000 7f92ad9df740 1 -- start start 2024-09-21T07:04:21.276 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.273+0000 7f92ad9df740 1 Processor -- start 2024-09-21T07:04:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.273+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f878c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.273+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f8791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92ad9df740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f8e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92ad9df740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b4510 con 0x55749f4b3070 2024-09-21T07:04:21.278 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92ad9df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b3e40 con 0x55749f492390 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92ad9df740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f92a01b10a0 con 0x55749f48a1b0 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92ad9df740 10 client.0 _ll_register_callbacks cb 0x7f929002ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f8e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92a4e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f8e0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37614/0 (socket says 172.21.15.114:37614) 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92a4e00640 1 -- 172.21.15.114:0/854810741 learned_addr learned my addr 172.21.15.114:0/854810741 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929f400640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f8791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929fe00640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f878c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929f400640 1 -- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 msgr2=0x55749f878c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929f400640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f878c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929f400640 1 -- 172.21.15.114:0/854810741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 msgr2=0x55749f8e0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929f400640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f8e0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f929f400640 1 -- 172.21.15.114:0/854810741 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749f8e11f0 con 0x55749f492390 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.274+0000 7f92a4e00640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f8e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f929f400640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f8791c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f928c0ad3c0 tx=0x7f928c0e5240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8795d2dedea6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:21.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f928a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f928a000640 1 -- 172.21.15.114:0/854810741 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c01c040 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f928a000640 1 -- 172.21.15.114:0/854810741 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f928c0b6bd0 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928c0acf40 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f928a000640 1 -- 172.21.15.114:0/854810741 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f928c0acba0 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f92ad9df740 10 client.28091 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55749f8e2450 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.275+0000 7f928a000640 1 -- 172.21.15.114:0/854810741 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f928c0acf40 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f928a000640 1 -- 172.21.15.114:0/854810741 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f928c0ac690 con 0x55749f492390 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f92ad9df740 10 client.28091 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f92ad9df740 10 client.28091 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f92ad9df740 20 client.28091 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f92ad9df740 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f446da0 0x55749f8e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f92ad9df740 4 client.28091 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:04:21.280 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55749f7ce9b0 con 0x55749f446da0 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.276+0000 7f929fe00640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f446da0 0x55749f8e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f929fe00640 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f446da0 0x55749f8e4be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1b05c985c603529 server_cookie=a933409722e0ab65 in_seq=0 out_seq=0 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f928a000640 10 client.28091 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f928a000640 1 -- 172.21.15.114:0/854810741 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55749f7ce9b0 con 0x55749f446da0 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f928a000640 10 client.28091 handle_command_reply: tid=0 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 client.28091 shutdown 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 msgr2=0x55749f8791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f8791c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f928c0ad3c0 tx=0x7f928c0e5240 comp rx=0 tx=0).stop 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 shutdown_connections 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55749f446da0 0x55749f8e4be0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55749f4b3070 0x55749f8e0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55749f492390 0x55749f8791c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 --2- 172.21.15.114:0/854810741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55749f48a1b0 0x55749f878c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f928a000640 0 client.28091 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:04:21.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 >> 172.21.15.114:0/854810741 conn(0x55749f4a7460 msgr2=0x55749f85add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:21.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 shutdown_connections 2024-09-21T07:04:21.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.277+0000 7f92ad9df740 1 -- 172.21.15.114:0/854810741 wait complete. 2024-09-21T07:04:21.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.278+0000 7f92ad9df740 20 client.28091 trim_cache size 0 max 16384 2024-09-21T07:04:21.283 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.278+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92840b6700 msgr2=0x7f92840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.278+0000 7f92a6e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92840b6700 0x7f92840b8c10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9298005fd0 tx=0x7f9298004a40 comp rx=0 tx=0).stop 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.278+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 msgr2=0x7f92a019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.278+0000 7f92a6e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 0x7f92a019cd00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f92a006b950 tx=0x7f928c00ee50 comp rx=0 tx=0).stop 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 shutdown_connections 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f92a010de30 0x7f92a01a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92a0105320 0x7f92a019cd00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92840b6700 0x7f92840b8c10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 --2- 172.21.15.114:0/1738135891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92a01049d0 0x7f92a019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 >> 172.21.15.114:0/1738135891 conn(0x7f92a0079740 msgr2=0x7f92a0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 shutdown_connections 2024-09-21T07:04:21.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:21.279+0000 7f92a6e00640 1 -- 172.21.15.114:0/1738135891 wait complete. 2024-09-21T07:04:22.350 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:04:22.738 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 585.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:04:23.010 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:22 smithi114 ceph-mon[30654]: pgmap v711: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-09-21T07:04:23.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:22 smithi183 ceph-mon[34219]: pgmap v711: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-09-21T07:04:23.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:22 smithi193 ceph-mon[34601]: pgmap v711: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 -- 172.21.15.114:0/405829334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 msgr2=0x7fef04103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 --2- 172.21.15.114:0/405829334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef04103dc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7feef4009900 tx=0x7feef402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 -- 172.21.15.114:0/405829334 shutdown_connections 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 --2- 172.21.15.114:0/405829334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef04109c40 0x7fef04110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 --2- 172.21.15.114:0/405829334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef041092f0 0x7fef04109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 --2- 172.21.15.114:0/405829334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef04103dc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 -- 172.21.15.114:0/405829334 >> 172.21.15.114:0/405829334 conn(0x7fef040ff440 msgr2=0x7fef04101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 -- 172.21.15.114:0/405829334 shutdown_connections 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.373+0000 7fef08a00640 1 -- 172.21.15.114:0/405829334 wait complete. 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.375+0000 7fef08a00640 1 Processor -- start 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.375+0000 7fef08a00640 1 -- start start 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef08a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef0419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef08a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef041092f0 0x7fef0419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef08a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef04109c40 0x7fef0419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef08a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113900 con 0x7fef041039b0 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef08a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113780 con 0x7fef04109c40 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef08a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113a80 con 0x7fef041092f0 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef0419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef0419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37632/0 (socket says 172.21.15.114:37632) 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 -- 172.21.15.114:0/2399195646 learned_addr learned my addr 172.21.15.114:0/2399195646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7feefbe00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef041092f0 0x7fef0419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef03200640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef04109c40 0x7fef0419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 -- 172.21.15.114:0/2399195646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef041092f0 msgr2=0x7fef0419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef041092f0 0x7fef0419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 -- 172.21.15.114:0/2399195646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef04109c40 msgr2=0x7fef0419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef04109c40 0x7fef0419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.376+0000 7fef02800640 1 -- 172.21.15.114:0/2399195646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a02a0 con 0x7fef041039b0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.377+0000 7fef02800640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef0419a330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7feef4005940 tx=0x7feef4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.377+0000 7feefb400640 1 -- 172.21.15.114:0/2399195646 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef403f030 con 0x7fef041039b0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.377+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a0530 con 0x7fef041039b0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.377+0000 7feefb400640 1 -- 172.21.15.114:0/2399195646 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef4003aa0 con 0x7fef041039b0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.377+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a6cf0 con 0x7fef041039b0 2024-09-21T07:04:23.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.377+0000 7feefb400640 1 -- 172.21.15.114:0/2399195646 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4049060 con 0x7fef041039b0 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.380+0000 7fef10092740 1 Processor -- start 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.380+0000 7fef10092740 1 -- start start 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef10092740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 0x5649c7ae3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef10092740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c770f1b0 0x5649c7ae38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef10092740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7ae3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef10092740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649c7a55fb0 con 0x5649c770f1b0 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef10092740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649c7a453e0 con 0x5649c7717390 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef10092740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649c7a491f0 con 0x5649c7738070 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7feefb400640 1 -- 172.21.15.114:0/2399195646 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7feef4042030 con 0x7fef041039b0 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef03200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 0x5649c7ae3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef03200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 0x5649c7ae3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46234/0 (socket says 172.21.15.114:46234) 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef03200640 1 -- 172.21.15.114:0/3606259563 learned_addr learned my addr 172.21.15.114:0/3606259563 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7feefbe00640 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7ae3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.381+0000 7fef02800640 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c770f1b0 0x5649c7ae38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7fef03200640 1 -- 172.21.15.114:0/3606259563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 msgr2=0x5649c7ae3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7fef03200640 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7ae3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7fef03200640 1 -- 172.21.15.114:0/3606259563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c770f1b0 msgr2=0x5649c7ae38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7fef03200640 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c770f1b0 0x5649c7ae38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7fef03200640 1 -- 172.21.15.114:0/3606259563 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c7a0fcd0 con 0x5649c7738070 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7feefbe00640 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7ae3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7fef03200640 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 0x5649c7ae3380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feefc009f50 tx=0x7feefc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfebc6cec0e6ca9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7feef3400640 1 -- 172.21.15.114:0/3606259563 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004d20 con 0x5649c7738070 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7feef3400640 1 -- 172.21.15.114:0/3606259563 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc004050 con 0x5649c7738070 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7feef3400640 1 -- 172.21.15.114:0/3606259563 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc005280 con 0x5649c7738070 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.382+0000 7feefb400640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feee80b6680 0x7feee80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7feefb400640 1 -- 172.21.15.114:0/2399195646 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7feef4108160 con 0x7fef041039b0 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef02800640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feee80b6680 0x7feee80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 -- 172.21.15.114:0/3606259563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 msgr2=0x5649c7ae3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 0x5649c7ae3380 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7feefc009f50 tx=0x7feefc0118d0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 -- 172.21.15.114:0/3606259563 shutdown_connections 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7ae3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c770f1b0 0x5649c7ae38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 --2- 172.21.15.114:0/3606259563 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7738070 0x5649c7ae3380 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 -- 172.21.15.114:0/3606259563 >> 172.21.15.114:0/3606259563 conn(0x5649c772c460 msgr2=0x5649c7ae1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 -- 172.21.15.114:0/3606259563 shutdown_connections 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef10092740 1 -- 172.21.15.114:0/3606259563 wait complete. 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.383+0000 7fef02800640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feee80b6680 0x7feee80b8b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5649c7ae2000 tx=0x7feef402a3f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.384+0000 7fef10092740 1 -- start start 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.384+0000 7fef10092740 1 Processor -- start 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7aeeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7aef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7afae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5649c78a6700 con 0x5649c7738070 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5649c79e0580 con 0x5649c7717390 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5649c7af1b60 con 0x5649c770f1b0 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef10092740 10 client.0 _ll_register_callbacks cb 0x5649c79f0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef03200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7afae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef03200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7afae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37638/0 (socket says 172.21.15.114:37638) 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7feefbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7aef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef03200640 1 -- 172.21.15.114:0/1091617668 learned_addr learned my addr 172.21.15.114:0/1091617668 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7fef03200640 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7aeeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7feefbe00640 1 -- 172.21.15.114:0/1091617668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 msgr2=0x5649c7aeeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7feefbe00640 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7aeeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7feefbe00640 1 -- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 msgr2=0x5649c7afae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7feefbe00640 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7afae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.385+0000 7feefbe00640 1 -- 172.21.15.114:0/1091617668 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c77a3650 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7feefbe00640 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7aef0a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feeec005d00 tx=0x7feeec0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17f894a321eafc01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7feee7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7feee7e00640 1 -- 172.21.15.114:0/1091617668 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec004020 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7feee7e00640 1 -- 172.21.15.114:0/1091617668 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7feeec0041c0 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5649c79e2b90 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7feee7e00640 1 -- 172.21.15.114:0/1091617668 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7feeec0043c0 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7fef10092740 10 client.28103 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.386+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5649c79fd950 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7feee7e00640 1 -- 172.21.15.114:0/1091617668 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5649c79e2b90 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7feee7e00640 1 -- 172.21.15.114:0/1091617668 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7feeec004c60 con 0x5649c7717390 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef10092740 10 client.28103 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:23.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef10092740 10 client.28103 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef10092740 20 client.28103 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef10092740 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76f69f0 0x5649c7b00d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef10092740 4 client.28103 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c7a57f00 con 0x5649c76f69f0 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.387+0000 7fef03200640 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76f69f0 0x5649c7b00d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.388+0000 7fef03200640 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76f69f0 0x5649c7b00d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec00e45bad20992a server_cookie=d1aab6b976dd25ad in_seq=0 out_seq=0 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.388+0000 7feee7e00640 10 client.28103 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.388+0000 7feee7e00640 1 -- 172.21.15.114:0/1091617668 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5649c7a57f00 con 0x5649c76f69f0 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.388+0000 7feee7e00640 10 client.28103 handle_command_reply: tid=0 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.388+0000 7fef10092740 1 client.28103 shutdown 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 msgr2=0x5649c7aef0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7aef0a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feeec005d00 tx=0x7feeec0118d0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 shutdown_connections 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76f69f0 0x5649c7b00d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7afae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7aef0a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7feee7e00640 0 client.28103 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 --2- 172.21.15.114:0/1091617668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7aeeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 >> 172.21.15.114:0/1091617668 conn(0x5649c772c460 msgr2=0x5649c7ae6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 shutdown_connections 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 1 -- 172.21.15.114:0/1091617668 wait complete. 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.389+0000 7fef10092740 20 client.28103 trim_cache size 0 max 16384 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.403+0000 7fef10092740 1 Processor -- start 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.403+0000 7fef10092740 1 -- start start 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef10092740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c770f1b0 0x5649c7aea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef10092740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7717390 0x5649c7ae4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef10092740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7adbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef10092740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fef041aeb20 con 0x5649c7738070 2024-09-21T07:04:23.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef10092740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b0660 con 0x5649c770f1b0 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef10092740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef041af1f0 con 0x5649c7717390 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef03200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7adbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7feefbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7717390 0x5649c7ae4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef03200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7adbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37646/0 (socket says 172.21.15.114:37646) 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef03200640 1 -- 172.21.15.114:0/1786562934 learned_addr learned my addr 172.21.15.114:0/1786562934 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7fef03200640 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c770f1b0 0x5649c7aea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7feefbe00640 1 -- 172.21.15.114:0/1786562934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c770f1b0 msgr2=0x5649c7aea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7feefbe00640 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c770f1b0 0x5649c7aea310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7feefbe00640 1 -- 172.21.15.114:0/1786562934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 msgr2=0x5649c7adbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7feefbe00640 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7adbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.404+0000 7feefbe00640 1 -- 172.21.15.114:0/1786562934 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c7ae5c10 con 0x5649c7717390 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.405+0000 7feefbe00640 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7717390 0x5649c7ae4d70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feeec006010 tx=0x7feeec00b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9df713cd18c629b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.405+0000 7feef2a00640 1 -- 172.21.15.114:0/1786562934 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec02c020 con 0x5649c7717390 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.405+0000 7feef2a00640 1 -- 172.21.15.114:0/1786562934 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeec005850 con 0x5649c7717390 2024-09-21T07:04:23.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.405+0000 7feef2a00640 1 -- 172.21.15.114:0/1786562934 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec0048c0 con 0x5649c7717390 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 -- 172.21.15.114:0/1786562934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7717390 msgr2=0x5649c7ae4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7717390 0x5649c7ae4d70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feeec006010 tx=0x7feeec00b540 comp rx=0 tx=0).stop 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 -- 172.21.15.114:0/1786562934 shutdown_connections 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7adbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c7717390 0x5649c7ae4d70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 --2- 172.21.15.114:0/1786562934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c770f1b0 0x5649c7aea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 -- 172.21.15.114:0/1786562934 >> 172.21.15.114:0/1786562934 conn(0x5649c772c460 msgr2=0x5649c7adb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 -- 172.21.15.114:0/1786562934 shutdown_connections 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.406+0000 7fef10092740 1 -- 172.21.15.114:0/1786562934 wait complete. 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 -- start start 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 Processor -- start 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7af6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7af7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7af1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fef041aff90 con 0x5649c7738070 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef041af8c0 con 0x5649c7717390 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef041acb20 con 0x5649c770f1b0 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.407+0000 7fef10092740 10 client.0 _ll_register_callbacks cb 0x7feeec00ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7fef03200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7af1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7af7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7af7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42050/0 (socket says 172.21.15.114:42050) 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 -- 172.21.15.114:0/3459958139 learned_addr learned my addr 172.21.15.114:0/3459958139 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7fef03200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7af1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37652/0 (socket says 172.21.15.114:37652) 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7fef03200640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7af6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 -- 172.21.15.114:0/3459958139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 msgr2=0x5649c7af6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7af6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 -- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 msgr2=0x5649c7af1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7af1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 -- 172.21.15.114:0/3459958139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c7af78f0 con 0x5649c7717390 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7fef03200640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7af1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feefbe00640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7af7330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec028650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c433b3472bb48e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.408+0000 7feee7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7feee7e00640 1 -- 172.21.15.114:0/3459958139 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec02c070 con 0x5649c7717390 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec005850 con 0x5649c7717390 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7feee7e00640 1 -- 172.21.15.114:0/3459958139 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7feeec017070 con 0x5649c7717390 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7feee7e00640 1 -- 172.21.15.114:0/3459958139 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7feeec041050 con 0x5649c7717390 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 10 client.28109 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5649c7b6b9e0 con 0x5649c7717390 2024-09-21T07:04:23.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7feee7e00640 1 -- 172.21.15.114:0/3459958139 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeec005850 con 0x5649c7717390 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7feee7e00640 1 -- 172.21.15.114:0/3459958139 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7feeec01b920 con 0x5649c7717390 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 10 client.28109 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 10 client.28109 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 20 client.28109 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76cbda0 0x5649c7b6e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 4 client.28109 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:04:23.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.409+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649c7a539b0 con 0x5649c76cbda0 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.410+0000 7fef03200640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76cbda0 0x5649c7b6e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.410+0000 7fef03200640 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76cbda0 0x5649c7b6e100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f23ae03a25c6495e server_cookie=5d174bb9d47d77da in_seq=0 out_seq=0 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.410+0000 7feee7e00640 10 client.28109 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.410+0000 7feee7e00640 1 -- 172.21.15.114:0/3459958139 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649c7a539b0 con 0x5649c76cbda0 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.410+0000 7feee7e00640 10 client.28109 handle_command_reply: tid=0 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.410+0000 7fef10092740 1 client.28109 shutdown 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 msgr2=0x5649c7af7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7af7330 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feeec000c00 tx=0x7feeec028650 comp rx=0 tx=0).stop 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 shutdown_connections 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5649c76cbda0 0x5649c7b6e100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5649c7738070 0x5649c7af1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5649c7717390 0x5649c7af7330 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 --2- 172.21.15.114:0/3459958139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5649c770f1b0 0x5649c7af6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7feee7e00640 0 client.28109 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 >> 172.21.15.114:0/3459958139 conn(0x5649c772c460 msgr2=0x5649c7ae4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 shutdown_connections 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 1 -- 172.21.15.114:0/3459958139 wait complete. 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.411+0000 7fef10092740 20 client.28109 trim_cache size 0 max 16384 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feee80b6680 msgr2=0x7feee80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feee80b6680 0x7feee80b8b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x5649c7ae2000 tx=0x7feef402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 msgr2=0x7fef0419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef0419a330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7feef4005940 tx=0x7feef4004270 comp rx=0 tx=0).stop 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 shutdown_connections 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7feee80b6680 0x7feee80b8b90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef04109c40 0x7fef0419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef041092f0 0x7fef0419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 --2- 172.21.15.114:0/2399195646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fef041039b0 0x7fef0419a330 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:23.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 >> 172.21.15.114:0/2399195646 conn(0x7fef040ff440 msgr2=0x7fef04101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:23.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 shutdown_connections 2024-09-21T07:04:23.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:23.412+0000 7fef08a00640 1 -- 172.21.15.114:0/2399195646 wait complete. 2024-09-21T07:04:24.424 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:04:24.796 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:25.232 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:24 smithi114 ceph-mon[30654]: pgmap v712: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-21T07:04:25.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:24 smithi183 ceph-mon[34219]: pgmap v712: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 -- 172.21.15.114:0/2634275050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc105bc0 msgr2=0x7f89bc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 --2- 172.21.15.114:0/2634275050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc105bc0 0x7f89bc105fd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009900 tx=0x7f89b002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 -- 172.21.15.114:0/2634275050 shutdown_connections 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 --2- 172.21.15.114:0/2634275050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10be50 0x7f89bc112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 --2- 172.21.15.114:0/2634275050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89bc10b500 0x7f89bc10b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 --2- 172.21.15.114:0/2634275050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc105bc0 0x7f89bc105fd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 -- 172.21.15.114:0/2634275050 >> 172.21.15.114:0/2634275050 conn(0x7f89bc101670 msgr2=0x7f89bc103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 -- 172.21.15.114:0/2634275050 shutdown_connections 2024-09-21T07:04:25.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.406+0000 7f89c3000640 1 -- 172.21.15.114:0/2634275050 wait complete. 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 Processor -- start 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 -- start start 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89bc105bc0 0x7f89bc19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10b500 0x7f89bc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 0x7f89bc1a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc115b60 con 0x7f89bc10be50 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1159e0 con 0x7f89bc105bc0 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.407+0000 7f89c3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc115ce0 con 0x7f89bc10b500 2024-09-21T07:04:25.410 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 0x7f89bc1a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 0x7f89bc1a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33308/0 (socket says 172.21.15.114:33308) 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 -- 172.21.15.114:0/1102867330 learned_addr learned my addr 172.21.15.114:0/1102867330 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89bc105bc0 0x7f89bc19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c1200640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10b500 0x7f89bc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 -- 172.21.15.114:0/1102867330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10b500 msgr2=0x7f89bc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10b500 0x7f89bc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 -- 172.21.15.114:0/1102867330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89bc105bc0 msgr2=0x7f89bc19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89bc105bc0 0x7f89bc19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c2600640 1 -- 172.21.15.114:0/1102867330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc1a2480 con 0x7f89bc10be50 2024-09-21T07:04:25.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.408+0000 7f89c1200640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10b500 0x7f89bc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:25.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:24 smithi193 ceph-mon[34601]: pgmap v712: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-21T07:04:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.409+0000 7f89c2600640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 0x7f89bc1a1d50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f89b400eb10 tx=0x7f89b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.409+0000 7f89ba000640 1 -- 172.21.15.114:0/1102867330 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4004b20 con 0x7f89bc10be50 2024-09-21T07:04:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.409+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc1a2770 con 0x7f89bc10be50 2024-09-21T07:04:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.409+0000 7f89ba000640 1 -- 172.21.15.114:0/1102867330 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b4004cc0 con 0x7f89bc10be50 2024-09-21T07:04:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.409+0000 7f89ba000640 1 -- 172.21.15.114:0/1102867330 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4010670 con 0x7f89bc10be50 2024-09-21T07:04:25.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.409+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc1a8ed0 con 0x7f89bc10be50 2024-09-21T07:04:25.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.412+0000 7f89cb1de740 1 Processor -- start 2024-09-21T07:04:25.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89ba000640 1 -- 172.21.15.114:0/1102867330 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f89b4005020 con 0x7f89bc10be50 2024-09-21T07:04:25.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 -- start start 2024-09-21T07:04:25.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2a6ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 0x5591e2a6f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a6f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5591e29e5fb0 con 0x5591e26a7390 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591e29d53e0 con 0x5591e269f1b0 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.413+0000 7f89cb1de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591e29d91f0 con 0x5591e26c8070 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a6f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 0x5591e2a6f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 0x5591e2a6f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45734/0 (socket says 172.21.15.114:45734) 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 -- 172.21.15.114:0/1424508142 learned_addr learned my addr 172.21.15.114:0/1424508142 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:25.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1200640 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2a6ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 -- 172.21.15.114:0/1424508142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 msgr2=0x5591e2a6ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2a6ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 -- 172.21.15.114:0/1424508142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 msgr2=0x5591e2a6f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a6f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1c00640 1 -- 172.21.15.114:0/1424508142 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e299fcd0 con 0x5591e269f1b0 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.414+0000 7f89c1200640 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2a6ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 0x5591e2a6f270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009230 tx=0x7f89b0004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5db8d246ab341c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89aea00640 1 -- 172.21.15.114:0/1424508142 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0035040 con 0x5591e269f1b0 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89ba000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89a40b6700 0x7f89a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89aea00640 1 -- 172.21.15.114:0/1424508142 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b00054f0 con 0x5591e269f1b0 2024-09-21T07:04:25.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89aea00640 1 -- 172.21.15.114:0/1424508142 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0003850 con 0x5591e269f1b0 2024-09-21T07:04:25.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89c1200640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89a40b6700 0x7f89a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.415+0000 7f89ba000640 1 -- 172.21.15.114:0/1102867330 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f89b40e7ff0 con 0x7f89bc10be50 2024-09-21T07:04:25.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 -- 172.21.15.114:0/1424508142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 msgr2=0x5591e2a6f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89c1200640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89a40b6700 0x7f89a40b8c10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f89bc104f50 tx=0x7f89a800bb90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 0x5591e2a6f270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009230 tx=0x7f89b0004170 comp rx=0 tx=0).stop 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 -- 172.21.15.114:0/1424508142 shutdown_connections 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a6f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e269f1b0 0x5591e2a6f270 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1424508142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2a6ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 -- 172.21.15.114:0/1424508142 >> 172.21.15.114:0/1424508142 conn(0x5591e26bc460 msgr2=0x5591e2a6cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 -- 172.21.15.114:0/1424508142 shutdown_connections 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.416+0000 7f89cb1de740 1 -- 172.21.15.114:0/1424508142 wait complete. 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.417+0000 7f89cb1de740 1 -- start start 2024-09-21T07:04:25.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.417+0000 7f89cb1de740 1 Processor -- start 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.417+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 0x5591e2a7a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a7ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a86800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89cb1de740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5591e2836700 con 0x5591e26a7390 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89cb1de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591e2970580 con 0x5591e26c8070 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89cb1de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591e2a7d4b0 con 0x5591e269f1b0 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89cb1de740 10 client.0 _ll_register_callbacks cb 0x5591e2980470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a7ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 0x5591e2a7a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 0x5591e2a7a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47090/0 (socket says 172.21.15.114:47090) 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 -- 172.21.15.114:0/727013792 learned_addr learned my addr 172.21.15.114:0/727013792 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a7ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33322/0 (socket says 172.21.15.114:33322) 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c2600640 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a86800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 -- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 msgr2=0x5591e2a86800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a86800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 -- 172.21.15.114:0/727013792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 msgr2=0x5591e2a7ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a7ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.418+0000 7f89c1c00640 1 -- 172.21.15.114:0/727013792 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e2733650 con 0x5591e269f1b0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89c1c00640 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 0x5591e2a7a850 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f89b0031440 tx=0x7f89b002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7f889eb3ca6017 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89a3400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89a3400640 1 -- 172.21.15.114:0/727013792 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0035040 con 0x5591e269f1b0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89a3400640 1 -- 172.21.15.114:0/727013792 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f89b0004dc0 con 0x5591e269f1b0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591e2972b90 con 0x5591e269f1b0 2024-09-21T07:04:25.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89a3400640 1 -- 172.21.15.114:0/727013792 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f89b0004fc0 con 0x5591e269f1b0 2024-09-21T07:04:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89cb1de740 10 client.28057 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:25.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.419+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5591e298d950 con 0x5591e269f1b0 2024-09-21T07:04:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89a3400640 1 -- 172.21.15.114:0/727013792 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5591e2972b90 con 0x5591e269f1b0 2024-09-21T07:04:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89a3400640 1 -- 172.21.15.114:0/727013792 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f89b0055030 con 0x5591e269f1b0 2024-09-21T07:04:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89cb1de740 10 client.28057 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:25.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89cb1de740 10 client.28057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89cb1de740 20 client.28057 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89cb1de740 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e26869f0 0x5591e2a8c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89cb1de740 4 client.28057 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e29e7f00 con 0x5591e26869f0 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.420+0000 7f89c1200640 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e26869f0 0x5591e2a8c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.421+0000 7f89c1200640 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e26869f0 0x5591e2a8c610 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f3deb057731b27 server_cookie=b1df7872453e2c3e in_seq=0 out_seq=0 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.421+0000 7f89a3400640 10 client.28057 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89a3400640 1 -- 172.21.15.114:0/727013792 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591e29e7f00 con 0x5591e26869f0 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89a3400640 10 client.28057 handle_command_reply: tid=0 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 client.28057 shutdown 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 msgr2=0x5591e2a7a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 0x5591e2a7a850 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f89b0031440 tx=0x7f89b002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 shutdown_connections 2024-09-21T07:04:25.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a86800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e26a7390 0x5591e2a7ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e26869f0 0x5591e2a8c610 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 --2- 172.21.15.114:0/727013792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e269f1b0 0x5591e2a7a850 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89a3400640 0 client.28057 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 >> 172.21.15.114:0/727013792 conn(0x5591e26bc460 msgr2=0x5591e2a71ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.422+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 shutdown_connections 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.423+0000 7f89cb1de740 1 -- 172.21.15.114:0/727013792 wait complete. 2024-09-21T07:04:25.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.423+0000 7f89cb1de740 20 client.28057 trim_cache size 0 max 16384 2024-09-21T07:04:25.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.436+0000 7f89cb1de740 1 Processor -- start 2024-09-21T07:04:25.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.436+0000 7f89cb1de740 1 -- start start 2024-09-21T07:04:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26a7390 0x5591e2a77840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a77d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89cb1de740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b2f20 con 0x5591e269f1b0 2024-09-21T07:04:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89cb1de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b4a60 con 0x5591e26c8070 2024-09-21T07:04:25.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89cb1de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b35f0 con 0x5591e26a7390 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a6cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33328/0 (socket says 172.21.15.114:33328) 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 -- 172.21.15.114:0/985374889 learned_addr learned my addr 172.21.15.114:0/985374889 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c2600640 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a77d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1200640 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26a7390 0x5591e2a77840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 -- 172.21.15.114:0/985374889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26a7390 msgr2=0x5591e2a77840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26a7390 0x5591e2a77840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 -- 172.21.15.114:0/985374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 msgr2=0x5591e2a77d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a77d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.437+0000 7f89c1c00640 1 -- 172.21.15.114:0/985374889 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e2a71020 con 0x5591e269f1b0 2024-09-21T07:04:25.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.438+0000 7f89c1c00640 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a6cb60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f89b002a910 tx=0x7f89b0004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f3c94f3a7c3493 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.438+0000 7f89ae000640 1 -- 172.21.15.114:0/985374889 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0063070 con 0x5591e269f1b0 2024-09-21T07:04:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.438+0000 7f89ae000640 1 -- 172.21.15.114:0/985374889 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b0005610 con 0x5591e269f1b0 2024-09-21T07:04:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.438+0000 7f89ae000640 1 -- 172.21.15.114:0/985374889 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0003ac0 con 0x5591e269f1b0 2024-09-21T07:04:25.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 -- 172.21.15.114:0/985374889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 msgr2=0x5591e2a6cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a6cb60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f89b002a910 tx=0x7f89b0004080 comp rx=0 tx=0).stop 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 -- 172.21.15.114:0/985374889 shutdown_connections 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26c8070 0x5591e2a77d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26a7390 0x5591e2a77840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 --2- 172.21.15.114:0/985374889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a6cb60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 -- 172.21.15.114:0/985374889 >> 172.21.15.114:0/985374889 conn(0x5591e26bc460 msgr2=0x5591e2a6c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 -- 172.21.15.114:0/985374889 shutdown_connections 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.439+0000 7f89cb1de740 1 -- 172.21.15.114:0/985374889 wait complete. 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 -- start start 2024-09-21T07:04:25.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 Processor -- start 2024-09-21T07:04:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26a7390 0x5591e2a8c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2aefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b4390 con 0x5591e269f1b0 2024-09-21T07:04:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b3cc0 con 0x5591e26a7390 2024-09-21T07:04:25.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc1b0f20 con 0x5591e26c8070 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.440+0000 7f89cb1de740 10 client.0 _ll_register_callbacks cb 0x7f89b0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a8c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33342/0 (socket says 172.21.15.114:33342) 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 -- 172.21.15.114:0/1640713393 learned_addr learned my addr 172.21.15.114:0/1640713393 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1200640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26a7390 0x5591e2a8c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c2600640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2aefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 -- 172.21.15.114:0/1640713393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 msgr2=0x5591e2aefd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2aefd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 -- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26a7390 msgr2=0x5591e2a8c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26a7390 0x5591e2a8c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 -- 172.21.15.114:0/1640713393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e2af03c0 con 0x5591e269f1b0 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1200640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26a7390 0x5591e2a8c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:25.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89c1c00640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a8c000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f89b00096c0 tx=0x7f89b002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21634241f19950fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.441+0000 7f89a3400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89a3400640 1 -- 172.21.15.114:0/1640713393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0035070 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89a3400640 1 -- 172.21.15.114:0/1640713393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f89b0005360 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b0005610 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89a3400640 1 -- 172.21.15.114:0/1640713393 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f89b0063040 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89cb1de740 10 client.17655 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5591e2af7790 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89a3400640 1 -- 172.21.15.114:0/1640713393 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89b0005610 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89a3400640 1 -- 172.21.15.114:0/1640713393 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f89b0055040 con 0x5591e269f1b0 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89cb1de740 10 client.17655 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.442+0000 7f89cb1de740 10 client.17655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:25.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89cb1de740 20 client.17655 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e265bda0 0x5591e2af9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89cb1de740 4 client.17655 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:04:25.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591e29e39b0 con 0x5591e265bda0 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89c1200640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e265bda0 0x5591e2af9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89c1200640 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e265bda0 0x5591e2af9cb0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88db9dc1f16fddc server_cookie=3e79127b03750bef in_seq=0 out_seq=0 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.443+0000 7f89a3400640 10 client.17655 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89a3400640 1 -- 172.21.15.114:0/1640713393 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591e29e39b0 con 0x5591e265bda0 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89a3400640 10 client.17655 handle_command_reply: tid=0 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 client.17655 shutdown 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 msgr2=0x5591e2a8c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a8c000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f89b00096c0 tx=0x7f89b002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 shutdown_connections 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5591e265bda0 0x5591e2af9cb0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591e26c8070 0x5591e2aefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591e26a7390 0x5591e2a8c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 --2- 172.21.15.114:0/1640713393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591e269f1b0 0x5591e2a8c000 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89a3400640 0 client.17655 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 >> 172.21.15.114:0/1640713393 conn(0x5591e26bc460 msgr2=0x5591e2a71e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 shutdown_connections 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 1 -- 172.21.15.114:0/1640713393 wait complete. 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.444+0000 7f89cb1de740 20 client.17655 trim_cache size 0 max 16384 2024-09-21T07:04:25.449 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89a40b6700 msgr2=0x7f89a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89a40b6700 0x7f89a40b8c10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f89bc104f50 tx=0x7f89a800bb90 comp rx=0 tx=0).stop 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 msgr2=0x7f89bc1a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 0x7f89bc1a1d50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f89b400eb10 tx=0x7f89b400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 shutdown_connections 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f89bc10be50 0x7f89bc1a1d50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89bc10b500 0x7f89bc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f89a40b6700 0x7f89a40b8c10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 --2- 172.21.15.114:0/1102867330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f89bc105bc0 0x7f89bc19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 >> 172.21.15.114:0/1102867330 conn(0x7f89bc101670 msgr2=0x7f89bc10abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:25.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 shutdown_connections 2024-09-21T07:04:25.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:25.445+0000 7f89c3000640 1 -- 172.21.15.114:0/1102867330 wait complete. 2024-09-21T07:04:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:26 smithi114 ceph-mon[30654]: pgmap v713: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-21T07:04:27.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:26 smithi183 ceph-mon[34219]: pgmap v713: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-21T07:04:27.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:04:27.378 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:04:27.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:26 smithi193 ceph-mon[34601]: pgmap v713: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-09-21T07:04:27.760 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:27.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 590.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:04:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 -- 172.21.15.114:0/860441314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f4105320 msgr2=0x7f91f410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 --2- 172.21.15.114:0/860441314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f4105320 0x7f91f410d870 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009900 tx=0x7f91e002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 -- 172.21.15.114:0/860441314 shutdown_connections 2024-09-21T07:04:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 --2- 172.21.15.114:0/860441314 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f410de30 0x7f91f4110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 --2- 172.21.15.114:0/860441314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f4105320 0x7f91f410d870 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 --2- 172.21.15.114:0/860441314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f41049d0 0x7f91f4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.361+0000 7f91fb600640 1 -- 172.21.15.114:0/860441314 >> 172.21.15.114:0/860441314 conn(0x7f91f4079740 msgr2=0x7f91f4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.362+0000 7f91fb600640 1 -- 172.21.15.114:0/860441314 shutdown_connections 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.362+0000 7f91fb600640 1 -- 172.21.15.114:0/860441314 wait complete. 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 Processor -- start 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 -- start start 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 0x7f91f419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 0x7f91f419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f410de30 0x7f91f41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f91f4113a90 con 0x7f91f41049d0 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91f4113910 con 0x7f91f410de30 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91fb600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91f4113c10 con 0x7f91f4105320 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.363+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 0x7f91f419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 0x7f91f419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.479 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 0x7f91f419c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33354/0 (socket says 172.21.15.114:33354) 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f8c00640 1 -- 172.21.15.114:0/3734118735 learned_addr learned my addr 172.21.15.114:0/3734118735 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 0x7f91f419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47138/0 (socket says 172.21.15.114:47138) 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f9600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f410de30 0x7f91f41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 -- 172.21.15.114:0/3734118735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f410de30 msgr2=0x7f91f41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f410de30 0x7f91f41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 -- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 msgr2=0x7f91f419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 0x7f91f419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 -- 172.21.15.114:0/3734118735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f41a2630 con 0x7f91f4105320 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 0x7f91f419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.364+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 0x7f91f419cc60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f91e002a910 tx=0x7f91e0031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.365+0000 7f91f1600640 1 -- 172.21.15.114:0/3734118735 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e003f070 con 0x7f91f4105320 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.365+0000 7f91f1600640 1 -- 172.21.15.114:0/3734118735 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e0003aa0 con 0x7f91f4105320 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.365+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f41a28c0 con 0x7f91f4105320 2024-09-21T07:04:28.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.365+0000 7f91f1600640 1 -- 172.21.15.114:0/3734118735 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0035070 con 0x7f91f4105320 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.365+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f41a9080 con 0x7f91f4105320 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.368+0000 7f9202263740 1 Processor -- start 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 -- start start 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8beb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563de8b60e90 con 0x563de8822670 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563de8b502c0 con 0x563de8843350 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f9202263740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563de8b540d0 con 0x563de881a490 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f91f9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.369+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8beb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bead10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45784/0 (socket says 172.21.15.114:45784) 2024-09-21T07:04:28.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f9600640 1 -- 172.21.15.114:0/3605522286 learned_addr learned my addr 172.21.15.114:0/3605522286 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f1600640 1 -- 172.21.15.114:0/3734118735 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f91e0003c50 con 0x7f91f4105320 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f8c00640 1 -- 172.21.15.114:0/3605522286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 msgr2=0x563de8bead10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bead10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f8c00640 1 -- 172.21.15.114:0/3605522286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 msgr2=0x563de8beb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f8c00640 1 -- 172.21.15.114:0/3605522286 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563de88ae930 con 0x563de881a490 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f9600640 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.370+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8beb250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f91e80114c0 tx=0x7f91e8011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca116cbbd3b0c075 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.371+0000 7f91e5c00640 1 -- 172.21.15.114:0/3605522286 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005020 con 0x563de881a490 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.371+0000 7f91e5c00640 1 -- 172.21.15.114:0/3605522286 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e80047a0 con 0x563de881a490 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.371+0000 7f91e5c00640 1 -- 172.21.15.114:0/3605522286 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e800f800 con 0x563de881a490 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.371+0000 7f9202263740 1 -- 172.21.15.114:0/3605522286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 msgr2=0x563de8beb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.371+0000 7f9202263740 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8beb250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f91e80114c0 tx=0x7f91e8011980 comp rx=0 tx=0).stop 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 -- 172.21.15.114:0/3605522286 shutdown_connections 2024-09-21T07:04:28.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8beb250 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 --2- 172.21.15.114:0/3605522286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bead10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 -- 172.21.15.114:0/3605522286 >> 172.21.15.114:0/3605522286 conn(0x563de8837740 msgr2=0x563de8be8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 -- 172.21.15.114:0/3605522286 shutdown_connections 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 -- 172.21.15.114:0/3605522286 wait complete. 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f91f1600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f91dc0b6590 0x7f91dc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f9202263740 1 -- start start 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f91f1600640 1 -- 172.21.15.114:0/3734118735 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f91e0107c00 con 0x7f91f4105320 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.372+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f91dc0b6590 0x7f91dc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 Processor -- start 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f91dc0b6590 0x7f91dc0b8aa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f91e801b040 tx=0x7f91e8000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bf68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8822670 0x563de8bf6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8c027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563de89b1890 con 0x563de881a490 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563de8aeb160 con 0x563de8843350 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563de8bf94f0 con 0x563de8822670 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f9202263740 10 client.0 _ll_register_callbacks cb 0x563de8b07a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.373+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bf68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bf68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33382/0 (socket says 172.21.15.114:33382) 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 -- 172.21.15.114:0/655337891 learned_addr learned my addr 172.21.15.114:0/655337891 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f9600640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8c027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f3e00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8822670 0x563de8bf6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 -- 172.21.15.114:0/655337891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8822670 msgr2=0x563de8bf6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8822670 0x563de8bf6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 -- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 msgr2=0x563de8c027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8c027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 -- 172.21.15.114:0/655337891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563de8c02da0 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f9600640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8c027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f3e00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8822670 0x563de8bf6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91f8c00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bf68e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f91e8004e90 tx=0x7f91e8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567c73812f16073f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.374+0000 7f91da000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f91da000640 1 -- 172.21.15.114:0/655337891 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8028070 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f91da000640 1 -- 172.21.15.114:0/655337891 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f91e8003b70 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e80047a0 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f91da000640 1 -- 172.21.15.114:0/655337891 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f91e803d050 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f9202263740 10 client.17664 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563de8c056d0 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.375+0000 7f91da000640 1 -- 172.21.15.114:0/655337891 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e80047a0 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f91da000640 1 -- 172.21.15.114:0/655337891 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f91e802f470 con 0x563de881a490 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f9202263740 10 client.17664 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:28.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f9202263740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f9202263740 20 client.17664 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f9202263740 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de8801cd0 0x563de8c08ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f9202263740 4 client.17664 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563de8b62de0 con 0x563de8801cd0 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.376+0000 7f91f3e00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de8801cd0 0x563de8c08ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.377+0000 7f91da000640 10 client.17664 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.377+0000 7f91f3e00640 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de8801cd0 0x563de8c08ae0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce6777ac9c3e072 server_cookie=e0b69cace6f947ff in_seq=0 out_seq=0 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.377+0000 7f91da000640 1 -- 172.21.15.114:0/655337891 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563de8b62de0 con 0x563de8801cd0 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f91da000640 10 client.17664 handle_command_reply: tid=0 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 client.17664 shutdown 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 msgr2=0x563de8bf68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bf68e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f91e8004e90 tx=0x7f91e8009310 comp rx=0 tx=0).stop 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 shutdown_connections 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8c027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de8801cd0 0x563de8c08ae0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8822670 0x563de8bf6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 --2- 172.21.15.114:0/655337891 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bf68e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f91da000640 0 client.17664 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 >> 172.21.15.114:0/655337891 conn(0x563de8837740 msgr2=0x563de8be7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 shutdown_connections 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 1 -- 172.21.15.114:0/655337891 wait complete. 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.378+0000 7f9202263740 20 client.17664 trim_cache size 0 max 16384 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.396+0000 7f9202263740 1 Processor -- start 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.396+0000 7f9202263740 1 -- start start 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f9202263740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8c026e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f9202263740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f9202263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bf04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f9202263740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41b3120 con 0x563de8822670 2024-09-21T07:04:28.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f9202263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41b4c60 con 0x563de8843350 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f9202263740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41b37f0 con 0x563de881a490 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8c026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33392/0 (socket says 172.21.15.114:33392) 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 -- 172.21.15.114:0/2353107548 learned_addr learned my addr 172.21.15.114:0/2353107548 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f9600640 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bf04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 -- 172.21.15.114:0/2353107548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 msgr2=0x563de8c026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8c026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 -- 172.21.15.114:0/2353107548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 msgr2=0x563de8bf04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bf04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f3e00640 1 -- 172.21.15.114:0/2353107548 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563de8bf7630 con 0x563de8822670 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.397+0000 7f91f8c00640 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8c026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.398+0000 7f91f3e00640 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beff90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f91e00cd320 tx=0x7f91e00045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3132f5b74cb3e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.398+0000 7f91e5200640 1 -- 172.21.15.114:0/2353107548 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0048040 con 0x563de8822670 2024-09-21T07:04:28.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.398+0000 7f91e5200640 1 -- 172.21.15.114:0/2353107548 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e00cdd20 con 0x563de8822670 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.398+0000 7f91e5200640 1 -- 172.21.15.114:0/2353107548 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e003bb60 con 0x563de8822670 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 -- 172.21.15.114:0/2353107548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 msgr2=0x563de8beff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beff90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f91e00cd320 tx=0x7f91e00045d0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 -- 172.21.15.114:0/2353107548 shutdown_connections 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8843350 0x563de8bf04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de8822670 0x563de8beff90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 --2- 172.21.15.114:0/2353107548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de881a490 0x563de8c026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 -- 172.21.15.114:0/2353107548 >> 172.21.15.114:0/2353107548 conn(0x563de8837740 msgr2=0x563de8bf4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 -- 172.21.15.114:0/2353107548 shutdown_connections 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 -- 172.21.15.114:0/2353107548 wait complete. 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.399+0000 7f9202263740 1 -- start start 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 Processor -- start 2024-09-21T07:04:28.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bfedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8822670 0x563de8c0dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8843350 0x563de8c0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41b4590 con 0x563de881a490 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41b3ec0 con 0x563de8822670 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f91f41b1120 con 0x563de8843350 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f9202263740 10 client.0 _ll_register_callbacks cb 0x7f91ec009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bfedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f91f8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bfedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33408/0 (socket says 172.21.15.114:33408) 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f91f8c00640 1 -- 172.21.15.114:0/3076211662 learned_addr learned my addr 172.21.15.114:0/3076211662 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.400+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8822670 0x563de8c0dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f9600640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8843350 0x563de8c0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f8c00640 1 -- 172.21.15.114:0/3076211662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8843350 msgr2=0x563de8c0e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8843350 0x563de8c0e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f8c00640 1 -- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8822670 msgr2=0x563de8c0dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8822670 0x563de8c0dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f8c00640 1 -- 172.21.15.114:0/3076211662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563de8bf7280 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8822670 0x563de8c0dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91f8c00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bfedb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009230 tx=0x7f91e8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c959cdb3283b6dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91da000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91da000640 1 -- 172.21.15.114:0/3076211662 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8028040 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f91da000640 1 -- 172.21.15.114:0/3076211662 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f91e802fb00 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.401+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e00cdd20 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f91da000640 1 -- 172.21.15.114:0/3076211662 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f91e8050050 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 10 client.17670 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563de8bfc430 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f91da000640 1 -- 172.21.15.114:0/3076211662 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91e00cdd20 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f91da000640 1 -- 172.21.15.114:0/3076211662 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f91e80110d0 con 0x563de881a490 2024-09-21T07:04:28.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 10 client.17670 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 20 client.17670 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de87d7080 0x563de8c758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 4 client.17670 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.402+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563de8b5e890 con 0x563de87d7080 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.403+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de87d7080 0x563de8c758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.403+0000 7f91f3e00640 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de87d7080 0x563de8c758b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6753f92856c72fcb server_cookie=991bd4ea8001f3e1 in_seq=0 out_seq=0 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.403+0000 7f91da000640 10 client.17670 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f91da000640 1 -- 172.21.15.114:0/3076211662 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563de8b5e890 con 0x563de87d7080 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f91da000640 10 client.17670 handle_command_reply: tid=0 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 client.17670 shutdown 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 msgr2=0x563de8bfedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bfedb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009230 tx=0x7f91e8004c80 comp rx=0 tx=0).stop 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 shutdown_connections 2024-09-21T07:04:28.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563de87d7080 0x563de8c758b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563de8843350 0x563de8c0e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563de8822670 0x563de8c0dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f91da000640 0 client.17670 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 --2- 172.21.15.114:0/3076211662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563de881a490 0x563de8bfedb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 >> 172.21.15.114:0/3076211662 conn(0x563de8837740 msgr2=0x563de8be7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "73ddb0e8-eb68-4bca-bf13-582ff62ccebc", 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:04:28.493 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 shutdown_connections 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.405+0000 7f9202263740 1 -- 172.21.15.114:0/3076211662 wait complete. 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f9202263740 20 client.17670 trim_cache size 0 max 16384 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f91dc0b6590 msgr2=0x7f91dc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f91dc0b6590 0x7f91dc0b8aa0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f91e801b040 tx=0x7f91e8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 msgr2=0x7f91f419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 0x7f91f419cc60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f91e002a910 tx=0x7f91e0031a20 comp rx=0 tx=0).stop 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 shutdown_connections 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f91dc0b6590 0x7f91dc0b8aa0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91f410de30 0x7f91f41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91f4105320 0x7f91f419cc60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 --2- 172.21.15.114:0/3734118735 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f91f41049d0 0x7f91f419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 >> 172.21.15.114:0/3734118735 conn(0x7f91f4079740 msgr2=0x7f91f410cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.406+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 shutdown_connections 2024-09-21T07:04:28.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:28.407+0000 7f91fb600640 1 -- 172.21.15.114:0/3734118735 wait complete. 2024-09-21T07:04:28.669 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:04:29.298 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: pgmap v714: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.299 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:29.300 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:28 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.334 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:04:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: pgmap v714: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-09-21T07:04:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:04:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:29.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:29.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:29.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:28 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: pgmap v714: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:28 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:29.440 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:30.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 -- 172.21.15.114:0/604411585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0108d00 msgr2=0x7fb5f010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:30.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 --2- 172.21.15.114:0/604411585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0108d00 0x7fb5f010f530 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009ca0 tx=0x7fb5e802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 -- 172.21.15.114:0/604411585 shutdown_connections 2024-09-21T07:04:30.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 --2- 172.21.15.114:0/604411585 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0108d00 0x7fb5f010f530 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 --2- 172.21.15.114:0/604411585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f01083b0 0x7fb5f01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 --2- 172.21.15.114:0/604411585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f0102ac0 0x7fb5f0102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 -- 172.21.15.114:0/604411585 >> 172.21.15.114:0/604411585 conn(0x7fb5f0079720 msgr2=0x7fb5f0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 -- 172.21.15.114:0/604411585 shutdown_connections 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.105+0000 7fb5f5800640 1 -- 172.21.15.114:0/604411585 wait complete. 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 Processor -- start 2024-09-21T07:04:30.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 -- start start 2024-09-21T07:04:30.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0102ac0 0x7fb5f01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:30.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 0x7fb5f0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:30.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108d00 0x7fb5f019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:30.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0112a10 con 0x7fb5f0102ac0 2024-09-21T07:04:30.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0112890 con 0x7fb5f0108d00 2024-09-21T07:04:30.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.107+0000 7fb5f5800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0112b90 con 0x7fb5f01083b0 2024-09-21T07:04:30.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 0x7fb5f0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:30.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5eea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0102ac0 0x7fb5f01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:30.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 0x7fb5f0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47194/0 (socket says 172.21.15.114:47194) 2024-09-21T07:04:30.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 -- 172.21.15.114:0/3579276263 learned_addr learned my addr 172.21.15.114:0/3579276263 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ef400640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108d00 0x7fb5f019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 -- 172.21.15.114:0/3579276263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108d00 msgr2=0x7fb5f019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108d00 0x7fb5f019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 -- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0102ac0 msgr2=0x7fb5f01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0102ac0 0x7fb5f01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.108+0000 7fb5ee000640 1 -- 172.21.15.114:0/3579276263 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f019f350 con 0x7fb5f01083b0 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.109+0000 7fb5ee000640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 0x7fb5f0199980 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:30.111 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.109+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc00ccc0 con 0x7fb5f01083b0 2024-09-21T07:04:30.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.109+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5dc0108e0 con 0x7fb5f01083b0 2024-09-21T07:04:30.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.109+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc018860 con 0x7fb5f01083b0 2024-09-21T07:04:30.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.109+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f019f640 con 0x7fb5f01083b0 2024-09-21T07:04:30.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.109+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f006b820 con 0x7fb5f01083b0 2024-09-21T07:04:30.112 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.110+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f0102ed0 con 0x7fb5f01083b0 2024-09-21T07:04:30.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.113+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb5dc010430 con 0x7fb5f01083b0 2024-09-21T07:04:30.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.115+0000 7fb5e7400640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5d40b6700 0x7fb5d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:30.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.115+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fb5dc0e81a0 con 0x7fb5f01083b0 2024-09-21T07:04:30.117 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.116+0000 7fb5eea00640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5d40b6700 0x7fb5d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:30.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.116+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5dc0b26a0 con 0x7fb5f01083b0 2024-09-21T07:04:30.118 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.116+0000 7fb5eea00640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5d40b6700 0x7fb5d40b8c10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009bd0 tx=0x7fb5e0010040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:30.155 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:29 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:29 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.369+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5f019a510 con 0x7fb5f01083b0 2024-09-21T07:04:30.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.370+0000 7fb5e7400640 1 -- 172.21.15.114:0/3579276263 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7fb5dc0b75e0 con 0x7fb5f01083b0 2024-09-21T07:04:30.373 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:04:30.373 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":49,"btime":"2024-09-21T07:03:23:885749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:03:22.931940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:04:30.373 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 49 2024-09-21T07:04:30.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5d40b6700 msgr2=0x7fb5d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5d40b6700 0x7fb5d40b8c10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009bd0 tx=0x7fb5e0010040 comp rx=0 tx=0).stop 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 msgr2=0x7fb5f0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 0x7fb5f0199980 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00e9f0 tx=0x7fb5dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 shutdown_connections 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb5d40b6700 0x7fb5d40b8c10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f0108d00 0x7fb5f019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5f01083b0 0x7fb5f0199980 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 --2- 172.21.15.114:0/3579276263 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb5f0102ac0 0x7fb5f01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 >> 172.21.15.114:0/3579276263 conn(0x7fb5f0079720 msgr2=0x7fb5f0104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:30.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 shutdown_connections 2024-09-21T07:04:30.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:30.373+0000 7fb5f5800640 1 -- 172.21.15.114:0/3579276263 wait complete. 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:30.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:29 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:04:31.112 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:30 smithi114 ceph-mon[30654]: pgmap v715: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 386 KiB/s wr, 67 op/s 2024-09-21T07:04:31.112 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:30 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3579276263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:30 smithi183 ceph-mon[34219]: pgmap v715: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 386 KiB/s wr, 67 op/s 2024-09-21T07:04:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:30 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3579276263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:31.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:30 smithi193 ceph-mon[34601]: pgmap v715: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 386 KiB/s wr, 67 op/s 2024-09-21T07:04:31.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:30 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3579276263' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:31.430 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 230636ca --release --await' 2024-09-21T07:04:31.430 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 230636ca --release --await 2024-09-21T07:04:31.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 -- 172.21.15.114:0/3666056288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c8105320 msgr2=0x7f72c810d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 --2- 172.21.15.114:0/3666056288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c8105320 0x7f72c810d870 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009900 tx=0x7f72b402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 -- 172.21.15.114:0/3666056288 shutdown_connections 2024-09-21T07:04:31.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 --2- 172.21.15.114:0/3666056288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c8110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 --2- 172.21.15.114:0/3666056288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c8105320 0x7f72c810d870 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 --2- 172.21.15.114:0/3666056288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72c81049d0 0x7f72c8104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 -- 172.21.15.114:0/3666056288 >> 172.21.15.114:0/3666056288 conn(0x7f72c8079740 msgr2=0x7f72c8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.110+0000 7f72d0400640 1 -- 172.21.15.114:0/3666056288 shutdown_connections 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.111+0000 7f72d0400640 1 -- 172.21.15.114:0/3666056288 wait complete. 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 Processor -- start 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 -- start start 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c81049d0 0x7f72c819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72c8105320 0x7f72c819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c81a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8113ae0 con 0x7f72c810de30 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8113960 con 0x7f72c81049d0 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72d0400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8113c60 con 0x7f72c8105320 2024-09-21T07:04:31.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72ce400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c81a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72ce400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c81a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33434/0 (socket says 172.21.15.114:33434) 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.112+0000 7f72ce400640 1 -- 172.21.15.114:0/2985282027 learned_addr learned my addr 172.21.15.114:0/2985282027 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72cd000640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72c8105320 0x7f72c819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72cda00640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c81049d0 0x7f72c819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72ce400640 1 -- 172.21.15.114:0/2985282027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72c8105320 msgr2=0x7f72c819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72ce400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72c8105320 0x7f72c819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72ce400640 1 -- 172.21.15.114:0/2985282027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c81049d0 msgr2=0x7f72c819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72ce400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c81049d0 0x7f72c819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72ce400640 1 -- 172.21.15.114:0/2985282027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c81a2660 con 0x7f72c810de30 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72cda00640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c81049d0 0x7f72c819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72ce400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c81a1f30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f72c000eb10 tx=0x7f72c000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.113+0000 7f72c6000640 1 -- 172.21.15.114:0/2985282027 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0004b20 con 0x7f72c810de30 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.114+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c81a2950 con 0x7f72c810de30 2024-09-21T07:04:31.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.114+0000 7f72c6000640 1 -- 172.21.15.114:0/2985282027 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72c0004cc0 con 0x7f72c810de30 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.114+0000 7f72c6000640 1 -- 172.21.15.114:0/2985282027 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0010670 con 0x7f72c810de30 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.114+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81a90b0 con 0x7f72c810de30 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 Processor -- start 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72c6000640 1 -- 172.21.15.114:0/2985282027 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f72c0005020 con 0x7f72c810de30 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 -- start start 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55693035bf70 con 0x55693001d390 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55693034b3a0 con 0x5569300151b0 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.117+0000 7f72d7084740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55693034f1b0 con 0x55693003e070 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303e55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33436/0 (socket says 172.21.15.114:33436) 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cd000640 1 -- 172.21.15.114:0/290757476 learned_addr learned my addr 172.21.15.114:0/290757476 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cd000640 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cda00640 1 -- 172.21.15.114:0/290757476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 msgr2=0x5569303e4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cda00640 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303e4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cda00640 1 -- 172.21.15.114:0/290757476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 msgr2=0x5569303e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cda00640 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.118+0000 7f72cda00640 1 -- 172.21.15.114:0/290757476 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556930315c90 con 0x5569300151b0 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72cda00640 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e50a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009fa0 tx=0x7f72bc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cddcc2fdf99bbb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72c6000640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6700 0x7f72b00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72baa00640 1 -- 172.21.15.114:0/290757476 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc004310 con 0x5569300151b0 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72baa00640 1 -- 172.21.15.114:0/290757476 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72bc0044b0 con 0x5569300151b0 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72baa00640 1 -- 172.21.15.114:0/290757476 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0035f0 con 0x5569300151b0 2024-09-21T07:04:31.570 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72c6000640 1 -- 172.21.15.114:0/2985282027 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f72c00e80a0 con 0x7f72c810de30 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.119+0000 7f72cd000640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6700 0x7f72b00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 -- 172.21.15.114:0/290757476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 msgr2=0x5569303e50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e50a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009fa0 tx=0x7f72bc0118d0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72cd000640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6700 0x7f72b00b8c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f72b4005ab0 tx=0x7f72b4005780 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 -- 172.21.15.114:0/290757476 shutdown_connections 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e50a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 --2- 172.21.15.114:0/290757476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 -- 172.21.15.114:0/290757476 >> 172.21.15.114:0/290757476 conn(0x556930032460 msgr2=0x5569303e2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 -- 172.21.15.114:0/290757476 shutdown_connections 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.120+0000 7f72d7084740 1 -- 172.21.15.114:0/290757476 wait complete. 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 -- start start 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 Processor -- start 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 0x5569303f0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.571 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693001d390 0x5569303f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5569301ac700 con 0x5569300151b0 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5569302e6540 con 0x55693001d390 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.121+0000 7f72d7084740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5569303f3340 con 0x55693003e070 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72d7084740 10 client.0 _ll_register_callbacks cb 0x5569302f6430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 0x5569303f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 0x5569303f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33440/0 (socket says 172.21.15.114:33440) 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 -- 172.21.15.114:0/1144923915 learned_addr learned my addr 172.21.15.114:0/1144923915 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cd000640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693001d390 0x5569303f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72ce400640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 -- 172.21.15.114:0/1144923915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 msgr2=0x5569303fc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.572 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 -- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693001d390 msgr2=0x5569303f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693001d390 0x5569303f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.122+0000 7f72cda00640 1 -- 172.21.15.114:0/1144923915 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569300a9650 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72cda00640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 0x5569303f0720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009700 tx=0x7f72bc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ae39308ba41740 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72af400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72af400640 1 -- 172.21.15.114:0/1144923915 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc027030 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72af400640 1 -- 172.21.15.114:0/1144923915 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f72bc0035f0 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5569302e8b50 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72af400640 1 -- 172.21.15.114:0/1144923915 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f72bc034050 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72d7084740 10 client.17688 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556930303910 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.123+0000 7f72af400640 1 -- 172.21.15.114:0/1144923915 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5569302e8b50 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72af400640 1 -- 172.21.15.114:0/1144923915 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f72bc004f10 con 0x5569300151b0 2024-09-21T07:04:31.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72d7084740 10 client.17688 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72d7084740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72d7084740 20 client.17688 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72d7084740 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692fffc9f0 0x556930402520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72d7084740 4 client.17688 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693035dec0 con 0x55692fffc9f0 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.124+0000 7f72cd000640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692fffc9f0 0x556930402520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.125+0000 7f72cd000640 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692fffc9f0 0x556930402520 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805aaa287f4a3869 server_cookie=a579f9f7c02e9ef8 in_seq=0 out_seq=0 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.125+0000 7f72af400640 10 client.17688 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72af400640 1 -- 172.21.15.114:0/1144923915 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55693035dec0 con 0x55692fffc9f0 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72af400640 10 client.17688 handle_command_reply: tid=0 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 client.17688 shutdown 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 msgr2=0x5569303f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 0x5569303f0720 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009700 tx=0x7f72bc004340 comp rx=0 tx=0).stop 2024-09-21T07:04:31.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 shutdown_connections 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692fffc9f0 0x556930402520 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55693001d390 0x5569303f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 --2- 172.21.15.114:0/1144923915 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5569300151b0 0x5569303f0720 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72af400640 0 client.17688 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 >> 172.21.15.114:0/1144923915 conn(0x556930032460 msgr2=0x5569303e7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 shutdown_connections 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 1 -- 172.21.15.114:0/1144923915 wait complete. 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.126+0000 7f72d7084740 20 client.17688 trim_cache size 0 max 16384 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.144+0000 7f72d7084740 1 Processor -- start 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.144+0000 7f72d7084740 1 -- start start 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72d7084740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72d7084740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72d7084740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72d7084740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b3210 con 0x55693001d390 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72d7084740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b4d50 con 0x5569300151b0 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72d7084740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b38e0 con 0x55693003e070 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303fc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33446/0 (socket says 172.21.15.114:33446) 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 -- 172.21.15.114:0/844254393 learned_addr learned my addr 172.21.15.114:0/844254393 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72ce400640 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cda00640 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 -- 172.21.15.114:0/844254393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 msgr2=0x5569303fcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 -- 172.21.15.114:0/844254393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 msgr2=0x5569303e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72cd000640 1 -- 172.21.15.114:0/844254393 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569303f2eb0 con 0x55693001d390 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.145+0000 7f72ce400640 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:31.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.146+0000 7f72cd000640 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303fc560 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009230 tx=0x7f72b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e11f3bdc0f22d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.146+0000 7f72ba000640 1 -- 172.21.15.114:0/844254393 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4031560 con 0x55693001d390 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.146+0000 7f72ba000640 1 -- 172.21.15.114:0/844254393 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b4031bb0 con 0x55693001d390 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.146+0000 7f72ba000640 1 -- 172.21.15.114:0/844254393 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b4055bc0 con 0x55693001d390 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 -- 172.21.15.114:0/844254393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 msgr2=0x5569303fc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303fc560 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009230 tx=0x7f72b4009260 comp rx=0 tx=0).stop 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 -- 172.21.15.114:0/844254393 shutdown_connections 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693003e070 0x5569303fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693001d390 0x5569303fc560 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 --2- 172.21.15.114:0/844254393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 -- 172.21.15.114:0/844254393 >> 172.21.15.114:0/844254393 conn(0x556930032460 msgr2=0x5569303e5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 -- 172.21.15.114:0/844254393 shutdown_connections 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 -- 172.21.15.114:0/844254393 wait complete. 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 -- start start 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.147+0000 7f72d7084740 1 Processor -- start 2024-09-21T07:04:31.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693001d390 0x5569303f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 0x556930465e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b4680 con 0x55693003e070 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b3fb0 con 0x5569300151b0 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b1210 con 0x55693001d390 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72d7084740 10 client.0 _ll_register_callbacks cb 0x7f72bc009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72ce400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 0x556930465e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72ce400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 0x556930465e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33458/0 (socket says 172.21.15.114:33458) 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72cda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72ce400640 1 -- 172.21.15.114:0/1374795965 learned_addr learned my addr 172.21.15.114:0/1374795965 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.148+0000 7f72cd000640 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693001d390 0x5569303f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72ce400640 1 -- 172.21.15.114:0/1374795965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693001d390 msgr2=0x5569303f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72ce400640 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693001d390 0x5569303f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72ce400640 1 -- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 msgr2=0x5569303f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72ce400640 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72ce400640 1 -- 172.21.15.114:0/1374795965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569304664d0 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72ce400640 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 0x556930465e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f72c00950b0 tx=0x7f72c00d12b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e3a446dce14bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72af400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72af400640 1 -- 172.21.15.114:0/1374795965 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c00d5070 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72af400640 1 -- 172.21.15.114:0/1374795965 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f72c00b7050 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b4031bb0 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72af400640 1 -- 172.21.15.114:0/1374795965 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f72c00ad5d0 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72d7084740 10 client.17694 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.149+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55693046d880 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72af400640 1 -- 172.21.15.114:0/1374795965 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b4031bb0 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72af400640 1 -- 172.21.15.114:0/1374795965 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f72c00e5b10 con 0x55693003e070 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72d7084740 10 client.17694 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:31.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72d7084740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72d7084740 20 client.17694 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72d7084740 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692ffd1da0 0x55693046ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72d7084740 4 client.17694 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556930359970 con 0x55692ffd1da0 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.150+0000 7f72cda00640 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692ffd1da0 0x55693046ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72cda00640 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692ffd1da0 0x55693046ffe0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe63176abda45d01 server_cookie=f82b4f7d66360379 in_seq=0 out_seq=0 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72af400640 10 client.17694 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72af400640 1 -- 172.21.15.114:0/1374795965 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x556930359970 con 0x55692ffd1da0 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72af400640 10 client.17694 handle_command_reply: tid=0 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 client.17694 shutdown 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 msgr2=0x556930465e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 0x556930465e60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f72c00950b0 tx=0x7f72c00d12b0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 shutdown_connections 2024-09-21T07:04:31.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55693003e070 0x556930465e60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55692ffd1da0 0x55693046ffe0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55693001d390 0x5569303f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72af400640 0 client.17694 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 --2- 172.21.15.114:0/1374795965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5569300151b0 0x5569303f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.151+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 >> 172.21.15.114:0/1374795965 conn(0x556930032460 msgr2=0x5569303eeda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:04:31.581 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "73ddb0e8-eb68-4bca-bf13-582ff62ccebc", 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.582 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.583 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.584 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.585 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:04:31.586 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:31.587 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 shutdown_connections 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d7084740 1 -- 172.21.15.114:0/1374795965 wait complete. 2024-09-21T07:04:31.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d7084740 20 client.17694 trim_cache size 0 max 16384 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6700 msgr2=0x7f72b00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6700 0x7f72b00b8c10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f72b4005ab0 tx=0x7f72b4005780 comp rx=0 tx=0).stop 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 msgr2=0x7f72c81a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c81a1f30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f72c000eb10 tx=0x7f72c000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 shutdown_connections 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f72c810de30 0x7f72c81a1f30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.152+0000 7f72d0400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72c8105320 0x7f72c819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.153+0000 7f72d0400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f72b00b6700 0x7f72b00b8c10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.153+0000 7f72d0400640 1 --2- 172.21.15.114:0/2985282027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72c81049d0 0x7f72c819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.153+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 >> 172.21.15.114:0/2985282027 conn(0x7f72c8079740 msgr2=0x7f72c81036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.153+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 shutdown_connections 2024-09-21T07:04:31.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:31.153+0000 7f72d0400640 1 -- 172.21.15.114:0/2985282027 wait complete. 2024-09-21T07:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 -- 172.21.15.114:0/3435564007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0109d80 msgr2=0x7f7fb01105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 --2- 172.21.15.114:0/3435564007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0109d80 0x7f7fb01105b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009ca0 tx=0x7f7fa802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 -- 172.21.15.114:0/3435564007 shutdown_connections 2024-09-21T07:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 --2- 172.21.15.114:0/3435564007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0109d80 0x7f7fb01105b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 --2- 172.21.15.114:0/3435564007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fb0109430 0x7f7fb0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 --2- 172.21.15.114:0/3435564007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fb0103b00 0x7f7fb0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 -- 172.21.15.114:0/3435564007 >> 172.21.15.114:0/3435564007 conn(0x7f7fb00ff500 msgr2=0x7f7fb0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 -- 172.21.15.114:0/3435564007 shutdown_connections 2024-09-21T07:04:32.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.166+0000 7f7fb8600640 1 -- 172.21.15.114:0/3435564007 wait complete. 2024-09-21T07:04:32.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.167+0000 7f7fb8600640 1 Processor -- start 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.167+0000 7f7fb8600640 1 -- start start 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb8600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 0x7f7fb019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb8600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fb0109430 0x7f7fb019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb8600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fb0109d80 0x7f7fb019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb8600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0113a40 con 0x7f7fb0103b00 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb8600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01138c0 con 0x7f7fb0109430 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb8600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0113bc0 con 0x7f7fb0109d80 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 0x7f7fb019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 0x7f7fb019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33490/0 (socket says 172.21.15.114:33490) 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 -- 172.21.15.114:0/1643832118 learned_addr learned my addr 172.21.15.114:0/1643832118 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:32.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fb0109430 0x7f7fb019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb6600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fb0109d80 0x7f7fb019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 -- 172.21.15.114:0/1643832118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fb0109d80 msgr2=0x7f7fb019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fb0109d80 0x7f7fb019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 -- 172.21.15.114:0/1643832118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fb0109430 msgr2=0x7f7fb019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fb0109430 0x7f7fb019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.168+0000 7f7fb5c00640 1 -- 172.21.15.114:0/1643832118 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb01a0350 con 0x7f7fb0103b00 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.169+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 0x7f7fb019a420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d950 tx=0x7f7fa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.169+0000 7f7fae000640 1 -- 172.21.15.114:0/1643832118 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4017030 con 0x7f7fb0103b00 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.169+0000 7f7fae000640 1 -- 172.21.15.114:0/1643832118 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa4004020 con 0x7f7fb0103b00 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.169+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb01a0640 con 0x7f7fb0103b00 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.169+0000 7f7fae000640 1 -- 172.21.15.114:0/1643832118 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40047a0 con 0x7f7fb0103b00 2024-09-21T07:04:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.169+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb01a6dd0 con 0x7f7fb0103b00 2024-09-21T07:04:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.172+0000 7f7fbf673740 1 Processor -- start 2024-09-21T07:04:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.172+0000 7f7fbf673740 1 -- start start 2024-09-21T07:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.172+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde891b0 0x55aabe232040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe232580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 0x55aabe232ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fbf673740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55aabe1a36b0 con 0x55aabde684d0 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fbf673740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aabe192ae0 con 0x55aabde891b0 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fbf673740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aabe1968f0 con 0x55aabde602f0 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fae000640 1 -- 172.21.15.114:0/1643832118 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7fa4009a30 con 0x7f7fb0103b00 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 0x55aabe232ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde891b0 0x55aabe232040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 0x55aabe232ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33504/0 (socket says 172.21.15.114:33504) 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 -- 172.21.15.114:0/3540608373 learned_addr learned my addr 172.21.15.114:0/3540608373 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe232580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 -- 172.21.15.114:0/3540608373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 msgr2=0x55aabe232580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe232580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 -- 172.21.15.114:0/3540608373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde891b0 msgr2=0x55aabe232040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde891b0 0x55aabe232040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5200640 1 -- 172.21.15.114:0/3540608373 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabe15f780 con 0x55aabde684d0 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe232580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.173+0000 7f7fb6600640 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde891b0 0x55aabe232040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.174+0000 7f7fb5200640 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 0x55aabe232ac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009c00 tx=0x7f7f9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b4ce56d7dda435 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.174+0000 7f7fa2a00640 1 -- 172.21.15.114:0/3540608373 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c004050 con 0x55aabde684d0 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.174+0000 7f7fa2a00640 1 -- 172.21.15.114:0/3540608373 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9c0054d0 con 0x55aabde684d0 2024-09-21T07:04:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.174+0000 7f7fa2a00640 1 -- 172.21.15.114:0/3540608373 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c004c60 con 0x55aabde684d0 2024-09-21T07:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.174+0000 7f7fae000640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7f980b6700 0x7f7f980b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 -- 172.21.15.114:0/3540608373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 msgr2=0x55aabe232ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 0x55aabe232ac0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009c00 tx=0x7f7f9c011810 comp rx=0 tx=0).stop 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fae000640 1 -- 172.21.15.114:0/1643832118 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7fa40e7530 con 0x7f7fb0103b00 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fb6600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7f980b6700 0x7f7f980b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 -- 172.21.15.114:0/3540608373 shutdown_connections 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde684d0 0x55aabe232ac0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe232580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3540608373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde891b0 0x55aabe232040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 -- 172.21.15.114:0/3540608373 >> 172.21.15.114:0/3540608373 conn(0x55aabde7d300 msgr2=0x55aabe22ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 -- 172.21.15.114:0/3540608373 shutdown_connections 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fbf673740 1 -- 172.21.15.114:0/3540608373 wait complete. 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.175+0000 7f7fb6600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7f980b6700 0x7f7f980b8c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7fa802f500 tx=0x7f7fa802f490 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 -- start start 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 Processor -- start 2024-09-21T07:04:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 0x55aabe23d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe23da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde891b0 0x55aabe23dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55aabe0caab0 con 0x55aabde602f0 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aabe139570 con 0x55aabde684d0 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55aabe2407c0 con 0x55aabde891b0 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.176+0000 7f7fbf673740 10 client.0 _ll_register_callbacks cb 0x55aabe148100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 0x55aabe23d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 0x55aabe23d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33518/0 (socket says 172.21.15.114:33518) 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 -- 172.21.15.114:0/2486485697 learned_addr learned my addr 172.21.15.114:0/2486485697 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5200640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe23da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb6600640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde891b0 0x55aabe23dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 -- 172.21.15.114:0/2486485697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde891b0 msgr2=0x55aabe23dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde891b0 0x55aabe23dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 -- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 msgr2=0x55aabe23da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe23da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 -- 172.21.15.114:0/2486485697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabdef4790 con 0x55aabde602f0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5200640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe23da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 0x55aabe23d530 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40aa1c0 tx=0x7f7fa40acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfba1ab8e793722 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7f97400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.177+0000 7f7f97400640 1 -- 172.21.15.114:0/2486485697 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40ac6c0 con 0x55aabde602f0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7f97400640 1 -- 172.21.15.114:0/2486485697 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7fa40e7130 con 0x55aabde602f0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9c0054d0 con 0x55aabde602f0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7f97400640 1 -- 172.21.15.114:0/2486485697 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7fa40e7330 con 0x55aabde602f0 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 10 client.17706 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55aabdf6d620 con 0x55aabde602f0 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7f97400640 1 -- 172.21.15.114:0/2486485697 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f9c0054d0 con 0x55aabde602f0 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7f97400640 1 -- 172.21.15.114:0/2486485697 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f7fa40e4ad0 con 0x55aabde602f0 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 10 client.17706 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 10 client.17706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 20 client.17706 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde47d80 0x55aabe24f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 4 client.17706 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.178+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aabe1a5600 con 0x55aabde47d80 2024-09-21T07:04:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.179+0000 7f7fb5200640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde47d80 0x55aabe24f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.179+0000 7f7fb5200640 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde47d80 0x55aabe24f640 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ff4eb07711ca2e server_cookie=441f6c20c56b2f5e in_seq=0 out_seq=0 2024-09-21T07:04:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.179+0000 7f7f97400640 10 client.17706 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.180+0000 7f7f97400640 1 -- 172.21.15.114:0/2486485697 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aabe1a5600 con 0x55aabde47d80 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.180+0000 7f7f97400640 10 client.17706 handle_command_reply: tid=0 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.180+0000 7f7fbf673740 1 client.17706 shutdown 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.180+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 msgr2=0x55aabe23d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.180+0000 7f7fbf673740 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 0x55aabe23d530 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40aa1c0 tx=0x7f7fa40acd20 comp rx=0 tx=0).stop 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 shutdown_connections 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde47d80 0x55aabe24f640 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde891b0 0x55aabe23dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe23da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7f97400640 0 client.17706 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 --2- 172.21.15.114:0/2486485697 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde602f0 0x55aabe23d530 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 >> 172.21.15.114:0/2486485697 conn(0x55aabde7d300 msgr2=0x55aabe233070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 shutdown_connections 2024-09-21T07:04:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 1 -- 172.21.15.114:0/2486485697 wait complete. 2024-09-21T07:04:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.181+0000 7f7fbf673740 20 client.17706 trim_cache size 0 max 16384 2024-09-21T07:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.196+0000 7f7fbf673740 1 Processor -- start 2024-09-21T07:04:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.196+0000 7f7fbf673740 1 -- start start 2024-09-21T07:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe22aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe233780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe23a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fbf673740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01aec40 con 0x55aabde891b0 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fbf673740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b0780 con 0x55aabde684d0 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fbf673740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01af310 con 0x55aabde602f0 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe22aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb6600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe23a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe22aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47294/0 (socket says 172.21.15.114:47294) 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 -- 172.21.15.114:0/4135601479 learned_addr learned my addr 172.21.15.114:0/4135601479 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5200640 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe233780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 -- 172.21.15.114:0/4135601479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 msgr2=0x55aabe233780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe233780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 -- 172.21.15.114:0/4135601479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 msgr2=0x55aabe23a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe23a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.197+0000 7f7fb5c00640 1 -- 172.21.15.114:0/4135601479 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabe234570 con 0x55aabde602f0 2024-09-21T07:04:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.198+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe22aaa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005fd0 tx=0x7f7fa4004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e6d6b11496d559c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.198+0000 7f7fa2000640 1 -- 172.21.15.114:0/4135601479 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40ac6c0 con 0x55aabde602f0 2024-09-21T07:04:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.198+0000 7f7fa2000640 1 -- 172.21.15.114:0/4135601479 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40aca60 con 0x55aabde602f0 2024-09-21T07:04:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.198+0000 7f7fa2000640 1 -- 172.21.15.114:0/4135601479 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40a9350 con 0x55aabde602f0 2024-09-21T07:04:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 -- 172.21.15.114:0/4135601479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 msgr2=0x55aabe22aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe22aaa0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005fd0 tx=0x7f7fa4004560 comp rx=0 tx=0).stop 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 -- 172.21.15.114:0/4135601479 shutdown_connections 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe23a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde684d0 0x55aabe233780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 --2- 172.21.15.114:0/4135601479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde602f0 0x55aabe22aaa0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 -- 172.21.15.114:0/4135601479 >> 172.21.15.114:0/4135601479 conn(0x55aabde7d300 msgr2=0x55aabe233070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 -- 172.21.15.114:0/4135601479 shutdown_connections 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 -- 172.21.15.114:0/4135601479 wait complete. 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 -- start start 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.199+0000 7f7fbf673740 1 Processor -- start 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde602f0 0x55aabe243c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 0x55aabe244180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe2446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01b00b0 con 0x55aabde891b0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01af9e0 con 0x55aabde602f0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01acc40 con 0x55aabde684d0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fbf673740 10 client.0 _ll_register_callbacks cb 0x7f7fa40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fb5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 0x55aabe244180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fb5200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 0x55aabe244180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:47310/0 (socket says 172.21.15.114:47310) 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fb5200640 1 -- 172.21.15.114:0/3218735041 learned_addr learned my addr 172.21.15.114:0/3218735041 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fb6600640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe2446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.200+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde602f0 0x55aabe243c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5200640 1 -- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde602f0 msgr2=0x55aabe243c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5200640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde602f0 0x55aabe243c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5200640 1 -- 172.21.15.114:0/3218735041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 msgr2=0x55aabe2446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5200640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe2446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5200640 1 -- 172.21.15.114:0/3218735041 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabe2b9240 con 0x55aabde684d0 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde602f0 0x55aabe243c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fb5200640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 0x55aabe244180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009c00 tx=0x7f7f9c009f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f643492d9decd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7f97400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7f97400640 1 -- 172.21.15.114:0/3218735041 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0037f0 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7f97400640 1 -- 172.21.15.114:0/3218735041 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7f9c003990 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.201+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa40aca60 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7f97400640 1 -- 172.21.15.114:0/3218735041 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7f9c03f050 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 10 client.28123 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55aabe2ba4d0 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7f97400640 1 -- 172.21.15.114:0/3218735041 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40aca60 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7f97400640 1 -- 172.21.15.114:0/3218735041 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f7f9c019bd0 con 0x55aabde684d0 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 10 client.28123 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 10 client.28123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 20 client.28123 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde1d1c0 0x55aabe2bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 4 client.28123 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:04:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.202+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aabe1a10b0 con 0x55aabde1d1c0 2024-09-21T07:04:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.203+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde1d1c0 0x55aabe2bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.203+0000 7f7f97400640 10 client.28123 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.203+0000 7f7fb5c00640 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde1d1c0 0x55aabe2bca10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=726a4778f54d5195 server_cookie=4b6ca9406806421c in_seq=0 out_seq=0 2024-09-21T07:04:32.213 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.211+0000 7f7f97400640 1 -- 172.21.15.114:0/3218735041 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aabe1a10b0 con 0x55aabde1d1c0 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7f97400640 10 client.28123 handle_command_reply: tid=0 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 client.28123 shutdown 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 msgr2=0x55aabe244180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 0x55aabe244180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009c00 tx=0x7f7f9c009f40 comp rx=0 tx=0).stop 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 shutdown_connections 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55aabde891b0 0x55aabe2446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55aabde1d1c0 0x55aabe2bca10 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aabde684d0 0x55aabe244180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 --2- 172.21.15.114:0/3218735041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aabde602f0 0x55aabe243c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7f97400640 0 client.28123 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 >> 172.21.15.114:0/3218735041 conn(0x55aabde7d300 msgr2=0x55aabe23f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 shutdown_connections 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 1 -- 172.21.15.114:0/3218735041 wait complete. 2024-09-21T07:04:32.214 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.212+0000 7f7fbf673740 20 client.28123 trim_cache size 0 max 16384 2024-09-21T07:04:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:04:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:04:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:04:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7f980b6700 msgr2=0x7f7f980b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7f980b6700 0x7f7f980b8c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7fa802f500 tx=0x7f7fa802f490 comp rx=0 tx=0).stop 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 msgr2=0x7f7fb019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 0x7f7fb019a420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d950 tx=0x7f7fa400de10 comp rx=0 tx=0).stop 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 shutdown_connections 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7fb0109d80 0x7f7fb019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7f980b6700 0x7f7f980b8c10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7fb0109430 0x7f7fb019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 --2- 172.21.15.114:0/1643832118 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7fb0103b00 0x7f7fb019a420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 >> 172.21.15.114:0/1643832118 conn(0x7f7fb00ff500 msgr2=0x7f7fb0108a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 shutdown_connections 2024-09-21T07:04:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:32.213+0000 7f7fb8600640 1 -- 172.21.15.114:0/1643832118 wait complete. 2024-09-21T07:04:32.266 INFO:tasks.cephfs.filesystem:scrub status for tag:73ddb0e8-eb68-4bca-bf13-582ff62ccebc - {'path': '/', 'tag': '73ddb0e8-eb68-4bca-bf13-582ff62ccebc', 'options': 'recursive,force'} 2024-09-21T07:04:32.377 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:32.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 595.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:04:33.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 -- 172.21.15.114:0/763529918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c105bb0 msgr2=0x7f903c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 --2- 172.21.15.114:0/763529918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c105bb0 0x7f903c105fc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9030009900 tx=0x7f903002a410 comp rx=0 tx=0).stop 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 -- 172.21.15.114:0/763529918 shutdown_connections 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 --2- 172.21.15.114:0/763529918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903c10be40 0x7f903c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 --2- 172.21.15.114:0/763529918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c10b4f0 0x7f903c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 --2- 172.21.15.114:0/763529918 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c105bb0 0x7f903c105fc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 -- 172.21.15.114:0/763529918 >> 172.21.15.114:0/763529918 conn(0x7f903c101680 msgr2=0x7f903c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.028+0000 7f9042a00640 1 -- 172.21.15.114:0/763529918 shutdown_connections 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.029+0000 7f9042a00640 1 -- 172.21.15.114:0/763529918 wait complete. 2024-09-21T07:04:33.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.030+0000 7f9042a00640 1 Processor -- start 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.030+0000 7f9042a00640 1 -- start start 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.030+0000 7f9042a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c105bb0 0x7f903c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903c10b4f0 0x7f903c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 0x7f903c1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f903c115b00 con 0x7f903c10be40 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f903c115980 con 0x7f903c10b4f0 2024-09-21T07:04:33.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f903c115c80 con 0x7f903c105bb0 2024-09-21T07:04:33.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 0x7f903c1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 0x7f903c1a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33554/0 (socket says 172.21.15.114:33554) 2024-09-21T07:04:33.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9041600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c105bb0 0x7f903c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 -- 172.21.15.114:0/4118681848 learned_addr learned my addr 172.21.15.114:0/4118681848 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9040c00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903c10b4f0 0x7f903c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 -- 172.21.15.114:0/4118681848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c105bb0 msgr2=0x7f903c19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c105bb0 0x7f903c19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 -- 172.21.15.114:0/4118681848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903c10b4f0 msgr2=0x7f903c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903c10b4f0 0x7f903c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9042000640 1 -- 172.21.15.114:0/4118681848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903c1a2500 con 0x7f903c10be40 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.031+0000 7f9041600640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c105bb0 0x7f903c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.032+0000 7f9042000640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 0x7f903c1a1dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f903400eb10 tx=0x7f903400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.032+0000 7f903a000640 1 -- 172.21.15.114:0/4118681848 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034004b20 con 0x7f903c10be40 2024-09-21T07:04:33.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.032+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903c1a27f0 con 0x7f903c10be40 2024-09-21T07:04:33.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.032+0000 7f903a000640 1 -- 172.21.15.114:0/4118681848 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9034004cc0 con 0x7f903c10be40 2024-09-21T07:04:33.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.032+0000 7f903a000640 1 -- 172.21.15.114:0/4118681848 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034010670 con 0x7f903c10be40 2024-09-21T07:04:33.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.032+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f903c1a8f50 con 0x7f903c10be40 2024-09-21T07:04:33.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.035+0000 7f904ac5c740 1 Processor -- start 2024-09-21T07:04:33.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.036+0000 7f903a000640 1 -- 172.21.15.114:0/4118681848 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9034005020 con 0x7f903c10be40 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.036+0000 7f904ac5c740 1 -- start start 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.036+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.036+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f904ac5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562b57654290 con 0x562b57315840 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f904ac5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b576436c0 con 0x562b57336520 2024-09-21T07:04:33.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f904ac5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b576474d0 con 0x562b5730d660 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576dd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45976/0 (socket says 172.21.15.114:45976) 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 -- 172.21.15.114:0/25091275 learned_addr learned my addr 172.21.15.114:0/25091275 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9041600640 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 -- 172.21.15.114:0/25091275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 msgr2=0x562b576dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 -- 172.21.15.114:0/25091275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 msgr2=0x562b576dddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576dddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.037+0000 7f9040c00640 1 -- 172.21.15.114:0/25091275 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b575d0460 con 0x562b57336520 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f9040c00640 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576dd340 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90280058c0 tx=0x7f9028005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872504bf8223b406 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:33.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f903a000640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90240b6680 0x7f90240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f902e600640 1 -- 172.21.15.114:0/25091275 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028005020 con 0x562b57336520 2024-09-21T07:04:33.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f902e600640 1 -- 172.21.15.114:0/25091275 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90280047a0 con 0x562b57336520 2024-09-21T07:04:33.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f902e600640 1 -- 172.21.15.114:0/25091275 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902800fa20 con 0x562b57336520 2024-09-21T07:04:33.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f903a000640 1 -- 172.21.15.114:0/4118681848 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f90340e80d0 con 0x7f903c10be40 2024-09-21T07:04:33.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.038+0000 7f9041600640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90240b6680 0x7f90240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 -- 172.21.15.114:0/25091275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 msgr2=0x562b576dd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576dd340 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90280058c0 tx=0x7f9028005850 comp rx=0 tx=0).stop 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f9041600640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90240b6680 0x7f90240b8b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9030031440 tx=0x7f9030032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 -- 172.21.15.114:0/25091275 shutdown_connections 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576dddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 --2- 172.21.15.114:0/25091275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576dd340 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 -- 172.21.15.114:0/25091275 >> 172.21.15.114:0/25091275 conn(0x562b5732a910 msgr2=0x562b576db230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 -- 172.21.15.114:0/25091275 shutdown_connections 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.039+0000 7f904ac5c740 1 -- 172.21.15.114:0/25091275 wait complete. 2024-09-21T07:04:33.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.040+0000 7f904ac5c740 1 -- start start 2024-09-21T07:04:33.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.040+0000 7f904ac5c740 1 Processor -- start 2024-09-21T07:04:33.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b5730d660 0x562b576f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b57336520 0x562b576f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562b574a4bb0 con 0x562b57315840 2024-09-21T07:04:33.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b575de6c0 con 0x562b5730d660 2024-09-21T07:04:33.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b576ebb20 con 0x562b57336520 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f904ac5c740 10 client.0 _ll_register_callbacks cb 0x562b575f9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9041600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b5730d660 0x562b576f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9040c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9041600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b5730d660 0x562b576f1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:45984/0 (socket says 172.21.15.114:45984) 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9041600640 1 -- 172.21.15.114:0/3719724702 learned_addr learned my addr 172.21.15.114:0/3719724702 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9042000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b57336520 0x562b576f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9040c00640 1 -- 172.21.15.114:0/3719724702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b57336520 msgr2=0x562b576f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9040c00640 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b57336520 0x562b576f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9040c00640 1 -- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b5730d660 msgr2=0x562b576f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9040c00640 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b5730d660 0x562b576f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.041+0000 7f9040c00640 1 -- 172.21.15.114:0/3719724702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b573a1b00 con 0x562b57315840 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f9040c00640 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f2410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9028009660 tx=0x7f9028004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69231c19db89219b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f9022a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f9022a00640 1 -- 172.21.15.114:0/3719724702 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902802b060 con 0x562b57315840 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f9022a00640 1 -- 172.21.15.114:0/3719724702 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f9028004be0 con 0x562b57315840 2024-09-21T07:04:33.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90280047a0 con 0x562b57315840 2024-09-21T07:04:33.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f9022a00640 1 -- 172.21.15.114:0/3719724702 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9028032050 con 0x562b57315840 2024-09-21T07:04:33.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f904ac5c740 10 client.17724 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:33.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.042+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562b57418960 con 0x562b57315840 2024-09-21T07:04:33.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f9022a00640 1 -- 172.21.15.114:0/3719724702 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90280047a0 con 0x562b57315840 2024-09-21T07:04:33.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f9022a00640 1 -- 172.21.15.114:0/3719724702 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f902801d030 con 0x562b57315840 2024-09-21T07:04:33.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f904ac5c740 10 client.17724 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:33.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f904ac5c740 10 client.17724 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f904ac5c740 20 client.17724 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f904ac5c740 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572f4ea0 0x562b57703570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f904ac5c740 4 client.17724 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.043+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b576561e0 con 0x562b572f4ea0 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.044+0000 7f9041600640 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572f4ea0 0x562b57703570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.044+0000 7f9041600640 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572f4ea0 0x562b57703570 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41d7a265e0702bf9 server_cookie=585b6181cd5116cf in_seq=0 out_seq=0 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.044+0000 7f9022a00640 10 client.17724 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.045+0000 7f9022a00640 1 -- 172.21.15.114:0/3719724702 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b576561e0 con 0x562b572f4ea0 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.045+0000 7f9022a00640 10 client.17724 handle_command_reply: tid=0 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.045+0000 7f904ac5c740 1 client.17724 shutdown 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.045+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 msgr2=0x562b576f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.045+0000 7f904ac5c740 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f2410 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9028009660 tx=0x7f9028004f20 comp rx=0 tx=0).stop 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 shutdown_connections 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572f4ea0 0x562b57703570 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b57336520 0x562b576f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f2410 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f9022a00640 0 client.17724 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:33.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 --2- 172.21.15.114:0/3719724702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b5730d660 0x562b576f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 >> 172.21.15.114:0/3719724702 conn(0x562b5732a910 msgr2=0x562b576e04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:33.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 shutdown_connections 2024-09-21T07:04:33.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 1 -- 172.21.15.114:0/3719724702 wait complete. 2024-09-21T07:04:33.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.046+0000 7f904ac5c740 20 client.17724 trim_cache size 0 max 16384 2024-09-21T07:04:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 Processor -- start 2024-09-21T07:04:33.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 -- start start 2024-09-21T07:04:33.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57315840 0x562b576f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 0x562b576f7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1aebb0 con 0x562b57336520 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b06f0 con 0x562b57315840 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f904ac5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1af280 con 0x562b5730d660 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f9042000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 0x562b576f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f9042000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 0x562b576f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33586/0 (socket says 172.21.15.114:33586) 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.064+0000 7f9042000640 1 -- 172.21.15.114:0/1198091423 learned_addr learned my addr 172.21.15.114:0/1198091423 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9041600640 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9040c00640 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57315840 0x562b576f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9042000640 1 -- 172.21.15.114:0/1198091423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 msgr2=0x562b576dea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9042000640 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9042000640 1 -- 172.21.15.114:0/1198091423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57315840 msgr2=0x562b576f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9042000640 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57315840 0x562b576f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9042000640 1 -- 172.21.15.114:0/1198091423 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b576eb650 con 0x562b57336520 2024-09-21T07:04:33.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f9042000640 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 0x562b576f7750 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f90340aa0c0 tx=0x7f9034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2818409c026181 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:33.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f902dc00640 1 -- 172.21.15.114:0/1198091423 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340b2070 con 0x562b57336520 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f902dc00640 1 -- 172.21.15.114:0/1198091423 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90340ad600 con 0x562b57336520 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.065+0000 7f902dc00640 1 -- 172.21.15.114:0/1198091423 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90340b7510 con 0x562b57336520 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 -- 172.21.15.114:0/1198091423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 msgr2=0x562b576f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 0x562b576f7750 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f90340aa0c0 tx=0x7f9034004a40 comp rx=0 tx=0).stop 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 -- 172.21.15.114:0/1198091423 shutdown_connections 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57336520 0x562b576f7750 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57315840 0x562b576f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 --2- 172.21.15.114:0/1198091423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576dea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 -- 172.21.15.114:0/1198091423 >> 172.21.15.114:0/1198091423 conn(0x562b5732a910 msgr2=0x562b576de370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 -- 172.21.15.114:0/1198091423 shutdown_connections 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.066+0000 7f904ac5c740 1 -- 172.21.15.114:0/1198091423 wait complete. 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.067+0000 7f904ac5c740 1 -- start start 2024-09-21T07:04:33.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.067+0000 7f904ac5c740 1 Processor -- start 2024-09-21T07:04:33.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.067+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.067+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.067+0000 7f904ac5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f904ac5c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1b0020 con 0x562b57315840 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f904ac5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1af950 con 0x562b57336520 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f904ac5c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f903c1acbb0 con 0x562b5730d660 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f904ac5c740 10 client.0 _ll_register_callbacks cb 0x7f9028009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9042000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9042000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576f91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46006/0 (socket says 172.21.15.114:46006) 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:33596/0 (socket says 172.21.15.114:33596) 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9042000640 1 -- 172.21.15.114:0/2542213057 learned_addr learned my addr 172.21.15.114:0/2542213057 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9041600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 -- 172.21.15.114:0/2542213057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 msgr2=0x562b576f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 -- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 msgr2=0x562b576f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9040c00640 1 -- 172.21.15.114:0/2542213057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b5776dda0 con 0x562b57315840 2024-09-21T07:04:33.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9041600640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.068+0000 7f9042000640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f9040c00640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f8cb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9028009710 tx=0x7f902800fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd26d50a0c4d12e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f9022a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f9022a00640 1 -- 172.21.15.114:0/2542213057 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902802b070 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f9022a00640 1 -- 172.21.15.114:0/2542213057 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f9028004be0 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90340ad600 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f9022a00640 1 -- 172.21.15.114:0/2542213057 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f902800f790 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f904ac5c740 10 client.17730 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562b5776ef80 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.069+0000 7f9022a00640 1 -- 172.21.15.114:0/2542213057 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90340ad600 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f9022a00640 1 -- 172.21.15.114:0/2542213057 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f902801d040 con 0x562b57315840 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f904ac5c740 10 client.17730 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f904ac5c740 10 client.17730 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f904ac5c740 20 client.17730 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f904ac5c740 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572ca250 0x562b57771700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:33.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f904ac5c740 4 client.17730 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "230636ca", "release": true, "await": true}] 2024-09-21T07:04:33.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "230636ca", "release": true, "await": true}) -- 0x562b57651c90 con 0x562b572ca250 2024-09-21T07:04:33.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.070+0000 7f9041600640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572ca250 0x562b57771700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:33.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.071+0000 7f9022a00640 10 client.17730 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:33.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.071+0000 7f9041600640 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572ca250 0x562b57771700 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac38fa5cf098b3bb server_cookie=c319abf7086e7303 in_seq=0 out_seq=0 2024-09-21T07:04:33.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.247+0000 7f9022a00640 1 -- 172.21.15.114:0/2542213057 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562b57651c90 con 0x562b572ca250 2024-09-21T07:04:33.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.247+0000 7f9022a00640 10 client.17730 handle_command_reply: tid=0 2024-09-21T07:04:33.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.247+0000 7f904ac5c740 1 client.17730 shutdown 2024-09-21T07:04:33.249 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.247+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 msgr2=0x562b576f8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.250 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.247+0000 7f904ac5c740 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f8cb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9028009710 tx=0x7f902800fdb0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.250 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:32 smithi114 ceph-mon[30654]: pgmap v716: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 36 KiB/s wr, 45 op/s 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 shutdown_connections 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b57336520 0x562b576f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b57315840 0x562b576f8cb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b572ca250 0x562b57771700 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 --2- 172.21.15.114:0/2542213057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b5730d660 0x562b576f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 >> 172.21.15.114:0/2542213057 conn(0x562b5732a910 msgr2=0x562b576de350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f9022a00640 0 client.17730 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 shutdown_connections 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.248+0000 7f904ac5c740 1 -- 172.21.15.114:0/2542213057 wait complete. 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.249+0000 7f904ac5c740 20 client.17730 trim_cache size 0 max 16384 2024-09-21T07:04:33.251 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 49, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 40, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "230636ca": { 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 40, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:04:33.252 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 144.5, 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:33.253 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.249+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90240b6680 msgr2=0x7f90240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.249+0000 7f9042a00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90240b6680 0x7f90240b8b90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9030031440 tx=0x7f9030032ed0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.249+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 msgr2=0x7f903c1a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.249+0000 7f9042a00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 0x7f903c1a1dd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f903400eb10 tx=0x7f903400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 shutdown_connections 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f903c10be40 0x7f903c1a1dd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90240b6680 0x7f90240b8b90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903c10b4f0 0x7f903c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 --2- 172.21.15.114:0/4118681848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f903c105bb0 0x7f903c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:33.254 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 >> 172.21.15.114:0/4118681848 conn(0x7f903c101680 msgr2=0x7f903c103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 shutdown_connections 2024-09-21T07:04:33.255 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:33.250+0000 7f9042a00640 1 -- 172.21.15.114:0/4118681848 wait complete. 2024-09-21T07:04:33.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:32 smithi183 ceph-mon[34219]: pgmap v716: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 36 KiB/s wr, 45 op/s 2024-09-21T07:04:33.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:32 smithi193 ceph-mon[34601]: pgmap v716: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 36 KiB/s wr, 45 op/s 2024-09-21T07:04:34.582 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '230636ca', total seconds elapsed: 30.4 2024-09-21T07:04:34.582 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-21T07:04:34.582 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:04:35.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:35 smithi114 ceph-mon[30654]: pgmap v717: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 21 KiB/s wr, 100 op/s 2024-09-21T07:04:35.347 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:35.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:35 smithi183 ceph-mon[34219]: pgmap v717: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 21 KiB/s wr, 100 op/s 2024-09-21T07:04:35.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:35 smithi193 ceph-mon[34601]: pgmap v717: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 21 KiB/s wr, 100 op/s 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.983+0000 7f0b91600640 1 -- 172.21.15.114:0/267421710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 msgr2=0x7f0b8c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.983+0000 7f0b91600640 1 --2- 172.21.15.114:0/267421710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c102c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009900 tx=0x7f0b7c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 -- 172.21.15.114:0/267421710 shutdown_connections 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 --2- 172.21.15.114:0/267421710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c108b00 0x7f0b8c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 --2- 172.21.15.114:0/267421710 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b8c1081b0 0x7f0b8c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 --2- 172.21.15.114:0/267421710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c102c80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 -- 172.21.15.114:0/267421710 >> 172.21.15.114:0/267421710 conn(0x7f0b8c0fe2c0 msgr2=0x7f0b8c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 -- 172.21.15.114:0/267421710 shutdown_connections 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.984+0000 7f0b91600640 1 -- 172.21.15.114:0/267421710 wait complete. 2024-09-21T07:04:36.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.986+0000 7f0b91600640 1 Processor -- start 2024-09-21T07:04:36.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 -- start start 2024-09-21T07:04:36.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c1126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b8c1081b0 0x7f0b8c112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c108b00 0x7f0b8c113140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c110bd0 con 0x7f0b8c1081b0 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c110a50 con 0x7f0b8c102870 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.989+0000 7f0b91600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c110d50 con 0x7f0b8c108b00 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.990+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c1126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:36.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.990+0000 7f0b8be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c1126c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60854/0 (socket says 172.21.15.114:60854) 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.990+0000 7f0b8be00640 1 -- 172.21.15.114:0/3425674071 learned_addr learned my addr 172.21.15.114:0/3425674071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.990+0000 7f0b90c00640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c108b00 0x7f0b8c113140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.991+0000 7f0b8be00640 1 -- 172.21.15.114:0/3425674071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c108b00 msgr2=0x7f0b8c113140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.991+0000 7f0b8be00640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c108b00 0x7f0b8c113140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.991+0000 7f0b8be00640 1 -- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b8c1081b0 msgr2=0x7f0b8c112c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.991+0000 7f0b8be00640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b8c1081b0 0x7f0b8c112c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:36.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.991+0000 7f0b8be00640 1 -- 172.21.15.114:0/3425674071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1b51a0 con 0x7f0b8c102870 2024-09-21T07:04:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.992+0000 7f0b8be00640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c1126c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c031440 tx=0x7f0b7c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.992+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c03f070 con 0x7f0b8c102870 2024-09-21T07:04:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.992+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c003e40 con 0x7f0b8c102870 2024-09-21T07:04:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.992+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c035030 con 0x7f0b8c102870 2024-09-21T07:04:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.993+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1b5430 con 0x7f0b8c102870 2024-09-21T07:04:36.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.994+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1b58c0 con 0x7f0b8c102870 2024-09-21T07:04:36.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:36.998+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c102c80 con 0x7f0b8c102870 2024-09-21T07:04:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.001+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0b7c042070 con 0x7f0b8c102870 2024-09-21T07:04:37.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.004+0000 7f0b88c00640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b700b6800 0x7f0b700b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.004+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f0b7c107ef0 con 0x7f0b8c102870 2024-09-21T07:04:37.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.006+0000 7f0b8b400640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b700b6800 0x7f0b700b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:37.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.010+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0b7c0ce070 con 0x7f0b8c102870 2024-09-21T07:04:37.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.011+0000 7f0b8b400640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b700b6800 0x7f0b700b8d10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005fd0 tx=0x7f0b78005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:37.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:37 smithi114 ceph-mon[30654]: pgmap v718: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 55 KiB/s wr, 104 op/s 2024-09-21T07:04:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:37 smithi183 ceph-mon[34219]: pgmap v718: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 55 KiB/s wr, 104 op/s 2024-09-21T07:04:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:37 smithi193 ceph-mon[34601]: pgmap v718: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 55 KiB/s wr, 104 op/s 2024-09-21T07:04:37.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.441+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8c10a8d0 con 0x7f0b8c102870 2024-09-21T07:04:37.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.442+0000 7f0b88c00640 1 -- 172.21.15.114:0/3425674071 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7f0b7c0d2450 con 0x7f0b8c102870 2024-09-21T07:04:37.444 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:04:37.448 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":49,"btime":"2024-09-21T07:03:23:885749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:03:22.931940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:04:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 49 2024-09-21T07:04:37.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.447+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b700b6800 msgr2=0x7f0b700b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:37.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.447+0000 7f0b91600640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b700b6800 0x7f0b700b8d10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005fd0 tx=0x7f0b78005c70 comp rx=0 tx=0).stop 2024-09-21T07:04:37.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.447+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 msgr2=0x7f0b8c1126c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:37.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.447+0000 7f0b91600640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c1126c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c031440 tx=0x7f0b7c003cd0 comp rx=0 tx=0).stop 2024-09-21T07:04:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 shutdown_connections 2024-09-21T07:04:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b8c108b00 0x7f0b8c113140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0b8c1081b0 0x7f0b8c112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0b700b6800 0x7f0b700b8d10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 --2- 172.21.15.114:0/3425674071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b8c102870 0x7f0b8c1126c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 >> 172.21.15.114:0/3425674071 conn(0x7f0b8c0fe2c0 msgr2=0x7f0b8c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:37.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.448+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 shutdown_connections 2024-09-21T07:04:37.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:37.449+0000 7f0b91600640 1 -- 172.21.15.114:0/3425674071 wait complete. 2024-09-21T07:04:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 600.0 s, 368.8 tps, lat 1159.928 ms stddev 3782.627 2024-09-21T07:04:38.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:38 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3425674071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:38.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:38 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3425674071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:38.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:38 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3425674071' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:04:38.471 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-21T07:04:38.471 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-21T07:04:39.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:39 smithi114 ceph-mon[30654]: pgmap v719: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-21T07:04:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:39 smithi183 ceph-mon[34219]: pgmap v719: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-21T07:04:39.411 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:39.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:39 smithi193 ceph-mon[34601]: pgmap v719: 129 pgs: 129 active+clean; 2.5 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-21T07:04:41.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:41 smithi114 ceph-mon[30654]: pgmap v720: 129 pgs: 129 active+clean; 2.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 3.0 MiB/s wr, 302 op/s 2024-09-21T07:04:41.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:41 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:04:41.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:41 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:04:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:41 smithi183 ceph-mon[34219]: pgmap v720: 129 pgs: 129 active+clean; 2.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 3.0 MiB/s wr, 302 op/s 2024-09-21T07:04:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:41 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:04:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:41 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:04:41.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:41 smithi193 ceph-mon[34601]: pgmap v720: 129 pgs: 129 active+clean; 2.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 3.0 MiB/s wr, 302 op/s 2024-09-21T07:04:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:41 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:04:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:41 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:04:41.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 -- 172.21.15.114:0/683259748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7820105bc0 msgr2=0x7f7820105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:41.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 --2- 172.21.15.114:0/683259748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7820105bc0 0x7f7820105fd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7810009ca0 tx=0x7f781002a8e0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 -- 172.21.15.114:0/683259748 shutdown_connections 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 --2- 172.21.15.114:0/683259748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 0x7f7820112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 --2- 172.21.15.114:0/683259748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f782010b500 0x7f782010b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 --2- 172.21.15.114:0/683259748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7820105bc0 0x7f7820105fd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 -- 172.21.15.114:0/683259748 >> 172.21.15.114:0/683259748 conn(0x7f7820101610 msgr2=0x7f7820103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 -- 172.21.15.114:0/683259748 shutdown_connections 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.607+0000 7f7824a00640 1 -- 172.21.15.114:0/683259748 wait complete. 2024-09-21T07:04:41.610 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.608+0000 7f7824a00640 1 Processor -- start 2024-09-21T07:04:41.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 -- start start 2024-09-21T07:04:41.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820105bc0 0x7f782019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 0x7f782019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 0x7f78201a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.611 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7820115be0 con 0x7f782010be50 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7820115a60 con 0x7f7820105bc0 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.609+0000 7f7824a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7820115d60 con 0x7f782010b500 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f781f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820105bc0 0x7f782019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 0x7f782019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 0x7f782019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52846/0 (socket says 172.21.15.114:52846) 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 -- 172.21.15.114:0/445748945 learned_addr learned my addr 172.21.15.114:0/445748945 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f781fe00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 0x7f78201a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 -- 172.21.15.114:0/445748945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820105bc0 msgr2=0x7f782019c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820105bc0 0x7f782019c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 -- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 msgr2=0x7f78201a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 0x7f78201a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f7816a00640 1 -- 172.21.15.114:0/445748945 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78201a2520 con 0x7f782010b500 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.610+0000 7f781fe00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 0x7f78201a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.611+0000 7f7816a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 0x7f782019cb50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f780800e970 tx=0x7f780800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.611+0000 7f781cc00640 1 -- 172.21.15.114:0/445748945 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780800cd80 con 0x7f782010b500 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.611+0000 7f781cc00640 1 -- 172.21.15.114:0/445748945 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78080035f0 con 0x7f782010b500 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.611+0000 7f781cc00640 1 -- 172.21.15.114:0/445748945 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808010690 con 0x7f782010b500 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.611+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78201a2810 con 0x7f782010b500 2024-09-21T07:04:41.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.611+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201a8f70 con 0x7f782010b500 2024-09-21T07:04:41.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.615+0000 7f782cc46740 1 Processor -- start 2024-09-21T07:04:41.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 -- start start 2024-09-21T07:04:41.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba0d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba0da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c2db982ca0 con 0x55c2db665840 2024-09-21T07:04:41.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2db9720d0 con 0x55c2db644b60 2024-09-21T07:04:41.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f782cc46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c2db975ee0 con 0x55c2db63c980 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f781fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba0d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f781fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba0d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39932/0 (socket says 172.21.15.114:39932) 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f781fe00640 1 -- 172.21.15.114:0/194254855 learned_addr learned my addr 172.21.15.114:0/194254855 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.616+0000 7f781f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba0da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.617+0000 7f781f400640 1 -- 172.21.15.114:0/194254855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 msgr2=0x55c2dba0df90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.617+0000 7f781f400640 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba0df90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.617+0000 7f781f400640 1 -- 172.21.15.114:0/194254855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 msgr2=0x55c2dba0d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.617+0000 7f781f400640 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba0d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.617+0000 7f781f400640 1 -- 172.21.15.114:0/194254855 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2db924400 con 0x55c2db63c980 2024-09-21T07:04:41.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.618+0000 7f781f400640 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba0da50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f781002aed0 tx=0x7f7810009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf158c5a8346f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:41.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.618+0000 7f781cc00640 1 -- 172.21.15.114:0/445748945 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7808003aa0 con 0x7f782010b500 2024-09-21T07:04:41.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.618+0000 7f780fe00640 1 -- 172.21.15.114:0/194254855 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810031ad0 con 0x55c2db63c980 2024-09-21T07:04:41.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.618+0000 7f780fe00640 1 -- 172.21.15.114:0/194254855 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7810036040 con 0x55c2db63c980 2024-09-21T07:04:41.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.618+0000 7f780fe00640 1 -- 172.21.15.114:0/194254855 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100046b0 con 0x55c2db63c980 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 -- 172.21.15.114:0/194254855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 msgr2=0x55c2dba0da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba0da50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f781002aed0 tx=0x7f7810009a20 comp rx=0 tx=0).stop 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 -- 172.21.15.114:0/194254855 shutdown_connections 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba0df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba0da50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 --2- 172.21.15.114:0/194254855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba0d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 -- 172.21.15.114:0/194254855 >> 172.21.15.114:0/194254855 conn(0x55c2db659990 msgr2=0x55c2dba0b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.619+0000 7f782cc46740 1 -- 172.21.15.114:0/194254855 shutdown_connections 2024-09-21T07:04:41.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.620+0000 7f782cc46740 1 -- 172.21.15.114:0/194254855 wait complete. 2024-09-21T07:04:41.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.620+0000 7f781cc00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78040b6600 0x7f78040b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.620+0000 7f782cc46740 1 -- start start 2024-09-21T07:04:41.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.620+0000 7f781cc00640 1 -- 172.21.15.114:0/445748945 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f78080e7db0 con 0x7f782010b500 2024-09-21T07:04:41.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.620+0000 7f781fe00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78040b6600 0x7f78040b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 Processor -- start 2024-09-21T07:04:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db63c980 0x55c2dba18c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 0x55c2dba24fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c2db7d37a0 con 0x55c2db63c980 2024-09-21T07:04:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2db917a10 con 0x55c2db665840 2024-09-21T07:04:41.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c2dba1bc90 con 0x55c2db644b60 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f781fe00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78040b6600 0x7f78040b8b10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7818006eb0 tx=0x7f7818008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.621+0000 7f782cc46740 10 client.0 _ll_register_callbacks cb 0x55c2db927be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 0x55c2dba24fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db63c980 0x55c2dba18c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 0x55c2dba24fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60882/0 (socket says 172.21.15.114:60882) 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 -- 172.21.15.114:0/1646244920 learned_addr learned my addr 172.21.15.114:0/1646244920 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:41.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f7816a00640 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 -- 172.21.15.114:0/1646244920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 msgr2=0x55c2dba191c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba191c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 -- 172.21.15.114:0/1646244920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db63c980 msgr2=0x55c2dba18c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db63c980 0x55c2dba18c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.622+0000 7f781fe00640 1 -- 172.21.15.114:0/1646244920 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2db6dfb90 con 0x55c2db665840 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.623+0000 7f781fe00640 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 0x55c2dba24fd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f781800b190 tx=0x7f7818004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af9929c222ae479 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.623+0000 7f780cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.623+0000 7f780cc00640 1 -- 172.21.15.114:0/1646244920 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781801c050 con 0x55c2db665840 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.623+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7810036040 con 0x55c2db665840 2024-09-21T07:04:41.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.623+0000 7f780cc00640 1 -- 172.21.15.114:0/1646244920 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7818014bc0 con 0x55c2db665840 2024-09-21T07:04:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.623+0000 7f780cc00640 1 -- 172.21.15.114:0/1646244920 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7818014e00 con 0x55c2db665840 2024-09-21T07:04:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.624+0000 7f782cc46740 10 client.28220 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.624+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c2dba27b60 con 0x55c2db665840 2024-09-21T07:04:41.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.624+0000 7f780cc00640 1 -- 172.21.15.114:0/1646244920 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7810036040 con 0x55c2db665840 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f780cc00640 1 -- 172.21.15.114:0/1646244920 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f78180065b0 con 0x55c2db665840 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f782cc46740 10 client.28220 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f782cc46740 10 client.28220 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f782cc46740 20 client.28220 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f782cc46740 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db6241c0 0x55c2dba2b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f782cc46740 4 client.28220 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:41.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.625+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2db984bf0 con 0x55c2db6241c0 2024-09-21T07:04:41.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.626+0000 7f781f400640 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db6241c0 0x55c2dba2b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.626+0000 7f780cc00640 10 client.28220 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:41.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.626+0000 7f781f400640 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db6241c0 0x55c2dba2b050 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a384eaa9c883466a server_cookie=a4df4fbdee5dfd3b in_seq=0 out_seq=0 2024-09-21T07:04:41.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.627+0000 7f780cc00640 1 -- 172.21.15.114:0/1646244920 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2db984bf0 con 0x55c2db6241c0 2024-09-21T07:04:41.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.627+0000 7f780cc00640 10 client.28220 handle_command_reply: tid=0 2024-09-21T07:04:41.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.628+0000 7f782cc46740 1 client.28220 shutdown 2024-09-21T07:04:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.628+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 msgr2=0x55c2dba24fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.628+0000 7f782cc46740 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 0x55c2dba24fd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f781800b190 tx=0x7f7818004700 comp rx=0 tx=0).stop 2024-09-21T07:04:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 shutdown_connections 2024-09-21T07:04:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db665840 0x55c2dba24fd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db6241c0 0x55c2dba2b050 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 --2- 172.21.15.114:0/1646244920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db63c980 0x55c2dba18c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 >> 172.21.15.114:0/1646244920 conn(0x55c2db659990 msgr2=0x55c2dba106f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f780cc00640 0 client.28220 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 shutdown_connections 2024-09-21T07:04:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 1 -- 172.21.15.114:0/1646244920 wait complete. 2024-09-21T07:04:41.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.629+0000 7f782cc46740 20 client.28220 trim_cache size 0 max 16384 2024-09-21T07:04:41.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.656+0000 7f782cc46740 1 Processor -- start 2024-09-21T07:04:41.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.656+0000 7f782cc46740 1 -- start start 2024-09-21T07:04:41.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f782cc46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db63c980 0x55c2dba1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f782cc46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba0ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f782cc46740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba24ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f782cc46740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f78201b30f0 con 0x55c2db665840 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f782cc46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78201b4c30 con 0x55c2db63c980 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f782cc46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78201b37c0 con 0x55c2db644b60 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f7816a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba0ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f7816a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba0ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52876/0 (socket says 172.21.15.114:52876) 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f7816a00640 1 -- 172.21.15.114:0/1454546637 learned_addr learned my addr 172.21.15.114:0/1454546637 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f781f400640 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db63c980 0x55c2dba1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.657+0000 7f781fe00640 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba24ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f7816a00640 1 -- 172.21.15.114:0/1454546637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db63c980 msgr2=0x55c2dba1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f7816a00640 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db63c980 0x55c2dba1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f7816a00640 1 -- 172.21.15.114:0/1454546637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 msgr2=0x55c2dba24ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f7816a00640 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba24ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f7816a00640 1 -- 172.21.15.114:0/1454546637 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2db6243f0 con 0x55c2db644b60 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f7816a00640 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba0ec50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f78080a9ff0 tx=0x7f78080ad060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15499636cf5b8e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f780f400640 1 -- 172.21.15.114:0/1454546637 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78080b2310 con 0x55c2db644b60 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f780f400640 1 -- 172.21.15.114:0/1454546637 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78080b7250 con 0x55c2db644b60 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.658+0000 7f780f400640 1 -- 172.21.15.114:0/1454546637 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78080a9180 con 0x55c2db644b60 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 -- 172.21.15.114:0/1454546637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 msgr2=0x55c2dba0ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba0ec50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f78080a9ff0 tx=0x7f78080ad060 comp rx=0 tx=0).stop 2024-09-21T07:04:41.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 -- 172.21.15.114:0/1454546637 shutdown_connections 2024-09-21T07:04:41.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba24ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db644b60 0x55c2dba0ec50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 --2- 172.21.15.114:0/1454546637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db63c980 0x55c2dba1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 -- 172.21.15.114:0/1454546637 >> 172.21.15.114:0/1454546637 conn(0x55c2db659990 msgr2=0x55c2dba0e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:41.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 -- 172.21.15.114:0/1454546637 shutdown_connections 2024-09-21T07:04:41.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.660+0000 7f782cc46740 1 -- 172.21.15.114:0/1454546637 wait complete. 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.661+0000 7f782cc46740 1 -- start start 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.661+0000 7f782cc46740 1 Processor -- start 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.661+0000 7f782cc46740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba1ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.661+0000 7f782cc46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba1f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f782cc46740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba1f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f782cc46740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f78201b4560 con 0x55c2db665840 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f782cc46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78201b3e90 con 0x55c2db644b60 2024-09-21T07:04:41.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f782cc46740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f78201b10f0 con 0x55c2db63c980 2024-09-21T07:04:41.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f782cc46740 10 client.0 _ll_register_callbacks cb 0x7f781800b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:41.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f781f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba1ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f781f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba1ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52880/0 (socket says 172.21.15.114:52880) 2024-09-21T07:04:41.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f781f400640 1 -- 172.21.15.114:0/3740989144 learned_addr learned my addr 172.21.15.114:0/3740989144 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:41.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f7816a00640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba1f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.662+0000 7f781fe00640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba1f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.663+0000 7f781f400640 1 -- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 msgr2=0x55c2dba1f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.663+0000 7f781f400640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba1f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.663+0000 7f781f400640 1 -- 172.21.15.114:0/3740989144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 msgr2=0x55c2dba1f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:41.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.663+0000 7f781f400640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba1f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:41.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.663+0000 7f781f400640 1 -- 172.21.15.114:0/3740989144 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2dba94a20 con 0x55c2db63c980 2024-09-21T07:04:41.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.664+0000 7f781f400640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba1ef30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7810031fd0 tx=0x7f7810005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da3be487bb5d32a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:41.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.664+0000 7f780cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:41.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.665+0000 7f780cc00640 1 -- 172.21.15.114:0/3740989144 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78100318d0 con 0x55c2db63c980 2024-09-21T07:04:41.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.665+0000 7f780cc00640 1 -- 172.21.15.114:0/3740989144 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7810031a70 con 0x55c2db63c980 2024-09-21T07:04:41.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.665+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78080b7250 con 0x55c2db63c980 2024-09-21T07:04:41.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.665+0000 7f780cc00640 1 -- 172.21.15.114:0/3740989144 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7810031c70 con 0x55c2db63c980 2024-09-21T07:04:41.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.665+0000 7f782cc46740 10 client.28165 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:41.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c2dba95c80 con 0x55c2db63c980 2024-09-21T07:04:41.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f780cc00640 1 -- 172.21.15.114:0/3740989144 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78080b7250 con 0x55c2db63c980 2024-09-21T07:04:41.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f780cc00640 1 -- 172.21.15.114:0/3740989144 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f781005f480 con 0x55c2db63c980 2024-09-21T07:04:41.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 10 client.28165 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 10 client.28165 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 20 client.28165 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db5f9570 0x55c2dba97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 4 client.28165 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.666+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55c2db9806a0 con 0x55c2db5f9570 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.667+0000 7f7816a00640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db5f9570 0x55c2dba97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.667+0000 7f780cc00640 10 client.28165 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:41.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:41.668+0000 7f7816a00640 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db5f9570 0x55c2dba97f20 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39c76c324003dcaa server_cookie=cb230525a3c50e32 in_seq=0 out_seq=0 2024-09-21T07:04:42.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.010+0000 7f780cc00640 1 -- 172.21.15.114:0/3740989144 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c2db9806a0 con 0x55c2db5f9570 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.010+0000 7f780cc00640 10 client.28165 handle_command_reply: tid=0 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.010+0000 7f782cc46740 1 client.28165 shutdown 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.010+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 msgr2=0x55c2dba1ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.010+0000 7f782cc46740 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba1ef30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7810031fd0 tx=0x7f7810005bf0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 shutdown_connections 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2db665840 0x55c2dba1f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db644b60 0x55c2dba1f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2db5f9570 0x55c2dba97f20 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 --2- 172.21.15.114:0/3740989144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2db63c980 0x55c2dba1ef30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 >> 172.21.15.114:0/3740989144 conn(0x55c2db659990 msgr2=0x55c2dba1a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f780cc00640 0 client.28165 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 shutdown_connections 2024-09-21T07:04:42.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 1 -- 172.21.15.114:0/3740989144 wait complete. 2024-09-21T07:04:42.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.011+0000 7f782cc46740 20 client.28165 trim_cache size 0 max 16384 2024-09-21T07:04:42.014 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:04:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 49, 2024-09-21T07:04:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:04:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 42, 2024-09-21T07:04:42.014 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "a66baf88": { 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 42, 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 144.5, 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:04:42.015 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78040b6600 msgr2=0x7f78040b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78040b6600 0x7f78040b8b10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7818006eb0 tx=0x7f7818008040 comp rx=0 tx=0).stop 2024-09-21T07:04:42.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 msgr2=0x7f782019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 0x7f782019cb50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f780800e970 tx=0x7f780800ee30 comp rx=0 tx=0).stop 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 shutdown_connections 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f782010be50 0x7f78201a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f782010b500 0x7f782019cb50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78040b6600 0x7f78040b8b10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 --2- 172.21.15.114:0/445748945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7820105bc0 0x7f782019c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.012+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 >> 172.21.15.114:0/445748945 conn(0x7f7820101610 msgr2=0x7f7820103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.013+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 shutdown_connections 2024-09-21T07:04:42.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:04:42.013+0000 7f7824a00640 1 -- 172.21.15.114:0/445748945 wait complete. 2024-09-21T07:04:42.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:42 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:04:42.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:42 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:04:42.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:42 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:04:42.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:42 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:04:42.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:42 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:04:42.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:42 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:04:42.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:42 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:04:42.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:42 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:04:42.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:42 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:04:42.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 605.0 s, 405.3 tps, lat 187.700 ms stddev 212.603 2024-09-21T07:04:43.045 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a66baf88', quiesce took 8.5 seconds. Will release after: 16.0 2024-09-21T07:04:43.313 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:43 smithi193 ceph-mon[34601]: pgmap v721: 129 pgs: 129 active+clean; 2.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.5 MiB/s wr, 322 op/s 2024-09-21T07:04:43.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:43 smithi114 ceph-mon[30654]: pgmap v721: 129 pgs: 129 active+clean; 2.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.5 MiB/s wr, 322 op/s 2024-09-21T07:04:43.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:43 smithi183 ceph-mon[34219]: pgmap v721: 129 pgs: 129 active+clean; 2.5 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 3.5 MiB/s wr, 322 op/s 2024-09-21T07:04:45.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:45 smithi114 ceph-mon[30654]: pgmap v722: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 5.5 MiB/s wr, 437 op/s 2024-09-21T07:04:45.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:45 smithi183 ceph-mon[34219]: pgmap v722: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 5.5 MiB/s wr, 437 op/s 2024-09-21T07:04:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.408+0000 7f590e600640 1 -- 172.21.15.114:0/956274144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5908075dd0 msgr2=0x7f59080761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.408+0000 7f590e600640 1 --2- 172.21.15.114:0/956274144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5908075dd0 0x7f59080761e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009900 tx=0x7f58f002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 -- 172.21.15.114:0/956274144 shutdown_connections 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 --2- 172.21.15.114:0/956274144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59081143a0 0x7f5908116870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 --2- 172.21.15.114:0/956274144 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5908075dd0 0x7f59080761e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 --2- 172.21.15.114:0/956274144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 0x7f59080757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 -- 172.21.15.114:0/956274144 >> 172.21.15.114:0/956274144 conn(0x7f59080ff520 msgr2=0x7f5908101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 -- 172.21.15.114:0/956274144 shutdown_connections 2024-09-21T07:04:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.409+0000 7f590e600640 1 -- 172.21.15.114:0/956274144 wait complete. 2024-09-21T07:04:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 Processor -- start 2024-09-21T07:04:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 -- start start 2024-09-21T07:04:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908075dd0 0x7f59081a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 0x7f59081a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 0x7f59081a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f590811a000 con 0x7f59081143a0 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5908119e80 con 0x7f5908075dd0 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.410+0000 7f590e600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f590811a180 con 0x7f5908077390 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 0x7f59081a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 0x7f59081a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 0x7f59081a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52484/0 (socket says 172.21.15.114:52484) 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 0x7f59081a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34168/0 (socket says 172.21.15.114:34168) 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 -- 172.21.15.114:0/370846723 learned_addr learned my addr 172.21.15.114:0/370846723 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:45.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907400640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908075dd0 0x7f59081a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.414 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:45 smithi193 ceph-mon[34601]: pgmap v722: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 5.5 MiB/s wr, 437 op/s 2024-09-21T07:04:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 -- 172.21.15.114:0/370846723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 msgr2=0x7f59081a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 0x7f59081a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 -- 172.21.15.114:0/370846723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908075dd0 msgr2=0x7f59081a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908075dd0 0x7f59081a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 -- 172.21.15.114:0/370846723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081a6930 con 0x7f59081143a0 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.411+0000 7f5907e00640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 0x7f59081a6230 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f590000eb10 tx=0x7f590000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.412+0000 7f5904c00640 1 -- 172.21.15.114:0/370846723 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900004b20 con 0x7f59081143a0 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.412+0000 7f5904c00640 1 -- 172.21.15.114:0/370846723 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5900004cc0 con 0x7f59081143a0 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.412+0000 7f5904c00640 1 -- 172.21.15.114:0/370846723 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900010670 con 0x7f59081143a0 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.412+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59081a6c20 con 0x7f59081143a0 2024-09-21T07:04:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.412+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081ad3b0 con 0x7f59081143a0 2024-09-21T07:04:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.415+0000 7f59157fe740 1 Processor -- start 2024-09-21T07:04:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.415+0000 7f5904c00640 1 -- 172.21.15.114:0/370846723 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5900005020 con 0x7f59081143a0 2024-09-21T07:04:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 -- start start 2024-09-21T07:04:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 0x55c244846c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c244847170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444814d0 0x55c2448476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c2447bc6b0 con 0x55c2444a21b0 2024-09-21T07:04:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2447abae0 con 0x55c2444814d0 2024-09-21T07:04:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f59157fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c2447af8f0 con 0x55c2444792f0 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 0x55c244846c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 0x55c244846c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52500/0 (socket says 172.21.15.114:52500) 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f58ffe00640 1 -- 172.21.15.114:0/2812681817 learned_addr learned my addr 172.21.15.114:0/2812681817 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.416+0000 7f5907400640 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c244847170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444814d0 0x55c2448476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5907400640 1 -- 172.21.15.114:0/2812681817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444814d0 msgr2=0x55c2448476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5907400640 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444814d0 0x55c2448476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5907400640 1 -- 172.21.15.114:0/2812681817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 msgr2=0x55c244846c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5907400640 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 0x55c244846c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5907400640 1 -- 172.21.15.114:0/2812681817 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c244778780 con 0x55c2444792f0 2024-09-21T07:04:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 0x55c244846c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5904c00640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58ec0b6700 0x7f58ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5907400640 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c244847170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009c50 tx=0x7f58f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4d5c377a8f7963 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f5904c00640 1 -- 172.21.15.114:0/370846723 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f59000e80a0 con 0x7f59081143a0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f58fcc00640 1 -- 172.21.15.114:0/2812681817 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8004050 con 0x55c2444792f0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f58ffe00640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58ec0b6700 0x7f58ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f58fcc00640 1 -- 172.21.15.114:0/2812681817 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f80054d0 con 0x55c2444792f0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.417+0000 7f58fcc00640 1 -- 172.21.15.114:0/2812681817 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8004d00 con 0x55c2444792f0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 -- 172.21.15.114:0/2812681817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 msgr2=0x55c244847170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f58ffe00640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58ec0b6700 0x7f58ec0b8c10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f58f0004650 tx=0x7f58f0005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c244847170 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009c50 tx=0x7f58f8011810 comp rx=0 tx=0).stop 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 -- 172.21.15.114:0/2812681817 shutdown_connections 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444814d0 0x55c2448476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c244847170 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 --2- 172.21.15.114:0/2812681817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444a21b0 0x55c244846c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.418+0000 7f59157fe740 1 -- 172.21.15.114:0/2812681817 >> 172.21.15.114:0/2812681817 conn(0x55c244496300 msgr2=0x55c244844b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.419+0000 7f59157fe740 1 -- 172.21.15.114:0/2812681817 shutdown_connections 2024-09-21T07:04:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.419+0000 7f59157fe740 1 -- 172.21.15.114:0/2812681817 wait complete. 2024-09-21T07:04:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.419+0000 7f59157fe740 1 -- start start 2024-09-21T07:04:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.419+0000 7f59157fe740 1 Processor -- start 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 0x55c244852250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244852790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444a21b0 0x55c244852d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c2446e3ab0 con 0x55c2444814d0 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c244752570 con 0x55c2444792f0 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c2448553b0 con 0x55c2444a21b0 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f59157fe740 10 client.0 _ll_register_callbacks cb 0x55c244761100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244852790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444a21b0 0x55c244852d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 0x55c244852250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 0x55c244852250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:51870/0 (socket says 172.21.15.114:51870) 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 -- 172.21.15.114:0/2790827927 learned_addr learned my addr 172.21.15.114:0/2790827927 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444a21b0 0x55c244852d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34184/0 (socket says 172.21.15.114:34184) 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 -- 172.21.15.114:0/2790827927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444a21b0 msgr2=0x55c244852d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444a21b0 0x55c244852d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 -- 172.21.15.114:0/2790827927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 msgr2=0x55c244852790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244852790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f5907400640 1 -- 172.21.15.114:0/2790827927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24450d790 con 0x55c2444792f0 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.420+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244852790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f5907400640 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 0x55c244852250 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f58f80044f0 tx=0x7f58f8004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=745d89548051af88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f58f5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:04:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f58f5600640 1 -- 172.21.15.114:0/2790827927 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8027070 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f80054d0 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f58f5600640 1 -- 172.21.15.114:0/2790827927 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f58f8003a00 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f58f5600640 1 -- 172.21.15.114:0/2790827927 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f58f8033050 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f59157fe740 10 client.28238 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.421+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c244586620 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f58f5600640 1 -- 172.21.15.114:0/2790827927 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f80054d0 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f58f5600640 1 -- 172.21.15.114:0/2790827927 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f58f8005030 con 0x55c2444792f0 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f59157fe740 10 client.28238 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f59157fe740 10 client.28238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f59157fe740 20 client.28238 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f59157fe740 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c244460d80 0x55c244864330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f59157fe740 4 client.28238 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2447be600 con 0x55c244460d80 2024-09-21T07:04:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.422+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c244460d80 0x55c244864330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.423+0000 7f58f5600640 10 client.28238 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.423+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c244460d80 0x55c244864330 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b7993ba27fd1a1 server_cookie=a3d60271df1f7e8d in_seq=0 out_seq=0 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.424+0000 7f58f5600640 1 -- 172.21.15.114:0/2790827927 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2447be600 con 0x55c244460d80 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.424+0000 7f58f5600640 10 client.28238 handle_command_reply: tid=0 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.424+0000 7f59157fe740 1 client.28238 shutdown 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.424+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 msgr2=0x55c244852250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.424+0000 7f59157fe740 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 0x55c244852250 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f58f80044f0 tx=0x7f58f8004550 comp rx=0 tx=0).stop 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.424+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 shutdown_connections 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c244460d80 0x55c244864330 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444a21b0 0x55c244852d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244852790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 --2- 172.21.15.114:0/2790827927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444792f0 0x55c244852250 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f58f5600640 0 client.28238 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 >> 172.21.15.114:0/2790827927 conn(0x55c244496300 msgr2=0x55c244847c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 shutdown_connections 2024-09-21T07:04:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 1 -- 172.21.15.114:0/2790827927 wait complete. 2024-09-21T07:04:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.425+0000 7f59157fe740 20 client.28238 trim_cache size 0 max 16384 2024-09-21T07:04:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 Processor -- start 2024-09-21T07:04:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 -- start start 2024-09-21T07:04:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c2448437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244869990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244869ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b7550 con 0x55c2444814d0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b9090 con 0x55c2444a21b0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f59157fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b7c20 con 0x55c2444792f0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244869990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244869990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52516/0 (socket says 172.21.15.114:52516) 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f58ffe00640 1 -- 172.21.15.114:0/2795550729 learned_addr learned my addr 172.21.15.114:0/2795550729 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.440+0000 7f5907400640 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c2448437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f5907e00640 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244869ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58ffe00640 1 -- 172.21.15.114:0/2795550729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 msgr2=0x55c2448437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c2448437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58ffe00640 1 -- 172.21.15.114:0/2795550729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 msgr2=0x55c244869ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244869ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58ffe00640 1 -- 172.21.15.114:0/2795550729 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c244848f30 con 0x55c2444814d0 2024-09-21T07:04:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f5907e00640 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244869ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244869990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f58f002a9e0 tx=0x7f58f003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68280b1d9945021c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58f7e00640 1 -- 172.21.15.114:0/2795550729 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0035040 con 0x55c2444814d0 2024-09-21T07:04:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58f7e00640 1 -- 172.21.15.114:0/2795550729 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f0004dc0 con 0x55c2444814d0 2024-09-21T07:04:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.441+0000 7f58f7e00640 1 -- 172.21.15.114:0/2795550729 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f003b5d0 con 0x55c2444814d0 2024-09-21T07:04:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 -- 172.21.15.114:0/2795550729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 msgr2=0x55c244869990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244869990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f58f002a9e0 tx=0x7f58f003b000 comp rx=0 tx=0).stop 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 -- 172.21.15.114:0/2795550729 shutdown_connections 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244869ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444814d0 0x55c244869990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 --2- 172.21.15.114:0/2795550729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444792f0 0x55c2448437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 -- 172.21.15.114:0/2795550729 >> 172.21.15.114:0/2795550729 conn(0x55c244496300 msgr2=0x55c2448430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 -- 172.21.15.114:0/2795550729 shutdown_connections 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.442+0000 7f59157fe740 1 -- 172.21.15.114:0/2795550729 wait complete. 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 -- start start 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 Processor -- start 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 0x55c24485b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444814d0 0x55c2448585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244858cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b89c0 con 0x55c2444792f0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b82f0 con 0x55c2444a21b0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f59081b5550 con 0x55c2444814d0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f59157fe740 10 client.0 _ll_register_callbacks cb 0x7f58f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f5907400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 0x55c24485b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f5907400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 0x55c24485b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52526/0 (socket says 172.21.15.114:52526) 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.443+0000 7f5907400640 1 -- 172.21.15.114:0/2119451416 learned_addr learned my addr 172.21.15.114:0/2119451416 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444814d0 0x55c2448585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907e00640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244858cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907400640 1 -- 172.21.15.114:0/2119451416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444814d0 msgr2=0x55c2448585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907400640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444814d0 0x55c2448585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907400640 1 -- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 msgr2=0x55c244858cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907400640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244858cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907400640 1 -- 172.21.15.114:0/2119451416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2448cdba0 con 0x55c2444792f0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907e00640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244858cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f5907400640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 0x55c24485b050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f58f8023460 tx=0x7f58f8009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8752d79182441ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f58f5600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f58f5600640 1 -- 172.21.15.114:0/2119451416 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8027070 con 0x55c2444792f0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f58f5600640 1 -- 172.21.15.114:0/2119451416 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f58f80045c0 con 0x55c2444792f0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.444+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f0004dc0 con 0x55c2444792f0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f58f5600640 1 -- 172.21.15.114:0/2119451416 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f58f8044050 con 0x55c2444792f0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 10 client.17763 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c2448ced30 con 0x55c2444792f0 2024-09-21T07:04:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f58f5600640 1 -- 172.21.15.114:0/2119451416 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f0004dc0 con 0x55c2444792f0 2024-09-21T07:04:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f58f5600640 1 -- 172.21.15.114:0/2119451416 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f58f803f030 con 0x55c2444792f0 2024-09-21T07:04:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 10 client.17763 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 20 client.17763 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2444361c0 0x55c2448d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 4 client.17763 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:04:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.445+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2447ba0b0 con 0x55c2444361c0 2024-09-21T07:04:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.446+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2444361c0 0x55c2448d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.446+0000 7f58ffe00640 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2444361c0 0x55c2448d1480 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7470fbfb7b62b2 server_cookie=74361245c2fcd9af in_seq=0 out_seq=0 2024-09-21T07:04:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.446+0000 7f58f5600640 10 client.17763 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f58f5600640 1 -- 172.21.15.114:0/2119451416 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2447ba0b0 con 0x55c2444361c0 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f58f5600640 10 client.17763 handle_command_reply: tid=0 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 client.17763 shutdown 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 msgr2=0x55c24485b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 0x55c24485b050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f58f8023460 tx=0x7f58f8009af0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 shutdown_connections 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2444a21b0 0x55c244858cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c2444361c0 0x55c2448d1480 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c2444814d0 0x55c2448585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 --2- 172.21.15.114:0/2119451416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c2444792f0 0x55c24485b050 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f58f5600640 0 client.17763 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 >> 172.21.15.114:0/2119451416 conn(0x55c244496300 msgr2=0x55c244847c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 shutdown_connections 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 1 -- 172.21.15.114:0/2119451416 wait complete. 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.455+0000 7f59157fe740 20 client.17763 trim_cache size 0 max 16384 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:04:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58ec0b6700 msgr2=0x7f58ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58ec0b6700 0x7f58ec0b8c10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f58f0004650 tx=0x7f58f0005fd0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 msgr2=0x7f59081a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 0x7f59081a6230 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f590000eb10 tx=0x7f590000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 shutdown_connections 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f59081143a0 0x7f59081a6230 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5908077390 0x7f59081a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f58ec0b6700 0x7f58ec0b8c10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 --2- 172.21.15.114:0/370846723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5908075dd0 0x7f59081a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 >> 172.21.15.114:0/370846723 conn(0x7f59080ff520 msgr2=0x7f59081102a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 shutdown_connections 2024-09-21T07:04:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:45.456+0000 7f590e600640 1 -- 172.21.15.114:0/370846723 wait complete. 2024-09-21T07:04:47.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:47 smithi114 ceph-mon[30654]: pgmap v723: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.7 MiB/s wr, 397 op/s 2024-09-21T07:04:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:47 smithi183 ceph-mon[34219]: pgmap v723: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.7 MiB/s wr, 397 op/s 2024-09-21T07:04:47.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:47 smithi193 ceph-mon[34601]: pgmap v723: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 5.7 MiB/s wr, 397 op/s 2024-09-21T07:04:47.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 610.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:04:49.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:49 smithi114 ceph-mon[30654]: pgmap v724: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.7 MiB/s wr, 394 op/s 2024-09-21T07:04:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:49 smithi183 ceph-mon[34219]: pgmap v724: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.7 MiB/s wr, 394 op/s 2024-09-21T07:04:49.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:49 smithi193 ceph-mon[34601]: pgmap v724: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.7 MiB/s wr, 394 op/s 2024-09-21T07:04:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:51 smithi183 ceph-mon[34219]: pgmap v725: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-21T07:04:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:51 smithi193 ceph-mon[34601]: pgmap v725: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-21T07:04:51.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:51 smithi114 ceph-mon[30654]: pgmap v725: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 4.2 MiB/s wr, 302 op/s 2024-09-21T07:04:52.835 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 615.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:04:53.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:53 smithi114 ceph-mon[30654]: pgmap v726: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-09-21T07:04:53.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:53 smithi183 ceph-mon[34219]: pgmap v726: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-09-21T07:04:53.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:53 smithi193 ceph-mon[34601]: pgmap v726: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-09-21T07:04:55.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:55 smithi183 ceph-mon[34219]: pgmap v727: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-21T07:04:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:55 smithi193 ceph-mon[34601]: pgmap v727: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-21T07:04:55.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:55 smithi114 ceph-mon[30654]: pgmap v727: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-21T07:04:57.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:57 smithi183 ceph-mon[34219]: pgmap v728: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 180 KiB/s wr, 34 op/s 2024-09-21T07:04:57.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:57 smithi193 ceph-mon[34601]: pgmap v728: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 180 KiB/s wr, 34 op/s 2024-09-21T07:04:57.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:57 smithi114 ceph-mon[30654]: pgmap v728: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 180 KiB/s wr, 34 op/s 2024-09-21T07:04:57.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 620.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 -- 172.21.15.114:0/3999894247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 msgr2=0x7fb240112780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 --2- 172.21.15.114:0/3999894247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb240112780 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb238009ca0 tx=0x7fb23802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 -- 172.21.15.114:0/3999894247 shutdown_connections 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 --2- 172.21.15.114:0/3999894247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb240112780 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 --2- 172.21.15.114:0/3999894247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb24010b600 0x7fb24010ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 --2- 172.21.15.114:0/3999894247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb240105cd0 0x7fb2401060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 -- 172.21.15.114:0/3999894247 >> 172.21.15.114:0/3999894247 conn(0x7fb2401016f0 msgr2=0x7fb240103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 -- 172.21.15.114:0/3999894247 shutdown_connections 2024-09-21T07:04:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.653+0000 7fb246e00640 1 -- 172.21.15.114:0/3999894247 wait complete. 2024-09-21T07:04:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 Processor -- start 2024-09-21T07:04:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 -- start start 2024-09-21T07:04:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb240105cd0 0x7fb24019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb24010b600 0x7fb24019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb2401a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb240115c10 con 0x7fb24010bf50 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb240115a90 con 0x7fb240105cd0 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.654+0000 7fb246e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb240115d90 con 0x7fb24010b600 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb2401a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb2401a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35032/0 (socket says 172.21.15.114:35032) 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 -- 172.21.15.114:0/4163619195 learned_addr learned my addr 172.21.15.114:0/4163619195 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb237400640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb24010b600 0x7fb24019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb23fe00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb240105cd0 0x7fb24019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 -- 172.21.15.114:0/4163619195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb24010b600 msgr2=0x7fb24019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb24010b600 0x7fb24019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 -- 172.21.15.114:0/4163619195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb240105cd0 msgr2=0x7fb24019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb240105cd0 0x7fb24019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 -- 172.21.15.114:0/4163619195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2401a2580 con 0x7fb24010bf50 2024-09-21T07:04:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb23fe00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb240105cd0 0x7fb24019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:04:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.655+0000 7fb244e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb2401a1e80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2380098d0 tx=0x7fb2380317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.656+0000 7fb23d600640 1 -- 172.21.15.114:0/4163619195 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238003b70 con 0x7fb24010bf50 2024-09-21T07:04:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.656+0000 7fb23d600640 1 -- 172.21.15.114:0/4163619195 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb238003d10 con 0x7fb24010bf50 2024-09-21T07:04:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.656+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2401a2810 con 0x7fb24010bf50 2024-09-21T07:04:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.656+0000 7fb23d600640 1 -- 172.21.15.114:0/4163619195 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23802fa20 con 0x7fb24010bf50 2024-09-21T07:04:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.656+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2401a9000 con 0x7fb24010bf50 2024-09-21T07:04:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.659+0000 7fb24de54740 1 Processor -- start 2024-09-21T07:04:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.659+0000 7fb23d600640 1 -- 172.21.15.114:0/4163619195 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb23802fc40 con 0x7fb24010bf50 2024-09-21T07:04:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.659+0000 7fb24de54740 1 -- start start 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb24de54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb646c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb24de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb647170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb24de54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb6476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb24de54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d8cb5bc6b0 con 0x55d8cb2814d0 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb24de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8cb5abae0 con 0x55d8cb2792f0 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb24de54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8cb5af8f0 con 0x55d8cb2a21b0 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb23fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb647170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb23fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb647170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41370/0 (socket says 172.21.15.114:41370) 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb23fe00640 1 -- 172.21.15.114:0/1627334463 learned_addr learned my addr 172.21.15.114:0/1627334463 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb237400640 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb646c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.660+0000 7fb237400640 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb6476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23fe00640 1 -- 172.21.15.114:0/1627334463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 msgr2=0x55d8cb646c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23fe00640 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb646c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23fe00640 1 -- 172.21.15.114:0/1627334463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 msgr2=0x55d8cb6476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23fe00640 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb6476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23fe00640 1 -- 172.21.15.114:0/1627334463 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cb578780 con 0x55d8cb2792f0 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23fe00640 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb647170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb230009d70 tx=0x7fb2300117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a9fd3309b6afd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23d600640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2240b6680 0x7fb2240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb235600640 1 -- 172.21.15.114:0/1627334463 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230009e60 con 0x55d8cb2792f0 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb235600640 1 -- 172.21.15.114:0/1627334463 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230016070 con 0x55d8cb2792f0 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb23d600640 1 -- 172.21.15.114:0/4163619195 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fb238108dd0 con 0x7fb24010bf50 2024-09-21T07:04:58.665 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb235600640 1 -- 172.21.15.114:0/1627334463 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230012a00 con 0x55d8cb2792f0 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.661+0000 7fb237400640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2240b6680 0x7fb2240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 -- 172.21.15.114:0/1627334463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 msgr2=0x55d8cb647170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb237400640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2240b6680 0x7fb2240b8b90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb22800bfd0 tx=0x7fb228004730 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb647170 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb230009d70 tx=0x7fb2300117b0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 -- 172.21.15.114:0/1627334463 shutdown_connections 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb6476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb647170 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 --2- 172.21.15.114:0/1627334463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb646c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 -- 172.21.15.114:0/1627334463 >> 172.21.15.114:0/1627334463 conn(0x55d8cb296300 msgr2=0x55d8cb644b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 -- 172.21.15.114:0/1627334463 shutdown_connections 2024-09-21T07:04:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.662+0000 7fb24de54740 1 -- 172.21.15.114:0/1627334463 wait complete. 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.663+0000 7fb24de54740 1 -- start start 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.663+0000 7fb24de54740 1 Processor -- start 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb6521d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb652710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb652c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d8cb4e3ab0 con 0x55d8cb2814d0 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8cb552570 con 0x55d8cb2792f0 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d8cb6553b0 con 0x55d8cb2a21b0 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb24de54740 10 client.0 _ll_register_callbacks cb 0x55d8cb561100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb652710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb652710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35050/0 (socket says 172.21.15.114:35050) 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 -- 172.21.15.114:0/1006644240 learned_addr learned my addr 172.21.15.114:0/1006644240 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb23fe00640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb6521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb244e00640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb652c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 -- 172.21.15.114:0/1006644240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 msgr2=0x55d8cb652c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb652c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 -- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 msgr2=0x55d8cb6521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb6521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.664+0000 7fb237400640 1 -- 172.21.15.114:0/1006644240 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cb30d790 con 0x55d8cb2814d0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb237400640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb652710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b9d0 tx=0x7fb2280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a84430bf88c3af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb22e000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb22e000640 1 -- 172.21.15.114:0/1006644240 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22802e060 con 0x55d8cb2814d0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb22e000640 1 -- 172.21.15.114:0/1006644240 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fb22801b030 con 0x55d8cb2814d0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb230016070 con 0x55d8cb2814d0 2024-09-21T07:04:58.670 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb22e000640 1 -- 172.21.15.114:0/1006644240 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb228035050 con 0x55d8cb2814d0 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb24de54740 10 client.17775 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.665+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d8cb386620 con 0x55d8cb2814d0 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb22e000640 1 -- 172.21.15.114:0/1006644240 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230016070 con 0x55d8cb2814d0 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb22e000640 1 -- 172.21.15.114:0/1006644240 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fb228029b30 con 0x55d8cb2814d0 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb24de54740 10 client.17775 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb24de54740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb24de54740 20 client.17775 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb24de54740 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb260d80 0x55d8cb664530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb24de54740 4 client.17775 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8cb5be600 con 0x55d8cb260d80 2024-09-21T07:04:58.671 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.666+0000 7fb23fe00640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb260d80 0x55d8cb664530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.667+0000 7fb23fe00640 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb260d80 0x55d8cb664530 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb27c736147d572 server_cookie=2fc10566138f75c4 in_seq=0 out_seq=0 2024-09-21T07:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.667+0000 7fb22e000640 10 client.17775 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:58.672 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb22e000640 1 -- 172.21.15.114:0/1006644240 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8cb5be600 con 0x55d8cb260d80 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb22e000640 10 client.17775 handle_command_reply: tid=0 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 client.17775 shutdown 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 msgr2=0x55d8cb652710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb652710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb22800b9d0 tx=0x7fb2280046c0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 shutdown_connections 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb260d80 0x55d8cb664530 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb652c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2814d0 0x55d8cb652710 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 --2- 172.21.15.114:0/1006644240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2792f0 0x55d8cb6521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb22e000640 0 client.17775 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 >> 172.21.15.114:0/1006644240 conn(0x55d8cb296300 msgr2=0x55d8cb647c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 shutdown_connections 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 1 -- 172.21.15.114:0/1006644240 wait complete. 2024-09-21T07:04:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.668+0000 7fb24de54740 20 client.17775 trim_cache size 0 max 16384 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.683+0000 7fb24de54740 1 Processor -- start 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.683+0000 7fb24de54740 1 -- start start 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb24de54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb643aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb24de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb653e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb24de54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb64f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb24de54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2401b3020 con 0x55d8cb2792f0 2024-09-21T07:04:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb24de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2401b4b60 con 0x55d8cb2814d0 2024-09-21T07:04:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb24de54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2401b36f0 con 0x55d8cb2a21b0 2024-09-21T07:04:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb643aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb643aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35060/0 (socket says 172.21.15.114:35060) 2024-09-21T07:04:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 -- 172.21.15.114:0/2883923069 learned_addr learned my addr 172.21.15.114:0/2883923069 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb244e00640 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb64f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb237400640 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb653e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 -- 172.21.15.114:0/2883923069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 msgr2=0x55d8cb64f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb64f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 -- 172.21.15.114:0/2883923069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 msgr2=0x55d8cb653e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb653e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 -- 172.21.15.114:0/2883923069 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cb64dd00 con 0x55d8cb2792f0 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.684+0000 7fb23fe00640 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb643aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb230009c60 tx=0x7fb230012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f71bb5749bf564b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.685+0000 7fb234c00640 1 -- 172.21.15.114:0/2883923069 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230005240 con 0x55d8cb2792f0 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.685+0000 7fb234c00640 1 -- 172.21.15.114:0/2883923069 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230016050 con 0x55d8cb2792f0 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.685+0000 7fb234c00640 1 -- 172.21.15.114:0/2883923069 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230004660 con 0x55d8cb2792f0 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.685+0000 7fb24de54740 1 -- 172.21.15.114:0/2883923069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 msgr2=0x55d8cb643aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.685+0000 7fb24de54740 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb643aa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb230009c60 tx=0x7fb230012700 comp rx=0 tx=0).stop 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 -- 172.21.15.114:0/2883923069 shutdown_connections 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb64f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb653e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 --2- 172.21.15.114:0/2883923069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb643aa0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 -- 172.21.15.114:0/2883923069 >> 172.21.15.114:0/2883923069 conn(0x55d8cb296300 msgr2=0x55d8cb6536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 -- 172.21.15.114:0/2883923069 shutdown_connections 2024-09-21T07:04:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 -- 172.21.15.114:0/2883923069 wait complete. 2024-09-21T07:04:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 -- start start 2024-09-21T07:04:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.686+0000 7fb24de54740 1 Processor -- start 2024-09-21T07:04:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb658850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb658d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb6592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.692 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2401b4490 con 0x55d8cb2792f0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2401b3dc0 con 0x55d8cb2814d0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2401b1020 con 0x55d8cb2a21b0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb24de54740 10 client.0 _ll_register_callbacks cb 0x7fb22800b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb23fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb658850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb237400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb658d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb6592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb237400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb658d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41396/0 (socket says 172.21.15.114:41396) 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb6592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42222/0 (socket says 172.21.15.114:42222) 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb237400640 1 -- 172.21.15.114:0/3158128282 learned_addr learned my addr 172.21.15.114:0/3158128282 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 -- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 msgr2=0x55d8cb658d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb658d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 -- 172.21.15.114:0/3158128282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 msgr2=0x55d8cb658850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:04:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb658850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb244e00640 1 -- 172.21.15.114:0/3158128282 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cb6ce100 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb23fe00640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb658850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.687+0000 7fb237400640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb658d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb244e00640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb6592d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb238106150 tx=0x7fb238009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd1ae1a9eb7ab4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb22e000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb22e000640 1 -- 172.21.15.114:0/3158128282 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2380ce1a0 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb22e000640 1 -- 172.21.15.114:0/3158128282 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fb2380d7fe0 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb230016050 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb22e000640 1 -- 172.21.15.114:0/3158128282 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb2380cdfc0 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb24de54740 10 client.28201 fetch_fsmap learned FSMap version 49 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d8cb6cf420 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.688+0000 7fb22e000640 1 -- 172.21.15.114:0/3158128282 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb230016050 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb22e000640 1 -- 172.21.15.114:0/3158128282 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fb2380cfb80 con 0x55d8cb2a21b0 2024-09-21T07:04:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb24de54740 10 client.28201 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb24de54740 10 client.28201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb24de54740 20 client.28201 populate_metadata read hostname 'smithi114' 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb24de54740 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb2361c0 0x55d8cb6d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb24de54740 4 client.28201 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8cb5ba0b0 con 0x55d8cb2361c0 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb23fe00640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb2361c0 0x55d8cb6d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb22e000640 10 client.28201 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.689+0000 7fb23fe00640 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb2361c0 0x55d8cb6d1b90 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad63925a12af1b0f server_cookie=caf512c81377267c in_seq=0 out_seq=0 2024-09-21T07:04:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb22e000640 1 -- 172.21.15.114:0/3158128282 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8cb5ba0b0 con 0x55d8cb2361c0 2024-09-21T07:04:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb22e000640 10 client.28201 handle_command_reply: tid=0 2024-09-21T07:04:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 client.28201 shutdown 2024-09-21T07:04:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 msgr2=0x55d8cb6592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb6592d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb238106150 tx=0x7fb238009970 comp rx=0 tx=0).stop 2024-09-21T07:04:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 shutdown_connections 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d8cb2361c0 0x55d8cb6d1b90 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d8cb2a21b0 0x55d8cb6592d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cb2814d0 0x55d8cb658d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb22e000640 0 client.28201 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 --2- 172.21.15.114:0/3158128282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d8cb2792f0 0x55d8cb658850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 >> 172.21.15.114:0/3158128282 conn(0x55d8cb296300 msgr2=0x55d8cb653680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 shutdown_connections 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.698+0000 7fb24de54740 1 -- 172.21.15.114:0/3158128282 wait complete. 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb24de54740 20 client.28201 trim_cache size 0 max 16384 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:04:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:04:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:04:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:04:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2240b6680 msgr2=0x7fb2240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2240b6680 0x7fb2240b8b90 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb22800bfd0 tx=0x7fb228004730 comp rx=0 tx=0).stop 2024-09-21T07:04:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 msgr2=0x7fb2401a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:04:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb2401a1e80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb2380098d0 tx=0x7fb2380317b0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 shutdown_connections 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb24010bf50 0x7fb2401a1e80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb24010b600 0x7fb24019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2240b6680 0x7fb2240b8b90 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 --2- 172.21.15.114:0/4163619195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb240105cd0 0x7fb24019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.699+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 >> 172.21.15.114:0/4163619195 conn(0x7fb2401016f0 msgr2=0x7fb240103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.700+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 shutdown_connections 2024-09-21T07:04:58.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:04:58.700+0000 7fb246e00640 1 -- 172.21.15.114:0/4163619195 wait complete. 2024-09-21T07:04:59.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:04:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:04:59 smithi183 ceph-mon[34219]: pgmap v729: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-21T07:04:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:04:59 smithi193 ceph-mon[34601]: pgmap v729: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-21T07:04:59.436 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:04:59.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:04:59 smithi114 ceph-mon[30654]: pgmap v729: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-09-21T07:05:00.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 -- 172.21.15.114:0/2583407967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941039b0 msgr2=0x7f9f94103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:00.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 --2- 172.21.15.114:0/2583407967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941039b0 0x7f9f94103dc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009860 tx=0x7f9f8402a340 comp rx=0 tx=0).stop 2024-09-21T07:05:00.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 -- 172.21.15.114:0/2583407967 shutdown_connections 2024-09-21T07:05:00.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 --2- 172.21.15.114:0/2583407967 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 0x7f9f94110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 --2- 172.21.15.114:0/2583407967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941092f0 0x7f9f94109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 --2- 172.21.15.114:0/2583407967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941039b0 0x7f9f94103dc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 -- 172.21.15.114:0/2583407967 >> 172.21.15.114:0/2583407967 conn(0x7f9f940ff440 msgr2=0x7f9f94101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:00.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 -- 172.21.15.114:0/2583407967 shutdown_connections 2024-09-21T07:05:00.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.138+0000 7f9f93200640 1 -- 172.21.15.114:0/2583407967 wait complete. 2024-09-21T07:05:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.139+0000 7f9f93200640 1 Processor -- start 2024-09-21T07:05:00.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.139+0000 7f9f93200640 1 -- start start 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f93200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 0x7f9f9419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f93200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941092f0 0x7f9f9419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f93200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 0x7f9f9419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f93200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94113900 con 0x7f9f94109c40 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f93200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94113780 con 0x7f9f941039b0 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f93200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94113a80 con 0x7f9f941092f0 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f91e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 0x7f9f9419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f91e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 0x7f9f9419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41412/0 (socket says 172.21.15.114:41412) 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f91e00640 1 -- 172.21.15.114:0/2574804765 learned_addr learned my addr 172.21.15.114:0/2574804765 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.140+0000 7f9f92800640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 0x7f9f9419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f91400640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941092f0 0x7f9f9419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f92800640 1 -- 172.21.15.114:0/2574804765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941092f0 msgr2=0x7f9f9419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f92800640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941092f0 0x7f9f9419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.144 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f92800640 1 -- 172.21.15.114:0/2574804765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 msgr2=0x7f9f9419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f92800640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 0x7f9f9419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f92800640 1 -- 172.21.15.114:0/2574804765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f941a0280 con 0x7f9f94109c40 2024-09-21T07:05:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f91e00640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 0x7f9f9419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f91400640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941092f0 0x7f9f9419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:00.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f92800640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 0x7f9f9419fb50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00ef00 tx=0x7f9f8c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c004b20 con 0x7f9f94109c40 2024-09-21T07:05:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f941a0570 con 0x7f9f94109c40 2024-09-21T07:05:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c003e20 con 0x7f9f94109c40 2024-09-21T07:05:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.141+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c00b570 con 0x7f9f94109c40 2024-09-21T07:05:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.142+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f941a6cd0 con 0x7f9f94109c40 2024-09-21T07:05:00.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.143+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f94103dc0 con 0x7f9f94109c40 2024-09-21T07:05:00.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.145+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9f8c01d020 con 0x7f9f94109c40 2024-09-21T07:05:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.147+0000 7f9f8aa00640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9f780b6680 0x7f9f780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.147+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f9f8c0e8490 con 0x7f9f94109c40 2024-09-21T07:05:00.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.147+0000 7f9f91e00640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9f780b6680 0x7f9f780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:00.150 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.148+0000 7f9f91e00640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9f780b6680 0x7f9f780b8b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9f840095f0 tx=0x7f9f84030340 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:00.152 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.149+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9f8c024030 con 0x7f9f94109c40 2024-09-21T07:05:00.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.396+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f9410ba10 con 0x7f9f94109c40 2024-09-21T07:05:00.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.397+0000 7f9f8aa00640 1 -- 172.21.15.114:0/2574804765 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7f9f8c004cc0 con 0x7f9f94109c40 2024-09-21T07:05:00.400 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:05:00.401 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":49,"btime":"2024-09-21T07:03:23:885749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:03:22.931940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:05:00.401 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 49 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.400+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9f780b6680 msgr2=0x7f9f780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.400+0000 7f9f93200640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9f780b6680 0x7f9f780b8b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9f840095f0 tx=0x7f9f84030340 comp rx=0 tx=0).stop 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.400+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 msgr2=0x7f9f9419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.400+0000 7f9f93200640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 0x7f9f9419fb50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00ef00 tx=0x7f9f8c00c740 comp rx=0 tx=0).stop 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 shutdown_connections 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9f94109c40 0x7f9f9419fb50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f941092f0 0x7f9f9419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9f780b6680 0x7f9f780b8b90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 --2- 172.21.15.114:0/2574804765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f941039b0 0x7f9f9419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 >> 172.21.15.114:0/2574804765 conn(0x7f9f940ff440 msgr2=0x7f9f94101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 shutdown_connections 2024-09-21T07:05:00.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:00.401+0000 7f9f93200640 1 -- 172.21.15.114:0/2574804765 wait complete. 2024-09-21T07:05:01.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:01 smithi114 ceph-mon[30654]: pgmap v730: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T07:05:01.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:01 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2574804765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:05:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:01 smithi183 ceph-mon[34219]: pgmap v730: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T07:05:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:01 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2574804765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:05:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:01 smithi193 ceph-mon[34601]: pgmap v730: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-09-21T07:05:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:01 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2574804765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:05:01.513 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id a66baf88 --release --await' 2024-09-21T07:05:01.513 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id a66baf88 --release --await 2024-09-21T07:05:01.888 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:02.267 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:05:02.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 -- 172.21.15.114:0/808730576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c81047c0 msgr2=0x7f07c8104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 --2- 172.21.15.114:0/808730576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c81047c0 0x7f07c8104bd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009900 tx=0x7f07b802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 -- 172.21.15.114:0/808730576 shutdown_connections 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 --2- 172.21.15.114:0/808730576 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07c810dc20 0x7f07c8110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 --2- 172.21.15.114:0/808730576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 0x7f07c810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 --2- 172.21.15.114:0/808730576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c81047c0 0x7f07c8104bd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 -- 172.21.15.114:0/808730576 >> 172.21.15.114:0/808730576 conn(0x7f07c80fe2c0 msgr2=0x7f07c8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 -- 172.21.15.114:0/808730576 shutdown_connections 2024-09-21T07:05:02.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.581+0000 7f07c7200640 1 -- 172.21.15.114:0/808730576 wait complete. 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.582+0000 7f07c7200640 1 Processor -- start 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.582+0000 7f07c7200640 1 -- start start 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07c81047c0 0x7f07c819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 0x7f07c819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c810dc20 0x7f07c819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81138a0 con 0x7f07c81047c0 2024-09-21T07:05:02.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8113720 con 0x7f07c8105110 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c8113a20 con 0x7f07c810dc20 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c810dc20 0x7f07c819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 0x7f07c819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c810dc20 0x7f07c819fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42260/0 (socket says 172.21.15.114:42260) 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c6800640 1 -- 172.21.15.114:0/4158052677 learned_addr learned my addr 172.21.15.114:0/4158052677 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c5e00640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07c81047c0 0x7f07c819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c5400640 1 -- 172.21.15.114:0/4158052677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c810dc20 msgr2=0x7f07c819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.583+0000 7f07c5400640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c810dc20 0x7f07c819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07c5400640 1 -- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07c81047c0 msgr2=0x7f07c819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07c5400640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07c81047c0 0x7f07c819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07c5400640 1 -- 172.21.15.114:0/4158052677 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c81a0260 con 0x7f07c8105110 2024-09-21T07:05:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07c5400640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 0x7f07c819a890 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f07c806aa90 tx=0x7f07b400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:02.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07bea00640 1 -- 172.21.15.114:0/4158052677 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b400cc40 con 0x7f07c8105110 2024-09-21T07:05:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07bea00640 1 -- 172.21.15.114:0/4158052677 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b400cde0 con 0x7f07c8105110 2024-09-21T07:05:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c81a0550 con 0x7f07c8105110 2024-09-21T07:05:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07bea00640 1 -- 172.21.15.114:0/4158052677 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4004a00 con 0x7f07c8105110 2024-09-21T07:05:02.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.584+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81a6cb0 con 0x7f07c8105110 2024-09-21T07:05:02.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.588+0000 7f07d3c7e740 1 Processor -- start 2024-09-21T07:05:02.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.588+0000 7f07d3c7e740 1 -- start start 2024-09-21T07:05:02.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b549340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b549880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b549dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564a8b4c0290 con 0x564a8b179660 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a8b4af6c0 con 0x564a8b181840 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a8b4b34d0 con 0x564a8b1a2520 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07c6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b549340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07c5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b549880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07c6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b549340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:42262/0 (socket says 172.21.15.114:42262) 2024-09-21T07:05:02.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07c6800640 1 -- 172.21.15.114:0/479038606 learned_addr learned my addr 172.21.15.114:0/479038606 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:02.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.589+0000 7f07bea00640 1 -- 172.21.15.114:0/4158052677 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f07b4004ba0 con 0x7f07c8105110 2024-09-21T07:05:02.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5e00640 1 -- 172.21.15.114:0/479038606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 msgr2=0x564a8b549340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5e00640 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b549340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5400640 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b549dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5e00640 1 -- 172.21.15.114:0/479038606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 msgr2=0x564a8b549dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5e00640 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b549dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5e00640 1 -- 172.21.15.114:0/479038606 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8b43c460 con 0x564a8b179660 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c6800640 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b549340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5400640 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b549dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.590+0000 7f07c5e00640 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b549880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009230 tx=0x7f07b8032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0d5eab5f4f7c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.591+0000 7f07b3400640 1 -- 172.21.15.114:0/479038606 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8040070 con 0x564a8b179660 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.591+0000 7f07b3400640 1 -- 172.21.15.114:0/479038606 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8004030 con 0x564a8b179660 2024-09-21T07:05:02.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.591+0000 7f07b3400640 1 -- 172.21.15.114:0/479038606 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8035040 con 0x564a8b179660 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07bea00640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07ac0b6700 0x7f07ac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 -- 172.21.15.114:0/479038606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 msgr2=0x564a8b549880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b549880 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009230 tx=0x7f07b8032b90 comp rx=0 tx=0).stop 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07bea00640 1 -- 172.21.15.114:0/4158052677 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f07b40e7ac0 con 0x7f07c8105110 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 -- 172.21.15.114:0/479038606 shutdown_connections 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b549dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b549880 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/479038606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b549340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 -- 172.21.15.114:0/479038606 >> 172.21.15.114:0/479038606 conn(0x564a8b196910 msgr2=0x564a8b547230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07c6800640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07ac0b6700 0x7f07ac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.592+0000 7f07d3c7e740 1 -- 172.21.15.114:0/479038606 shutdown_connections 2024-09-21T07:05:02.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.593+0000 7f07d3c7e740 1 -- 172.21.15.114:0/479038606 wait complete. 2024-09-21T07:05:02.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.593+0000 7f07c6800640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07ac0b6700 0x7f07ac0b8c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f07c0013a80 tx=0x7f07c0015040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:02.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.593+0000 7f07d3c7e740 1 -- start start 2024-09-21T07:05:02.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.593+0000 7f07d3c7e740 1 Processor -- start 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b554ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b555400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b560db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564a8b310bb0 con 0x564a8b179660 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a8b44a6c0 con 0x564a8b181840 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a8b557ac0 con 0x564a8b1a2520 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07d3c7e740 10 client.0 _ll_register_callbacks cb 0x564a8b465490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07c5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b555400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07c5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b555400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41428/0 (socket says 172.21.15.114:41428) 2024-09-21T07:05:02.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07c5400640 1 -- 172.21.15.114:0/1544639748 learned_addr learned my addr 172.21.15.114:0/1544639748 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.594+0000 7f07c5e00640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b554ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c6800640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b560db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5400640 1 -- 172.21.15.114:0/1544639748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 msgr2=0x564a8b560db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5400640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b560db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5400640 1 -- 172.21.15.114:0/1544639748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 msgr2=0x564a8b554ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5400640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b554ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5400640 1 -- 172.21.15.114:0/1544639748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8b20db00 con 0x564a8b181840 2024-09-21T07:05:02.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5e00640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b554ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c6800640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b560db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07c5400640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b555400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f07b40ad1e0 tx=0x7f07b40a9b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f31223b28e1c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.595+0000 7f07a7e00640 1 -- 172.21.15.114:0/1544639748 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40d0e60 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07a7e00640 1 -- 172.21.15.114:0/1544639748 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f07b40ac740 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8004030 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07a7e00640 1 -- 172.21.15.114:0/1544639748 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f07b401c040 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 10 client.28277 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564a8b284960 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07a7e00640 1 -- 172.21.15.114:0/1544639748 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07b8004030 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07a7e00640 1 -- 172.21.15.114:0/1544639748 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f07b40ac1b0 con 0x564a8b181840 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 10 client.28277 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 10 client.28277 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 20 client.28277 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:02.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b160ea0 0x564a8b566d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 4 client.28277 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:02.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.596+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a8b4c21e0 con 0x564a8b160ea0 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.597+0000 7f07c5e00640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b160ea0 0x564a8b566d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.598+0000 7f07c5e00640 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b160ea0 0x564a8b566d60 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6444015fc888019 server_cookie=ba421ddbac6701d7 in_seq=0 out_seq=0 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.598+0000 7f07a7e00640 10 client.28277 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.598+0000 7f07a7e00640 1 -- 172.21.15.114:0/1544639748 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a8b4c21e0 con 0x564a8b160ea0 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.598+0000 7f07a7e00640 10 client.28277 handle_command_reply: tid=0 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.598+0000 7f07d3c7e740 1 client.28277 shutdown 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 msgr2=0x564a8b555400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b555400 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f07b40ad1e0 tx=0x7f07b40a9b30 comp rx=0 tx=0).stop 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 shutdown_connections 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b160ea0 0x564a8b566d60 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b560db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b181840 0x564a8b555400 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/1544639748 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b179660 0x564a8b554ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 >> 172.21.15.114:0/1544639748 conn(0x564a8b196910 msgr2=0x564a8b54c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:02.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07a7e00640 0 client.28277 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:02.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 shutdown_connections 2024-09-21T07:05:02.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 1 -- 172.21.15.114:0/1544639748 wait complete. 2024-09-21T07:05:02.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.599+0000 7f07d3c7e740 20 client.28277 trim_cache size 0 max 16384 2024-09-21T07:05:02.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.617+0000 7f07d3c7e740 1 Processor -- start 2024-09-21T07:05:02.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 -- start start 2024-09-21T07:05:02.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b179660 0x564a8b560cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b5611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 0x564a8b551eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81aeb50 con 0x564a8b181840 2024-09-21T07:05:02.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81b0690 con 0x564a8b1a2520 2024-09-21T07:05:02.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81af220 con 0x564a8b179660 2024-09-21T07:05:02.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07c6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 0x564a8b551eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07c5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b5611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b179660 0x564a8b560cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07c6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 0x564a8b551eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41430/0 (socket says 172.21.15.114:41430) 2024-09-21T07:05:02.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.618+0000 7f07c6800640 1 -- 172.21.15.114:0/86353186 learned_addr learned my addr 172.21.15.114:0/86353186 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c6800640 1 -- 172.21.15.114:0/86353186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b179660 msgr2=0x564a8b560cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c6800640 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b179660 0x564a8b560cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c6800640 1 -- 172.21.15.114:0/86353186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 msgr2=0x564a8b5611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c6800640 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b5611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c6800640 1 -- 172.21.15.114:0/86353186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8b553c20 con 0x564a8b1a2520 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c5e00640 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b179660 0x564a8b560cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07c6800640 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 0x564a8b551eb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f07c000ee00 tx=0x7f07c001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33e36e5e616f9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.619+0000 7f07b2a00640 1 -- 172.21.15.114:0/86353186 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0025050 con 0x564a8b1a2520 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07b2a00640 1 -- 172.21.15.114:0/86353186 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c002f070 con 0x564a8b1a2520 2024-09-21T07:05:02.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07b2a00640 1 -- 172.21.15.114:0/86353186 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c001d3c0 con 0x564a8b1a2520 2024-09-21T07:05:02.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 -- 172.21.15.114:0/86353186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 msgr2=0x564a8b551eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 0x564a8b551eb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f07c000ee00 tx=0x7f07c001b000 comp rx=0 tx=0).stop 2024-09-21T07:05:02.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 -- 172.21.15.114:0/86353186 shutdown_connections 2024-09-21T07:05:02.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b1a2520 0x564a8b551eb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b5611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.624 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/86353186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b179660 0x564a8b560cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.620+0000 7f07d3c7e740 1 -- 172.21.15.114:0/86353186 >> 172.21.15.114:0/86353186 conn(0x564a8b196910 msgr2=0x564a8b54fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:02.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.621+0000 7f07d3c7e740 1 -- 172.21.15.114:0/86353186 shutdown_connections 2024-09-21T07:05:02.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.621+0000 7f07d3c7e740 1 -- 172.21.15.114:0/86353186 wait complete. 2024-09-21T07:05:02.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.621+0000 7f07d3c7e740 1 -- start start 2024-09-21T07:05:02.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.621+0000 7f07d3c7e740 1 Processor -- start 2024-09-21T07:05:02.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b179660 0x564a8b559dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b55a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b55a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81affc0 con 0x564a8b181840 2024-09-21T07:05:02.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81af8f0 con 0x564a8b179660 2024-09-21T07:05:02.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81acb50 con 0x564a8b1a2520 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07d3c7e740 10 client.0 _ll_register_callbacks cb 0x7f07b4004f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b179660 0x564a8b559dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07c5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b55a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07c5400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b55a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:35142/0 (socket says 172.21.15.114:35142) 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07c5400640 1 -- 172.21.15.114:0/2950737914 learned_addr learned my addr 172.21.15.114:0/2950737914 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.622+0000 7f07c6800640 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b55a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07c5400640 1 -- 172.21.15.114:0/2950737914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 msgr2=0x564a8b55a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07c5400640 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b55a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07c5400640 1 -- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b179660 msgr2=0x564a8b559dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07c5400640 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b179660 0x564a8b559dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07c5400640 1 -- 172.21.15.114:0/2950737914 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8b5d1800 con 0x564a8b181840 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07c5400640 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b55a310 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f07b40053a0 tx=0x7f07b40ac970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1748042e953262d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:05:02.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.623+0000 7f07a7e00640 1 -- 172.21.15.114:0/2950737914 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b40d0e60 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07a7e00640 1 -- 172.21.15.114:0/2950737914 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f07b40ac9e0 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c002f070 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07a7e00640 1 -- 172.21.15.114:0/2950737914 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f07b40ace30 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07d3c7e740 10 client.17802 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564a8b5d2ac0 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07a7e00640 1 -- 172.21.15.114:0/2950737914 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c002f070 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.624+0000 7f07a7e00640 1 -- 172.21.15.114:0/2950737914 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f07b40cf0d0 con 0x564a8b181840 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07d3c7e740 10 client.17802 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07d3c7e740 10 client.17802 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07d3c7e740 20 client.17802 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b136250 0x564a8b5d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07d3c7e740 4 client.17802 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a66baf88", "release": true, "await": true}] 2024-09-21T07:05:02.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a66baf88", "release": true, "await": true}) -- 0x564a8b4bdc90 con 0x564a8b136250 2024-09-21T07:05:02.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.625+0000 7f07c5e00640 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b136250 0x564a8b5d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:02.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.626+0000 7f07c5e00640 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b136250 0x564a8b5d5250 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a94f9e82af3fcb1 server_cookie=b5faaf4ee22ee1b1 in_seq=0 out_seq=0 2024-09-21T07:05:02.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.626+0000 7f07a7e00640 10 client.17802 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:02.662 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:02.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.663+0000 7f07a7e00640 1 -- 172.21.15.114:0/2950737914 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a8b4bdc90 con 0x564a8b136250 2024-09-21T07:05:02.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.663+0000 7f07a7e00640 10 client.17802 handle_command_reply: tid=0 2024-09-21T07:05:02.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.663+0000 7f07d3c7e740 1 client.17802 shutdown 2024-09-21T07:05:02.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 msgr2=0x564a8b55a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b55a310 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f07b40053a0 tx=0x7f07b40ac970 comp rx=0 tx=0).stop 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 shutdown_connections 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a8b136250 0x564a8b5d5250 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a8b1a2520 0x564a8b55a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a8b181840 0x564a8b55a310 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 --2- 172.21.15.114:0/2950737914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a8b179660 0x564a8b559dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 >> 172.21.15.114:0/2950737914 conn(0x564a8b196910 msgr2=0x564a8b54fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:02.668 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.665+0000 7f07a7e00640 0 client.17802 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:02.669 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.666+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 shutdown_connections 2024-09-21T07:05:02.670 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:05:02.670 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 49, 2024-09-21T07:05:02.670 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:05:02.670 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 44, 2024-09-21T07:05:02.670 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "a66baf88": { 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 44, 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 144.5, 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:05:02.671 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:05:02.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.666+0000 7f07d3c7e740 1 -- 172.21.15.114:0/2950737914 wait complete. 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.666+0000 7f07d3c7e740 20 client.17802 trim_cache size 0 max 16384 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07ac0b6700 msgr2=0x7f07ac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07ac0b6700 0x7f07ac0b8c10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f07c0013a80 tx=0x7f07c0015040 comp rx=0 tx=0).stop 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 msgr2=0x7f07c819a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 0x7f07c819a890 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f07c806aa90 tx=0x7f07b400ee50 comp rx=0 tx=0).stop 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 shutdown_connections 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07c810dc20 0x7f07c819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f07ac0b6700 0x7f07ac0b8c10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07c8105110 0x7f07c819a890 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 --2- 172.21.15.114:0/4158052677 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f07c81047c0 0x7f07c819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 >> 172.21.15.114:0/4158052677 conn(0x7f07c80fe2c0 msgr2=0x7f07c8107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 shutdown_connections 2024-09-21T07:05:02.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:02.669+0000 7f07c7200640 1 -- 172.21.15.114:0/4158052677 wait complete. 2024-09-21T07:05:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 625.0 s, 8.4 tps, lat 20533.277 ms stddev 3203.238 2024-09-21T07:05:03.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:03 smithi183 ceph-mon[34219]: pgmap v731: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-21T07:05:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:03 smithi193 ceph-mon[34601]: pgmap v731: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-21T07:05:03.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:03 smithi114 ceph-mon[30654]: pgmap v731: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-09-21T07:05:05.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:05 smithi183 ceph-mon[34219]: pgmap v732: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 37 op/s 2024-09-21T07:05:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:05 smithi193 ceph-mon[34601]: pgmap v732: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 37 op/s 2024-09-21T07:05:05.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:05 smithi114 ceph-mon[30654]: pgmap v732: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 37 op/s 2024-09-21T07:05:05.840 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a66baf88', total seconds elapsed: 31.3 2024-09-21T07:05:06.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.681+0000 7f9a0ca00640 1 -- 172.21.15.114:0/186317151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a0009ba50 msgr2=0x7f9a0009be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.681+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/186317151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a0009ba50 0x7f9a0009be60 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009900 tx=0x7f99f402a410 comp rx=0 tx=0).stop 2024-09-21T07:05:06.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.681+0000 7f9a0ca00640 1 -- 172.21.15.114:0/186317151 shutdown_connections 2024-09-21T07:05:06.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.682+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/186317151 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009c3a0 0x7f9a000a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.682+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/186317151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a0009ba50 0x7f9a0009be60 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.682+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/186317151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a00096110 0x7f9a00096520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.682+0000 7f9a0ca00640 1 -- 172.21.15.114:0/186317151 >> 172.21.15.114:0/186317151 conn(0x7f9a0000de60 msgr2=0x7f9a0000e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:06.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.682+0000 7f9a0ca00640 1 -- 172.21.15.114:0/186317151 shutdown_connections 2024-09-21T07:05:06.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.682+0000 7f9a0ca00640 1 -- 172.21.15.114:0/186317151 wait complete. 2024-09-21T07:05:06.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.684+0000 7f9a0ca00640 1 Processor -- start 2024-09-21T07:05:06.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.684+0000 7f9a0ca00640 1 -- start start 2024-09-21T07:05:06.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.686+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 0x7f9a000a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.686+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 0x7f9a000a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a0009c3a0 0x7f9a000b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a0ca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00008fe0 con 0x7f9a0009ba50 2024-09-21T07:05:06.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a0ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00008e60 con 0x7f9a00096110 2024-09-21T07:05:06.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a0ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00009160 con 0x7f9a0009c3a0 2024-09-21T07:05:06.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 0x7f9a000a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 0x7f9a000a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49562/0 (socket says 172.21.15.114:49562) 2024-09-21T07:05:06.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a05400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 0x7f9a000a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a05400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 0x7f9a000a6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58590/0 (socket says 172.21.15.114:58590) 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a05e00640 1 -- 172.21.15.114:0/2911477665 learned_addr learned my addr 172.21.15.114:0/2911477665 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.687+0000 7f9a06800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a0009c3a0 0x7f9a000b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a06800640 1 -- 172.21.15.114:0/2911477665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 msgr2=0x7f9a000a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a06800640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 0x7f9a000a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a06800640 1 -- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 msgr2=0x7f9a000a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a06800640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 0x7f9a000a6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.688+0000 7f9a06800640 1 -- 172.21.15.114:0/2911477665 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a000b0cb0 con 0x7f9a0009c3a0 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.689+0000 7f9a06800640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a0009c3a0 0x7f9a000b0580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a08052020 tx=0x7f9a080755f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.689+0000 7f9a05e00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 0x7f9a000a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T07:05:06.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.689+0000 7f9a05400640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 0x7f9a000a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T07:05:06.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.690+0000 7f99fea00640 1 -- 172.21.15.114:0/2911477665 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0806a5d0 con 0x7f9a0009c3a0 2024-09-21T07:05:06.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.690+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a000b0f40 con 0x7f9a0009c3a0 2024-09-21T07:05:06.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.690+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a000ab3f0 con 0x7f9a0009c3a0 2024-09-21T07:05:06.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.690+0000 7f99fea00640 1 -- 172.21.15.114:0/2911477665 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08085070 con 0x7f9a0009c3a0 2024-09-21T07:05:06.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.690+0000 7f99fea00640 1 -- 172.21.15.114:0/2911477665 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0807f890 con 0x7f9a0009c3a0 2024-09-21T07:05:06.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.699+0000 7f9a1367c740 1 Processor -- start 2024-09-21T07:05:06.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.699+0000 7f99fea00640 1 -- 172.21.15.114:0/2911477665 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9a0807fa30 con 0x7f9a0009c3a0 2024-09-21T07:05:06.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.704+0000 7f99fea00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99ec0b6800 0x7f99ec0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.704+0000 7f9a05400640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99ec0b6800 0x7f99ec0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 -- start start 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 0x55618de3af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da6b1b0 0x55618de3b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de3b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55618ddb1f70 con 0x55618da6b1b0 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55618dda13a0 con 0x55618da73390 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a1367c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55618dda51b0 con 0x55618da94070 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f99fea00640 1 -- 172.21.15.114:0/2911477665 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f9a08118580 con 0x7f9a0009c3a0 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 0x55618de3af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de3b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.705+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 0x55618de3af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54486/0 (socket says 172.21.15.114:54486) 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.706+0000 7f9a06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de3b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49570/0 (socket says 172.21.15.114:49570) 2024-09-21T07:05:06.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.706+0000 7f9a05e00640 1 -- 172.21.15.114:0/3477251826 learned_addr learned my addr 172.21.15.114:0/3477251826 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:06.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.706+0000 7f9a06800640 1 -- 172.21.15.114:0/3477251826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 msgr2=0x55618de3af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.707+0000 7f9a06800640 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 0x55618de3af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.707+0000 7f9a06800640 1 -- 172.21.15.114:0/3477251826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da6b1b0 msgr2=0x55618de3b470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:06.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.707+0000 7f9a06800640 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da6b1b0 0x55618de3b470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.707+0000 7f9a06800640 1 -- 172.21.15.114:0/3477251826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618dd6bc90 con 0x55618da73390 2024-09-21T07:05:06.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.708+0000 7f9a05e00640 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 0x55618de3af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:06.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.708+0000 7f9a06800640 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de3b9b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a08153c10 tx=0x7f9a08124100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cfb0382d35d796f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:06.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.709+0000 7f9a05400640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99ec0b6800 0x7f99ec0b8d10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009850 tx=0x7f99f4005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:06.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.709+0000 7f99f3000640 1 -- 172.21.15.114:0/3477251826 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08118b00 con 0x55618da73390 2024-09-21T07:05:06.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.709+0000 7f99f3000640 1 -- 172.21.15.114:0/3477251826 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a08154090 con 0x55618da73390 2024-09-21T07:05:06.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.710+0000 7f99f3000640 1 -- 172.21.15.114:0/3477251826 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08118ca0 con 0x55618da73390 2024-09-21T07:05:06.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.711+0000 7f9a1367c740 1 -- 172.21.15.114:0/3477251826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 msgr2=0x55618de3b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.711+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de3b9b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9a08153c10 tx=0x7f9a08124100 comp rx=0 tx=0).stop 2024-09-21T07:05:06.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.714+0000 7f9a1367c740 1 -- 172.21.15.114:0/3477251826 shutdown_connections 2024-09-21T07:05:06.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.714+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de3b9b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.714+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da6b1b0 0x55618de3b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.714+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3477251826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da94070 0x55618de3af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.715+0000 7f9a1367c740 1 -- 172.21.15.114:0/3477251826 >> 172.21.15.114:0/3477251826 conn(0x55618da88460 msgr2=0x55618de38e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:06.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.715+0000 7f9a1367c740 1 -- 172.21.15.114:0/3477251826 shutdown_connections 2024-09-21T07:05:06.718 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.715+0000 7f9a1367c740 1 -- 172.21.15.114:0/3477251826 wait complete. 2024-09-21T07:05:06.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.716+0000 7f9a1367c740 1 -- start start 2024-09-21T07:05:06.719 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.717+0000 7f9a1367c740 1 Processor -- start 2024-09-21T07:05:06.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.718+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.718+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de46cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.718+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da94070 0x55618de63a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a1367c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55618dc02700 con 0x55618da94070 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a1367c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55618dd3c540 con 0x55618da6b1b0 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a1367c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55618de5a6e0 con 0x55618da73390 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a1367c740 10 client.0 _ll_register_callbacks cb 0x55618dd4c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de46cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de46cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54496/0 (socket says 172.21.15.114:54496) 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a05400640 1 -- 172.21.15.114:0/3922995166 learned_addr learned my addr 172.21.15.114:0/3922995166 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:06.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.719+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de46770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49586/0 (socket says 172.21.15.114:49586) 2024-09-21T07:05:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.720+0000 7f9a05e00640 1 -- 172.21.15.114:0/3922995166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 msgr2=0x55618de46cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.720+0000 7f9a05e00640 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de46cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.720+0000 7f9a05e00640 1 -- 172.21.15.114:0/3922995166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da94070 msgr2=0x55618de63a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.720+0000 7f9a05e00640 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da94070 0x55618de63a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.723 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.720+0000 7f9a05e00640 1 -- 172.21.15.114:0/3922995166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618daff650 con 0x55618da6b1b0 2024-09-21T07:05:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.721+0000 7f9a05e00640 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de46770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f99f800f4a0 tx=0x7f99f800f960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9417ae7eed25d129 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.721+0000 7f99e7400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:06.724 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.722+0000 7f99e7400640 1 -- 172.21.15.114:0/3922995166 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8013850 con 0x55618da6b1b0 2024-09-21T07:05:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.722+0000 7f99e7400640 1 -- 172.21.15.114:0/3922995166 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f99f80139f0 con 0x55618da6b1b0 2024-09-21T07:05:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.722+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55618dd3eb50 con 0x55618da6b1b0 2024-09-21T07:05:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.722+0000 7f99e7400640 1 -- 172.21.15.114:0/3922995166 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f99f8024050 con 0x55618da6b1b0 2024-09-21T07:05:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.722+0000 7f9a1367c740 10 client.28292 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.722+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55618dd59910 con 0x55618da6b1b0 2024-09-21T07:05:06.726 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.723+0000 7f99e7400640 1 -- 172.21.15.114:0/3922995166 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55618dd3eb50 con 0x55618da6b1b0 2024-09-21T07:05:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f99e7400640 1 -- 172.21.15.114:0/3922995166 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f99f80113e0 con 0x55618da6b1b0 2024-09-21T07:05:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f9a1367c740 10 client.28292 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f9a1367c740 10 client.28292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:06.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f9a1367c740 20 client.28292 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da529f0 0x55618de69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f9a1367c740 4 client.28292 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.724+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55618ddb3ec0 con 0x55618da529f0 2024-09-21T07:05:06.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.725+0000 7f9a05400640 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da529f0 0x55618de69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.727+0000 7f99e7400640 10 client.28292 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:06.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.727+0000 7f9a05400640 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da529f0 0x55618de69a60 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe8db59d9923a3 server_cookie=d878392097ff936d in_seq=0 out_seq=0 2024-09-21T07:05:06.730 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.728+0000 7f99e7400640 1 -- 172.21.15.114:0/3922995166 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55618ddb3ec0 con 0x55618da529f0 2024-09-21T07:05:06.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.728+0000 7f99e7400640 10 client.28292 handle_command_reply: tid=0 2024-09-21T07:05:06.731 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.728+0000 7f9a1367c740 1 client.28292 shutdown 2024-09-21T07:05:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.730+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 msgr2=0x55618de46770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.730+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de46770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f99f800f4a0 tx=0x7f99f800f960 comp rx=0 tx=0).stop 2024-09-21T07:05:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 shutdown_connections 2024-09-21T07:05:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da94070 0x55618de63a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da529f0 0x55618de69a60 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f99e7400640 0 client.28292 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de46cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f9a1367c740 1 --2- 172.21.15.114:0/3922995166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de46770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.731+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 >> 172.21.15.114:0/3922995166 conn(0x55618da88460 msgr2=0x55618de375c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.732+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 shutdown_connections 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.732+0000 7f9a1367c740 1 -- 172.21.15.114:0/3922995166 wait complete. 2024-09-21T07:05:06.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.732+0000 7f9a1367c740 20 client.28292 trim_cache size 0 max 16384 2024-09-21T07:05:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.770+0000 7f9a1367c740 1 Processor -- start 2024-09-21T07:05:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 -- start start 2024-09-21T07:05:06.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de41f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 0x55618de483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de63920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8010020 con 0x55618da529f0 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8011b60 con 0x55618da73390 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.771+0000 7f9a1367c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99e80106f0 con 0x55618da6b1b0 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.772+0000 7f9a05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 0x55618de483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.772+0000 7f9a05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 0x55618de483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54500/0 (socket says 172.21.15.114:54500) 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.772+0000 7f9a05400640 1 -- 172.21.15.114:0/87849324 learned_addr learned my addr 172.21.15.114:0/87849324 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.772+0000 7f9a06800640 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de63920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05e00640 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de41f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05400640 1 -- 172.21.15.114:0/87849324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 msgr2=0x55618de63920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05400640 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de63920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05400640 1 -- 172.21.15.114:0/87849324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 msgr2=0x55618de41f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05400640 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de41f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05400640 1 -- 172.21.15.114:0/87849324 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618de47830 con 0x55618da6b1b0 2024-09-21T07:05:06.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.773+0000 7f9a05e00640 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de41f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.774+0000 7f9a05400640 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 0x55618de483a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009230 tx=0x7f99f402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e629f474803c0567 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.775+0000 7f99f2600640 1 -- 172.21.15.114:0/87849324 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4058070 con 0x55618da6b1b0 2024-09-21T07:05:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.775+0000 7f99f2600640 1 -- 172.21.15.114:0/87849324 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55618ddb3ec0 con 0x55618da6b1b0 2024-09-21T07:05:06.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.775+0000 7f99f2600640 1 -- 172.21.15.114:0/87849324 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4035050 con 0x55618da6b1b0 2024-09-21T07:05:06.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.777+0000 7f9a1367c740 1 -- 172.21.15.114:0/87849324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 msgr2=0x55618de483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.777+0000 7f9a1367c740 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 0x55618de483a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009230 tx=0x7f99f402a410 comp rx=0 tx=0).stop 2024-09-21T07:05:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.777+0000 7f9a1367c740 1 -- 172.21.15.114:0/87849324 shutdown_connections 2024-09-21T07:05:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.778+0000 7f9a1367c740 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da73390 0x55618de63920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.778+0000 7f9a1367c740 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da6b1b0 0x55618de483a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.778+0000 7f9a1367c740 1 --2- 172.21.15.114:0/87849324 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de41f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.778+0000 7f9a1367c740 1 -- 172.21.15.114:0/87849324 >> 172.21.15.114:0/87849324 conn(0x55618da88460 msgr2=0x55618de47c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:06.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.780+0000 7f9a1367c740 1 -- 172.21.15.114:0/87849324 shutdown_connections 2024-09-21T07:05:06.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.780+0000 7f9a1367c740 1 -- 172.21.15.114:0/87849324 wait complete. 2024-09-21T07:05:06.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.781+0000 7f9a1367c740 1 -- start start 2024-09-21T07:05:06.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.781+0000 7f9a1367c740 1 Processor -- start 2024-09-21T07:05:06.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.782+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.783+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de5a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.783+0000 7f9a05e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.783+0000 7f9a1367c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de5ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.783+0000 7f9a1367c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8011490 con 0x55618da529f0 2024-09-21T07:05:06.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.783+0000 7f9a1367c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99e8010dc0 con 0x55618da6b1b0 2024-09-21T07:05:06.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.783+0000 7f9a1367c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99e800e020 con 0x55618da73390 2024-09-21T07:05:06.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de5a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de5a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49606/0 (socket says 172.21.15.114:49606) 2024-09-21T07:05:06.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05400640 1 -- 172.21.15.114:0/498224661 learned_addr learned my addr 172.21.15.114:0/498224661 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:06.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a1367c740 10 client.0 _ll_register_callbacks cb 0x7f99f800aea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:06.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a06800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de5ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05e00640 1 -- 172.21.15.114:0/498224661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 msgr2=0x55618de5ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05e00640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de5ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05e00640 1 -- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 msgr2=0x55618de5a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05e00640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de5a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.784+0000 7f9a05e00640 1 -- 172.21.15.114:0/498224661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55618de5b4f0 con 0x55618da529f0 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.785+0000 7f9a06800640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de5ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.785+0000 7f9a05400640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de5a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.785+0000 7f9a05e00640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de60050 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f99f800f470 tx=0x7f99f800a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a97935bc0390986 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:06.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.785+0000 7f99e7400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:05:06.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.786+0000 7f99e7400640 1 -- 172.21.15.114:0/498224661 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f802e020 con 0x55618da529f0 2024-09-21T07:05:06.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.786+0000 7f99e7400640 1 -- 172.21.15.114:0/498224661 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f99f80119a0 con 0x55618da529f0 2024-09-21T07:05:06.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.786+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55618ddb3ec0 con 0x55618da529f0 2024-09-21T07:05:06.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.786+0000 7f99e7400640 1 -- 172.21.15.114:0/498224661 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f99f8011ba0 con 0x55618da529f0 2024-09-21T07:05:06.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.786+0000 7f9a1367c740 10 client.17808 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:06.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.786+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55618ded5280 con 0x55618da529f0 2024-09-21T07:05:06.793 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.790+0000 7f99e7400640 1 -- 172.21.15.114:0/498224661 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55618ddb3ec0 con 0x55618da529f0 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.790+0000 7f99e7400640 1 -- 172.21.15.114:0/498224661 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f99f8013bd0 con 0x55618da529f0 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.791+0000 7f9a1367c740 10 client.17808 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.791+0000 7f9a1367c740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.791+0000 7f9a1367c740 20 client.17808 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.791+0000 7f9a1367c740 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da94070 0x55618ded7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.791+0000 7f9a1367c740 4 client.17808 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.791+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55618ddaf970 con 0x55618da94070 2024-09-21T07:05:06.794 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.792+0000 7f9a05400640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da94070 0x55618ded7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:06.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.796+0000 7f9a05400640 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da94070 0x55618ded7ac0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af1f18f588d01be server_cookie=f768463de019d7d6 in_seq=0 out_seq=0 2024-09-21T07:05:06.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.796+0000 7f99e7400640 10 client.17808 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:06.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.800+0000 7f99e7400640 1 -- 172.21.15.114:0/498224661 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x55618ddaf970 con 0x55618da94070 2024-09-21T07:05:06.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.800+0000 7f99e7400640 10 client.17808 handle_command_reply: tid=0 2024-09-21T07:05:06.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.801+0000 7f9a1367c740 1 client.17808 shutdown 2024-09-21T07:05:06.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 msgr2=0x55618de60050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de60050 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f99f800f470 tx=0x7f99f800a120 comp rx=0 tx=0).stop 2024-09-21T07:05:06.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 shutdown_connections 2024-09-21T07:05:06.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55618da94070 0x55618ded7ac0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55618da73390 0x55618de5ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55618da6b1b0 0x55618de5a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 --2- 172.21.15.114:0/498224661 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55618da529f0 0x55618de60050 secure :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f99f800f470 tx=0x7f99f800a120 comp rx=0 tx=0).stop 2024-09-21T07:05:06.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 >> 172.21.15.114:0/498224661 conn(0x55618da88460 msgr2=0x55618de3bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:06.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.802+0000 7f99e7400640 0 client.17808 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:06.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.803+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 shutdown_connections 2024-09-21T07:05:06.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.803+0000 7f9a1367c740 1 -- 172.21.15.114:0/498224661 wait complete. 2024-09-21T07:05:06.806 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.804+0000 7f9a1367c740 20 client.17808 trim_cache size 0 max 16384 2024-09-21T07:05:06.811 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:05:06.811 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:05:06.811 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:05:06.811 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:05:06.811 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.813 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.814 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:05:06.815 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:05:06.816 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:06.817 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.807+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99ec0b6800 msgr2=0x7f99ec0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.807+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99ec0b6800 0x7f99ec0b8d10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009850 tx=0x7f99f4005fd0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.807+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a0009c3a0 msgr2=0x7f9a000b0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.807+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a0009c3a0 0x7f9a000b0580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9a08052020 tx=0x7f9a080755f0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 shutdown_connections 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a0009c3a0 0x7f9a000b0580 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9a0009ba50 0x7f9a000a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f99ec0b6800 0x7f99ec0b8d10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 --2- 172.21.15.114:0/2911477665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a00096110 0x7f9a000a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:06.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 >> 172.21.15.114:0/2911477665 conn(0x7f9a0000de60 msgr2=0x7f9a000988d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:06.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.808+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 shutdown_connections 2024-09-21T07:05:06.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:06.809+0000 7f9a0ca00640 1 -- 172.21.15.114:0/2911477665 wait complete. 2024-09-21T07:05:07.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:07 smithi183 ceph-mon[34219]: pgmap v733: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 252 KiB/s wr, 52 op/s 2024-09-21T07:05:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:07 smithi193 ceph-mon[34601]: pgmap v733: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 252 KiB/s wr, 52 op/s 2024-09-21T07:05:07.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:07 smithi114 ceph-mon[30654]: pgmap v733: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 252 KiB/s wr, 52 op/s 2024-09-21T07:05:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 630.0 s, 412.6 tps, lat 849.674 ms stddev 3720.454 2024-09-21T07:05:08.097 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:05:08.098 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:05:08.769 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:09.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:09 smithi114 ceph-mon[30654]: pgmap v734: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-21T07:05:09.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:09 smithi193 ceph-mon[34601]: pgmap v734: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-21T07:05:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:09 smithi183 ceph-mon[34219]: pgmap v734: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1.6 MiB/s wr, 142 op/s 2024-09-21T07:05:11.162 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 -- 172.21.15.114:0/335818521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 msgr2=0x7f53840a2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 --2- 172.21.15.114:0/335818521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f53840a2410 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5378009860 tx=0x7f537802a340 comp rx=0 tx=0).stop 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 -- 172.21.15.114:0/335818521 shutdown_connections 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 --2- 172.21.15.114:0/335818521 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53840a2950 0x7f53840a91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 --2- 172.21.15.114:0/335818521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f53840a2410 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 --2- 172.21.15.114:0/335818521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f538409cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.972+0000 7f5394400640 1 -- 172.21.15.114:0/335818521 >> 172.21.15.114:0/335818521 conn(0x7f538400bdb0 msgr2=0x7f538400e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.973+0000 7f5394400640 1 -- 172.21.15.114:0/335818521 shutdown_connections 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.973+0000 7f5394400640 1 -- 172.21.15.114:0/335818521 wait complete. 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.976+0000 7f5394400640 1 Processor -- start 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.977+0000 7f5394400640 1 -- start start 2024-09-21T07:05:11.163 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.980+0000 7f5394400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f5384144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.980+0000 7f5394400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f5384144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.980+0000 7f5394400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53840a2950 0x7f53841498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.980+0000 7f5391a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f5384144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5394400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f53840ac5e0 con 0x7f53840a2950 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5394400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53840ac460 con 0x7f53840a2000 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5394400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f53840ac760 con 0x7f538409c9b0 2024-09-21T07:05:11.164 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f5384144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f5384144080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54518/0 (socket says 172.21.15.114:54518) 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391a00640 1 -- 172.21.15.114:0/2059043040 learned_addr learned my addr 172.21.15.114:0/2059043040 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5392400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53840a2950 0x7f53841498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391a00640 1 -- 172.21.15.114:0/2059043040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 msgr2=0x7f5384144620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391a00640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f5384144620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391a00640 1 -- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53840a2950 msgr2=0x7f53841498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.981+0000 7f5391a00640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53840a2950 0x7f53841498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f5391a00640 1 -- 172.21.15.114:0/2059043040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5384149ff0 con 0x7f538409c9b0 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f5391000640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f5384144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f5391a00640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f5384144080 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f538000d950 tx=0x7f538000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380017030 con 0x7f538409c9b0 2024-09-21T07:05:11.165 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5380004020 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53800047a0 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.982+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538414a2e0 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.983+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5384150a40 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.986+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c0052c0 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.988+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5380009a30 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.994+0000 7f538a000640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53740b6780 0x7f53740b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.995+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f53800e7500 con 0x7f538409c9b0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.997+0000 7f5391000640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53740b6780 0x7f53740b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:10.999+0000 7f5391000640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53740b6780 0x7f53740b8c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f538400f550 tx=0x7f5378000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:11.166 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.000+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f53800ad620 con 0x7f538409c9b0 2024-09-21T07:05:11.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:11 smithi114 ceph-mon[30654]: pgmap v735: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2024-09-21T07:05:11.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:11 smithi193 ceph-mon[34601]: pgmap v735: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2024-09-21T07:05:11.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.424+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f536c005880 con 0x7f538409c9b0 2024-09-21T07:05:11.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.428+0000 7f538a000640 1 -- 172.21.15.114:0/2059043040 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11989 (secure 0 0 0) 0x7f53800b1a00 con 0x7f538409c9b0 2024-09-21T07:05:11.430 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:05:11.431 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 49 2024-09-21T07:05:11.431 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":49,"btime":"2024-09-21T07:03:23:885749+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:03:22.931940+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:05:11.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.431+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53740b6780 msgr2=0x7f53740b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:11.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.431+0000 7f5394400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53740b6780 0x7f53740b8c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f538400f550 tx=0x7f5378000bc0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.431+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 msgr2=0x7f5384144080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:11.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.431+0000 7f5394400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f5384144080 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f538000d950 tx=0x7f538000de10 comp rx=0 tx=0).stop 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 shutdown_connections 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f53840a2950 0x7f53841498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f53740b6780 0x7f53740b8c90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53840a2000 0x7f5384144620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 --2- 172.21.15.114:0/2059043040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f538409c9b0 0x7f5384144080 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 >> 172.21.15.114:0/2059043040 conn(0x7f538400bdb0 msgr2=0x7f538409f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 shutdown_connections 2024-09-21T07:05:11.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:11.432+0000 7f5394400640 1 -- 172.21.15.114:0/2059043040 wait complete. 2024-09-21T07:05:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:11 smithi183 ceph-mon[34219]: pgmap v735: 129 pgs: 129 active+clean; 2.6 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.0 MiB/s wr, 237 op/s 2024-09-21T07:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 -- 172.21.15.114:0/361141686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 msgr2=0x7f8b6810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 --2- 172.21.15.114:0/361141686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b6810a7d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009900 tx=0x7f8b5802a410 comp rx=0 tx=0).stop 2024-09-21T07:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 -- 172.21.15.114:0/361141686 shutdown_connections 2024-09-21T07:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 --2- 172.21.15.114:0/361141686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b68116510 0x7f8b68118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 --2- 172.21.15.114:0/361141686 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b68115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 --2- 172.21.15.114:0/361141686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b6810a7d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 -- 172.21.15.114:0/361141686 >> 172.21.15.114:0/361141686 conn(0x7f8b68070c00 msgr2=0x7f8b68071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 -- 172.21.15.114:0/361141686 shutdown_connections 2024-09-21T07:05:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.993+0000 7f8b67200640 1 -- 172.21.15.114:0/361141686 wait complete. 2024-09-21T07:05:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.995+0000 7f8b67200640 1 Processor -- start 2024-09-21T07:05:11.999 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 -- start start 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b681a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b681a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b68116510 0x7f8b681a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6811c0c0 con 0x7f8b6810ad10 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6811bf40 con 0x7f8b6810a3c0 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.996+0000 7f8b67200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6811c240 con 0x7f8b68116510 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.997+0000 7f8b65400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b681a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.997+0000 7f8b65400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b681a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58658/0 (socket says 172.21.15.114:58658) 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.997+0000 7f8b65400640 1 -- 172.21.15.114:0/922586928 learned_addr learned my addr 172.21.15.114:0/922586928 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65e00640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b681a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65e00640 1 -- 172.21.15.114:0/922586928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b68116510 msgr2=0x7f8b681a2010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65e00640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b68116510 0x7f8b681a2010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65e00640 1 -- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 msgr2=0x7f8b681a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65e00640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b681a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65e00640 1 -- 172.21.15.114:0/922586928 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b681a8830 con 0x7f8b6810a3c0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.998+0000 7f8b65400640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b681a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.999+0000 7f8b65e00640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b681a1590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009850 tx=0x7f8b58032910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.999+0000 7f8b5ea00640 1 -- 172.21.15.114:0/922586928 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58040030 con 0x7f8b6810a3c0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.999+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b681a8ac0 con 0x7f8b6810a3c0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.999+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a9000 con 0x7f8b6810a3c0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.999+0000 7f8b5ea00640 1 -- 172.21.15.114:0/922586928 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b58031400 con 0x7f8b6810a3c0 2024-09-21T07:05:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:11.999+0000 7f8b5ea00640 1 -- 172.21.15.114:0/922586928 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58035020 con 0x7f8b6810a3c0 2024-09-21T07:05:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.015+0000 7f8b73dfe740 1 Processor -- start 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 -- start start 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f200be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f200c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f200c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5622f1f816b0 con 0x5622f1c3e2f0 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622f1f70ae0 con 0x5622f1c671b0 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5622f1f748f0 con 0x5622f1c464d0 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.016+0000 7f8b65400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f200c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.017+0000 7f8b65400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f200c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54528/0 (socket says 172.21.15.114:54528) 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.017+0000 7f8b65400640 1 -- 172.21.15.114:0/1307831892 learned_addr learned my addr 172.21.15.114:0/1307831892 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.017+0000 7f8b5ea00640 1 -- 172.21.15.114:0/922586928 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8b58043030 con 0x7f8b6810a3c0 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.017+0000 7f8b65400640 1 -- 172.21.15.114:0/1307831892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 msgr2=0x5622f200be10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.017+0000 7f8b65400640 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f200be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.017+0000 7f8b65400640 1 -- 172.21.15.114:0/1307831892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 msgr2=0x5622f200c350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.019+0000 7f8b5ea00640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b4c0b6780 0x7f8b4c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.020+0000 7f8b5ea00640 1 -- 172.21.15.114:0/922586928 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f8b581082b0 con 0x7f8b6810a3c0 2024-09-21T07:05:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.022+0000 7f8b65e00640 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f200c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.023+0000 7f8b65400640 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f200c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.023+0000 7f8b65400640 1 -- 172.21.15.114:0/1307831892 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f1f3d780 con 0x5622f1c464d0 2024-09-21T07:05:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.023+0000 7f8b65e00640 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f200c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.023+0000 7f8b66800640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b4c0b6780 0x7f8b4c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.024+0000 7f8b66800640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b4c0b6780 0x7f8b4c0b8c90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000aab0 tx=0x7f8b60009230 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.024+0000 7f8b65400640 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f200c890 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b500325c0 tx=0x7f8b50032a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec979a43f802e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:12.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.025+0000 7f8b57000640 1 -- 172.21.15.114:0/1307831892 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b500307e0 con 0x5622f1c464d0 2024-09-21T07:05:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.025+0000 7f8b57000640 1 -- 172.21.15.114:0/1307831892 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b50030e30 con 0x5622f1c464d0 2024-09-21T07:05:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.025+0000 7f8b57000640 1 -- 172.21.15.114:0/1307831892 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50033450 con 0x5622f1c464d0 2024-09-21T07:05:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.026+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1307831892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 msgr2=0x5622f200c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.026+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f200c890 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b500325c0 tx=0x7f8b50032a80 comp rx=0 tx=0).stop 2024-09-21T07:05:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.027+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1307831892 shutdown_connections 2024-09-21T07:05:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.027+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f200c890 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.027+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f200c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.027+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1307831892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f200be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.027+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1307831892 >> 172.21.15.114:0/1307831892 conn(0x5622f1c5b300 msgr2=0x5622f2009d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.028+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1307831892 shutdown_connections 2024-09-21T07:05:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.029+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1307831892 wait complete. 2024-09-21T07:05:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.030+0000 7f8b73dfe740 1 -- start start 2024-09-21T07:05:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.030+0000 7f8b73dfe740 1 Processor -- start 2024-09-21T07:05:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.034+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f2017480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f20179c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f20238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5622f1ea8ab0 con 0x5622f1c3e2f0 2024-09-21T07:05:12.038 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622f1f17570 con 0x5622f1c671b0 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5622f201a590 con 0x5622f1c464d0 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b66800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f20238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.035+0000 7f8b66800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f20238e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49638/0 (socket says 172.21.15.114:49638) 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 -- 172.21.15.114:0/3908294312 learned_addr learned my addr 172.21.15.114:0/3908294312 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b73dfe740 10 client.0 _ll_register_callbacks cb 0x5622f1f26100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 -- 172.21.15.114:0/3908294312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 msgr2=0x5622f20179c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f20179c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 -- 172.21.15.114:0/3908294312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 msgr2=0x5622f2017480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f2017480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 -- 172.21.15.114:0/3908294312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f1cd2790 con 0x5622f1c671b0 2024-09-21T07:05:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.036+0000 7f8b66800640 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f20238e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000da30 tx=0x7f8b60004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43880802219fec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.038+0000 7f8b4b400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.038+0000 7f8b4b400640 1 -- 172.21.15.114:0/3908294312 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60018d90 con 0x5622f1c671b0 2024-09-21T07:05:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.038+0000 7f8b4b400640 1 -- 172.21.15.114:0/3908294312 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f8b60023480 con 0x5622f1c671b0 2024-09-21T07:05:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.038+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b50030e30 con 0x5622f1c671b0 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.039+0000 7f8b4b400640 1 -- 172.21.15.114:0/3908294312 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8b6002c050 con 0x5622f1c671b0 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.039+0000 7f8b73dfe740 10 client.28310 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.039+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5622f1d4b620 con 0x5622f1c671b0 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b4b400640 1 -- 172.21.15.114:0/3908294312 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b60023b70 con 0x5622f1c671b0 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b4b400640 1 -- 172.21.15.114:0/3908294312 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f8b6001aa80 con 0x5622f1c671b0 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b73dfe740 10 client.28310 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b73dfe740 10 client.28310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b73dfe740 20 client.28310 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1c25d80 0x5622f20296e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b73dfe740 4 client.28310 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.040+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b50030e30 con 0x5622f1c25d80 2024-09-21T07:05:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.042+0000 7f8b65e00640 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1c25d80 0x5622f20296e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.042+0000 7f8b65e00640 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1c25d80 0x5622f20296e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31b28af0bae0415 server_cookie=8bcaa3131a08376d in_seq=0 out_seq=0 2024-09-21T07:05:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.042+0000 7f8b4b400640 10 client.28310 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:12.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.043+0000 7f8b4b400640 1 -- 172.21.15.114:0/3908294312 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8b50030e30 con 0x5622f1c25d80 2024-09-21T07:05:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.043+0000 7f8b4b400640 10 client.28310 handle_command_reply: tid=0 2024-09-21T07:05:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.043+0000 7f8b73dfe740 1 client.28310 shutdown 2024-09-21T07:05:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 msgr2=0x5622f20238e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.046 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f20238e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000da30 tx=0x7f8b60004ad0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 shutdown_connections 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c671b0 0x5622f20238e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1c25d80 0x5622f20296e0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c464d0 0x5622f20179c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/3908294312 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f2017480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 >> 172.21.15.114:0/3908294312 conn(0x5622f1c5b300 msgr2=0x5622f200ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:12.047 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.045+0000 7f8b4b400640 0 client.28310 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.046+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 shutdown_connections 2024-09-21T07:05:12.048 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.046+0000 7f8b73dfe740 1 -- 172.21.15.114:0/3908294312 wait complete. 2024-09-21T07:05:12.049 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.048+0000 7f8b73dfe740 20 client.28310 trim_cache size 0 max 16384 2024-09-21T07:05:12.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.103+0000 7f8b73dfe740 1 Processor -- start 2024-09-21T07:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.103+0000 7f8b73dfe740 1 -- start start 2024-09-21T07:05:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.104+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f20190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.104+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f20237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.104+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f2014650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.104+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440100d0 con 0x5622f1c3e2f0 2024-09-21T07:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.104+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44011c10 con 0x5622f1c464d0 2024-09-21T07:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.104+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440107a0 con 0x5622f1c671b0 2024-09-21T07:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.105+0000 7f8b66800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f2014650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.105+0000 7f8b66800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f2014650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54540/0 (socket says 172.21.15.114:54540) 2024-09-21T07:05:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.105+0000 7f8b66800640 1 -- 172.21.15.114:0/1302289936 learned_addr learned my addr 172.21.15.114:0/1302289936 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.105+0000 7f8b65400640 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f20237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.105+0000 7f8b65e00640 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f20190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.106+0000 7f8b66800640 1 -- 172.21.15.114:0/1302289936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 msgr2=0x5622f20237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.106+0000 7f8b66800640 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f20237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.106+0000 7f8b66800640 1 -- 172.21.15.114:0/1302289936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 msgr2=0x5622f20190b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.106+0000 7f8b66800640 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f20190b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.106+0000 7f8b66800640 1 -- 172.21.15.114:0/1302289936 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f2004000 con 0x5622f1c671b0 2024-09-21T07:05:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.106+0000 7f8b66800640 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f2014650 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000c760 tx=0x7f8b6000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3aa880912eaa2ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.107+0000 7f8b56600640 1 -- 172.21.15.114:0/1302289936 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60019580 con 0x5622f1c671b0 2024-09-21T07:05:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.107+0000 7f8b56600640 1 -- 172.21.15.114:0/1302289936 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b60003a10 con 0x5622f1c671b0 2024-09-21T07:05:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.107+0000 7f8b56600640 1 -- 172.21.15.114:0/1302289936 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60019de0 con 0x5622f1c671b0 2024-09-21T07:05:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.109+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1302289936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 msgr2=0x5622f2014650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.109+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f2014650 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000c760 tx=0x7f8b6000c7e0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.110+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1302289936 shutdown_connections 2024-09-21T07:05:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.110+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f2014650 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.110+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f20237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.110+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/1302289936 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f20190b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.110+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1302289936 >> 172.21.15.114:0/1302289936 conn(0x5622f1c5b300 msgr2=0x5622f20189a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.113+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1302289936 shutdown_connections 2024-09-21T07:05:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.113+0000 7f8b73dfe740 1 -- 172.21.15.114:0/1302289936 wait complete. 2024-09-21T07:05:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.114+0000 7f8b73dfe740 1 -- start start 2024-09-21T07:05:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.114+0000 7f8b73dfe740 1 Processor -- start 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f201d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f201d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f208b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44011540 con 0x5622f1c3e2f0 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b44010e70 con 0x5622f1c464d0 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4400e0d0 con 0x5622f1c671b0 2024-09-21T07:05:12.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b73dfe740 10 client.0 _ll_register_callbacks cb 0x7f8b6000cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.115+0000 7f8b66800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f208b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.116+0000 7f8b66800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f208b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54554/0 (socket says 172.21.15.114:54554) 2024-09-21T07:05:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.116+0000 7f8b66800640 1 -- 172.21.15.114:0/2944137069 learned_addr learned my addr 172.21.15.114:0/2944137069 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.116+0000 7f8b65400640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f201d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.117+0000 7f8b66800640 1 -- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 msgr2=0x5622f201d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.117+0000 7f8b66800640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f201d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.117+0000 7f8b66800640 1 -- 172.21.15.114:0/2944137069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 msgr2=0x5622f201d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.117+0000 7f8b66800640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f201d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.117+0000 7f8b66800640 1 -- 172.21.15.114:0/2944137069 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f208b740 con 0x5622f1c671b0 2024-09-21T07:05:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.118+0000 7f8b65400640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f201d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.118+0000 7f8b66800640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f208b0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000da30 tx=0x7f8b600197d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55396da505b67eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.118+0000 7f8b4b400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.119+0000 7f8b4b400640 1 -- 172.21.15.114:0/2944137069 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60042030 con 0x5622f1c671b0 2024-09-21T07:05:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.120+0000 7f8b4b400640 1 -- 172.21.15.114:0/2944137069 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f8b60029070 con 0x5622f1c671b0 2024-09-21T07:05:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.120+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b60003a10 con 0x5622f1c671b0 2024-09-21T07:05:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.127+0000 7f8b4b400640 1 -- 172.21.15.114:0/2944137069 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8b60019a20 con 0x5622f1c671b0 2024-09-21T07:05:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.127+0000 7f8b73dfe740 10 client.28249 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.127+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5622f2093c60 con 0x5622f1c671b0 2024-09-21T07:05:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b4b400640 1 -- 172.21.15.114:0/2944137069 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b60003a10 con 0x5622f1c671b0 2024-09-21T07:05:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b4b400640 1 -- 172.21.15.114:0/2944137069 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f8b6003b480 con 0x5622f1c671b0 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b73dfe740 10 client.28249 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b73dfe740 10 client.28249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b73dfe740 20 client.28249 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1bfb1c0 0x5622f2096340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b73dfe740 4 client.28249 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.128+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622f1f83600 con 0x5622f1bfb1c0 2024-09-21T07:05:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.129+0000 7f8b65e00640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1bfb1c0 0x5622f2096340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.130+0000 7f8b4b400640 10 client.28249 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.130+0000 7f8b65e00640 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1bfb1c0 0x5622f2096340 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bee49f102ba1e27 server_cookie=530d6edbe19dfb62 in_seq=0 out_seq=0 2024-09-21T07:05:12.132 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:12 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2059043040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:05:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.145+0000 7f8b4b400640 1 -- 172.21.15.114:0/2944137069 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622f1f83600 con 0x5622f1bfb1c0 2024-09-21T07:05:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b4b400640 10 client.28249 handle_command_reply: tid=0 2024-09-21T07:05:12.147 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 client.28249 shutdown 2024-09-21T07:05:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 msgr2=0x5622f208b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f208b0a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000da30 tx=0x7f8b600197d0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 shutdown_connections 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5622f1bfb1c0 0x5622f2096340 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5622f1c671b0 0x5622f208b0a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622f1c464d0 0x5622f201d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b4b400640 0 client.28249 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 --2- 172.21.15.114:0/2944137069 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5622f1c3e2f0 0x5622f201d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.146+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 >> 172.21.15.114:0/2944137069 conn(0x5622f1c5b300 msgr2=0x5622f200ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.147+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 shutdown_connections 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.147+0000 7f8b73dfe740 1 -- 172.21.15.114:0/2944137069 wait complete. 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.147+0000 7f8b73dfe740 20 client.28249 trim_cache size 0 max 16384 2024-09-21T07:05:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:05:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:05:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:05:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:05:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b4c0b6780 msgr2=0x7f8b4c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b4c0b6780 0x7f8b4c0b8c90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000aab0 tx=0x7f8b60009230 comp rx=0 tx=0).stop 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 msgr2=0x7f8b681a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b681a1590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009850 tx=0x7f8b58032910 comp rx=0 tx=0).stop 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 shutdown_connections 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b68116510 0x7f8b681a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8b6810ad10 0x7f8b681a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8b4c0b6780 0x7f8b4c0b8c90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 --2- 172.21.15.114:0/922586928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b6810a3c0 0x7f8b681a1590 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 >> 172.21.15.114:0/922586928 conn(0x7f8b68070c00 msgr2=0x7f8b68073b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 shutdown_connections 2024-09-21T07:05:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:12.148+0000 7f8b67200640 1 -- 172.21.15.114:0/922586928 wait complete. 2024-09-21T07:05:12.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:12 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2059043040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:05:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:12 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2059043040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:05:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 635.0 s, 605.2 tps, lat 163.325 ms stddev 186.841 2024-09-21T07:05:13.105 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:05:13.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:13 smithi114 ceph-mon[30654]: pgmap v736: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.6 MiB/s wr, 258 op/s 2024-09-21T07:05:13.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:13 smithi193 ceph-mon[34601]: pgmap v736: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.6 MiB/s wr, 258 op/s 2024-09-21T07:05:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:13 smithi183 ceph-mon[34219]: pgmap v736: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.6 MiB/s wr, 258 op/s 2024-09-21T07:05:13.713 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:14.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 -- 172.21.15.114:0/1347352928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8077380 msgr2=0x7f45f80757b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 --2- 172.21.15.114:0/1347352928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8077380 0x7f45f80757b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45e801a910 tx=0x7f45e803b430 comp rx=0 tx=0).stop 2024-09-21T07:05:14.670 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 -- 172.21.15.114:0/1347352928 shutdown_connections 2024-09-21T07:05:14.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 --2- 172.21.15.114:0/1347352928 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8114280 0x7f45f8116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 --2- 172.21.15.114:0/1347352928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8075dd0 0x7f45f80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 --2- 172.21.15.114:0/1347352928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8077380 0x7f45f80757b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 -- 172.21.15.114:0/1347352928 >> 172.21.15.114:0/1347352928 conn(0x7f45f80ff440 msgr2=0x7f45f8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:14.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 -- 172.21.15.114:0/1347352928 shutdown_connections 2024-09-21T07:05:14.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.667+0000 7f45f7e00640 1 -- 172.21.15.114:0/1347352928 wait complete. 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.669+0000 7f45f7e00640 1 Processor -- start 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.669+0000 7f45f7e00640 1 -- start start 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8075dd0 0x7f45f8113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 0x7f45f810fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8114280 0x7f45f81102c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f7e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f45f810a200 con 0x7f45f8114280 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f7e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45f810a080 con 0x7f45f8075dd0 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f7e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45f810a380 con 0x7f45f8077380 2024-09-21T07:05:14.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.670+0000 7f45f6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 0x7f45f810fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 0x7f45f810fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54570/0 (socket says 172.21.15.114:54570) 2024-09-21T07:05:14.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 -- 172.21.15.114:0/3889127299 learned_addr learned my addr 172.21.15.114:0/3889127299 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:14.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6a00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8075dd0 0x7f45f8113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 -- 172.21.15.114:0/3889127299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8075dd0 msgr2=0x7f45f8113a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8075dd0 0x7f45f8113a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.673 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 -- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8114280 msgr2=0x7f45f81102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8114280 0x7f45f81102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.671+0000 7f45f6000640 1 -- 172.21.15.114:0/3889127299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f8110ae0 con 0x7f45f8077380 2024-09-21T07:05:14.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.673+0000 7f45f6000640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 0x7f45f810fd20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f45e400e9a0 tx=0x7f45e400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:14.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.674+0000 7f45fca00640 1 -- 172.21.15.114:0/3889127299 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e400cd80 con 0x7f45f8077380 2024-09-21T07:05:14.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.674+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f81a44a0 con 0x7f45f8077380 2024-09-21T07:05:14.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.674+0000 7f45fca00640 1 -- 172.21.15.114:0/3889127299 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e40035f0 con 0x7f45f8077380 2024-09-21T07:05:14.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.674+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f81a4a30 con 0x7f45f8077380 2024-09-21T07:05:14.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.675+0000 7f45fca00640 1 -- 172.21.15.114:0/3889127299 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e400cd80 con 0x7f45f8077380 2024-09-21T07:05:14.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.680+0000 7f4604a80740 1 Processor -- start 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.680+0000 7f4604a80740 1 -- start start 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.681+0000 7f4604a80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c68d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.681+0000 7f4604a80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748991b0 0x55d274c69270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.681+0000 7f4604a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.681+0000 7f4604a80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d274bdffb0 con 0x55d2748991b0 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.681+0000 7f4604a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d274bcf3e0 con 0x55d2748a1390 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.681+0000 7f4604a80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d274bd31f0 con 0x55d2748c2070 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.682+0000 7f45f6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748991b0 0x55d274c69270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.682+0000 7f45f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c68d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.682+0000 7f45f7400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c68d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54578/0 (socket says 172.21.15.114:54578) 2024-09-21T07:05:14.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.682+0000 7f45f7400640 1 -- 172.21.15.114:0/3636448646 learned_addr learned my addr 172.21.15.114:0/3636448646 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:14.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.683+0000 7f45f7400640 1 -- 172.21.15.114:0/3636448646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 msgr2=0x55d274c697b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:14.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.683+0000 7f45f7400640 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c697b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.683+0000 7f45f7400640 1 -- 172.21.15.114:0/3636448646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748991b0 msgr2=0x55d274c69270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:14.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.683+0000 7f45f7400640 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748991b0 0x55d274c69270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.683+0000 7f45f7400640 1 -- 172.21.15.114:0/3636448646 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d274b99cd0 con 0x55d2748c2070 2024-09-21T07:05:14.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.685+0000 7f45f7400640 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c68d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009d50 tx=0x7f45f000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c8afdac7ffe50f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:14.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.685+0000 7f45fca00640 1 -- 172.21.15.114:0/3889127299 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f45e4004000 con 0x7f45f8077380 2024-09-21T07:05:14.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.685+0000 7f45e3e00640 1 -- 172.21.15.114:0/3636448646 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0016070 con 0x55d2748c2070 2024-09-21T07:05:14.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.688+0000 7f45e3e00640 1 -- 172.21.15.114:0/3636448646 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f0005020 con 0x55d2748c2070 2024-09-21T07:05:14.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.688+0000 7f45e3e00640 1 -- 172.21.15.114:0/3636448646 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0011690 con 0x55d2748c2070 2024-09-21T07:05:14.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.689+0000 7f45fca00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45dc0b6870 0x7f45dc0b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.690+0000 7f45fca00640 1 -- 172.21.15.114:0/3889127299 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f45e40e78b0 con 0x7f45f8077380 2024-09-21T07:05:14.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.690+0000 7f45f6a00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45dc0b6870 0x7f45dc0b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 -- 172.21.15.114:0/3636448646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 msgr2=0x55d274c68d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c68d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009d50 tx=0x7f45f000e6c0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 -- 172.21.15.114:0/3636448646 shutdown_connections 2024-09-21T07:05:14.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748991b0 0x55d274c69270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 --2- 172.21.15.114:0/3636448646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c68d30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.691+0000 7f4604a80740 1 -- 172.21.15.114:0/3636448646 >> 172.21.15.114:0/3636448646 conn(0x55d2748b6460 msgr2=0x55d274c66bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:14.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.692+0000 7f4604a80740 1 -- 172.21.15.114:0/3636448646 shutdown_connections 2024-09-21T07:05:14.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.692+0000 7f4604a80740 1 -- 172.21.15.114:0/3636448646 wait complete. 2024-09-21T07:05:14.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.693+0000 7f45f6a00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45dc0b6870 0x7f45dc0b8d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f45e8006010 tx=0x7f45e8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:14.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.694+0000 7f4604a80740 1 -- start start 2024-09-21T07:05:14.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.694+0000 7f4604a80740 1 Processor -- start 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 0x55d274c74930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c74e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c80860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d274a30700 con 0x55d2748a1390 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d274b6a580 con 0x55d2748c2070 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d274c77510 con 0x55d2748991b0 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.696+0000 7f4604a80740 10 client.0 _ll_register_callbacks cb 0x55d274b7a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 0x55d274c74930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 0x55d274c74930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54594/0 (socket says 172.21.15.114:54594) 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 -- 172.21.15.114:0/3568907337 learned_addr learned my addr 172.21.15.114:0/3568907337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 -- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 msgr2=0x55d274c80860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:14.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c80860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 -- 172.21.15.114:0/3568907337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 msgr2=0x55d274c74e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c74e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.697+0000 7f45f6a00640 1 -- 172.21.15.114:0/3568907337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27492d650 con 0x55d2748991b0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.698+0000 7f45f6a00640 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 0x55d274c74930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45e8044000 tx=0x7f45e801a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39bfa782441ef1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.698+0000 7f45e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.700+0000 7f45e0c00640 1 -- 172.21.15.114:0/3568907337 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e805f070 con 0x55d2748991b0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.700+0000 7f45e0c00640 1 -- 172.21.15.114:0/3568907337 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f45e8040510 con 0x55d2748991b0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.700+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d274b6cb90 con 0x55d2748991b0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.700+0000 7f45e0c00640 1 -- 172.21.15.114:0/3568907337 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f45e8069050 con 0x55d2748991b0 2024-09-21T07:05:14.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.700+0000 7f4604a80740 10 client.28261 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.700+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d274b87950 con 0x55d2748991b0 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f45e0c00640 1 -- 172.21.15.114:0/3568907337 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d274b6cb90 con 0x55d2748991b0 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f45e0c00640 1 -- 172.21.15.114:0/3568907337 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f45e805bb20 con 0x55d2748991b0 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f4604a80740 10 client.28261 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f4604a80740 10 client.28261 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f4604a80740 20 client.28261 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:14.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f4604a80740 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d2748809f0 0x55d274c86a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f4604a80740 4 client.28261 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:14.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.701+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d274be1f00 con 0x55d2748809f0 2024-09-21T07:05:14.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.702+0000 7f45f6000640 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d2748809f0 0x55d274c86a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.704+0000 7f45e0c00640 10 client.28261 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:05:14.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.706+0000 7f45f6000640 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d2748809f0 0x55d274c86a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=18b0aba18ae8b00 server_cookie=9104ff927a3c3825 in_seq=0 out_seq=0 2024-09-21T07:05:14.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.708+0000 7f45e0c00640 1 -- 172.21.15.114:0/3568907337 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d274be1f00 con 0x55d2748809f0 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.708+0000 7f45e0c00640 10 client.28261 handle_command_reply: tid=0 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.708+0000 7f4604a80740 1 client.28261 shutdown 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.708+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 msgr2=0x55d274c74930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.708+0000 7f4604a80740 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 0x55d274c74930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45e8044000 tx=0x7f45e801a270 comp rx=0 tx=0).stop 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.709+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 shutdown_connections 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.709+0000 7f4604a80740 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d2748809f0 0x55d274c86a20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.709+0000 7f4604a80740 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c80860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.709+0000 7f4604a80740 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c74e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.709+0000 7f4604a80740 1 --2- 172.21.15.114:0/3568907337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748991b0 0x55d274c74930 secure :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45e8044000 tx=0x7f45e801a270 comp rx=0 tx=0).stop 2024-09-21T07:05:14.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.709+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 >> 172.21.15.114:0/3568907337 conn(0x55d2748b6460 msgr2=0x55d274c6bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:14.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.710+0000 7f45e0c00640 0 client.28261 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:05:14.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.711+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 shutdown_connections 2024-09-21T07:05:14.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.711+0000 7f4604a80740 1 -- 172.21.15.114:0/3568907337 wait complete. 2024-09-21T07:05:14.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.713+0000 7f4604a80740 20 client.28261 trim_cache size 0 max 16384 2024-09-21T07:05:14.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.741+0000 7f4604a80740 1 Processor -- start 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 -- start start 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c72fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d274855da0 0x55d274c6de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8010040 con 0x55d274855da0 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8011b80 con 0x55d2748a1390 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f4604a80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8010710 con 0x55d2748c2070 2024-09-21T07:05:14.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.742+0000 7f45f6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54610/0 (socket says 172.21.15.114:54610) 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 -- 172.21.15.114:0/2080623421 learned_addr learned my addr 172.21.15.114:0/2080623421 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6a00640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c72fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 -- 172.21.15.114:0/2080623421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 msgr2=0x55d274c72fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c72fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f7400640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d274855da0 0x55d274c6de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 -- 172.21.15.114:0/2080623421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d274855da0 msgr2=0x55d274c6de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:14.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d274855da0 0x55d274c6de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.743+0000 7f45f6000640 1 -- 172.21.15.114:0/2080623421 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d274c759f0 con 0x55d2748c2070 2024-09-21T07:05:14.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.745+0000 7f45f6a00640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:14.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.745+0000 7f45f7400640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d274855da0 0x55d274c6de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:14.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.745+0000 7f45f6000640 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c734f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45e40a9900 tx=0x7f45e40acda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278b233fa146338 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:14.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.745+0000 7f45e3400640 1 -- 172.21.15.114:0/2080623421 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40b1e10 con 0x55d2748c2070 2024-09-21T07:05:14.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.745+0000 7f45e3400640 1 -- 172.21.15.114:0/2080623421 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e40b6d50 con 0x55d2748c2070 2024-09-21T07:05:14.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.745+0000 7f45e3400640 1 -- 172.21.15.114:0/2080623421 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40a8ee0 con 0x55d2748c2070 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 -- 172.21.15.114:0/2080623421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 msgr2=0x55d274c734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c734f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45e40a9900 tx=0x7f45e40acda0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 -- 172.21.15.114:0/2080623421 shutdown_connections 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d274855da0 0x55d274c6de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d2748c2070 0x55d274c734f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 --2- 172.21.15.114:0/2080623421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748a1390 0x55d274c72fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 -- 172.21.15.114:0/2080623421 >> 172.21.15.114:0/2080623421 conn(0x55d2748b6460 msgr2=0x55d274c74910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:14.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.747+0000 7f4604a80740 1 -- 172.21.15.114:0/2080623421 shutdown_connections 2024-09-21T07:05:14.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.748+0000 7f4604a80740 1 -- 172.21.15.114:0/2080623421 wait complete. 2024-09-21T07:05:14.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.748+0000 7f4604a80740 1 -- start start 2024-09-21T07:05:14.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.748+0000 7f4604a80740 1 Processor -- start 2024-09-21T07:05:14.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.749+0000 7f4604a80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d274855da0 0x55d274c78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.749+0000 7f4604a80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c78ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.749+0000 7f4604a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f4604a80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80114b0 con 0x55d2748a1390 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f4604a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8010de0 con 0x55d2748c2070 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f4604a80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45d800e040 con 0x55d274855da0 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f4604a80740 10 client.0 _ll_register_callbacks cb 0x7f45e8004a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f45f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f45f7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49706/0 (socket says 172.21.15.114:49706) 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f45f7400640 1 -- 172.21.15.114:0/122744647 learned_addr learned my addr 172.21.15.114:0/122744647 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:14.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.751+0000 7f45f6a00640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d274855da0 0x55d274c78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.752+0000 7f45f6000640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c78ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.753+0000 7f45f7400640 1 -- 172.21.15.114:0/122744647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d274855da0 msgr2=0x55d274c78760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:14.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.753+0000 7f45f7400640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d274855da0 0x55d274c78760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.753+0000 7f45f7400640 1 -- 172.21.15.114:0/122744647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 msgr2=0x55d274c78ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:14.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.753+0000 7f45f7400640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c78ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.753+0000 7f45f7400640 1 -- 172.21.15.114:0/122744647 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d274cf03e0 con 0x55d2748c2070 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.754+0000 7f45f7400640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c791e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000f80 tx=0x7f45f0009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=160d763643626787 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.754+0000 7f45e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.754+0000 7f45e0c00640 1 -- 172.21.15.114:0/122744647 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0028060 con 0x55d2748c2070 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.754+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e40b6d50 con 0x55d2748c2070 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.754+0000 7f45e0c00640 1 -- 172.21.15.114:0/122744647 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f45f000edd0 con 0x55d2748c2070 2024-09-21T07:05:14.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.755+0000 7f45e0c00640 1 -- 172.21.15.114:0/122744647 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f45f002f050 con 0x55d2748c2070 2024-09-21T07:05:14.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.755+0000 7f4604a80740 10 client.28328 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:14.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.755+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55d274cf15a0 con 0x55d2748c2070 2024-09-21T07:05:14.757 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f45e0c00640 1 -- 172.21.15.114:0/122744647 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e40b6d50 con 0x55d2748c2070 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f45e0c00640 1 -- 172.21.15.114:0/122744647 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f45f000f040 con 0x55d2748c2070 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f4604a80740 10 client.28328 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f4604a80740 10 client.28328 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f4604a80740 20 client.28328 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f4604a80740 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d274cf1990 0x55d274cf3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f4604a80740 4 client.28328 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d274bdd9b0 con 0x55d274cf1990 2024-09-21T07:05:14.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.756+0000 7f45f6a00640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d274cf1990 0x55d274cf3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:14.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.757+0000 7f45e0c00640 10 client.28328 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:05:14.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.757+0000 7f45f6a00640 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d274cf1990 0x55d274cf3db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee1ade0a13671343 server_cookie=2ac346d049af9749 in_seq=0 out_seq=0 2024-09-21T07:05:14.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f45e0c00640 1 -- 172.21.15.114:0/122744647 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d274bdd9b0 con 0x55d274cf1990 2024-09-21T07:05:14.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f45e0c00640 10 client.28328 handle_command_reply: tid=0 2024-09-21T07:05:14.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 client.28328 shutdown 2024-09-21T07:05:14.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 msgr2=0x55d274c791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c791e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f45f0000f80 tx=0x7f45f0009a90 comp rx=0 tx=0).stop 2024-09-21T07:05:14.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 shutdown_connections 2024-09-21T07:05:14.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d274cf1990 0x55d274cf3db0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f45e0c00640 0 client.28328 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:05:14.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2748c2070 0x55d274c791e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d2748a1390 0x55d274c78ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 --2- 172.21.15.114:0/122744647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d274855da0 0x55d274c78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.758+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 >> 172.21.15.114:0/122744647 conn(0x55d2748b6460 msgr2=0x55d274c6de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.759+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 shutdown_connections 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.759+0000 7f4604a80740 1 -- 172.21.15.114:0/122744647 wait complete. 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.759+0000 7f4604a80740 20 client.28328 trim_cache size 0 max 16384 2024-09-21T07:05:14.762 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:05:14.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45dc0b6870 msgr2=0x7f45dc0b8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45dc0b6870 0x7f45dc0b8d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f45e8006010 tx=0x7f45e8005a20 comp rx=0 tx=0).stop 2024-09-21T07:05:14.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 msgr2=0x7f45f810fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:14.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 0x7f45f810fd20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f45e400e9a0 tx=0x7f45e400ee60 comp rx=0 tx=0).stop 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 shutdown_connections 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f45f8114280 0x7f45f81102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45f8077380 0x7f45f810fd20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f45dc0b6870 0x7f45dc0b8d80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 --2- 172.21.15.114:0/3889127299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45f8075dd0 0x7f45f8113a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.761+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 >> 172.21.15.114:0/3889127299 conn(0x7f45f80ff440 msgr2=0x7f45f81033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.762+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 shutdown_connections 2024-09-21T07:05:14.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:14.762+0000 7f45f7e00640 1 -- 172.21.15.114:0/3889127299 wait complete. 2024-09-21T07:05:15.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:15 smithi193 ceph-mon[34601]: pgmap v737: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.9 MiB/s wr, 390 op/s 2024-09-21T07:05:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:15 smithi114 ceph-mon[30654]: pgmap v737: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.9 MiB/s wr, 390 op/s 2024-09-21T07:05:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:15 smithi183 ceph-mon[34219]: pgmap v737: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.9 MiB/s wr, 390 op/s 2024-09-21T07:05:15.790 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:05:16.486 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:17.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:17 smithi193 ceph-mon[34601]: pgmap v738: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-21T07:05:17.415 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:17 smithi183 ceph-mon[34219]: pgmap v738: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-21T07:05:17.533 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:17 smithi114 ceph-mon[30654]: pgmap v738: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 6.3 MiB/s wr, 389 op/s 2024-09-21T07:05:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 640.0 s, 495.8 tps, lat 195.643 ms stddev 241.636 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.865+0000 7fa3c7200640 1 -- 172.21.15.114:0/2518005862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 msgr2=0x7fa3c8110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.865+0000 7fa3c7200640 1 --2- 172.21.15.114:0/2518005862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c8110110 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009ca0 tx=0x7fa3c002a8d0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.866+0000 7fa3c7200640 1 -- 172.21.15.114:0/2518005862 shutdown_connections 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.866+0000 7fa3c7200640 1 --2- 172.21.15.114:0/2518005862 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c8110110 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.866+0000 7fa3c7200640 1 --2- 172.21.15.114:0/2518005862 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 0x7fa3c810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.866+0000 7fa3c7200640 1 --2- 172.21.15.114:0/2518005862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 0x7fa3c8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.866+0000 7fa3c7200640 1 -- 172.21.15.114:0/2518005862 >> 172.21.15.114:0/2518005862 conn(0x7fa3c80fe2c0 msgr2=0x7fa3c8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:18.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.868+0000 7fa3c7200640 1 -- 172.21.15.114:0/2518005862 shutdown_connections 2024-09-21T07:05:18.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.868+0000 7fa3c7200640 1 -- 172.21.15.114:0/2518005862 wait complete. 2024-09-21T07:05:18.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.871+0000 7fa3c7200640 1 Processor -- start 2024-09-21T07:05:18.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.872+0000 7fa3c7200640 1 -- start start 2024-09-21T07:05:18.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.875+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 0x7fa3c8198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.876+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 0x7fa3c81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.876+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c819fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.876+0000 7fa3c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 0x7fa3c8198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3c7200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81139b0 con 0x7fa3c8105080 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3c7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8113830 con 0x7fa3c8104650 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3c7200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8113b30 con 0x7fa3c810dc00 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3c5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 0x7fa3c8198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49616/0 (socket says 172.21.15.114:49616) 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 0x7fa3c81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3c6800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c819fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 0x7fa3c81993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46126/0 (socket says 172.21.15.114:46126) 2024-09-21T07:05:18.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.877+0000 7fa3bd400640 1 -- 172.21.15.114:0/240218471 learned_addr learned my addr 172.21.15.114:0/240218471 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:18.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.878+0000 7fa3c6800640 1 -- 172.21.15.114:0/240218471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 msgr2=0x7fa3c8198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.878+0000 7fa3c6800640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 0x7fa3c8198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.878+0000 7fa3c6800640 1 -- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 msgr2=0x7fa3c81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.878+0000 7fa3c6800640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 0x7fa3c81993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.878+0000 7fa3c6800640 1 -- 172.21.15.114:0/240218471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c8199c90 con 0x7fa3c810dc00 2024-09-21T07:05:18.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.879+0000 7fa3c6800640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c819fee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa3c002adf0 tx=0x7fa3c0031a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:18.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.879+0000 7fa3bf400640 1 -- 172.21.15.114:0/240218471 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0003db0 con 0x7fa3c810dc00 2024-09-21T07:05:18.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.880+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81a04e0 con 0x7fa3c810dc00 2024-09-21T07:05:18.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.880+0000 7fa3bf400640 1 -- 172.21.15.114:0/240218471 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c00044f0 con 0x7fa3c810dc00 2024-09-21T07:05:18.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.880+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c81a0ac0 con 0x7fa3c810dc00 2024-09-21T07:05:18.884 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.882+0000 7fa3bf400640 1 -- 172.21.15.114:0/240218471 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c002fca0 con 0x7fa3c810dc00 2024-09-21T07:05:18.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.890+0000 7fa3d3e84740 1 Processor -- start 2024-09-21T07:05:18.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.890+0000 7fa3bf400640 1 -- 172.21.15.114:0/240218471 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa3c00305d0 con 0x7fa3c810dc00 2024-09-21T07:05:18.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.893+0000 7fa3d3e84740 1 -- start start 2024-09-21T07:05:18.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.894+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.894+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d11b0 0x5643108a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.894+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x5643108a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.894+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564310817fb0 con 0x5643104fa070 2024-09-21T07:05:18.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.894+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643108073e0 con 0x5643104d11b0 2024-09-21T07:05:18.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.894+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56431080b1f0 con 0x5643104d9390 2024-09-21T07:05:18.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.895+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.895+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46140/0 (socket says 172.21.15.114:46140) 2024-09-21T07:05:18.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.895+0000 7fa3bd400640 1 -- 172.21.15.114:0/3072854342 learned_addr learned my addr 172.21.15.114:0/3072854342 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:18.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.895+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x5643108a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.895+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d11b0 0x5643108a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bd400640 1 -- 172.21.15.114:0/3072854342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d11b0 msgr2=0x5643108a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d11b0 0x5643108a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bd400640 1 -- 172.21.15.114:0/3072854342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 msgr2=0x5643108a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bd400640 1 -- 172.21.15.114:0/3072854342 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643107d1cd0 con 0x5643104d9390 2024-09-21T07:05:18.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bf400640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3a80b6700 0x7fa3a80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.896+0000 7fa3bf400640 1 -- 172.21.15.114:0/240218471 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fa3c0109000 con 0x7fa3c810dc00 2024-09-21T07:05:18.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.897+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x5643108a17b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00177b0 tx=0x7fa3b0009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f0fcfdbd1f1742c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:18.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.898+0000 7fa3b7400640 1 -- 172.21.15.114:0/3072854342 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0015760 con 0x5643104d9390 2024-09-21T07:05:18.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.898+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3a80b6700 0x7fa3a80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.898+0000 7fa3b7400640 1 -- 172.21.15.114:0/3072854342 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b0015db0 con 0x5643104d9390 2024-09-21T07:05:18.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.898+0000 7fa3b7400640 1 -- 172.21.15.114:0/3072854342 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0018a70 con 0x5643104d9390 2024-09-21T07:05:18.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.899+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3072854342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 msgr2=0x5643108a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.899+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x5643108a17b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa3b00177b0 tx=0x7fa3b0009190 comp rx=0 tx=0).stop 2024-09-21T07:05:18.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.900+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3a80b6700 0x7fa3a80b8c10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x56431089f9b0 tx=0x7fa3b8008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:18.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.901+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3072854342 shutdown_connections 2024-09-21T07:05:18.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.901+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x5643108a17b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.901+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d11b0 0x5643108a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.901+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3072854342 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.901+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3072854342 >> 172.21.15.114:0/3072854342 conn(0x5643104ee460 msgr2=0x56431089ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:18.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.903+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3072854342 shutdown_connections 2024-09-21T07:05:18.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.903+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3072854342 wait complete. 2024-09-21T07:05:18.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.903+0000 7fa3d3e84740 1 -- start start 2024-09-21T07:05:18.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.904+0000 7fa3d3e84740 1 Processor -- start 2024-09-21T07:05:18.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.910+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.910+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.910+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.910+0000 7fa3c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.910+0000 7fa3c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54924/0 (socket says 172.21.15.114:54924) 2024-09-21T07:05:18.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.911+0000 7fa3c5e00640 1 -- 172.21.15.114:0/3921413306 learned_addr learned my addr 172.21.15.114:0/3921413306 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:18.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.911+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564310668700 con 0x5643104fa070 2024-09-21T07:05:18.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.911+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643107a2580 con 0x5643104d9390 2024-09-21T07:05:18.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.911+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5643108af4b0 con 0x5643104d11b0 2024-09-21T07:05:18.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.911+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.911+0000 7fa3d3e84740 10 client.0 _ll_register_callbacks cb 0x5643107b2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.912+0000 7fa3bd400640 1 -- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 msgr2=0x5643108ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.912+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.912+0000 7fa3bd400640 1 -- 172.21.15.114:0/3921413306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 msgr2=0x5643108b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.912+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.912+0000 7fa3bd400640 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564310565650 con 0x5643104d9390 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.912+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:18.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.913+0000 7fa3bd400640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108acff0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009030 tx=0x7fa3b00186e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27498d20fe7b5a23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:18.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.913+0000 7fa3afe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:18.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.915+0000 7fa3afe00640 1 -- 172.21.15.114:0/3921413306 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0025020 con 0x5643104d9390 2024-09-21T07:05:18.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.915+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5643107a4b90 con 0x5643104d9390 2024-09-21T07:05:18.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.915+0000 7fa3afe00640 1 -- 172.21.15.114:0/3921413306 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fa3b0018c00 con 0x5643104d9390 2024-09-21T07:05:18.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.916+0000 7fa3afe00640 1 -- 172.21.15.114:0/3921413306 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa3b0017440 con 0x5643104d9390 2024-09-21T07:05:18.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.916+0000 7fa3d3e84740 10 client.28337 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:18.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.916+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5643107bf950 con 0x5643104d9390 2024-09-21T07:05:18.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3afe00640 1 -- 172.21.15.114:0/3921413306 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5643107a4b90 con 0x5643104d9390 2024-09-21T07:05:18.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3afe00640 1 -- 172.21.15.114:0/3921413306 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fa3b0021440 con 0x5643104d9390 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3d3e84740 10 client.28337 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3d3e84740 10 client.28337 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3d3e84740 20 client.28337 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5643104b89f0 0x5643108be780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3d3e84740 4 client.28337 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564310819f00 con 0x5643104b89f0 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.919+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5643104b89f0 0x5643108be780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.920+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5643104b89f0 0x5643108be780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9dc12f63f51a4c44 server_cookie=fbeebcecfdb3b810 in_seq=0 out_seq=0 2024-09-21T07:05:18.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.920+0000 7fa3afe00640 10 client.28337 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:05:18.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.921+0000 7fa3afe00640 1 -- 172.21.15.114:0/3921413306 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564310819f00 con 0x5643104b89f0 2024-09-21T07:05:18.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.921+0000 7fa3afe00640 10 client.28337 handle_command_reply: tid=0 2024-09-21T07:05:18.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.922+0000 7fa3d3e84740 1 client.28337 shutdown 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 msgr2=0x5643108acff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108acff0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009030 tx=0x7fa3b00186e0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 shutdown_connections 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x5643104b89f0 0x5643108be780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x5643108b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108acff0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/3921413306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3afe00640 0 client.28337 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:05:18.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.927+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 >> 172.21.15.114:0/3921413306 conn(0x5643104ee460 msgr2=0x5643108a3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:18.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.929+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 shutdown_connections 2024-09-21T07:05:18.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.929+0000 7fa3d3e84740 1 -- 172.21.15.114:0/3921413306 wait complete. 2024-09-21T07:05:18.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.929+0000 7fa3d3e84740 20 client.28337 trim_cache size 0 max 16384 2024-09-21T07:05:18.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.962+0000 7fa3d3e84740 1 Processor -- start 2024-09-21T07:05:18.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.963+0000 7fa3d3e84740 1 -- start start 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.964+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104d11b0 0x5643108ab780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.964+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x56431089e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.964+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104fa070 0x56431089e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.964+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81bfd30 con 0x5643104d11b0 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.964+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81c1870 con 0x5643104fa070 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.964+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81c0400 con 0x5643104d9390 2024-09-21T07:05:18.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.965+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x56431089e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.965+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x56431089e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54938/0 (socket says 172.21.15.114:54938) 2024-09-21T07:05:18.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.965+0000 7fa3bd400640 1 -- 172.21.15.114:0/127927161 learned_addr learned my addr 172.21.15.114:0/127927161 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:18.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.965+0000 7fa3c6800640 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104fa070 0x56431089e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.965+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104d11b0 0x5643108ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.966+0000 7fa3bd400640 1 -- 172.21.15.114:0/127927161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104fa070 msgr2=0x56431089e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.966+0000 7fa3bd400640 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104fa070 0x56431089e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.966+0000 7fa3bd400640 1 -- 172.21.15.114:0/127927161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104d11b0 msgr2=0x5643108ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.966+0000 7fa3bd400640 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104d11b0 0x5643108ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.966+0000 7fa3bd400640 1 -- 172.21.15.114:0/127927161 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643108af3f0 con 0x5643104d9390 2024-09-21T07:05:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.969+0000 7fa3c6800640 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104fa070 0x56431089e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.969+0000 7fa3bd400640 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x56431089e210 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009e20 tx=0x7fa3b00187d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e30b680177888ff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:18.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.969+0000 7fa3b6a00640 1 -- 172.21.15.114:0/127927161 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0021440 con 0x5643104d9390 2024-09-21T07:05:18.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.970+0000 7fa3b6a00640 1 -- 172.21.15.114:0/127927161 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b00215e0 con 0x5643104d9390 2024-09-21T07:05:18.974 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.972+0000 7fa3b6a00640 1 -- 172.21.15.114:0/127927161 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0005dd0 con 0x5643104d9390 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 -- 172.21.15.114:0/127927161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 msgr2=0x56431089e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x56431089e210 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009e20 tx=0x7fa3b00187d0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 -- 172.21.15.114:0/127927161 shutdown_connections 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104fa070 0x56431089e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d9390 0x56431089e210 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/127927161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104d11b0 0x5643108ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 -- 172.21.15.114:0/127927161 >> 172.21.15.114:0/127927161 conn(0x5643104ee460 msgr2=0x5643108ab070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 -- 172.21.15.114:0/127927161 shutdown_connections 2024-09-21T07:05:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.974+0000 7fa3d3e84740 1 -- 172.21.15.114:0/127927161 wait complete. 2024-09-21T07:05:18.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.975+0000 7fa3d3e84740 1 -- start start 2024-09-21T07:05:18.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.975+0000 7fa3d3e84740 1 Processor -- start 2024-09-21T07:05:18.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x564310927f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81c11a0 con 0x5643104fa070 2024-09-21T07:05:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81c0ad0 con 0x5643104d9390 2024-09-21T07:05:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81bdcd0 con 0x5643104d11b0 2024-09-21T07:05:18.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.976+0000 7fa3d3e84740 10 client.0 _ll_register_callbacks cb 0x7fa3b0017680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:18.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.978+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.979+0000 7fa3bd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108bffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49630/0 (socket says 172.21.15.114:49630) 2024-09-21T07:05:18.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.979+0000 7fa3bd400640 1 -- 172.21.15.114:0/2038596712 learned_addr learned my addr 172.21.15.114:0/2038596712 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:18.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.978+0000 7fa3c5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.979+0000 7fa3c6800640 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x564310927f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.980+0000 7fa3c5e00640 1 -- 172.21.15.114:0/2038596712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 msgr2=0x5643108bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.980+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108bffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.980+0000 7fa3c5e00640 1 -- 172.21.15.114:0/2038596712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 msgr2=0x564310927f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:18.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.980+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x564310927f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.980+0000 7fa3c5e00640 1 -- 172.21.15.114:0/2038596712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643108c0800 con 0x5643104d11b0 2024-09-21T07:05:18.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.982+0000 7fa3c5e00640 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108bfa70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x5643106687d0 tx=0x7fa3b80047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8053ff5c141d700 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:18.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.982+0000 7fa3afe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:18.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.983+0000 7fa3afe00640 1 -- 172.21.15.114:0/2038596712 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801c040 con 0x5643104d11b0 2024-09-21T07:05:18.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.984+0000 7fa3afe00640 1 -- 172.21.15.114:0/2038596712 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fa3b8023640 con 0x5643104d11b0 2024-09-21T07:05:18.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.984+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b00215e0 con 0x5643104d11b0 2024-09-21T07:05:18.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.985+0000 7fa3afe00640 1 -- 172.21.15.114:0/2038596712 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa3b8023840 con 0x5643104d11b0 2024-09-21T07:05:18.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.985+0000 7fa3d3e84740 10 client.28285 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:18.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.985+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5643109294e0 con 0x5643104d11b0 2024-09-21T07:05:18.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.987+0000 7fa3afe00640 1 -- 172.21.15.114:0/2038596712 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b00215e0 con 0x5643104d11b0 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.987+0000 7fa3afe00640 1 -- 172.21.15.114:0/2038596712 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fa3b8014b40 con 0x5643104d11b0 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.987+0000 7fa3d3e84740 10 client.28285 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.987+0000 7fa3d3e84740 10 client.28285 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.987+0000 7fa3d3e84740 20 client.28285 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.987+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56431048dda0 0x56431092bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.988+0000 7fa3d3e84740 4 client.28285 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.988+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643108159b0 con 0x56431048dda0 2024-09-21T07:05:18.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.988+0000 7fa3bd400640 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56431048dda0 0x56431092bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:18.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.988+0000 7fa3afe00640 10 client.28285 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:05:18.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.988+0000 7fa3bd400640 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56431048dda0 0x56431092bc80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd5fe2cd4eca8df0 server_cookie=32f07b98c8c3bcc7 in_seq=0 out_seq=0 2024-09-21T07:05:18.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.989+0000 7fa3afe00640 1 -- 172.21.15.114:0/2038596712 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643108159b0 con 0x56431048dda0 2024-09-21T07:05:18.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.989+0000 7fa3afe00640 10 client.28285 handle_command_reply: tid=0 2024-09-21T07:05:18.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.989+0000 7fa3d3e84740 1 client.28285 shutdown 2024-09-21T07:05:18.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 msgr2=0x5643108bfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108bfa70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x5643106687d0 tx=0x7fa3b80047d0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 shutdown_connections 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x56431048dda0 0x56431092bc80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643104fa070 0x564310927f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643104d9390 0x5643108bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 --2- 172.21.15.114:0/2038596712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643104d11b0 0x5643108bfa70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 >> 172.21.15.114:0/2038596712 conn(0x5643104ee460 msgr2=0x5643108a1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:18.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.990+0000 7fa3afe00640 0 client.28285 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:05:18.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.991+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 shutdown_connections 2024-09-21T07:05:18.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.991+0000 7fa3d3e84740 1 -- 172.21.15.114:0/2038596712 wait complete. 2024-09-21T07:05:18.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.991+0000 7fa3d3e84740 20 client.28285 trim_cache size 0 max 16384 2024-09-21T07:05:18.995 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:05:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.994+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3a80b6700 msgr2=0x7fa3a80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.994+0000 7fa3c7200640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3a80b6700 0x7fa3a80b8c10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x56431089f9b0 tx=0x7fa3b8008040 comp rx=0 tx=0).stop 2024-09-21T07:05:18.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.994+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 msgr2=0x7fa3c819fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.994+0000 7fa3c7200640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c819fee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa3c002adf0 tx=0x7fa3c0031a00 comp rx=0 tx=0).stop 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.995+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 shutdown_connections 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.995+0000 7fa3c7200640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3c810dc00 0x7fa3c819fee0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.995+0000 7fa3c7200640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa3c8105080 0x7fa3c81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.995+0000 7fa3c7200640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa3a80b6700 0x7fa3a80b8c10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.995+0000 7fa3c7200640 1 --2- 172.21.15.114:0/240218471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3c8104650 0x7fa3c8198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:18.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.995+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 >> 172.21.15.114:0/240218471 conn(0x7fa3c80fe2c0 msgr2=0x7fa3c8102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:18.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.998+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 shutdown_connections 2024-09-21T07:05:19.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:18.998+0000 7fa3c7200640 1 -- 172.21.15.114:0/240218471 wait complete. 2024-09-21T07:05:19.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:19 smithi193 ceph-mon[34601]: pgmap v739: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.9 MiB/s wr, 460 op/s 2024-09-21T07:05:19.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:19 smithi114 ceph-mon[30654]: pgmap v739: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.9 MiB/s wr, 460 op/s 2024-09-21T07:05:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:19 smithi183 ceph-mon[34219]: pgmap v739: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 7.9 MiB/s wr, 460 op/s 2024-09-21T07:05:20.569 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:20 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:20.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:20 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:20.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:20 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:05:21.151 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:05:21.470 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:21 smithi114 ceph-mon[30654]: pgmap v740: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.7 MiB/s wr, 462 op/s 2024-09-21T07:05:21.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:21 smithi183 ceph-mon[34219]: pgmap v740: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.7 MiB/s wr, 462 op/s 2024-09-21T07:05:21.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:21 smithi193 ceph-mon[34601]: pgmap v740: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 7.7 MiB/s wr, 462 op/s 2024-09-21T07:05:21.845 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:22.415 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:22 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:22.415 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:22 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:22.415 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:22 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:05:22.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:22 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:22.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:22 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:22.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:22 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:05:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:22 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:05:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:22 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:05:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:22 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:05:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 645.0 s, 440.2 tps, lat 219.706 ms stddev 303.293 2024-09-21T07:05:22.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 -- 172.21.15.114:0/4262220856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d8113eb0 msgr2=0x7f04d81142c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 --2- 172.21.15.114:0/4262220856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d8113eb0 0x7f04d81142c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009900 tx=0x7f04c002a410 comp rx=0 tx=0).stop 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 -- 172.21.15.114:0/4262220856 shutdown_connections 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 --2- 172.21.15.114:0/4262220856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04d8114800 0x7f04d811b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 --2- 172.21.15.114:0/4262220856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d8113eb0 0x7f04d81142c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 --2- 172.21.15.114:0/4262220856 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d810e5c0 0x7f04d810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 -- 172.21.15.114:0/4262220856 >> 172.21.15.114:0/4262220856 conn(0x7f04d810a3c0 msgr2=0x7f04d810c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 -- 172.21.15.114:0/4262220856 shutdown_connections 2024-09-21T07:05:22.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.831+0000 7f04dd800640 1 -- 172.21.15.114:0/4262220856 wait complete. 2024-09-21T07:05:22.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 Processor -- start 2024-09-21T07:05:22.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 -- start start 2024-09-21T07:05:22.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 0x7f04d81184b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04d8113eb0 0x7f04d81189f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d8114800 0x7f04d8118f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f04d811e490 con 0x7f04d8114800 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04d811e310 con 0x7f04d8113eb0 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.833+0000 7f04dd800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04d811e610 con 0x7f04d810e5c0 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.834+0000 7f04d6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 0x7f04d81184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.834+0000 7f04d6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 0x7f04d81184b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54952/0 (socket says 172.21.15.114:54952) 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.834+0000 7f04d6a00640 1 -- 172.21.15.114:0/2403952783 learned_addr learned my addr 172.21.15.114:0/2403952783 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.834+0000 7f04cfe00640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04d8113eb0 0x7f04d81189f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d7400640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d8114800 0x7f04d8118f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d6a00640 1 -- 172.21.15.114:0/2403952783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04d8113eb0 msgr2=0x7f04d81189f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04d8113eb0 0x7f04d81189f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d6a00640 1 -- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d8114800 msgr2=0x7f04d8118f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d8114800 0x7f04d8118f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d6a00640 1 -- 172.21.15.114:0/2403952783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d81b5330 con 0x7f04d810e5c0 2024-09-21T07:05:22.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.835+0000 7f04d7400640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d8114800 0x7f04d8118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:22.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.836+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 0x7f04d81184b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e9f0 tx=0x7f04c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:22.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.837+0000 7f04cf400640 1 -- 172.21.15.114:0/2403952783 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800cd80 con 0x7f04d810e5c0 2024-09-21T07:05:22.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.837+0000 7f04cf400640 1 -- 172.21.15.114:0/2403952783 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c80108e0 con 0x7f04d810e5c0 2024-09-21T07:05:22.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.837+0000 7f04cf400640 1 -- 172.21.15.114:0/2403952783 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8018980 con 0x7f04d810e5c0 2024-09-21T07:05:22.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.837+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d81b5620 con 0x7f04d810e5c0 2024-09-21T07:05:22.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.838+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d81b5ae0 con 0x7f04d810e5c0 2024-09-21T07:05:22.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.842+0000 7f04cf400640 1 -- 172.21.15.114:0/2403952783 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f04c8010430 con 0x7f04d810e5c0 2024-09-21T07:05:22.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.844+0000 7f04e45d6740 1 Processor -- start 2024-09-21T07:05:22.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.844+0000 7f04e45d6740 1 -- start start 2024-09-21T07:05:22.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.844+0000 7f04cf400640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04b80b6700 0x7f04b80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.844+0000 7f04cf400640 1 -- 172.21.15.114:0/2403952783 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f04c80e8240 con 0x7f04d810e5c0 2024-09-21T07:05:22.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.844+0000 7f04d7400640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04b80b6700 0x7f04b80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 0x55e4b8635ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8636620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04e45d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e4b85acfb0 con 0x55e4b828f070 2024-09-21T07:05:22.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04e45d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4b859c3e0 con 0x55e4b82661b0 2024-09-21T07:05:22.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04e45d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4b85a01f0 con 0x55e4b826e390 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.845+0000 7f04d7400640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04b80b6700 0x7f04b80b8c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005fd0 tx=0x7f04d0009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 0x55e4b8635ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 0x55e4b8635ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46200/0 (socket says 172.21.15.114:46200) 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 -- 172.21.15.114:0/2149824684 learned_addr learned my addr 172.21.15.114:0/2149824684 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04cfe00640 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 -- 172.21.15.114:0/2149824684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 msgr2=0x55e4b8636620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8636620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 -- 172.21.15.114:0/2149824684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 msgr2=0x55e4b86360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04d6a00640 1 -- 172.21.15.114:0/2149824684 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b8566cd0 con 0x55e4b828f070 2024-09-21T07:05:22.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.846+0000 7f04cfe00640 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:22.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.847+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 0x55e4b8635ba0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04c80ad9c0 tx=0x7f04c8010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96bc8dd20f4e873 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:22.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.848+0000 7f04c7e00640 1 -- 172.21.15.114:0/2149824684 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80950b0 con 0x55e4b828f070 2024-09-21T07:05:22.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.849+0000 7f04c7e00640 1 -- 172.21.15.114:0/2149824684 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c80ae3c0 con 0x55e4b828f070 2024-09-21T07:05:22.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.849+0000 7f04c7e00640 1 -- 172.21.15.114:0/2149824684 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80d10d0 con 0x55e4b828f070 2024-09-21T07:05:22.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.850+0000 7f04e45d6740 1 -- 172.21.15.114:0/2149824684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 msgr2=0x55e4b8635ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.850+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 0x55e4b8635ba0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f04c80ad9c0 tx=0x7f04c8010720 comp rx=0 tx=0).stop 2024-09-21T07:05:22.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.851+0000 7f04e45d6740 1 -- 172.21.15.114:0/2149824684 shutdown_connections 2024-09-21T07:05:22.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.852+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8636620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.852+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.852+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2149824684 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b828f070 0x55e4b8635ba0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.852+0000 7f04e45d6740 1 -- 172.21.15.114:0/2149824684 >> 172.21.15.114:0/2149824684 conn(0x55e4b8283460 msgr2=0x55e4b8633a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:22.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.853+0000 7f04e45d6740 1 -- 172.21.15.114:0/2149824684 shutdown_connections 2024-09-21T07:05:22.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.853+0000 7f04e45d6740 1 -- 172.21.15.114:0/2149824684 wait complete. 2024-09-21T07:05:22.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.855+0000 7f04e45d6740 1 -- start start 2024-09-21T07:05:22.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.855+0000 7f04e45d6740 1 Processor -- start 2024-09-21T07:05:22.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.857+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b82661b0 0x55e4b8641830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.857+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8641e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.857+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b828f070 0x55e4b8647630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.857+0000 7f04e45d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e4b83fd700 con 0x55e4b82661b0 2024-09-21T07:05:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.857+0000 7f04e45d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e4b8537580 con 0x55e4b828f070 2024-09-21T07:05:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.857+0000 7f04e45d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e4b8644320 con 0x55e4b826e390 2024-09-21T07:05:22.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.858+0000 7f04e45d6740 10 client.0 _ll_register_callbacks cb 0x55e4b8547470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.858+0000 7f04d7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b828f070 0x55e4b8647630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.858+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8641e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.858+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8641e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54970/0 (socket says 172.21.15.114:54970) 2024-09-21T07:05:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.859+0000 7f04cfe00640 1 -- 172.21.15.114:0/3709602431 learned_addr learned my addr 172.21.15.114:0/3709602431 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:22.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.859+0000 7f04d6a00640 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b82661b0 0x55e4b8641830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.860+0000 7f04d7400640 1 -- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 msgr2=0x55e4b8641e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.860+0000 7f04d7400640 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8641e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.860+0000 7f04d7400640 1 -- 172.21.15.114:0/3709602431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b82661b0 msgr2=0x55e4b8641830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.860+0000 7f04d7400640 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b82661b0 0x55e4b8641830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.860+0000 7f04d7400640 1 -- 172.21.15.114:0/3709602431 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b82fa650 con 0x55e4b828f070 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.860+0000 7f04d7400640 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b828f070 0x55e4b8647630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f04d00116c0 tx=0x7f04d0011ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5927e20eae366763 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.861+0000 7f04c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.861+0000 7f04c4c00640 1 -- 172.21.15.114:0/3709602431 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0022070 con 0x55e4b828f070 2024-09-21T07:05:22.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.861+0000 7f04c4c00640 1 -- 172.21.15.114:0/3709602431 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f04d0010a90 con 0x55e4b828f070 2024-09-21T07:05:22.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.861+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4b8539b90 con 0x55e4b828f070 2024-09-21T07:05:22.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.862+0000 7f04c4c00640 1 -- 172.21.15.114:0/3709602431 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f04d002e050 con 0x55e4b828f070 2024-09-21T07:05:22.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.862+0000 7f04e45d6740 10 client.28352 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.862+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e4b8554950 con 0x55e4b828f070 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.862+0000 7f04c4c00640 1 -- 172.21.15.114:0/3709602431 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e4b8539b90 con 0x55e4b828f070 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.863+0000 7f04c4c00640 1 -- 172.21.15.114:0/3709602431 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f04d00263b0 con 0x55e4b828f070 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04e45d6740 10 client.28352 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04e45d6740 10 client.28352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04e45d6740 20 client.28352 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:22.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04e45d6740 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b824d9f0 0x55e4b8653740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04e45d6740 4 client.28352 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:22.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4b85aef00 con 0x55e4b824d9f0 2024-09-21T07:05:22.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.864+0000 7f04d6a00640 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b824d9f0 0x55e4b8653740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.865+0000 7f04d6a00640 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b824d9f0 0x55e4b8653740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3b0e5e4b6ca0cd4 server_cookie=ea802392db42442b in_seq=0 out_seq=0 2024-09-21T07:05:22.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.865+0000 7f04c4c00640 10 client.28352 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:05:22.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.866+0000 7f04c4c00640 1 -- 172.21.15.114:0/3709602431 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4b85aef00 con 0x55e4b824d9f0 2024-09-21T07:05:22.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.866+0000 7f04c4c00640 10 client.28352 handle_command_reply: tid=0 2024-09-21T07:05:22.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.867+0000 7f04e45d6740 1 client.28352 shutdown 2024-09-21T07:05:22.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.867+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b828f070 msgr2=0x55e4b8647630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.867+0000 7f04e45d6740 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b828f070 0x55e4b8647630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f04d00116c0 tx=0x7f04d0011ac0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.867+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 shutdown_connections 2024-09-21T07:05:22.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.868+0000 7f04e45d6740 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b824d9f0 0x55e4b8653740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.868+0000 7f04c4c00640 0 client.28352 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:05:22.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.868+0000 7f04e45d6740 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b828f070 0x55e4b8647630 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.868+0000 7f04e45d6740 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b8641e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.868+0000 7f04e45d6740 1 --2- 172.21.15.114:0/3709602431 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b82661b0 0x55e4b8641830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.868+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 >> 172.21.15.114:0/3709602431 conn(0x55e4b8283460 msgr2=0x55e4b8638d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:22.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.869+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 shutdown_connections 2024-09-21T07:05:22.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.869+0000 7f04e45d6740 1 -- 172.21.15.114:0/3709602431 wait complete. 2024-09-21T07:05:22.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.869+0000 7f04e45d6740 20 client.28352 trim_cache size 0 max 16384 2024-09-21T07:05:22.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 Processor -- start 2024-09-21T07:05:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 -- start start 2024-09-21T07:05:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b824d9f0 0x55e4b863b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b826e390 0x55e4b86338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81bfea0 con 0x55e4b826e390 2024-09-21T07:05:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81c19e0 con 0x55e4b82661b0 2024-09-21T07:05:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.890+0000 7f04e45d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81c0570 con 0x55e4b824d9f0 2024-09-21T07:05:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.891+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.891+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86333b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49674/0 (socket says 172.21.15.114:49674) 2024-09-21T07:05:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.891+0000 7f04d6a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b824d9f0 0x55e4b863b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.893 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.892+0000 7f04cfe00640 1 -- 172.21.15.114:0/2198958079 learned_addr learned my addr 172.21.15.114:0/2198958079 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.893+0000 7f04cfe00640 1 -- 172.21.15.114:0/2198958079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b824d9f0 msgr2=0x55e4b863b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.893+0000 7f04cfe00640 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b824d9f0 0x55e4b863b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.893+0000 7f04cfe00640 1 -- 172.21.15.114:0/2198958079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b826e390 msgr2=0x55e4b86338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.893+0000 7f04d7400640 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b826e390 0x55e4b86338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.893+0000 7f04cfe00640 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b826e390 0x55e4b86338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.894+0000 7f04cfe00640 1 -- 172.21.15.114:0/2198958079 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4b8638160 con 0x55e4b82661b0 2024-09-21T07:05:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.894+0000 7f04d6a00640 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b824d9f0 0x55e4b863b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.894+0000 7f04d7400640 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b826e390 0x55e4b86338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.894+0000 7f04cfe00640 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86333b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f04c0032df0 tx=0x7f04c0032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490a26894ea8901e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:22.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.895+0000 7f04c7400640 1 -- 172.21.15.114:0/2198958079 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0035040 con 0x55e4b82661b0 2024-09-21T07:05:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.895+0000 7f04c7400640 1 -- 172.21.15.114:0/2198958079 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c00316a0 con 0x55e4b82661b0 2024-09-21T07:05:22.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.895+0000 7f04c7400640 1 -- 172.21.15.114:0/2198958079 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0004750 con 0x55e4b82661b0 2024-09-21T07:05:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 -- 172.21.15.114:0/2198958079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 msgr2=0x55e4b86333b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86333b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f04c0032df0 tx=0x7f04c0032e20 comp rx=0 tx=0).stop 2024-09-21T07:05:22.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 -- 172.21.15.114:0/2198958079 shutdown_connections 2024-09-21T07:05:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b826e390 0x55e4b86338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b86333b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 --2- 172.21.15.114:0/2198958079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b824d9f0 0x55e4b863b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.896+0000 7f04e45d6740 1 -- 172.21.15.114:0/2198958079 >> 172.21.15.114:0/2198958079 conn(0x55e4b8283460 msgr2=0x55e4b863acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.897+0000 7f04e45d6740 1 -- 172.21.15.114:0/2198958079 shutdown_connections 2024-09-21T07:05:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.897+0000 7f04e45d6740 1 -- 172.21.15.114:0/2198958079 wait complete. 2024-09-21T07:05:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.897+0000 7f04e45d6740 1 -- start start 2024-09-21T07:05:22.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.897+0000 7f04e45d6740 1 Processor -- start 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b824d9f0 0x55e4b864d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b864d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b864c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81c1310 con 0x55e4b824d9f0 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81c0c40 con 0x55e4b82661b0 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f04d81bde60 con 0x55e4b826e390 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04e45d6740 10 client.0 _ll_register_callbacks cb 0x7f04d0009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b864d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b864d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:49682/0 (socket says 172.21.15.114:49682) 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.898+0000 7f04cfe00640 1 -- 172.21.15.114:0/568988004 learned_addr learned my addr 172.21.15.114:0/568988004 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:22.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04d7400640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b864c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04d6a00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b824d9f0 0x55e4b864d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04cfe00640 1 -- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 msgr2=0x55e4b864c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04cfe00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b864c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04cfe00640 1 -- 172.21.15.114:0/568988004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b824d9f0 msgr2=0x55e4b864d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04cfe00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b824d9f0 0x55e4b864d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04cfe00640 1 -- 172.21.15.114:0/568988004 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4b8643d00 con 0x55e4b82661b0 2024-09-21T07:05:22.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.899+0000 7f04d6a00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b824d9f0 0x55e4b864d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04cfe00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b864d940 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f04c0005f70 tx=0x7f04c0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20009c56902e18a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04c4c00640 1 -- 172.21.15.114:0/568988004 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0035070 con 0x55e4b82661b0 2024-09-21T07:05:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04c4c00640 1 -- 172.21.15.114:0/568988004 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f04c0031d10 con 0x55e4b82661b0 2024-09-21T07:05:22.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c00316a0 con 0x55e4b82661b0 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04c4c00640 1 -- 172.21.15.114:0/568988004 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f04c0031910 con 0x55e4b82661b0 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04e45d6740 10 client.28358 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.900+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e4b86cf450 con 0x55e4b82661b0 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04c4c00640 1 -- 172.21.15.114:0/568988004 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c00316a0 con 0x55e4b82661b0 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04c4c00640 1 -- 172.21.15.114:0/568988004 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f04c0003b90 con 0x55e4b82661b0 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04e45d6740 10 client.28358 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04e45d6740 10 client.28358 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04e45d6740 20 client.28358 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04e45d6740 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b828f070 0x55e4b86d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04e45d6740 4 client.28358 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:05:22.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.901+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4b85aa9b0 con 0x55e4b828f070 2024-09-21T07:05:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.903+0000 7f04d6a00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b828f070 0x55e4b86d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.903+0000 7f04d6a00640 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b828f070 0x55e4b86d1bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fd2eb36c156ddd9 server_cookie=f1bcbccd752f8600 in_seq=0 out_seq=0 2024-09-21T07:05:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.903+0000 7f04c4c00640 10 client.28358 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:05:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.904+0000 7f04c4c00640 1 -- 172.21.15.114:0/568988004 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4b85aa9b0 con 0x55e4b828f070 2024-09-21T07:05:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.904+0000 7f04c4c00640 10 client.28358 handle_command_reply: tid=0 2024-09-21T07:05:22.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.904+0000 7f04e45d6740 1 client.28358 shutdown 2024-09-21T07:05:22.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.904+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 msgr2=0x55e4b864d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.904+0000 7f04e45d6740 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b864d940 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f04c0005f70 tx=0x7f04c0031400 comp rx=0 tx=0).stop 2024-09-21T07:05:22.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 shutdown_connections 2024-09-21T07:05:22.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e4b828f070 0x55e4b86d1bb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04c4c00640 0 client.28358 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:05:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e4b826e390 0x55e4b864c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e4b82661b0 0x55e4b864d940 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 --2- 172.21.15.114:0/568988004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e4b824d9f0 0x55e4b864d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 >> 172.21.15.114:0/568988004 conn(0x55e4b8283460 msgr2=0x55e4b86332c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:22.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 shutdown_connections 2024-09-21T07:05:22.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.905+0000 7f04e45d6740 1 -- 172.21.15.114:0/568988004 wait complete. 2024-09-21T07:05:22.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.906+0000 7f04e45d6740 20 client.28358 trim_cache size 0 max 16384 2024-09-21T07:05:22.908 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:05:22.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04b80b6700 msgr2=0x7f04b80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04b80b6700 0x7f04b80b8c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005fd0 tx=0x7f04d0009500 comp rx=0 tx=0).stop 2024-09-21T07:05:22.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 msgr2=0x7f04d81184b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 0x7f04d81184b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f04c800e9f0 tx=0x7f04c800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 shutdown_connections 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f04d8114800 0x7f04d8118f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f04b80b6700 0x7f04b80b8c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04d8113eb0 0x7f04d81189f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 --2- 172.21.15.114:0/2403952783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d810e5c0 0x7f04d81184b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:22.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.907+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 >> 172.21.15.114:0/2403952783 conn(0x7f04d810a3c0 msgr2=0x7f04d81134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:22.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.908+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 shutdown_connections 2024-09-21T07:05:22.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:22.908+0000 7f04dd800640 1 -- 172.21.15.114:0/2403952783 wait complete. 2024-09-21T07:05:23.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:23 smithi114 ceph-mon[30654]: pgmap v741: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.5 MiB/s wr, 388 op/s 2024-09-21T07:05:23.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:23 smithi183 ceph-mon[34219]: pgmap v741: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.5 MiB/s wr, 388 op/s 2024-09-21T07:05:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:23 smithi193 ceph-mon[34601]: pgmap v741: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.5 MiB/s wr, 388 op/s 2024-09-21T07:05:25.319 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:05:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 -- 172.21.15.114:0/2256066997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120111ca0 msgr2=0x7f712010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 --2- 172.21.15.114:0/2256066997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120111ca0 0x7f712010a3c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7118009ca0 tx=0x7f711802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 -- 172.21.15.114:0/2256066997 shutdown_connections 2024-09-21T07:05:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 --2- 172.21.15.114:0/2256066997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712011cba0 0x7f712011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 --2- 172.21.15.114:0/2256066997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f712010a900 0x7f712010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 --2- 172.21.15.114:0/2256066997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120111ca0 0x7f712010a3c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 -- 172.21.15.114:0/2256066997 >> 172.21.15.114:0/2256066997 conn(0x7f7120070c00 msgr2=0x7f7120071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 -- 172.21.15.114:0/2256066997 shutdown_connections 2024-09-21T07:05:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.480+0000 7f7125400640 1 -- 172.21.15.114:0/2256066997 wait complete. 2024-09-21T07:05:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.484+0000 7f7125400640 1 Processor -- start 2024-09-21T07:05:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.485+0000 7f7125400640 1 -- start start 2024-09-21T07:05:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f7125400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 0x7f71201a9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f7125400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7120111ca0 0x7f71201a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f7125400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f712011cba0 0x7f71201ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f7125400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7120122630 con 0x7f7120111ca0 2024-09-21T07:05:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f7125400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71201224b0 con 0x7f712011cba0 2024-09-21T07:05:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f7125400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71201227b0 con 0x7f712010a900 2024-09-21T07:05:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f711fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 0x7f71201a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f711fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 0x7f71201a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48930/0 (socket says 172.21.15.114:48930) 2024-09-21T07:05:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f711fe00640 1 -- 172.21.15.114:0/1252609072 learned_addr learned my addr 172.21.15.114:0/1252609072 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.486+0000 7f711f400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7120111ca0 0x7f71201a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.487+0000 7f711fe00640 1 -- 172.21.15.114:0/1252609072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f712011cba0 msgr2=0x7f71201ae8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:05:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.487+0000 7f711fe00640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f712011cba0 0x7f71201ae8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.487+0000 7f711fe00640 1 -- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7120111ca0 msgr2=0x7f71201a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.487+0000 7f711fe00640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7120111ca0 0x7f71201a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.487+0000 7f711fe00640 1 -- 172.21.15.114:0/1252609072 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71201aefa0 con 0x7f712010a900 2024-09-21T07:05:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.488+0000 7f711fe00640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 0x7f71201a9070 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f711802ade0 tx=0x7f7118031770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.488+0000 7f711cc00640 1 -- 172.21.15.114:0/1252609072 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118003b70 con 0x7f712010a900 2024-09-21T07:05:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.488+0000 7f711cc00640 1 -- 172.21.15.114:0/1252609072 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118003d10 con 0x7f712010a900 2024-09-21T07:05:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.489+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71201af230 con 0x7f712010a900 2024-09-21T07:05:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.489+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201b5a20 con 0x7f712010a900 2024-09-21T07:05:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.489+0000 7f711cc00640 1 -- 172.21.15.114:0/1252609072 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711802fa80 con 0x7f712010a900 2024-09-21T07:05:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.501+0000 7f711cc00640 1 -- 172.21.15.114:0/1252609072 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7118004580 con 0x7f712010a900 2024-09-21T07:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 Processor -- start 2024-09-21T07:05:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 -- start start 2024-09-21T07:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49afece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49afed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49afed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a49af626b0 con 0x55a49ac1f2f0 2024-09-21T07:05:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a49af51ae0 con 0x55a49ac274d0 2024-09-21T07:05:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.503+0000 7f712dc00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a49af558f0 con 0x55a49ac481b0 2024-09-21T07:05:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.504+0000 7f711f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49afed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.504+0000 7f711f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49afed890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54650/0 (socket says 172.21.15.114:54650) 2024-09-21T07:05:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.504+0000 7f711f400640 1 -- 172.21.15.114:0/3012806347 learned_addr learned my addr 172.21.15.114:0/3012806347 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.504+0000 7f7124a00640 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49afece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711fe00640 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49afed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711f400640 1 -- 172.21.15.114:0/3012806347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 msgr2=0x55a49afece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711f400640 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49afece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711cc00640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71040b6700 0x7f71040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711f400640 1 -- 172.21.15.114:0/3012806347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 msgr2=0x55a49afed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711f400640 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49afed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.505+0000 7f711f400640 1 -- 172.21.15.114:0/3012806347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49af1e780 con 0x55a49ac274d0 2024-09-21T07:05:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.506+0000 7f711cc00640 1 -- 172.21.15.114:0/1252609072 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7118108ce0 con 0x7f712010a900 2024-09-21T07:05:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.506+0000 7f711f400640 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49afed890 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f711000abe0 tx=0x7f711000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0b1f2f8a2da9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.507+0000 7f7115200640 1 -- 172.21.15.114:0/3012806347 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711001d070 con 0x55a49ac274d0 2024-09-21T07:05:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.507+0000 7f7124a00640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71040b6700 0x7f71040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.507+0000 7f7115200640 1 -- 172.21.15.114:0/3012806347 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110018af0 con 0x55a49ac274d0 2024-09-21T07:05:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.508+0000 7f7115200640 1 -- 172.21.15.114:0/3012806347 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010b00 con 0x55a49ac274d0 2024-09-21T07:05:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.508+0000 7f712dc00740 1 -- 172.21.15.114:0/3012806347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 msgr2=0x55a49afed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.508+0000 7f712dc00740 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49afed890 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f711000abe0 tx=0x7f711000f790 comp rx=0 tx=0).stop 2024-09-21T07:05:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.508+0000 7f7124a00640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71040b6700 0x7f71040b8c10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f710c009bf0 tx=0x7f710c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.510+0000 7f712dc00740 1 -- 172.21.15.114:0/3012806347 shutdown_connections 2024-09-21T07:05:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.511+0000 7f712dc00740 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49afed890 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.511+0000 7f712dc00740 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49afed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.511+0000 7f712dc00740 1 --2- 172.21.15.114:0/3012806347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49afece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.511+0000 7f712dc00740 1 -- 172.21.15.114:0/3012806347 >> 172.21.15.114:0/3012806347 conn(0x55a49ac3c300 msgr2=0x55a49afead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.511+0000 7f712dc00740 1 -- 172.21.15.114:0/3012806347 shutdown_connections 2024-09-21T07:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.511+0000 7f712dc00740 1 -- 172.21.15.114:0/3012806347 wait complete. 2024-09-21T07:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.512+0000 7f712dc00740 1 -- start start 2024-09-21T07:05:25.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.512+0000 7f712dc00740 1 Processor -- start 2024-09-21T07:05:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 0x55a49aff83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac274d0 0x55a49aff88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f711fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 0x55a49aff83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f711fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 0x55a49aff83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54664/0 (socket says 172.21.15.114:54664) 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49aff8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a49ae89ab0 con 0x55a49ac274d0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a49aef8570 con 0x55a49ac1f2f0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a49affb630 con 0x55a49ac481b0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.514+0000 7f712dc00740 10 client.0 _ll_register_callbacks cb 0x55a49af07100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49aff8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f711f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac274d0 0x55a49aff88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49aff8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48952/0 (socket says 172.21.15.114:48952) 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 -- 172.21.15.114:0/1911900310 learned_addr learned my addr 172.21.15.114:0/1911900310 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 -- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 msgr2=0x55a49aff83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 0x55a49aff83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 -- 172.21.15.114:0/1911900310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac274d0 msgr2=0x55a49aff88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac274d0 0x55a49aff88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 -- 172.21.15.114:0/1911900310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49acb3790 con 0x55a49ac481b0 2024-09-21T07:05:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f711f400640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac274d0 0x55a49aff88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.515+0000 7f7124a00640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49aff8e20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f710c005e00 tx=0x7f710c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6444dd4eb928d349 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:25.518 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.516+0000 7f711fe00640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 0x55a49aff83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.516+0000 7f7109600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.516+0000 7f7109600640 1 -- 172.21.15.114:0/1911900310 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c01e830 con 0x55a49ac481b0 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.516+0000 7f7109600640 1 -- 172.21.15.114:0/1911900310 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f710c01e9d0 con 0x55a49ac481b0 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.516+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110018af0 con 0x55a49ac481b0 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.517+0000 7f7109600640 1 -- 172.21.15.114:0/1911900310 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f710c01ebd0 con 0x55a49ac481b0 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.517+0000 7f712dc00740 10 client.28312 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.517+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a49ad2c620 con 0x55a49ac481b0 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f7109600640 1 -- 172.21.15.114:0/1911900310 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110018af0 con 0x55a49ac481b0 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f7109600640 1 -- 172.21.15.114:0/1911900310 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f710c026400 con 0x55a49ac481b0 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f712dc00740 10 client.28312 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f712dc00740 10 client.28312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f712dc00740 20 client.28312 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f712dc00740 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49ac06d80 0x55a49b00a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f712dc00740 4 client.28312 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:25.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.518+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a49af64600 con 0x55a49ac06d80 2024-09-21T07:05:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.520+0000 7f711fe00640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49ac06d80 0x55a49b00a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.520+0000 7f711fe00640 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49ac06d80 0x55a49b00a5d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da55ed691557d27 server_cookie=6c5362acfb9ac45e in_seq=0 out_seq=0 2024-09-21T07:05:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.520+0000 7f7109600640 10 client.28312 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.521+0000 7f7109600640 1 -- 172.21.15.114:0/1911900310 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a49af64600 con 0x55a49ac06d80 2024-09-21T07:05:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.521+0000 7f7109600640 10 client.28312 handle_command_reply: tid=0 2024-09-21T07:05:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.521+0000 7f712dc00740 1 client.28312 shutdown 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 msgr2=0x55a49aff8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49aff8e20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f710c005e00 tx=0x7f710c004730 comp rx=0 tx=0).stop 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 shutdown_connections 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49ac06d80 0x55a49b00a5d0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49aff8e20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac274d0 0x55a49aff88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 --2- 172.21.15.114:0/1911900310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac1f2f0 0x55a49aff83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 >> 172.21.15.114:0/1911900310 conn(0x55a49ac3c300 msgr2=0x55a49afede40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.524+0000 7f7109600640 0 client.28312 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.525+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 shutdown_connections 2024-09-21T07:05:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.525+0000 7f712dc00740 1 -- 172.21.15.114:0/1911900310 wait complete. 2024-09-21T07:05:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.527+0000 7f712dc00740 20 client.28312 trim_cache size 0 max 16384 2024-09-21T07:05:25.551 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:25 smithi114 ceph-mon[30654]: pgmap v742: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.9 MiB/s wr, 510 op/s 2024-09-21T07:05:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.560+0000 7f712dc00740 1 Processor -- start 2024-09-21T07:05:25.561 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.560+0000 7f712dc00740 1 -- start start 2024-09-21T07:05:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.561+0000 7f712dc00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 0x55a49afee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.561+0000 7f712dc00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49aff8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.561+0000 7f712dc00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac481b0 0x55a49aff8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.561+0000 7f712dc00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71201bfbc0 con 0x55a49ac481b0 2024-09-21T07:05:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.561+0000 7f712dc00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71201c1700 con 0x55a49ac274d0 2024-09-21T07:05:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.561+0000 7f712dc00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71201c0290 con 0x55a49ac1f2f0 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f711fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 0x55a49afee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f711fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 0x55a49afee550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48964/0 (socket says 172.21.15.114:48964) 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f711fe00640 1 -- 172.21.15.114:0/496333438 learned_addr learned my addr 172.21.15.114:0/496333438 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f711fe00640 1 -- 172.21.15.114:0/496333438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 msgr2=0x55a49aff8880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f7124a00640 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac481b0 0x55a49aff8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f711fe00640 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49aff8880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.562+0000 7f711fe00640 1 -- 172.21.15.114:0/496333438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac481b0 msgr2=0x55a49aff8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.563+0000 7f711fe00640 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac481b0 0x55a49aff8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.563+0000 7f711fe00640 1 -- 172.21.15.114:0/496333438 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a49afedbf0 con 0x55a49ac1f2f0 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.563+0000 7f7124a00640 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac481b0 0x55a49aff8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.563+0000 7f711fe00640 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 0x55a49afee550 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f71180094e0 tx=0x7f711810c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbfc15c2d604e7fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.563+0000 7f710be00640 1 -- 172.21.15.114:0/496333438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180c8e60 con 0x55a49ac1f2f0 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.563+0000 7f710be00640 1 -- 172.21.15.114:0/496333438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118036040 con 0x55a49ac1f2f0 2024-09-21T07:05:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.564+0000 7f710be00640 1 -- 172.21.15.114:0/496333438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180cee60 con 0x55a49ac1f2f0 2024-09-21T07:05:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 -- 172.21.15.114:0/496333438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 msgr2=0x55a49afee550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 0x55a49afee550 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f71180094e0 tx=0x7f711810c830 comp rx=0 tx=0).stop 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 -- 172.21.15.114:0/496333438 shutdown_connections 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac481b0 0x55a49aff8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49aff8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 --2- 172.21.15.114:0/496333438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac1f2f0 0x55a49afee550 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 -- 172.21.15.114:0/496333438 >> 172.21.15.114:0/496333438 conn(0x55a49ac3c300 msgr2=0x55a49afede40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 -- 172.21.15.114:0/496333438 shutdown_connections 2024-09-21T07:05:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.565+0000 7f712dc00740 1 -- 172.21.15.114:0/496333438 wait complete. 2024-09-21T07:05:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 -- start start 2024-09-21T07:05:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 Processor -- start 2024-09-21T07:05:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49affe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49affe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49b06d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71201c1030 con 0x55a49ac1f2f0 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71201c0960 con 0x55a49ac274d0 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71201bdbc0 con 0x55a49ac481b0 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.566+0000 7f712dc00740 10 client.0 _ll_register_callbacks cb 0x7f710c009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.567+0000 7f711fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49affe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.567+0000 7f711fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49affe220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46980/0 (socket says 172.21.15.114:46980) 2024-09-21T07:05:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.567+0000 7f711fe00640 1 -- 172.21.15.114:0/3822747081 learned_addr learned my addr 172.21.15.114:0/3822747081 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f711f400640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49affe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f7124a00640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49b06d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f711fe00640 1 -- 172.21.15.114:0/3822747081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 msgr2=0x55a49b06d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f711fe00640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49b06d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f711fe00640 1 -- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 msgr2=0x55a49affe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f711fe00640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49affe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f711fe00640 1 -- 172.21.15.114:0/3822747081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49b06d680 con 0x55a49ac1f2f0 2024-09-21T07:05:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.568+0000 7f7124a00640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49b06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.569+0000 7f711fe00640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49affe220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7118106090 tx=0x7f71180ce3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34c6cb8cfbc8b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.569+0000 7f7109600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:05:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.571+0000 7f7109600640 1 -- 172.21.15.114:0/3822747081 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180ce530 con 0x55a49ac1f2f0 2024-09-21T07:05:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.571+0000 7f7109600640 1 -- 172.21.15.114:0/3822747081 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f71180ca9d0 con 0x55a49ac1f2f0 2024-09-21T07:05:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.571+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7118036040 con 0x55a49ac1f2f0 2024-09-21T07:05:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.572+0000 7f7109600640 1 -- 172.21.15.114:0/3822747081 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f71180cded0 con 0x55a49ac1f2f0 2024-09-21T07:05:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.572+0000 7f712dc00740 10 client.17865 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.572+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a49b074920 con 0x55a49ac1f2f0 2024-09-21T07:05:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.573+0000 7f7109600640 1 -- 172.21.15.114:0/3822747081 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7118036040 con 0x55a49ac1f2f0 2024-09-21T07:05:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f7109600640 1 -- 172.21.15.114:0/3822747081 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f71180f02a0 con 0x55a49ac1f2f0 2024-09-21T07:05:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f712dc00740 10 client.17865 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f712dc00740 10 client.17865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f712dc00740 20 client.17865 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f712dc00740 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49abdc1c0 0x55a49b077020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f712dc00740 4 client.17865 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:05:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.574+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a49af600b0 con 0x55a49abdc1c0 2024-09-21T07:05:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.575+0000 7f711f400640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49abdc1c0 0x55a49b077020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.577+0000 7f711f400640 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49abdc1c0 0x55a49b077020 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afb830198bddb54e server_cookie=864b0c4238f9a65f in_seq=0 out_seq=0 2024-09-21T07:05:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.577+0000 7f7109600640 10 client.17865 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.590+0000 7f7109600640 1 -- 172.21.15.114:0/3822747081 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a49af600b0 con 0x55a49abdc1c0 2024-09-21T07:05:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.590+0000 7f7109600640 10 client.17865 handle_command_reply: tid=0 2024-09-21T07:05:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 client.17865 shutdown 2024-09-21T07:05:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 msgr2=0x55a49affe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49affe220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7118106090 tx=0x7f71180ce3d0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 shutdown_connections 2024-09-21T07:05:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a49abdc1c0 0x55a49b077020 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a49ac481b0 0x55a49b06d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a49ac274d0 0x55a49affe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 --2- 172.21.15.114:0/3822747081 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a49ac1f2f0 0x55a49affe220 secure :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7118106090 tx=0x7f71180ce3d0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 >> 172.21.15.114:0/3822747081 conn(0x55a49ac3c300 msgr2=0x55a49aff97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f7109600640 0 client.17865 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 shutdown_connections 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.591+0000 7f712dc00740 1 -- 172.21.15.114:0/3822747081 wait complete. 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.592+0000 7f712dc00740 20 client.17865 trim_cache size 0 max 16384 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:05:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.592+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71040b6700 msgr2=0x7f71040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71040b6700 0x7f71040b8c10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f710c009bf0 tx=0x7f710c005f90 comp rx=0 tx=0).stop 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 msgr2=0x7f71201a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 0x7f71201a9070 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f711802ade0 tx=0x7f7118031770 comp rx=0 tx=0).stop 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 shutdown_connections 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71040b6700 0x7f71040b8c10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f712011cba0 0x7f71201ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7120111ca0 0x7f71201a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 --2- 172.21.15.114:0/1252609072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f712010a900 0x7f71201a9070 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 >> 172.21.15.114:0/1252609072 conn(0x7f7120070c00 msgr2=0x7f712011ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 shutdown_connections 2024-09-21T07:05:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:25.593+0000 7f7125400640 1 -- 172.21.15.114:0/1252609072 wait complete. 2024-09-21T07:05:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:25 smithi183 ceph-mon[34219]: pgmap v742: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.9 MiB/s wr, 510 op/s 2024-09-21T07:05:25.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:25 smithi193 ceph-mon[34601]: pgmap v742: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 7.9 MiB/s wr, 510 op/s 2024-09-21T07:05:25.998 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:27.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 -- 172.21.15.114:0/131183908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 msgr2=0x7fa920110110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 --2- 172.21.15.114:0/131183908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa920110110 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa918009ca0 tx=0x7fa91802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 -- 172.21.15.114:0/131183908 shutdown_connections 2024-09-21T07:05:27.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 --2- 172.21.15.114:0/131183908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa920110110 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 --2- 172.21.15.114:0/131183908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa920105110 0x7fa92010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 --2- 172.21.15.114:0/131183908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9201047c0 0x7fa920104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:26.998+0000 7fa91fe00640 1 -- 172.21.15.114:0/131183908 >> 172.21.15.114:0/131183908 conn(0x7fa9200fe2c0 msgr2=0x7fa920100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:27.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.001+0000 7fa91fe00640 1 -- 172.21.15.114:0/131183908 shutdown_connections 2024-09-21T07:05:27.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.001+0000 7fa91fe00640 1 -- 172.21.15.114:0/131183908 wait complete. 2024-09-21T07:05:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.003+0000 7fa91fe00640 1 Processor -- start 2024-09-21T07:05:27.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.003+0000 7fa91fe00640 1 -- start start 2024-09-21T07:05:27.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.003+0000 7fa91fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9201047c0 0x7fa920198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.003+0000 7fa91fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa920105110 0x7fa9201985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.003+0000 7fa91fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa92019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.004+0000 7fa91fe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa920111690 con 0x7fa9201047c0 2024-09-21T07:05:27.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.004+0000 7fa91fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa920111510 con 0x7fa920105110 2024-09-21T07:05:27.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.004+0000 7fa91fe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa920111810 con 0x7fa92010dc20 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.004+0000 7fa91f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa92019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.004+0000 7fa91f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa92019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:48998/0 (socket says 172.21.15.114:48998) 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.004+0000 7fa91f400640 1 -- 172.21.15.114:0/1202446873 learned_addr learned my addr 172.21.15.114:0/1202446873 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91e000640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa920105110 0x7fa9201985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91f400640 1 -- 172.21.15.114:0/1202446873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa920105110 msgr2=0x7fa9201985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91f400640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa920105110 0x7fa9201985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91f400640 1 -- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9201047c0 msgr2=0x7fa920198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91f400640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9201047c0 0x7fa920198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91f400640 1 -- 172.21.15.114:0/1202446873 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92019ded0 con 0x7fa92010dc20 2024-09-21T07:05:27.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.005+0000 7fa91e000640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa920105110 0x7fa9201985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:27.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.007+0000 7fa91f400640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa92019d860 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098d0 tx=0x7fa9180319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:27.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.007+0000 7fa917400640 1 -- 172.21.15.114:0/1202446873 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa918003b70 con 0x7fa92010dc20 2024-09-21T07:05:27.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.007+0000 7fa917400640 1 -- 172.21.15.114:0/1202446873 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa918003d10 con 0x7fa92010dc20 2024-09-21T07:05:27.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.007+0000 7fa917400640 1 -- 172.21.15.114:0/1202446873 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91802f990 con 0x7fa92010dc20 2024-09-21T07:05:27.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.008+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92019e160 con 0x7fa92010dc20 2024-09-21T07:05:27.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.008+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92019e650 con 0x7fa92010dc20 2024-09-21T07:05:27.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.014+0000 7fa917400640 1 -- 172.21.15.114:0/1202446873 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa91802fb30 con 0x7fa92010dc20 2024-09-21T07:05:27.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.017+0000 7fa917400640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9040b6770 0x7fa9040b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.017+0000 7fa917400640 1 -- 172.21.15.114:0/1202446873 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fa918112ad0 con 0x7fa92010dc20 2024-09-21T07:05:27.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.018+0000 7fa91ea00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9040b6770 0x7fa9040b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.023+0000 7fa91ea00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9040b6770 0x7fa9040b8c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9080058c0 tx=0x7fa908005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:27.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.024+0000 7fa92c485740 1 Processor -- start 2024-09-21T07:05:27.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.024+0000 7fa92c485740 1 -- start start 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa92c485740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b5af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa92c485740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b5b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa92c485740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b5b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa92c485740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5591c2ad1fb0 con 0x5591c27b4070 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa92c485740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591c2ac13e0 con 0x5591c2793390 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa92c485740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591c2ac51f0 con 0x5591c278b1b0 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa91e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b5b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa91e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b5b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:49008/0 (socket says 172.21.15.114:49008) 2024-09-21T07:05:27.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.025+0000 7fa91e000640 1 -- 172.21.15.114:0/1285372183 learned_addr learned my addr 172.21.15.114:0/1285372183 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:27.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa91e000640 1 -- 172.21.15.114:0/1285372183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 msgr2=0x5591c2b5b9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:27.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa91e000640 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b5b9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa91e000640 1 -- 172.21.15.114:0/1285372183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 msgr2=0x5591c2b5af70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa91e000640 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b5af70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa91e000640 1 -- 172.21.15.114:0/1285372183 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c2a8bcd0 con 0x5591c278b1b0 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa91e000640 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b5b4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa910008bb0 tx=0x7fa91000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8273c68bf413182d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.026+0000 7fa90fe00640 1 -- 172.21.15.114:0/1285372183 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910014070 con 0x5591c278b1b0 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.027+0000 7fa90fe00640 1 -- 172.21.15.114:0/1285372183 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9100047a0 con 0x5591c278b1b0 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.027+0000 7fa90fe00640 1 -- 172.21.15.114:0/1285372183 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910010650 con 0x5591c278b1b0 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 -- 172.21.15.114:0/1285372183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 msgr2=0x5591c2b5b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b5b4b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa910008bb0 tx=0x7fa91000e720 comp rx=0 tx=0).stop 2024-09-21T07:05:27.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 -- 172.21.15.114:0/1285372183 shutdown_connections 2024-09-21T07:05:27.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b5b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b5b4b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 --2- 172.21.15.114:0/1285372183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b5af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.028+0000 7fa92c485740 1 -- 172.21.15.114:0/1285372183 >> 172.21.15.114:0/1285372183 conn(0x5591c27a8460 msgr2=0x5591c2b58e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:27.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.031+0000 7fa92c485740 1 -- 172.21.15.114:0/1285372183 shutdown_connections 2024-09-21T07:05:27.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.031+0000 7fa92c485740 1 -- 172.21.15.114:0/1285372183 wait complete. 2024-09-21T07:05:27.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.032+0000 7fa92c485740 1 -- start start 2024-09-21T07:05:27.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.032+0000 7fa92c485740 1 Processor -- start 2024-09-21T07:05:27.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.034+0000 7fa92c485740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 0x5591c2b667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.034+0000 7fa92c485740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c2793390 0x5591c2b66dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.034+0000 7fa92c485740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 0x5591c2b72a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa92c485740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5591c2922700 con 0x5591c2793390 2024-09-21T07:05:27.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa92c485740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5591c2a5c580 con 0x5591c278b1b0 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa91ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 0x5591c2b667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa91f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 0x5591c2b72a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa91f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 0x5591c2b72a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:49014/0 (socket says 172.21.15.114:49014) 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa91f400640 1 -- 172.21.15.114:0/3695052481 learned_addr learned my addr 172.21.15.114:0/3695052481 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa91ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 0x5591c2b667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54708/0 (socket says 172.21.15.114:54708) 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5591c2b69750 con 0x5591c27b4070 2024-09-21T07:05:27.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.035+0000 7fa92c485740 10 client.0 _ll_register_callbacks cb 0x5591c2a6c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:27.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91f400640 1 -- 172.21.15.114:0/3695052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 msgr2=0x5591c2b667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:27.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91f400640 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 0x5591c2b667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91f400640 1 -- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c2793390 msgr2=0x5591c2b66dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:27.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91f400640 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c2793390 0x5591c2b66dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91f400640 1 -- 172.21.15.114:0/3695052481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c281f650 con 0x5591c27b4070 2024-09-21T07:05:27.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91ea00640 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 0x5591c2b667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:27.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa91f400640 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 0x5591c2b72a40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa918003fd0 tx=0x7fa9180d83e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c25716fb3df26d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:27.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.036+0000 7fa90cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:27.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.037+0000 7fa90cc00640 1 -- 172.21.15.114:0/3695052481 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180d8060 con 0x5591c27b4070 2024-09-21T07:05:27.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.037+0000 7fa90cc00640 1 -- 172.21.15.114:0/3695052481 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fa918004b40 con 0x5591c27b4070 2024-09-21T07:05:27.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.037+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591c2a5eb90 con 0x5591c27b4070 2024-09-21T07:05:27.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.039+0000 7fa90cc00640 1 -- 172.21.15.114:0/3695052481 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa9180d8c50 con 0x5591c27b4070 2024-09-21T07:05:27.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.039+0000 7fa92c485740 10 client.28330 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:27.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.039+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5591c2a79950 con 0x5591c27b4070 2024-09-21T07:05:27.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.040+0000 7fa90cc00640 1 -- 172.21.15.114:0/3695052481 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5591c2a5eb90 con 0x5591c27b4070 2024-09-21T07:05:27.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.040+0000 7fa90cc00640 1 -- 172.21.15.114:0/3695052481 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fa9180f3c70 con 0x5591c27b4070 2024-09-21T07:05:27.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.040+0000 7fa92c485740 10 client.28330 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:27.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.040+0000 7fa92c485740 10 client.28330 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:05:27.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.041+0000 7fa92c485740 20 client.28330 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:27.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.041+0000 7fa92c485740 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c27729f0 0x5591c2b78860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.041+0000 7fa92c485740 4 client.28330 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:27.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.041+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c2ad3f00 con 0x5591c27729f0 2024-09-21T07:05:27.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.041+0000 7fa91ea00640 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c27729f0 0x5591c2b78860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.043+0000 7fa90cc00640 10 client.28330 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:05:27.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.044+0000 7fa91ea00640 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c27729f0 0x5591c2b78860 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4279378d3ea51f6 server_cookie=395c902b7288ad36 in_seq=0 out_seq=0 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.045+0000 7fa90cc00640 1 -- 172.21.15.114:0/3695052481 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591c2ad3f00 con 0x5591c27729f0 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.045+0000 7fa90cc00640 10 client.28330 handle_command_reply: tid=0 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.045+0000 7fa92c485740 1 client.28330 shutdown 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.045+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 msgr2=0x5591c2b72a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.045+0000 7fa92c485740 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 0x5591c2b72a40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa918003fd0 tx=0x7fa9180d83e0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 shutdown_connections 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c27729f0 0x5591c2b78860 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c27b4070 0x5591c2b72a40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c2793390 0x5591c2b66dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 --2- 172.21.15.114:0/3695052481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c278b1b0 0x5591c2b667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 >> 172.21.15.114:0/3695052481 conn(0x5591c27a8460 msgr2=0x5591c2b57600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa90cc00640 0 client.28330 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 shutdown_connections 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 1 -- 172.21.15.114:0/3695052481 wait complete. 2024-09-21T07:05:27.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.046+0000 7fa92c485740 20 client.28330 trim_cache size 0 max 16384 2024-09-21T07:05:27.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.075+0000 7fa92c485740 1 Processor -- start 2024-09-21T07:05:27.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.076+0000 7fa92c485740 1 -- start start 2024-09-21T07:05:27.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa92c485740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c278b1b0 0x5591c2b68410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa92c485740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c2793390 0x5591c2b58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa92c485740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 0x5591c2b58b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa92c485740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201aea40 con 0x5591c278b1b0 2024-09-21T07:05:27.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa92c485740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201b0580 con 0x5591c27b4070 2024-09-21T07:05:27.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa92c485740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201af110 con 0x5591c2793390 2024-09-21T07:05:27.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa91e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c2793390 0x5591c2b58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.077+0000 7fa91f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 0x5591c2b58b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 0x5591c2b58b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54710/0 (socket says 172.21.15.114:54710) 2024-09-21T07:05:27.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 -- 172.21.15.114:0/3014730616 learned_addr learned my addr 172.21.15.114:0/3014730616 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:27.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91ea00640 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c278b1b0 0x5591c2b68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 -- 172.21.15.114:0/3014730616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c2793390 msgr2=0x5591c2b58610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c2793390 0x5591c2b58610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 -- 172.21.15.114:0/3014730616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c278b1b0 msgr2=0x5591c2b68410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c278b1b0 0x5591c2b68410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.078+0000 7fa91f400640 1 -- 172.21.15.114:0/3014730616 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c2772c20 con 0x5591c27b4070 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.079+0000 7fa91e000640 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c2793390 0x5591c2b58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.079+0000 7fa91f400640 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 0x5591c2b58b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa918110450 tx=0x7fa9180d8450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=322d6d47c00e2595 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:27.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.079+0000 7fa91ea00640 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c278b1b0 0x5591c2b68410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:27.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.080+0000 7fa90f400640 1 -- 172.21.15.114:0/3014730616 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180f7070 con 0x5591c27b4070 2024-09-21T07:05:27.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.081+0000 7fa90f400640 1 -- 172.21.15.114:0/3014730616 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa918036030 con 0x5591c27b4070 2024-09-21T07:05:27.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.081+0000 7fa90f400640 1 -- 172.21.15.114:0/3014730616 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9180f3ba0 con 0x5591c27b4070 2024-09-21T07:05:27.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.082+0000 7fa92c485740 1 -- 172.21.15.114:0/3014730616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 msgr2=0x5591c2b58b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.082+0000 7fa92c485740 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 0x5591c2b58b50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa918110450 tx=0x7fa9180d8450 comp rx=0 tx=0).stop 2024-09-21T07:05:27.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.083+0000 7fa92c485740 1 -- 172.21.15.114:0/3014730616 shutdown_connections 2024-09-21T07:05:27.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.083+0000 7fa92c485740 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c27b4070 0x5591c2b58b50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.083+0000 7fa92c485740 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c2793390 0x5591c2b58610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.083+0000 7fa92c485740 1 --2- 172.21.15.114:0/3014730616 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c278b1b0 0x5591c2b68410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.083+0000 7fa92c485740 1 -- 172.21.15.114:0/3014730616 >> 172.21.15.114:0/3014730616 conn(0x5591c27a8460 msgr2=0x5591c2b67d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:27.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.083+0000 7fa92c485740 1 -- 172.21.15.114:0/3014730616 shutdown_connections 2024-09-21T07:05:27.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.084+0000 7fa92c485740 1 -- 172.21.15.114:0/3014730616 wait complete. 2024-09-21T07:05:27.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.084+0000 7fa92c485740 1 -- start start 2024-09-21T07:05:27.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.084+0000 7fa92c485740 1 Processor -- start 2024-09-21T07:05:27.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b6ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b6b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201afeb0 con 0x5591c27b4070 2024-09-21T07:05:27.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201af7e0 con 0x5591c2793390 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9201aca40 con 0x5591c278b1b0 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.085+0000 7fa92c485740 10 client.0 _ll_register_callbacks cb 0x7fa9180d43f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b6b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b6ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b6b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47042/0 (socket says 172.21.15.114:47042) 2024-09-21T07:05:27.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b6ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:49030/0 (socket says 172.21.15.114:49030) 2024-09-21T07:05:27.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91f400640 1 -- 172.21.15.114:0/2458168140 learned_addr learned my addr 172.21.15.114:0/2458168140 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:27.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.086+0000 7fa91e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b6b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54712/0 (socket says 172.21.15.114:54712) 2024-09-21T07:05:27.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa91e000640 1 -- 172.21.15.114:0/2458168140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 msgr2=0x5591c2b6ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:27.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa91e000640 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b6ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa91e000640 1 -- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 msgr2=0x5591c2b6b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:27.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa91e000640 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b6b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa91e000640 1 -- 172.21.15.114:0/2458168140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c2bf3400 con 0x5591c2793390 2024-09-21T07:05:27.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa91e000640 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b6b090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa91000ec20 tx=0x7fa910003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bd9cb5c80197e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:27.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.087+0000 7fa90cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:27.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.088+0000 7fa90cc00640 1 -- 172.21.15.114:0/2458168140 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910014070 con 0x5591c2793390 2024-09-21T07:05:27.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.088+0000 7fa90cc00640 1 -- 172.21.15.114:0/2458168140 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fa910003da0 con 0x5591c2793390 2024-09-21T07:05:27.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.088+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa918036030 con 0x5591c2793390 2024-09-21T07:05:27.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.089+0000 7fa90cc00640 1 -- 172.21.15.114:0/2458168140 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa910005610 con 0x5591c2793390 2024-09-21T07:05:27.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa92c485740 10 client.28382 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:27.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5591c2bf4660 con 0x5591c2793390 2024-09-21T07:05:27.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa90cc00640 1 -- 172.21.15.114:0/2458168140 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa918036030 con 0x5591c2793390 2024-09-21T07:05:27.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa90cc00640 1 -- 172.21.15.114:0/2458168140 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7fa91002a030 con 0x5591c2793390 2024-09-21T07:05:27.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa92c485740 10 client.28382 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:27.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa92c485740 10 client.28382 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:05:27.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.090+0000 7fa92c485740 20 client.28382 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:27.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.091+0000 7fa92c485740 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c2747da0 0x5591c2bf6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:27.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.091+0000 7fa92c485740 4 client.28382 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:05:27.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.091+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591c2acf9b0 con 0x5591c2747da0 2024-09-21T07:05:27.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.093+0000 7fa91ea00640 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c2747da0 0x5591c2bf6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:27.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.095+0000 7fa91ea00640 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c2747da0 0x5591c2bf6e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ae3f9afaead032d server_cookie=7b7b57a24f814361 in_seq=0 out_seq=0 2024-09-21T07:05:27.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.095+0000 7fa90cc00640 10 client.28382 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:05:27.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.095+0000 7fa90cc00640 1 -- 172.21.15.114:0/2458168140 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591c2acf9b0 con 0x5591c2747da0 2024-09-21T07:05:27.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.095+0000 7fa90cc00640 10 client.28382 handle_command_reply: tid=0 2024-09-21T07:05:27.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.095+0000 7fa92c485740 1 client.28382 shutdown 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 msgr2=0x5591c2b6b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b6b090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa91000ec20 tx=0x7fa910003820 comp rx=0 tx=0).stop 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 shutdown_connections 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5591c2747da0 0x5591c2bf6e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5591c27b4070 0x5591c2b6b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5591c2793390 0x5591c2b6b090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa90cc00640 0 client.28382 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:05:27.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 --2- 172.21.15.114:0/2458168140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5591c278b1b0 0x5591c2b6ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 >> 172.21.15.114:0/2458168140 conn(0x5591c27a8460 msgr2=0x5591c2b5bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:27.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 shutdown_connections 2024-09-21T07:05:27.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.096+0000 7fa92c485740 1 -- 172.21.15.114:0/2458168140 wait complete. 2024-09-21T07:05:27.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.097+0000 7fa92c485740 20 client.28382 trim_cache size 0 max 16384 2024-09-21T07:05:27.100 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:05:27.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9040b6770 msgr2=0x7fa9040b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9040b6770 0x7fa9040b8c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa9080058c0 tx=0x7fa908005850 comp rx=0 tx=0).stop 2024-09-21T07:05:27.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 msgr2=0x7fa92019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:27.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa92019d860 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098d0 tx=0x7fa9180319f0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 shutdown_connections 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa92010dc20 0x7fa92019d860 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9040b6770 0x7fa9040b8c80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa920105110 0x7fa9201985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 --2- 172.21.15.114:0/1202446873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9201047c0 0x7fa920198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 >> 172.21.15.114:0/1202446873 conn(0x7fa9200fe2c0 msgr2=0x7fa920100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 shutdown_connections 2024-09-21T07:05:27.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:27.098+0000 7fa91fe00640 1 -- 172.21.15.114:0/1202446873 wait complete. 2024-09-21T07:05:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:27 smithi114 ceph-mon[30654]: pgmap v743: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.9 MiB/s wr, 402 op/s 2024-09-21T07:05:27.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:27 smithi183 ceph-mon[34219]: pgmap v743: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.9 MiB/s wr, 402 op/s 2024-09-21T07:05:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:27 smithi193 ceph-mon[34601]: pgmap v743: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 5.9 MiB/s wr, 402 op/s 2024-09-21T07:05:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 650.0 s, 449.6 tps, lat 239.833 ms stddev 302.838 2024-09-21T07:05:29.110 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:05:29.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:29 smithi114 ceph-mon[30654]: pgmap v744: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.6 MiB/s wr, 460 op/s 2024-09-21T07:05:29.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:29 smithi183 ceph-mon[34219]: pgmap v744: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.6 MiB/s wr, 460 op/s 2024-09-21T07:05:29.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:29 smithi193 ceph-mon[34601]: pgmap v744: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 6.6 MiB/s wr, 460 op/s 2024-09-21T07:05:29.803 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:30.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.718+0000 7f6e24200640 1 -- 172.21.15.114:0/2521273524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 msgr2=0x7f6e1c117a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.718+0000 7f6e24200640 1 --2- 172.21.15.114:0/2521273524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 0x7f6e1c117a80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009900 tx=0x7f6e1002a490 comp rx=0 tx=0).stop 2024-09-21T07:05:30.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.719+0000 7f6e24200640 1 -- 172.21.15.114:0/2521273524 shutdown_connections 2024-09-21T07:05:30.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.719+0000 7f6e24200640 1 --2- 172.21.15.114:0/2521273524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 0x7f6e1c117a80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.719+0000 7f6e24200640 1 --2- 172.21.15.114:0/2521273524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 0x7f6e1c114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.719+0000 7f6e24200640 1 --2- 172.21.15.114:0/2521273524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c10c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.719+0000 7f6e24200640 1 -- 172.21.15.114:0/2521273524 >> 172.21.15.114:0/2521273524 conn(0x7f6e1c071310 msgr2=0x7f6e1c071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:30.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.720+0000 7f6e24200640 1 -- 172.21.15.114:0/2521273524 shutdown_connections 2024-09-21T07:05:30.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.720+0000 7f6e24200640 1 -- 172.21.15.114:0/2521273524 wait complete. 2024-09-21T07:05:30.725 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.723+0000 7f6e24200640 1 Processor -- start 2024-09-21T07:05:30.728 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.724+0000 7f6e24200640 1 -- start start 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.724+0000 7f6e24200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c111dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.724+0000 7f6e24200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 0x7f6e1c112370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.724+0000 7f6e24200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 0x7f6e1c1ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.724+0000 7f6e24200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c10b260 con 0x7f6e1c115570 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.725+0000 7f6e24200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c10b0e0 con 0x7f6e1c10bff0 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.725+0000 7f6e24200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c10b3e0 con 0x7f6e1c10c940 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.725+0000 7f6e21800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c111dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.726+0000 7f6e21800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c111dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54730/0 (socket says 172.21.15.114:54730) 2024-09-21T07:05:30.729 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.726+0000 7f6e21800640 1 -- 172.21.15.114:0/235679913 learned_addr learned my addr 172.21.15.114:0/235679913 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:30.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.730+0000 7f6e20e00640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 0x7f6e1c112370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.731+0000 7f6e21800640 1 -- 172.21.15.114:0/235679913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 msgr2=0x7f6e1c112370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.732+0000 7f6e21800640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 0x7f6e1c112370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.733+0000 7f6e21800640 1 -- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 msgr2=0x7f6e1c1ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.733+0000 7f6e21800640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 0x7f6e1c1ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.733+0000 7f6e21800640 1 -- 172.21.15.114:0/235679913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c1ac020 con 0x7f6e1c10bff0 2024-09-21T07:05:30.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.733+0000 7f6e20e00640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 0x7f6e1c112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:30.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.735+0000 7f6e21800640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c111dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400cd10 tx=0x7f6e1400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.735+0000 7f6e1a000640 1 -- 172.21.15.114:0/235679913 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14004b20 con 0x7f6e1c10bff0 2024-09-21T07:05:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.735+0000 7f6e1a000640 1 -- 172.21.15.114:0/235679913 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e14004cc0 con 0x7f6e1c10bff0 2024-09-21T07:05:30.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.735+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1c1ac310 con 0x7f6e1c10bff0 2024-09-21T07:05:30.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.737+0000 7f6e1a000640 1 -- 172.21.15.114:0/235679913 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400f6b0 con 0x7f6e1c10bff0 2024-09-21T07:05:30.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.737+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1c1a1b00 con 0x7f6e1c10bff0 2024-09-21T07:05:30.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.743+0000 7f6e1a000640 1 -- 172.21.15.114:0/235679913 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6e1400f950 con 0x7f6e1c10bff0 2024-09-21T07:05:30.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 Processor -- start 2024-09-21T07:05:30.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 -- start start 2024-09-21T07:05:30.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862d9070 0x55c08667fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b01b0 0x55c0866802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 0x55c086680820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c0865f6fb0 con 0x55c0862b01b0 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0865e63e0 con 0x55c0862d9070 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.745+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0865ea1f0 con 0x55c0862b8390 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.746+0000 7f6e1a000640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6e040b66b0 0x7f6e040b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.746+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 0x55c086680820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.746+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 0x55c086680820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:49062/0 (socket says 172.21.15.114:49062) 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.746+0000 7f6e20e00640 1 -- 172.21.15.114:0/3236887721 learned_addr learned my addr 172.21.15.114:0/3236887721 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e22200640 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862d9070 0x55c08667fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e20e00640 1 -- 172.21.15.114:0/3236887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862d9070 msgr2=0x55c08667fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e20e00640 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862d9070 0x55c08667fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e20e00640 1 -- 172.21.15.114:0/3236887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b01b0 msgr2=0x55c0866802e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e20e00640 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b01b0 0x55c0866802e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e20e00640 1 -- 172.21.15.114:0/3236887721 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0865b0cd0 con 0x55c0862b8390 2024-09-21T07:05:30.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.747+0000 7f6e22200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6e040b66b0 0x7f6e040b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.748+0000 7f6e20e00640 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 0x55c086680820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800a9a0 tx=0x7f6e0800ae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cef76b7c6fa93a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.749+0000 7f6e1a000640 1 -- 172.21.15.114:0/235679913 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f6e140e7280 con 0x7f6e1c10bff0 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.749+0000 7f6e22200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6e040b66b0 0x7f6e040b8bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6e1002a9b0 tx=0x7f6e10009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.749+0000 7f6e0ea00640 1 -- 172.21.15.114:0/3236887721 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08037070 con 0x55c0862b8390 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.749+0000 7f6e0ea00640 1 -- 172.21.15.114:0/3236887721 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e08033400 con 0x55c0862b8390 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.749+0000 7f6e0ea00640 1 -- 172.21.15.114:0/3236887721 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0803a5c0 con 0x55c0862b8390 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.750+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3236887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 msgr2=0x55c086680820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.750+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 0x55c086680820 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800a9a0 tx=0x7f6e0800ae60 comp rx=0 tx=0).stop 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3236887721 shutdown_connections 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b8390 0x55c086680820 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b01b0 0x55c0866802e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.755 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3236887721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862d9070 0x55c08667fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3236887721 >> 172.21.15.114:0/3236887721 conn(0x55c0862cd460 msgr2=0x55c08667dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:30.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3236887721 shutdown_connections 2024-09-21T07:05:30.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.752+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3236887721 wait complete. 2024-09-21T07:05:30.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.753+0000 7f6e2b00c740 1 -- start start 2024-09-21T07:05:30.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.753+0000 7f6e2b00c740 1 Processor -- start 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c08668b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c08668bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c0866978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e21800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c08668b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c086447700 con 0x55c0862b8390 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c086581580 con 0x55c0862b01b0 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c08668e580 con 0x55c0862d9070 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.756+0000 7f6e22200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c0866978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.757+0000 7f6e22200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c0866978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:49078/0 (socket says 172.21.15.114:49078) 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.757+0000 7f6e22200640 1 -- 172.21.15.114:0/3301402887 learned_addr learned my addr 172.21.15.114:0/3301402887 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:30.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.757+0000 7f6e2b00c740 10 client.0 _ll_register_callbacks cb 0x55c086591470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:30.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.758+0000 7f6e20e00640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c08668bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.758+0000 7f6e22200640 1 -- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 msgr2=0x55c08668b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.759+0000 7f6e22200640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c08668b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.761 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.759+0000 7f6e22200640 1 -- 172.21.15.114:0/3301402887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 msgr2=0x55c08668bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.760+0000 7f6e22200640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c08668bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.763 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.762+0000 7f6e22200640 1 -- 172.21.15.114:0/3301402887 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c086344650 con 0x55c0862d9070 2024-09-21T07:05:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.762+0000 7f6e20e00640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c08668bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.762+0000 7f6e03400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.762+0000 7f6e22200640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c0866978a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6e10031750 tx=0x7f6e10031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69252ac6e9a3e69f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:30.764 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.762+0000 7f6e21800640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c08668b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:30.765 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.763+0000 7f6e03400640 1 -- 172.21.15.114:0/3301402887 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1004d020 con 0x55c0862d9070 2024-09-21T07:05:30.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.763+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c086583b90 con 0x55c0862d9070 2024-09-21T07:05:30.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.763+0000 7f6e03400640 1 -- 172.21.15.114:0/3301402887 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f6e10031840 con 0x55c0862d9070 2024-09-21T07:05:30.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.763+0000 7f6e03400640 1 -- 172.21.15.114:0/3301402887 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6e10058050 con 0x55c0862d9070 2024-09-21T07:05:30.766 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.764+0000 7f6e2b00c740 10 client.28342 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:30.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.764+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c08659e950 con 0x55c0862d9070 2024-09-21T07:05:30.767 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.765+0000 7f6e03400640 1 -- 172.21.15.114:0/3301402887 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c086583b90 con 0x55c0862d9070 2024-09-21T07:05:30.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e03400640 1 -- 172.21.15.114:0/3301402887 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f6e1003a040 con 0x55c0862d9070 2024-09-21T07:05:30.770 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e2b00c740 10 client.28342 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:30.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e2b00c740 10 client.28342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:30.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e2b00c740 20 client.28342 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:30.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0862979f0 0x55c08669d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e2b00c740 4 client.28342 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:30.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.768+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0865f8f00 con 0x55c0862979f0 2024-09-21T07:05:30.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.769+0000 7f6e21800640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0862979f0 0x55c08669d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.772+0000 7f6e03400640 10 client.28342 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:30.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.772+0000 7f6e21800640 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0862979f0 0x55c08669d730 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc95fe07845cb76c server_cookie=7d0396bef92f18b0 in_seq=0 out_seq=0 2024-09-21T07:05:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e03400640 1 -- 172.21.15.114:0/3301402887 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0865f8f00 con 0x55c0862979f0 2024-09-21T07:05:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e03400640 10 client.28342 handle_command_reply: tid=0 2024-09-21T07:05:30.777 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 client.28342 shutdown 2024-09-21T07:05:30.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 msgr2=0x55c0866978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c0866978a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6e10031750 tx=0x7f6e10031400 comp rx=0 tx=0).stop 2024-09-21T07:05:30.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 shutdown_connections 2024-09-21T07:05:30.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c0862979f0 0x55c08669d730 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c0866978a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c08668bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/3301402887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c08668b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.776+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 >> 172.21.15.114:0/3301402887 conn(0x55c0862cd460 msgr2=0x55c086682f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.777+0000 7f6e03400640 0 client.28342 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.777+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 shutdown_connections 2024-09-21T07:05:30.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.777+0000 7f6e2b00c740 1 -- 172.21.15.114:0/3301402887 wait complete. 2024-09-21T07:05:30.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.777+0000 7f6e2b00c740 20 client.28342 trim_cache size 0 max 16384 2024-09-21T07:05:30.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 Processor -- start 2024-09-21T07:05:30.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 -- start start 2024-09-21T07:05:30.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c0866835b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c0866977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c086697ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1b7250 con 0x55c0862b8390 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1b8d90 con 0x55c0862b01b0 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.813+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1b7920 con 0x55c0862d9070 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.814+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c0866977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.814+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c0866977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47068/0 (socket says 172.21.15.114:47068) 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.814+0000 7f6e20e00640 1 -- 172.21.15.114:0/2372597038 learned_addr learned my addr 172.21.15.114:0/2372597038 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.814+0000 7f6e21800640 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c0866835b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.814+0000 7f6e22200640 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c086697ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.815+0000 7f6e20e00640 1 -- 172.21.15.114:0/2372597038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 msgr2=0x55c086697ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.815+0000 7f6e20e00640 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c086697ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.815+0000 7f6e20e00640 1 -- 172.21.15.114:0/2372597038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 msgr2=0x55c0866835b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.815+0000 7f6e20e00640 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c0866835b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.815+0000 7f6e20e00640 1 -- 172.21.15.114:0/2372597038 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08668c890 con 0x55c0862b8390 2024-09-21T07:05:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.816+0000 7f6e22200640 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c086697ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.816+0000 7f6e20e00640 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c0866977a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800a940 tx=0x7f6e08006c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16dee27eaa8c749 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:30.819 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.817+0000 7f6e0e000640 1 -- 172.21.15.114:0/2372597038 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08037040 con 0x55c0862b8390 2024-09-21T07:05:30.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.819+0000 7f6e0e000640 1 -- 172.21.15.114:0/2372597038 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0800a210 con 0x55c0862b8390 2024-09-21T07:05:30.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.819+0000 7f6e0e000640 1 -- 172.21.15.114:0/2372597038 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08033a40 con 0x55c0862b8390 2024-09-21T07:05:30.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.823+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2372597038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 msgr2=0x55c0866977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.823+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c0866977a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800a940 tx=0x7f6e08006c70 comp rx=0 tx=0).stop 2024-09-21T07:05:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2372597038 shutdown_connections 2024-09-21T07:05:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862d9070 0x55c086697ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862b8390 0x55c0866977a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2372597038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b01b0 0x55c0866835b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2372597038 >> 172.21.15.114:0/2372597038 conn(0x55c0862cd460 msgr2=0x55c086682ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2372597038 shutdown_connections 2024-09-21T07:05:30.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.825+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2372597038 wait complete. 2024-09-21T07:05:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.826+0000 7f6e2b00c740 1 -- start start 2024-09-21T07:05:30.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.826+0000 7f6e2b00c740 1 Processor -- start 2024-09-21T07:05:30.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b01b0 0x55c08668f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b8390 0x55c08668fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e2b00c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862d9070 0x55c0866902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1b86c0 con 0x55c0862d9070 2024-09-21T07:05:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1b7ff0 con 0x55c0862b8390 2024-09-21T07:05:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e2b00c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c10a5d0 con 0x55c0862b01b0 2024-09-21T07:05:30.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b8390 0x55c08668fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e21800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b01b0 0x55c08668f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.828+0000 7f6e22200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862d9070 0x55c0866902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.829+0000 7f6e22200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862d9070 0x55c0866902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47076/0 (socket says 172.21.15.114:47076) 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.831+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b8390 0x55c08668fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54750/0 (socket says 172.21.15.114:54750) 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.831+0000 7f6e20e00640 1 -- 172.21.15.114:0/2153693446 learned_addr learned my addr 172.21.15.114:0/2153693446 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.831+0000 7f6e2b00c740 10 client.0 _ll_register_callbacks cb 0x7f6e10032e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.832+0000 7f6e21800640 1 -- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b8390 msgr2=0x55c08668fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.832+0000 7f6e21800640 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b8390 0x55c08668fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.832+0000 7f6e21800640 1 -- 172.21.15.114:0/2153693446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862d9070 msgr2=0x55c0866902d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.832+0000 7f6e21800640 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862d9070 0x55c0866902d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.832+0000 7f6e21800640 1 -- 172.21.15.114:0/2153693446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c086707120 con 0x55c0862b01b0 2024-09-21T07:05:30.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.833+0000 7f6e21800640 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b01b0 0x55c08668f850 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6e140e4600 tx=0x7f6e140ac7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39b25a24f1ed47d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:30.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.833+0000 7f6e03400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:30.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.833+0000 7f6e03400640 1 -- 172.21.15.114:0/2153693446 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14098070 con 0x55c0862b01b0 2024-09-21T07:05:30.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.833+0000 7f6e03400640 1 -- 172.21.15.114:0/2153693446 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f6e140abfe0 con 0x55c0862b01b0 2024-09-21T07:05:30.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.834+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e0800a210 con 0x55c0862b01b0 2024-09-21T07:05:30.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.835+0000 7f6e03400640 1 -- 172.21.15.114:0/2153693446 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6e140ac890 con 0x55c0862b01b0 2024-09-21T07:05:30.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.835+0000 7f6e2b00c740 10 client.28348 fetch_fsmap learned FSMap version 49 2024-09-21T07:05:30.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.835+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c0867083a0 con 0x55c0862b01b0 2024-09-21T07:05:30.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.835+0000 7f6e03400640 1 -- 172.21.15.114:0/2153693446 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0800a210 con 0x55c0862b01b0 2024-09-21T07:05:30.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e03400640 1 -- 172.21.15.114:0/2153693446 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f6e140aba50 con 0x55c0862b01b0 2024-09-21T07:05:30.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e2b00c740 10 client.28348 fetch_fsmap finished waiting for FSMap version 49 2024-09-21T07:05:30.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e2b00c740 10 client.28348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:30.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e2b00c740 20 client.28348 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:30.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c08626cda0 0x55c08670ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:30.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e2b00c740 4 client.28348 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:05:30.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.837+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0865f49b0 con 0x55c08626cda0 2024-09-21T07:05:30.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.838+0000 7f6e20e00640 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c08626cda0 0x55c08670ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:30.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.839+0000 7f6e20e00640 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c08626cda0 0x55c08670ab00 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feb60e54c9f97afa server_cookie=d41d6102d925fdef in_seq=0 out_seq=0 2024-09-21T07:05:30.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.839+0000 7f6e03400640 10 client.28348 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.840+0000 7f6e03400640 1 -- 172.21.15.114:0/2153693446 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0865f49b0 con 0x55c08626cda0 2024-09-21T07:05:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.840+0000 7f6e03400640 10 client.28348 handle_command_reply: tid=0 2024-09-21T07:05:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.840+0000 7f6e2b00c740 1 client.28348 shutdown 2024-09-21T07:05:30.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b01b0 msgr2=0x55c08668f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b01b0 0x55c08668f850 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6e140e4600 tx=0x7f6e140ac7e0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 shutdown_connections 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c0862d9070 0x55c0866902d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0862b8390 0x55c08668fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c08626cda0 0x55c08670ab00 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 --2- 172.21.15.114:0/2153693446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0862b01b0 0x55c08668f850 secure :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6e140e4600 tx=0x7f6e140ac7e0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 >> 172.21.15.114:0/2153693446 conn(0x55c0862cd460 msgr2=0x55c086689e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:30.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.841+0000 7f6e03400640 0 client.28348 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:30.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.842+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 shutdown_connections 2024-09-21T07:05:30.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.842+0000 7f6e2b00c740 1 -- 172.21.15.114:0/2153693446 wait complete. 2024-09-21T07:05:30.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.843+0000 7f6e2b00c740 20 client.28348 trim_cache size 0 max 16384 2024-09-21T07:05:30.844 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:05:30.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6e040b66b0 msgr2=0x7f6e040b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6e040b66b0 0x7f6e040b8bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6e1002a9b0 tx=0x7f6e10009260 comp rx=0 tx=0).stop 2024-09-21T07:05:30.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 msgr2=0x7f6e1c111dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:30.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c111dd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6e1400cd10 tx=0x7f6e1400a620 comp rx=0 tx=0).stop 2024-09-21T07:05:30.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 shutdown_connections 2024-09-21T07:05:30.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6e1c115570 0x7f6e1c1ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e1c10c940 0x7f6e1c112370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6e040b66b0 0x7f6e040b8bc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 --2- 172.21.15.114:0/235679913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6e1c10bff0 0x7f6e1c111dd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:30.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.847+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 >> 172.21.15.114:0/235679913 conn(0x7f6e1c071310 msgr2=0x7f6e1c11b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:30.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.848+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 shutdown_connections 2024-09-21T07:05:30.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:30.848+0000 7f6e24200640 1 -- 172.21.15.114:0/235679913 wait complete. 2024-09-21T07:05:31.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:30 smithi114 ceph-mon[30654]: pgmap v745: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 6.2 MiB/s wr, 466 op/s 2024-09-21T07:05:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:30 smithi183 ceph-mon[34219]: pgmap v745: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 6.2 MiB/s wr, 466 op/s 2024-09-21T07:05:31.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:30 smithi193 ceph-mon[34601]: pgmap v745: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 6.2 MiB/s wr, 466 op/s 2024-09-21T07:05:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:31 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:05:32.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:31 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:05:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:31 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:05:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 655.0 s, 463.4 tps, lat 208.131 ms stddev 247.550 2024-09-21T07:05:33.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:05:33.074 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:05:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:32 smithi114 ceph-mon[30654]: pgmap v746: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.2 MiB/s wr, 397 op/s 2024-09-21T07:05:33.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:32 smithi183 ceph-mon[34219]: pgmap v746: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.2 MiB/s wr, 397 op/s 2024-09-21T07:05:33.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:32 smithi193 ceph-mon[34601]: pgmap v746: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 5.2 MiB/s wr, 397 op/s 2024-09-21T07:05:33.772 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:35.314 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:35 smithi114 ceph-mon[30654]: pgmap v747: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 7.2 MiB/s wr, 529 op/s 2024-09-21T07:05:35.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:35 smithi183 ceph-mon[34219]: pgmap v747: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 7.2 MiB/s wr, 529 op/s 2024-09-21T07:05:35.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:35 smithi193 ceph-mon[34601]: pgmap v747: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 7.2 MiB/s wr, 529 op/s 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.584+0000 7f627d800640 1 -- 172.21.15.114:0/1547042772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278075dd0 msgr2=0x7f62780761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.584+0000 7f627d800640 1 --2- 172.21.15.114:0/1547042772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278075dd0 0x7f62780761e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f626402a440 comp rx=0 tx=0).stop 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.584+0000 7f627d800640 1 -- 172.21.15.114:0/1547042772 shutdown_connections 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.584+0000 7f627d800640 1 --2- 172.21.15.114:0/1547042772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278114280 0x7f6278116750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.584+0000 7f627d800640 1 --2- 172.21.15.114:0/1547042772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6278075dd0 0x7f62780761e0 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6264009900 tx=0x7f626402a440 comp rx=0 tx=0).stop 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.585+0000 7f627d800640 1 --2- 172.21.15.114:0/1547042772 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6278077380 0x7f62780757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.585+0000 7f627d800640 1 -- 172.21.15.114:0/1547042772 >> 172.21.15.114:0/1547042772 conn(0x7f62780ff440 msgr2=0x7f6278101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:36.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.586+0000 7f627d800640 1 -- 172.21.15.114:0/1547042772 shutdown_connections 2024-09-21T07:05:36.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.587+0000 7f627d800640 1 -- 172.21.15.114:0/1547042772 wait complete. 2024-09-21T07:05:36.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.589+0000 7f627d800640 1 Processor -- start 2024-09-21T07:05:36.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.589+0000 7f627d800640 1 -- start start 2024-09-21T07:05:36.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.591+0000 7f627d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278077380 0x7f6278113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.591+0000 7f627d800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6278114280 0x7f627810ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.592+0000 7f6277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278077380 0x7f6278113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.592+0000 7f6277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278077380 0x7f6278113bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37442/0 (socket says 172.21.15.114:37442) 2024-09-21T07:05:36.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.592+0000 7f6277e00640 1 -- 172.21.15.114:0/3549630268 learned_addr learned my addr 172.21.15.114:0/3549630268 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:36.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.593+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f627810d480 0x7f62781127a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.593+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6278119f10 con 0x7f6278114280 2024-09-21T07:05:36.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.593+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6278119d90 con 0x7f6278077380 2024-09-21T07:05:36.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.593+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f627811a090 con 0x7f627810d480 2024-09-21T07:05:36.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.594+0000 7f627ce00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f627810d480 0x7f62781127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.594+0000 7f6277400640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6278114280 0x7f627810ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.594+0000 7f627ce00640 1 -- 172.21.15.114:0/3549630268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278077380 msgr2=0x7f6278113bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.594+0000 7f627ce00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278077380 0x7f6278113bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.594+0000 7f627ce00640 1 -- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6278114280 msgr2=0x7f627810ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.596+0000 7f627ce00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6278114280 0x7f627810ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.596+0000 7f627ce00640 1 -- 172.21.15.114:0/3549630268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627810dbd0 con 0x7f627810d480 2024-09-21T07:05:36.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.597+0000 7f627ce00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f627810d480 0x7f62781127a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f627000ec50 tx=0x7f627000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:36.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.597+0000 7f6274c00640 1 -- 172.21.15.114:0/3549630268 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270003b70 con 0x7f627810d480 2024-09-21T07:05:36.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.599+0000 7f6274c00640 1 -- 172.21.15.114:0/3549630268 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6270003d10 con 0x7f627810d480 2024-09-21T07:05:36.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.602+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6278112da0 con 0x7f627810d480 2024-09-21T07:05:36.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.602+0000 7f6274c00640 1 -- 172.21.15.114:0/3549630268 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700107f0 con 0x7f627810d480 2024-09-21T07:05:36.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.602+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6278113260 con 0x7f627810d480 2024-09-21T07:05:36.612 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.609+0000 7f6274c00640 1 -- 172.21.15.114:0/3549630268 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6270004000 con 0x7f627810d480 2024-09-21T07:05:36.613 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.612+0000 7f6274c00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f625c0b67b0 0x7f625c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.612+0000 7f6274c00640 1 -- 172.21.15.114:0/3549630268 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f62700e8250 con 0x7f627810d480 2024-09-21T07:05:36.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.612+0000 7f6285ac7740 1 Processor -- start 2024-09-21T07:05:36.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.613+0000 7f6277e00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f625c0b67b0 0x7f625c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.613+0000 7f6285ac7740 1 -- start start 2024-09-21T07:05:36.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.614+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065d5350 0x56060697cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x56060697d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065b4670 0x56060697d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.616 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6285ac7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5606068f2e90 con 0x5606065ac490 2024-09-21T07:05:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6285ac7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5606068e22c0 con 0x5606065d5350 2024-09-21T07:05:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6285ac7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606068e60d0 con 0x5606065b4670 2024-09-21T07:05:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065d5350 0x56060697cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065d5350 0x56060697cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37448/0 (socket says 172.21.15.114:37448) 2024-09-21T07:05:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.615+0000 7f6277e00640 1 -- 172.21.15.114:0/3652390612 learned_addr learned my addr 172.21.15.114:0/3652390612 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:36.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.616+0000 7f627ce00640 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065b4670 0x56060697d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.616+0000 7f6277400640 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x56060697d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.617+0000 7f627ce00640 1 -- 172.21.15.114:0/3652390612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065d5350 msgr2=0x56060697cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.617+0000 7f627ce00640 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065d5350 0x56060697cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.617+0000 7f627ce00640 1 -- 172.21.15.114:0/3652390612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 msgr2=0x56060697d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.617+0000 7f627ce00640 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x56060697d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.617+0000 7f627ce00640 1 -- 172.21.15.114:0/3652390612 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560606640930 con 0x5606065b4670 2024-09-21T07:05:36.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.618+0000 7f6277e00640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f625c0b67b0 0x7f625c0b8cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f62680129d0 tx=0x7f6268012550 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.618+0000 7f627ce00640 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065b4670 0x56060697d980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f62700e5570 tx=0x7f62700adb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ea13ea5fdd33f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:36.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.619+0000 7f626d600640 1 -- 172.21.15.114:0/3652390612 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700ad780 con 0x5606065b4670 2024-09-21T07:05:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.619+0000 7f626d600640 1 -- 172.21.15.114:0/3652390612 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62700ae370 con 0x5606065b4670 2024-09-21T07:05:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.619+0000 7f626d600640 1 -- 172.21.15.114:0/3652390612 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700b7690 con 0x5606065b4670 2024-09-21T07:05:36.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.620+0000 7f6285ac7740 1 -- 172.21.15.114:0/3652390612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065b4670 msgr2=0x56060697d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.620+0000 7f6285ac7740 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065b4670 0x56060697d980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f62700e5570 tx=0x7f62700adb60 comp rx=0 tx=0).stop 2024-09-21T07:05:36.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 -- 172.21.15.114:0/3652390612 shutdown_connections 2024-09-21T07:05:36.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065b4670 0x56060697d980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x56060697d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 --2- 172.21.15.114:0/3652390612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065d5350 0x56060697cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 -- 172.21.15.114:0/3652390612 >> 172.21.15.114:0/3652390612 conn(0x5606065c9740 msgr2=0x56060697adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 -- 172.21.15.114:0/3652390612 shutdown_connections 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.622+0000 7f6285ac7740 1 -- 172.21.15.114:0/3652390612 wait complete. 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.623+0000 7f6285ac7740 1 -- start start 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.623+0000 7f6285ac7740 1 Processor -- start 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x560606988b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 0x560606994970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x560606743890 con 0x5606065ac490 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56060687d160 con 0x5606065b4670 2024-09-21T07:05:36.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56060698b680 con 0x5606065d5350 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.624+0000 7f6285ac7740 10 client.0 _ll_register_callbacks cb 0x560606899a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f6277400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 0x560606994970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f6277400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069890c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37458/0 (socket says 172.21.15.114:37458) 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 0x560606994970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43774/0 (socket says 172.21.15.114:43774) 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f6277400640 1 -- 172.21.15.114:0/2448031079 learned_addr learned my addr 172.21.15.114:0/2448031079 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f6277e00640 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x560606988b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 -- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 msgr2=0x5606069890c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069890c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 -- 172.21.15.114:0/2448031079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 msgr2=0x560606988b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x560606988b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.625+0000 7f627ce00640 1 -- 172.21.15.114:0/2448031079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560606994fb0 con 0x5606065d5350 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.626+0000 7f627ce00640 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 0x560606994970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62700ad920 tx=0x7f62700e5a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff182bb29b5d128b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:36.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.626+0000 7f6262000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:36.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.626+0000 7f6262000640 1 -- 172.21.15.114:0/2448031079 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700ad780 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.626+0000 7f6262000640 1 -- 172.21.15.114:0/2448031079 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f62700aced0 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.627+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62700ae370 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.627+0000 7f6262000640 1 -- 172.21.15.114:0/2448031079 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f62700950b0 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.627+0000 7f6285ac7740 10 client.28360 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.627+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5606069978e0 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.627+0000 7f6262000640 1 -- 172.21.15.114:0/2448031079 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62700ae370 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.627+0000 7f6262000640 1 -- 172.21.15.114:0/2448031079 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7f62700b21c0 con 0x5606065d5350 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.628+0000 7f6285ac7740 10 client.28360 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.628+0000 7f6285ac7740 10 client.28360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.628+0000 7f6285ac7740 20 client.28360 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.628+0000 7f6285ac7740 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560606593cd0 0x56060699ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.628+0000 7f6285ac7740 4 client.28360 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.628+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606068f4de0 con 0x560606593cd0 2024-09-21T07:05:36.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.629+0000 7f6277e00640 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560606593cd0 0x56060699ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.633+0000 7f6262000640 10 client.28360 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:36.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.633+0000 7f6277e00640 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560606593cd0 0x56060699ac40 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e482a7e4eb1830 server_cookie=471708fcd6431e01 in_seq=0 out_seq=0 2024-09-21T07:05:36.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.634+0000 7f6262000640 1 -- 172.21.15.114:0/2448031079 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606068f4de0 con 0x560606593cd0 2024-09-21T07:05:36.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.634+0000 7f6262000640 10 client.28360 handle_command_reply: tid=0 2024-09-21T07:05:36.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.636+0000 7f6285ac7740 1 client.28360 shutdown 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 msgr2=0x560606994970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 0x560606994970 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62700ad920 tx=0x7f62700e5a20 comp rx=0 tx=0).stop 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 shutdown_connections 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x560606593cd0 0x56060699ac40 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065d5350 0x560606994970 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6262000640 0 client.28360 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 --2- 172.21.15.114:0/2448031079 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065ac490 0x560606988b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 >> 172.21.15.114:0/2448031079 conn(0x5606065c9740 msgr2=0x5606069800e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:36.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.637+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 shutdown_connections 2024-09-21T07:05:36.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.638+0000 7f6285ac7740 1 -- 172.21.15.114:0/2448031079 wait complete. 2024-09-21T07:05:36.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.640+0000 7f6285ac7740 20 client.28360 trim_cache size 0 max 16384 2024-09-21T07:05:36.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.676+0000 7f6285ac7740 1 Processor -- start 2024-09-21T07:05:36.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.676+0000 7f6285ac7740 1 -- start start 2024-09-21T07:05:36.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065c9740 0x5606069948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065ac490 0x560606982720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6285ac7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f62781c02b0 con 0x5606065c9740 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6285ac7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62781c1df0 con 0x5606065b4670 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6285ac7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62781c0980 con 0x5606065ac490 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f627ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f627ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069881b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37474/0 (socket says 172.21.15.114:37474) 2024-09-21T07:05:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f627ce00640 1 -- 172.21.15.114:0/1460373053 learned_addr learned my addr 172.21.15.114:0/1460373053 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.677+0000 7f6277400640 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065ac490 0x560606982720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.678+0000 7f627ce00640 1 -- 172.21.15.114:0/1460373053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065ac490 msgr2=0x560606982720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.678+0000 7f627ce00640 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065ac490 0x560606982720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.678+0000 7f627ce00640 1 -- 172.21.15.114:0/1460373053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065c9740 msgr2=0x5606069948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.678+0000 7f627ce00640 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065c9740 0x5606069948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.678+0000 7f627ce00640 1 -- 172.21.15.114:0/1460373053 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56060697f1f0 con 0x5606065b4670 2024-09-21T07:05:36.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.678+0000 7f627ce00640 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069881b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f62700ad970 tx=0x7f627000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7bc1012d8992826 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:36.682 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.681+0000 7f626cc00640 1 -- 172.21.15.114:0/1460373053 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700c7020 con 0x5606065b4670 2024-09-21T07:05:36.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.681+0000 7f626cc00640 1 -- 172.21.15.114:0/1460373053 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62700ae370 con 0x5606065b4670 2024-09-21T07:05:36.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.681+0000 7f626cc00640 1 -- 172.21.15.114:0/1460373053 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62700d5200 con 0x5606065b4670 2024-09-21T07:05:36.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.682+0000 7f6285ac7740 1 -- 172.21.15.114:0/1460373053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 msgr2=0x5606069881b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.684 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.682+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069881b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f62700ad970 tx=0x7f627000cf90 comp rx=0 tx=0).stop 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 -- 172.21.15.114:0/1460373053 shutdown_connections 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065b4670 0x5606069881b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065ac490 0x560606982720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1460373053 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065c9740 0x5606069948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 -- 172.21.15.114:0/1460373053 >> 172.21.15.114:0/1460373053 conn(0x560606983310 msgr2=0x560606982010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 -- 172.21.15.114:0/1460373053 shutdown_connections 2024-09-21T07:05:36.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.683+0000 7f6285ac7740 1 -- 172.21.15.114:0/1460373053 wait complete. 2024-09-21T07:05:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.684+0000 7f6285ac7740 1 -- start start 2024-09-21T07:05:36.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.684+0000 7f6285ac7740 1 Processor -- start 2024-09-21T07:05:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.685+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 0x56060699bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.685+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065b4670 0x56060699c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6285ac7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065c9740 0x56060699c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6285ac7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f62781c1720 con 0x5606065b4670 2024-09-21T07:05:36.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6285ac7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62781c1050 con 0x5606065ac490 2024-09-21T07:05:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6285ac7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62781be2b0 con 0x5606065c9740 2024-09-21T07:05:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6285ac7740 10 client.0 _ll_register_callbacks cb 0x7f62700093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 0x56060699bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 0x56060699bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37478/0 (socket says 172.21.15.114:37478) 2024-09-21T07:05:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.686+0000 7f6277e00640 1 -- 172.21.15.114:0/1558640829 learned_addr learned my addr 172.21.15.114:0/1558640829 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:36.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f6277400640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065b4670 0x56060699c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f627ce00640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065c9740 0x56060699c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f6277e00640 1 -- 172.21.15.114:0/1558640829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065c9740 msgr2=0x56060699c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f6277e00640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065c9740 0x56060699c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f6277e00640 1 -- 172.21.15.114:0/1558640829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065b4670 msgr2=0x56060699c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f6277e00640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065b4670 0x56060699c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.687+0000 7f6277e00640 1 -- 172.21.15.114:0/1558640829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560606a043c0 con 0x5606065ac490 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.688+0000 7f6277e00640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 0x56060699bc30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6268012470 tx=0x7f6268004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9289e2c6eaec969 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:36.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.688+0000 7f6262000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6262000640 1 -- 172.21.15.114:0/1558640829 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268020b00 con 0x5606065ac490 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6262000640 1 -- 172.21.15.114:0/1558640829 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f6268020ca0 con 0x5606065ac490 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62700ae370 con 0x5606065ac490 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6262000640 1 -- 172.21.15.114:0/1558640829 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6268035050 con 0x5606065ac490 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6285ac7740 10 client.28415 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560606a055f0 con 0x5606065ac490 2024-09-21T07:05:36.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.689+0000 7f6262000640 1 -- 172.21.15.114:0/1558640829 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62700ae370 con 0x5606065ac490 2024-09-21T07:05:36.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6262000640 1 -- 172.21.15.114:0/1558640829 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7f6268030030 con 0x5606065ac490 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6285ac7740 10 client.28415 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6285ac7740 10 client.28415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6285ac7740 20 client.28415 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606065d5350 0x560606a07de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6285ac7740 4 client.28415 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5606068f0890 con 0x5606065d5350 2024-09-21T07:05:36.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.690+0000 7f6277400640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606065d5350 0x560606a07de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:36.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.693+0000 7f6277400640 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606065d5350 0x560606a07de0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0ddee65313261b server_cookie=9036676386a42cbb in_seq=0 out_seq=0 2024-09-21T07:05:36.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.693+0000 7f6262000640 10 client.28415 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:36.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6262000640 1 -- 172.21.15.114:0/1558640829 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5606068f0890 con 0x5606065d5350 2024-09-21T07:05:36.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6262000640 10 client.28415 handle_command_reply: tid=0 2024-09-21T07:05:36.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 client.28415 shutdown 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 msgr2=0x56060699bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 0x56060699bc30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6268012470 tx=0x7f6268004730 comp rx=0 tx=0).stop 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 shutdown_connections 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606065d5350 0x560606a07de0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606065c9740 0x56060699c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606065b4670 0x56060699c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 --2- 172.21.15.114:0/1558640829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606065ac490 0x56060699bc30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 >> 172.21.15.114:0/1558640829 conn(0x560606983310 msgr2=0x560606989dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:36.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6262000640 0 client.28415 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.695+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 shutdown_connections 2024-09-21T07:05:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.696+0000 7f6285ac7740 1 -- 172.21.15.114:0/1558640829 wait complete. 2024-09-21T07:05:36.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.696+0000 7f6285ac7740 20 client.28415 trim_cache size 0 max 16384 2024-09-21T07:05:36.698 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:05:36.698 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "4d99c123-582b-4c1f-afb2-2a0a31e59673", 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f625c0b67b0 msgr2=0x7f625c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f625c0b67b0 0x7f625c0b8cc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f62680129d0 tx=0x7f6268012550 comp rx=0 tx=0).stop 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f627810d480 msgr2=0x7f62781127a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:36.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f627810d480 0x7f62781127a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f627000ec50 tx=0x7f627000c6a0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 shutdown_connections 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f627810d480 0x7f62781127a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6278114280 0x7f627810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f625c0b67b0 0x7f625c0b8cc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 --2- 172.21.15.114:0/3549630268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6278077380 0x7f6278113bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 >> 172.21.15.114:0/3549630268 conn(0x7f62780ff440 msgr2=0x7f62781033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.697+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 shutdown_connections 2024-09-21T07:05:36.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:36.698+0000 7f627d800640 1 -- 172.21.15.114:0/3549630268 wait complete. 2024-09-21T07:05:37.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: pgmap v748: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.7 MiB/s wr, 420 op/s 2024-09-21T07:05:37.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:05:37.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:37.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:37.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:37.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:37.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:37 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: pgmap v748: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.7 MiB/s wr, 420 op/s 2024-09-21T07:05:37.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:05:37.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:37.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:37.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:37.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:37.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:37 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: pgmap v748: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.7 MiB/s wr, 420 op/s 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:37.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:37 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 660.0 s, 505.2 tps, lat 204.070 ms stddev 257.403 2024-09-21T07:05:38.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:38 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:38.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:05:38.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:38 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:38.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:38.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:38.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:38.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:05:38.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:38 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:38.736 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 -- 172.21.15.114:0/2903780673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 msgr2=0x7ff8ac115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 --2- 172.21.15.114:0/2903780673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 0x7ff8ac115fd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff894009860 tx=0x7ff89402a380 comp rx=0 tx=0).stop 2024-09-21T07:05:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 -- 172.21.15.114:0/2903780673 shutdown_connections 2024-09-21T07:05:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 --2- 172.21.15.114:0/2903780673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ac116510 0x7ff8ac118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 --2- 172.21.15.114:0/2903780673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 0x7ff8ac115fd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 --2- 172.21.15.114:0/2903780673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac10a3c0 0x7ff8ac10a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.953+0000 7ff8ab200640 1 -- 172.21.15.114:0/2903780673 >> 172.21.15.114:0/2903780673 conn(0x7ff8ac070c00 msgr2=0x7ff8ac071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.954+0000 7ff8ab200640 1 -- 172.21.15.114:0/2903780673 shutdown_connections 2024-09-21T07:05:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.954+0000 7ff8ab200640 1 -- 172.21.15.114:0/2903780673 wait complete. 2024-09-21T07:05:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.957+0000 7ff8ab200640 1 Processor -- start 2024-09-21T07:05:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.957+0000 7ff8ab200640 1 -- start start 2024-09-21T07:05:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.958+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ac10a3c0 0x7ff8ac1a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.958+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 0x7ff8ac1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8ab200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 0x7ff8ac1a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ac10a3c0 0x7ff8ac1a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8ab200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac11c060 con 0x7ff8ac10a3c0 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8ab200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac11bee0 con 0x7ff8ac10ad10 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8ab200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac11c1e0 con 0x7ff8ac116510 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8a9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 0x7ff8ac1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8aa800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 0x7ff8ac1a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8aa800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 0x7ff8ac1a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43798/0 (socket says 172.21.15.114:43798) 2024-09-21T07:05:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.959+0000 7ff8aa800640 1 -- 172.21.15.114:0/3638303042 learned_addr learned my addr 172.21.15.114:0/3638303042 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.960+0000 7ff8aa800640 1 -- 172.21.15.114:0/3638303042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 msgr2=0x7ff8ac1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.960+0000 7ff8aa800640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 0x7ff8ac1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.960+0000 7ff8aa800640 1 -- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ac10a3c0 msgr2=0x7ff8ac1a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.960+0000 7ff8aa800640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ac10a3c0 0x7ff8ac1a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.960+0000 7ff8aa800640 1 -- 172.21.15.114:0/3638303042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac1a87d0 con 0x7ff8ac116510 2024-09-21T07:05:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.960+0000 7ff8aa800640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 0x7ff8ac1a1fb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fd0 tx=0x7ff89c00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.961+0000 7ff8a2a00640 1 -- 172.21.15.114:0/3638303042 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c011830 con 0x7ff8ac116510 2024-09-21T07:05:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.961+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac1a8ac0 con 0x7ff8ac116510 2024-09-21T07:05:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.961+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1a9060 con 0x7ff8ac116510 2024-09-21T07:05:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.961+0000 7ff8a2a00640 1 -- 172.21.15.114:0/3638303042 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff89c016070 con 0x7ff8ac116510 2024-09-21T07:05:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.961+0000 7ff8a2a00640 1 -- 172.21.15.114:0/3638303042 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c010de0 con 0x7ff8ac116510 2024-09-21T07:05:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.966+0000 7ff8a2a00640 1 -- 172.21.15.114:0/3638303042 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff89c01b460 con 0x7ff8ac116510 2024-09-21T07:05:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.967+0000 7ff8b7aa0740 1 Processor -- start 2024-09-21T07:05:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.969+0000 7ff8a2a00640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8900b6700 0x7ff8900b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.970+0000 7ff8a2a00640 1 -- 172.21.15.114:0/3638303042 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7ff89c0a9c90 con 0x7ff8ac116510 2024-09-21T07:05:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.970+0000 7ff8a9400640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8900b6700 0x7ff8900b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.970+0000 7ff8b7aa0740 1 -- start start 2024-09-21T07:05:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d881e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d882350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d882890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.974 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56542d7f76b0 con 0x56542d4bc4d0 2024-09-21T07:05:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56542d7e6ae0 con 0x56542d4dd1b0 2024-09-21T07:05:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56542d7ea8f0 con 0x56542d4b42f0 2024-09-21T07:05:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d881e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d881e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37500/0 (socket says 172.21.15.114:37500) 2024-09-21T07:05:38.975 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.971+0000 7ff8a9e00640 1 -- 172.21.15.114:0/4026651628 learned_addr learned my addr 172.21.15.114:0/4026651628 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d882350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8900b6700 0x7ff8900b8c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff894009780 tx=0x7ff89400b040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8aa800640 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d882890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 -- 172.21.15.114:0/4026651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 msgr2=0x56542d881e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:38.976 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d881e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.977 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 -- 172.21.15.114:0/4026651628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 msgr2=0x56542d882890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d882890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.972+0000 7ff8a9400640 1 -- 172.21.15.114:0/4026651628 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56542d7b3780 con 0x56542d4b42f0 2024-09-21T07:05:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.973+0000 7ff8a9400640 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d882350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac06c430 tx=0x7ff894041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=209aa9851174b430 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:38.981 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.973+0000 7ff89b000640 1 -- 172.21.15.114:0/4026651628 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894042070 con 0x56542d4b42f0 2024-09-21T07:05:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.973+0000 7ff89b000640 1 -- 172.21.15.114:0/4026651628 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8940035f0 con 0x56542d4b42f0 2024-09-21T07:05:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.973+0000 7ff89b000640 1 -- 172.21.15.114:0/4026651628 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894037040 con 0x56542d4b42f0 2024-09-21T07:05:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.975+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/4026651628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 msgr2=0x56542d882350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.975+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d882350 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac06c430 tx=0x7ff894041000 comp rx=0 tx=0).stop 2024-09-21T07:05:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.975+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/4026651628 shutdown_connections 2024-09-21T07:05:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.975+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d882890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.975+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d882350 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.975+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/4026651628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d881e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.976+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/4026651628 >> 172.21.15.114:0/4026651628 conn(0x56542d4d1300 msgr2=0x56542d87fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.976+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/4026651628 shutdown_connections 2024-09-21T07:05:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.976+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/4026651628 wait complete. 2024-09-21T07:05:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.977+0000 7ff8b7aa0740 1 -- start start 2024-09-21T07:05:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.977+0000 7ff8b7aa0740 1 Processor -- start 2024-09-21T07:05:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d88d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d88d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d88d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d88d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43814/0 (socket says 172.21.15.114:43814) 2024-09-21T07:05:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d88de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56542d71eab0 con 0x56542d4bc4d0 2024-09-21T07:05:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56542d78d570 con 0x56542d4dd1b0 2024-09-21T07:05:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56542d890530 con 0x56542d4b42f0 2024-09-21T07:05:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 -- 172.21.15.114:0/1497338345 learned_addr learned my addr 172.21.15.114:0/1497338345 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 -- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 msgr2=0x56542d88de50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:05:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d88de50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 -- 172.21.15.114:0/1497338345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 msgr2=0x56542d88d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d88d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 -- 172.21.15.114:0/1497338345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56542d548790 con 0x56542d4b42f0 2024-09-21T07:05:38.989 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.978+0000 7ff8b7aa0740 10 client.0 _ll_register_callbacks cb 0x56542d79c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8940035f0 con 0x56542d4b42f0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d88d3d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4010bc0 tx=0x7ff8a400e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0adc8b1e013e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff88f400640 10 client.28378 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.979+0000 7ff88f400640 1 -- 172.21.15.114:0/1497338345 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a401e060 con 0x56542d4b42f0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.980+0000 7ff88f400640 1 -- 172.21.15.114:0/1497338345 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7ff8a400c5f0 con 0x56542d4b42f0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.980+0000 7ff88f400640 1 -- 172.21.15.114:0/1497338345 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff8a400c810 con 0x56542d4b42f0 2024-09-21T07:05:38.990 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.980+0000 7ff8b7aa0740 10 client.28378 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.980+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=50}) -- 0x56542d5c1620 con 0x56542d4b42f0 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff88f400640 1 -- 172.21.15.114:0/1497338345 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7ff8a400b040 con 0x56542d4b42f0 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff8b7aa0740 10 client.28378 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff8b7aa0740 10 client.28378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff8b7aa0740 20 client.28378 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d49bd80 0x56542d89f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff8b7aa0740 4 client.28378 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.981+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56542d7f9600 con 0x56542d49bd80 2024-09-21T07:05:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.982+0000 7ff8a9400640 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d49bd80 0x56542d89f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.984+0000 7ff8a9400640 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d49bd80 0x56542d89f430 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546c82dd6e5689b9 server_cookie=32f986246bc773f in_seq=0 out_seq=0 2024-09-21T07:05:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.984+0000 7ff88f400640 10 client.28378 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.985+0000 7ff88f400640 1 -- 172.21.15.114:0/1497338345 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56542d7f9600 con 0x56542d49bd80 2024-09-21T07:05:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.985+0000 7ff88f400640 10 client.28378 handle_command_reply: tid=0 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.985+0000 7ff8b7aa0740 1 client.28378 shutdown 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 msgr2=0x56542d88d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d88d3d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4010bc0 tx=0x7ff8a400e7c0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 shutdown_connections 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4dd1b0 0x56542d88de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4bc4d0 0x56542d88d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d49bd80 0x56542d89f430 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/1497338345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d88d3d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 >> 172.21.15.114:0/1497338345 conn(0x56542d4d1300 msgr2=0x56542d882e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.986+0000 7ff88f400640 0 client.28378 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.987+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 shutdown_connections 2024-09-21T07:05:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.987+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/1497338345 wait complete. 2024-09-21T07:05:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:38.987+0000 7ff8b7aa0740 20 client.28378 trim_cache size 0 max 16384 2024-09-21T07:05:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.007+0000 7ff8b7aa0740 1 Processor -- start 2024-09-21T07:05:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.007+0000 7ff8b7aa0740 1 -- start start 2024-09-21T07:05:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d87ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d883550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d88a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff88800ffa0 con 0x56542d4dd1b0 2024-09-21T07:05:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff888011ae0 con 0x56542d4bc4d0 2024-09-21T07:05:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff888010670 con 0x56542d4b42f0 2024-09-21T07:05:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d87ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d87ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43822/0 (socket says 172.21.15.114:43822) 2024-09-21T07:05:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.008+0000 7ff8a9e00640 1 -- 172.21.15.114:0/734196439 learned_addr learned my addr 172.21.15.114:0/734196439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8aa800640 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d88a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9400640 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d883550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9e00640 1 -- 172.21.15.114:0/734196439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 msgr2=0x56542d883550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d883550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9e00640 1 -- 172.21.15.114:0/734196439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 msgr2=0x56542d88a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d88a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9e00640 1 -- 172.21.15.114:0/734196439 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56542d884c20 con 0x56542d4b42f0 2024-09-21T07:05:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d87ed20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4002230 tx=0x7ff8a4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18bea3dd34f61190 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff89a600640 1 -- 172.21.15.114:0/734196439 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4003970 con 0x56542d4b42f0 2024-09-21T07:05:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff89a600640 1 -- 172.21.15.114:0/734196439 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8940035f0 con 0x56542d4b42f0 2024-09-21T07:05:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.009+0000 7ff89a600640 1 -- 172.21.15.114:0/734196439 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a401a450 con 0x56542d4b42f0 2024-09-21T07:05:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/734196439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 msgr2=0x56542d87ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d87ed20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4002230 tx=0x7ff8a4016000 comp rx=0 tx=0).stop 2024-09-21T07:05:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/734196439 shutdown_connections 2024-09-21T07:05:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d88a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d883550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/734196439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d87ed20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.021 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/734196439 >> 172.21.15.114:0/734196439 conn(0x56542d4d1300 msgr2=0x56542d882e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/734196439 shutdown_connections 2024-09-21T07:05:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.011+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/734196439 wait complete. 2024-09-21T07:05:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.012+0000 7ff8b7aa0740 1 -- start start 2024-09-21T07:05:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.012+0000 7ff8b7aa0740 1 Processor -- start 2024-09-21T07:05:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.012+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d8960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.012+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d893590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8b7aa0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d893be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff888011410 con 0x56542d4dd1b0 2024-09-21T07:05:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff888010d40 con 0x56542d4bc4d0 2024-09-21T07:05:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8b7aa0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff88800dfa0 con 0x56542d4b42f0 2024-09-21T07:05:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8b7aa0740 10 client.0 _ll_register_callbacks cb 0x7ff8a4009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d8960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8a9e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d8960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43824/0 (socket says 172.21.15.114:43824) 2024-09-21T07:05:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8a9e00640 1 -- 172.21.15.114:0/3431481596 learned_addr learned my addr 172.21.15.114:0/3431481596 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8a9400640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d893590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.013+0000 7ff8aa800640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d893be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9400640 1 -- 172.21.15.114:0/3431481596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 msgr2=0x56542d8960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9400640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d8960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9400640 1 -- 172.21.15.114:0/3431481596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 msgr2=0x56542d893be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9400640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d893be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9400640 1 -- 172.21.15.114:0/3431481596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56542d894370 con 0x56542d4bc4d0 2024-09-21T07:05:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d8960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8a9400640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d893590 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff89404a040 tx=0x7ff894030c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6145eac0e558092a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff88f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff88f400640 1 -- 172.21.15.114:0/3431481596 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894042030 con 0x56542d4bc4d0 2024-09-21T07:05:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff88f400640 1 -- 172.21.15.114:0/3431481596 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7ff8940039a0 con 0x56542d4bc4d0 2024-09-21T07:05:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.014+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8940035f0 con 0x56542d4bc4d0 2024-09-21T07:05:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.015+0000 7ff88f400640 1 -- 172.21.15.114:0/3431481596 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff894068050 con 0x56542d4bc4d0 2024-09-21T07:05:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.015+0000 7ff8b7aa0740 10 client.28427 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.015+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56542d909f00 con 0x56542d4bc4d0 2024-09-21T07:05:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.015+0000 7ff88f400640 1 -- 172.21.15.114:0/3431481596 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff894003ba0 con 0x56542d4bc4d0 2024-09-21T07:05:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff88f400640 1 -- 172.21.15.114:0/3431481596 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7ff894063030 con 0x56542d4bc4d0 2024-09-21T07:05:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8b7aa0740 10 client.28427 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8b7aa0740 10 client.28427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8b7aa0740 20 client.28427 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d4711c0 0x56542d90c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8b7aa0740 4 client.28427 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:05:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff8940035f0 con 0x56542d4711c0 2024-09-21T07:05:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.016+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d4711c0 0x56542d90c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.017+0000 7ff88f400640 10 client.28427 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.017+0000 7ff8a9e00640 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d4711c0 0x56542d90c310 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4bc15b67d87712 server_cookie=4e8c1e1ee7ebe921 in_seq=0 out_seq=0 2024-09-21T07:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff88f400640 1 -- 172.21.15.114:0/3431481596 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff8940035f0 con 0x56542d4711c0 2024-09-21T07:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff88f400640 10 client.28427 handle_command_reply: tid=0 2024-09-21T07:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 client.28427 shutdown 2024-09-21T07:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 msgr2=0x56542d893590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d893590 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff89404a040 tx=0x7ff894030c00 comp rx=0 tx=0).stop 2024-09-21T07:05:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 shutdown_connections 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56542d4dd1b0 0x56542d893be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56542d4bc4d0 0x56542d893590 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56542d4711c0 0x56542d90c310 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 --2- 172.21.15.114:0/3431481596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56542d4b42f0 0x56542d8960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 >> 172.21.15.114:0/3431481596 conn(0x56542d4d1300 msgr2=0x56542d87eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.037+0000 7ff88f400640 0 client.28427 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.038+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 shutdown_connections 2024-09-21T07:05:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.038+0000 7ff8b7aa0740 1 -- 172.21.15.114:0/3431481596 wait complete. 2024-09-21T07:05:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.038+0000 7ff8b7aa0740 20 client.28427 trim_cache size 0 max 16384 2024-09-21T07:05:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:05:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:05:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:05:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:05:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8900b6700 msgr2=0x7ff8900b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8900b6700 0x7ff8900b8c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff894009780 tx=0x7ff89400b040 comp rx=0 tx=0).stop 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 msgr2=0x7ff8ac1a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 0x7ff8ac1a1fb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009fd0 tx=0x7ff89c00b8e0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 shutdown_connections 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8ac116510 0x7ff8ac1a1fb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff8900b6700 0x7ff8900b8c10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8ac10ad10 0x7ff8ac1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 --2- 172.21.15.114:0/3638303042 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff8ac10a3c0 0x7ff8ac1a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 >> 172.21.15.114:0/3638303042 conn(0x7ff8ac070c00 msgr2=0x7ff8ac073b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 shutdown_connections 2024-09-21T07:05:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:39.039+0000 7ff8ab200640 1 -- 172.21.15.114:0/3638303042 wait complete. 2024-09-21T07:05:39.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:39 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:39.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:39 smithi114 ceph-mon[30654]: pgmap v749: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 6.9 MiB/s wr, 542 op/s 2024-09-21T07:05:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:39 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:39.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:39 smithi183 ceph-mon[34219]: pgmap v749: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 6.9 MiB/s wr, 542 op/s 2024-09-21T07:05:39.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:39 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:05:39.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:39 smithi193 ceph-mon[34601]: pgmap v749: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 6.9 MiB/s wr, 542 op/s 2024-09-21T07:05:39.441 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:05:40.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.887+0000 7f791b000640 1 -- 172.21.15.114:0/3667471061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410f8c0 msgr2=0x7f791407bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:40.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.887+0000 7f791b000640 1 --2- 172.21.15.114:0/3667471061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410f8c0 0x7f791407bf20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f790c009ca0 tx=0x7f790c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.887+0000 7f791b000640 1 -- 172.21.15.114:0/3667471061 shutdown_connections 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.888+0000 7f791b000640 1 --2- 172.21.15.114:0/3667471061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410f8c0 0x7f791407bf20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.888+0000 7f791b000640 1 --2- 172.21.15.114:0/3667471061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410fd10 0x7f7914110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.888+0000 7f791b000640 1 --2- 172.21.15.114:0/3667471061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410b8e0 0x7f791410bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.888+0000 7f791b000640 1 -- 172.21.15.114:0/3667471061 >> 172.21.15.114:0/3667471061 conn(0x7f791406e9f0 msgr2=0x7f791406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.888+0000 7f791b000640 1 -- 172.21.15.114:0/3667471061 shutdown_connections 2024-09-21T07:05:40.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.888+0000 7f791b000640 1 -- 172.21.15.114:0/3667471061 wait complete. 2024-09-21T07:05:40.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.891+0000 7f791b000640 1 Processor -- start 2024-09-21T07:05:40.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.892+0000 7f791b000640 1 -- start start 2024-09-21T07:05:40.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.893+0000 7f791b000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410b8e0 0x7f7914082840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.893+0000 7f791b000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 0x7f7914082d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.893+0000 7f791b000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 0x7f79141bf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.894+0000 7f7919200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 0x7f7914082d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f7919200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 0x7f7914082d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43844/0 (socket says 172.21.15.114:43844) 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f791a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 0x7f79141bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f791a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 0x7f79141bf730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37542/0 (socket says 172.21.15.114:37542) 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f791a600640 1 -- 172.21.15.114:0/1284582207 learned_addr learned my addr 172.21.15.114:0/1284582207 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7914080d50 con 0x7f791410b8e0 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7914080bd0 con 0x7f791410fd10 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7914080ed0 con 0x7f791410f8c0 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.896+0000 7f791a600640 1 -- 172.21.15.114:0/1284582207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 msgr2=0x7f7914082d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:40.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.895+0000 7f7919c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410b8e0 0x7f7914082840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.896+0000 7f791a600640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 0x7f7914082d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.896+0000 7f791a600640 1 -- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410b8e0 msgr2=0x7f7914082840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.896+0000 7f791a600640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410b8e0 0x7f7914082840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.896+0000 7f791a600640 1 -- 172.21.15.114:0/1284582207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79141bfc70 con 0x7f791410fd10 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.897+0000 7f7919c00640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410b8e0 0x7f7914082840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.897+0000 7f7919200640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 0x7f7914082d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:05:40.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.897+0000 7f791a600640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 0x7f79141bf730 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f790c003bd0 tx=0x7f790c003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:40.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.897+0000 7f7912000640 1 -- 172.21.15.114:0/1284582207 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c031900 con 0x7f791410fd10 2024-09-21T07:05:40.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.898+0000 7f7912000640 1 -- 172.21.15.114:0/1284582207 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790c0044f0 con 0x7f791410fd10 2024-09-21T07:05:40.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.898+0000 7f7912000640 1 -- 172.21.15.114:0/1284582207 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c02fcb0 con 0x7f791410fd10 2024-09-21T07:05:40.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.898+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79141bfe40 con 0x7f791410fd10 2024-09-21T07:05:40.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.898+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79141c02c0 con 0x7f791410fd10 2024-09-21T07:05:40.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.909+0000 7f7912000640 1 -- 172.21.15.114:0/1284582207 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f790c0046a0 con 0x7f791410fd10 2024-09-21T07:05:40.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.912+0000 7f7912000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.912+0000 7f7912000640 1 -- 172.21.15.114:0/1284582207 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f790c109360 con 0x7f791410fd10 2024-09-21T07:05:40.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.914+0000 7f7919c00640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.915+0000 7f79233d8740 1 Processor -- start 2024-09-21T07:05:40.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.915+0000 7f79233d8740 1 -- start start 2024-09-21T07:05:40.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.915+0000 7f7919c00640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f79000058c0 tx=0x7f7900005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.916+0000 7f79233d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 0x55d55f8ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.916+0000 7f7919c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 0x55d55f8ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.916+0000 7f79233d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f900230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.916+0000 7f79233d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f900770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.916+0000 7f7919c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 0x55d55f8ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59876/0 (socket says 172.21.15.114:59876) 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f791a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f900770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f791a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f900770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37550/0 (socket says 172.21.15.114:37550) 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f79233d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d55f876f70 con 0x55d55f559070 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f79233d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d55f8663a0 con 0x55d55f538390 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f79233d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d55f86a1b0 con 0x55d55f5301b0 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f7919c00640 1 -- 172.21.15.114:0/4193347039 learned_addr learned my addr 172.21.15.114:0/4193347039 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:40.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.917+0000 7f7919200640 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f900230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.918+0000 7f7919c00640 1 -- 172.21.15.114:0/4193347039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 msgr2=0x55d55f900230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.918+0000 7f7919c00640 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f900230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.918+0000 7f7919c00640 1 -- 172.21.15.114:0/4193347039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 msgr2=0x55d55f900770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.918+0000 7f7919c00640 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f900770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.918+0000 7f7919c00640 1 -- 172.21.15.114:0/4193347039 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d55f830c90 con 0x55d55f559070 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.918+0000 7f791a600640 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f900770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.920+0000 7f7919c00640 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 0x55d55f8ffcf0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f790000de10 tx=0x7f7900010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a32a3821c4f7829 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:40.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.920+0000 7f7906a00640 1 -- 172.21.15.114:0/4193347039 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900019650 con 0x55d55f559070 2024-09-21T07:05:40.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.920+0000 7f7906a00640 1 -- 172.21.15.114:0/4193347039 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790001f070 con 0x55d55f559070 2024-09-21T07:05:40.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.920+0000 7f7906a00640 1 -- 172.21.15.114:0/4193347039 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790001ab10 con 0x55d55f559070 2024-09-21T07:05:40.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 -- 172.21.15.114:0/4193347039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 msgr2=0x55d55f8ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:40.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 0x55d55f8ffcf0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f790000de10 tx=0x7f7900010000 comp rx=0 tx=0).stop 2024-09-21T07:05:40.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 -- 172.21.15.114:0/4193347039 shutdown_connections 2024-09-21T07:05:40.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f900770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f900230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 --2- 172.21.15.114:0/4193347039 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f559070 0x55d55f8ffcf0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.922+0000 7f79233d8740 1 -- 172.21.15.114:0/4193347039 >> 172.21.15.114:0/4193347039 conn(0x55d55f54d460 msgr2=0x55d55f8fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.923+0000 7f79233d8740 1 -- 172.21.15.114:0/4193347039 shutdown_connections 2024-09-21T07:05:40.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.923+0000 7f79233d8740 1 -- 172.21.15.114:0/4193347039 wait complete. 2024-09-21T07:05:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.924+0000 7f79233d8740 1 -- start start 2024-09-21T07:05:40.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.924+0000 7f79233d8740 1 Processor -- start 2024-09-21T07:05:40.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.926+0000 7f79233d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f5301b0 0x55d55f90c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f79233d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 0x55d55f90c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f79233d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f911630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f79233d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d55f6c7700 con 0x55d55f5301b0 2024-09-21T07:05:40.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f79233d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d55f801540 con 0x55d55f5179f0 2024-09-21T07:05:40.930 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f79233d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d55f90a8c0 con 0x55d55f538390 2024-09-21T07:05:40.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f79233d8740 10 client.0 _ll_register_callbacks cb 0x55d55f811430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:40.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f7919200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 0x55d55f90c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f7919200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 0x55d55f90c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43856/0 (socket says 172.21.15.114:43856) 2024-09-21T07:05:40.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.927+0000 7f7919200640 1 -- 172.21.15.114:0/359757251 learned_addr learned my addr 172.21.15.114:0/359757251 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:40.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.928+0000 7f7919c00640 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f5301b0 0x55d55f90c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.928+0000 7f7919200640 1 -- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 msgr2=0x55d55f911630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.928+0000 7f7919200640 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f911630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.928+0000 7f7919200640 1 -- 172.21.15.114:0/359757251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f5301b0 msgr2=0x55d55f90c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.928+0000 7f7919200640 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f5301b0 0x55d55f90c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.928+0000 7f7919200640 1 -- 172.21.15.114:0/359757251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d55f5c4650 con 0x55d55f538390 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.930+0000 7f7919200640 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 0x55d55f90c610 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f790800cfd0 tx=0x7f790800e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2f3888e596ccea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.930+0000 7f78fb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:40.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.930+0000 7f78fb400640 1 -- 172.21.15.114:0/359757251 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908018020 con 0x55d55f538390 2024-09-21T07:05:40.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.931+0000 7f78fb400640 1 -- 172.21.15.114:0/359757251 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f7908013660 con 0x55d55f538390 2024-09-21T07:05:40.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.933+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d55f803b50 con 0x55d55f538390 2024-09-21T07:05:40.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.934+0000 7f78fb400640 1 -- 172.21.15.114:0/359757251 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7908013860 con 0x55d55f538390 2024-09-21T07:05:40.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.934+0000 7f79233d8740 10 client.28390 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:40.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.934+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d55f81e910 con 0x55d55f538390 2024-09-21T07:05:40.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.936+0000 7f78fb400640 1 -- 172.21.15.114:0/359757251 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d55f803b50 con 0x55d55f538390 2024-09-21T07:05:40.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f78fb400640 1 -- 172.21.15.114:0/359757251 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7f7908025020 con 0x55d55f538390 2024-09-21T07:05:40.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f79233d8740 10 client.28390 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f79233d8740 10 client.28390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f79233d8740 20 client.28390 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f79233d8740 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f4ecda0 0x55d55f91da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f79233d8740 4 client.28390 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d55f878ec0 con 0x55d55f4ecda0 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.938+0000 7f7919c00640 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f4ecda0 0x55d55f91da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.940+0000 7f7919c00640 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f4ecda0 0x55d55f91da10 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d50260b9ca49a32 server_cookie=bedad5fda6c5a662 in_seq=0 out_seq=0 2024-09-21T07:05:40.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.940+0000 7f78fb400640 10 client.28390 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:40.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.941+0000 7f78fb400640 1 -- 172.21.15.114:0/359757251 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d55f878ec0 con 0x55d55f4ecda0 2024-09-21T07:05:40.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.941+0000 7f78fb400640 10 client.28390 handle_command_reply: tid=0 2024-09-21T07:05:40.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.941+0000 7f79233d8740 1 client.28390 shutdown 2024-09-21T07:05:40.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.943+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 msgr2=0x55d55f90c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:40.945 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.943+0000 7f79233d8740 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 0x55d55f90c610 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f790800cfd0 tx=0x7f790800e730 comp rx=0 tx=0).stop 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 shutdown_connections 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f79233d8740 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f911630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f79233d8740 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f4ecda0 0x55d55f91da10 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f79233d8740 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f538390 0x55d55f90c610 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f79233d8740 1 --2- 172.21.15.114:0/359757251 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f5301b0 0x55d55f90c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f78fb400640 0 client.28390 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.944+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 >> 172.21.15.114:0/359757251 conn(0x55d55f54d460 msgr2=0x55d55f8fc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:40.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.945+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 shutdown_connections 2024-09-21T07:05:40.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.945+0000 7f79233d8740 1 -- 172.21.15.114:0/359757251 wait complete. 2024-09-21T07:05:40.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.945+0000 7f79233d8740 20 client.28390 trim_cache size 0 max 16384 2024-09-21T07:05:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.990+0000 7f79233d8740 1 Processor -- start 2024-09-21T07:05:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.990+0000 7f79233d8740 1 -- start start 2024-09-21T07:05:40.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.990+0000 7f79233d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f9064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.990+0000 7f79233d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f9116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.990+0000 7f79233d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f538390 0x55d55f911bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.991+0000 7f7919c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f9064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.991+0000 7f7919c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f9064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37554/0 (socket says 172.21.15.114:37554) 2024-09-21T07:05:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.992+0000 7f7919c00640 1 -- 172.21.15.114:0/520111738 learned_addr learned my addr 172.21.15.114:0/520111738 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.992+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79141c8480 con 0x55d55f538390 2024-09-21T07:05:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.992+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79141c9fc0 con 0x55d55f5179f0 2024-09-21T07:05:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.992+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79141c8b50 con 0x55d55f5301b0 2024-09-21T07:05:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.992+0000 7f791a600640 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f538390 0x55d55f911bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.992+0000 7f7919200640 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f9116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.993+0000 7f7919200640 1 -- 172.21.15.114:0/520111738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 msgr2=0x55d55f9064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.993+0000 7f7919200640 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f9064a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.993+0000 7f7919200640 1 -- 172.21.15.114:0/520111738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f538390 msgr2=0x55d55f911bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.993+0000 7f7919200640 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f538390 0x55d55f911bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.993+0000 7f7919200640 1 -- 172.21.15.114:0/520111738 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d55f90a1c0 con 0x55d55f5301b0 2024-09-21T07:05:40.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.993+0000 7f7919c00640 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f9064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:40.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.994+0000 7f7919200640 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f9116b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7908008010 tx=0x7f790800e140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d09b3b0fbe61c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:40.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.995+0000 7f7906000640 1 -- 172.21.15.114:0/520111738 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908018040 con 0x55d55f5301b0 2024-09-21T07:05:40.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.995+0000 7f7906000640 1 -- 172.21.15.114:0/520111738 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908013eb0 con 0x55d55f5301b0 2024-09-21T07:05:40.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.995+0000 7f7906000640 1 -- 172.21.15.114:0/520111738 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790801d8f0 con 0x55d55f5301b0 2024-09-21T07:05:41.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.996+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 msgr2=0x55d55f9116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:41.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.996+0000 7f79233d8740 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f9116b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7908008010 tx=0x7f790800e140 comp rx=0 tx=0).stop 2024-09-21T07:05:41.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 shutdown_connections 2024-09-21T07:05:41.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f538390 0x55d55f911bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5301b0 0x55d55f9116b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 --2- 172.21.15.114:0/520111738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f5179f0 0x55d55f9064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 >> 172.21.15.114:0/520111738 conn(0x55d55f54d460 msgr2=0x55d55f90c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 shutdown_connections 2024-09-21T07:05:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:40.999+0000 7f79233d8740 1 -- 172.21.15.114:0/520111738 wait complete. 2024-09-21T07:05:41.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.000+0000 7f79233d8740 1 -- start start 2024-09-21T07:05:41.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.000+0000 7f79233d8740 1 Processor -- start 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5179f0 0x55d55f917e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f918430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f912660 0x55d55f912b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79141c98f0 con 0x55d55f912660 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79141c9220 con 0x55d55f538390 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79141c6480 con 0x55d55f5179f0 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.004+0000 7f79233d8740 10 client.0 _ll_register_callbacks cb 0x7f790800cc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.005+0000 7f7919200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f918430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.005+0000 7f7919c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5179f0 0x55d55f917e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.005+0000 7f7919c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5179f0 0x55d55f917e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:43872/0 (socket says 172.21.15.114:43872) 2024-09-21T07:05:41.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.005+0000 7f7919c00640 1 -- 172.21.15.114:0/3320094594 learned_addr learned my addr 172.21.15.114:0/3320094594 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f7919200640 1 -- 172.21.15.114:0/3320094594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5179f0 msgr2=0x55d55f917e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f7919200640 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5179f0 0x55d55f917e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f7919200640 1 -- 172.21.15.114:0/3320094594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f912660 msgr2=0x55d55f912b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f7919200640 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f912660 0x55d55f912b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f7919200640 1 -- 172.21.15.114:0/3320094594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d55f9132a0 con 0x55d55f538390 2024-09-21T07:05:41.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f7919200640 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f918430 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55d55f91e110 tx=0x7f79080046a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad48252381dea5b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:41.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.006+0000 7f78fb400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:05:41.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.011+0000 7f78fb400640 1 -- 172.21.15.114:0/3320094594 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908018040 con 0x55d55f538390 2024-09-21T07:05:41.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.011+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d55f874970 con 0x55d55f538390 2024-09-21T07:05:41.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.012+0000 7f78fb400640 1 -- 172.21.15.114:0/3320094594 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f79080059c0 con 0x55d55f538390 2024-09-21T07:05:41.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.019+0000 7f78fb400640 1 -- 172.21.15.114:0/3320094594 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7908005bc0 con 0x55d55f538390 2024-09-21T07:05:41.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.019+0000 7f79233d8740 10 client.28442 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:41.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.019+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d55f989290 con 0x55d55f538390 2024-09-21T07:05:41.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.020+0000 7f78fb400640 1 -- 172.21.15.114:0/3320094594 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d55f874970 con 0x55d55f538390 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f78fb400640 1 -- 172.21.15.114:0/3320094594 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7f7908025030 con 0x55d55f538390 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f79233d8740 10 client.28442 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f79233d8740 10 client.28442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f79233d8740 20 client.28442 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f79233d8740 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f989710 0x55d55f98bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f79233d8740 4 client.28442 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:05:41.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.021+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d55f871da0 con 0x55d55f989710 2024-09-21T07:05:41.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.022+0000 7f7919c00640 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f989710 0x55d55f98bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:41.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.022+0000 7f7919c00640 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f989710 0x55d55f98bb30 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491ca29a497d201b server_cookie=9aecbec76e11a6a0 in_seq=0 out_seq=0 2024-09-21T07:05:41.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.022+0000 7f78fb400640 10 client.28442 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:41.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.023+0000 7f78fb400640 1 -- 172.21.15.114:0/3320094594 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x55d55f871da0 con 0x55d55f989710 2024-09-21T07:05:41.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.023+0000 7f78fb400640 10 client.28442 handle_command_reply: tid=0 2024-09-21T07:05:41.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.023+0000 7f79233d8740 1 client.28442 shutdown 2024-09-21T07:05:41.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 msgr2=0x55d55f918430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:41.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f918430 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55d55f91e110 tx=0x7f79080046a0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 shutdown_connections 2024-09-21T07:05:41.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d55f912660 0x55d55f912b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d55f538390 0x55d55f918430 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d55f989710 0x55d55f98bb30 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.029 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "4d99c123-582b-4c1f-afb2-2a0a31e59673", 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:05:41.030 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.031 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.032 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.033 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.034 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.035 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.036 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.036 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.036 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.036 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:05:41.037 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:05:41.037 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.037 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.037 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.037 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:05:41.038 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 --2- 172.21.15.114:0/3320094594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d55f5179f0 0x55d55f917e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 >> 172.21.15.114:0/3320094594 conn(0x55d55f54d460 msgr2=0x55d55f9064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f78fb400640 0 client.28442 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 shutdown_connections 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 1 -- 172.21.15.114:0/3320094594 wait complete. 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.024+0000 7f79233d8740 20 client.28442 trim_cache size 0 max 16384 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.025+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78fc0b66b0 msgr2=0x7f78fc0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:41.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.025+0000 7f791b000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f79000058c0 tx=0x7f7900005d50 comp rx=0 tx=0).stop 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.025+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 msgr2=0x7f79141bf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.025+0000 7f791b000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 0x7f79141bf730 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f790c003bd0 tx=0x7f790c003c00 comp rx=0 tx=0).stop 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 shutdown_connections 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f78fc0b66b0 0x7f78fc0b8bc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f791410fd10 0x7f79141bf730 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791410f8c0 0x7f7914082d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 --2- 172.21.15.114:0/1284582207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f791410b8e0 0x7f7914082840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 >> 172.21.15.114:0/1284582207 conn(0x7f791406e9f0 msgr2=0x7f791410d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.026+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 shutdown_connections 2024-09-21T07:05:41.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:05:41.028+0000 7f791b000640 1 -- 172.21.15.114:0/1284582207 wait complete. 2024-09-21T07:05:41.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:41 smithi114 ceph-mon[30654]: pgmap v750: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 6.8 MiB/s wr, 599 op/s 2024-09-21T07:05:41.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:41 smithi183 ceph-mon[34219]: pgmap v750: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 6.8 MiB/s wr, 599 op/s 2024-09-21T07:05:41.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:41 smithi193 ceph-mon[34601]: pgmap v750: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 6.8 MiB/s wr, 599 op/s 2024-09-21T07:05:42.129 INFO:tasks.cephfs.filesystem:scrub status for tag:4d99c123-582b-4c1f-afb2-2a0a31e59673 - {'path': '/', 'tag': '4d99c123-582b-4c1f-afb2-2a0a31e59673', 'options': 'recursive,force'} 2024-09-21T07:05:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 665.0 s, 486.0 tps, lat 196.430 ms stddev 232.644 2024-09-21T07:05:43.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:43 smithi183 ceph-mon[34219]: pgmap v751: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2024-09-21T07:05:43.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:43 smithi193 ceph-mon[34601]: pgmap v751: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2024-09-21T07:05:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:43 smithi114 ceph-mon[30654]: pgmap v751: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.7 MiB/s wr, 538 op/s 2024-09-21T07:05:45.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:45 smithi183 ceph-mon[34219]: pgmap v752: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-09-21T07:05:45.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:45 smithi193 ceph-mon[34601]: pgmap v752: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-09-21T07:05:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:45 smithi114 ceph-mon[30654]: pgmap v752: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 7.4 MiB/s wr, 646 op/s 2024-09-21T07:05:47.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:47 smithi183 ceph-mon[34219]: pgmap v753: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 6.0 MiB/s wr, 557 op/s 2024-09-21T07:05:47.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:47 smithi193 ceph-mon[34601]: pgmap v753: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 6.0 MiB/s wr, 557 op/s 2024-09-21T07:05:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:47 smithi114 ceph-mon[30654]: pgmap v753: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 6.0 MiB/s wr, 557 op/s 2024-09-21T07:05:47.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 670.0 s, 533.0 tps, lat 191.624 ms stddev 243.037 2024-09-21T07:05:49.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:49 smithi183 ceph-mon[34219]: pgmap v754: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 7.0 MiB/s wr, 627 op/s 2024-09-21T07:05:49.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:49 smithi193 ceph-mon[34601]: pgmap v754: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 7.0 MiB/s wr, 627 op/s 2024-09-21T07:05:49.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:49 smithi114 ceph-mon[30654]: pgmap v754: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 7.0 MiB/s wr, 627 op/s 2024-09-21T07:05:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:51 smithi193 ceph-mon[34601]: pgmap v755: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 7.0 MiB/s wr, 593 op/s 2024-09-21T07:05:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:51 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:05:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:51 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:05:51.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:51 smithi114 ceph-mon[30654]: pgmap v755: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 7.0 MiB/s wr, 593 op/s 2024-09-21T07:05:51.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:51 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:05:51.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:51 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:05:51.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:51 smithi183 ceph-mon[34219]: pgmap v755: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 7.0 MiB/s wr, 593 op/s 2024-09-21T07:05:51.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:51 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:05:51.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:51 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 -- 172.21.15.114:0/3984915934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0109250 msgr2=0x7fefc0109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 --2- 172.21.15.114:0/3984915934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0109250 0x7fefc0109660 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009900 tx=0x7fefb002a410 comp rx=0 tx=0).stop 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 -- 172.21.15.114:0/3984915934 shutdown_connections 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 --2- 172.21.15.114:0/3984915934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 0x7fefc01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 --2- 172.21.15.114:0/3984915934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0109250 0x7fefc0109660 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 --2- 172.21.15.114:0/3984915934 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0103920 0x7fefc0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 -- 172.21.15.114:0/3984915934 >> 172.21.15.114:0/3984915934 conn(0x7fefc00ff320 msgr2=0x7fefc0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.369+0000 7fefc5a00640 1 -- 172.21.15.114:0/3984915934 shutdown_connections 2024-09-21T07:05:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.370+0000 7fefc5a00640 1 -- 172.21.15.114:0/3984915934 wait complete. 2024-09-21T07:05:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.372+0000 7fefc5a00640 1 Processor -- start 2024-09-21T07:05:52.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.372+0000 7fefc5a00640 1 -- start start 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.373+0000 7fefc5a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 0x7fefc01a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.373+0000 7fefc5a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0109250 0x7fefc01a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefc5a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 0x7fefc01ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefc5a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01138b0 con 0x7fefc0109250 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefc5a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0113730 con 0x7fefc0103920 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 0x7fefc01a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefc5a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc0113a30 con 0x7fefc0109ba0 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 0x7fefc01a1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52646/0 (socket says 172.21.15.114:52646) 2024-09-21T07:05:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.374+0000 7fefbfe00640 1 -- 172.21.15.114:0/398034349 learned_addr learned my addr 172.21.15.114:0/398034349 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.375+0000 7fefc5000640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 0x7fefc01ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.375+0000 7fefbf400640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0109250 0x7fefc01a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.376+0000 7fefbf400640 1 -- 172.21.15.114:0/398034349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 msgr2=0x7fefc01ac5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.378+0000 7fefbf400640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 0x7fefc01ac5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.378+0000 7fefbf400640 1 -- 172.21.15.114:0/398034349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 msgr2=0x7fefc01a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.378+0000 7fefbf400640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 0x7fefc01a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.378+0000 7fefbf400640 1 -- 172.21.15.114:0/398034349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc01a2320 con 0x7fefc0109250 2024-09-21T07:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.378+0000 7fefbfe00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 0x7fefc01a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:05:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.378+0000 7fefc5000640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 0x7fefc01ac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.379+0000 7fefbf400640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0109250 0x7fefc01a1a70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009230 tx=0x7fefb0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.379+0000 7fefbcc00640 1 -- 172.21.15.114:0/398034349 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb003f070 con 0x7fefc0109250 2024-09-21T07:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.379+0000 7fefbcc00640 1 -- 172.21.15.114:0/398034349 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb0031d60 con 0x7fefc0109250 2024-09-21T07:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.380+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc01acbb0 con 0x7fefc0109250 2024-09-21T07:05:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.380+0000 7fefbcc00640 1 -- 172.21.15.114:0/398034349 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb0035030 con 0x7fefc0109250 2024-09-21T07:05:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.380+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc01ad070 con 0x7fefc0109250 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.385+0000 7fefcdffe740 1 Processor -- start 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.385+0000 7fefcdffe740 1 -- start start 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.386+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.386+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.386+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.386+0000 7fefcdffe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5587274286b0 con 0x5587270ed4d0 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.386+0000 7fefcdffe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558727417ae0 con 0x55872710e1b0 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.386+0000 7fefcdffe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55872741b8f0 con 0x5587270e52f0 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52652/0 (socket says 172.21.15.114:52652) 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 -- 172.21.15.114:0/3110083887 learned_addr learned my addr 172.21.15.114:0/3110083887 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefbfe00640 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 -- 172.21.15.114:0/3110083887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 msgr2=0x5587274b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 -- 172.21.15.114:0/3110083887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 msgr2=0x5587274b3890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:05:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274b3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.387+0000 7fefc5000640 1 -- 172.21.15.114:0/3110083887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587273e4780 con 0x55872710e1b0 2024-09-21T07:05:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.388+0000 7fefc5000640 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274b2e10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fefac032490 tx=0x7fefac032950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4322854212559e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.388+0000 7fefb5600640 1 -- 172.21.15.114:0/3110083887 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac033770 con 0x55872710e1b0 2024-09-21T07:05:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.390+0000 7fefb5600640 1 -- 172.21.15.114:0/3110083887 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac033dc0 con 0x55872710e1b0 2024-09-21T07:05:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.390+0000 7fefb5600640 1 -- 172.21.15.114:0/3110083887 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac03c380 con 0x55872710e1b0 2024-09-21T07:05:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.391+0000 7fefbcc00640 1 -- 172.21.15.114:0/398034349 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fefb0042070 con 0x7fefc0109250 2024-09-21T07:05:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.391+0000 7fefcdffe740 1 -- 172.21.15.114:0/3110083887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 msgr2=0x5587274b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.392+0000 7fefcdffe740 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274b2e10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fefac032490 tx=0x7fefac032950 comp rx=0 tx=0).stop 2024-09-21T07:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.393+0000 7fefbcc00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fefa40b6780 0x7fefa40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.393+0000 7fefbcc00640 1 -- 172.21.15.114:0/398034349 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fefb0108280 con 0x7fefc0109250 2024-09-21T07:05:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.394+0000 7fefcdffe740 1 -- 172.21.15.114:0/3110083887 shutdown_connections 2024-09-21T07:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.394+0000 7fefcdffe740 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.394+0000 7fefcdffe740 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.394+0000 7fefcdffe740 1 --2- 172.21.15.114:0/3110083887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274b2e10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.394+0000 7fefcdffe740 1 -- 172.21.15.114:0/3110083887 >> 172.21.15.114:0/3110083887 conn(0x558727102300 msgr2=0x5587274b0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.394+0000 7fefc5000640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fefa40b6780 0x7fefa40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.395+0000 7fefcdffe740 1 -- 172.21.15.114:0/3110083887 shutdown_connections 2024-09-21T07:05:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.397+0000 7fefcdffe740 1 -- 172.21.15.114:0/3110083887 wait complete. 2024-09-21T07:05:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.397+0000 7fefc5000640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fefa40b6780 0x7fefa40b8c90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fefac032460 tx=0x7fefac033f70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.399+0000 7fefcdffe740 1 -- start start 2024-09-21T07:05:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.399+0000 7fefcdffe740 1 Processor -- start 2024-09-21T07:05:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.401+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.401+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.401+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.403+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.403+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274beab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34410/0 (socket says 172.21.15.114:34410) 2024-09-21T07:05:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.404+0000 7fefbf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.404+0000 7fefc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.404+0000 7fefc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52668/0 (socket says 172.21.15.114:52668) 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.404+0000 7fefbf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41120/0 (socket says 172.21.15.114:41120) 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.405+0000 7fefbfe00640 1 -- 172.21.15.114:0/860134114 learned_addr learned my addr 172.21.15.114:0/860134114 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.405+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55872734fab0 con 0x5587270ed4d0 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.405+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587273be570 con 0x55872710e1b0 2024-09-21T07:05:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.405+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5587274b9380 con 0x5587270e52f0 2024-09-21T07:05:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefbfe00640 1 -- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 msgr2=0x5587274c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefbfe00640 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefbfe00640 1 -- 172.21.15.114:0/860134114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 msgr2=0x5587274bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefbfe00640 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefcdffe740 10 client.0 _ll_register_callbacks cb 0x5587273cd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefbfe00640 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558727179790 con 0x5587270e52f0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefac033dc0 con 0x5587270e52f0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefbfe00640 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274beab0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fefb8010b90 tx=0x7fefb800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d11f456f76655b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefaa000640 10 client.28408 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.406+0000 7fefaa000640 1 -- 172.21.15.114:0/860134114 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb801c680 con 0x5587270e52f0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.407+0000 7fefaa000640 1 -- 172.21.15.114:0/860134114 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fefb801c820 con 0x5587270e52f0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.407+0000 7fefaa000640 1 -- 172.21.15.114:0/860134114 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fefb801ca40 con 0x5587270e52f0 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.407+0000 7fefcdffe740 10 client.28408 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.407+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=50}) -- 0x5587271f2620 con 0x5587270e52f0 2024-09-21T07:05:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefaa000640 1 -- 172.21.15.114:0/860134114 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7fefb801cbe0 con 0x5587270e52f0 2024-09-21T07:05:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefcdffe740 10 client.28408 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefcdffe740 10 client.28408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefcdffe740 20 client.28408 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefcdffe740 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270ccd80 0x5587274d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefcdffe740 4 client.28408 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:05:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.408+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55872742a600 con 0x5587270ccd80 2024-09-21T07:05:52.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:52 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:05:52.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:52 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:05:52.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:52 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:05:52.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.410+0000 7fefbf400640 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270ccd80 0x5587274d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.411+0000 7fefbf400640 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270ccd80 0x5587274d0280 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87eacb6a6ff8a33f server_cookie=ec00f22834343bc7 in_seq=0 out_seq=0 2024-09-21T07:05:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.411+0000 7fefaa000640 10 client.28408 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:52.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.412+0000 7fefaa000640 1 -- 172.21.15.114:0/860134114 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55872742a600 con 0x5587270ccd80 2024-09-21T07:05:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.414+0000 7fefaa000640 10 client.28408 handle_command_reply: tid=0 2024-09-21T07:05:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.414+0000 7fefcdffe740 1 client.28408 shutdown 2024-09-21T07:05:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.414+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 msgr2=0x5587274beab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.414+0000 7fefcdffe740 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274beab0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fefb8010b90 tx=0x7fefb800e720 comp rx=0 tx=0).stop 2024-09-21T07:05:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 shutdown_connections 2024-09-21T07:05:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270ccd80 0x5587274d0280 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefaa000640 0 client.28408 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 --2- 172.21.15.114:0/860134114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270e52f0 0x5587274beab0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 >> 172.21.15.114:0/860134114 conn(0x558727102300 msgr2=0x5587274b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 shutdown_connections 2024-09-21T07:05:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.415+0000 7fefcdffe740 1 -- 172.21.15.114:0/860134114 wait complete. 2024-09-21T07:05:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.416+0000 7fefcdffe740 20 client.28408 trim_cache size 0 max 16384 2024-09-21T07:05:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.450+0000 7fefcdffe740 1 Processor -- start 2024-09-21T07:05:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.451+0000 7fefcdffe740 1 -- start start 2024-09-21T07:05:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.452+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587270e52f0 0x5587274bd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.452+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587270e52f0 0x5587274bd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefbfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587270e52f0 0x5587274bd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52676/0 (socket says 172.21.15.114:52676) 2024-09-21T07:05:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefbfe00640 1 -- 172.21.15.114:0/1731467770 learned_addr learned my addr 172.21.15.114:0/1731467770 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55872710e1b0 0x5587274b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01ae9b0 con 0x5587270ed4d0 2024-09-21T07:05:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01b04f0 con 0x5587270e52f0 2024-09-21T07:05:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01af080 con 0x55872710e1b0 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefbf400640 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.453+0000 7fefc5000640 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55872710e1b0 0x5587274b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.454+0000 7fefc5000640 1 -- 172.21.15.114:0/1731467770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587270e52f0 msgr2=0x5587274bd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.454+0000 7fefc5000640 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587270e52f0 0x5587274bd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.454+0000 7fefc5000640 1 -- 172.21.15.114:0/1731467770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 msgr2=0x5587274bd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.454+0000 7fefc5000640 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.454+0000 7fefc5000640 1 -- 172.21.15.114:0/1731467770 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587274b5c70 con 0x55872710e1b0 2024-09-21T07:05:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.454+0000 7fefc5000640 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55872710e1b0 0x5587274b0620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fefac03e9b0 tx=0x7fefac035070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132a4c842f61ff77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.455+0000 7fefb4c00640 1 -- 172.21.15.114:0/1731467770 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac056480 con 0x55872710e1b0 2024-09-21T07:05:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.455+0000 7fefb4c00640 1 -- 172.21.15.114:0/1731467770 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac05b070 con 0x55872710e1b0 2024-09-21T07:05:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.455+0000 7fefb4c00640 1 -- 172.21.15.114:0/1731467770 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac0498e0 con 0x55872710e1b0 2024-09-21T07:05:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55872710e1b0 msgr2=0x5587274b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55872710e1b0 0x5587274b0620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fefac03e9b0 tx=0x7fefac035070 comp rx=0 tx=0).stop 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 shutdown_connections 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55872710e1b0 0x5587274b0620 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270ed4d0 0x5587274bd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1731467770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587270e52f0 0x5587274bd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 >> 172.21.15.114:0/1731467770 conn(0x558727102300 msgr2=0x5587274b7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 shutdown_connections 2024-09-21T07:05:52.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.456+0000 7fefcdffe740 1 -- 172.21.15.114:0/1731467770 wait complete. 2024-09-21T07:05:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.457+0000 7fefcdffe740 1 -- start start 2024-09-21T07:05:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.457+0000 7fefcdffe740 1 Processor -- start 2024-09-21T07:05:52.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270e52f0 0x5587274ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270ed4d0 0x5587274c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01afe20 con 0x5587270e52f0 2024-09-21T07:05:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01af750 con 0x55872710e1b0 2024-09-21T07:05:52.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fefc01a0c30 con 0x5587270ed4d0 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefcdffe740 10 client.0 _ll_register_callbacks cb 0x7fefb8007720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.458+0000 7fefc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefc5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52684/0 (socket says 172.21.15.114:52684) 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefc5000640 1 -- 172.21.15.114:0/1495086879 learned_addr learned my addr 172.21.15.114:0/1495086879 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270ed4d0 0x5587274c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 -- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 msgr2=0x5587274c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 -- 172.21.15.114:0/1495086879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270e52f0 msgr2=0x5587274ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:05:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270e52f0 0x5587274ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 -- 172.21.15.114:0/1495086879 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587274c7940 con 0x5587270ed4d0 2024-09-21T07:05:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefbf400640 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270ed4d0 0x5587274c6dc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fefb01055a0 tx=0x7fefb00f1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=978a6a6122750bac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:05:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.459+0000 7fefaa000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:05:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefaa000640 1 -- 172.21.15.114:0/1495086879 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00f5070 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefaa000640 1 -- 172.21.15.114:0/1495086879 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7fefb003b460 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefac05b070 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefaa000640 1 -- 172.21.15.114:0/1495086879 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fefb00cd7b0 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefcdffe740 10 client.28414 fetch_fsmap learned FSMap version 50 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55872753b360 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.460+0000 7fefaa000640 1 -- 172.21.15.114:0/1495086879 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac05b070 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefaa000640 1 -- 172.21.15.114:0/1495086879 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7fefb0105820 con 0x5587270ed4d0 2024-09-21T07:05:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefcdffe740 10 client.28414 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefcdffe740 10 client.28414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefcdffe740 20 client.28414 populate_metadata read hostname 'smithi114' 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270a21c0 0x55872753da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefcdffe740 4 client.28414 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587274260b0 con 0x5587270a21c0 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.461+0000 7fefbfe00640 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270a21c0 0x55872753da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.462+0000 7fefaa000640 10 client.28414 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.462+0000 7fefbfe00640 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270a21c0 0x55872753da90 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb8cc5367fc9a6f server_cookie=505db83e306054f4 in_seq=0 out_seq=0 2024-09-21T07:05:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.476+0000 7fefaa000640 1 -- 172.21.15.114:0/1495086879 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587274260b0 con 0x5587270a21c0 2024-09-21T07:05:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.476+0000 7fefaa000640 10 client.28414 handle_command_reply: tid=0 2024-09-21T07:05:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.476+0000 7fefcdffe740 1 client.28414 shutdown 2024-09-21T07:05:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270ed4d0 msgr2=0x5587274c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270ed4d0 0x5587274c6dc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fefb01055a0 tx=0x7fefb00f1040 comp rx=0 tx=0).stop 2024-09-21T07:05:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 shutdown_connections 2024-09-21T07:05:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872710e1b0 0x5587274c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5587270a21c0 0x55872753da90 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefaa000640 0 client.28414 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587270ed4d0 0x5587274c6dc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 --2- 172.21.15.114:0/1495086879 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5587270e52f0 0x5587274ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.477+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 >> 172.21.15.114:0/1495086879 conn(0x558727102300 msgr2=0x5587274b7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.478+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 shutdown_connections 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.478+0000 7fefcdffe740 1 -- 172.21.15.114:0/1495086879 wait complete. 2024-09-21T07:05:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.478+0000 7fefcdffe740 20 client.28414 trim_cache size 0 max 16384 2024-09-21T07:05:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:05:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:05:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:05:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fefa40b6780 msgr2=0x7fefa40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fefa40b6780 0x7fefa40b8c90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fefac032460 tx=0x7fefac033f70 comp rx=0 tx=0).stop 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0109250 msgr2=0x7fefc01a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0109250 0x7fefc01a1a70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009230 tx=0x7fefb0003c80 comp rx=0 tx=0).stop 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 shutdown_connections 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefc0109ba0 0x7fefc01ac5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fefc0109250 0x7fefc01a1a70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fefa40b6780 0x7fefa40b8c90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 --2- 172.21.15.114:0/398034349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefc0103920 0x7fefc01a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 >> 172.21.15.114:0/398034349 conn(0x7fefc00ff320 msgr2=0x7fefc0108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 shutdown_connections 2024-09-21T07:05:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:05:52.480+0000 7fefc5a00640 1 -- 172.21.15.114:0/398034349 wait complete. 2024-09-21T07:05:52.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:52 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:05:52.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:52 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:05:52.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:52 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:05:52.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:52 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:05:52.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:52 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:05:52.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:52 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:05:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 675.0 s, 459.4 tps, lat 220.923 ms stddev 286.563 2024-09-21T07:05:53.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:53 smithi114 ceph-mon[30654]: pgmap v756: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 6.0 MiB/s wr, 480 op/s 2024-09-21T07:05:53.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:53 smithi183 ceph-mon[34219]: pgmap v756: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 6.0 MiB/s wr, 480 op/s 2024-09-21T07:05:53.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:53 smithi193 ceph-mon[34601]: pgmap v756: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 6.0 MiB/s wr, 480 op/s 2024-09-21T07:05:55.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:55 smithi114 ceph-mon[30654]: pgmap v757: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 7.1 MiB/s wr, 575 op/s 2024-09-21T07:05:55.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:55 smithi183 ceph-mon[34219]: pgmap v757: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 7.1 MiB/s wr, 575 op/s 2024-09-21T07:05:55.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:55 smithi193 ceph-mon[34601]: pgmap v757: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 7.1 MiB/s wr, 575 op/s 2024-09-21T07:05:57.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:57 smithi114 ceph-mon[30654]: pgmap v758: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-09-21T07:05:57.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:57 smithi183 ceph-mon[34219]: pgmap v758: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-09-21T07:05:57.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:57 smithi193 ceph-mon[34601]: pgmap v758: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 6.0 MiB/s wr, 510 op/s 2024-09-21T07:05:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 680.0 s, 467.2 tps, lat 211.266 ms stddev 267.434 2024-09-21T07:05:59.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:05:59 smithi114 ceph-mon[30654]: pgmap v759: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 6.4 MiB/s wr, 533 op/s 2024-09-21T07:05:59.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:05:59 smithi183 ceph-mon[34219]: pgmap v759: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 6.4 MiB/s wr, 533 op/s 2024-09-21T07:05:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:05:59 smithi193 ceph-mon[34601]: pgmap v759: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 6.4 MiB/s wr, 533 op/s 2024-09-21T07:06:01.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:01 smithi114 ceph-mon[30654]: pgmap v760: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-09-21T07:06:01.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:01 smithi183 ceph-mon[34219]: pgmap v760: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-09-21T07:06:01.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:01 smithi193 ceph-mon[34601]: pgmap v760: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 6.3 MiB/s wr, 528 op/s 2024-09-21T07:06:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 685.0 s, 486.2 tps, lat 202.238 ms stddev 241.490 2024-09-21T07:06:03.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:03 smithi114 ceph-mon[30654]: pgmap v761: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.5 MiB/s wr, 447 op/s 2024-09-21T07:06:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:03 smithi183 ceph-mon[34219]: pgmap v761: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.5 MiB/s wr, 447 op/s 2024-09-21T07:06:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:03 smithi193 ceph-mon[34601]: pgmap v761: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 5.5 MiB/s wr, 447 op/s 2024-09-21T07:06:05.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:05 smithi114 ceph-mon[30654]: pgmap v762: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 6.2 MiB/s wr, 541 op/s 2024-09-21T07:06:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:05 smithi183 ceph-mon[34219]: pgmap v762: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 6.2 MiB/s wr, 541 op/s 2024-09-21T07:06:05.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:05 smithi193 ceph-mon[34601]: pgmap v762: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 6.2 MiB/s wr, 541 op/s 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 -- 172.21.15.114:0/2954650999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc105b20 msgr2=0x7f42bc105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2954650999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc105b20 0x7f42bc105f30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009900 tx=0x7f42ac02a410 comp rx=0 tx=0).stop 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 -- 172.21.15.114:0/2954650999 shutdown_connections 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2954650999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc10bda0 0x7f42bc1125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2954650999 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42bc10b450 0x7f42bc10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 --2- 172.21.15.114:0/2954650999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc105b20 0x7f42bc105f30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 -- 172.21.15.114:0/2954650999 >> 172.21.15.114:0/2954650999 conn(0x7f42bc101560 msgr2=0x7f42bc1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:05.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 -- 172.21.15.114:0/2954650999 shutdown_connections 2024-09-21T07:06:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.818+0000 7f42bbe00640 1 -- 172.21.15.114:0/2954650999 wait complete. 2024-09-21T07:06:05.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.820+0000 7f42bbe00640 1 Processor -- start 2024-09-21T07:06:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.820+0000 7f42bbe00640 1 -- start start 2024-09-21T07:06:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42bbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc105b20 0x7f42bc19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc10b450 0x7f42bc19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42bbe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42bc10bda0 0x7f42bc1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42bbe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc115ab0 con 0x7f42bc10bda0 2024-09-21T07:06:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42bbe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc115930 con 0x7f42bc10b450 2024-09-21T07:06:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42bbe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc115c30 con 0x7f42bc105b20 2024-09-21T07:06:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc105b20 0x7f42bc19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc105b20 0x7f42bc19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36346/0 (socket says 172.21.15.114:36346) 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.821+0000 7f42baa00640 1 -- 172.21.15.114:0/1683701360 learned_addr learned my addr 172.21.15.114:0/1683701360 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42bb400640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42bc10bda0 0x7f42bc1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42b3e00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc10b450 0x7f42bc19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42b3e00640 1 -- 172.21.15.114:0/1683701360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc105b20 msgr2=0x7f42bc19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42b3e00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc105b20 0x7f42bc19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42b3e00640 1 -- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42bc10bda0 msgr2=0x7f42bc1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42b3e00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42bc10bda0 0x7f42bc1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.822+0000 7f42b3e00640 1 -- 172.21.15.114:0/1683701360 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc1a23e0 con 0x7f42bc10b450 2024-09-21T07:06:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.823+0000 7f42b3e00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc10b450 0x7f42bc19ca50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f42bc06c850 tx=0x7f42a400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.823+0000 7f42b3400640 1 -- 172.21.15.114:0/1683701360 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a400cc40 con 0x7f42bc10b450 2024-09-21T07:06:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.823+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc1a26d0 con 0x7f42bc10b450 2024-09-21T07:06:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.823+0000 7f42b3400640 1 -- 172.21.15.114:0/1683701360 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a400cde0 con 0x7f42bc10b450 2024-09-21T07:06:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.823+0000 7f42b3400640 1 -- 172.21.15.114:0/1683701360 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4018760 con 0x7f42bc10b450 2024-09-21T07:06:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.824+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc077800 con 0x7f42bc10b450 2024-09-21T07:06:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.834+0000 7f42b3400640 1 -- 172.21.15.114:0/1683701360 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f42a4018900 con 0x7f42bc10b450 2024-09-21T07:06:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.836+0000 7f42c8c05740 1 Processor -- start 2024-09-21T07:06:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.836+0000 7f42c8c05740 1 -- start start 2024-09-21T07:06:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.836+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x557426488e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.836+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426489350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 0x557426489890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42c8c05740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5574263fe6b0 con 0x5574260c34d0 2024-09-21T07:06:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42c8c05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574263edae0 con 0x5574260e41b0 2024-09-21T07:06:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42c8c05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574263f18f0 con 0x5574260bb2f0 2024-09-21T07:06:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42b3400640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42a00b6700 0x7f42a00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42bb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x557426488e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426489350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42baa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426489350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36350/0 (socket says 172.21.15.114:36350) 2024-09-21T07:06:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42baa00640 1 -- 172.21.15.114:0/2226310948 learned_addr learned my addr 172.21.15.114:0/2226310948 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.837+0000 7f42b3400640 1 -- 172.21.15.114:0/1683701360 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f42a40e7d30 con 0x7f42bc10b450 2024-09-21T07:06:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.838+0000 7f42baa00640 1 -- 172.21.15.114:0/2226310948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 msgr2=0x557426488e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.838+0000 7f42baa00640 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x557426488e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.838+0000 7f42baa00640 1 -- 172.21.15.114:0/2226310948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 msgr2=0x557426489890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.838+0000 7f42baa00640 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 0x557426489890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.838+0000 7f42baa00640 1 -- 172.21.15.114:0/2226310948 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574263ba780 con 0x5574260bb2f0 2024-09-21T07:06:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.839+0000 7f42baa00640 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426489350 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f42ac004100 tx=0x7f42ac004130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467a1e11dcb0f472 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.839+0000 7f42bb400640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42a00b6700 0x7f42a00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.839+0000 7f42abe00640 1 -- 172.21.15.114:0/2226310948 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac035040 con 0x5574260bb2f0 2024-09-21T07:06:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.839+0000 7f42abe00640 1 -- 172.21.15.114:0/2226310948 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac0054d0 con 0x5574260bb2f0 2024-09-21T07:06:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.841+0000 7f42c8c05740 1 -- 172.21.15.114:0/2226310948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 msgr2=0x557426489350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.841+0000 7f42c8c05740 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426489350 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f42ac004100 tx=0x7f42ac004130 comp rx=0 tx=0).stop 2024-09-21T07:06:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.841+0000 7f42bb400640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42a00b6700 0x7f42a00b8c10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f42b4005c40 tx=0x7f42b4004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 -- 172.21.15.114:0/2226310948 shutdown_connections 2024-09-21T07:06:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 0x557426489890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426489350 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 --2- 172.21.15.114:0/2226310948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x557426488e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 -- 172.21.15.114:0/2226310948 >> 172.21.15.114:0/2226310948 conn(0x5574260d8300 msgr2=0x557426486d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 -- 172.21.15.114:0/2226310948 shutdown_connections 2024-09-21T07:06:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.843+0000 7f42c8c05740 1 -- 172.21.15.114:0/2226310948 wait complete. 2024-09-21T07:06:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.844+0000 7f42c8c05740 1 -- start start 2024-09-21T07:06:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.844+0000 7f42c8c05740 1 Processor -- start 2024-09-21T07:06:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426494390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 0x5574264948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260e41b0 0x557426494e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557426325ab0 con 0x5574260e41b0 2024-09-21T07:06:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557426394570 con 0x5574260c34d0 2024-09-21T07:06:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5574264975d0 con 0x5574260bb2f0 2024-09-21T07:06:05.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 0x5574264948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 0x5574264948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41484/0 (socket says 172.21.15.114:41484) 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42b3e00640 1 -- 172.21.15.114:0/3465644861 learned_addr learned my addr 172.21.15.114:0/3465644861 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.845+0000 7f42c8c05740 10 client.0 _ll_register_callbacks cb 0x5574263a3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42baa00640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426494390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42bb400640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260e41b0 0x557426494e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42b3e00640 1 -- 172.21.15.114:0/3465644861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 msgr2=0x557426494390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42b3e00640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426494390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42b3e00640 1 -- 172.21.15.114:0/3465644861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260e41b0 msgr2=0x557426494e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42b3e00640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260e41b0 0x557426494e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42b3e00640 1 -- 172.21.15.114:0/3465644861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742614f790 con 0x5574260c34d0 2024-09-21T07:06:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.846+0000 7f42bb400640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260e41b0 0x557426494e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:06:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42b3e00640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 0x5574264948d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42a40e5110 tx=0x7f42a4010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bdc43f8e47816c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42a8c00640 1 -- 172.21.15.114:0/3465644861 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a40ad370 con 0x5574260c34d0 2024-09-21T07:06:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac0054d0 con 0x5574260c34d0 2024-09-21T07:06:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42a8c00640 1 -- 172.21.15.114:0/3465644861 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f42a40a93c0 con 0x5574260c34d0 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42a8c00640 1 -- 172.21.15.114:0/3465644861 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f42a40ad370 con 0x5574260c34d0 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42c8c05740 10 client.28472 fetch_fsmap learned FSMap version 50 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.847+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5574261c8620 con 0x5574260c34d0 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.848+0000 7f42a8c00640 1 -- 172.21.15.114:0/3465644861 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac0054d0 con 0x5574260c34d0 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.848+0000 7f42a8c00640 1 -- 172.21.15.114:0/3465644861 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7f42a40b3420 con 0x5574260c34d0 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.848+0000 7f42c8c05740 10 client.28472 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.848+0000 7f42c8c05740 10 client.28472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.848+0000 7f42c8c05740 20 client.28472 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.848+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260a2d80 0x5574264a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.849+0000 7f42c8c05740 4 client.28472 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.849+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557426400600 con 0x5574260a2d80 2024-09-21T07:06:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.849+0000 7f42baa00640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260a2d80 0x5574264a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.850+0000 7f42baa00640 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260a2d80 0x5574264a6a10 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cee9d0d275757914 server_cookie=58b2a0210b948e1e in_seq=0 out_seq=0 2024-09-21T07:06:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.850+0000 7f42a8c00640 10 client.28472 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.851+0000 7f42a8c00640 1 -- 172.21.15.114:0/3465644861 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557426400600 con 0x5574260a2d80 2024-09-21T07:06:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.851+0000 7f42a8c00640 10 client.28472 handle_command_reply: tid=0 2024-09-21T07:06:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 client.28472 shutdown 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 msgr2=0x5574264948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 0x5574264948d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42a40e5110 tx=0x7f42a4010d70 comp rx=0 tx=0).stop 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 shutdown_connections 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260e41b0 0x557426494e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260c34d0 0x5574264948d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260a2d80 0x5574264a6a10 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3465644861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260bb2f0 0x557426494390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.852+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 >> 172.21.15.114:0/3465644861 conn(0x5574260d8300 msgr2=0x557426489e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.853+0000 7f42a8c00640 0 client.28472 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.853+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 shutdown_connections 2024-09-21T07:06:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.853+0000 7f42c8c05740 1 -- 172.21.15.114:0/3465644861 wait complete. 2024-09-21T07:06:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.856+0000 7f42c8c05740 20 client.28472 trim_cache size 0 max 16384 2024-09-21T07:06:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.893+0000 7f42c8c05740 1 Processor -- start 2024-09-21T07:06:05.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.893+0000 7f42c8c05740 1 -- start start 2024-09-21T07:06:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.894+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 0x557426485d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.894+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 0x55742648a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.894+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 0x557426491560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.894+0000 7f42c8c05740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b7590 con 0x5574260c34d0 2024-09-21T07:06:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.894+0000 7f42c8c05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b90d0 con 0x5574260bb2f0 2024-09-21T07:06:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.894+0000 7f42c8c05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b7c60 con 0x5574260e41b0 2024-09-21T07:06:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.895+0000 7f42bb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 0x557426491560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.895+0000 7f42bb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 0x557426491560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:36368/0 (socket says 172.21.15.114:36368) 2024-09-21T07:06:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.895+0000 7f42baa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 0x557426485d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.895+0000 7f42baa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 0x557426485d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41500/0 (socket says 172.21.15.114:41500) 2024-09-21T07:06:05.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.895+0000 7f42baa00640 1 -- 172.21.15.114:0/3216517341 learned_addr learned my addr 172.21.15.114:0/3216517341 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.896+0000 7f42baa00640 1 -- 172.21.15.114:0/3216517341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 msgr2=0x557426491560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.896+0000 7f42baa00640 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 0x557426491560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.896+0000 7f42baa00640 1 -- 172.21.15.114:0/3216517341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 msgr2=0x55742648a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.896+0000 7f42baa00640 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 0x55742648a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.896+0000 7f42baa00640 1 -- 172.21.15.114:0/3216517341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55742648fee0 con 0x5574260bb2f0 2024-09-21T07:06:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.897+0000 7f42bb400640 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 0x557426491560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.897+0000 7f42baa00640 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 0x557426485d20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42ac005ec0 tx=0x7f42ac031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6dde5a5c8aacab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.898+0000 7f42ab400640 1 -- 172.21.15.114:0/3216517341 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac035040 con 0x5574260bb2f0 2024-09-21T07:06:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.898+0000 7f42ab400640 1 -- 172.21.15.114:0/3216517341 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac004410 con 0x5574260bb2f0 2024-09-21T07:06:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.898+0000 7f42ab400640 1 -- 172.21.15.114:0/3216517341 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac005260 con 0x5574260bb2f0 2024-09-21T07:06:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.899+0000 7f42c8c05740 1 -- 172.21.15.114:0/3216517341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 msgr2=0x557426485d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.900+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 0x557426485d20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f42ac005ec0 tx=0x7f42ac031400 comp rx=0 tx=0).stop 2024-09-21T07:06:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.900+0000 7f42c8c05740 1 -- 172.21.15.114:0/3216517341 shutdown_connections 2024-09-21T07:06:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.900+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260e41b0 0x557426491560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.900+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260c34d0 0x55742648a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.900+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3216517341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260bb2f0 0x557426485d20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.900+0000 7f42c8c05740 1 -- 172.21.15.114:0/3216517341 >> 172.21.15.114:0/3216517341 conn(0x5574260d8300 msgr2=0x557426489e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.901+0000 7f42c8c05740 1 -- 172.21.15.114:0/3216517341 shutdown_connections 2024-09-21T07:06:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.901+0000 7f42c8c05740 1 -- 172.21.15.114:0/3216517341 wait complete. 2024-09-21T07:06:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.902+0000 7f42c8c05740 1 -- start start 2024-09-21T07:06:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.902+0000 7f42c8c05740 1 Processor -- start 2024-09-21T07:06:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.903+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260bb2f0 0x557426499950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260c34d0 0x557426499e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42c8c05740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x55742649a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42c8c05740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b8a00 con 0x5574260bb2f0 2024-09-21T07:06:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42c8c05740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b8330 con 0x5574260e41b0 2024-09-21T07:06:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42c8c05740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc1b5510 con 0x5574260c34d0 2024-09-21T07:06:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42c8c05740 10 client.0 _ll_register_callbacks cb 0x7f42a40056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42baa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260bb2f0 0x557426499950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.904+0000 7f42bb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x55742649a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.905+0000 7f42bb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x55742649a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:41512/0 (socket says 172.21.15.114:41512) 2024-09-21T07:06:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.905+0000 7f42bb400640 1 -- 172.21.15.114:0/3131324782 learned_addr learned my addr 172.21.15.114:0/3131324782 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.905+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260c34d0 0x557426499e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.906+0000 7f42bb400640 1 -- 172.21.15.114:0/3131324782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260c34d0 msgr2=0x557426499e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.906+0000 7f42bb400640 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260c34d0 0x557426499e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.906+0000 7f42bb400640 1 -- 172.21.15.114:0/3131324782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260bb2f0 msgr2=0x557426499950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.906+0000 7f42bb400640 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260bb2f0 0x557426499950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.906+0000 7f42bb400640 1 -- 172.21.15.114:0/3131324782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55742650f4b0 con 0x5574260e41b0 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.907+0000 7f42bb400640 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x55742649a3d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f42b400ff90 tx=0x7f42b4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd463f5ca8bdb739 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.907+0000 7f42a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.907+0000 7f42a8c00640 1 -- 172.21.15.114:0/3131324782 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b402d060 con 0x5574260e41b0 2024-09-21T07:06:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.907+0000 7f42a8c00640 1 -- 172.21.15.114:0/3131324782 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9490+0+0 (secure 0 0 0) 0x7f42b4003b80 con 0x5574260e41b0 2024-09-21T07:06:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.907+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac004410 con 0x5574260e41b0 2024-09-21T07:06:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.908+0000 7f42a8c00640 1 -- 172.21.15.114:0/3131324782 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f42b401be40 con 0x5574260e41b0 2024-09-21T07:06:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.908+0000 7f42c8c05740 10 client.28478 fetch_fsmap learned FSMap version 50 2024-09-21T07:06:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.908+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5574265107a0 con 0x5574260e41b0 2024-09-21T07:06:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.908+0000 7f42a8c00640 1 -- 172.21.15.114:0/3131324782 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac004410 con 0x5574260e41b0 2024-09-21T07:06:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.909+0000 7f42a8c00640 1 -- 172.21.15.114:0/3131324782 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 50) ==== 8374+0+0 (secure 0 0 0) 0x7f42b4028610 con 0x5574260e41b0 2024-09-21T07:06:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42c8c05740 10 client.28478 fetch_fsmap finished waiting for FSMap version 50 2024-09-21T07:06:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42c8c05740 10 client.28478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42c8c05740 20 client.28478 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260781c0 0x557426512f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42c8c05740 4 client.28478 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:06:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5574263fc0b0 con 0x5574260781c0 2024-09-21T07:06:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.910+0000 7f42baa00640 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260781c0 0x557426512f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.911+0000 7f42baa00640 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260781c0 0x557426512f60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a5ed80ac7fad03 server_cookie=4f0b80eabc4c9af6 in_seq=0 out_seq=0 2024-09-21T07:06:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:05.911+0000 7f42a8c00640 10 client.28478 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.176+0000 7f42a8c00640 1 -- 172.21.15.114:0/3131324782 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5574263fc0b0 con 0x5574260781c0 2024-09-21T07:06:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.177+0000 7f42a8c00640 10 client.28478 handle_command_reply: tid=0 2024-09-21T07:06:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.177+0000 7f42c8c05740 1 client.28478 shutdown 2024-09-21T07:06:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.177+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 msgr2=0x55742649a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.177+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x55742649a3d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f42b400ff90 tx=0x7f42b4004ab0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.177+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 shutdown_connections 2024-09-21T07:06:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.177+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574260e41b0 0x55742649a3d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5574260781c0 0x557426512f60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5574260c34d0 0x557426499e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42c8c05740 1 --2- 172.21.15.114:0/3131324782 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5574260bb2f0 0x557426499950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 >> 172.21.15.114:0/3131324782 conn(0x5574260d8300 msgr2=0x5574264961d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42a8c00640 0 client.28478 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 shutdown_connections 2024-09-21T07:06:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.178+0000 7f42c8c05740 1 -- 172.21.15.114:0/3131324782 wait complete. 2024-09-21T07:06:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.179+0000 7f42c8c05740 20 client.28478 trim_cache size 0 max 16384 2024-09-21T07:06:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:06:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:06:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:06:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42a00b6700 msgr2=0x7f42a00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42a00b6700 0x7f42a00b8c10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f42b4005c40 tx=0x7f42b4004a40 comp rx=0 tx=0).stop 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc10b450 msgr2=0x7f42bc19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc10b450 0x7f42bc19ca50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f42bc06c850 tx=0x7f42a400ee50 comp rx=0 tx=0).stop 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 shutdown_connections 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f42bc10bda0 0x7f42bc1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f42a00b6700 0x7f42a00b8c10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42bc10b450 0x7f42bc19ca50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 --2- 172.21.15.114:0/1683701360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42bc105b20 0x7f42bc19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 >> 172.21.15.114:0/1683701360 conn(0x7f42bc101560 msgr2=0x7f42bc103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 shutdown_connections 2024-09-21T07:06:06.183 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:06.181+0000 7f42bbe00640 1 -- 172.21.15.114:0/1683701360 wait complete. 2024-09-21T07:06:07.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:07 smithi114 ceph-mon[30654]: pgmap v763: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 5.8 MiB/s wr, 503 op/s 2024-09-21T07:06:07.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:07 smithi114 ceph-mon[30654]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T07:06:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:07 smithi183 ceph-mon[34219]: pgmap v763: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 5.8 MiB/s wr, 503 op/s 2024-09-21T07:06:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:07 smithi183 ceph-mon[34219]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T07:06:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:07 smithi193 ceph-mon[34601]: pgmap v763: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 5.8 MiB/s wr, 503 op/s 2024-09-21T07:06:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:07 smithi193 ceph-mon[34601]: osdmap e98: 12 total, 12 up, 12 in 2024-09-21T07:06:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 690.0 s, 550.0 tps, lat 190.932 ms stddev 231.137 2024-09-21T07:06:09.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:09 smithi114 ceph-mon[30654]: pgmap v765: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 6.9 MiB/s wr, 569 op/s 2024-09-21T07:06:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:09 smithi183 ceph-mon[34219]: pgmap v765: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 6.9 MiB/s wr, 569 op/s 2024-09-21T07:06:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:09 smithi193 ceph-mon[34601]: pgmap v765: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 6.9 MiB/s wr, 569 op/s 2024-09-21T07:06:11.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:11 smithi114 ceph-mon[30654]: pgmap v766: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 6.8 MiB/s wr, 581 op/s 2024-09-21T07:06:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:11 smithi183 ceph-mon[34219]: pgmap v766: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 6.8 MiB/s wr, 581 op/s 2024-09-21T07:06:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:11 smithi193 ceph-mon[34601]: pgmap v766: 129 pgs: 6 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 111 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 6.8 MiB/s wr, 581 op/s 2024-09-21T07:06:12.130 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:06:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 695.0 s, 446.4 tps, lat 219.471 ms stddev 283.618 2024-09-21T07:06:12.836 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:13.273 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:13 smithi114 ceph-mon[30654]: pgmap v767: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-09-21T07:06:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:13 smithi183 ceph-mon[34219]: pgmap v767: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-09-21T07:06:13.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:13 smithi193 ceph-mon[34601]: pgmap v767: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 6.2 MiB/s wr, 574 op/s 2024-09-21T07:06:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:15 smithi114 ceph-mon[30654]: pgmap v768: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2024-09-21T07:06:15.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:15 smithi114 ceph-mon[30654]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T07:06:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:15 smithi183 ceph-mon[34219]: pgmap v768: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2024-09-21T07:06:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:15 smithi183 ceph-mon[34219]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T07:06:15.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:15 smithi193 ceph-mon[34601]: pgmap v768: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 6.3 MiB/s wr, 570 op/s 2024-09-21T07:06:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:15 smithi193 ceph-mon[34601]: osdmap e99: 12 total, 12 up, 12 in 2024-09-21T07:06:17.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:17 smithi114 ceph-mon[30654]: pgmap v770: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-09-21T07:06:17.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:17 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:06:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:17 smithi183 ceph-mon[34219]: pgmap v770: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-09-21T07:06:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:17 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:06:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:17 smithi193 ceph-mon[34601]: pgmap v770: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 6.7 MiB/s wr, 562 op/s 2024-09-21T07:06:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:17 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:06:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 700.0 s, 417.6 tps, lat 229.608 ms stddev 293.515 2024-09-21T07:06:19.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:19 smithi114 ceph-mon[30654]: pgmap v771: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 5.5 MiB/s wr, 501 op/s 2024-09-21T07:06:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:19 smithi183 ceph-mon[34219]: pgmap v771: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 5.5 MiB/s wr, 501 op/s 2024-09-21T07:06:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 -- 172.21.15.114:0/2167322335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f603010bd90 msgr2=0x7f60301125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 --2- 172.21.15.114:0/2167322335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f603010bd90 0x7f60301125c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6028009ca0 tx=0x7f602802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 -- 172.21.15.114:0/2167322335 shutdown_connections 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 --2- 172.21.15.114:0/2167322335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f603010bd90 0x7f60301125c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 --2- 172.21.15.114:0/2167322335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f603010b440 0x7f603010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 --2- 172.21.15.114:0/2167322335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6030105b10 0x7f6030105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 -- 172.21.15.114:0/2167322335 >> 172.21.15.114:0/2167322335 conn(0x7f6030101550 msgr2=0x7f60301039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 -- 172.21.15.114:0/2167322335 shutdown_connections 2024-09-21T07:06:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.642+0000 7f602f200640 1 -- 172.21.15.114:0/2167322335 wait complete. 2024-09-21T07:06:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 Processor -- start 2024-09-21T07:06:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 -- start start 2024-09-21T07:06:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 0x7f603019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f603010b440 0x7f603019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603010bd90 0x7f60301a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6030115aa0 con 0x7f6030105b10 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6030115920 con 0x7f603010b440 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.643+0000 7f602f200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6030115c20 con 0x7f603010bd90 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 0x7f603019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 0x7f603019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58034/0 (socket says 172.21.15.114:58034) 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 -- 172.21.15.114:0/1443043696 learned_addr learned my addr 172.21.15.114:0/1443043696 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602e800640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603010bd90 0x7f60301a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f6025400640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f603010b440 0x7f603019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 -- 172.21.15.114:0/1443043696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603010bd90 msgr2=0x7f60301a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603010bd90 0x7f60301a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 -- 172.21.15.114:0/1443043696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f603010b440 msgr2=0x7f603019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f603010b440 0x7f603019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 -- 172.21.15.114:0/1443043696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60301a2410 con 0x7f6030105b10 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.644+0000 7f602de00640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 0x7f603019c4e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9f0 tx=0x7f602000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.647 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.645+0000 7f6027400640 1 -- 172.21.15.114:0/1443043696 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602000cd80 con 0x7f6030105b10 2024-09-21T07:06:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.645+0000 7f6027400640 1 -- 172.21.15.114:0/1443043696 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60200054d0 con 0x7f6030105b10 2024-09-21T07:06:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.645+0000 7f6027400640 1 -- 172.21.15.114:0/1443043696 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020010690 con 0x7f6030105b10 2024-09-21T07:06:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.645+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301a2700 con 0x7f6030105b10 2024-09-21T07:06:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.645+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a8e90 con 0x7f6030105b10 2024-09-21T07:06:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.648+0000 7f603bc11740 1 Processor -- start 2024-09-21T07:06:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.648+0000 7f6027400640 1 -- 172.21.15.114:0/1443043696 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6020004e90 con 0x7f6030105b10 2024-09-21T07:06:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.648+0000 7f603bc11740 1 -- start start 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f603bc11740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276ee8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f603bc11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b1b2f0 0x558276ee9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f603bc11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b234d0 0x558276ee9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f603bc11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558276e5e6b0 con 0x558276b1b2f0 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f603bc11740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558276e4dae0 con 0x558276b441b0 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f603bc11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558276e518f0 con 0x558276b234d0 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f602e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276ee8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f602de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b1b2f0 0x558276ee9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f602e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276ee8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38076/0 (socket says 172.21.15.114:38076) 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f602e800640 1 -- 172.21.15.114:0/2724845650 learned_addr learned my addr 172.21.15.114:0/2724845650 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.649+0000 7f6025400640 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b234d0 0x558276ee9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602e800640 1 -- 172.21.15.114:0/2724845650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b234d0 msgr2=0x558276ee9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602e800640 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b234d0 0x558276ee9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602e800640 1 -- 172.21.15.114:0/2724845650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b1b2f0 msgr2=0x558276ee9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602e800640 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b1b2f0 0x558276ee9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602e800640 1 -- 172.21.15.114:0/2724845650 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558276e1a780 con 0x558276b441b0 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f6025400640 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b234d0 0x558276ee9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f6027400640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60140b6680 0x7f60140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602e800640 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276ee8e10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6028035000 tx=0x7f6028009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212259ed27f2eacb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f6027400640 1 -- 172.21.15.114:0/1443043696 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f60200e7ec0 con 0x7f6030105b10 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f602de00640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60140b6680 0x7f60140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.650+0000 7f601f000640 1 -- 172.21.15.114:0/2724845650 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028031cf0 con 0x558276b441b0 2024-09-21T07:06:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.651+0000 7f601f000640 1 -- 172.21.15.114:0/2724845650 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6028036070 con 0x558276b441b0 2024-09-21T07:06:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.651+0000 7f601f000640 1 -- 172.21.15.114:0/2724845650 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028004280 con 0x558276b441b0 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.651+0000 7f603bc11740 1 -- 172.21.15.114:0/2724845650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 msgr2=0x558276ee8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.651+0000 7f602de00640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60140b6680 0x7f60140b8b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f60200e52f0 tx=0x7f60200eba10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.651+0000 7f603bc11740 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276ee8e10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6028035000 tx=0x7f6028009430 comp rx=0 tx=0).stop 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 -- 172.21.15.114:0/2724845650 shutdown_connections 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b234d0 0x558276ee9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b1b2f0 0x558276ee9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 --2- 172.21.15.114:0/2724845650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276ee8e10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 -- 172.21.15.114:0/2724845650 >> 172.21.15.114:0/2724845650 conn(0x558276b38300 msgr2=0x558276ee6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 -- 172.21.15.114:0/2724845650 shutdown_connections 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 -- 172.21.15.114:0/2724845650 wait complete. 2024-09-21T07:06:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 -- start start 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.652+0000 7f603bc11740 1 Processor -- start 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b1b2f0 0x558276ef4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276ef49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276f006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558276d85ab0 con 0x558276b234d0 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558276df4570 con 0x558276b441b0 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558276ef73b0 con 0x558276b1b2f0 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f603bc11740 10 client.0 _ll_register_callbacks cb 0x558276e03100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f602e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b1b2f0 0x558276ef4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f6025400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276ef49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f602e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b1b2f0 0x558276ef4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46204/0 (socket says 172.21.15.114:46204) 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f602e800640 1 -- 172.21.15.114:0/2122414351 learned_addr learned my addr 172.21.15.114:0/2122414351 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f6025400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276ef49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58064/0 (socket says 172.21.15.114:58064) 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.653+0000 7f602e800640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276f006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6025400640 1 -- 172.21.15.114:0/2122414351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b1b2f0 msgr2=0x558276ef4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6025400640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b1b2f0 0x558276ef4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6025400640 1 -- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 msgr2=0x558276f006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6025400640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276f006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6025400640 1 -- 172.21.15.114:0/2122414351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558276baf790 con 0x558276b234d0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f602e800640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276f006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6025400640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276ef49b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f603006c620 tx=0x7f6018012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc71db3cbc2c4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6013400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6013400640 1 -- 172.21.15.114:0/2122414351 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601800f8f0 con 0x558276b234d0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6013400640 1 -- 172.21.15.114:0/2122414351 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f601800fa90 con 0x558276b234d0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6028036070 con 0x558276b234d0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f6013400640 1 -- 172.21.15.114:0/2122414351 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6018025050 con 0x558276b234d0 2024-09-21T07:06:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f603bc11740 10 client.17943 fetch_fsmap learned FSMap version 51 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.654+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558276c28620 con 0x558276b234d0 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f6013400640 1 -- 172.21.15.114:0/2122414351 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6028036070 con 0x558276b234d0 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f6013400640 1 -- 172.21.15.114:0/2122414351 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f6018010430 con 0x558276b234d0 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f603bc11740 10 client.17943 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f603bc11740 10 client.17943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f603bc11740 20 client.17943 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f603bc11740 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276b02d80 0x558276f06450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f603bc11740 4 client.17943 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558276e60600 con 0x558276b02d80 2024-09-21T07:06:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.655+0000 7f602e800640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276b02d80 0x558276f06450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.656+0000 7f602e800640 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276b02d80 0x558276f06450 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4037d5fb250351 server_cookie=9028bc55842890a1 in_seq=0 out_seq=0 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.656+0000 7f6013400640 10 client.17943 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.656+0000 7f6013400640 1 -- 172.21.15.114:0/2122414351 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558276e60600 con 0x558276b02d80 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.656+0000 7f6013400640 10 client.17943 handle_command_reply: tid=0 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.656+0000 7f603bc11740 1 client.17943 shutdown 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 msgr2=0x558276ef49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276ef49b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f603006c620 tx=0x7f6018012b60 comp rx=0 tx=0).stop 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 shutdown_connections 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b441b0 0x558276f006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276ef49b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276b02d80 0x558276f06450 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 --2- 172.21.15.114:0/2122414351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b1b2f0 0x558276ef4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f6013400640 0 client.17943 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 >> 172.21.15.114:0/2122414351 conn(0x558276b38300 msgr2=0x558276ee9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 shutdown_connections 2024-09-21T07:06:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 1 -- 172.21.15.114:0/2122414351 wait complete. 2024-09-21T07:06:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.657+0000 7f603bc11740 20 client.17943 trim_cache size 0 max 16384 2024-09-21T07:06:19.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:19 smithi193 ceph-mon[34601]: pgmap v771: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 5.5 MiB/s wr, 501 op/s 2024-09-21T07:06:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 Processor -- start 2024-09-21T07:06:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 -- start start 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276ee5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276eea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276ef1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b2f30 con 0x558276b234d0 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b4aa0 con 0x558276b1b2f0 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.672+0000 7f603bc11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b3600 con 0x558276b441b0 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276eea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276eea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58076/0 (socket says 172.21.15.114:58076) 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 -- 172.21.15.114:0/2115454810 learned_addr learned my addr 172.21.15.114:0/2115454810 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f602e800640 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276ef1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f602e800640 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276ee5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 -- 172.21.15.114:0/2115454810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 msgr2=0x558276ef1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276ef1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 -- 172.21.15.114:0/2115454810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 msgr2=0x558276ee5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276ee5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 -- 172.21.15.114:0/2115454810 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558276ef2950 con 0x558276b234d0 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f6025400640 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276eea550 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6018000c00 tx=0x7f6018005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb46221f85e3330 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f601e600640 1 -- 172.21.15.114:0/2115454810 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018009ca0 con 0x558276b234d0 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f601e600640 1 -- 172.21.15.114:0/2115454810 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601802e070 con 0x558276b234d0 2024-09-21T07:06:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.673+0000 7f601e600640 1 -- 172.21.15.114:0/2115454810 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601801c470 con 0x558276b234d0 2024-09-21T07:06:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 -- 172.21.15.114:0/2115454810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 msgr2=0x558276eea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276eea550 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6018000c00 tx=0x7f6018005880 comp rx=0 tx=0).stop 2024-09-21T07:06:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 -- 172.21.15.114:0/2115454810 shutdown_connections 2024-09-21T07:06:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276ef1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276eea550 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 --2- 172.21.15.114:0/2115454810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276ee5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 -- 172.21.15.114:0/2115454810 >> 172.21.15.114:0/2115454810 conn(0x558276b38300 msgr2=0x558276ee9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 -- 172.21.15.114:0/2115454810 shutdown_connections 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.674+0000 7f603bc11740 1 -- 172.21.15.114:0/2115454810 wait complete. 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.675+0000 7f603bc11740 1 -- start start 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.675+0000 7f603bc11740 1 Processor -- start 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.675+0000 7f603bc11740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276efcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.675+0000 7f603bc11740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276efa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f603bc11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276efaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f603bc11740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b43a0 con 0x558276b234d0 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f603bc11740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b3cd0 con 0x558276b1b2f0 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f603bc11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f60301b0f30 con 0x558276b441b0 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f603bc11740 10 client.0 _ll_register_callbacks cb 0x7f6018009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f6025400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276efa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276efcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276efcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38096/0 (socket says 172.21.15.114:38096) 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f6025400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276efa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58090/0 (socket says 172.21.15.114:58090) 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 -- 172.21.15.114:0/69292508 learned_addr learned my addr 172.21.15.114:0/69292508 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276efaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 -- 172.21.15.114:0/69292508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 msgr2=0x558276efaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276efaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 -- 172.21.15.114:0/69292508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 msgr2=0x558276efa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276efa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 -- 172.21.15.114:0/69292508 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558276efb190 con 0x558276b1b2f0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f602e800640 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276efcef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6028003e60 tx=0x7f6028052040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8492af2d623df87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.676+0000 7f6013400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f6013400640 1 -- 172.21.15.114:0/69292508 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028031710 con 0x558276b1b2f0 2024-09-21T07:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601802e070 con 0x558276b1b2f0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f6013400640 1 -- 172.21.15.114:0/69292508 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f60280318b0 con 0x558276b1b2f0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f6013400640 1 -- 172.21.15.114:0/69292508 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6028004040 con 0x558276b1b2f0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 10 client.28496 fetch_fsmap learned FSMap version 51 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x558276f70e90 con 0x558276b1b2f0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f6013400640 1 -- 172.21.15.114:0/69292508 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f601802e070 con 0x558276b1b2f0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f6013400640 1 -- 172.21.15.114:0/69292508 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f6028043020 con 0x558276b1b2f0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 10 client.28496 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 10 client.28496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 20 client.28496 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276ad81c0 0x558276f73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 4 client.28496 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.677+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558276e5c0b0 con 0x558276ad81c0 2024-09-21T07:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.678+0000 7f6025400640 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276ad81c0 0x558276f73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.678+0000 7f6025400640 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276ad81c0 0x558276f73350 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa599f2c9a5e844a server_cookie=19fdd9cc69d680e in_seq=0 out_seq=0 2024-09-21T07:06:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.678+0000 7f6013400640 10 client.28496 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.763+0000 7f6013400640 1 -- 172.21.15.114:0/69292508 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558276e5c0b0 con 0x558276ad81c0 2024-09-21T07:06:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.763+0000 7f6013400640 10 client.28496 handle_command_reply: tid=0 2024-09-21T07:06:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.764+0000 7f603bc11740 1 client.28496 shutdown 2024-09-21T07:06:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.764+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 msgr2=0x558276efcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.764+0000 7f603bc11740 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276efcef0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6028003e60 tx=0x7f6028052040 comp rx=0 tx=0).stop 2024-09-21T07:06:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 shutdown_connections 2024-09-21T07:06:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558276ad81c0 0x558276f73350 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558276b441b0 0x558276efaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558276b234d0 0x558276efa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 --2- 172.21.15.114:0/69292508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558276b1b2f0 0x558276efcef0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 >> 172.21.15.114:0/69292508 conn(0x558276b38300 msgr2=0x558276ee5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f6013400640 0 client.28496 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 shutdown_connections 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 1 -- 172.21.15.114:0/69292508 wait complete. 2024-09-21T07:06:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.765+0000 7f603bc11740 20 client.28496 trim_cache size 0 max 16384 2024-09-21T07:06:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:06:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:06:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:06:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.767+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60140b6680 msgr2=0x7f60140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.767+0000 7f602f200640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60140b6680 0x7f60140b8b90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f60200e52f0 tx=0x7f60200eba10 comp rx=0 tx=0).stop 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.767+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 msgr2=0x7f603019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.767+0000 7f602f200640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 0x7f603019c4e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9f0 tx=0x7f602000edf0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 shutdown_connections 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f603010bd90 0x7f60301a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f60140b6680 0x7f60140b8b90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f603010b440 0x7f603019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 --2- 172.21.15.114:0/1443043696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6030105b10 0x7f603019c4e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 >> 172.21.15.114:0/1443043696 conn(0x7f6030101550 msgr2=0x7f603010aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.768+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 shutdown_connections 2024-09-21T07:06:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:19.770+0000 7f602f200640 1 -- 172.21.15.114:0/1443043696 wait complete. 2024-09-21T07:06:21.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:21 smithi114 ceph-mon[30654]: pgmap v772: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-09-21T07:06:21.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:21 smithi183 ceph-mon[34219]: pgmap v772: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-09-21T07:06:21.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:21 smithi193 ceph-mon[34601]: pgmap v772: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-09-21T07:06:21.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.877+0000 7fee43e00640 1 -- 172.21.15.114:0/915899321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410af90 msgr2=0x7fee44115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:21.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.877+0000 7fee43e00640 1 --2- 172.21.15.114:0/915899321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410af90 0x7fee44115fd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fee30009860 tx=0x7fee3002a340 comp rx=0 tx=0).stop 2024-09-21T07:06:21.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 -- 172.21.15.114:0/915899321 shutdown_connections 2024-09-21T07:06:21.882 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 --2- 172.21.15.114:0/915899321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee44118990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 --2- 172.21.15.114:0/915899321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410af90 0x7fee44115fd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 --2- 172.21.15.114:0/915899321 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410a640 0x7fee4410aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 -- 172.21.15.114:0/915899321 >> 172.21.15.114:0/915899321 conn(0x7fee4406e9f0 msgr2=0x7fee4406ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 -- 172.21.15.114:0/915899321 shutdown_connections 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.878+0000 7fee43e00640 1 -- 172.21.15.114:0/915899321 wait complete. 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.880+0000 7fee43e00640 1 Processor -- start 2024-09-21T07:06:21.883 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.880+0000 7fee43e00640 1 -- start start 2024-09-21T07:06:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.885+0000 7fee43e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410a640 0x7fee4410be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.885+0000 7fee43e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 0x7fee4410c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.887 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.885+0000 7fee43e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee4410c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.885+0000 7fee42a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410a640 0x7fee4410be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.885+0000 7fee42a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410a640 0x7fee4410be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46236/0 (socket says 172.21.15.114:46236) 2024-09-21T07:06:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.886+0000 7fee42000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 0x7fee4410c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.888 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.886+0000 7fee42000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 0x7fee4410c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58112/0 (socket says 172.21.15.114:58112) 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.887+0000 7fee43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee4410c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.887+0000 7fee43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee4410c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38120/0 (socket says 172.21.15.114:38120) 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.887+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fee4411c070 con 0x7fee4410af90 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.887+0000 7fee42a00640 1 -- 172.21.15.114:0/3837149214 learned_addr learned my addr 172.21.15.114:0/3837149214 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.887+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee4411bef0 con 0x7fee44116510 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.887+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee4411c1f0 con 0x7fee4410a640 2024-09-21T07:06:21.890 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.888+0000 7fee43400640 1 -- 172.21.15.114:0/3837149214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410a640 msgr2=0x7fee4410be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:21.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.888+0000 7fee43400640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410a640 0x7fee4410be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.888+0000 7fee43400640 1 -- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 msgr2=0x7fee4410c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:21.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.888+0000 7fee43400640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 0x7fee4410c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.888+0000 7fee43400640 1 -- 172.21.15.114:0/3837149214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee441b0ba0 con 0x7fee44116510 2024-09-21T07:06:21.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.889+0000 7fee42000640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 0x7fee4410c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:21.891 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.889+0000 7fee43400640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee4410c900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00ef00 tx=0x7fee3c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:21.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.889+0000 7fee48a00640 1 -- 172.21.15.114:0/3837149214 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c004b20 con 0x7fee44116510 2024-09-21T07:06:21.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.889+0000 7fee48a00640 1 -- 172.21.15.114:0/3837149214 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c003e20 con 0x7fee44116510 2024-09-21T07:06:21.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.889+0000 7fee48a00640 1 -- 172.21.15.114:0/3837149214 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00b570 con 0x7fee44116510 2024-09-21T07:06:21.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.889+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee441a2650 con 0x7fee44116510 2024-09-21T07:06:21.892 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.890+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee441a2b10 con 0x7fee44116510 2024-09-21T07:06:21.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.902+0000 7fee48a00640 1 -- 172.21.15.114:0/3837149214 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fee3c01d020 con 0x7fee44116510 2024-09-21T07:06:21.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.904+0000 7fee48a00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee280b6850 0x7fee280b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.904+0000 7fee48a00640 1 -- 172.21.15.114:0/3837149214 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fee3c0e7970 con 0x7fee44116510 2024-09-21T07:06:21.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.908+0000 7fee42a00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee280b6850 0x7fee280b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.909+0000 7fee42a00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee280b6850 0x7fee280b8d60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fee34005fd0 tx=0x7fee34013c90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.936+0000 7fee50a4a740 1 Processor -- start 2024-09-21T07:06:21.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.936+0000 7fee50a4a740 1 -- start start 2024-09-21T07:06:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.937+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563970107070 0x5639704adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.937+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 0x5639704ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.937+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700e6390 0x5639704ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee50a4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563970424f70 con 0x563970107070 2024-09-21T07:06:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee50a4a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639704143a0 con 0x5639700e6390 2024-09-21T07:06:21.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee50a4a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639704181b0 con 0x5639700de1b0 2024-09-21T07:06:21.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee42a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563970107070 0x5639704adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee42000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 0x5639704ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee42000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 0x5639704ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46250/0 (socket says 172.21.15.114:46250) 2024-09-21T07:06:21.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee42000640 1 -- 172.21.15.114:0/1050068334 learned_addr learned my addr 172.21.15.114:0/1050068334 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:21.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.938+0000 7fee43400640 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700e6390 0x5639704ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.939+0000 7fee42000640 1 -- 172.21.15.114:0/1050068334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700e6390 msgr2=0x5639704ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:21.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.939+0000 7fee42000640 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700e6390 0x5639704ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.939+0000 7fee42000640 1 -- 172.21.15.114:0/1050068334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563970107070 msgr2=0x5639704adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:21.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.939+0000 7fee42000640 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563970107070 0x5639704adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.939+0000 7fee42000640 1 -- 172.21.15.114:0/1050068334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639703dec90 con 0x5639700de1b0 2024-09-21T07:06:21.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.939+0000 7fee42000640 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 0x5639704ae470 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fee3002a860 tx=0x7fee30005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b83a48a516b5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.940+0000 7fee2fe00640 1 -- 172.21.15.114:0/1050068334 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee30037040 con 0x5639700de1b0 2024-09-21T07:06:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.940+0000 7fee2fe00640 1 -- 172.21.15.114:0/1050068334 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee30041070 con 0x5639700de1b0 2024-09-21T07:06:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.940+0000 7fee2fe00640 1 -- 172.21.15.114:0/1050068334 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee300054c0 con 0x5639700de1b0 2024-09-21T07:06:21.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 -- 172.21.15.114:0/1050068334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 msgr2=0x5639704ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 0x5639704ae470 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fee3002a860 tx=0x7fee30005db0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 -- 172.21.15.114:0/1050068334 shutdown_connections 2024-09-21T07:06:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700e6390 0x5639704ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700de1b0 0x5639704ae470 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1050068334 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563970107070 0x5639704adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.941+0000 7fee50a4a740 1 -- 172.21.15.114:0/1050068334 >> 172.21.15.114:0/1050068334 conn(0x5639700fb460 msgr2=0x5639704abe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.942+0000 7fee50a4a740 1 -- 172.21.15.114:0/1050068334 shutdown_connections 2024-09-21T07:06:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.942+0000 7fee50a4a740 1 -- 172.21.15.114:0/1050068334 wait complete. 2024-09-21T07:06:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.942+0000 7fee50a4a740 1 -- start start 2024-09-21T07:06:21.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.943+0000 7fee50a4a740 1 Processor -- start 2024-09-21T07:06:21.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.944+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.944+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee50a4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563970275700 con 0x5639700de1b0 2024-09-21T07:06:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee50a4a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639703af540 con 0x563970107070 2024-09-21T07:06:21.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38136/0 (socket says 172.21.15.114:38136) 2024-09-21T07:06:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee50a4a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639704bc6b0 con 0x5639700e6390 2024-09-21T07:06:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee43400640 1 -- 172.21.15.114:0/2802981198 learned_addr learned my addr 172.21.15.114:0/2802981198 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee42000640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.945+0000 7fee50a4a740 10 client.0 _ll_register_callbacks cb 0x5639703bf430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.946+0000 7fee42a00640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.946+0000 7fee43400640 1 -- 172.21.15.114:0/2802981198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 msgr2=0x5639704b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.946+0000 7fee43400640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.946+0000 7fee43400640 1 -- 172.21.15.114:0/2802981198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 msgr2=0x5639704b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.946+0000 7fee43400640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.946+0000 7fee43400640 1 -- 172.21.15.114:0/2802981198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563970172650 con 0x563970107070 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.947+0000 7fee43400640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704c59a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005fd0 tx=0x7fee3c0e5250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37a5cf2fef1fc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.947+0000 7fee2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:21.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.947+0000 7fee2cc00640 1 -- 172.21.15.114:0/2802981198 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c01a030 con 0x563970107070 2024-09-21T07:06:21.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.949+0000 7fee2cc00640 1 -- 172.21.15.114:0/2802981198 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fee3c003970 con 0x563970107070 2024-09-21T07:06:21.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.949+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639703b1b50 con 0x563970107070 2024-09-21T07:06:21.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.952+0000 7fee2cc00640 1 -- 172.21.15.114:0/2802981198 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fee3c0ada90 con 0x563970107070 2024-09-21T07:06:21.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.952+0000 7fee50a4a740 10 client.28508 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:21.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.952+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5639703cc910 con 0x563970107070 2024-09-21T07:06:21.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.953+0000 7fee2cc00640 1 -- 172.21.15.114:0/2802981198 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5639703b1b50 con 0x563970107070 2024-09-21T07:06:21.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee2cc00640 1 -- 172.21.15.114:0/2802981198 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fee3c00b040 con 0x563970107070 2024-09-21T07:06:21.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee50a4a740 10 client.28508 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:21.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee50a4a740 10 client.28508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee50a4a740 20 client.28508 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5639700c59f0 0x5639704cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee50a4a740 4 client.28508 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563970426ec0 con 0x5639700c59f0 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.954+0000 7fee42a00640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5639700c59f0 0x5639704cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.955+0000 7fee2cc00640 10 client.28508 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:21.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.955+0000 7fee42a00640 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5639700c59f0 0x5639704cbc00 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f150bd221243f03 server_cookie=55dd91083344a4a2 in_seq=0 out_seq=0 2024-09-21T07:06:21.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.958+0000 7fee2cc00640 1 -- 172.21.15.114:0/2802981198 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563970426ec0 con 0x5639700c59f0 2024-09-21T07:06:21.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.958+0000 7fee2cc00640 10 client.28508 handle_command_reply: tid=0 2024-09-21T07:06:21.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.958+0000 7fee50a4a740 1 client.28508 shutdown 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 msgr2=0x5639704c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704c59a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005fd0 tx=0x7fee3c0e5250 comp rx=0 tx=0).stop 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 shutdown_connections 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704c59a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5639700c59f0 0x5639704cbc00 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2802981198 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee2cc00640 0 client.28508 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:21.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 >> 172.21.15.114:0/2802981198 conn(0x5639700fb460 msgr2=0x5639704aa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:21.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 shutdown_connections 2024-09-21T07:06:21.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.960+0000 7fee50a4a740 1 -- 172.21.15.114:0/2802981198 wait complete. 2024-09-21T07:06:21.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.961+0000 7fee50a4a740 20 client.28508 trim_cache size 0 max 16384 2024-09-21T07:06:21.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.997+0000 7fee50a4a740 1 Processor -- start 2024-09-21T07:06:21.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.998+0000 7fee50a4a740 1 -- start start 2024-09-21T07:06:22.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.998+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700de1b0 0x5639704b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.998+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700e6390 0x5639704af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 0x5639704c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee50a4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b71d0 con 0x5639700e6390 2024-09-21T07:06:22.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee50a4a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b8d70 con 0x5639700de1b0 2024-09-21T07:06:22.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee50a4a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b78a0 con 0x563970107070 2024-09-21T07:06:22.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee43400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 0x5639704c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:22.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee43400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 0x5639704c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:46258/0 (socket says 172.21.15.114:46258) 2024-09-21T07:06:22.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee43400640 1 -- 172.21.15.114:0/2534264948 learned_addr learned my addr 172.21.15.114:0/2534264948 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:22.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:21.999+0000 7fee43400640 1 -- 172.21.15.114:0/2534264948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700de1b0 msgr2=0x5639704b4f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:22.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.001+0000 7fee43400640 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700de1b0 0x5639704b4f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.001+0000 7fee43400640 1 -- 172.21.15.114:0/2534264948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700e6390 msgr2=0x5639704af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:22.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.001+0000 7fee43400640 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700e6390 0x5639704af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.001+0000 7fee43400640 1 -- 172.21.15.114:0/2534264948 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639704baa10 con 0x563970107070 2024-09-21T07:06:22.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.002+0000 7fee43400640 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 0x5639704c58a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fee3c05a040 tx=0x7fee3c0ad1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d957cb54c29477 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:22.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.002+0000 7fee2f400640 1 -- 172.21.15.114:0/2534264948 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0c7070 con 0x563970107070 2024-09-21T07:06:22.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.002+0000 7fee2f400640 1 -- 172.21.15.114:0/2534264948 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0b6db0 con 0x563970107070 2024-09-21T07:06:22.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.002+0000 7fee2f400640 1 -- 172.21.15.114:0/2534264948 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c01a040 con 0x563970107070 2024-09-21T07:06:22.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.004+0000 7fee50a4a740 1 -- 172.21.15.114:0/2534264948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 msgr2=0x5639704c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.004+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 0x5639704c58a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fee3c05a040 tx=0x7fee3c0ad1a0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 -- 172.21.15.114:0/2534264948 shutdown_connections 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563970107070 0x5639704c58a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700e6390 0x5639704af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 --2- 172.21.15.114:0/2534264948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5639700de1b0 0x5639704b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 -- 172.21.15.114:0/2534264948 >> 172.21.15.114:0/2534264948 conn(0x5639700fb460 msgr2=0x5639704aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:22.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 -- 172.21.15.114:0/2534264948 shutdown_connections 2024-09-21T07:06:22.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.005+0000 7fee50a4a740 1 -- 172.21.15.114:0/2534264948 wait complete. 2024-09-21T07:06:22.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.006+0000 7fee50a4a740 1 -- start start 2024-09-21T07:06:22.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.006+0000 7fee50a4a740 1 Processor -- start 2024-09-21T07:06:22.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.007+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.007+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.007+0000 7fee50a4a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee42a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee50a4a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b8640 con 0x5639700de1b0 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee50a4a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b7f70 con 0x563970107070 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee50a4a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee441b51f0 con 0x5639700e6390 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee50a4a740 10 client.0 _ll_register_callbacks cb 0x7fee3c0acce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704bc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38150/0 (socket says 172.21.15.114:38150) 2024-09-21T07:06:22.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 -- 172.21.15.114:0/1327473409 learned_addr learned my addr 172.21.15.114:0/1327473409 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee42000640 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 -- 172.21.15.114:0/1327473409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 msgr2=0x5639704c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704c1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 -- 172.21.15.114:0/1327473409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 msgr2=0x5639704c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.008+0000 7fee43400640 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.009+0000 7fee43400640 1 -- 172.21.15.114:0/1327473409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639704bce00 con 0x563970107070 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.009+0000 7fee43400640 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704bc8c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fee3c021040 tx=0x7fee3c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f5d57f0d527fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.009+0000 7fee2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.009+0000 7fee2cc00640 1 -- 172.21.15.114:0/1327473409 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0c7030 con 0x563970107070 2024-09-21T07:06:22.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.010+0000 7fee2cc00640 1 -- 172.21.15.114:0/1327473409 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fee3c0a8e20 con 0x563970107070 2024-09-21T07:06:22.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.014+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c0b6db0 con 0x563970107070 2024-09-21T07:06:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.014+0000 7fee2cc00640 1 -- 172.21.15.114:0/1327473409 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fee3c01a040 con 0x563970107070 2024-09-21T07:06:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.014+0000 7fee50a4a740 10 client.28511 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:22.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.014+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563970536c70 con 0x563970107070 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee2cc00640 1 -- 172.21.15.114:0/1327473409 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee3c0a9020 con 0x563970107070 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee2cc00640 1 -- 172.21.15.114:0/1327473409 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fee3c094ad0 con 0x563970107070 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee50a4a740 10 client.28511 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee50a4a740 10 client.28511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee50a4a740 20 client.28511 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56397009ada0 0x563970539160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee50a4a740 4 client.28511 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:06:22.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fee3c0b6db0 con 0x56397009ada0 2024-09-21T07:06:22.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.015+0000 7fee42a00640 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56397009ada0 0x563970539160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:22.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.016+0000 7fee42a00640 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56397009ada0 0x563970539160 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8614276f126ae1 server_cookie=77f3b7cae381c4f7 in_seq=0 out_seq=0 2024-09-21T07:06:22.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.017+0000 7fee2cc00640 10 client.28511 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:22.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.017+0000 7fee2cc00640 1 -- 172.21.15.114:0/1327473409 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x7fee3c0b6db0 con 0x56397009ada0 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.017+0000 7fee2cc00640 10 client.28511 handle_command_reply: tid=0 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.017+0000 7fee50a4a740 1 client.28511 shutdown 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.017+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 msgr2=0x5639704bc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.017+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704bc8c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fee3c021040 tx=0x7fee3c00b040 comp rx=0 tx=0).stop 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 shutdown_connections 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563970107070 0x5639704bc8c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56397009ada0 0x563970539160 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5639700e6390 0x5639704c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee50a4a740 1 --2- 172.21.15.114:0/1327473409 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5639700de1b0 0x5639704c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 >> 172.21.15.114:0/1327473409 conn(0x5639700fb460 msgr2=0x5639704aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:22.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.018+0000 7fee2cc00640 0 client.28511 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.019+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 shutdown_connections 2024-09-21T07:06:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.019+0000 7fee50a4a740 1 -- 172.21.15.114:0/1327473409 wait complete. 2024-09-21T07:06:22.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.019+0000 7fee50a4a740 20 client.28511 trim_cache size 0 max 16384 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.023 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:06:22.024 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.026 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:06:22.027 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.028 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:22.029 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:22.030 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:06:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.023+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee280b6850 msgr2=0x7fee280b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.023+0000 7fee43e00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee280b6850 0x7fee280b8d60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fee34005fd0 tx=0x7fee34013c90 comp rx=0 tx=0).stop 2024-09-21T07:06:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.023+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 msgr2=0x7fee4410c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.023+0000 7fee43e00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee4410c900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00ef00 tx=0x7fee3c00c740 comp rx=0 tx=0).stop 2024-09-21T07:06:22.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.024+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 shutdown_connections 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.024+0000 7fee43e00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee280b6850 0x7fee280b8d60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.024+0000 7fee43e00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee44116510 0x7fee4410c900 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.024+0000 7fee43e00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee4410af90 0x7fee4410c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.024+0000 7fee43e00640 1 --2- 172.21.15.114:0/3837149214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee4410a640 0x7fee4410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.024+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 >> 172.21.15.114:0/3837149214 conn(0x7fee4406e9f0 msgr2=0x7fee4410df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.025+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 shutdown_connections 2024-09-21T07:06:22.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:22.025+0000 7fee43e00640 1 -- 172.21.15.114:0/3837149214 wait complete. 2024-09-21T07:06:22.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:22 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:06:22.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:22 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:22.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:22 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:06:22.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:22 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:22.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:22 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:06:22.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:22 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:06:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 705.0 s, 396.0 tps, lat 256.638 ms stddev 316.351 2024-09-21T07:06:23.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:23 smithi183 ceph-mon[34219]: pgmap v773: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-09-21T07:06:23.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:23 smithi193 ceph-mon[34601]: pgmap v773: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-09-21T07:06:23.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:23 smithi114 ceph-mon[30654]: pgmap v773: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 4.9 MiB/s wr, 475 op/s 2024-09-21T07:06:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:25 smithi183 ceph-mon[34219]: pgmap v774: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-09-21T07:06:25.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:25 smithi193 ceph-mon[34601]: pgmap v774: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-09-21T07:06:25.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:25 smithi114 ceph-mon[30654]: pgmap v774: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.4 MiB/s wr, 473 op/s 2024-09-21T07:06:25.892 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:06:25.892 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:06:26.658 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:27 smithi183 ceph-mon[34219]: pgmap v775: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-21T07:06:27.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:27.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:27.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:27 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:06:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:27 smithi193 ceph-mon[34601]: pgmap v775: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-21T07:06:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:27 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:06:27.665 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:27 smithi114 ceph-mon[30654]: pgmap v775: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 4.5 MiB/s wr, 446 op/s 2024-09-21T07:06:27.665 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:06:27.665 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:06:27.665 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:27 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:06:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 710.0 s, 469.2 tps, lat 212.180 ms stddev 297.258 2024-09-21T07:06:28.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.576+0000 7fe448800640 1 -- 172.21.15.114:0/2497716004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe44010a590 msgr2=0x7fe44010a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.576+0000 7fe448800640 1 --2- 172.21.15.114:0/2497716004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe44010a590 0x7fe44010a9a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe430009900 tx=0x7fe43002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 -- 172.21.15.114:0/2497716004 shutdown_connections 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 --2- 172.21.15.114:0/2497716004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe440116640 0x7fe440118b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 --2- 172.21.15.114:0/2497716004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 0x7fe440115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 --2- 172.21.15.114:0/2497716004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe44010a590 0x7fe44010a9a0 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe430009900 tx=0x7fe43002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 -- 172.21.15.114:0/2497716004 >> 172.21.15.114:0/2497716004 conn(0x7fe440070ec0 msgr2=0x7fe4400712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 -- 172.21.15.114:0/2497716004 shutdown_connections 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.577+0000 7fe448800640 1 -- 172.21.15.114:0/2497716004 wait complete. 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.579+0000 7fe448800640 1 Processor -- start 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.581+0000 7fe448800640 1 -- start start 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.582+0000 7fe448800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 0x7fe4401a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.582+0000 7fe448800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe440116640 0x7fe4401a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:28.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.582+0000 7fe448800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4401a85d0 0x7fe4401a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.582+0000 7fe445e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 0x7fe4401a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.582+0000 7fe445e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 0x7fe4401a2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34472/0 (socket says 172.21.15.114:34472) 2024-09-21T07:06:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.583+0000 7fe445e00640 1 -- 172.21.15.114:0/1427805296 learned_addr learned my addr 172.21.15.114:0/1427805296 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.583+0000 7fe446800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4401a85d0 0x7fe4401a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.583+0000 7fe445400640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe440116640 0x7fe4401a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:28.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.583+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe44011c320 con 0x7fe440116640 2024-09-21T07:06:28.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.583+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe44011c1a0 con 0x7fe44010aee0 2024-09-21T07:06:28.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.583+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe44011c4a0 con 0x7fe4401a85d0 2024-09-21T07:06:28.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.586+0000 7fe445400640 1 -- 172.21.15.114:0/1427805296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4401a85d0 msgr2=0x7fe4401a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:28.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.586+0000 7fe445400640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4401a85d0 0x7fe4401a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:28.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.586+0000 7fe445400640 1 -- 172.21.15.114:0/1427805296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 msgr2=0x7fe4401a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:28.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.586+0000 7fe445400640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 0x7fe4401a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:28.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.586+0000 7fe445400640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4401a91b0 con 0x7fe440116640 2024-09-21T07:06:28.589 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.587+0000 7fe445400640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe440116640 0x7fe4401a33f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe44010bd30 tx=0x7fe434007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:28.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.589+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434010050 con 0x7fe440116640 2024-09-21T07:06:28.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.590+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe434005020 con 0x7fe440116640 2024-09-21T07:06:28.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.590+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434004ba0 con 0x7fe440116640 2024-09-21T07:06:28.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.591+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44010bd60 con 0x7fe440116640 2024-09-21T07:06:28.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.591+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44010c220 con 0x7fe440116640 2024-09-21T07:06:28.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.593+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44010b5e0 con 0x7fe440116640 2024-09-21T07:06:28.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.605+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe4340101f0 con 0x7fe440116640 2024-09-21T07:06:28.614 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.611+0000 7fe43aa00640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4280b6850 0x7fe4280b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.611+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fe4340ebcb0 con 0x7fe440116640 2024-09-21T07:06:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.612+0000 7fe445e00640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4280b6850 0x7fe4280b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:28.615 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.612+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4340b1e50 con 0x7fe440116640 2024-09-21T07:06:28.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:28.615+0000 7fe445e00640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4280b6850 0x7fe4280b8d60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe430005ab0 tx=0x7fe430005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:29.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.062+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe44010a9a0 con 0x7fe440116640 2024-09-21T07:06:29.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.063+0000 7fe43aa00640 1 -- 172.21.15.114:0/1427805296 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11993 (secure 0 0 0) 0x7fe4340b6230 con 0x7fe440116640 2024-09-21T07:06:29.066 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:06:29.067 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 52 2024-09-21T07:06:29.067 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":52,"btime":"2024-09-21T07:06:21:307707+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:06:20.622407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:06:29.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.067+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4280b6850 msgr2=0x7fe4280b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:29.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.067+0000 7fe448800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4280b6850 0x7fe4280b8d60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fe430005ab0 tx=0x7fe430005a20 comp rx=0 tx=0).stop 2024-09-21T07:06:29.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.067+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe440116640 msgr2=0x7fe4401a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:29.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.067+0000 7fe448800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe440116640 0x7fe4401a33f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe44010bd30 tx=0x7fe434007590 comp rx=0 tx=0).stop 2024-09-21T07:06:29.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.068+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 shutdown_connections 2024-09-21T07:06:29.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.068+0000 7fe448800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4401a85d0 0x7fe4401a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:29.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.068+0000 7fe448800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe440116640 0x7fe4401a33f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:29.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.068+0000 7fe448800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe4280b6850 0x7fe4280b8d60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:29.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.068+0000 7fe448800640 1 --2- 172.21.15.114:0/1427805296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe44010aee0 0x7fe4401a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:29.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.068+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 >> 172.21.15.114:0/1427805296 conn(0x7fe440070ec0 msgr2=0x7fe440073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:29.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.069+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 shutdown_connections 2024-09-21T07:06:29.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:29.069+0000 7fe448800640 1 -- 172.21.15.114:0/1427805296 wait complete. 2024-09-21T07:06:29.645 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:29 smithi114 ceph-mon[30654]: pgmap v776: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.0 MiB/s wr, 480 op/s 2024-09-21T07:06:29.645 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:29 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1427805296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:29.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:29 smithi193 ceph-mon[34601]: pgmap v776: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.0 MiB/s wr, 480 op/s 2024-09-21T07:06:29.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:29 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1427805296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:29.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:29 smithi183 ceph-mon[34219]: pgmap v776: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 5.0 MiB/s wr, 480 op/s 2024-09-21T07:06:29.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:29 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1427805296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:30.042 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:06:30.743 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:31.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:31 smithi193 ceph-mon[34601]: pgmap v777: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-21T07:06:31.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:31 smithi114 ceph-mon[30654]: pgmap v777: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-21T07:06:31.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:31 smithi183 ceph-mon[34219]: pgmap v777: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 4.6 MiB/s wr, 455 op/s 2024-09-21T07:06:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 715.0 s, 433.6 tps, lat 236.464 ms stddev 302.648 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 -- 172.21.15.114:0/1301926964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481050f0 msgr2=0x7fc64810d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 --2- 172.21.15.114:0/1301926964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481050f0 0x7fc64810d630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc634009900 tx=0x7fc63402a420 comp rx=0 tx=0).stop 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 -- 172.21.15.114:0/1301926964 shutdown_connections 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 --2- 172.21.15.114:0/1301926964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc64810dbc0 0x7fc6481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 --2- 172.21.15.114:0/1301926964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481050f0 0x7fc64810d630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 --2- 172.21.15.114:0/1301926964 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc6481047a0 0x7fc648104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.079+0000 7fc64ec00640 1 -- 172.21.15.114:0/1301926964 >> 172.21.15.114:0/1301926964 conn(0x7fc6480fe230 msgr2=0x7fc648100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.080+0000 7fc64ec00640 1 -- 172.21.15.114:0/1301926964 shutdown_connections 2024-09-21T07:06:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.080+0000 7fc64ec00640 1 -- 172.21.15.114:0/1301926964 wait complete. 2024-09-21T07:06:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.081+0000 7fc64ec00640 1 Processor -- start 2024-09-21T07:06:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 -- start start 2024-09-21T07:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481047a0 0x7fc648109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 0x7fc64810a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc64810dbc0 0x7fc64810a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc648102e00 con 0x7fc64810dbc0 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc648102c80 con 0x7fc6481047a0 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.082+0000 7fc64ec00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc648102f80 con 0x7fc6481050f0 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 0x7fc64810a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 0x7fc64810a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59720/0 (socket says 172.21.15.114:59720) 2024-09-21T07:06:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 -- 172.21.15.114:0/3256777247 learned_addr learned my addr 172.21.15.114:0/3256777247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 -- 172.21.15.114:0/3256777247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481047a0 msgr2=0x7fc648109b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481047a0 0x7fc648109b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 -- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc64810dbc0 msgr2=0x7fc64810a600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc64810dbc0 0x7fc64810a600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.083+0000 7fc64ce00640 1 -- 172.21.15.114:0/3256777247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6481a42a0 con 0x7fc6481050f0 2024-09-21T07:06:33.087 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.085+0000 7fc64ce00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 0x7fc64810a0c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc63402aa10 tx=0x7fc634003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.085+0000 7fc646000640 1 -- 172.21.15.114:0/3256777247 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63403f070 con 0x7fc6481050f0 2024-09-21T07:06:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.086+0000 7fc646000640 1 -- 172.21.15.114:0/3256777247 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc634031d60 con 0x7fc6481050f0 2024-09-21T07:06:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.086+0000 7fc646000640 1 -- 172.21.15.114:0/3256777247 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634035030 con 0x7fc6481050f0 2024-09-21T07:06:33.089 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.086+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6481a4530 con 0x7fc6481050f0 2024-09-21T07:06:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.086+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6481b5a90 con 0x7fc6481050f0 2024-09-21T07:06:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.093+0000 7fc657239740 1 Processor -- start 2024-09-21T07:06:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.093+0000 7fc646000640 1 -- 172.21.15.114:0/3256777247 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc634042070 con 0x7fc6481050f0 2024-09-21T07:06:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.094+0000 7fc657239740 1 -- start start 2024-09-21T07:06:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.094+0000 7fc657239740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 0x55f2dbcaae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.095+0000 7fc657239740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.095+0000 7fc657239740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.095+0000 7fc657239740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2dbc206b0 con 0x55f2db9061b0 2024-09-21T07:06:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.095+0000 7fc657239740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2dbc0fae0 con 0x55f2db8dd2f0 2024-09-21T07:06:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.095+0000 7fc657239740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f2dbc138f0 con 0x55f2db8e54d0 2024-09-21T07:06:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.096+0000 7fc64e200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 0x55f2dbcaae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.096+0000 7fc646000640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6300b6780 0x7fc6300b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.097+0000 7fc646000640 1 -- 172.21.15.114:0/3256777247 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fc6341085b0 con 0x7fc6481050f0 2024-09-21T07:06:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.098+0000 7fc64e200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 0x55f2dbcaae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52844/0 (socket says 172.21.15.114:52844) 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.098+0000 7fc64ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.098+0000 7fc64ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59736/0 (socket says 172.21.15.114:59736) 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.098+0000 7fc64ce00640 1 -- 172.21.15.114:0/3254295141 learned_addr learned my addr 172.21.15.114:0/3254295141 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.098+0000 7fc64ce00640 1 -- 172.21.15.114:0/3254295141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8dd2f0 msgr2=0x55f2dbcab370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.099+0000 7fc64d800640 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.099+0000 7fc64ce00640 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.099+0000 7fc64ce00640 1 -- 172.21.15.114:0/3254295141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 msgr2=0x55f2dbcaae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.099+0000 7fc64ce00640 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 0x55f2dbcaae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.099+0000 7fc64ce00640 1 -- 172.21.15.114:0/3254295141 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2dbbdc780 con 0x55f2db8e54d0 2024-09-21T07:06:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.100+0000 7fc64e200640 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 0x55f2dbcaae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.100+0000 7fc64e200640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6300b6780 0x7fc6300b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.100+0000 7fc64ce00640 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcab8b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc6340cb240 tx=0x7fc634031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd5de65bca34959 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.101+0000 7fc63aa00640 1 -- 172.21.15.114:0/3254295141 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634049070 con 0x55f2db8e54d0 2024-09-21T07:06:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.101+0000 7fc63aa00640 1 -- 172.21.15.114:0/3254295141 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6340cdae0 con 0x55f2db8e54d0 2024-09-21T07:06:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.101+0000 7fc63aa00640 1 -- 172.21.15.114:0/3254295141 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6340d79f0 con 0x55f2db8e54d0 2024-09-21T07:06:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.102+0000 7fc64e200640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6300b6780 0x7fc6300b8c90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc640009f20 tx=0x7fc640004a80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.102+0000 7fc657239740 1 -- 172.21.15.114:0/3254295141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 msgr2=0x55f2dbcab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.102+0000 7fc657239740 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcab8b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc6340cb240 tx=0x7fc634031f10 comp rx=0 tx=0).stop 2024-09-21T07:06:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.102+0000 7fc657239740 1 -- 172.21.15.114:0/3254295141 shutdown_connections 2024-09-21T07:06:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.103+0000 7fc657239740 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcab8b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.103+0000 7fc657239740 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.103+0000 7fc657239740 1 --2- 172.21.15.114:0/3254295141 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db9061b0 0x55f2dbcaae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.103+0000 7fc657239740 1 -- 172.21.15.114:0/3254295141 >> 172.21.15.114:0/3254295141 conn(0x55f2db8fa300 msgr2=0x55f2dbca8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.103+0000 7fc657239740 1 -- 172.21.15.114:0/3254295141 shutdown_connections 2024-09-21T07:06:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.103+0000 7fc657239740 1 -- 172.21.15.114:0/3254295141 wait complete. 2024-09-21T07:06:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.104+0000 7fc657239740 1 -- start start 2024-09-21T07:06:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 Processor -- start 2024-09-21T07:06:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8e54d0 0x55f2dbcb6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcc2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f2dbb47ab0 con 0x55f2db8e54d0 2024-09-21T07:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f2dbbb6570 con 0x55f2db9061b0 2024-09-21T07:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f2dbcb9370 con 0x55f2db8dd2f0 2024-09-21T07:06:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.105+0000 7fc657239740 10 client.0 _ll_register_callbacks cb 0x55f2dbbc5100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.106+0000 7fc64d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.106+0000 7fc64d800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59752/0 (socket says 172.21.15.114:59752) 2024-09-21T07:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.106+0000 7fc64d800640 1 -- 172.21.15.114:0/523554004 learned_addr learned my addr 172.21.15.114:0/523554004 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.106+0000 7fc64e200640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcc2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64d800640 1 -- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 msgr2=0x55f2dbcc2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64d800640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcc2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64ce00640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8e54d0 0x55f2dbcb6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64d800640 1 -- 172.21.15.114:0/523554004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8e54d0 msgr2=0x55f2dbcb6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64d800640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8e54d0 0x55f2dbcb6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64d800640 1 -- 172.21.15.114:0/523554004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2db971790 con 0x55f2db8dd2f0 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.107+0000 7fc64e200640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcc2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.108+0000 7fc64d800640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00aee0 tx=0x7fc63c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13305630c8145280 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.108+0000 7fc62f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.108+0000 7fc62f400640 1 -- 172.21.15.114:0/523554004 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c015070 con 0x55f2db8dd2f0 2024-09-21T07:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.108+0000 7fc62f400640 1 -- 172.21.15.114:0/523554004 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fc63c0115f0 con 0x55f2db8dd2f0 2024-09-21T07:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.109+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6340cdae0 con 0x55f2db8dd2f0 2024-09-21T07:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.109+0000 7fc62f400640 1 -- 172.21.15.114:0/523554004 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc63c01f050 con 0x55f2db8dd2f0 2024-09-21T07:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.110+0000 7fc657239740 10 client.28480 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.110+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f2db9ea620 con 0x55f2db8dd2f0 2024-09-21T07:06:33.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.110+0000 7fc62f400640 1 -- 172.21.15.114:0/523554004 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6340cdae0 con 0x55f2db8dd2f0 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc62f400640 1 -- 172.21.15.114:0/523554004 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fc63c01a960 con 0x55f2db8dd2f0 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc657239740 10 client.28480 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc657239740 10 client.28480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc657239740 20 client.28480 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc657239740 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db8c4d80 0x55f2dbcc84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc657239740 4 client.28480 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.112+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2dbc22600 con 0x55f2db8c4d80 2024-09-21T07:06:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.113+0000 7fc64ce00640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db8c4d80 0x55f2dbcc84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.114+0000 7fc62f400640 10 client.28480 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.114+0000 7fc64ce00640 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db8c4d80 0x55f2dbcc84e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56cb00d0f8558a3 server_cookie=fb9e49703b59a041 in_seq=0 out_seq=0 2024-09-21T07:06:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.115+0000 7fc62f400640 1 -- 172.21.15.114:0/523554004 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f2dbc22600 con 0x55f2db8c4d80 2024-09-21T07:06:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.115+0000 7fc62f400640 10 client.28480 handle_command_reply: tid=0 2024-09-21T07:06:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.115+0000 7fc657239740 1 client.28480 shutdown 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 msgr2=0x55f2dbcb6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00aee0 tx=0x7fc63c00c520 comp rx=0 tx=0).stop 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 shutdown_connections 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcc2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8e54d0 0x55f2dbcb6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db8c4d80 0x55f2dbcc84e0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 --2- 172.21.15.114:0/523554004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6540 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc62f400640 0 client.28480 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.116+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 >> 172.21.15.114:0/523554004 conn(0x55f2db8fa300 msgr2=0x55f2dbcabe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.117+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 shutdown_connections 2024-09-21T07:06:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.117+0000 7fc657239740 1 -- 172.21.15.114:0/523554004 wait complete. 2024-09-21T07:06:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.118+0000 7fc657239740 20 client.28480 trim_cache size 0 max 16384 2024-09-21T07:06:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.164+0000 7fc657239740 1 Processor -- start 2024-09-21T07:06:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.164+0000 7fc657239740 1 -- start start 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc657239740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc657239740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8e54d0 0x55f2dbcac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc657239740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 0x55f2dbcc24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc657239740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481bfca0 con 0x55f2db8dd2f0 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc657239740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481c1840 con 0x55f2db8e54d0 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc657239740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481c0370 con 0x55f2db9061b0 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc64e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 0x55f2dbcc24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc64e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 0x55f2dbcc24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59762/0 (socket says 172.21.15.114:59762) 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc64e200640 1 -- 172.21.15.114:0/1608160487 learned_addr learned my addr 172.21.15.114:0/1608160487 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.165+0000 7fc64d800640 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.166+0000 7fc64e200640 1 -- 172.21.15.114:0/1608160487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8e54d0 msgr2=0x55f2dbcac570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.166+0000 7fc64ce00640 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8e54d0 0x55f2dbcac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.166+0000 7fc64e200640 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8e54d0 0x55f2dbcac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.166+0000 7fc64e200640 1 -- 172.21.15.114:0/1608160487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 msgr2=0x55f2dbcb6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.166+0000 7fc64e200640 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.166+0000 7fc64e200640 1 -- 172.21.15.114:0/1608160487 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2dbcb0a30 con 0x55f2db9061b0 2024-09-21T07:06:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.167+0000 7fc64ce00640 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8e54d0 0x55f2dbcac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:06:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.167+0000 7fc64e200640 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 0x55f2dbcc24a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc640016bc0 tx=0x7fc640011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca75d0c8d010967 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.168+0000 7fc63a000640 1 -- 172.21.15.114:0/1608160487 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000acf0 con 0x55f2db9061b0 2024-09-21T07:06:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.168+0000 7fc63a000640 1 -- 172.21.15.114:0/1608160487 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc640038070 con 0x55f2db9061b0 2024-09-21T07:06:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.168+0000 7fc63a000640 1 -- 172.21.15.114:0/1608160487 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640014480 con 0x55f2db9061b0 2024-09-21T07:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 -- 172.21.15.114:0/1608160487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 msgr2=0x55f2dbcc24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 0x55f2dbcc24a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc640016bc0 tx=0x7fc640011000 comp rx=0 tx=0).stop 2024-09-21T07:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 -- 172.21.15.114:0/1608160487 shutdown_connections 2024-09-21T07:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db9061b0 0x55f2dbcc24a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db8e54d0 0x55f2dbcac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 --2- 172.21.15.114:0/1608160487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcb6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 -- 172.21.15.114:0/1608160487 >> 172.21.15.114:0/1608160487 conn(0x55f2db8fa300 msgr2=0x55f2dbcabe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 -- 172.21.15.114:0/1608160487 shutdown_connections 2024-09-21T07:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.169+0000 7fc657239740 1 -- 172.21.15.114:0/1608160487 wait complete. 2024-09-21T07:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 -- start start 2024-09-21T07:06:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 Processor -- start 2024-09-21T07:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcbee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481c1110 con 0x55f2db8dd2f0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481c0a40 con 0x55f2db9061b0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481bdca0 con 0x55f2db8e54d0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.170+0000 7fc657239740 10 client.0 _ll_register_callbacks cb 0x7fc63c006e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcba2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59774/0 (socket says 172.21.15.114:59774) 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 -- 172.21.15.114:0/807662080 learned_addr learned my addr 172.21.15.114:0/807662080 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64e200640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64d800640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcbee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 -- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 msgr2=0x55f2dbcba960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcba960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 -- 172.21.15.114:0/807662080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 msgr2=0x55f2dbcbee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcbee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64ce00640 1 -- 172.21.15.114:0/807662080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2dbcbb0d0 con 0x55f2db8e54d0 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.171+0000 7fc64d800640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcbee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc64ce00640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcba2c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc6340cdcd0 tx=0x7fc6340cd840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=323006c315587425 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc62f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc62f400640 1 -- 172.21.15.114:0/807662080 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc634049040 con 0x55f2db8e54d0 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc62f400640 1 -- 172.21.15.114:0/807662080 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fc634108120 con 0x55f2db8e54d0 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc640038070 con 0x55f2db8e54d0 2024-09-21T07:06:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc62f400640 1 -- 172.21.15.114:0/807662080 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc634105d50 con 0x55f2db8e54d0 2024-09-21T07:06:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc657239740 10 client.28486 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.172+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55f2dbd32ea0 con 0x55f2db8e54d0 2024-09-21T07:06:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.173+0000 7fc62f400640 1 -- 172.21.15.114:0/807662080 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc640038070 con 0x55f2db8e54d0 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc62f400640 1 -- 172.21.15.114:0/807662080 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fc6340b9070 con 0x55f2db8e54d0 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc657239740 10 client.28486 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc657239740 10 client.28486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc657239740 20 client.28486 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc657239740 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db89a1c0 0x55f2dbd351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc657239740 4 client.28486 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2dbc1e0b0 con 0x55f2db89a1c0 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.174+0000 7fc64d800640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db89a1c0 0x55f2dbd351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.175+0000 7fc64d800640 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db89a1c0 0x55f2dbd351f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454b927db3cfde74 server_cookie=a401542f16791801 in_seq=0 out_seq=0 2024-09-21T07:06:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.175+0000 7fc62f400640 10 client.28486 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.195+0000 7fc62f400640 1 -- 172.21.15.114:0/807662080 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2dbc1e0b0 con 0x55f2db89a1c0 2024-09-21T07:06:33.198 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.195+0000 7fc62f400640 10 client.28486 handle_command_reply: tid=0 2024-09-21T07:06:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.196+0000 7fc657239740 1 client.28486 shutdown 2024-09-21T07:06:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.196+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 msgr2=0x55f2dbcba2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.196+0000 7fc657239740 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcba2c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc6340cdcd0 tx=0x7fc6340cd840 comp rx=0 tx=0).stop 2024-09-21T07:06:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.196+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 shutdown_connections 2024-09-21T07:06:33.199 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc657239740 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f2db9061b0 0x55f2dbcba960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc657239740 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f2db89a1c0 0x55f2dbd351f0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc657239740 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f2db8e54d0 0x55f2dbcba2c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc657239740 1 --2- 172.21.15.114:0/807662080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f2db8dd2f0 0x55f2dbcbee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 >> 172.21.15.114:0/807662080 conn(0x55f2db8fa300 msgr2=0x55f2dbcabe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc62f400640 0 client.28486 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.197+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 shutdown_connections 2024-09-21T07:06:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.198+0000 7fc657239740 1 -- 172.21.15.114:0/807662080 wait complete. 2024-09-21T07:06:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.198+0000 7fc657239740 20 client.28486 trim_cache size 0 max 16384 2024-09-21T07:06:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:06:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:06:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:06:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:06:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6300b6780 msgr2=0x7fc6300b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6300b6780 0x7fc6300b8c90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc640009f20 tx=0x7fc640004a80 comp rx=0 tx=0).stop 2024-09-21T07:06:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 msgr2=0x7fc64810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 0x7fc64810a0c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc63402aa10 tx=0x7fc634003d70 comp rx=0 tx=0).stop 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 shutdown_connections 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc64810dbc0 0x7fc64810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6481050f0 0x7fc64810a0c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc6300b6780 0x7fc6300b8c90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.200+0000 7fc64ec00640 1 --2- 172.21.15.114:0/3256777247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6481047a0 0x7fc648109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.201+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 >> 172.21.15.114:0/3256777247 conn(0x7fc6480fe230 msgr2=0x7fc64810fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.201+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 shutdown_connections 2024-09-21T07:06:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:33.201+0000 7fc64ec00640 1 -- 172.21.15.114:0/3256777247 wait complete. 2024-09-21T07:06:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:33 smithi193 ceph-mon[34601]: pgmap v778: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 4.3 MiB/s wr, 414 op/s 2024-09-21T07:06:33.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:33 smithi114 ceph-mon[30654]: pgmap v778: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 4.3 MiB/s wr, 414 op/s 2024-09-21T07:06:33.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:33 smithi183 ceph-mon[34219]: pgmap v778: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 4.3 MiB/s wr, 414 op/s 2024-09-21T07:06:33.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.978+0000 7f792a200640 1 -- 172.21.15.114:0/2987885863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7924102440 msgr2=0x7f7924102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.978+0000 7f792a200640 1 --2- 172.21.15.114:0/2987885863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7924102440 0x7f7924102850 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7918009900 tx=0x7f791802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.979+0000 7f792a200640 1 -- 172.21.15.114:0/2987885863 shutdown_connections 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.979+0000 7f792a200640 1 --2- 172.21.15.114:0/2987885863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f792410fff0 0x7f7924112410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.979+0000 7f792a200640 1 --2- 172.21.15.114:0/2987885863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102d90 0x7f792410fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.979+0000 7f792a200640 1 --2- 172.21.15.114:0/2987885863 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7924102440 0x7f7924102850 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.979+0000 7f792a200640 1 -- 172.21.15.114:0/2987885863 >> 172.21.15.114:0/2987885863 conn(0x7f79240fe2c0 msgr2=0x7f7924100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.979+0000 7f792a200640 1 -- 172.21.15.114:0/2987885863 shutdown_connections 2024-09-21T07:06:33.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.980+0000 7f792a200640 1 -- 172.21.15.114:0/2987885863 wait complete. 2024-09-21T07:06:33.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.981+0000 7f792a200640 1 Processor -- start 2024-09-21T07:06:33.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.982+0000 7f792a200640 1 -- start start 2024-09-21T07:06:33.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.982+0000 7f792a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 0x7f792419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.983+0000 7f792a200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7924102d90 0x7f792419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.983+0000 7f792a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792410fff0 0x7f79241a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.983+0000 7f792a200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7924115ba0 con 0x7f792410fff0 2024-09-21T07:06:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.983+0000 7f792a200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7924115a20 con 0x7f7924102d90 2024-09-21T07:06:33.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.983+0000 7f792a200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7924115d20 con 0x7f7924102440 2024-09-21T07:06:33.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.986+0000 7f7928e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 0x7f792419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.986+0000 7f7928e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 0x7f792419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59788/0 (socket says 172.21.15.114:59788) 2024-09-21T07:06:33.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.986+0000 7f7928e00640 1 -- 172.21.15.114:0/3517678482 learned_addr learned my addr 172.21.15.114:0/3517678482 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:33.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.986+0000 7f7929800640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792410fff0 0x7f79241a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.986+0000 7f7923e00640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7924102d90 0x7f792419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.987+0000 7f7929800640 1 -- 172.21.15.114:0/3517678482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 msgr2=0x7f792419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.987+0000 7f7929800640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 0x7f792419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.987+0000 7f7929800640 1 -- 172.21.15.114:0/3517678482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7924102d90 msgr2=0x7f792419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.987+0000 7f7929800640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7924102d90 0x7f792419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:33.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.987+0000 7f7929800640 1 -- 172.21.15.114:0/3517678482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241a25a0 con 0x7f792410fff0 2024-09-21T07:06:33.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.987+0000 7f7928e00640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 0x7f792419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:33.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.988+0000 7f7929800640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792410fff0 0x7f79241a1e70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f791c00be20 tx=0x7f791c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:33.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.990+0000 7f7921600640 1 -- 172.21.15.114:0/3517678482 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c019070 con 0x7f792410fff0 2024-09-21T07:06:33.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.990+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79241a2890 con 0x7f792410fff0 2024-09-21T07:06:33.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.990+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241a8ff0 con 0x7f792410fff0 2024-09-21T07:06:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.991+0000 7f7921600640 1 -- 172.21.15.114:0/3517678482 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791c0148f0 con 0x7f792410fff0 2024-09-21T07:06:33.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:33.991+0000 7f7921600640 1 -- 172.21.15.114:0/3517678482 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c019070 con 0x7f792410fff0 2024-09-21T07:06:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.002+0000 7f7921600640 1 -- 172.21.15.114:0/3517678482 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f791c01c070 con 0x7f792410fff0 2024-09-21T07:06:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.005+0000 7f7921600640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f790c0b6980 0x7f790c0b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.005+0000 7f7921600640 1 -- 172.21.15.114:0/3517678482 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f791c0e7e50 con 0x7f792410fff0 2024-09-21T07:06:34.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.005+0000 7f7928e00640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f790c0b6980 0x7f790c0b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.017+0000 7f7928e00640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f790c0b6980 0x7f790c0b8e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7918000c00 tx=0x7f7918031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:34.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.018+0000 7f79323c8740 1 Processor -- start 2024-09-21T07:06:34.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.021+0000 7f79323c8740 1 -- start start 2024-09-21T07:06:34.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.022+0000 7f79323c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.022+0000 7f79323c8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.022+0000 7f79323c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.022+0000 7f79323c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575e0647fb0 con 0x5575e032a070 2024-09-21T07:06:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.022+0000 7f79323c8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575e06373e0 con 0x5575e03011b0 2024-09-21T07:06:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.022+0000 7f79323c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575e063b1f0 con 0x5575e0309390 2024-09-21T07:06:34.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.023+0000 7f7923e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.023+0000 7f7923e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34580/0 (socket says 172.21.15.114:34580) 2024-09-21T07:06:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.023+0000 7f7923e00640 1 -- 172.21.15.114:0/1258821645 learned_addr learned my addr 172.21.15.114:0/1258821645 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.023+0000 7f7929800640 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.024+0000 7f7928e00640 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.024+0000 7f7929800640 1 -- 172.21.15.114:0/1258821645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 msgr2=0x5575e06d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.024+0000 7f7929800640 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.024+0000 7f7929800640 1 -- 172.21.15.114:0/1258821645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 msgr2=0x5575e06d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.024+0000 7f7929800640 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.024+0000 7f7929800640 1 -- 172.21.15.114:0/1258821645 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e0601cd0 con 0x5575e0309390 2024-09-21T07:06:34.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.025+0000 7f7929800640 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06d17b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f791c0e5170 tx=0x7f791c0ad640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2be82c3280f46c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.030+0000 7f7912000640 1 -- 172.21.15.114:0/1258821645 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c0e55f0 con 0x5575e0309390 2024-09-21T07:06:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.030+0000 7f7912000640 1 -- 172.21.15.114:0/1258821645 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791c0ad380 con 0x5575e0309390 2024-09-21T07:06:34.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.030+0000 7f7912000640 1 -- 172.21.15.114:0/1258821645 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c014aa0 con 0x5575e0309390 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 -- 172.21.15.114:0/1258821645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 msgr2=0x5575e06d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06d17b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f791c0e5170 tx=0x7f791c0ad640 comp rx=0 tx=0).stop 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 -- 172.21.15.114:0/1258821645 shutdown_connections 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06d17b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 --2- 172.21.15.114:0/1258821645 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.031+0000 7f79323c8740 1 -- 172.21.15.114:0/1258821645 >> 172.21.15.114:0/1258821645 conn(0x5575e031e460 msgr2=0x5575e06cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:34.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.032+0000 7f79323c8740 1 -- 172.21.15.114:0/1258821645 shutdown_connections 2024-09-21T07:06:34.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.032+0000 7f79323c8740 1 -- 172.21.15.114:0/1258821645 wait complete. 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.033+0000 7f79323c8740 1 -- start start 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.035+0000 7f79323c8740 1 Processor -- start 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.036+0000 7f79323c8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.036+0000 7f79323c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06dcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.036+0000 7f79323c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06e87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.036+0000 7f79323c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575e0498700 con 0x5575e032a070 2024-09-21T07:06:34.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.036+0000 7f79323c8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575e05d2580 con 0x5575e03011b0 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.036+0000 7f79323c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575e06df450 con 0x5575e0309390 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.037+0000 7f79323c8740 10 client.0 _ll_register_callbacks cb 0x5575e05e2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.037+0000 7f7928e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.037+0000 7f7928e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06dca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34582/0 (socket says 172.21.15.114:34582) 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.037+0000 7f7928e00640 1 -- 172.21.15.114:0/788275720 learned_addr learned my addr 172.21.15.114:0/788275720 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.037+0000 7f7929800640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06e87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.037+0000 7f7923e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06dcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.038+0000 7f7928e00640 1 -- 172.21.15.114:0/788275720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 msgr2=0x5575e06dcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.038+0000 7f7928e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06dcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.038+0000 7f7928e00640 1 -- 172.21.15.114:0/788275720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 msgr2=0x5575e06e87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.038+0000 7f7928e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06e87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.038+0000 7f7928e00640 1 -- 172.21.15.114:0/788275720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e0395650 con 0x5575e03011b0 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.039+0000 7f7928e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06dca00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f79180316b0 tx=0x7f7918009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6744907e6fc4323d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:34.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.039+0000 7f7906a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:34.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.039+0000 7f7923e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06dcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:34.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.039+0000 7f7929800640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06e87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:34.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.039+0000 7f7906a00640 1 -- 172.21.15.114:0/788275720 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791804d070 con 0x5575e03011b0 2024-09-21T07:06:34.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.039+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575e05d4b90 con 0x5575e03011b0 2024-09-21T07:06:34.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.040+0000 7f7906a00640 1 -- 172.21.15.114:0/788275720 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f7918049460 con 0x5575e03011b0 2024-09-21T07:06:34.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.040+0000 7f7906a00640 1 -- 172.21.15.114:0/788275720 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7918059050 con 0x5575e03011b0 2024-09-21T07:06:34.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.040+0000 7f79323c8740 10 client.28532 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:34.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.040+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5575e05ef950 con 0x5575e03011b0 2024-09-21T07:06:34.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.041+0000 7f7906a00640 1 -- 172.21.15.114:0/788275720 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575e05d4b90 con 0x5575e03011b0 2024-09-21T07:06:34.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.042+0000 7f7906a00640 1 -- 172.21.15.114:0/788275720 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f7918049870 con 0x5575e03011b0 2024-09-21T07:06:34.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.043+0000 7f79323c8740 10 client.28532 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:34.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.044+0000 7f79323c8740 10 client.28532 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:06:34.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.044+0000 7f79323c8740 20 client.28532 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:34.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.044+0000 7f79323c8740 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02e89f0 0x5575e06ee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.044+0000 7f79323c8740 4 client.28532 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:34.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.044+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e0649f00 con 0x5575e02e89f0 2024-09-21T07:06:34.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.044+0000 7f7923e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02e89f0 0x5575e06ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.045+0000 7f7923e00640 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02e89f0 0x5575e06ee980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abf25ff63b2d5cac server_cookie=12d245338a31b058 in_seq=0 out_seq=0 2024-09-21T07:06:34.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.045+0000 7f7906a00640 10 client.28532 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:06:34.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.046+0000 7f7906a00640 1 -- 172.21.15.114:0/788275720 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575e0649f00 con 0x5575e02e89f0 2024-09-21T07:06:34.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.046+0000 7f7906a00640 10 client.28532 handle_command_reply: tid=0 2024-09-21T07:06:34.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.046+0000 7f79323c8740 1 client.28532 shutdown 2024-09-21T07:06:34.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 msgr2=0x5575e06dca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:34.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06dca00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f79180316b0 tx=0x7f7918009260 comp rx=0 tx=0).stop 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 shutdown_connections 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06e87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e0309390 0x5575e06dcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02e89f0 0x5575e06ee980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 --2- 172.21.15.114:0/788275720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06dca00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 >> 172.21.15.114:0/788275720 conn(0x5575e031e460 msgr2=0x5575e06d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:34.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.047+0000 7f7906a00640 0 client.28532 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:06:34.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.048+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 shutdown_connections 2024-09-21T07:06:34.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.050+0000 7f79323c8740 1 -- 172.21.15.114:0/788275720 wait complete. 2024-09-21T07:06:34.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.050+0000 7f79323c8740 20 client.28532 trim_cache size 0 max 16384 2024-09-21T07:06:34.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.076+0000 7f79323c8740 1 Processor -- start 2024-09-21T07:06:34.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.076+0000 7f79323c8740 1 -- start start 2024-09-21T07:06:34.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f79323c8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f79323c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e0309390 0x5575e06d6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f79323c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e032a070 0x5575e06ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f79323c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7908010030 con 0x5575e0309390 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f79323c8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7908011bd0 con 0x5575e03011b0 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f79323c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7908010700 con 0x5575e032a070 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f7928e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f7928e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34590/0 (socket says 172.21.15.114:34590) 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.078+0000 7f7928e00640 1 -- 172.21.15.114:0/1264167571 learned_addr learned my addr 172.21.15.114:0/1264167571 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7928e00640 1 -- 172.21.15.114:0/1264167571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e032a070 msgr2=0x5575e06ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7929800640 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e032a070 0x5575e06ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7923e00640 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e0309390 0x5575e06d6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7928e00640 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e032a070 0x5575e06ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7928e00640 1 -- 172.21.15.114:0/1264167571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e0309390 msgr2=0x5575e06d6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7928e00640 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e0309390 0x5575e06d6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7928e00640 1 -- 172.21.15.114:0/1264167571 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e06d3250 con 0x5575e03011b0 2024-09-21T07:06:34.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.079+0000 7f7929800640 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e032a070 0x5575e06ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:06:34.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.082+0000 7f7928e00640 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d5e40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f79180059f0 tx=0x7f7918005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66414619f5bf42a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:34.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.083+0000 7f7911600640 1 -- 172.21.15.114:0/1264167571 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791804d070 con 0x5575e03011b0 2024-09-21T07:06:34.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.083+0000 7f7911600640 1 -- 172.21.15.114:0/1264167571 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7918049a70 con 0x5575e03011b0 2024-09-21T07:06:34.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.083+0000 7f7911600640 1 -- 172.21.15.114:0/1264167571 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918035050 con 0x5575e03011b0 2024-09-21T07:06:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 -- 172.21.15.114:0/1264167571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 msgr2=0x5575e06d5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:34.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d5e40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f79180059f0 tx=0x7f7918005a20 comp rx=0 tx=0).stop 2024-09-21T07:06:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 -- 172.21.15.114:0/1264167571 shutdown_connections 2024-09-21T07:06:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e032a070 0x5575e06ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e0309390 0x5575e06d6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 --2- 172.21.15.114:0/1264167571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e03011b0 0x5575e06d5e40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 -- 172.21.15.114:0/1264167571 >> 172.21.15.114:0/1264167571 conn(0x5575e031e460 msgr2=0x5575e06db080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:34.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.086+0000 7f79323c8740 1 -- 172.21.15.114:0/1264167571 shutdown_connections 2024-09-21T07:06:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.089+0000 7f79323c8740 1 -- 172.21.15.114:0/1264167571 wait complete. 2024-09-21T07:06:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.089+0000 7f79323c8740 1 -- start start 2024-09-21T07:06:34.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.089+0000 7f79323c8740 1 Processor -- start 2024-09-21T07:06:34.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.093+0000 7f79323c8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e03011b0 0x5575e06e4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.094+0000 7f79323c8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 0x5575e06e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.094+0000 7f79323c8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.094+0000 7f79323c8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79080114a0 con 0x5575e032a070 2024-09-21T07:06:34.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.094+0000 7f7928e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e03011b0 0x5575e06e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.094+0000 7f79323c8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7908010dd0 con 0x5575e0309390 2024-09-21T07:06:34.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f79323c8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f790800e030 con 0x5575e03011b0 2024-09-21T07:06:34.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f79323c8740 10 client.0 _ll_register_callbacks cb 0x7f7918031570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 0x5575e06e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 0x5575e06e4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34592/0 (socket says 172.21.15.114:34592) 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 -- 172.21.15.114:0/2754699477 learned_addr learned my addr 172.21.15.114:0/2754699477 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7928e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e03011b0 0x5575e06e4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59822/0 (socket says 172.21.15.114:59822) 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 -- 172.21.15.114:0/2754699477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e03011b0 msgr2=0x5575e06e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e03011b0 0x5575e06e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 -- 172.21.15.114:0/2754699477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 msgr2=0x5575e06df6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:34.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.095+0000 7f7923e00640 1 -- 172.21.15.114:0/2754699477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e06dfdd0 con 0x5575e0309390 2024-09-21T07:06:34.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.097+0000 7f7923e00640 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 0x5575e06e4ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7914006fd0 tx=0x7f7914017810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3789936974eec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:34.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.097+0000 7f7906a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:34.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.099+0000 7f7906a00640 1 -- 172.21.15.114:0/2754699477 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79140170d0 con 0x5575e0309390 2024-09-21T07:06:34.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.099+0000 7f7906a00640 1 -- 172.21.15.114:0/2754699477 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f7914015420 con 0x5575e0309390 2024-09-21T07:06:34.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.100+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7918049a70 con 0x5575e0309390 2024-09-21T07:06:34.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.101+0000 7f7906a00640 1 -- 172.21.15.114:0/2754699477 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f791402a050 con 0x5575e0309390 2024-09-21T07:06:34.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.101+0000 7f79323c8740 10 client.28538 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:34.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.101+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5575e07624d0 con 0x5575e0309390 2024-09-21T07:06:34.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f7906a00640 1 -- 172.21.15.114:0/2754699477 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7918049a70 con 0x5575e0309390 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f7906a00640 1 -- 172.21.15.114:0/2754699477 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f79140254e0 con 0x5575e0309390 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f79323c8740 10 client.28538 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f79323c8740 10 client.28538 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f79323c8740 20 client.28538 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f79323c8740 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02bdda0 0x5575e0764cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f79323c8740 4 client.28538 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.104+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575e06459b0 con 0x5575e02bdda0 2024-09-21T07:06:34.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.105+0000 7f7928e00640 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02bdda0 0x5575e0764cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.106+0000 7f7906a00640 10 client.28538 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:06:34.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.106+0000 7f7928e00640 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02bdda0 0x5575e0764cc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a32decdfab81da9 server_cookie=f73a549e502d610c in_seq=0 out_seq=0 2024-09-21T07:06:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.106+0000 7f7906a00640 1 -- 172.21.15.114:0/2754699477 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575e06459b0 con 0x5575e02bdda0 2024-09-21T07:06:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.106+0000 7f7906a00640 10 client.28538 handle_command_reply: tid=0 2024-09-21T07:06:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.106+0000 7f79323c8740 1 client.28538 shutdown 2024-09-21T07:06:34.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.107+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 msgr2=0x5575e06e4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.107+0000 7f79323c8740 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 0x5575e06e4ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7914006fd0 tx=0x7f7914017810 comp rx=0 tx=0).stop 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.107+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 shutdown_connections 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.107+0000 7f79323c8740 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575e032a070 0x5575e06df6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.107+0000 7f79323c8740 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x5575e02bdda0 0x5575e0764cc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.108+0000 7f79323c8740 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575e0309390 0x5575e06e4ef0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.108+0000 7f79323c8740 1 --2- 172.21.15.114:0/2754699477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575e03011b0 0x5575e06e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.108+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 >> 172.21.15.114:0/2754699477 conn(0x5575e031e460 msgr2=0x5575e06d5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.108+0000 7f7906a00640 0 client.28538 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:06:34.114 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.111+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 shutdown_connections 2024-09-21T07:06:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.111+0000 7f79323c8740 1 -- 172.21.15.114:0/2754699477 wait complete. 2024-09-21T07:06:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.111+0000 7f79323c8740 20 client.28538 trim_cache size 0 max 16384 2024-09-21T07:06:34.115 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:06:34.115 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.112+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f790c0b6980 msgr2=0x7f790c0b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.112+0000 7f792a200640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f790c0b6980 0x7f790c0b8e90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7918000c00 tx=0x7f7918031400 comp rx=0 tx=0).stop 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.112+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792410fff0 msgr2=0x7f79241a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.112+0000 7f792a200640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792410fff0 0x7f79241a1e70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f791c00be20 tx=0x7f791c00bf10 comp rx=0 tx=0).stop 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 shutdown_connections 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f792410fff0 0x7f79241a1e70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f790c0b6980 0x7f790c0b8e90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7924102d90 0x7f792419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 --2- 172.21.15.114:0/3517678482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7924102440 0x7f792419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 >> 172.21.15.114:0/3517678482 conn(0x7f79240fe2c0 msgr2=0x7f7924101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.113+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 shutdown_connections 2024-09-21T07:06:34.116 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:34.114+0000 7f792a200640 1 -- 172.21.15.114:0/3517678482 wait complete. 2024-09-21T07:06:34.965 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:06:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:35 smithi193 ceph-mon[34601]: pgmap v779: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.9 MiB/s wr, 515 op/s 2024-09-21T07:06:35.707 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:35.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:35 smithi114 ceph-mon[30654]: pgmap v779: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.9 MiB/s wr, 515 op/s 2024-09-21T07:06:35.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:35 smithi183 ceph-mon[34219]: pgmap v779: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.9 MiB/s wr, 515 op/s 2024-09-21T07:06:36.841 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:06:36.841 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-09-21T07:06:36.842 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:06:37.617 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 -- 172.21.15.114:0/840908117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95081094b0 msgr2=0x7f95081098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 --2- 172.21.15.114:0/840908117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95081094b0 0x7f95081098c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009860 tx=0x7f94fc02a380 comp rx=0 tx=0).stop 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 -- 172.21.15.114:0/840908117 shutdown_connections 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 --2- 172.21.15.114:0/840908117 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9508109e00 0x7f9508110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 --2- 172.21.15.114:0/840908117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95081094b0 0x7f95081098c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 --2- 172.21.15.114:0/840908117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.613+0000 7f9510800640 1 -- 172.21.15.114:0/840908117 >> 172.21.15.114:0/840908117 conn(0x7f95080ff5e0 msgr2=0x7f9508101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.614+0000 7f9510800640 1 -- 172.21.15.114:0/840908117 shutdown_connections 2024-09-21T07:06:37.618 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.614+0000 7f9510800640 1 -- 172.21.15.114:0/840908117 wait complete. 2024-09-21T07:06:37.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.616+0000 7f9510800640 1 Processor -- start 2024-09-21T07:06:37.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.616+0000 7f9510800640 1 -- start start 2024-09-21T07:06:37.619 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f9510800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.620 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f9510800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081094b0 0x7f9508199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.621 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f9510800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9508109e00 0x7f95081999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f9510800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9508113ac0 con 0x7f95081094b0 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f9510800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9508113940 con 0x7f9508109e00 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f9510800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9508113c40 con 0x7f9508103b70 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f950de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f950de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52704/0 (socket says 172.21.15.114:52704) 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.617+0000 7f950de00640 1 -- 172.21.15.114:0/319055441 learned_addr learned my addr 172.21.15.114:0/319055441 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950e800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9508109e00 0x7f95081999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950de00640 1 -- 172.21.15.114:0/319055441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9508109e00 msgr2=0x7f95081999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950de00640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9508109e00 0x7f95081999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950d400640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081094b0 0x7f9508199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950de00640 1 -- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081094b0 msgr2=0x7f9508199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950de00640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081094b0 0x7f9508199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.618+0000 7f950de00640 1 -- 172.21.15.114:0/319055441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081a01d0 con 0x7f9508103b70 2024-09-21T07:06:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.619+0000 7f950de00640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508198f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f950400ed80 tx=0x7f950400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.619+0000 7f9502a00640 1 -- 172.21.15.114:0/319055441 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950401c070 con 0x7f9508103b70 2024-09-21T07:06:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.619+0000 7f9502a00640 1 -- 172.21.15.114:0/319055441 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9504017af0 con 0x7f9508103b70 2024-09-21T07:06:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.619+0000 7f9502a00640 1 -- 172.21.15.114:0/319055441 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950400fb70 con 0x7f9508103b70 2024-09-21T07:06:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.620+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95081a04c0 con 0x7f9508103b70 2024-09-21T07:06:37.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.620+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95081a09b0 con 0x7f9508103b70 2024-09-21T07:06:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.633+0000 7f9517428740 1 Processor -- start 2024-09-21T07:06:37.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.634+0000 7f9502a00640 1 -- 172.21.15.114:0/319055441 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f950401f070 con 0x7f9508103b70 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 -- start start 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e26eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e26f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e26f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556d7e1e5fb0 con 0x556d7de9f1b0 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d7e1d53e0 con 0x556d7dea7390 2024-09-21T07:06:37.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9517428740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d7e1d91f0 con 0x556d7dec8070 2024-09-21T07:06:37.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.636+0000 7f9502a00640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f00b6780 0x7f94f00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.637+0000 7f9502a00640 1 -- 172.21.15.114:0/319055441 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f95040e79b0 con 0x7f9508103b70 2024-09-21T07:06:37.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.637+0000 7f950e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e26eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.637+0000 7f950e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e26eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52710/0 (socket says 172.21.15.114:52710) 2024-09-21T07:06:37.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.638+0000 7f950e800640 1 -- 172.21.15.114:0/1859620014 learned_addr learned my addr 172.21.15.114:0/1859620014 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:37.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.638+0000 7f950d400640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e26f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.638+0000 7f950de00640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e26f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.638+0000 7f950d400640 1 -- 172.21.15.114:0/1859620014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 msgr2=0x556d7e26eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.639+0000 7f950d400640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e26eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.639+0000 7f950d400640 1 -- 172.21.15.114:0/1859620014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 msgr2=0x556d7e26f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.639+0000 7f950d400640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e26f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.639+0000 7f950d400640 1 -- 172.21.15.114:0/1859620014 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7e19fcd0 con 0x556d7dea7390 2024-09-21T07:06:37.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.639+0000 7f950e800640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e26eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.640+0000 7f950d400640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e26f820 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009830 tx=0x7f94fc00b0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca6f865415fd306b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:37.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.641+0000 7f94f7000640 1 -- 172.21.15.114:0/1859620014 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc040030 con 0x556d7dea7390 2024-09-21T07:06:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.641+0000 7f94f7000640 1 -- 172.21.15.114:0/1859620014 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc038040 con 0x556d7dea7390 2024-09-21T07:06:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.641+0000 7f94f7000640 1 -- 172.21.15.114:0/1859620014 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037070 con 0x556d7dea7390 2024-09-21T07:06:37.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.641+0000 7f950e800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f00b6780 0x7f94f00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.642+0000 7f9517428740 1 -- 172.21.15.114:0/1859620014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 msgr2=0x556d7e26f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.642+0000 7f9517428740 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e26f820 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009830 tx=0x7f94fc00b0b0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.642+0000 7f950de00640 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e26f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:37.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.644+0000 7f9517428740 1 -- 172.21.15.114:0/1859620014 shutdown_connections 2024-09-21T07:06:37.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.644+0000 7f9517428740 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e26f820 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.644+0000 7f9517428740 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e26f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.644+0000 7f9517428740 1 --2- 172.21.15.114:0/1859620014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e26eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.644+0000 7f9517428740 1 -- 172.21.15.114:0/1859620014 >> 172.21.15.114:0/1859620014 conn(0x556d7debc460 msgr2=0x556d7e26cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:37.647 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.644+0000 7f950e800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f00b6780 0x7f94f00b8c90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005d80 tx=0x7f94f800c3d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:37.648 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.645+0000 7f9517428740 1 -- 172.21.15.114:0/1859620014 shutdown_connections 2024-09-21T07:06:37.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.647+0000 7f9517428740 1 -- 172.21.15.114:0/1859620014 wait complete. 2024-09-21T07:06:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.648+0000 7f9517428740 1 -- start start 2024-09-21T07:06:37.651 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.648+0000 7f9517428740 1 Processor -- start 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 0x556d7e27a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7dea7390 0x556d7e27aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e286870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556d7e036700 con 0x556d7dea7390 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d7e170580 con 0x556d7de9f1b0 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d7e27d520 con 0x556d7dec8070 2024-09-21T07:06:37.653 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.649+0000 7f9517428740 10 client.0 _ll_register_callbacks cb 0x556d7e180470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.650+0000 7f950de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 0x556d7e27a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.650+0000 7f950de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 0x556d7e27a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43796/0 (socket says 172.21.15.114:43796) 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.650+0000 7f950de00640 1 -- 172.21.15.114:0/3375814739 learned_addr learned my addr 172.21.15.114:0/3375814739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.650+0000 7f950de00640 1 -- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 msgr2=0x556d7e286870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.650+0000 7f950de00640 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e286870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.650+0000 7f950de00640 1 -- 172.21.15.114:0/3375814739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7dea7390 msgr2=0x556d7e27aa60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.651+0000 7f950de00640 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7dea7390 0x556d7e27aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.651+0000 7f950de00640 1 -- 172.21.15.114:0/3375814739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7df33650 con 0x556d7de9f1b0 2024-09-21T07:06:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.652+0000 7f950de00640 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 0x556d7e27a520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f95040a99a0 tx=0x7f950400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91ff7733ee01ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.652+0000 7f94eb400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.652+0000 7f94eb400640 1 -- 172.21.15.114:0/3375814739 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040e5150 con 0x556d7de9f1b0 2024-09-21T07:06:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.652+0000 7f94eb400640 1 -- 172.21.15.114:0/3375814739 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f9504017640 con 0x556d7de9f1b0 2024-09-21T07:06:37.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.652+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d7e172b90 con 0x556d7de9f1b0 2024-09-21T07:06:37.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.653+0000 7f94eb400640 1 -- 172.21.15.114:0/3375814739 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f95040b6df0 con 0x556d7de9f1b0 2024-09-21T07:06:37.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.653+0000 7f9517428740 10 client.28550 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:37.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.653+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556d7e18d950 con 0x556d7de9f1b0 2024-09-21T07:06:37.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f94eb400640 1 -- 172.21.15.114:0/3375814739 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d7e172b90 con 0x556d7de9f1b0 2024-09-21T07:06:37.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f94eb400640 1 -- 172.21.15.114:0/3375814739 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f950400b030 con 0x556d7de9f1b0 2024-09-21T07:06:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f9517428740 10 client.28550 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f9517428740 10 client.28550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:06:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f9517428740 20 client.28550 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f9517428740 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de869f0 0x556d7e29d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f9517428740 4 client.28550 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:37.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d7e1e7f00 con 0x556d7de869f0 2024-09-21T07:06:37.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.654+0000 7f950d400640 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de869f0 0x556d7e29d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.656+0000 7f950d400640 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de869f0 0x556d7e29d750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4961cf01c334eccd server_cookie=c277b2d1771e3d06 in_seq=0 out_seq=0 2024-09-21T07:06:37.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.656+0000 7f94eb400640 10 client.28550 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:06:37.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.657+0000 7f94eb400640 1 -- 172.21.15.114:0/3375814739 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d7e1e7f00 con 0x556d7de869f0 2024-09-21T07:06:37.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.657+0000 7f94eb400640 10 client.28550 handle_command_reply: tid=0 2024-09-21T07:06:37.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.657+0000 7f9517428740 1 client.28550 shutdown 2024-09-21T07:06:37.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 msgr2=0x556d7e27a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 0x556d7e27a520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f95040a99a0 tx=0x7f950400ff90 comp rx=0 tx=0).stop 2024-09-21T07:06:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:37 smithi193 ceph-mon[34601]: pgmap v780: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.1 MiB/s wr, 481 op/s 2024-09-21T07:06:37.662 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:37 smithi114 ceph-mon[30654]: pgmap v780: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.1 MiB/s wr, 481 op/s 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 shutdown_connections 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de869f0 0x556d7e29d750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e286870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7dea7390 0x556d7e27aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 --2- 172.21.15.114:0/3375814739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7de9f1b0 0x556d7e27a520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 >> 172.21.15.114:0/3375814739 conn(0x556d7debc460 msgr2=0x556d7e271f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f94eb400640 0 client.28550 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.659+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 shutdown_connections 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.660+0000 7f9517428740 1 -- 172.21.15.114:0/3375814739 wait complete. 2024-09-21T07:06:37.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.660+0000 7f9517428740 20 client.28550 trim_cache size 0 max 16384 2024-09-21T07:06:37.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.682+0000 7f9517428740 1 Processor -- start 2024-09-21T07:06:37.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.682+0000 7f9517428740 1 -- start start 2024-09-21T07:06:37.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f9517428740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e2725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f9517428740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 0x556d7e286770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f9517428740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dec8070 0x556d7e277710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f9517428740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec010100 con 0x556d7de9f1b0 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f9517428740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec011ca0 con 0x556d7dec8070 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f9517428740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec0107d0 con 0x556d7dea7390 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f950d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 0x556d7e286770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f950d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 0x556d7e286770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52714/0 (socket says 172.21.15.114:52714) 2024-09-21T07:06:37.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f950d400640 1 -- 172.21.15.114:0/3195443703 learned_addr learned my addr 172.21.15.114:0/3195443703 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:37.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f950e800640 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dec8070 0x556d7e277710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.683+0000 7f950de00640 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e2725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.684+0000 7f950d400640 1 -- 172.21.15.114:0/3195443703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dec8070 msgr2=0x556d7e277710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:37.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.684+0000 7f950d400640 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dec8070 0x556d7e277710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.684+0000 7f950d400640 1 -- 172.21.15.114:0/3195443703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 msgr2=0x556d7e2725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.684+0000 7f950d400640 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e2725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.684+0000 7f950d400640 1 -- 172.21.15.114:0/3195443703 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d7e27b100 con 0x556d7dea7390 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.684+0000 7f950d400640 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 0x556d7e286770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0057c0 tx=0x7f94fc0058b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad90902725b9ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.685+0000 7f94f6600640 1 -- 172.21.15.114:0/3195443703 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc037040 con 0x556d7dea7390 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.685+0000 7f94f6600640 1 -- 172.21.15.114:0/3195443703 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc038040 con 0x556d7dea7390 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.685+0000 7f94f6600640 1 -- 172.21.15.114:0/3195443703 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc040040 con 0x556d7dea7390 2024-09-21T07:06:37.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 -- 172.21.15.114:0/3195443703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 msgr2=0x556d7e286770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 0x556d7e286770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f94fc0057c0 tx=0x7f94fc0058b0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 -- 172.21.15.114:0/3195443703 shutdown_connections 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dec8070 0x556d7e277710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dea7390 0x556d7e286770 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 --2- 172.21.15.114:0/3195443703 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e2725b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 -- 172.21.15.114:0/3195443703 >> 172.21.15.114:0/3195443703 conn(0x556d7debc460 msgr2=0x556d7e271ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 -- 172.21.15.114:0/3195443703 shutdown_connections 2024-09-21T07:06:37.689 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:37.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.686+0000 7f9517428740 1 -- 172.21.15.114:0/3195443703 wait complete. 2024-09-21T07:06:37.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 -- start start 2024-09-21T07:06:37.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 Processor -- start 2024-09-21T07:06:37.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e282e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e27e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e27e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec011570 con 0x556d7de9f1b0 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec010ea0 con 0x556d7dea7390 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec00e100 con 0x556d7dec8070 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.687+0000 7f9517428740 10 client.0 _ll_register_callbacks cb 0x7f95040acdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.688+0000 7f950de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e282e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.688+0000 7f950de00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e282e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59806/0 (socket says 172.21.15.114:59806) 2024-09-21T07:06:37.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.688+0000 7f950de00640 1 -- 172.21.15.114:0/1490656872 learned_addr learned my addr 172.21.15.114:0/1490656872 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.688+0000 7f950e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e27e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.688+0000 7f950d400640 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e27e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.689+0000 7f950e800640 1 -- 172.21.15.114:0/1490656872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 msgr2=0x556d7e27e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.689+0000 7f950e800640 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e27e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.689+0000 7f950e800640 1 -- 172.21.15.114:0/1490656872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 msgr2=0x556d7e282e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.689+0000 7f950e800640 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e282e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.689+0000 7f950e800640 1 -- 172.21.15.114:0/1490656872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d7e27ee40 con 0x556d7dec8070 2024-09-21T07:06:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.689+0000 7f950e800640 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e27e830 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f94f8012010 tx=0x7f94f8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1c4b9542db6e587 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.690+0000 7f94eb400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.690+0000 7f94eb400640 1 -- 172.21.15.114:0/1490656872 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f801d850 con 0x556d7dec8070 2024-09-21T07:06:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.690+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94fc038040 con 0x556d7dec8070 2024-09-21T07:06:37.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.690+0000 7f94eb400640 1 -- 172.21.15.114:0/1490656872 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f94f801d9f0 con 0x556d7dec8070 2024-09-21T07:06:37.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.693+0000 7f94eb400640 1 -- 172.21.15.114:0/1490656872 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f94f801dbf0 con 0x556d7dec8070 2024-09-21T07:06:37.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.694+0000 7f9517428740 10 client.28513 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:37.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.694+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556d7e307fd0 con 0x556d7dec8070 2024-09-21T07:06:37.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.694+0000 7f94eb400640 1 -- 172.21.15.114:0/1490656872 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc038040 con 0x556d7dec8070 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f94eb400640 1 -- 172.21.15.114:0/1490656872 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f94f8019050 con 0x556d7dec8070 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f9517428740 10 client.28513 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f9517428740 10 client.28513 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f9517428740 20 client.28513 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f9517428740 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de5bda0 0x556d7e30a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f9517428740 4 client.28513 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:06:37.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.699+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d7e1e39b0 con 0x556d7de5bda0 2024-09-21T07:06:37.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.700+0000 7f950de00640 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de5bda0 0x556d7e30a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:37.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.705+0000 7f94eb400640 10 client.28513 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:06:37.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.705+0000 7f950de00640 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de5bda0 0x556d7e30a6d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f86b7a9ecee6d50 server_cookie=dde509a9d82a3d3d in_seq=0 out_seq=0 2024-09-21T07:06:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.706+0000 7f94eb400640 1 -- 172.21.15.114:0/1490656872 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d7e1e39b0 con 0x556d7de5bda0 2024-09-21T07:06:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.706+0000 7f94eb400640 10 client.28513 handle_command_reply: tid=0 2024-09-21T07:06:37.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.706+0000 7f9517428740 1 client.28513 shutdown 2024-09-21T07:06:37.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.710+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 msgr2=0x556d7e27e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.710+0000 7f9517428740 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e27e830 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f94f8012010 tx=0x7f94f8004730 comp rx=0 tx=0).stop 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 shutdown_connections 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x556d7de5bda0 0x556d7e30a6d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f94eb400640 0 client.28513 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d7dec8070 0x556d7e27e830 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d7dea7390 0x556d7e27e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 --2- 172.21.15.114:0/1490656872 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556d7de9f1b0 0x556d7e282e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.714 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 >> 172.21.15.114:0/1490656872 conn(0x556d7debc460 msgr2=0x556d7e26fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.711+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 shutdown_connections 2024-09-21T07:06:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.712+0000 7f9517428740 1 -- 172.21.15.114:0/1490656872 wait complete. 2024-09-21T07:06:37.715 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.712+0000 7f9517428740 20 client.28513 trim_cache size 0 max 16384 2024-09-21T07:06:37.715 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:06:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.715+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f00b6780 msgr2=0x7f94f00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.715+0000 7f9510800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f00b6780 0x7f94f00b8c90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005d80 tx=0x7f94f800c3d0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.715+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 msgr2=0x7f9508198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:37.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.715+0000 7f9510800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508198f30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f950400ed80 tx=0x7f950400c6e0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.718+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 shutdown_connections 2024-09-21T07:06:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.718+0000 7f9510800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f94f00b6780 0x7f94f00b8c90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.718+0000 7f9510800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9508109e00 0x7f95081999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.718+0000 7f9510800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f95081094b0 0x7f9508199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.720 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.718+0000 7f9510800640 1 --2- 172.21.15.114:0/319055441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9508103b70 0x7f9508198f30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:37.721 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.718+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 >> 172.21.15.114:0/319055441 conn(0x7f95080ff5e0 msgr2=0x7f9508106910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:37.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.721+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 shutdown_connections 2024-09-21T07:06:37.727 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:37.722+0000 7f9510800640 1 -- 172.21.15.114:0/319055441 wait complete. 2024-09-21T07:06:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 720.0 s, 391.2 tps, lat 237.903 ms stddev 281.929 2024-09-21T07:06:37.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:37 smithi183 ceph-mon[34219]: pgmap v780: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 5.1 MiB/s wr, 481 op/s 2024-09-21T07:06:39.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:39 smithi114 ceph-mon[30654]: pgmap v781: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 5.5 MiB/s wr, 501 op/s 2024-09-21T07:06:39.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:39 smithi183 ceph-mon[34219]: pgmap v781: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 5.5 MiB/s wr, 501 op/s 2024-09-21T07:06:39.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:39 smithi193 ceph-mon[34601]: pgmap v781: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 5.5 MiB/s wr, 501 op/s 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 -- 172.21.15.114:0/1458843404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a41039d0 msgr2=0x7fb7a4103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/1458843404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a41039d0 0x7fb7a4103de0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb794009900 tx=0x7fb79402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 -- 172.21.15.114:0/1458843404 shutdown_connections 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/1458843404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7a4109c60 0x7fb7a4110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/1458843404 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a4109310 0x7fb7a4109720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/1458843404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a41039d0 0x7fb7a4103de0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 -- 172.21.15.114:0/1458843404 >> 172.21.15.114:0/1458843404 conn(0x7fb7a40ff460 msgr2=0x7fb7a41018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 -- 172.21.15.114:0/1458843404 shutdown_connections 2024-09-21T07:06:40.732 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.728+0000 7fb7a3e00640 1 -- 172.21.15.114:0/1458843404 wait complete. 2024-09-21T07:06:40.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.731+0000 7fb7a3e00640 1 Processor -- start 2024-09-21T07:06:40.733 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.731+0000 7fb7a3e00640 1 -- start start 2024-09-21T07:06:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.732+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 0x7fb7a419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:40.734 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.732+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7a4109310 0x7fb7a419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.732+0000 7fb7a3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a4109c60 0x7fb7a41a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.732+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a4113970 con 0x7fb7a41039d0 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.732+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a41137f0 con 0x7fb7a4109310 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.732+0000 7fb7a3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a4113af0 con 0x7fb7a4109c60 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 0x7fb7a419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 0x7fb7a419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:59820/0 (socket says 172.21.15.114:59820) 2024-09-21T07:06:40.735 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 -- 172.21.15.114:0/2243482741 learned_addr learned my addr 172.21.15.114:0/2243482741 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2000640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7a4109310 0x7fb7a419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 -- 172.21.15.114:0/2243482741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a4109c60 msgr2=0x7fb7a41a1d80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a4109c60 0x7fb7a41a1d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 -- 172.21.15.114:0/2243482741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7a4109310 msgr2=0x7fb7a419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7a4109310 0x7fb7a419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.733+0000 7fb7a2a00640 1 -- 172.21.15.114:0/2243482741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a41a24b0 con 0x7fb7a41039d0 2024-09-21T07:06:40.736 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.734+0000 7fb7a2a00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 0x7fb7a419c540 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb794005ab0 tx=0x7fb794003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:40.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.735+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79403f070 con 0x7fb7a41039d0 2024-09-21T07:06:40.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.735+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a41a2740 con 0x7fb7a41039d0 2024-09-21T07:06:40.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.735+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb794031d60 con 0x7fb7a41039d0 2024-09-21T07:06:40.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.735+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794035030 con 0x7fb7a41039d0 2024-09-21T07:06:40.737 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.735+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a41a8f00 con 0x7fb7a41039d0 2024-09-21T07:06:40.739 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.738+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a4103de0 con 0x7fb7a41039d0 2024-09-21T07:06:40.742 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.740+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb794042070 con 0x7fb7a41039d0 2024-09-21T07:06:40.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.742+0000 7fb79b400640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7880b6880 0x7fb7880b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:40.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.742+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fb79403bbc0 con 0x7fb7a41039d0 2024-09-21T07:06:40.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.743+0000 7fb7a2000640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7880b6880 0x7fb7880b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:40.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.744+0000 7fb7a2000640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7880b6880 0x7fb7880b8d90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4105970 tx=0x7fb790005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:40.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:40.749+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb79403be20 con 0x7fb7a41039d0 2024-09-21T07:06:40.908 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:06:41.201 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.197+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7a41a9130 con 0x7fb7a41039d0 2024-09-21T07:06:41.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.202+0000 7fb79b400640 1 -- 172.21.15.114:0/2243482741 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11993 (secure 0 0 0) 0x7fb7940ce970 con 0x7fb7a41039d0 2024-09-21T07:06:41.209 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:06:41.209 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":52,"btime":"2024-09-21T07:06:21:307707+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:06:20.622407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:06:41.209 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 52 2024-09-21T07:06:41.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.209+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7880b6880 msgr2=0x7fb7880b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:41.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.209+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7880b6880 0x7fb7880b8d90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4105970 tx=0x7fb790005f90 comp rx=0 tx=0).stop 2024-09-21T07:06:41.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.209+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 msgr2=0x7fb7a419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:41.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.209+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 0x7fb7a419c540 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb794005ab0 tx=0x7fb794003c80 comp rx=0 tx=0).stop 2024-09-21T07:06:41.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 shutdown_connections 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7a4109c60 0x7fb7a41a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7880b6880 0x7fb7880b8d90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7a4109310 0x7fb7a419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 --2- 172.21.15.114:0/2243482741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7a41039d0 0x7fb7a419c540 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 >> 172.21.15.114:0/2243482741 conn(0x7fb7a40ff460 msgr2=0x7fb7a4108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 shutdown_connections 2024-09-21T07:06:41.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:41.210+0000 7fb7a3e00640 1 -- 172.21.15.114:0/2243482741 wait complete. 2024-09-21T07:06:41.582 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:41 smithi114 ceph-mon[30654]: pgmap v782: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 5.1 MiB/s wr, 455 op/s 2024-09-21T07:06:41.583 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2243482741' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:41 smithi183 ceph-mon[34219]: pgmap v782: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 5.1 MiB/s wr, 455 op/s 2024-09-21T07:06:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2243482741' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:41 smithi193 ceph-mon[34601]: pgmap v782: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 5.1 MiB/s wr, 455 op/s 2024-09-21T07:06:41.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2243482741' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:42.143 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-09-21T07:06:42.143 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-09-21T07:06:42.258 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:42.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 725.0 s, 461.4 tps, lat 227.531 ms stddev 342.171 2024-09-21T07:06:43.094 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:43.174 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.172+0000 7fa742a00640 1 -- 172.21.15.114:0/3075980489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 msgr2=0x7fa73c10aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.172+0000 7fa742a00640 1 --2- 172.21.15.114:0/3075980489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c10aa60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009900 tx=0x7fa72c02a410 comp rx=0 tx=0).stop 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 -- 172.21.15.114:0/3075980489 shutdown_connections 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 --2- 172.21.15.114:0/3075980489 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10d620 0x7fa73c117090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 --2- 172.21.15.114:0/3075980489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10ccd0 0x7fa73c10d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 --2- 172.21.15.114:0/3075980489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c10aa60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 -- 172.21.15.114:0/3075980489 >> 172.21.15.114:0/3075980489 conn(0x7fa73c070ec0 msgr2=0x7fa73c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 -- 172.21.15.114:0/3075980489 shutdown_connections 2024-09-21T07:06:43.175 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.173+0000 7fa742a00640 1 -- 172.21.15.114:0/3075980489 wait complete. 2024-09-21T07:06:43.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.175+0000 7fa742a00640 1 Processor -- start 2024-09-21T07:06:43.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.175+0000 7fa742a00640 1 -- start start 2024-09-21T07:06:43.177 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa742a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c1a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa742a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10ccd0 0x7fa73c1a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa742a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10d620 0x7fa73c1aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa742a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c11a130 con 0x7fa73c10ccd0 2024-09-21T07:06:43.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa742a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c119fb0 con 0x7fa73c10a650 2024-09-21T07:06:43.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa742a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c11a2b0 con 0x7fa73c10d620 2024-09-21T07:06:43.178 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.176+0000 7fa73b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10ccd0 0x7fa73c1a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c1a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c1a4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43838/0 (socket says 172.21.15.114:43838) 2024-09-21T07:06:43.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73be00640 1 -- 172.21.15.114:0/4129286640 learned_addr learned my addr 172.21.15.114:0/4129286640 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:43.179 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa740a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10d620 0x7fa73c1aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73b400640 1 -- 172.21.15.114:0/4129286640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10d620 msgr2=0x7fa73c1aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73b400640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10d620 0x7fa73c1aa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73b400640 1 -- 172.21.15.114:0/4129286640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 msgr2=0x7fa73c1a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73b400640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c1a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.177+0000 7fa73b400640 1 -- 172.21.15.114:0/4129286640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c1aada0 con 0x7fa73c10ccd0 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.178+0000 7fa740a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10d620 0x7fa73c1aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:06:43.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.178+0000 7fa73b400640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10ccd0 0x7fa73c1a5490 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa73c06c6e0 tx=0x7fa72800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:43.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.178+0000 7fa738c00640 1 -- 172.21.15.114:0/4129286640 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72800cc10 con 0x7fa73c10ccd0 2024-09-21T07:06:43.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.179+0000 7fa738c00640 1 -- 172.21.15.114:0/4129286640 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa72800cdb0 con 0x7fa73c10ccd0 2024-09-21T07:06:43.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.179+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73c1ab090 con 0x7fa73c10ccd0 2024-09-21T07:06:43.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.179+0000 7fa738c00640 1 -- 172.21.15.114:0/4129286640 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728010430 con 0x7fa73c10ccd0 2024-09-21T07:06:43.181 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.179+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73c1b18b0 con 0x7fa73c10ccd0 2024-09-21T07:06:43.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.194+0000 7fa738c00640 1 -- 172.21.15.114:0/4129286640 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa7280105d0 con 0x7fa73c10ccd0 2024-09-21T07:06:43.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.197+0000 7fa738c00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa7200b6760 0x7fa7200b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.198+0000 7fa738c00640 1 -- 172.21.15.114:0/4129286640 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fa7280e8d10 con 0x7fa73c10ccd0 2024-09-21T07:06:43.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.198+0000 7fa73be00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa7200b6760 0x7fa7200b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.713 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:43 smithi193 ceph-mon[34601]: pgmap v783: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 5.2 MiB/s wr, 466 op/s 2024-09-21T07:06:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:43 smithi114 ceph-mon[30654]: pgmap v783: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 5.2 MiB/s wr, 466 op/s 2024-09-21T07:06:43.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.830+0000 7fa73be00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa7200b6760 0x7fa7200b8c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa72c004730 tx=0x7fa72c0046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:43.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.832+0000 7fa7497e3740 1 Processor -- start 2024-09-21T07:06:43.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.832+0000 7fa7497e3740 1 -- start start 2024-09-21T07:06:43.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067df620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa7497e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555c06755fb0 con 0x555c06438070 2024-09-21T07:06:43.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa7497e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c067453e0 con 0x555c0640f1b0 2024-09-21T07:06:43.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa7497e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555c067491f0 con 0x555c06417390 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa740a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067df620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa73b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa740a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067df620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:52784/0 (socket says 172.21.15.114:52784) 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa740a00640 1 -- 172.21.15.114:0/1933399538 learned_addr learned my addr 172.21.15.114:0/1933399538 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.833+0000 7fa73be00640 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.834+0000 7fa73b400640 1 -- 172.21.15.114:0/1933399538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 msgr2=0x555c067df620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.834+0000 7fa73b400640 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067df620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.834+0000 7fa73b400640 1 -- 172.21.15.114:0/1933399538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 msgr2=0x555c067deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.834+0000 7fa73b400640 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067deba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.834+0000 7fa73b400640 1 -- 172.21.15.114:0/1933399538 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0670fcd0 con 0x555c0640f1b0 2024-09-21T07:06:43.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.834+0000 7fa740a00640 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067df620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:43.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.836+0000 7fa73b400640 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067df0e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa7280e6090 tx=0x7fa7280aa760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7206349ea2ba0e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:43.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.837+0000 7fa731200640 1 -- 172.21.15.114:0/1933399538 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7280aee90 con 0x555c0640f1b0 2024-09-21T07:06:43.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.837+0000 7fa731200640 1 -- 172.21.15.114:0/1933399538 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7280b3270 con 0x555c0640f1b0 2024-09-21T07:06:43.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.837+0000 7fa731200640 1 -- 172.21.15.114:0/1933399538 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7280ad460 con 0x555c0640f1b0 2024-09-21T07:06:43.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.839+0000 7fa7497e3740 1 -- 172.21.15.114:0/1933399538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 msgr2=0x555c067df0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.839+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067df0e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa7280e6090 tx=0x7fa7280aa760 comp rx=0 tx=0).stop 2024-09-21T07:06:43.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.840+0000 7fa7497e3740 1 -- 172.21.15.114:0/1933399538 shutdown_connections 2024-09-21T07:06:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.840+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067df620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.840+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067df0e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.840+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1933399538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.840+0000 7fa7497e3740 1 -- 172.21.15.114:0/1933399538 >> 172.21.15.114:0/1933399538 conn(0x555c0642c460 msgr2=0x555c067dca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.840+0000 7fa7497e3740 1 -- 172.21.15.114:0/1933399538 shutdown_connections 2024-09-21T07:06:43.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.841+0000 7fa7497e3740 1 -- 172.21.15.114:0/1933399538 wait complete. 2024-09-21T07:06:43.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.841+0000 7fa7497e3740 1 -- start start 2024-09-21T07:06:43.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 Processor -- start 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c0640f1b0 0x555c067ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 0x555c067f6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x555c065a6700 con 0x555c0640f1b0 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c066e0580 con 0x555c06438070 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.842+0000 7fa7497e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555c067ed380 con 0x555c06417390 2024-09-21T07:06:43.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa7497e3740 10 client.0 _ll_register_callbacks cb 0x555c066f0470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 0x555c067f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 0x555c067f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43860/0 (socket says 172.21.15.114:43860) 2024-09-21T07:06:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 -- 172.21.15.114:0/1139368589 learned_addr learned my addr 172.21.15.114:0/1139368589 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa73b400640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa73be00640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c0640f1b0 0x555c067ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 -- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 msgr2=0x555c067eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067eada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 -- 172.21.15.114:0/1139368589 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c0640f1b0 msgr2=0x555c067ea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c0640f1b0 0x555c067ea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.843+0000 7fa740a00640 1 -- 172.21.15.114:0/1139368589 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c064a3650 con 0x555c06438070 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.844+0000 7fa740a00640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 0x555c067f6670 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa73400fa60 tx=0x7fa73400ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b07aa7a7c74b6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.844+0000 7fa725600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:43.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.844+0000 7fa725600640 1 -- 172.21.15.114:0/1139368589 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734004b20 con 0x555c06438070 2024-09-21T07:06:43.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.844+0000 7fa725600640 1 -- 172.21.15.114:0/1139368589 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fa734004cc0 con 0x555c06438070 2024-09-21T07:06:43.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.845+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555c066e2b90 con 0x555c06438070 2024-09-21T07:06:43.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.845+0000 7fa725600640 1 -- 172.21.15.114:0/1139368589 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa734023050 con 0x555c06438070 2024-09-21T07:06:43.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.845+0000 7fa7497e3740 10 client.28571 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:43.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.845+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555c066fd950 con 0x555c06438070 2024-09-21T07:06:43.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.845+0000 7fa725600640 1 -- 172.21.15.114:0/1139368589 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555c066e2b90 con 0x555c06438070 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.846+0000 7fa725600640 1 -- 172.21.15.114:0/1139368589 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fa734010550 con 0x555c06438070 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.846+0000 7fa7497e3740 10 client.28571 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.846+0000 7fa7497e3740 10 client.28571 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.846+0000 7fa7497e3740 20 client.28571 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.847+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063f69f0 0x555c067fc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.847+0000 7fa7497e3740 4 client.28571 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:43.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.847+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c06757f00 con 0x555c063f69f0 2024-09-21T07:06:43.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.853+0000 7fa73be00640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063f69f0 0x555c067fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.859+0000 7fa73be00640 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063f69f0 0x555c067fc780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7940f99d4cb1a03 server_cookie=157ab405ff97736b in_seq=0 out_seq=0 2024-09-21T07:06:43.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.859+0000 7fa725600640 10 client.28571 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:06:43.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.861+0000 7fa725600640 1 -- 172.21.15.114:0/1139368589 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c06757f00 con 0x555c063f69f0 2024-09-21T07:06:43.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.861+0000 7fa725600640 10 client.28571 handle_command_reply: tid=0 2024-09-21T07:06:43.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.862+0000 7fa7497e3740 1 client.28571 shutdown 2024-09-21T07:06:43.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:43 smithi183 ceph-mon[34219]: pgmap v783: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 634 KiB/s rd, 5.2 MiB/s wr, 466 op/s 2024-09-21T07:06:43.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 msgr2=0x555c067f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 0x555c067f6670 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa73400fa60 tx=0x7fa73400ff20 comp rx=0 tx=0).stop 2024-09-21T07:06:43.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 shutdown_connections 2024-09-21T07:06:43.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063f69f0 0x555c067fc780 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c06438070 0x555c067f6670 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067eada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1139368589 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c0640f1b0 0x555c067ea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa725600640 0 client.28571 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:06:43.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.866+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 >> 172.21.15.114:0/1139368589 conn(0x555c0642c460 msgr2=0x555c067e1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:43.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.867+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 shutdown_connections 2024-09-21T07:06:43.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.867+0000 7fa7497e3740 1 -- 172.21.15.114:0/1139368589 wait complete. 2024-09-21T07:06:43.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.867+0000 7fa7497e3740 20 client.28571 trim_cache size 0 max 16384 2024-09-21T07:06:43.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 Processor -- start 2024-09-21T07:06:43.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 -- start start 2024-09-21T07:06:43.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1bb960 con 0x555c06438070 2024-09-21T07:06:43.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1bd500 con 0x555c0640f1b0 2024-09-21T07:06:43.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.893+0000 7fa7497e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1bc030 con 0x555c06417390 2024-09-21T07:06:43.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.894+0000 7fa73be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.894+0000 7fa73be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067e3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43864/0 (socket says 172.21.15.114:43864) 2024-09-21T07:06:43.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.894+0000 7fa73be00640 1 -- 172.21.15.114:0/1052689583 learned_addr learned my addr 172.21.15.114:0/1052689583 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:43.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.894+0000 7fa73b400640 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.894+0000 7fa73be00640 1 -- 172.21.15.114:0/1052689583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 msgr2=0x555c067e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.895+0000 7fa73be00640 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.895+0000 7fa73be00640 1 -- 172.21.15.114:0/1052689583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 msgr2=0x555c067dc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.895+0000 7fa73be00640 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.895+0000 7fa73be00640 1 -- 172.21.15.114:0/1052689583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c067e9de0 con 0x555c0640f1b0 2024-09-21T07:06:43.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.895+0000 7fa73b400640 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:43.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.895+0000 7fa73be00640 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067e3cb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa72c031440 tx=0x7fa72c040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedef78be5239728 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:43.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.896+0000 7fa727e00640 1 -- 172.21.15.114:0/1052689583 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c035040 con 0x555c0640f1b0 2024-09-21T07:06:43.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.896+0000 7fa727e00640 1 -- 172.21.15.114:0/1052689583 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa72c003e40 con 0x555c0640f1b0 2024-09-21T07:06:43.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.896+0000 7fa727e00640 1 -- 172.21.15.114:0/1052689583 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c03c730 con 0x555c0640f1b0 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 -- 172.21.15.114:0/1052689583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 msgr2=0x555c067e3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067e3cb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa72c031440 tx=0x7fa72c040000 comp rx=0 tx=0).stop 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 -- 172.21.15.114:0/1052689583 shutdown_connections 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c067dc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1052689583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067e3cb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.899+0000 7fa7497e3740 1 -- 172.21.15.114:0/1052689583 >> 172.21.15.114:0/1052689583 conn(0x555c0642c460 msgr2=0x555c067e8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:43.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.900+0000 7fa7497e3740 1 -- 172.21.15.114:0/1052689583 shutdown_connections 2024-09-21T07:06:43.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.900+0000 7fa7497e3740 1 -- 172.21.15.114:0/1052689583 wait complete. 2024-09-21T07:06:43.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.901+0000 7fa7497e3740 1 -- start start 2024-09-21T07:06:43.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.901+0000 7fa7497e3740 1 Processor -- start 2024-09-21T07:06:43.907 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.905+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067ef060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.906+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067ef640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.906+0000 7fa7497e3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c06868a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.910+0000 7fa73be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067ef060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.910+0000 7fa73be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067ef060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:43876/0 (socket says 172.21.15.114:43876) 2024-09-21T07:06:43.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.911+0000 7fa7497e3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1bcdd0 con 0x555c06438070 2024-09-21T07:06:43.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.911+0000 7fa7497e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1bc700 con 0x555c0640f1b0 2024-09-21T07:06:43.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.911+0000 7fa7497e3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c1b9960 con 0x555c06417390 2024-09-21T07:06:43.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.911+0000 7fa7497e3740 10 client.0 _ll_register_callbacks cb 0x7fa73400f5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:43.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.911+0000 7fa73be00640 1 -- 172.21.15.114:0/1548940384 learned_addr learned my addr 172.21.15.114:0/1548940384 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:43.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.912+0000 7fa740a00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c06868a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.913+0000 7fa73b400640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.913+0000 7fa740a00640 1 -- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 msgr2=0x555c067ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.915+0000 7fa740a00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.915+0000 7fa740a00640 1 -- 172.21.15.114:0/1548940384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 msgr2=0x555c067ef060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:43.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.915+0000 7fa740a00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067ef060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.915+0000 7fa740a00640 1 -- 172.21.15.114:0/1548940384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c068690b0 con 0x555c06438070 2024-09-21T07:06:43.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.916+0000 7fa73be00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:43.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.917+0000 7fa73b400640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:06:43.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.920+0000 7fa725600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:43.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.920+0000 7fa740a00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c06868a40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa734010ae0 tx=0x7fa73400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fc3206ca80c29f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:43.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.922+0000 7fa725600640 1 -- 172.21.15.114:0/1548940384 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73402b070 con 0x555c06438070 2024-09-21T07:06:43.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.925+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72c003e40 con 0x555c06438070 2024-09-21T07:06:43.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.925+0000 7fa725600640 1 -- 172.21.15.114:0/1548940384 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fa734020040 con 0x555c06438070 2024-09-21T07:06:43.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.926+0000 7fa725600640 1 -- 172.21.15.114:0/1548940384 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fa73402b070 con 0x555c06438070 2024-09-21T07:06:43.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.927+0000 7fa7497e3740 10 client.18018 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:43.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.927+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555c068704d0 con 0x555c06438070 2024-09-21T07:06:43.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.929+0000 7fa725600640 1 -- 172.21.15.114:0/1548940384 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa72c003e40 con 0x555c06438070 2024-09-21T07:06:43.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.929+0000 7fa725600640 1 -- 172.21.15.114:0/1548940384 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fa734005040 con 0x555c06438070 2024-09-21T07:06:43.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.930+0000 7fa7497e3740 10 client.18018 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:43.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.930+0000 7fa7497e3740 10 client.18018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:06:43.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.930+0000 7fa7497e3740 20 client.18018 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:43.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.930+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063cbda0 0x555c06872cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:43.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.930+0000 7fa7497e3740 4 client.18018 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:06:43.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.930+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c067539b0 con 0x555c063cbda0 2024-09-21T07:06:43.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.932+0000 7fa73be00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063cbda0 0x555c06872cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.934+0000 7fa73be00640 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063cbda0 0x555c06872cc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13b4bd7a2f348921 server_cookie=d6d750ef48ab6098 in_seq=0 out_seq=0 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.935+0000 7fa725600640 10 client.18018 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.935+0000 7fa725600640 1 -- 172.21.15.114:0/1548940384 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c067539b0 con 0x555c063cbda0 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.935+0000 7fa725600640 10 client.18018 handle_command_reply: tid=0 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.935+0000 7fa7497e3740 1 client.18018 shutdown 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.936+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 msgr2=0x555c06868a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.938 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.936+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c06868a40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa734010ae0 tx=0x7fa73400df60 comp rx=0 tx=0).stop 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.937+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 shutdown_connections 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x555c063cbda0 0x555c06872cc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x555c06438070 0x555c06868a40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555c06417390 0x555c067ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa7497e3740 1 --2- 172.21.15.114:0/1548940384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c0640f1b0 0x555c067ef060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 >> 172.21.15.114:0/1548940384 conn(0x555c0642c460 msgr2=0x555c067e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa725600640 0 client.18018 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:06:43.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.938+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 shutdown_connections 2024-09-21T07:06:43.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.939+0000 7fa7497e3740 1 -- 172.21.15.114:0/1548940384 wait complete. 2024-09-21T07:06:43.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.939+0000 7fa7497e3740 20 client.18018 trim_cache size 0 max 16384 2024-09-21T07:06:43.942 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:06:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.940+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa7200b6760 msgr2=0x7fa7200b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.940+0000 7fa742a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa7200b6760 0x7fa7200b8c70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa72c004730 tx=0x7fa72c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.940+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10ccd0 msgr2=0x7fa73c1a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:43.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.940+0000 7fa742a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10ccd0 0x7fa73c1a5490 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa73c06c6e0 tx=0x7fa72800ee50 comp rx=0 tx=0).stop 2024-09-21T07:06:43.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.946+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 shutdown_connections 2024-09-21T07:06:43.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.946+0000 7fa742a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa73c10d620 0x7fa73c1aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.946+0000 7fa742a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa73c10ccd0 0x7fa73c1a5490 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.946+0000 7fa742a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa7200b6760 0x7fa7200b8c70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.946+0000 7fa742a00640 1 --2- 172.21.15.114:0/4129286640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa73c10a650 0x7fa73c1a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.946+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 >> 172.21.15.114:0/4129286640 conn(0x7fa73c070ec0 msgr2=0x7fa73c10ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.947+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 shutdown_connections 2024-09-21T07:06:43.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:43.947+0000 7fa742a00640 1 -- 172.21.15.114:0/4129286640 wait complete. 2024-09-21T07:06:45.568 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:45 smithi114 ceph-mon[30654]: pgmap v784: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 5.6 MiB/s wr, 512 op/s 2024-09-21T07:06:45.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:45 smithi183 ceph-mon[34219]: pgmap v784: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 5.6 MiB/s wr, 512 op/s 2024-09-21T07:06:45.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:45 smithi193 ceph-mon[34601]: pgmap v784: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 5.6 MiB/s wr, 512 op/s 2024-09-21T07:06:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.770+0000 7fcd05400640 1 -- 172.21.15.114:0/3789076890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd0010ad10 msgr2=0x7fcd00115fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.770+0000 7fcd05400640 1 --2- 172.21.15.114:0/3789076890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd0010ad10 0x7fcd00115fd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fccec009900 tx=0x7fccec02a410 comp rx=0 tx=0).stop 2024-09-21T07:06:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.774+0000 7fcd05400640 1 -- 172.21.15.114:0/3789076890 shutdown_connections 2024-09-21T07:06:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.774+0000 7fcd05400640 1 --2- 172.21.15.114:0/3789076890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd00116510 0x7fcd00118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.774+0000 7fcd05400640 1 --2- 172.21.15.114:0/3789076890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd0010ad10 0x7fcd00115fd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.774+0000 7fcd05400640 1 --2- 172.21.15.114:0/3789076890 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd0010a3c0 0x7fcd0010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.774+0000 7fcd05400640 1 -- 172.21.15.114:0/3789076890 >> 172.21.15.114:0/3789076890 conn(0x7fcd00070c00 msgr2=0x7fcd00071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.774+0000 7fcd05400640 1 -- 172.21.15.114:0/3789076890 shutdown_connections 2024-09-21T07:06:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.775+0000 7fcd05400640 1 -- 172.21.15.114:0/3789076890 wait complete. 2024-09-21T07:06:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.776+0000 7fcd05400640 1 Processor -- start 2024-09-21T07:06:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.777+0000 7fcd05400640 1 -- start start 2024-09-21T07:06:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.778+0000 7fcd05400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0010a3c0 0x7fcd001a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.778+0000 7fcd05400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd0010ad10 0x7fcd001a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.778+0000 7fcd05400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 0x7fcd001aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.778+0000 7fcd05400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0011c060 con 0x7fcd0010ad10 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.778+0000 7fcd05400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0011bee0 con 0x7fcd0010a3c0 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.778+0000 7fcd05400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0011c1e0 con 0x7fcd00116510 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 0x7fcd001aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 0x7fcd001aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35116/0 (socket says 172.21.15.114:35116) 2024-09-21T07:06:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 -- 172.21.15.114:0/4200745693 learned_addr learned my addr 172.21.15.114:0/4200745693 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fccff400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd0010ad10 0x7fcd001a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 -- 172.21.15.114:0/4200745693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0010a3c0 msgr2=0x7fcd001a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0010a3c0 0x7fcd001a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 -- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd0010ad10 msgr2=0x7fcd001a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd0010ad10 0x7fcd001a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.779+0000 7fcd04a00640 1 -- 172.21.15.114:0/4200745693 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd001aac50 con 0x7fcd00116510 2024-09-21T07:06:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.780+0000 7fcd04a00640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 0x7fcd001aa550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b970 tx=0x7fccf800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.780+0000 7fccfcc00640 1 -- 172.21.15.114:0/4200745693 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8019070 con 0x7fcd00116510 2024-09-21T07:06:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.780+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001aaf40 con 0x7fcd00116510 2024-09-21T07:06:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.780+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001b16d0 con 0x7fcd00116510 2024-09-21T07:06:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.781+0000 7fccfcc00640 1 -- 172.21.15.114:0/4200745693 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf8014440 con 0x7fcd00116510 2024-09-21T07:06:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.781+0000 7fccfcc00640 1 -- 172.21.15.114:0/4200745693 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf80075d0 con 0x7fcd00116510 2024-09-21T07:06:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.788+0000 7fcd0dc0c740 1 Processor -- start 2024-09-21T07:06:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.789+0000 7fccfcc00640 1 -- 172.21.15.114:0/4200745693 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fccf801c070 con 0x7fcd00116510 2024-09-21T07:06:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 -- start start 2024-09-21T07:06:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a29160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 0x564a29160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564a290d56b0 con 0x564a28d922f0 2024-09-21T07:06:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a290c4ae0 con 0x564a28dbb1b0 2024-09-21T07:06:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.790+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a290c88f0 con 0x564a28d9a4d0 2024-09-21T07:06:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.792+0000 7fccfcc00640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcce40b6680 0x7fcce40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.792+0000 7fccfcc00640 1 -- 172.21.15.114:0/4200745693 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fccf80e6ed0 con 0x7fcd00116510 2024-09-21T07:06:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 0x564a29160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 0x564a29160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35126/0 (socket says 172.21.15.114:35126) 2024-09-21T07:06:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 -- 172.21.15.114:0/3575952007 learned_addr learned my addr 172.21.15.114:0/3575952007 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 -- 172.21.15.114:0/3575952007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 msgr2=0x564a2915fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 -- 172.21.15.114:0/3575952007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 msgr2=0x564a29160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a29160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.793+0000 7fccff400640 1 -- 172.21.15.114:0/3575952007 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a29091780 con 0x564a28d9a4d0 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.794+0000 7fccff400640 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 0x564a29160890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fccec009900 tx=0x7fccec0036a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6bfc59c7590fd62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.796+0000 7fcd04a00640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcce40b6680 0x7fcce40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.797+0000 7fcd04a00640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcce40b6680 0x7fcce40b8b90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fccf80e4300 tx=0x7fccf8014e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.798+0000 7fccf5200640 1 -- 172.21.15.114:0/3575952007 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec042070 con 0x564a28d9a4d0 2024-09-21T07:06:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.798+0000 7fccf5200640 1 -- 172.21.15.114:0/3575952007 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec005000 con 0x564a28d9a4d0 2024-09-21T07:06:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.798+0000 7fccf5200640 1 -- 172.21.15.114:0/3575952007 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec0317c0 con 0x564a28d9a4d0 2024-09-21T07:06:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.799+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3575952007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 msgr2=0x564a29160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.799+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 0x564a29160890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fccec009900 tx=0x7fccec0036a0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.800+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3575952007 shutdown_connections 2024-09-21T07:06:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.800+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d9a4d0 0x564a29160890 secure :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fccec009900 tx=0x7fccec0036a0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.800+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a29160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.800+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3575952007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.800+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3575952007 >> 172.21.15.114:0/3575952007 conn(0x564a28daf300 msgr2=0x564a2915dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.800+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3575952007 shutdown_connections 2024-09-21T07:06:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.801+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3575952007 wait complete. 2024-09-21T07:06:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.802+0000 7fcd0dc0c740 1 -- start start 2024-09-21T07:06:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.803+0000 7fcd0dc0c740 1 Processor -- start 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a2916be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2916c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a29171430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564a28ffcab0 con 0x564a28d79d80 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564a2906b570 con 0x564a28dbb1b0 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564a29166490 con 0x564a28d922f0 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fcd0dc0c740 10 client.0 _ll_register_callbacks cb 0x564a2907a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fccff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a2916be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fccffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2916c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.804+0000 7fccff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a2916be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35140/0 (socket says 172.21.15.114:35140) 2024-09-21T07:06:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.805+0000 7fccff400640 1 -- 172.21.15.114:0/1867406214 learned_addr learned my addr 172.21.15.114:0/1867406214 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.805+0000 7fccff400640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a29171430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.807+0000 7fccff400640 1 -- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 msgr2=0x564a2916c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.807+0000 7fccff400640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2916c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.807+0000 7fccff400640 1 -- 172.21.15.114:0/1867406214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 msgr2=0x564a29171430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.807+0000 7fccff400640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a29171430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.809+0000 7fccff400640 1 -- 172.21.15.114:0/1867406214 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a28e26790 con 0x564a28d922f0 2024-09-21T07:06:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.809+0000 7fccffe00640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2916c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.809+0000 7fccff400640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a2916be40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fccec0314a0 tx=0x7fccec004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ef8e3c030a253f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.809+0000 7fcce9600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.810+0000 7fcce9600640 1 -- 172.21.15.114:0/1867406214 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec042040 con 0x564a28d922f0 2024-09-21T07:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.810+0000 7fcce9600640 1 -- 172.21.15.114:0/1867406214 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fccec005520 con 0x564a28d922f0 2024-09-21T07:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.810+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccec005000 con 0x564a28d922f0 2024-09-21T07:06:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.811+0000 7fcce9600640 1 -- 172.21.15.114:0/1867406214 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fccec05c050 con 0x564a28d922f0 2024-09-21T07:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.811+0000 7fcd0dc0c740 10 client.28540 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.811+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564a28e9f620 con 0x564a28d922f0 2024-09-21T07:06:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.812+0000 7fcce9600640 1 -- 172.21.15.114:0/1867406214 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec005000 con 0x564a28d922f0 2024-09-21T07:06:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.812+0000 7fcce9600640 1 -- 172.21.15.114:0/1867406214 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fccec047650 con 0x564a28d922f0 2024-09-21T07:06:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.813+0000 7fcd0dc0c740 10 client.28540 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.813+0000 7fcd0dc0c740 10 client.28540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.813+0000 7fcd0dc0c740 20 client.28540 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.813+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a28d4f1c0 0x564a2918e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.813+0000 7fcd0dc0c740 4 client.28540 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.813+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a290d7600 con 0x564a28d4f1c0 2024-09-21T07:06:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.814+0000 7fccffe00640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a28d4f1c0 0x564a2918e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.816+0000 7fccffe00640 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a28d4f1c0 0x564a2918e530 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8961d46cc860a0 server_cookie=b613f9bada93a903 in_seq=0 out_seq=0 2024-09-21T07:06:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.816+0000 7fcce9600640 10 client.28540 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.817+0000 7fcce9600640 1 -- 172.21.15.114:0/1867406214 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a290d7600 con 0x564a28d4f1c0 2024-09-21T07:06:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.817+0000 7fcce9600640 10 client.28540 handle_command_reply: tid=0 2024-09-21T07:06:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.817+0000 7fcd0dc0c740 1 client.28540 shutdown 2024-09-21T07:06:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 msgr2=0x564a2916be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a2916be40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fccec0314a0 tx=0x7fccec004ed0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 shutdown_connections 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a29171430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2916c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a28d4f1c0 0x564a2918e530 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/1867406214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a2916be40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 >> 172.21.15.114:0/1867406214 conn(0x564a28daf300 msgr2=0x564a29162fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcce9600640 0 client.28540 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 shutdown_connections 2024-09-21T07:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.818+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/1867406214 wait complete. 2024-09-21T07:06:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.819+0000 7fcd0dc0c740 20 client.28540 trim_cache size 0 max 16384 2024-09-21T07:06:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.848+0000 7fcd0dc0c740 1 Processor -- start 2024-09-21T07:06:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.848+0000 7fcd0dc0c740 1 -- start start 2024-09-21T07:06:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.849+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a291650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.849+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a29165620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.849+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.849+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcce000ffc0 con 0x564a28d79d80 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.849+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0011b60 con 0x564a28dbb1b0 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.849+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0010690 con 0x564a28d922f0 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a29165620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a29165620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35146/0 (socket says 172.21.15.114:35146) 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 -- 172.21.15.114:0/3193880543 learned_addr learned my addr 172.21.15.114:0/3193880543 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccff400640 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccff400640 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a291650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 -- 172.21.15.114:0/3193880543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 msgr2=0x564a2915c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 -- 172.21.15.114:0/3193880543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 msgr2=0x564a291650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a291650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.850+0000 7fccffe00640 1 -- 172.21.15.114:0/3193880543 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a29167780 con 0x564a28d922f0 2024-09-21T07:06:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.851+0000 7fccffe00640 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a29165620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fccf0003610 tx=0x7fccf000b290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5f72cdb75758b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.851+0000 7fccebe00640 1 -- 172.21.15.114:0/3193880543 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf000bcc0 con 0x564a28d922f0 2024-09-21T07:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.851+0000 7fccebe00640 1 -- 172.21.15.114:0/3193880543 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf001d030 con 0x564a28d922f0 2024-09-21T07:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.851+0000 7fccebe00640 1 -- 172.21.15.114:0/3193880543 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf0004d10 con 0x564a28d922f0 2024-09-21T07:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3193880543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 msgr2=0x564a29165620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a29165620 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fccf0003610 tx=0x7fccf000b290 comp rx=0 tx=0).stop 2024-09-21T07:06:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3193880543 shutdown_connections 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a2915c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d922f0 0x564a29165620 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/3193880543 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d79d80 0x564a291650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3193880543 >> 172.21.15.114:0/3193880543 conn(0x564a28daf300 msgr2=0x564a2916a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.852+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3193880543 shutdown_connections 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.853+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/3193880543 wait complete. 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.853+0000 7fcd0dc0c740 1 -- start start 2024-09-21T07:06:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.854+0000 7fcd0dc0c740 1 Processor -- start 2024-09-21T07:06:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.856+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 0x564a2918d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.857+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a2918dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.857+0000 7fcd0dc0c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a291f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.857+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0011430 con 0x564a28d922f0 2024-09-21T07:06:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.857+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0010d60 con 0x564a28dbb1b0 2024-09-21T07:06:46.859 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.857+0000 7fcd0dc0c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcce000dfc0 con 0x564a28d79d80 2024-09-21T07:06:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.857+0000 7fcd0dc0c740 10 client.0 _ll_register_callbacks cb 0x7fccec0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:46.860 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.859+0000 7fccff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 0x564a2918d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.860+0000 7fccff400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 0x564a2918d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35160/0 (socket says 172.21.15.114:35160) 2024-09-21T07:06:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.860+0000 7fccff400640 1 -- 172.21.15.114:0/832192855 learned_addr learned my addr 172.21.15.114:0/832192855 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:46.862 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.860+0000 7fccffe00640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a2918dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.861+0000 7fccffe00640 1 -- 172.21.15.114:0/832192855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 msgr2=0x564a2918d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.861+0000 7fccffe00640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 0x564a2918d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.861+0000 7fccffe00640 1 -- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 msgr2=0x564a291f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.861+0000 7fccffe00640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a291f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.861+0000 7fccffe00640 1 -- 172.21.15.114:0/832192855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a291f81e0 con 0x564a28d922f0 2024-09-21T07:06:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.861+0000 7fccff400640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 0x564a2918d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.862+0000 7fccffe00640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a2918dec0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fccf0004650 tx=0x7fccf000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a88c4d4484316cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.862+0000 7fcce9600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.863+0000 7fcce9600640 1 -- 172.21.15.114:0/832192855 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf000bc30 con 0x564a28d922f0 2024-09-21T07:06:46.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.863+0000 7fcce9600640 1 -- 172.21.15.114:0/832192855 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fccf000bdd0 con 0x564a28d922f0 2024-09-21T07:06:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.863+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccf001d030 con 0x564a28d922f0 2024-09-21T07:06:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.863+0000 7fcce9600640 1 -- 172.21.15.114:0/832192855 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fccf0037050 con 0x564a28d922f0 2024-09-21T07:06:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.863+0000 7fcd0dc0c740 10 client.18027 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.863+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564a291f9320 con 0x564a28d922f0 2024-09-21T07:06:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcce9600640 1 -- 172.21.15.114:0/832192855 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccf001d030 con 0x564a28d922f0 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcce9600640 1 -- 172.21.15.114:0/832192855 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fccf00224d0 con 0x564a28d922f0 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcd0dc0c740 10 client.18027 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcd0dc0c740 10 client.18027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcd0dc0c740 20 client.18027 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a291f9a40 0x564a291fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcd0dc0c740 4 client.18027 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.864+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a290d30b0 con 0x564a291f9a40 2024-09-21T07:06:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.865+0000 7fccff400640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a291f9a40 0x564a291fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.865+0000 7fcce9600640 10 client.18027 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.865+0000 7fccff400640 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a291f9a40 0x564a291fbe90 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee1ba26af849543 server_cookie=d8ae82cf6f656d3b in_seq=0 out_seq=0 2024-09-21T07:06:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.888+0000 7fcce9600640 1 -- 172.21.15.114:0/832192855 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a290d30b0 con 0x564a291f9a40 2024-09-21T07:06:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.888+0000 7fcce9600640 10 client.18027 handle_command_reply: tid=0 2024-09-21T07:06:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.888+0000 7fcd0dc0c740 1 client.18027 shutdown 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 msgr2=0x564a2918dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a2918dec0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fccf0004650 tx=0x7fccf000bac0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 shutdown_connections 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564a28dbb1b0 0x564a291f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564a28d922f0 0x564a2918dec0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564a291f9a40 0x564a291fbe90 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 --2- 172.21.15.114:0/832192855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564a28d79d80 0x564a2918d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 >> 172.21.15.114:0/832192855 conn(0x564a28daf300 msgr2=0x564a2915c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcce9600640 0 client.18027 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.889+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 shutdown_connections 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.890+0000 7fcd0dc0c740 1 -- 172.21.15.114:0/832192855 wait complete. 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.890+0000 7fcd0dc0c740 20 client.18027 trim_cache size 0 max 16384 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:06:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:06:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.891+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcce40b6680 msgr2=0x7fcce40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.891+0000 7fcd05400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcce40b6680 0x7fcce40b8b90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fccf80e4300 tx=0x7fccf8014e40 comp rx=0 tx=0).stop 2024-09-21T07:06:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.891+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 msgr2=0x7fcd001aa550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.891+0000 7fcd05400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 0x7fcd001aa550 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b970 tx=0x7fccf800be30 comp rx=0 tx=0).stop 2024-09-21T07:06:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.892+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 shutdown_connections 2024-09-21T07:06:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.892+0000 7fcd05400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd00116510 0x7fcd001aa550 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.892+0000 7fcd05400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcd0010ad10 0x7fcd001a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.892+0000 7fcd05400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcce40b6680 0x7fcce40b8b90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.892+0000 7fcd05400640 1 --2- 172.21.15.114:0/4200745693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0010a3c0 0x7fcd001a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.892+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 >> 172.21.15.114:0/4200745693 conn(0x7fcd00070c00 msgr2=0x7fcd0010c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.893+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 shutdown_connections 2024-09-21T07:06:46.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:06:46.893+0000 7fcd05400640 1 -- 172.21.15.114:0/4200745693 wait complete. 2024-09-21T07:06:46.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.988+0000 7f0ac3000640 1 -- 172.21.15.114:0/883010288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc1076f0 msgr2=0x7f0abc107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.988+0000 7f0ac3000640 1 --2- 172.21.15.114:0/883010288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc1076f0 0x7f0abc107b00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009900 tx=0x7f0aa802a410 comp rx=0 tx=0).stop 2024-09-21T07:06:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 -- 172.21.15.114:0/883010288 shutdown_connections 2024-09-21T07:06:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 --2- 172.21.15.114:0/883010288 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0abc108040 0x7f0abc10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 --2- 172.21.15.114:0/883010288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc1076f0 0x7f0abc107b00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 --2- 172.21.15.114:0/883010288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 -- 172.21.15.114:0/883010288 >> 172.21.15.114:0/883010288 conn(0x7f0abc0776b0 msgr2=0x7f0abc077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 -- 172.21.15.114:0/883010288 shutdown_connections 2024-09-21T07:06:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.990+0000 7f0ac3000640 1 -- 172.21.15.114:0/883010288 wait complete. 2024-09-21T07:06:46.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.991+0000 7f0ac3000640 1 Processor -- start 2024-09-21T07:06:46.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.991+0000 7f0ac3000640 1 -- start start 2024-09-21T07:06:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.992+0000 7f0ac3000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.992+0000 7f0ac3000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0abc1076f0 0x7f0abc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.992+0000 7f0ac3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc108040 0x7f0abc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.992+0000 7f0ac3000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc111900 con 0x7f0abc1076f0 2024-09-21T07:06:46.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.992+0000 7f0ac3000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc111780 con 0x7f0abc101e00 2024-09-21T07:06:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.992+0000 7f0ac3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc111a80 con 0x7f0abc108040 2024-09-21T07:06:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.993+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.993+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40652/0 (socket says 172.21.15.114:40652) 2024-09-21T07:06:46.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.993+0000 7f0abbe00640 1 -- 172.21.15.114:0/1309354853 learned_addr learned my addr 172.21.15.114:0/1309354853 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.993+0000 7f0ac1000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc108040 0x7f0abc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.994+0000 7f0abbe00640 1 -- 172.21.15.114:0/1309354853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc108040 msgr2=0x7f0abc19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.994+0000 7f0abbe00640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc108040 0x7f0abc19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.994+0000 7f0abbe00640 1 -- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0abc1076f0 msgr2=0x7f0abc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.994+0000 7f0abbe00640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0abc1076f0 0x7f0abc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:46.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.994+0000 7f0abbe00640 1 -- 172.21.15.114:0/1309354853 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc1a0540 con 0x7f0abc101e00 2024-09-21T07:06:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.994+0000 7f0abbe00640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc19a5d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00bec0 tx=0x7f0aac00dd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.995+0000 7f0ab8c00640 1 -- 172.21.15.114:0/1309354853 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac01c070 con 0x7f0abc101e00 2024-09-21T07:06:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.995+0000 7f0ab8c00640 1 -- 172.21.15.114:0/1309354853 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac01a070 con 0x7f0abc101e00 2024-09-21T07:06:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.995+0000 7f0ab8c00640 1 -- 172.21.15.114:0/1309354853 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003850 con 0x7f0abc101e00 2024-09-21T07:06:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.995+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc1a0830 con 0x7f0abc101e00 2024-09-21T07:06:46.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:46.995+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc0fe8f0 con 0x7f0abc101e00 2024-09-21T07:06:47.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.001+0000 7f0ab8c00640 1 -- 172.21.15.114:0/1309354853 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0aac020410 con 0x7f0abc101e00 2024-09-21T07:06:47.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.007+0000 7f0ab8c00640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0aa00b6760 0x7f0aa00b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.007+0000 7f0ab8c00640 1 -- 172.21.15.114:0/1309354853 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f0aac0ef9b0 con 0x7f0abc101e00 2024-09-21T07:06:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.007+0000 7f0abb400640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0aa00b6760 0x7f0aa00b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.008+0000 7f0abb400640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0aa00b6760 0x7f0aa00b8c70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009230 tx=0x7f0aa8000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:47.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.033+0000 7f0ac9ccd740 1 Processor -- start 2024-09-21T07:06:47.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.033+0000 7f0ac9ccd740 1 -- start start 2024-09-21T07:06:47.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.034+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.034+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d36890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.034+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.034+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c003cabca0 con 0x55c00396db60 2024-09-21T07:06:47.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.034+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c003c9b0d0 con 0x55c00398e840 2024-09-21T07:06:47.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.034+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c003c9eee0 con 0x55c003965980 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d36890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d36890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35180/0 (socket says 172.21.15.114:35180) 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 -- 172.21.15.114:0/1377897874 learned_addr learned my addr 172.21.15.114:0/1377897874 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 -- 172.21.15.114:0/1377897874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 msgr2=0x55c003d36350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d36350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 -- 172.21.15.114:0/1377897874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 msgr2=0x55c003d36dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:06:47.038 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d36dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.035+0000 7f0abb400640 1 -- 172.21.15.114:0/1377897874 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c003c4d400 con 0x55c003965980 2024-09-21T07:06:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.036+0000 7f0abb400640 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d36890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8033000 tx=0x7f0aa803a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1d4f0d6bed0e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.036+0000 7f0ab1600640 1 -- 172.21.15.114:0/1377897874 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8040070 con 0x55c003965980 2024-09-21T07:06:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.036+0000 7f0ab1600640 1 -- 172.21.15.114:0/1377897874 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa804c070 con 0x55c003965980 2024-09-21T07:06:47.039 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.036+0000 7f0ab1600640 1 -- 172.21.15.114:0/1377897874 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8035040 con 0x55c003965980 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1377897874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 msgr2=0x55c003d36890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d36890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8033000 tx=0x7f0aa803a000 comp rx=0 tx=0).stop 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1377897874 shutdown_connections 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d36dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d36890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1377897874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1377897874 >> 172.21.15.114:0/1377897874 conn(0x55c003982990 msgr2=0x55c003d34240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1377897874 shutdown_connections 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.037+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1377897874 wait complete. 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.038+0000 7f0ac9ccd740 1 -- start start 2024-09-21T07:06:47.040 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.038+0000 7f0ac9ccd740 1 Processor -- start 2024-09-21T07:06:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.039+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d41e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.039+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d42360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.039+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d6fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.041 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.039+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c003afc7a0 con 0x55c00396db60 2024-09-21T07:06:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c003c40a10 con 0x55c00398e840 2024-09-21T07:06:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c003d66970 con 0x55c003965980 2024-09-21T07:06:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0ac9ccd740 10 client.0 _ll_register_callbacks cb 0x55c003c50be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:47.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d42360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d41e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d41e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35196/0 (socket says 172.21.15.114:35196) 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 -- 172.21.15.114:0/1126163303 learned_addr learned my addr 172.21.15.114:0/1126163303 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0ac1000640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d6fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 -- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 msgr2=0x55c003d6fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d6fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 -- 172.21.15.114:0/1126163303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 msgr2=0x55c003d42360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d42360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abbe00640 1 -- 172.21.15.114:0/1126163303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c003a08b90 con 0x55c003965980 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.040+0000 7f0abb400640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d42360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0abbe00640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d41e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0aac000c00 tx=0x7f0aac0b51a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488fe0bd9999404e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0aa6000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0aa6000640 1 -- 172.21.15.114:0/1126163303 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0ed160 con 0x55c003965980 2024-09-21T07:06:47.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0aa6000640 1 -- 172.21.15.114:0/1126163303 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f0aac0b4830 con 0x55c003965980 2024-09-21T07:06:47.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aa804c070 con 0x55c003965980 2024-09-21T07:06:47.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0aa6000640 1 -- 172.21.15.114:0/1126163303 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0aac0d0050 con 0x55c003965980 2024-09-21T07:06:47.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0ac9ccd740 10 client.28558 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:47.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.041+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c003d729d0 con 0x55c003965980 2024-09-21T07:06:47.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.042+0000 7f0aa6000640 1 -- 172.21.15.114:0/1126163303 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa804c070 con 0x55c003965980 2024-09-21T07:06:47.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.043+0000 7f0aa6000640 1 -- 172.21.15.114:0/1126163303 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f0aac0b0d20 con 0x55c003965980 2024-09-21T07:06:47.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.043+0000 7f0ac9ccd740 10 client.28558 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:47.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.044+0000 7f0ac9ccd740 10 client.28558 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.044+0000 7f0ac9ccd740 20 client.28558 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.044+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c00394d1c0 0x55c003d75ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.044+0000 7f0ac9ccd740 4 client.28558 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.044+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c003cadbf0 con 0x55c00394d1c0 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.044+0000 7f0abb400640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c00394d1c0 0x55c003d75ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.045+0000 7f0abb400640 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c00394d1c0 0x55c003d75ea0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9be9b381aac636 server_cookie=574fc5970f3059f in_seq=0 out_seq=0 2024-09-21T07:06:47.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.045+0000 7f0aa6000640 10 client.28558 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:47.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.045+0000 7f0aa6000640 1 -- 172.21.15.114:0/1126163303 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c003cadbf0 con 0x55c00394d1c0 2024-09-21T07:06:47.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.045+0000 7f0aa6000640 10 client.28558 handle_command_reply: tid=0 2024-09-21T07:06:47.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.045+0000 7f0ac9ccd740 1 client.28558 shutdown 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 msgr2=0x55c003d41e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d41e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0aac000c00 tx=0x7f0aac0b51a0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 shutdown_connections 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d6fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d42360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c00394d1c0 0x55c003d75ea0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/1126163303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d41e20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0aa6000640 0 client.28558 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:47.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 >> 172.21.15.114:0/1126163303 conn(0x55c003982990 msgr2=0x55c003d37380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:47.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 shutdown_connections 2024-09-21T07:06:47.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.046+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/1126163303 wait complete. 2024-09-21T07:06:47.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.047+0000 7f0ac9ccd740 20 client.28558 trim_cache size 0 max 16384 2024-09-21T07:06:47.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.070+0000 7f0ac9ccd740 1 Processor -- start 2024-09-21T07:06:47.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.070+0000 7f0ac9ccd740 1 -- start start 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d43850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d6fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d70180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc074c80 con 0x55c00396db60 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a81c0 con 0x55c00398e840 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc075350 con 0x55c003965980 2024-09-21T07:06:47.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0ac1000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d70180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d43850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d43850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35204/0 (socket says 172.21.15.114:35204) 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.071+0000 7f0abbe00640 1 -- 172.21.15.114:0/4072331626 learned_addr learned my addr 172.21.15.114:0/4072331626 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abb400640 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d6fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abbe00640 1 -- 172.21.15.114:0/4072331626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 msgr2=0x55c003d70180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abbe00640 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d70180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abbe00640 1 -- 172.21.15.114:0/4072331626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 msgr2=0x55c003d6fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abbe00640 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d6fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abbe00640 1 -- 172.21.15.114:0/4072331626 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c003d426d0 con 0x55c003965980 2024-09-21T07:06:47.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.072+0000 7f0abb400640 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d6fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:06:47.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.073+0000 7f0abbe00640 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d43850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0b2640 tx=0x7f0aac0b11d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df37bb8d0b1914dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:47.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.073+0000 7f0ab0c00640 1 -- 172.21.15.114:0/4072331626 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0d4030 con 0x55c003965980 2024-09-21T07:06:47.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.073+0000 7f0ab0c00640 1 -- 172.21.15.114:0/4072331626 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac0b9eb0 con 0x55c003965980 2024-09-21T07:06:47.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.073+0000 7f0ab0c00640 1 -- 172.21.15.114:0/4072331626 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0b0e70 con 0x55c003965980 2024-09-21T07:06:47.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.074+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/4072331626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 msgr2=0x55c003d43850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:47.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.074+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d43850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0b2640 tx=0x7f0aac0b11d0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/4072331626 shutdown_connections 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00398e840 0x55c003d70180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00396db60 0x55c003d6fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/4072331626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d43850 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/4072331626 >> 172.21.15.114:0/4072331626 conn(0x55c003982990 msgr2=0x55c003d43140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/4072331626 shutdown_connections 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.078+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/4072331626 wait complete. 2024-09-21T07:06:47.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.079+0000 7f0ac9ccd740 1 -- start start 2024-09-21T07:06:47.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.079+0000 7f0ac9ccd740 1 Processor -- start 2024-09-21T07:06:47.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d67e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 0x55c003d68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00398e840 0x55c003d68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a7a90 con 0x55c00398e840 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1a73c0 con 0x55c00396db60 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc072c80 con 0x55c003965980 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0ac9ccd740 10 client.0 _ll_register_callbacks cb 0x7f0aac0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:47.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0abb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 0x55c003d68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0abb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 0x55c003d68340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40680/0 (socket says 172.21.15.114:40680) 2024-09-21T07:06:47.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0abb400640 1 -- 172.21.15.114:0/127810277 learned_addr learned my addr 172.21.15.114:0/127810277 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:47.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.080+0000 7f0abbe00640 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d67e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.083 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.081+0000 7f0abb400640 1 -- 172.21.15.114:0/127810277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 msgr2=0x55c003d67e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:47.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.081+0000 7f0abb400640 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d67e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.081+0000 7f0abb400640 1 -- 172.21.15.114:0/127810277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00398e840 msgr2=0x55c003d68880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:47.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.081+0000 7f0abb400640 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00398e840 0x55c003d68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.081+0000 7f0abb400640 1 -- 172.21.15.114:0/127810277 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c003ddf950 con 0x55c00396db60 2024-09-21T07:06:47.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.082+0000 7f0abb400640 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 0x55c003d68340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80095f0 tx=0x7f0aa8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56c41a4b6e5e484 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:47.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.082+0000 7f0aa6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.082+0000 7f0aa6000640 1 -- 172.21.15.114:0/127810277 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8035040 con 0x55c00396db60 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.082+0000 7f0aa6000640 1 -- 172.21.15.114:0/127810277 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f0aa80037f0 con 0x55c00396db60 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.082+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac0b9eb0 con 0x55c00396db60 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0aa6000640 1 -- 172.21.15.114:0/127810277 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0aa8061050 con 0x55c00396db60 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 10 client.28598 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c003de0b20 con 0x55c00396db60 2024-09-21T07:06:47.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0aa6000640 1 -- 172.21.15.114:0/127810277 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aac0b9eb0 con 0x55c00396db60 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0aa6000640 1 -- 172.21.15.114:0/127810277 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f0aa8031960 con 0x55c00396db60 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 10 client.28598 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 10 client.28598 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 20 client.28598 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c003922570 0x55c003de2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 4 client.28598 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.083+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x55c003ca96a0 con 0x55c003922570 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.084+0000 7f0abbe00640 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c003922570 0x55c003de2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.084+0000 7f0abbe00640 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c003922570 0x55c003de2f50 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87799f0065878e27 server_cookie=ddeec05853745738 in_seq=0 out_seq=0 2024-09-21T07:06:47.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.084+0000 7f0aa6000640 10 client.28598 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:47.143 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:06:47.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.479+0000 7f0aa6000640 1 -- 172.21.15.114:0/127810277 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c003ca96a0 con 0x55c003922570 2024-09-21T07:06:47.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.479+0000 7f0aa6000640 10 client.28598 handle_command_reply: tid=0 2024-09-21T07:06:47.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.479+0000 7f0ac9ccd740 1 client.28598 shutdown 2024-09-21T07:06:47.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.480+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 msgr2=0x55c003d68340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:47.482 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.480+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 0x55c003d68340 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80095f0 tx=0x7f0aa8009580 comp rx=0 tx=0).stop 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 shutdown_connections 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c00398e840 0x55c003d68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c00396db60 0x55c003d68340 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c003922570 0x55c003de2f50 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 --2- 172.21.15.114:0/127810277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c003965980 0x55c003d67e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 >> 172.21.15.114:0/127810277 conn(0x55c003982990 msgr2=0x55c003d3c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0aa6000640 0 client.28598 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:47.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 shutdown_connections 2024-09-21T07:06:47.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.481+0000 7f0ac9ccd740 1 -- 172.21.15.114:0/127810277 wait complete. 2024-09-21T07:06:47.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.482+0000 7f0ac9ccd740 20 client.28598 trim_cache size 0 max 16384 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 52, 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 46, 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "faeaa3be": { 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 46, 2024-09-21T07:06:47.485 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 137.8, 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:47.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.483+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0aa00b6760 msgr2=0x7f0aa00b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.483+0000 7f0ac3000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0aa00b6760 0x7f0aa00b8c70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009230 tx=0x7f0aa8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 msgr2=0x7f0abc19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:47.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc19a5d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00bec0 tx=0x7f0aac00dd20 comp rx=0 tx=0).stop 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 shutdown_connections 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0abc108040 0x7f0abc19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0abc1076f0 0x7f0abc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0aa00b6760 0x7f0aa00b8c70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 --2- 172.21.15.114:0/1309354853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0abc101e00 0x7f0abc19a5d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 >> 172.21.15.114:0/1309354853 conn(0x7f0abc0776b0 msgr2=0x7f0abc100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 shutdown_connections 2024-09-21T07:06:47.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:47.484+0000 7f0ac3000640 1 -- 172.21.15.114:0/1309354853 wait complete. 2024-09-21T07:06:47.525 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:47 smithi114 ceph-mon[30654]: pgmap v785: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.5 MiB/s wr, 460 op/s 2024-09-21T07:06:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 730.0 s, 404.2 tps, lat 221.859 ms stddev 332.513 2024-09-21T07:06:47.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:47 smithi183 ceph-mon[34219]: pgmap v785: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.5 MiB/s wr, 460 op/s 2024-09-21T07:06:47.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:47 smithi193 ceph-mon[34601]: pgmap v785: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 4.5 MiB/s wr, 460 op/s 2024-09-21T07:06:48.227 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'faeaa3be', quiesce took 11.4 seconds. Will release after: 6.4 2024-09-21T07:06:48.256 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:48.732 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:48 smithi114 ceph-mon[30654]: pgmap v786: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 5.2 MiB/s wr, 479 op/s 2024-09-21T07:06:48.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:48 smithi183 ceph-mon[34219]: pgmap v786: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 5.2 MiB/s wr, 479 op/s 2024-09-21T07:06:48.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:48 smithi193 ceph-mon[34601]: pgmap v786: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 5.2 MiB/s wr, 479 op/s 2024-09-21T07:06:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.986+0000 7f90be200640 1 -- 172.21.15.114:0/3163958656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 msgr2=0x7f90b8109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:48.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.986+0000 7f90be200640 1 --2- 172.21.15.114:0/3163958656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b8109700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009900 tx=0x7f90a402a410 comp rx=0 tx=0).stop 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.986+0000 7f90be200640 1 -- 172.21.15.114:0/3163958656 shutdown_connections 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.986+0000 7f90be200640 1 --2- 172.21.15.114:0/3163958656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b8109c40 0x7f90b8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.986+0000 7f90be200640 1 --2- 172.21.15.114:0/3163958656 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b8109700 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.987+0000 7f90be200640 1 --2- 172.21.15.114:0/3163958656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b81039b0 0x7f90b8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.987+0000 7f90be200640 1 -- 172.21.15.114:0/3163958656 >> 172.21.15.114:0/3163958656 conn(0x7f90b80ff440 msgr2=0x7f90b8101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.987+0000 7f90be200640 1 -- 172.21.15.114:0/3163958656 shutdown_connections 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.987+0000 7f90be200640 1 -- 172.21.15.114:0/3163958656 wait complete. 2024-09-21T07:06:48.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 Processor -- start 2024-09-21T07:06:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 -- start start 2024-09-21T07:06:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b81039b0 0x7f90b819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b8109c40 0x7f90b819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8113950 con 0x7f90b81092f0 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81137d0 con 0x7f90b8109c40 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.988+0000 7f90be200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8113ad0 con 0x7f90b81039b0 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90bce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b81039b0 0x7f90b819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b819a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55376/0 (socket says 172.21.15.114:55376) 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 -- 172.21.15.114:0/2887430215 learned_addr learned my addr 172.21.15.114:0/2887430215 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:48.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90bd800640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b8109c40 0x7f90b819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 -- 172.21.15.114:0/2887430215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b81039b0 msgr2=0x7f90b819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b81039b0 0x7f90b819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 -- 172.21.15.114:0/2887430215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b8109c40 msgr2=0x7f90b819fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b8109c40 0x7f90b819fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90b7e00640 1 -- 172.21.15.114:0/2887430215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b81a0250 con 0x7f90b81092f0 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.989+0000 7f90bd800640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b8109c40 0x7f90b819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.990+0000 7f90b7e00640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b819a880 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f90a4005ab0 tx=0x7f90a4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:48.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.990+0000 7f90b5600640 1 -- 172.21.15.114:0/2887430215 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a403f070 con 0x7f90b81092f0 2024-09-21T07:06:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.990+0000 7f90b5600640 1 -- 172.21.15.114:0/2887430215 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a4031b50 con 0x7f90b81092f0 2024-09-21T07:06:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.990+0000 7f90b5600640 1 -- 172.21.15.114:0/2887430215 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4035030 con 0x7f90b81092f0 2024-09-21T07:06:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.990+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b81a04e0 con 0x7f90b81092f0 2024-09-21T07:06:48.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.990+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b81a6ca0 con 0x7f90b81092f0 2024-09-21T07:06:48.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 Processor -- start 2024-09-21T07:06:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 -- start start 2024-09-21T07:06:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90b5600640 1 -- 172.21.15.114:0/2887430215 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f90a4042070 con 0x7f90b81092f0 2024-09-21T07:06:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed38f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e9691b0 0x55895ed394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed399f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55895ecaffb0 con 0x55895e992070 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55895ec9f3e0 con 0x55895e9691b0 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.994+0000 7f90c6490740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55895eca31f0 con 0x55895e971390 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed38f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e9691b0 0x55895ed394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed38f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55382/0 (socket says 172.21.15.114:55382) 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 -- 172.21.15.114:0/1244013555 learned_addr learned my addr 172.21.15.114:0/1244013555 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:48.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90b7e00640 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 -- 172.21.15.114:0/1244013555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 msgr2=0x55895ed399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 -- 172.21.15.114:0/1244013555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e9691b0 msgr2=0x55895ed394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e9691b0 0x55895ed394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bd800640 1 -- 172.21.15.114:0/1244013555 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895ec69cd0 con 0x55895e992070 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90bce00640 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e9691b0 0x55895ed394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:48.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.995+0000 7f90b7e00640 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90b5600640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90a00b6680 0x7f90a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90bd800640 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed38f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90b0010750 tx=0x7f90b0010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23fb92b422fdf2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90b5600640 1 -- 172.21.15.114:0/2887430215 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f90a4107e60 con 0x7f90b81092f0 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90bce00640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90a00b6680 0x7f90a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90aa000640 1 -- 172.21.15.114:0/1244013555 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b000ead0 con 0x55895e992070 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90aa000640 1 -- 172.21.15.114:0/1244013555 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b0010e60 con 0x55895e992070 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.996+0000 7f90aa000640 1 -- 172.21.15.114:0/1244013555 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0011640 con 0x55895e992070 2024-09-21T07:06:48.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 -- 172.21.15.114:0/1244013555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 msgr2=0x55895ed38f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed38f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90b0010750 tx=0x7f90b0010c10 comp rx=0 tx=0).stop 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90bce00640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90a00b6680 0x7f90a00b8b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55895ed37bf0 tx=0x7f90ac015040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 -- 172.21.15.114:0/1244013555 shutdown_connections 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed399f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e9691b0 0x55895ed394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 --2- 172.21.15.114:0/1244013555 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed38f70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.997+0000 7f90c6490740 1 -- 172.21.15.114:0/1244013555 >> 172.21.15.114:0/1244013555 conn(0x55895e986460 msgr2=0x55895ed36e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.998+0000 7f90c6490740 1 -- 172.21.15.114:0/1244013555 shutdown_connections 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.998+0000 7f90c6490740 1 -- 172.21.15.114:0/1244013555 wait complete. 2024-09-21T07:06:49.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.998+0000 7f90c6490740 1 -- start start 2024-09-21T07:06:49.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:48.999+0000 7f90c6490740 1 Processor -- start 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e9691b0 0x55895ed44830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed44e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e992070 0x55895ed50a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55895eb00700 con 0x55895e9691b0 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55895ec3a580 con 0x55895e992070 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55895ed47750 con 0x55895e971390 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90c6490740 10 client.0 _ll_register_callbacks cb 0x55895ec4a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90bce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e9691b0 0x55895ed44830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90b7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed44e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90b7e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed44e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35266/0 (socket says 172.21.15.114:35266) 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90b7e00640 1 -- 172.21.15.114:0/1517592026 learned_addr learned my addr 172.21.15.114:0/1517592026 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:49.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.000+0000 7f90bd800640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e992070 0x55895ed50a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90bd800640 1 -- 172.21.15.114:0/1517592026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 msgr2=0x55895ed44e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90bd800640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed44e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90bd800640 1 -- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e9691b0 msgr2=0x55895ed44830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90bd800640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e9691b0 0x55895ed44830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90bd800640 1 -- 172.21.15.114:0/1517592026 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895e9fd650 con 0x55895e992070 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90b7e00640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed44e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90bd800640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e992070 0x55895ed50a40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f90b0012e00 tx=0x7f90b000eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48ad3385a36fc7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f909ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f909ea00640 1 -- 172.21.15.114:0/1517592026 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b002b060 con 0x55895e992070 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55895ec3cb90 con 0x55895e992070 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.001+0000 7f909ea00640 1 -- 172.21.15.114:0/1517592026 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f90b0012450 con 0x55895e992070 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f909ea00640 1 -- 172.21.15.114:0/1517592026 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f90b0032050 con 0x55895e992070 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f90c6490740 10 client.28610 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:49.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55895ec57950 con 0x55895e992070 2024-09-21T07:06:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f909ea00640 1 -- 172.21.15.114:0/1517592026 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55895ec3cb90 con 0x55895e992070 2024-09-21T07:06:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f909ea00640 1 -- 172.21.15.114:0/1517592026 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f90b000f040 con 0x55895e992070 2024-09-21T07:06:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f90c6490740 10 client.28610 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f90c6490740 10 client.28610 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:06:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f90c6490740 20 client.28610 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:49.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.002+0000 7f90c6490740 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e9509f0 0x55895ed56800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.003+0000 7f90c6490740 4 client.28610 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.003+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895ecb1f00 con 0x55895e9509f0 2024-09-21T07:06:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.003+0000 7f90bce00640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e9509f0 0x55895ed56800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.003+0000 7f90bce00640 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e9509f0 0x55895ed56800 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8e81489759bbb3a server_cookie=6eb0d9ac7e83eb26 in_seq=0 out_seq=0 2024-09-21T07:06:49.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.003+0000 7f909ea00640 10 client.28610 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.004+0000 7f909ea00640 1 -- 172.21.15.114:0/1517592026 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55895ecb1f00 con 0x55895e9509f0 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.004+0000 7f909ea00640 10 client.28610 handle_command_reply: tid=0 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.004+0000 7f90c6490740 1 client.28610 shutdown 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e992070 msgr2=0x55895ed50a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e992070 0x55895ed50a40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f90b0012e00 tx=0x7f90b000eb00 comp rx=0 tx=0).stop 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 shutdown_connections 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e9509f0 0x55895ed56800 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e992070 0x55895ed50a40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e971390 0x55895ed44e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 --2- 172.21.15.114:0/1517592026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e9691b0 0x55895ed44830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f909ea00640 0 client.28610 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:06:49.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 >> 172.21.15.114:0/1517592026 conn(0x55895e986460 msgr2=0x55895ed35600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 shutdown_connections 2024-09-21T07:06:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 1 -- 172.21.15.114:0/1517592026 wait complete. 2024-09-21T07:06:49.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.005+0000 7f90c6490740 20 client.28610 trim_cache size 0 max 16384 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 Processor -- start 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 -- start start 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed45d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed437c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81aeb80 con 0x55895e992070 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81b0720 con 0x55895e971390 2024-09-21T07:06:49.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.019+0000 7f90c6490740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81af250 con 0x55895e9691b0 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed437c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55398/0 (socket says 172.21.15.114:55398) 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 -- 172.21.15.114:0/1072726670 learned_addr learned my addr 172.21.15.114:0/1072726670 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90b7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed45d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bce00640 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 -- 172.21.15.114:0/1072726670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 msgr2=0x55895ed3a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed3a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 -- 172.21.15.114:0/1072726670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 msgr2=0x55895ed45d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed45d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 -- 172.21.15.114:0/1072726670 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895ed46c00 con 0x55895e992070 2024-09-21T07:06:49.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.020+0000 7f90bd800640 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed437c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90b00106a0 tx=0x7f90b0010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ca7531087e4817 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:49.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.021+0000 7f90a9600640 1 -- 172.21.15.114:0/1072726670 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b002b040 con 0x55895e992070 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.021+0000 7f90a9600640 1 -- 172.21.15.114:0/1072726670 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b000e6c0 con 0x55895e992070 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.021+0000 7f90a9600640 1 -- 172.21.15.114:0/1072726670 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b00126b0 con 0x55895e992070 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.021+0000 7f90c6490740 1 -- 172.21.15.114:0/1072726670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 msgr2=0x55895ed437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.021+0000 7f90c6490740 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed437c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90b00106a0 tx=0x7f90b0010470 comp rx=0 tx=0).stop 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 -- 172.21.15.114:0/1072726670 shutdown_connections 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed437c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed45d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 --2- 172.21.15.114:0/1072726670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed3a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 -- 172.21.15.114:0/1072726670 >> 172.21.15.114:0/1072726670 conn(0x55895e986460 msgr2=0x55895ed430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 -- 172.21.15.114:0/1072726670 shutdown_connections 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 -- 172.21.15.114:0/1072726670 wait complete. 2024-09-21T07:06:49.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 -- start start 2024-09-21T07:06:49.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.022+0000 7f90c6490740 1 Processor -- start 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed5a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed57780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed57e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81afff0 con 0x55895e992070 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81af920 con 0x55895e971390 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f90b81acb80 con 0x55895e9691b0 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90c6490740 10 client.0 _ll_register_callbacks cb 0x7f90a410b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90bd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed57e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90bd800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed57e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55412/0 (socket says 172.21.15.114:55412) 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90bd800640 1 -- 172.21.15.114:0/33316887 learned_addr learned my addr 172.21.15.114:0/33316887 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90b7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed57780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.023+0000 7f90bce00640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed5a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90bd800640 1 -- 172.21.15.114:0/33316887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 msgr2=0x55895ed5a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:49.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90bd800640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed5a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90bd800640 1 -- 172.21.15.114:0/33316887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 msgr2=0x55895ed57780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90bd800640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed57780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90bd800640 1 -- 172.21.15.114:0/33316887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895edbff80 con 0x55895e992070 2024-09-21T07:06:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90b7e00640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed57780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90bd800640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed57e90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f90b000e8a0 tx=0x7f90b0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833e00e62a6446a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:49.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f909ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f909ea00640 1 -- 172.21.15.114:0/33316887 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0044060 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f909ea00640 1 -- 172.21.15.114:0/33316887 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f90b0004fd0 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.024+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b000e6c0 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f909ea00640 1 -- 172.21.15.114:0/33316887 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f90b004b050 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 10 client.18048 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55895edc1100 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f909ea00640 1 -- 172.21.15.114:0/33316887 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b000e6c0 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f909ea00640 1 -- 172.21.15.114:0/33316887 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f90b0037560 con 0x55895e992070 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 10 client.18048 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 10 client.18048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 20 client.18048 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e925da0 0x55895edc3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 4 client.18048 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:06:49.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.025+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55895ecad9b0 con 0x55895e925da0 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.026+0000 7f90bce00640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e925da0 0x55895edc3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.026+0000 7f90bce00640 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e925da0 0x55895edc3720 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73eef2f049650a2f server_cookie=dd96d644bd42b2bb in_seq=0 out_seq=0 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.026+0000 7f909ea00640 10 client.18048 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.026+0000 7f909ea00640 1 -- 172.21.15.114:0/33316887 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55895ecad9b0 con 0x55895e925da0 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.026+0000 7f909ea00640 10 client.18048 handle_command_reply: tid=0 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.026+0000 7f90c6490740 1 client.18048 shutdown 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 msgr2=0x55895ed57e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed57e90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f90b000e8a0 tx=0x7f90b0004d50 comp rx=0 tx=0).stop 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 shutdown_connections 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55895e925da0 0x55895edc3720 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55895e992070 0x55895ed57e90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55895e971390 0x55895ed57780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 --2- 172.21.15.114:0/33316887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55895e9691b0 0x55895ed5a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 >> 172.21.15.114:0/33316887 conn(0x55895e986460 msgr2=0x55895ed430b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:49.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f909ea00640 0 client.18048 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:06:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 shutdown_connections 2024-09-21T07:06:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 1 -- 172.21.15.114:0/33316887 wait complete. 2024-09-21T07:06:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.027+0000 7f90c6490740 20 client.18048 trim_cache size 0 max 16384 2024-09-21T07:06:49.031 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:06:49.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90a00b6680 msgr2=0x7f90a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90a00b6680 0x7f90a00b8b90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55895ed37bf0 tx=0x7f90ac015040 comp rx=0 tx=0).stop 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 msgr2=0x7f90b819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b819a880 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f90a4005ab0 tx=0x7f90a4003b90 comp rx=0 tx=0).stop 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 shutdown_connections 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f90a00b6680 0x7f90a00b8b90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90b8109c40 0x7f90b819fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f90b81092f0 0x7f90b819a880 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 --2- 172.21.15.114:0/2887430215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b81039b0 0x7f90b819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 >> 172.21.15.114:0/2887430215 conn(0x7f90b80ff440 msgr2=0x7f90b8101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 shutdown_connections 2024-09-21T07:06:49.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:49.028+0000 7f90be200640 1 -- 172.21.15.114:0/2887430215 wait complete. 2024-09-21T07:06:49.893 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:06:50.310 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:50.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 -- 172.21.15.114:0/1813086796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 msgr2=0x7fb2dc114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/1813086796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc114200 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009900 tx=0x7fb2c802a410 comp rx=0 tx=0).stop 2024-09-21T07:06:50.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 -- 172.21.15.114:0/1813086796 shutdown_connections 2024-09-21T07:06:50.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/1813086796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc114740 0x7fb2dc116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/1813086796 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc114200 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/1813086796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2dc077940 0x7fb2dc077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 -- 172.21.15.114:0/1813086796 >> 172.21.15.114:0/1813086796 conn(0x7fb2dc101810 msgr2=0x7fb2dc103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 -- 172.21.15.114:0/1813086796 shutdown_connections 2024-09-21T07:06:50.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.916+0000 7fb2e0e00640 1 -- 172.21.15.114:0/1813086796 wait complete. 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.917+0000 7fb2e0e00640 1 Processor -- start 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 -- start start 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc077940 0x7fb2dc1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc1a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2dc114740 0x7fb2dc1a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc11a090 con 0x7fb2dc078290 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc119f10 con 0x7fb2dc114740 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2e0e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc11a210 con 0x7fb2dc077940 2024-09-21T07:06:50.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc1a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc1a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55426/0 (socket says 172.21.15.114:55426) 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2d9400640 1 -- 172.21.15.114:0/2465160556 learned_addr learned my addr 172.21.15.114:0/2465160556 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.918+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc077940 0x7fb2dc1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2da800640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2dc114740 0x7fb2dc1a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9e00640 1 -- 172.21.15.114:0/2465160556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2dc114740 msgr2=0x7fb2dc1a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2dc114740 0x7fb2dc1a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9e00640 1 -- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 msgr2=0x7fb2dc1a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc1a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9e00640 1 -- 172.21.15.114:0/2465160556 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc1a6a90 con 0x7fb2dc077940 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9400640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc077940 0x7fb2dc1a0b20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00e9f0 tx=0x7fb2cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.919+0000 7fb2d2a00640 1 -- 172.21.15.114:0/2465160556 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00ccc0 con 0x7fb2dc077940 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.920+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc1a6d80 con 0x7fb2dc077940 2024-09-21T07:06:50.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.920+0000 7fb2d2a00640 1 -- 172.21.15.114:0/2465160556 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2cc0108e0 con 0x7fb2dc077940 2024-09-21T07:06:50.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.920+0000 7fb2d2a00640 1 -- 172.21.15.114:0/2465160556 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc018880 con 0x7fb2dc077940 2024-09-21T07:06:50.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.920+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1ad4e0 con 0x7fb2dc077940 2024-09-21T07:06:50.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 Processor -- start 2024-09-21T07:06:50.924 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 -- start start 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 0x562abc0aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbcdb1b0 0x562abc0ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0ab820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562abc021fb0 con 0x562abbd04070 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562abc0113e0 con 0x562abbce3390 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.923+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562abc0151f0 con 0x562abbcdb1b0 2024-09-21T07:06:50.925 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 0x562abc0aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2d2a00640 1 -- 172.21.15.114:0/2465160556 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb2cc010430 con 0x7fb2dc077940 2024-09-21T07:06:50.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 0x562abc0aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55440/0 (socket says 172.21.15.114:55440) 2024-09-21T07:06:50.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 -- 172.21.15.114:0/3008784926 learned_addr learned my addr 172.21.15.114:0/3008784926 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:50.926 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2d9400640 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbcdb1b0 0x562abc0ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 -- 172.21.15.114:0/3008784926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbcdb1b0 msgr2=0x562abc0ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbcdb1b0 0x562abc0ab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 -- 172.21.15.114:0/3008784926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 msgr2=0x562abc0ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0ab820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 -- 172.21.15.114:0/3008784926 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abbfdbcd0 con 0x562abbd04070 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.924+0000 7fb2da800640 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 0x562abc0aada0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009f50 tx=0x7fb2d40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162e9f3944f4cb87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.925+0000 7fb2c7400640 1 -- 172.21.15.114:0/3008784926 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4004b20 con 0x562abbd04070 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.925+0000 7fb2c7400640 1 -- 172.21.15.114:0/3008784926 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4004cc0 con 0x562abbd04070 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.925+0000 7fb2c7400640 1 -- 172.21.15.114:0/3008784926 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4005020 con 0x562abbd04070 2024-09-21T07:06:50.927 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.925+0000 7fb2d2a00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2c00b6680 0x7fb2c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.928 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3008784926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 msgr2=0x562abc0aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 0x562abc0aada0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009f50 tx=0x7fb2d40118d0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2d2a00640 1 -- 172.21.15.114:0/2465160556 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fb2cc0e7e80 con 0x7fb2dc077940 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3008784926 shutdown_connections 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbcdb1b0 0x562abc0ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3008784926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbd04070 0x562abc0aada0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3008784926 >> 172.21.15.114:0/3008784926 conn(0x562abbcf8460 msgr2=0x562abc0a8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3008784926 shutdown_connections 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2c00b6680 0x7fb2c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.926+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3008784926 wait complete. 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 -- start start 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2d9e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2c00b6680 0x7fb2c00b8b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x562abc0a9a20 tx=0x7fb2cc009700 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.929 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 Processor -- start 2024-09-21T07:06:50.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbcdb1b0 0x562abc0b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 0x562abc0b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562abbe72700 con 0x562abbce3390 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562abbfac580 con 0x562abbcdb1b0 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562abc0b9580 con 0x562abbd04070 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.927+0000 7fb2e7a4d740 10 client.0 _ll_register_callbacks cb 0x562abbfbc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 0x562abc0b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 0x562abc0b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55450/0 (socket says 172.21.15.114:55450) 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 -- 172.21.15.114:0/3532943201 learned_addr learned my addr 172.21.15.114:0/3532943201 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2da800640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbcdb1b0 0x562abc0b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2da800640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 -- 172.21.15.114:0/3532943201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 msgr2=0x562abc0c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0c28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 -- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbcdb1b0 msgr2=0x562abc0b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbcdb1b0 0x562abc0b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2d9400640 1 -- 172.21.15.114:0/3532943201 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abbd6f650 con 0x562abbce3390 2024-09-21T07:06:50.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.928+0000 7fb2da800640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2d9400640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 0x562abc0b6a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006010 tx=0x7fb2c8031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66271e6f8be38bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2bbe00640 1 -- 172.21.15.114:0/3532943201 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8035040 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2bbe00640 1 -- 172.21.15.114:0/3532943201 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fb2c8004450 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562abbfaeb90 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2bbe00640 1 -- 172.21.15.114:0/3532943201 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb2c8005020 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2e7a4d740 10 client.18060 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x562abbfc9950 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.929+0000 7fb2bbe00640 1 -- 172.21.15.114:0/3532943201 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562abbfaeb90 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2bbe00640 1 -- 172.21.15.114:0/3532943201 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fb2c80035f0 con 0x562abbce3390 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2e7a4d740 10 client.18060 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2e7a4d740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2e7a4d740 20 client.18060 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbcc29f0 0x562abc0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2e7a4d740 4 client.18060 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:50.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562abc023f00 con 0x562abbcc29f0 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.930+0000 7fb2da800640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbcc29f0 0x562abc0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.931+0000 7fb2bbe00640 10 client.18060 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.931+0000 7fb2da800640 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbcc29f0 0x562abc0c8770 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b8bc2b1bb3beae server_cookie=327e890d58b7028 in_seq=0 out_seq=0 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2bbe00640 1 -- 172.21.15.114:0/3532943201 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562abc023f00 con 0x562abbcc29f0 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2bbe00640 10 client.18060 handle_command_reply: tid=0 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 client.18060 shutdown 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 msgr2=0x562abc0b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 0x562abc0b6a80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006010 tx=0x7fb2c8031740 comp rx=0 tx=0).stop 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 shutdown_connections 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbcc29f0 0x562abc0c8770 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbce3390 0x562abc0b6a80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3532943201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbcdb1b0 0x562abc0b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 >> 172.21.15.114:0/3532943201 conn(0x562abbcf8460 msgr2=0x562abc0adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2bbe00640 0 client.18060 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:50.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 shutdown_connections 2024-09-21T07:06:50.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3532943201 wait complete. 2024-09-21T07:06:50.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.932+0000 7fb2e7a4d740 20 client.18060 trim_cache size 0 max 16384 2024-09-21T07:06:50.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.946+0000 7fb2e7a4d740 1 Processor -- start 2024-09-21T07:06:50.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.946+0000 7fb2e7a4d740 1 -- start start 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbce3390 0x562abc0b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbd04070 0x562abc0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b74d0 con 0x562abbcdb1b0 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b9070 con 0x562abbd04070 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b7ba0 con 0x562abbce3390 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55456/0 (socket says 172.21.15.114:55456) 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 -- 172.21.15.114:0/3004586852 learned_addr learned my addr 172.21.15.114:0/3004586852 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2d9400640 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbce3390 0x562abc0b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbd04070 0x562abc0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 -- 172.21.15.114:0/3004586852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbce3390 msgr2=0x562abc0b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbce3390 0x562abc0b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 -- 172.21.15.114:0/3004586852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbd04070 msgr2=0x562abc0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbd04070 0x562abc0adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.947+0000 7fb2da800640 1 -- 172.21.15.114:0/3004586852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abc0b5d60 con 0x562abbcdb1b0 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.948+0000 7fb2da800640 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0b4dc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4004550 tx=0x7fb2d4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8ec9d7b291fa83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.948+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3004586852 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4037070 con 0x562abbcdb1b0 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.948+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3004586852 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4004940 con 0x562abbcdb1b0 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.948+0000 7fb2c6a00640 1 -- 172.21.15.114:0/3004586852 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4028a00 con 0x562abbcdb1b0 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3004586852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 msgr2=0x562abc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0b4dc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4004550 tx=0x7fb2d4009500 comp rx=0 tx=0).stop 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3004586852 shutdown_connections 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbd04070 0x562abc0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbce3390 0x562abc0b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/3004586852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0b4dc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3004586852 >> 172.21.15.114:0/3004586852 conn(0x562abbcf8460 msgr2=0x562abc0b64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3004586852 shutdown_connections 2024-09-21T07:06:50.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/3004586852 wait complete. 2024-09-21T07:06:50.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 -- start start 2024-09-21T07:06:50.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.949+0000 7fb2e7a4d740 1 Processor -- start 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0bec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b8940 con 0x562abbcdb1b0 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b8270 con 0x562abbce3390 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1b54d0 con 0x562abbd04070 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2e7a4d740 10 client.0 _ll_register_callbacks cb 0x7fb2c8005e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0bec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2da800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0bec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55466/0 (socket says 172.21.15.114:55466) 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.950+0000 7fb2da800640 1 -- 172.21.15.114:0/2365677969 learned_addr learned my addr 172.21.15.114:0/2365677969 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2d9400640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 -- 172.21.15.114:0/2365677969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 msgr2=0x562abc0ba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0ba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 -- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 msgr2=0x562abc0bf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0bf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 -- 172.21.15.114:0/2365677969 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abc0baa80 con 0x562abbcdb1b0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2d9400640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2da800640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0bec70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009270 tx=0x7fb2d40285e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355af42ef73f5c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2bbe00640 1 -- 172.21.15.114:0/2365677969 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4037040 con 0x562abbcdb1b0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2bbe00640 1 -- 172.21.15.114:0/2365677969 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fb2d4028890 con 0x562abbcdb1b0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.951+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d4004940 con 0x562abbcdb1b0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2bbe00640 1 -- 172.21.15.114:0/2365677969 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fb2d4028a90 con 0x562abbcdb1b0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 10 client.18066 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x562abc132d50 con 0x562abbcdb1b0 2024-09-21T07:06:50.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2bbe00640 1 -- 172.21.15.114:0/2365677969 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d4004940 con 0x562abbcdb1b0 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2bbe00640 1 -- 172.21.15.114:0/2365677969 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7fb2d400f480 con 0x562abbcdb1b0 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 10 client.18066 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 20 client.18066 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbc97da0 0x562abc1352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 4 client.18066 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:06:50.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.952+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562abc01f9b0 con 0x562abbc97da0 2024-09-21T07:06:50.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.953+0000 7fb2d9400640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbc97da0 0x562abc1352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:50.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.953+0000 7fb2d9400640 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbc97da0 0x562abc1352a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e386e421cbbed1b4 server_cookie=eaf32df4dad36af5 in_seq=0 out_seq=0 2024-09-21T07:06:50.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.953+0000 7fb2bbe00640 10 client.18066 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:50.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.953+0000 7fb2bbe00640 1 -- 172.21.15.114:0/2365677969 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562abc01f9b0 con 0x562abbc97da0 2024-09-21T07:06:50.957 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.953+0000 7fb2bbe00640 10 client.18066 handle_command_reply: tid=0 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.953+0000 7fb2e7a4d740 1 client.18066 shutdown 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 msgr2=0x562abc0bec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0bec70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4009270 tx=0x7fb2d40285e0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 shutdown_connections 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562abbc97da0 0x562abc1352a0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562abbd04070 0x562abc0ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562abbce3390 0x562abc0bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 --2- 172.21.15.114:0/2365677969 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562abbcdb1b0 0x562abc0bec70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 >> 172.21.15.114:0/2365677969 conn(0x562abbcf8460 msgr2=0x562abc0b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2bbe00640 0 client.18066 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 shutdown_connections 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 1 -- 172.21.15.114:0/2365677969 wait complete. 2024-09-21T07:06:50.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.954+0000 7fb2e7a4d740 20 client.18066 trim_cache size 0 max 16384 2024-09-21T07:06:50.959 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:06:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2c00b6680 msgr2=0x7fb2c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2c00b6680 0x7fb2c00b8b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x562abc0a9a20 tx=0x7fb2cc009700 comp rx=0 tx=0).stop 2024-09-21T07:06:50.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc077940 msgr2=0x7fb2dc1a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc077940 0x7fb2dc1a0b20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00e9f0 tx=0x7fb2cc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 shutdown_connections 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2c00b6680 0x7fb2c00b8b90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2dc114740 0x7fb2dc1a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2dc078290 0x7fb2dc1a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 --2- 172.21.15.114:0/2465160556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2dc077940 0x7fb2dc1a0b20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 >> 172.21.15.114:0/2465160556 conn(0x7fb2dc101810 msgr2=0x7fb2dc1167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 shutdown_connections 2024-09-21T07:06:50.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:50.955+0000 7fb2e0e00640 1 -- 172.21.15.114:0/2465160556 wait complete. 2024-09-21T07:06:51.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:51 smithi114 ceph-mon[30654]: pgmap v787: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-09-21T07:06:51.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:51 smithi183 ceph-mon[34219]: pgmap v787: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-09-21T07:06:51.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:51 smithi193 ceph-mon[34601]: pgmap v787: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-09-21T07:06:52.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 735.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:06:52.903 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:06:52.903 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:06:53.293 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:53.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:53 smithi114 ceph-mon[30654]: pgmap v788: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-09-21T07:06:53.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:53 smithi183 ceph-mon[34219]: pgmap v788: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-09-21T07:06:53.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:53 smithi193 ceph-mon[34601]: pgmap v788: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 4.2 MiB/s wr, 414 op/s 2024-09-21T07:06:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 -- 172.21.15.114:0/410428817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 msgr2=0x7f039c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 --2- 172.21.15.114:0/410428817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 0x7f039c103dc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0390009900 tx=0x7f039002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 -- 172.21.15.114:0/410428817 shutdown_connections 2024-09-21T07:06:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 --2- 172.21.15.114:0/410428817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039c109c40 0x7f039c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 --2- 172.21.15.114:0/410428817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 --2- 172.21.15.114:0/410428817 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 0x7f039c103dc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 -- 172.21.15.114:0/410428817 >> 172.21.15.114:0/410428817 conn(0x7f039c0ff440 msgr2=0x7f039c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 -- 172.21.15.114:0/410428817 shutdown_connections 2024-09-21T07:06:53.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.987+0000 7f03a2a00640 1 -- 172.21.15.114:0/410428817 wait complete. 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.988+0000 7f03a2a00640 1 Processor -- start 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 -- start start 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 0x7f039c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039c109c40 0x7f039c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113900 con 0x7f039c1039b0 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113780 con 0x7f039c109c40 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a2a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f039c113a80 con 0x7f039c1092f0 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 0x7f039c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:35354/0 (socket says 172.21.15.114:35354) 2024-09-21T07:06:53.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.989+0000 7f03a0c00640 1 -- 172.21.15.114:0/2043025996 learned_addr learned my addr 172.21.15.114:0/2043025996 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:53.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a2000640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039c109c40 0x7f039c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:53.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a0c00640 1 -- 172.21.15.114:0/2043025996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039c109c40 msgr2=0x7f039c19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:53.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a0c00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039c109c40 0x7f039c19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a0c00640 1 -- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 msgr2=0x7f039c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a0c00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 0x7f039c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a0c00640 1 -- 172.21.15.114:0/2043025996 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c1a02a0 con 0x7f039c1092f0 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.990+0000 7f03a0c00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c19a8d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f038800d950 tx=0x7f038800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.991+0000 7f039a000640 1 -- 172.21.15.114:0/2043025996 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388017070 con 0x7f039c1092f0 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.991+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c1a0590 con 0x7f039c1092f0 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.991+0000 7f039a000640 1 -- 172.21.15.114:0/2043025996 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0388009c30 con 0x7f039c1092f0 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.991+0000 7f039a000640 1 -- 172.21.15.114:0/2043025996 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880047a0 con 0x7f039c1092f0 2024-09-21T07:06:53.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.991+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a6cf0 con 0x7f039c1092f0 2024-09-21T07:06:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 Processor -- start 2024-09-21T07:06:53.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 -- start start 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6add350 0x564fe6e84f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e85440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6abc670 0x564fe6e85980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564fe6dfae90 con 0x564fe6abc670 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564fe6dea2c0 con 0x564fe6ab4490 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.994+0000 7f03aacd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564fe6dee0d0 con 0x564fe6add350 2024-09-21T07:06:53.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f039a000640 1 -- 172.21.15.114:0/2043025996 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0388004020 con 0x7f039c1092f0 2024-09-21T07:06:53.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6add350 0x564fe6e84f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:53.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e85440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:53.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e85440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:40782/0 (socket says 172.21.15.114:40782) 2024-09-21T07:06:53.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a1600640 1 -- 172.21.15.114:0/1480344432 learned_addr learned my addr 172.21.15.114:0/1480344432 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:53.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a0c00640 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6abc670 0x564fe6e85980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a2000640 1 -- 172.21.15.114:0/1480344432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 msgr2=0x564fe6e85440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a2000640 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e85440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a2000640 1 -- 172.21.15.114:0/1480344432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6abc670 msgr2=0x564fe6e85980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a2000640 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6abc670 0x564fe6e85980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a2000640 1 -- 172.21.15.114:0/1480344432 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe6b48930 con 0x564fe6add350 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.995+0000 7f03a0c00640 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6abc670 0x564fe6e85980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:06:53.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.996+0000 7f03a2000640 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6add350 0x564fe6e84f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0394009e30 tx=0x7f0394012b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e60cfa4b23f90a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:53.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.996+0000 7f038e600640 1 -- 172.21.15.114:0/1480344432 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394003a30 con 0x564fe6add350 2024-09-21T07:06:53.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.996+0000 7f038e600640 1 -- 172.21.15.114:0/1480344432 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039400dd70 con 0x564fe6add350 2024-09-21T07:06:53.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.996+0000 7f038e600640 1 -- 172.21.15.114:0/1480344432 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394004520 con 0x564fe6add350 2024-09-21T07:06:53.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.996+0000 7f039a000640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f03840b6680 0x7f03840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:53.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f039a000640 1 -- 172.21.15.114:0/2043025996 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f03880e7790 con 0x7f039c1092f0 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03a1600640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f03840b6680 0x7f03840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 -- 172.21.15.114:0/1480344432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6add350 msgr2=0x564fe6e84f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6add350 0x564fe6e84f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0394009e30 tx=0x7f0394012b30 comp rx=0 tx=0).stop 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 -- 172.21.15.114:0/1480344432 shutdown_connections 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6abc670 0x564fe6e85980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e85440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1480344432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6add350 0x564fe6e84f00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 -- 172.21.15.114:0/1480344432 >> 172.21.15.114:0/1480344432 conn(0x564fe6ad1740 msgr2=0x564fe6e82df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.997+0000 7f03aacd8740 1 -- 172.21.15.114:0/1480344432 shutdown_connections 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.998+0000 7f03aacd8740 1 -- 172.21.15.114:0/1480344432 wait complete. 2024-09-21T07:06:54.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.998+0000 7f03a1600640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f03840b6680 0x7f03840b8b90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0390009680 tx=0x7f0390009610 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:54.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.998+0000 7f03aacd8740 1 -- start start 2024-09-21T07:06:54.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.998+0000 7f03aacd8740 1 Processor -- start 2024-09-21T07:06:54.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e90a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e90f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e9c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x564fe6c4b890 con 0x564fe6ab4490 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564fe6d85160 con 0x564fe6add350 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564fe6e936e0 con 0x564fe6abc670 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03aacd8740 10 client.0 _ll_register_callbacks cb 0x564fe6da1a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03a1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e90a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03a0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e90f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03a1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e90a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55494/0 (socket says 172.21.15.114:55494) 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03a1600640 1 -- 172.21.15.114:0/4128556532 learned_addr learned my addr 172.21.15.114:0/4128556532 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:53.999+0000 7f03a2000640 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e9c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03a0c00640 1 -- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 msgr2=0x564fe6e9c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03a0c00640 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e9c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03a0c00640 1 -- 172.21.15.114:0/4128556532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 msgr2=0x564fe6e90a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:54.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03a0c00640 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e90a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03a0c00640 1 -- 172.21.15.114:0/4128556532 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe6e9cf40 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03a0c00640 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e90f90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03880e4ab0 tx=0x7f03880a9050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a38999852f582462 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f0382a00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f0382a00640 1 -- 172.21.15.114:0/4128556532 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880ac920 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f0382a00640 1 -- 172.21.15.114:0/4128556532 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f03880e74b0 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.000+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039400dd70 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f0382a00640 1 -- 172.21.15.114:0/4128556532 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f03880e4f40 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 10 client.28612 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564fe6e9f870 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f0382a00640 1 -- 172.21.15.114:0/4128556532 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039400dd70 con 0x564fe6abc670 2024-09-21T07:06:54.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f0382a00640 1 -- 172.21.15.114:0/4128556532 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f03880e7b80 con 0x564fe6abc670 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 10 client.28612 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 10 client.28612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 20 client.28612 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a9bcd0 0x564fe6ea2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 4 client.28612 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.001+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fe6dfcde0 con 0x564fe6a9bcd0 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.002+0000 7f03a1600640 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a9bcd0 0x564fe6ea2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.002+0000 7f03a1600640 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a9bcd0 0x564fe6ea2cd0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3521381d5b8f54d server_cookie=b97a5d0b3afe9345 in_seq=0 out_seq=0 2024-09-21T07:06:54.005 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.002+0000 7f0382a00640 10 client.28612 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.003+0000 7f0382a00640 1 -- 172.21.15.114:0/4128556532 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fe6dfcde0 con 0x564fe6a9bcd0 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.003+0000 7f0382a00640 10 client.28612 handle_command_reply: tid=0 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.003+0000 7f03aacd8740 1 client.28612 shutdown 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 msgr2=0x564fe6e90f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e90f90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f03880e4ab0 tx=0x7f03880a9050 comp rx=0 tx=0).stop 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 shutdown_connections 2024-09-21T07:06:54.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e9c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a9bcd0 0x564fe6ea2cd0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e90f90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 --2- 172.21.15.114:0/4128556532 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e90a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f0382a00640 0 client.28612 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 >> 172.21.15.114:0/4128556532 conn(0x564fe6ad1740 msgr2=0x564fe6e88090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 shutdown_connections 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 1 -- 172.21.15.114:0/4128556532 wait complete. 2024-09-21T07:06:54.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.004+0000 7f03aacd8740 20 client.28612 trim_cache size 0 max 16384 2024-09-21T07:06:54.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.022+0000 7f03aacd8740 1 Processor -- start 2024-09-21T07:06:54.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.022+0000 7f03aacd8740 1 -- start start 2024-09-21T07:06:54.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e9c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e8ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e92480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03aacd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1aebb0 con 0x564fe6ab4490 2024-09-21T07:06:54.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03aacd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1b0750 con 0x564fe6add350 2024-09-21T07:06:54.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03aacd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1af280 con 0x564fe6abc670 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03a1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e9c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03a0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e8ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03a1600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e9c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55510/0 (socket says 172.21.15.114:55510) 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03a1600640 1 -- 172.21.15.114:0/2685665535 learned_addr learned my addr 172.21.15.114:0/2685665535 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.023+0000 7f03a2000640 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e92480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a0c00640 1 -- 172.21.15.114:0/2685665535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 msgr2=0x564fe6e92480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a0c00640 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e92480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a0c00640 1 -- 172.21.15.114:0/2685665535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 msgr2=0x564fe6e9c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a0c00640 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e9c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a0c00640 1 -- 172.21.15.114:0/2685665535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fe6e87790 con 0x564fe6abc670 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a1600640 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f03a0c00640 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e8ffb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f038800bfd0 tx=0x7f0388005190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29437895f4364afc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f038dc00640 1 -- 172.21.15.114:0/2685665535 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880accc0 con 0x564fe6abc670 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f038dc00640 1 -- 172.21.15.114:0/2685665535 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0388009de0 con 0x564fe6abc670 2024-09-21T07:06:54.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.024+0000 7f038dc00640 1 -- 172.21.15.114:0/2685665535 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880e7510 con 0x564fe6abc670 2024-09-21T07:06:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 -- 172.21.15.114:0/2685665535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 msgr2=0x564fe6e8ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e8ffb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f038800bfd0 tx=0x7f0388005190 comp rx=0 tx=0).stop 2024-09-21T07:06:54.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 -- 172.21.15.114:0/2685665535 shutdown_connections 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6add350 0x564fe6e92480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e8ffb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 --2- 172.21.15.114:0/2685665535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6ab4490 0x564fe6e9c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 -- 172.21.15.114:0/2685665535 >> 172.21.15.114:0/2685665535 conn(0x564fe6ad1740 msgr2=0x564fe6e91d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 -- 172.21.15.114:0/2685665535 shutdown_connections 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.025+0000 7f03aacd8740 1 -- 172.21.15.114:0/2685665535 wait complete. 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.026+0000 7f03aacd8740 1 -- start start 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.026+0000 7f03aacd8740 1 Processor -- start 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.026+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e95060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.026+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.026+0000 7f03aacd8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6add350 0x564fe6f0c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03aacd8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1b0020 con 0x564fe6add350 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03aacd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1af950 con 0x564fe6ab4490 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03aacd8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f039c1acbb0 con 0x564fe6abc670 2024-09-21T07:06:54.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03aacd8740 10 client.0 _ll_register_callbacks cb 0x7f03880056f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:54.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6add350 0x564fe6f0c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a1600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e95060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a2000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6add350 0x564fe6f0c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55514/0 (socket says 172.21.15.114:55514) 2024-09-21T07:06:54.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a2000640 1 -- 172.21.15.114:0/1176158455 learned_addr learned my addr 172.21.15.114:0/1176158455 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:54.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a0c00640 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a1600640 1 -- 172.21.15.114:0/1176158455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 msgr2=0x564fe6e955a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a1600640 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e955a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a1600640 1 -- 172.21.15.114:0/1176158455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6add350 msgr2=0x564fe6f0c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a1600640 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6add350 0x564fe6f0c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.027+0000 7f03a1600640 1 -- 172.21.15.114:0/1176158455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fe6f0cc00 con 0x564fe6ab4490 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f03a1600640 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e95060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f039002a910 tx=0x7f039002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec96c17188bc420 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f0382a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f0382a00640 1 -- 172.21.15.114:0/1176158455 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039005f060 con 0x564fe6ab4490 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0388009de0 con 0x564fe6ab4490 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f0382a00640 1 -- 172.21.15.114:0/1176158455 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f0390005040 con 0x564fe6ab4490 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f0382a00640 1 -- 172.21.15.114:0/1176158455 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0390066050 con 0x564fe6ab4490 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f03aacd8740 10 client.28652 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.028+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x564fe6f0dec0 con 0x564fe6ab4490 2024-09-21T07:06:54.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f0382a00640 1 -- 172.21.15.114:0/1176158455 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0388009de0 con 0x564fe6ab4490 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f0382a00640 1 -- 172.21.15.114:0/1176158455 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f03900035f0 con 0x564fe6ab4490 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f03aacd8740 10 client.28652 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f03aacd8740 10 client.28652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f03aacd8740 20 client.28652 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a71080 0x564fe6f106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f03aacd8740 4 client.28652 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.029+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564fe6e8c400 con 0x564fe6a71080 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.030+0000 7f03a0c00640 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a71080 0x564fe6f106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.030+0000 7f03a0c00640 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a71080 0x564fe6f106e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85451fc1e53d86ad server_cookie=d179b9515707ad6b in_seq=0 out_seq=0 2024-09-21T07:06:54.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.030+0000 7f0382a00640 10 client.28652 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f0382a00640 1 -- 172.21.15.114:0/1176158455 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564fe6e8c400 con 0x564fe6a71080 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f0382a00640 10 client.28652 handle_command_reply: tid=0 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 client.28652 shutdown 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 msgr2=0x564fe6e95060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e95060 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f039002a910 tx=0x7f039002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 shutdown_connections 2024-09-21T07:06:54.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x564fe6add350 0x564fe6f0c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x564fe6a71080 0x564fe6f106e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564fe6abc670 0x564fe6e955a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 --2- 172.21.15.114:0/1176158455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564fe6ab4490 0x564fe6e95060 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f0382a00640 0 client.28652 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 >> 172.21.15.114:0/1176158455 conn(0x564fe6ad1740 msgr2=0x564fe6e924d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 shutdown_connections 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.032+0000 7f03aacd8740 1 -- 172.21.15.114:0/1176158455 wait complete. 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03aacd8740 20 client.28652 trim_cache size 0 max 16384 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:06:54.035 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e", 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f03840b6680 msgr2=0x7f03840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f03840b6680 0x7f03840b8b90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0390009680 tx=0x7f0390009610 comp rx=0 tx=0).stop 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 msgr2=0x7f039c19a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c19a8d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f038800d950 tx=0x7f038800de10 comp rx=0 tx=0).stop 2024-09-21T07:06:54.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 shutdown_connections 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f03840b6680 0x7f03840b8b90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f039c109c40 0x7f039c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f039c1092f0 0x7f039c19a8d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 --2- 172.21.15.114:0/2043025996 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f039c1039b0 0x7f039c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.033+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 >> 172.21.15.114:0/2043025996 conn(0x7f039c0ff440 msgr2=0x7f039c101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.034+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 shutdown_connections 2024-09-21T07:06:54.037 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:54.034+0000 7f03a2a00640 1 -- 172.21.15.114:0/2043025996 wait complete. 2024-09-21T07:06:54.628 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:06:54.960 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:06:55.054 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: pgmap v789: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.202 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.203 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:55 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: pgmap v789: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.366 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:55 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: pgmap v789: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.3 MiB/s wr, 340 op/s 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:55 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 -- 172.21.15.114:0/562402773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 msgr2=0x7fea9c106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 --2- 172.21.15.114:0/562402773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 0x7fea9c106690 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea88009900 tx=0x7fea8802a410 comp rx=0 tx=0).stop 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 -- 172.21.15.114:0/562402773 shutdown_connections 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 --2- 172.21.15.114:0/562402773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c106bd0 0x7fea9c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 --2- 172.21.15.114:0/562402773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 0x7fea9c106690 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 --2- 172.21.15.114:0/562402773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c102030 0x7fea9c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 -- 172.21.15.114:0/562402773 >> 172.21.15.114:0/562402773 conn(0x7fea9c077c60 msgr2=0x7fea9c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 -- 172.21.15.114:0/562402773 shutdown_connections 2024-09-21T07:06:55.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.770+0000 7feaa3600640 1 -- 172.21.15.114:0/562402773 wait complete. 2024-09-21T07:06:55.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.771+0000 7feaa3600640 1 Processor -- start 2024-09-21T07:06:55.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.771+0000 7feaa3600640 1 -- start start 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa3600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 0x7fea9c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa3600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 0x7fea9c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa3600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c106bd0 0x7fea9c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa3600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c113af0 con 0x7fea9c106280 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa3600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c113970 con 0x7fea9c102030 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa3600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c113c70 con 0x7fea9c106bd0 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 0x7fea9c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 0x7fea9c19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:60976/0 (socket says 172.21.15.114:60976) 2024-09-21T07:06:55.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.772+0000 7feaa0c00640 1 -- 172.21.15.114:0/4117951570 learned_addr learned my addr 172.21.15.114:0/4117951570 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7fea9be00640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 0x7fea9c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7feaa0c00640 1 -- 172.21.15.114:0/4117951570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c106bd0 msgr2=0x7fea9c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7feaa0c00640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c106bd0 0x7fea9c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7feaa0c00640 1 -- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 msgr2=0x7fea9c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7feaa0c00640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 0x7fea9c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7feaa0c00640 1 -- 172.21.15.114:0/4117951570 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1a0260 con 0x7fea9c102030 2024-09-21T07:06:55.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.773+0000 7feaa0c00640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 0x7fea9c19a4e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00e9f0 tx=0x7fea8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:55.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.774+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00cd80 con 0x7fea9c102030 2024-09-21T07:06:55.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.774+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1a0550 con 0x7fea9c102030 2024-09-21T07:06:55.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.774+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a0ab0 con 0x7fea9c102030 2024-09-21T07:06:55.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.774+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c0035f0 con 0x7fea9c102030 2024-09-21T07:06:55.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.774+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c010690 con 0x7fea9c102030 2024-09-21T07:06:55.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.775+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c102440 con 0x7fea9c102030 2024-09-21T07:06:55.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.778+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fea8c004000 con 0x7fea9c102030 2024-09-21T07:06:55.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.779+0000 7fea99600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea800b6700 0x7fea800b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:55.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.779+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7fea8c0e8060 con 0x7fea9c102030 2024-09-21T07:06:55.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.780+0000 7fea9be00640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea800b6700 0x7fea800b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:55.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.781+0000 7fea9be00640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea800b6700 0x7fea800b8c10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fea8802aa00 tx=0x7fea88000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:55.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:55.781+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea8c0ae180 con 0x7fea9c102030 2024-09-21T07:06:55.796 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:56.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.027+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea9c19b610 con 0x7fea9c102030 2024-09-21T07:06:56.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.028+0000 7fea99600640 1 -- 172.21.15.114:0/4117951570 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11993 (secure 0 0 0) 0x7fea8c0b2560 con 0x7fea9c102030 2024-09-21T07:06:56.030 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:06:56.030 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":52,"btime":"2024-09-21T07:06:21:307707+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:06:20.622407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:06:56.030 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 52 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea800b6700 msgr2=0x7fea800b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea800b6700 0x7fea800b8c10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fea8802aa00 tx=0x7fea88000bc0 comp rx=0 tx=0).stop 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 msgr2=0x7fea9c19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 0x7fea9c19a4e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00e9f0 tx=0x7fea8c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 shutdown_connections 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c106bd0 0x7fea9c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea9c106280 0x7fea9c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:56.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea800b6700 0x7fea800b8c10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:56.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 --2- 172.21.15.114:0/4117951570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c102030 0x7fea9c19a4e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:56.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 >> 172.21.15.114:0/4117951570 conn(0x7fea9c077c60 msgr2=0x7fea9c107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:56.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.030+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 shutdown_connections 2024-09-21T07:06:56.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:56.031+0000 7feaa3600640 1 -- 172.21.15.114:0/4117951570 wait complete. 2024-09-21T07:06:56.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:56 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:56.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:56 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4117951570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:56.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:56 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:56.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:56 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4117951570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:56.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:56 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:06:56.417 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:56 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4117951570' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:06:57.045 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id faeaa3be --release --await' 2024-09-21T07:06:57.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id faeaa3be --release --await 2024-09-21T07:06:57.261 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:57 smithi114 ceph-mon[30654]: pgmap v790: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-21T07:06:57.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.303+0000 7f1976c00640 1 -- 172.21.15.114:0/2939406164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701092f0 msgr2=0x7f1970109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.303+0000 7f1976c00640 1 --2- 172.21.15.114:0/2939406164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701092f0 0x7f1970109700 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 -- 172.21.15.114:0/2939406164 shutdown_connections 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 --2- 172.21.15.114:0/2939406164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 0x7f1970110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 --2- 172.21.15.114:0/2939406164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701092f0 0x7f1970109700 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 --2- 172.21.15.114:0/2939406164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19701039b0 0x7f1970103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 -- 172.21.15.114:0/2939406164 >> 172.21.15.114:0/2939406164 conn(0x7f19700ff440 msgr2=0x7f1970101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 -- 172.21.15.114:0/2939406164 shutdown_connections 2024-09-21T07:06:57.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.304+0000 7f1976c00640 1 -- 172.21.15.114:0/2939406164 wait complete. 2024-09-21T07:06:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.305+0000 7f1976c00640 1 Processor -- start 2024-09-21T07:06:57.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.305+0000 7f1976c00640 1 -- start start 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701039b0 0x7f197019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19701092f0 0x7f197019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 0x7f19701a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1970113900 con 0x7f19701039b0 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1970113780 con 0x7f19701092f0 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1970113a80 con 0x7f1970109c40 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1975800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701039b0 0x7f197019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1975800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701039b0 0x7f197019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34290/0 (socket says 172.21.15.114:34290) 2024-09-21T07:06:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1974e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19701092f0 0x7f197019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1975800640 1 -- 172.21.15.114:0/595297176 learned_addr learned my addr 172.21.15.114:0/595297176 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.306+0000 7f1976200640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 0x7f19701a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976200640 1 -- 172.21.15.114:0/595297176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19701092f0 msgr2=0x7f197019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976200640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19701092f0 0x7f197019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976200640 1 -- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701039b0 msgr2=0x7f197019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976200640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701039b0 0x7f197019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976200640 1 -- 172.21.15.114:0/595297176 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19701a24c0 con 0x7f1970109c40 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976200640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 0x7f19701a1d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1968009fd0 tx=0x7f196800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f196e000640 1 -- 172.21.15.114:0/595297176 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196800ce20 con 0x7f1970109c40 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a27b0 con 0x7f1970109c40 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.307+0000 7f196e000640 1 -- 172.21.15.114:0/595297176 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1968005020 con 0x7f1970109c40 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.308+0000 7f196e000640 1 -- 172.21.15.114:0/595297176 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968010690 con 0x7f1970109c40 2024-09-21T07:06:57.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.308+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a8f10 con 0x7f1970109c40 2024-09-21T07:06:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.311+0000 7f196e000640 1 -- 172.21.15.114:0/595297176 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f19680054d0 con 0x7f1970109c40 2024-09-21T07:06:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.312+0000 7f197efeb740 1 Processor -- start 2024-09-21T07:06:57.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.312+0000 7f197efeb740 1 -- start start 2024-09-21T07:06:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f197efeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa7f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f197efeb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aa7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f197efeb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f197efeb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b13a9edf70 con 0x55b13a6a71b0 2024-09-21T07:06:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f197efeb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b13a9dd3a0 con 0x55b13a6af390 2024-09-21T07:06:57.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f197efeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b13a9e11b0 con 0x55b13a6d0070 2024-09-21T07:06:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f196e000640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f19580b6700 0x7f19580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.313+0000 7f196e000640 1 -- 172.21.15.114:0/595297176 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f19680e81b0 con 0x7f1970109c40 2024-09-21T07:06:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa7f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa7f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56974/0 (socket says 172.21.15.114:56974) 2024-09-21T07:06:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 -- 172.21.15.114:0/3491787515 learned_addr learned my addr 172.21.15.114:0/3491787515 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:57.316 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1976200640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f19580b6700 0x7f19580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1975800640 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aa7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 -- 172.21.15.114:0/3491787515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 msgr2=0x55b13aa801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 -- 172.21.15.114:0/3491787515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 msgr2=0x55b13aa7fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aa7fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.314+0000 7f1974e00640 1 -- 172.21.15.114:0/3491787515 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13a9a7c90 con 0x55b13a6d0070 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.315+0000 7f1976200640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f19580b6700 0x7f19580b8c10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f19680e5530 tx=0x7f1968005f70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:57.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.315+0000 7f1974e00640 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa7f740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f195c0057c0 tx=0x7f195c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f502f572d67899b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.315+0000 7f1962a00640 1 -- 172.21.15.114:0/3491787515 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c035070 con 0x55b13a6d0070 2024-09-21T07:06:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.315+0000 7f1962a00640 1 -- 172.21.15.114:0/3491787515 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c004030 con 0x55b13a6d0070 2024-09-21T07:06:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.315+0000 7f1962a00640 1 -- 172.21.15.114:0/3491787515 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c0053b0 con 0x55b13a6d0070 2024-09-21T07:06:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 -- 172.21.15.114:0/3491787515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 msgr2=0x55b13aa7f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa7f740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f195c0057c0 tx=0x7f195c000bc0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 -- 172.21.15.114:0/3491787515 shutdown_connections 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aa7fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 --2- 172.21.15.114:0/3491787515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa7f740 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 -- 172.21.15.114:0/3491787515 >> 172.21.15.114:0/3491787515 conn(0x55b13a6c4460 msgr2=0x55b13aa7d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.316+0000 7f197efeb740 1 -- 172.21.15.114:0/3491787515 shutdown_connections 2024-09-21T07:06:57.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.317+0000 7f197efeb740 1 -- 172.21.15.114:0/3491787515 wait complete. 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.317+0000 7f197efeb740 1 -- start start 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.317+0000 7f197efeb740 1 Processor -- start 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6a71b0 0x55b13aa79930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa79e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6d0070 0x55b13aa7a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b13a83e700 con 0x55b13a6d0070 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b13a978540 con 0x55b13a6af390 2024-09-21T07:06:57.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b13aa7c9f0 con 0x55b13a6a71b0 2024-09-21T07:06:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f197efeb740 10 client.0 _ll_register_callbacks cb 0x55b13a988430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f1974e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6a71b0 0x55b13aa79930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f1974e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6a71b0 0x55b13aa79930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56980/0 (socket says 172.21.15.114:56980) 2024-09-21T07:06:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f1974e00640 1 -- 172.21.15.114:0/2188140159 learned_addr learned my addr 172.21.15.114:0/2188140159 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f1975800640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa79e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.318+0000 7f1974e00640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6d0070 0x55b13aa7a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1974e00640 1 -- 172.21.15.114:0/2188140159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6a71b0 msgr2=0x55b13aa79930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1974e00640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6a71b0 0x55b13aa79930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1974e00640 1 -- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 msgr2=0x55b13aa79e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1974e00640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa79e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1974e00640 1 -- 172.21.15.114:0/2188140159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13a73b650 con 0x55b13a6d0070 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1974e00640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6d0070 0x55b13aa7a3b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f195c0096c0 tx=0x7f195c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d127c141cdd86a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.319+0000 7f1957400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.320+0000 7f1957400640 1 -- 172.21.15.114:0/2188140159 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c035070 con 0x55b13a6d0070 2024-09-21T07:06:57.322 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.320+0000 7f1957400640 1 -- 172.21.15.114:0/2188140159 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f195c031d10 con 0x55b13a6d0070 2024-09-21T07:06:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.320+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b13a97ab50 con 0x55b13a6d0070 2024-09-21T07:06:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.320+0000 7f1957400640 1 -- 172.21.15.114:0/2188140159 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f195c004020 con 0x55b13a6d0070 2024-09-21T07:06:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.320+0000 7f197efeb740 10 client.18102 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.320+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b13a995910 con 0x55b13a6d0070 2024-09-21T07:06:57.323 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f1957400640 1 -- 172.21.15.114:0/2188140159 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b13a97ab50 con 0x55b13a6d0070 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f1957400640 1 -- 172.21.15.114:0/2188140159 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f195c004750 con 0x55b13a6d0070 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f197efeb740 10 client.18102 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f197efeb740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f197efeb740 20 client.18102 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f197efeb740 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a68e9f0 0x55b13aaa55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f197efeb740 4 client.18102 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.321+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13a9efec0 con 0x55b13a68e9f0 2024-09-21T07:06:57.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.322+0000 7f1975800640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a68e9f0 0x55b13aaa55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.322+0000 7f1975800640 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a68e9f0 0x55b13aaa55b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f62638a2b9015e server_cookie=46757acafbccae8d in_seq=0 out_seq=0 2024-09-21T07:06:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.322+0000 7f1957400640 10 client.18102 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.323+0000 7f1957400640 1 -- 172.21.15.114:0/2188140159 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b13a9efec0 con 0x55b13a68e9f0 2024-09-21T07:06:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.323+0000 7f1957400640 10 client.18102 handle_command_reply: tid=0 2024-09-21T07:06:57.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.323+0000 7f197efeb740 1 client.18102 shutdown 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6d0070 msgr2=0x55b13aa7a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6d0070 0x55b13aa7a3b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f195c0096c0 tx=0x7f195c005fd0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 shutdown_connections 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6d0070 0x55b13aa7a3b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6af390 0x55b13aa79e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a68e9f0 0x55b13aaa55b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 --2- 172.21.15.114:0/2188140159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6a71b0 0x55b13aa79930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 >> 172.21.15.114:0/2188140159 conn(0x55b13a6c4460 msgr2=0x55b13aa81130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f1957400640 0 client.18102 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 shutdown_connections 2024-09-21T07:06:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 1 -- 172.21.15.114:0/2188140159 wait complete. 2024-09-21T07:06:57.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.324+0000 7f197efeb740 20 client.18102 trim_cache size 0 max 16384 2024-09-21T07:06:57.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.345+0000 7f197efeb740 1 Processor -- start 2024-09-21T07:06:57.347 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 -- start start 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6a71b0 0x55b13aa7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 0x55b13aa76b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa77060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19701aeb90 con 0x55b13a6af390 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19701b0730 con 0x55b13a6a71b0 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f197efeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19701af260 con 0x55b13a6d0070 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f1975800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 0x55b13aa76b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f1975800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 0x55b13aa76b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34312/0 (socket says 172.21.15.114:34312) 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f1975800640 1 -- 172.21.15.114:0/3397899541 learned_addr learned my addr 172.21.15.114:0/3397899541 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:57.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f1974e00640 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6a71b0 0x55b13aa7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.346+0000 7f1974e00640 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa77060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1975800640 1 -- 172.21.15.114:0/3397899541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 msgr2=0x55b13aa77060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1975800640 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa77060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1975800640 1 -- 172.21.15.114:0/3397899541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6a71b0 msgr2=0x55b13aa7b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1975800640 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6a71b0 0x55b13aa7b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1975800640 1 -- 172.21.15.114:0/3397899541 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13aa81c60 con 0x55b13a6af390 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1975800640 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 0x55b13aa76b20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x55b13aa7caf0 tx=0x7f1964005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9383fdb32f3061 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.347+0000 7f1962000640 1 -- 172.21.15.114:0/3397899541 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964003d20 con 0x55b13a6af390 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.348+0000 7f1962000640 1 -- 172.21.15.114:0/3397899541 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1964012280 con 0x55b13a6af390 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.348+0000 7f1962000640 1 -- 172.21.15.114:0/3397899541 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964020720 con 0x55b13a6af390 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 -- 172.21.15.114:0/3397899541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 msgr2=0x55b13aa76b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 0x55b13aa76b20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x55b13aa7caf0 tx=0x7f1964005f90 comp rx=0 tx=0).stop 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 -- 172.21.15.114:0/3397899541 shutdown_connections 2024-09-21T07:06:57.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6d0070 0x55b13aa77060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6af390 0x55b13aa76b20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 --2- 172.21.15.114:0/3397899541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6a71b0 0x55b13aa7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 -- 172.21.15.114:0/3397899541 >> 172.21.15.114:0/3397899541 conn(0x55b13a6c4460 msgr2=0x55b13aa7ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 -- 172.21.15.114:0/3397899541 shutdown_connections 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 -- 172.21.15.114:0/3397899541 wait complete. 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 -- start start 2024-09-21T07:06:57.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.349+0000 7f197efeb740 1 Processor -- start 2024-09-21T07:06:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aaa6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 0x55b13aaa6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6d0070 0x55b13aaa72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f19701b0000 con 0x55b13a6a71b0 2024-09-21T07:06:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19701af930 con 0x55b13a6d0070 2024-09-21T07:06:57.355 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f19701acc40 con 0x55b13a6af390 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f197efeb740 10 client.0 _ll_register_callbacks cb 0x7f195c0316c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f1975800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 0x55b13aaa6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f1974e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aaa6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f1975800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 0x55b13aaa6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56988/0 (socket says 172.21.15.114:56988) 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.350+0000 7f1975800640 1 -- 172.21.15.114:0/2425756045 learned_addr learned my addr 172.21.15.114:0/2425756045 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1974e00640 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6d0070 0x55b13aaa72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1975800640 1 -- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6d0070 msgr2=0x55b13aaa72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1975800640 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6d0070 0x55b13aaa72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1975800640 1 -- 172.21.15.114:0/2425756045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 msgr2=0x55b13aaa6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1975800640 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aaa6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1975800640 1 -- 172.21.15.114:0/2425756045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13ab0ef80 con 0x55b13a6af390 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1975800640 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 0x55b13aaa6d60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1964009730 tx=0x7f1964003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ca771ebd7dc08a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:57.356 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1957400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f1957400640 1 -- 172.21.15.114:0/2425756045 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964003e50 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.351+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1964012280 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.352+0000 7f1957400640 1 -- 172.21.15.114:0/2425756045 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f1964005260 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.352+0000 7f1957400640 1 -- 172.21.15.114:0/2425756045 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f196403b050 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.352+0000 7f197efeb740 10 client.28639 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.352+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b13ab10210 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.352+0000 7f1957400640 1 -- 172.21.15.114:0/2425756045 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1964012280 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.352+0000 7f1957400640 1 -- 172.21.15.114:0/2425756045 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f1964012590 con 0x55b13a6af390 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f197efeb740 10 client.28639 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f197efeb740 10 client.28639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f197efeb740 20 client.28639 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f197efeb740 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a663da0 0x55b13ab12a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f197efeb740 4 client.28639 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:06:57.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b13a9eb970 con 0x55b13a663da0 2024-09-21T07:06:57.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f1974e00640 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a663da0 0x55b13ab12a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:57.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f1974e00640 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a663da0 0x55b13ab12a00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a06bfde9a230039 server_cookie=c5df0af3d5172057 in_seq=0 out_seq=0 2024-09-21T07:06:57.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.353+0000 7f1957400640 10 client.28639 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:57.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f1957400640 1 -- 172.21.15.114:0/2425756045 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55b13a9eb970 con 0x55b13a663da0 2024-09-21T07:06:57.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f1957400640 10 client.28639 handle_command_reply: tid=0 2024-09-21T07:06:57.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 client.28639 shutdown 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 msgr2=0x55b13aaa6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 0x55b13aaa6d60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1964009730 tx=0x7f1964003c30 comp rx=0 tx=0).stop 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 shutdown_connections 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b13a6d0070 0x55b13aaa72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b13a663da0 0x55b13ab12a00 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b13a6af390 0x55b13aaa6d60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 --2- 172.21.15.114:0/2425756045 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b13a6a71b0 0x55b13aaa6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 >> 172.21.15.114:0/2425756045 conn(0x55b13a6c4460 msgr2=0x55b13aa9eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f1957400640 0 client.28639 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 shutdown_connections 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 1 -- 172.21.15.114:0/2425756045 wait complete. 2024-09-21T07:06:57.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.354+0000 7f197efeb740 20 client.28639 trim_cache size 0 max 16384 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e", 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:06:57.360 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:06:54.030598+0000", 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:06:57.361 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.363 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.364 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:06:57.365 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:06:57.366 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:57.367 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f19580b6700 msgr2=0x7f19580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f19580b6700 0x7f19580b8c10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f19680e5530 tx=0x7f1968005f70 comp rx=0 tx=0).stop 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 msgr2=0x7f19701a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 0x7f19701a1d90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1968009fd0 tx=0x7f196800ef10 comp rx=0 tx=0).stop 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 shutdown_connections 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1970109c40 0x7f19701a1d90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f19580b6700 0x7f19580b8c10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19701092f0 0x7f197019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 --2- 172.21.15.114:0/595297176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f19701039b0 0x7f197019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:57.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 >> 172.21.15.114:0/595297176 conn(0x7f19700ff440 msgr2=0x7f19701089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:57.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 shutdown_connections 2024-09-21T07:06:57.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:57.356+0000 7f1976c00640 1 -- 172.21.15.114:0/595297176 wait complete. 2024-09-21T07:06:57.369 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:57 smithi183 ceph-mon[34219]: pgmap v790: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-21T07:06:57.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:57 smithi193 ceph-mon[34601]: pgmap v790: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 2.5 MiB/s wr, 274 op/s 2024-09-21T07:06:57.836 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 740.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:06:58.219 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:06:58.248 INFO:tasks.cephfs.filesystem:scrub status for tag:fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e - {'path': '/', 'tag': 'fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e', 'options': 'recursive,force'} 2024-09-21T07:06:58.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 -- 172.21.15.114:0/682861343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc1039b0 msgr2=0x7f26dc103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:58.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 --2- 172.21.15.114:0/682861343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc1039b0 0x7f26dc103dc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009900 tx=0x7f26d002a410 comp rx=0 tx=0).stop 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 -- 172.21.15.114:0/682861343 shutdown_connections 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 --2- 172.21.15.114:0/682861343 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26dc109c40 0x7f26dc110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 --2- 172.21.15.114:0/682861343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1092f0 0x7f26dc109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 --2- 172.21.15.114:0/682861343 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc1039b0 0x7f26dc103dc0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 -- 172.21.15.114:0/682861343 >> 172.21.15.114:0/682861343 conn(0x7f26dc0ff440 msgr2=0x7f26dc101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 -- 172.21.15.114:0/682861343 shutdown_connections 2024-09-21T07:06:58.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.857+0000 7f26e3000640 1 -- 172.21.15.114:0/682861343 wait complete. 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.858+0000 7f26e3000640 1 Processor -- start 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 -- start start 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1039b0 0x7f26dc19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26dc1092f0 0x7f26dc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 0x7f26dc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc113900 con 0x7f26dc109c40 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc113780 con 0x7f26dc1039b0 2024-09-21T07:06:58.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e3000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc113a80 con 0x7f26dc1092f0 2024-09-21T07:06:58.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 0x7f26dc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1039b0 0x7f26dc19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e2600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 0x7f26dc1a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34336/0 (socket says 172.21.15.114:34336) 2024-09-21T07:06:58.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e2600640 1 -- 172.21.15.114:0/3919656168 learned_addr learned my addr 172.21.15.114:0/3919656168 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:58.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.859+0000 7f26e1200640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26dc1092f0 0x7f26dc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e1c00640 1 -- 172.21.15.114:0/3919656168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26dc1092f0 msgr2=0x7f26dc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e1c00640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26dc1092f0 0x7f26dc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e1c00640 1 -- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 msgr2=0x7f26dc1a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e1c00640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 0x7f26dc1a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e1c00640 1 -- 172.21.15.114:0/3919656168 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26dc1a24c0 con 0x7f26dc1039b0 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e2600640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 0x7f26dc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.860+0000 7f26e1c00640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1039b0 0x7f26dc19c550 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f26d002a930 tx=0x7f26d0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.861+0000 7f26da000640 1 -- 172.21.15.114:0/3919656168 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d003f030 con 0x7f26dc1039b0 2024-09-21T07:06:58.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.861+0000 7f26da000640 1 -- 172.21.15.114:0/3919656168 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d0004490 con 0x7f26dc1039b0 2024-09-21T07:06:58.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.861+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26dc1a2750 con 0x7f26dc1039b0 2024-09-21T07:06:58.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.861+0000 7f26da000640 1 -- 172.21.15.114:0/3919656168 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0035070 con 0x7f26dc1039b0 2024-09-21T07:06:58.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.861+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26dc1a8f10 con 0x7f26dc1039b0 2024-09-21T07:06:58.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 Processor -- start 2024-09-21T07:06:58.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 -- start start 2024-09-21T07:06:58.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e840444290 con 0x55e840105840 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8404336c0 con 0x55e8400fd660 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.865+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8404374d0 con 0x55e840126520 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34350/0 (socket says 172.21.15.114:34350) 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 -- 172.21.15.114:0/796251020 learned_addr learned my addr 172.21.15.114:0/796251020 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e2600640 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26da000640 1 -- 172.21.15.114:0/3919656168 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f26d0042030 con 0x7f26dc1039b0 2024-09-21T07:06:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 -- 172.21.15.114:0/796251020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 msgr2=0x55e8404cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 -- 172.21.15.114:0/796251020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 msgr2=0x55e8404cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e1200640 1 -- 172.21.15.114:0/796251020 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8403c0460 con 0x55e840105840 2024-09-21T07:06:58.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.866+0000 7f26e2600640 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.867+0000 7f26e1200640 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404cddc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009c20 tx=0x7f26c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1845d90ac4a762 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.867+0000 7f26cea00640 1 -- 172.21.15.114:0/796251020 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004030 con 0x55e840105840 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.867+0000 7f26cea00640 1 -- 172.21.15.114:0/796251020 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c8005020 con 0x55e840105840 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.867+0000 7f26cea00640 1 -- 172.21.15.114:0/796251020 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004c20 con 0x55e840105840 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26da000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f26c40b6680 0x7f26c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 -- 172.21.15.114:0/796251020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 msgr2=0x55e8404cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:58.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404cddc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009c20 tx=0x7f26c8011810 comp rx=0 tx=0).stop 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 -- 172.21.15.114:0/796251020 shutdown_connections 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404cddc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/796251020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26da000640 1 -- 172.21.15.114:0/3919656168 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f26d0107d30 con 0x7f26dc1039b0 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 -- 172.21.15.114:0/796251020 >> 172.21.15.114:0/796251020 conn(0x55e84011a910 msgr2=0x55e8404cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 -- 172.21.15.114:0/796251020 shutdown_connections 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26e2600640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f26c40b6680 0x7f26c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.868+0000 7f26eb2cc740 1 -- 172.21.15.114:0/796251020 wait complete. 2024-09-21T07:06:58.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.869+0000 7f26eb2cc740 1 -- start start 2024-09-21T07:06:58.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.869+0000 7f26e2600640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f26c40b6680 0x7f26c40b8b90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f26d40049d0 tx=0x7f26d4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:58.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.869+0000 7f26eb2cc740 1 Processor -- start 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404d8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404d93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e840294bb0 con 0x55e840105840 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8403ce6c0 con 0x55e8400fd660 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e8404dbb20 con 0x55e840126520 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26eb2cc740 10 client.0 _ll_register_callbacks cb 0x55e8403e9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404d93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26e1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404d8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26e1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404d93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34360/0 (socket says 172.21.15.114:34360) 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26e1200640 1 -- 172.21.15.114:0/389003820 learned_addr learned my addr 172.21.15.114:0/389003820 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:58.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.870+0000 7f26e2600640 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26e1c00640 1 -- 172.21.15.114:0/389003820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 msgr2=0x55e8404e4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26e1c00640 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26e1c00640 1 -- 172.21.15.114:0/389003820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 msgr2=0x55e8404d93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26e1c00640 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404d93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26e1c00640 1 -- 172.21.15.114:0/389003820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e840191b00 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26e1c00640 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404d8e80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f26d01050b0 tx=0x7f26d01055b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805028829be8cc62 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26c3400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26c3400640 1 -- 172.21.15.114:0/389003820 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0049070 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c8005020 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26c3400640 1 -- 172.21.15.114:0/389003820 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f26d00d6d40 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26c3400640 1 -- 172.21.15.114:0/389003820 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f26d00cd1b0 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26eb2cc740 10 client.28685 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.871+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e840208960 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26c3400640 1 -- 172.21.15.114:0/389003820 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c8005020 con 0x55e8400fd660 2024-09-21T07:06:58.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26c3400640 1 -- 172.21.15.114:0/389003820 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f26d00ceae0 con 0x55e8400fd660 2024-09-21T07:06:58.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26eb2cc740 10 client.28685 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:58.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26eb2cc740 10 client.28685 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:06:58.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26eb2cc740 20 client.28685 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:58.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400e4ea0 0x55e8404ead10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26eb2cc740 4 client.28685 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:06:58.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8404461e0 con 0x55e8400e4ea0 2024-09-21T07:06:58.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.872+0000 7f26e1200640 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400e4ea0 0x55e8404ead10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.873+0000 7f26e1200640 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400e4ea0 0x55e8404ead10 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70d0a997634556a server_cookie=b295faef5ec5dc1b in_seq=0 out_seq=0 2024-09-21T07:06:58.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.873+0000 7f26c3400640 10 client.28685 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:58.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26c3400640 1 -- 172.21.15.114:0/389003820 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8404461e0 con 0x55e8400e4ea0 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26c3400640 10 client.28685 handle_command_reply: tid=0 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 client.28685 shutdown 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 msgr2=0x55e8404d8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404d8e80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f26d01050b0 tx=0x7f26d01055b0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 shutdown_connections 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400e4ea0 0x55e8404ead10 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e840105840 0x55e8404d93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/389003820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8400fd660 0x55e8404d8e80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26c3400640 0 client.28685 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 >> 172.21.15.114:0/389003820 conn(0x55e84011a910 msgr2=0x55e8404d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 shutdown_connections 2024-09-21T07:06:58.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.874+0000 7f26eb2cc740 1 -- 172.21.15.114:0/389003820 wait complete. 2024-09-21T07:06:58.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.875+0000 7f26eb2cc740 20 client.28685 trim_cache size 0 max 16384 2024-09-21T07:06:58.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.892+0000 7f26eb2cc740 1 Processor -- start 2024-09-21T07:06:58.894 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.892+0000 7f26eb2cc740 1 -- start start 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.892+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840105840 0x55e8404cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1b2f90 con 0x55e8400fd660 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1b4b30 con 0x55e840105840 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1b3660 con 0x55e840126520 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34376/0 (socket says 172.21.15.114:34376) 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 -- 172.21.15.114:0/360184978 learned_addr learned my addr 172.21.15.114:0/360184978 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:58.895 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1200640 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840105840 0x55e8404cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e2600640 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 -- 172.21.15.114:0/360184978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 msgr2=0x55e8404e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 -- 172.21.15.114:0/360184978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840105840 msgr2=0x55e8404cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840105840 0x55e8404cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.896 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.893+0000 7f26e1c00640 1 -- 172.21.15.114:0/360184978 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8404d13c0 con 0x55e8400fd660 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.894+0000 7f26e1c00640 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404d0ab0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f26d0031440 tx=0x7f26d003b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e2c9cfc0f0018 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.894+0000 7f26ce000640 1 -- 172.21.15.114:0/360184978 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d00e8070 con 0x55e8400fd660 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.894+0000 7f26ce000640 1 -- 172.21.15.114:0/360184978 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d00f3030 con 0x55e8400fd660 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.894+0000 7f26ce000640 1 -- 172.21.15.114:0/360184978 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0049070 con 0x55e8400fd660 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 -- 172.21.15.114:0/360184978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 msgr2=0x55e8404d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404d0ab0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f26d0031440 tx=0x7f26d003b460 comp rx=0 tx=0).stop 2024-09-21T07:06:58.897 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 -- 172.21.15.114:0/360184978 shutdown_connections 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840126520 0x55e8404e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840105840 0x55e8404cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/360184978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404d0ab0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 -- 172.21.15.114:0/360184978 >> 172.21.15.114:0/360184978 conn(0x55e84011a910 msgr2=0x55e8404ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 -- 172.21.15.114:0/360184978 shutdown_connections 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 -- 172.21.15.114:0/360184978 wait complete. 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.895+0000 7f26eb2cc740 1 -- start start 2024-09-21T07:06:58.898 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 Processor -- start 2024-09-21T07:06:58.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404dc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840105840 0x55e8404dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840126520 0x55e8405556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1b4400 con 0x55e8400fd660 2024-09-21T07:06:58.900 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1b3d30 con 0x55e840126520 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f26dc1b0f90 con 0x55e840105840 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26eb2cc740 10 client.0 _ll_register_callbacks cb 0x7f26d0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404dc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26e1c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404dc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34390/0 (socket says 172.21.15.114:34390) 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26e1c00640 1 -- 172.21.15.114:0/3951993350 learned_addr learned my addr 172.21.15.114:0/3951993350 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.896+0000 7f26e1200640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840105840 0x55e8404dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e2600640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840126520 0x55e8405556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e1c00640 1 -- 172.21.15.114:0/3951993350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840105840 msgr2=0x55e8404dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e1c00640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840105840 0x55e8404dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e1c00640 1 -- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840126520 msgr2=0x55e8405556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e1c00640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840126520 0x55e8405556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e1c00640 1 -- 172.21.15.114:0/3951993350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8404ca960 con 0x55e8400fd660 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26e1c00640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404dc190 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f26d00cd7f0 tx=0x7f26d01055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce660997606ab07f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:06:58.901 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26c3400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26c3400640 1 -- 172.21.15.114:0/3951993350 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0049070 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26c3400640 1 -- 172.21.15.114:0/3951993350 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f26d00e8e00 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.897+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d00f3030 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26c3400640 1 -- 172.21.15.114:0/3951993350 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f26d00e8050 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 10 client.18126 fetch_fsmap learned FSMap version 52 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55e8404ca960 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26c3400640 1 -- 172.21.15.114:0/3951993350 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26d00f3030 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26c3400640 1 -- 172.21.15.114:0/3951993350 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f26d00d6d40 con 0x55e8400fd660 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 10 client.18126 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 10 client.18126 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 20 client.18126 populate_metadata read hostname 'smithi114' 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400ba250 0x55e840558e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 4 client.18126 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "faeaa3be", "release": true, "await": true}] 2024-09-21T07:06:58.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.898+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "faeaa3be", "release": true, "await": true}) -- 0x55e840441c90 con 0x55e8400ba250 2024-09-21T07:06:58.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.899+0000 7f26e1200640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400ba250 0x55e840558e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:06:58.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.899+0000 7f26e1200640 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400ba250 0x55e840558e20 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b575548f7e455fb server_cookie=e0be922d7be700ac in_seq=0 out_seq=0 2024-09-21T07:06:58.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:58.899+0000 7f26c3400640 10 client.18126 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.082+0000 7f26c3400640 1 -- 172.21.15.114:0/3951993350 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e840441c90 con 0x55e8400ba250 2024-09-21T07:06:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.082+0000 7f26c3400640 10 client.18126 handle_command_reply: tid=0 2024-09-21T07:06:59.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.082+0000 7f26eb2cc740 1 client.18126 shutdown 2024-09-21T07:06:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.083+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 msgr2=0x55e8404dc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.083+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404dc190 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f26d00cd7f0 tx=0x7f26d01055b0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 shutdown_connections 2024-09-21T07:06:59.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e840126520 0x55e8405556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55e8400ba250 0x55e840558e20 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e840105840 0x55e8404dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 --2- 172.21.15.114:0/3951993350 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e8400fd660 0x55e8404dc190 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 >> 172.21.15.114:0/3951993350 conn(0x55e84011a910 msgr2=0x55e8404da130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26c3400640 0 client.18126 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 shutdown_connections 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.084+0000 7f26eb2cc740 1 -- 172.21.15.114:0/3951993350 wait complete. 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.085+0000 7f26eb2cc740 20 client.18126 trim_cache size 0 max 16384 2024-09-21T07:06:59.087 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 52, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 48, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "faeaa3be": { 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 48, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 137.8, 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:06:59.088 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:06:59.089 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:06:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.087+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f26c40b6680 msgr2=0x7f26c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.087+0000 7f26e3000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f26c40b6680 0x7f26c40b8b90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f26d40049d0 tx=0x7f26d4009500 comp rx=0 tx=0).stop 2024-09-21T07:06:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.087+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1039b0 msgr2=0x7f26dc19c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:06:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.087+0000 7f26e3000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1039b0 0x7f26dc19c550 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f26d002a930 tx=0x7f26d0031a20 comp rx=0 tx=0).stop 2024-09-21T07:06:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 shutdown_connections 2024-09-21T07:06:59.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f26dc109c40 0x7f26dc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26dc1092f0 0x7f26dc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f26c40b6680 0x7f26c40b8b90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 --2- 172.21.15.114:0/3919656168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26dc1039b0 0x7f26dc19c550 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:06:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 >> 172.21.15.114:0/3919656168 conn(0x7f26dc0ff440 msgr2=0x7f26dc108a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:06:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 shutdown_connections 2024-09-21T07:06:59.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:06:59.088+0000 7f26e3000640 1 -- 172.21.15.114:0/3919656168 wait complete. 2024-09-21T07:06:59.137 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:06:59 smithi114 ceph-mon[30654]: pgmap v791: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-09-21T07:06:59.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:06:59 smithi183 ceph-mon[34219]: pgmap v791: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-09-21T07:06:59.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:06:59 smithi193 ceph-mon[34601]: pgmap v791: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-09-21T07:07:00.123 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'faeaa3be', total seconds elapsed: 23.3 2024-09-21T07:07:00.123 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.8 2024-09-21T07:07:00.124 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:07:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.253+0000 7f05c0600640 1 -- 172.21.15.114:0/2692021122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b810dd60 msgr2=0x7f05b8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.253+0000 7f05c0600640 1 --2- 172.21.15.114:0/2692021122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b810dd60 0x7f05b8110250 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009ca0 tx=0x7f05b002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.254+0000 7f05c0600640 1 -- 172.21.15.114:0/2692021122 shutdown_connections 2024-09-21T07:07:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.254+0000 7f05c0600640 1 --2- 172.21.15.114:0/2692021122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b810dd60 0x7f05b8110250 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.254+0000 7f05c0600640 1 --2- 172.21.15.114:0/2692021122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b8105250 0x7f05b810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.254+0000 7f05c0600640 1 --2- 172.21.15.114:0/2692021122 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b8104900 0x7f05b8104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.254+0000 7f05c0600640 1 -- 172.21.15.114:0/2692021122 >> 172.21.15.114:0/2692021122 conn(0x7f05b807a740 msgr2=0x7f05b807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.257+0000 7f05c0600640 1 -- 172.21.15.114:0/2692021122 shutdown_connections 2024-09-21T07:07:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.257+0000 7f05c0600640 1 -- 172.21.15.114:0/2692021122 wait complete. 2024-09-21T07:07:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.258+0000 7f05c0600640 1 Processor -- start 2024-09-21T07:07:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 -- start start 2024-09-21T07:07:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 0x7f05b819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b8105250 0x7f05b819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b810dd60 0x7f05b81a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8115c60 con 0x7f05b8105250 2024-09-21T07:07:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8115ae0 con 0x7f05b810dd60 2024-09-21T07:07:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.259+0000 7f05c0600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8115de0 con 0x7f05b8104900 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.260+0000 7f05bdc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 0x7f05b819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.260+0000 7f05be600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b810dd60 0x7f05b81a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.260+0000 7f05be600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b810dd60 0x7f05b81a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:32874/0 (socket says 172.21.15.114:32874) 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.260+0000 7f05bdc00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 0x7f05b819c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57072/0 (socket says 172.21.15.114:57072) 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.260+0000 7f05be600640 1 -- 172.21.15.114:0/4247355197 learned_addr learned my addr 172.21.15.114:0/4247355197 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.260+0000 7f05bd200640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b8105250 0x7f05b819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.261+0000 7f05bdc00640 1 -- 172.21.15.114:0/4247355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b810dd60 msgr2=0x7f05b81a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.261+0000 7f05bdc00640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b810dd60 0x7f05b81a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.261+0000 7f05bdc00640 1 -- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b8105250 msgr2=0x7f05b819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.261+0000 7f05bdc00640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b8105250 0x7f05b819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.261+0000 7f05bdc00640 1 -- 172.21.15.114:0/4247355197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b81a2670 con 0x7f05b8104900 2024-09-21T07:07:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.262+0000 7f05bdc00640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 0x7f05b819c740 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b3f0 tx=0x7f05a400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.263+0000 7f05b6000640 1 -- 172.21.15.114:0/4247355197 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4015020 con 0x7f05b8104900 2024-09-21T07:07:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.263+0000 7f05b6000640 1 -- 172.21.15.114:0/4247355197 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a4011ce0 con 0x7f05b8104900 2024-09-21T07:07:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.263+0000 7f05b6000640 1 -- 172.21.15.114:0/4247355197 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4010d80 con 0x7f05b8104900 2024-09-21T07:07:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.264+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b81a2960 con 0x7f05b8104900 2024-09-21T07:07:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.264+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a2d70 con 0x7f05b8104900 2024-09-21T07:07:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.273+0000 7f05c7695740 1 Processor -- start 2024-09-21T07:07:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.273+0000 7f05c7695740 1 -- start start 2024-09-21T07:07:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.274+0000 7f05c7695740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 0x558833b32c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.274+0000 7f05c7695740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.274+0000 7f05c7695740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833b336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.274+0000 7f05c7695740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558833aa86b0 con 0x5588337652f0 2024-09-21T07:07:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.274+0000 7f05c7695740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558833a97ae0 con 0x55883376d4d0 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.274+0000 7f05c7695740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558833a9b8f0 con 0x55883378e1b0 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 0x558833b32c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 0x558833b32c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57076/0 (socket says 172.21.15.114:57076) 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 -- 172.21.15.114:0/3652209621 learned_addr learned my addr 172.21.15.114:0/3652209621 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05bd200640 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833b336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 -- 172.21.15.114:0/3652209621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 msgr2=0x558833b336b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833b336b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 -- 172.21.15.114:0/3652209621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 msgr2=0x558833b33170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:07:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b33170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.275+0000 7f05be600640 1 -- 172.21.15.114:0/3652209621 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558833a64780 con 0x55883378e1b0 2024-09-21T07:07:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.276+0000 7f05b6000640 1 -- 172.21.15.114:0/4247355197 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f05a4018020 con 0x7f05b8104900 2024-09-21T07:07:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.277+0000 7f05be600640 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 0x558833b32c30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b00094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ff42b2dcec9ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.278+0000 7f05aaa00640 1 -- 172.21.15.114:0/3652209621 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0031450 con 0x55883378e1b0 2024-09-21T07:07:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.278+0000 7f05aaa00640 1 -- 172.21.15.114:0/3652209621 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b0036040 con 0x55883378e1b0 2024-09-21T07:07:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.278+0000 7f05aaa00640 1 -- 172.21.15.114:0/3652209621 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0004480 con 0x55883378e1b0 2024-09-21T07:07:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05b6000640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05a00b6700 0x7f05a00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 -- 172.21.15.114:0/3652209621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 msgr2=0x558833b32c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 0x558833b32c30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009990 tx=0x7f05b00094a0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05b6000640 1 -- 172.21.15.114:0/4247355197 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f05a40e70a0 con 0x7f05b8104900 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 -- 172.21.15.114:0/3652209621 shutdown_connections 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833b336b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 --2- 172.21.15.114:0/3652209621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883378e1b0 0x558833b32c30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.279+0000 7f05c7695740 1 -- 172.21.15.114:0/3652209621 >> 172.21.15.114:0/3652209621 conn(0x558833782300 msgr2=0x558833b30b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.284+0000 7f05bdc00640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05a00b6700 0x7f05a00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.284+0000 7f05c7695740 1 -- 172.21.15.114:0/3652209621 shutdown_connections 2024-09-21T07:07:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.284+0000 7f05c7695740 1 -- 172.21.15.114:0/3652209621 wait complete. 2024-09-21T07:07:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.285+0000 7f05bdc00640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05a00b6700 0x7f05a00b8c10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x558833b318b0 tx=0x7f05a4000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.285+0000 7f05c7695740 1 -- start start 2024-09-21T07:07:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.285+0000 7f05c7695740 1 Processor -- start 2024-09-21T07:07:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.289+0000 7f05c7695740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b3e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.290+0000 7f05be600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b3e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.290+0000 7f05be600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b3e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34416/0 (socket says 172.21.15.114:34416) 2024-09-21T07:07:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.290+0000 7f05be600640 1 -- 172.21.15.114:0/1859448671 learned_addr learned my addr 172.21.15.114:0/1859448671 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.290+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.291+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883378e1b0 0x558833b3ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.291+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5588339cfab0 con 0x5588337652f0 2024-09-21T07:07:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.291+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558833a3e570 con 0x55883378e1b0 2024-09-21T07:07:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.291+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558833b411d0 con 0x55883376d4d0 2024-09-21T07:07:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.291+0000 7f05bd200640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.292+0000 7f05be600640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883378e1b0 0x558833b3ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.293+0000 7f05c7695740 10 client.0 _ll_register_callbacks cb 0x558833a4d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.293+0000 7f05be600640 1 -- 172.21.15.114:0/1859448671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 msgr2=0x558833b3e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.294+0000 7f05be600640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.294+0000 7f05be600640 1 -- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883378e1b0 msgr2=0x558833b3ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.294+0000 7f05be600640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883378e1b0 0x558833b3ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.294+0000 7f05be600640 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588337f9790 con 0x5588337652f0 2024-09-21T07:07:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.294+0000 7f05bd200640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.295+0000 7f05be600640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b3e1f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f05b0003f20 tx=0x7f05b00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7443e0f658138b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.296+0000 7f059f400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.296+0000 7f059f400640 1 -- 172.21.15.114:0/1859448671 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0058060 con 0x5588337652f0 2024-09-21T07:07:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.296+0000 7f059f400640 1 -- 172.21.15.114:0/1859448671 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f05b00043d0 con 0x5588337652f0 2024-09-21T07:07:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.296+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b0036040 con 0x5588337652f0 2024-09-21T07:07:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.297+0000 7f059f400640 1 -- 172.21.15.114:0/1859448671 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f05b005f050 con 0x5588337652f0 2024-09-21T07:07:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.297+0000 7f05c7695740 10 client.18129 fetch_fsmap learned FSMap version 52 2024-09-21T07:07:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.297+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558833872620 con 0x5588337652f0 2024-09-21T07:07:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.297+0000 7f059f400640 1 -- 172.21.15.114:0/1859448671 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b0036040 con 0x5588337652f0 2024-09-21T07:07:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.297+0000 7f059f400640 1 -- 172.21.15.114:0/1859448671 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f05b0051040 con 0x5588337652f0 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05c7695740 10 client.18129 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05c7695740 10 client.18129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05c7695740 20 client.18129 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883374cd80 0x558833b4ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05c7695740 4 client.18129 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558833aaa600 con 0x55883374cd80 2024-09-21T07:07:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.298+0000 7f05bd200640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883374cd80 0x558833b4ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.299+0000 7f05bd200640 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883374cd80 0x558833b4ff80 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e2058e902fee07 server_cookie=38af5ff7a0ce6322 in_seq=0 out_seq=0 2024-09-21T07:07:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.299+0000 7f059f400640 10 client.18129 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.300+0000 7f059f400640 1 -- 172.21.15.114:0/1859448671 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558833aaa600 con 0x55883374cd80 2024-09-21T07:07:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.300+0000 7f059f400640 10 client.18129 handle_command_reply: tid=0 2024-09-21T07:07:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.302+0000 7f05c7695740 1 client.18129 shutdown 2024-09-21T07:07:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.307+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 msgr2=0x558833b3e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.307+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b3e1f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f05b0003f20 tx=0x7f05b00040e0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.307+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 shutdown_connections 2024-09-21T07:07:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.307+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883378e1b0 0x558833b3ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.307+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883374cd80 0x558833b4ff80 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f059f400640 0 client.18129 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:00.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f05c7695740 1 --2- 172.21.15.114:0/1859448671 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5588337652f0 0x558833b3e1f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 >> 172.21.15.114:0/1859448671 conn(0x558833782300 msgr2=0x558833b33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 shutdown_connections 2024-09-21T07:07:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f05c7695740 1 -- 172.21.15.114:0/1859448671 wait complete. 2024-09-21T07:07:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.308+0000 7f05c7695740 20 client.18129 trim_cache size 0 max 16384 2024-09-21T07:07:00.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.351+0000 7f05c7695740 1 Processor -- start 2024-09-21T07:07:00.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.354+0000 7f05c7695740 1 -- start start 2024-09-21T07:07:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.354+0000 7f05c7695740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b3fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.354+0000 7f05c7695740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588337652f0 0x558833b3b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.354+0000 7f05c7695740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.355+0000 7f05c7695740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b7670 con 0x55883374cd80 2024-09-21T07:07:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.355+0000 7f05c7695740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b9210 con 0x5588337652f0 2024-09-21T07:07:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.355+0000 7f05c7695740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b7d40 con 0x55883376d4d0 2024-09-21T07:07:00.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.356+0000 7f05be600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b3fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b3fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:34420/0 (socket says 172.21.15.114:34420) 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 -- 172.21.15.114:0/3436034219 learned_addr learned my addr 172.21.15.114:0/3436034219 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 -- 172.21.15.114:0/3436034219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 msgr2=0x558833b3b900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 -- 172.21.15.114:0/3436034219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588337652f0 msgr2=0x558833b3b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588337652f0 0x558833b3b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.359+0000 7f05be600640 1 -- 172.21.15.114:0/3436034219 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558833b351d0 con 0x55883374cd80 2024-09-21T07:07:00.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.360+0000 7f05be600640 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b3fe20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f05b0000f80 tx=0x7f05b0031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff38c097d1218eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.364+0000 7f05aa000640 1 -- 172.21.15.114:0/3436034219 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0058040 con 0x55883374cd80 2024-09-21T07:07:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.366+0000 7f05aa000640 1 -- 172.21.15.114:0/3436034219 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b0036040 con 0x55883374cd80 2024-09-21T07:07:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.366+0000 7f05aa000640 1 -- 172.21.15.114:0/3436034219 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b00318e0 con 0x55883374cd80 2024-09-21T07:07:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.367+0000 7f05c7695740 1 -- 172.21.15.114:0/3436034219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 msgr2=0x558833b3fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.367+0000 7f05c7695740 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b3fe20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f05b0000f80 tx=0x7f05b0031450 comp rx=0 tx=0).stop 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 -- 172.21.15.114:0/3436034219 shutdown_connections 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55883376d4d0 0x558833b3b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588337652f0 0x558833b3b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 --2- 172.21.15.114:0/3436034219 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b3fe20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 -- 172.21.15.114:0/3436034219 >> 172.21.15.114:0/3436034219 conn(0x558833782300 msgr2=0x558833b3f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 -- 172.21.15.114:0/3436034219 shutdown_connections 2024-09-21T07:07:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.368+0000 7f05c7695740 1 -- 172.21.15.114:0/3436034219 wait complete. 2024-09-21T07:07:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.369+0000 7f05c7695740 1 -- start start 2024-09-21T07:07:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.369+0000 7f05c7695740 1 Processor -- start 2024-09-21T07:07:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b52790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 0x558833b52cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833bb26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b8ae0 con 0x55883374cd80 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b8410 con 0x55883376d4d0 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81b5690 con 0x5588337652f0 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05c7695740 10 client.0 _ll_register_callbacks cb 0x7f05b0031590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.370+0000 7f05bd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 0x558833b52cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 0x558833b52cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57094/0 (socket says 172.21.15.114:57094) 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 -- 172.21.15.114:0/3932938292 learned_addr learned my addr 172.21.15.114:0/3932938292 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 -- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 msgr2=0x558833bb26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833bb26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 -- 172.21.15.114:0/3932938292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 msgr2=0x558833b52790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b52790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.371+0000 7f05bd200640 1 -- 172.21.15.114:0/3932938292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558833bb2cc0 con 0x5588337652f0 2024-09-21T07:07:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.372+0000 7f05bd200640 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 0x558833b52cd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f05b806c730 tx=0x7f05ac013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c004175b994b9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.372+0000 7f059f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.373+0000 7f059f400640 1 -- 172.21.15.114:0/3932938292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac005020 con 0x5588337652f0 2024-09-21T07:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.373+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b0036040 con 0x5588337652f0 2024-09-21T07:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.373+0000 7f059f400640 1 -- 172.21.15.114:0/3932938292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f05ac0051c0 con 0x5588337652f0 2024-09-21T07:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.373+0000 7f059f400640 1 -- 172.21.15.114:0/3932938292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f05ac02b050 con 0x5588337652f0 2024-09-21T07:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.373+0000 7f05c7695740 10 client.28672 fetch_fsmap learned FSMap version 52 2024-09-21T07:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.373+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x558833bb4030 con 0x5588337652f0 2024-09-21T07:07:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.374+0000 7f059f400640 1 -- 172.21.15.114:0/3932938292 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b0036040 con 0x5588337652f0 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.375+0000 7f059f400640 1 -- 172.21.15.114:0/3932938292 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 52) ==== 8382+0+0 (secure 0 0 0) 0x7f05ac021030 con 0x5588337652f0 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.375+0000 7f05c7695740 10 client.28672 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.376+0000 7f05c7695740 10 client.28672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.376+0000 7f05c7695740 20 client.28672 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.376+0000 7f05c7695740 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883378e1b0 0x558833bcd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.376+0000 7f05c7695740 4 client.28672 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:07:00.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.376+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558833aa60b0 con 0x55883378e1b0 2024-09-21T07:07:00.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.376+0000 7f05be600640 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883378e1b0 0x558833bcd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.379+0000 7f05be600640 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883378e1b0 0x558833bcd920 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a246e44f2f6f47a server_cookie=c6f7854cacc60a31 in_seq=0 out_seq=0 2024-09-21T07:07:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.379+0000 7f059f400640 10 client.28672 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.504+0000 7f059f400640 1 -- 172.21.15.114:0/3932938292 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558833aa60b0 con 0x55883378e1b0 2024-09-21T07:07:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.504+0000 7f059f400640 10 client.28672 handle_command_reply: tid=0 2024-09-21T07:07:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.504+0000 7f05c7695740 1 client.28672 shutdown 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 msgr2=0x558833b52cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 0x558833b52cd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f05b806c730 tx=0x7f05ac013830 comp rx=0 tx=0).stop 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 shutdown_connections 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55883376d4d0 0x558833bb26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55883378e1b0 0x558833bcd920 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5588337652f0 0x558833b52cd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 --2- 172.21.15.114:0/3932938292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55883374cd80 0x558833b52790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 >> 172.21.15.114:0/3932938292 conn(0x558833782300 msgr2=0x558833b2f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:00.509 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.507+0000 7f059f400640 0 client.28672 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.509+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 shutdown_connections 2024-09-21T07:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.509+0000 7f05c7695740 1 -- 172.21.15.114:0/3932938292 wait complete. 2024-09-21T07:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.509+0000 7f05c7695740 20 client.28672 trim_cache size 0 max 16384 2024-09-21T07:07:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:07:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:07:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:07:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:07:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05a00b6700 msgr2=0x7f05a00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05a00b6700 0x7f05a00b8c10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x558833b318b0 tx=0x7f05a4000bc0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 msgr2=0x7f05b819c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 0x7f05b819c740 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f05a400b3f0 tx=0x7f05a400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 shutdown_connections 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f05a00b6700 0x7f05a00b8c10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05b810dd60 0x7f05b81a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f05b8105250 0x7f05b819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 --2- 172.21.15.114:0/4247355197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05b8104900 0x7f05b819c740 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 >> 172.21.15.114:0/4247355197 conn(0x7f05b807a740 msgr2=0x7f05b80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 shutdown_connections 2024-09-21T07:07:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:00.513+0000 7f05c0600640 1 -- 172.21.15.114:0/4247355197 wait complete. 2024-09-21T07:07:00.901 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:01.341 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:01 smithi114 ceph-mon[30654]: pgmap v792: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 465 KiB/s wr, 161 op/s 2024-09-21T07:07:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:01 smithi183 ceph-mon[34219]: pgmap v792: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 465 KiB/s wr, 161 op/s 2024-09-21T07:07:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:01 smithi193 ceph-mon[34601]: pgmap v792: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 465 KiB/s wr, 161 op/s 2024-09-21T07:07:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 745.0 s, 254.8 tps, lat 1171.426 ms stddev 3219.590 2024-09-21T07:07:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:03 smithi193 ceph-mon[34601]: pgmap v793: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 411 KiB/s wr, 153 op/s 2024-09-21T07:07:03.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:03 smithi114 ceph-mon[30654]: pgmap v793: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 411 KiB/s wr, 153 op/s 2024-09-21T07:07:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:03 smithi183 ceph-mon[34219]: pgmap v793: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 411 KiB/s wr, 153 op/s 2024-09-21T07:07:04.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.183+0000 7f12ba000640 1 -- 172.21.15.114:0/4104791196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 msgr2=0x7f12ac0a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:04.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.183+0000 7f12ba000640 1 --2- 172.21.15.114:0/4104791196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 0x7f12ac0a4ac0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f12b4066480 tx=0x7f12b40981c0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.185 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.184+0000 7f12ba000640 1 -- 172.21.15.114:0/4104791196 shutdown_connections 2024-09-21T07:07:04.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.184+0000 7f12ba000640 1 --2- 172.21.15.114:0/4104791196 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 0x7f12ac0a4ac0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.184+0000 7f12ba000640 1 --2- 172.21.15.114:0/4104791196 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac094aa0 0x7f12ac0a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.184+0000 7f12ba000640 1 --2- 172.21.15.114:0/4104791196 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0944d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.186 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.184+0000 7f12ba000640 1 -- 172.21.15.114:0/4104791196 >> 172.21.15.114:0/4104791196 conn(0x7f12ac00c060 msgr2=0x7f12ac00c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.188+0000 7f12ba000640 1 -- 172.21.15.114:0/4104791196 shutdown_connections 2024-09-21T07:07:04.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.188+0000 7f12ba000640 1 -- 172.21.15.114:0/4104791196 wait complete. 2024-09-21T07:07:04.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.189+0000 7f12ba000640 1 Processor -- start 2024-09-21T07:07:04.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 -- start start 2024-09-21T07:07:04.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:04.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac094aa0 0x7f12ac0a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:04.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 0x7f12ac13eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:04.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac098750 con 0x7f12ac0a25b0 2024-09-21T07:07:04.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac0985d0 con 0x7f12ac0940c0 2024-09-21T07:07:04.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.190+0000 7f12ba000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac0988d0 con 0x7f12ac094aa0 2024-09-21T07:07:04.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.191+0000 7f12b3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac094aa0 0x7f12ac0a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:04.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.191+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:04.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.191+0000 7f12b8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:32938/0 (socket says 172.21.15.114:32938) 2024-09-21T07:07:04.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.191+0000 7f12b8c00640 1 -- 172.21.15.114:0/4062794483 learned_addr learned my addr 172.21.15.114:0/4062794483 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:04.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.191+0000 7f12b9600640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 0x7f12ac13eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:04.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.192+0000 7f12b8c00640 1 -- 172.21.15.114:0/4062794483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac094aa0 msgr2=0x7f12ac0a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:04.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.192+0000 7f12b8c00640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac094aa0 0x7f12ac0a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.192+0000 7f12b8c00640 1 -- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 msgr2=0x7f12ac13eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:04.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.192+0000 7f12b8c00640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 0x7f12ac13eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.192+0000 7f12b8c00640 1 -- 172.21.15.114:0/4062794483 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac13f210 con 0x7f12ac0940c0 2024-09-21T07:07:04.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.195+0000 7f12b8c00640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0a0b40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f12a800b450 tx=0x7f12a800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:04.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.195+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8015020 con 0x7f12ac0940c0 2024-09-21T07:07:04.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.195+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac13f500 con 0x7f12ac0940c0 2024-09-21T07:07:04.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.196+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac13fa90 con 0x7f12ac0940c0 2024-09-21T07:07:04.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.196+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8011d40 con 0x7f12ac0940c0 2024-09-21T07:07:04.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.196+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f340 con 0x7f12ac0940c0 2024-09-21T07:07:04.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.199+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12940052c0 con 0x7f12ac0940c0 2024-09-21T07:07:04.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.205+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f12a8018020 con 0x7f12ac0940c0 2024-09-21T07:07:04.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.208+0000 7f12b1600640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f129c0b6880 0x7f129c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:04.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.209+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f12a8010c50 con 0x7f12ac0940c0 2024-09-21T07:07:04.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.210+0000 7f12b3e00640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f129c0b6880 0x7f129c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:04.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.211+0000 7f12b3e00640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f129c0b6880 0x7f129c0b8d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0055a0 tx=0x7f12a0009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:04.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.217+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12a801f020 con 0x7f12ac0940c0 2024-09-21T07:07:04.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:04 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:07:04.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:04 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:07:04.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:04 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:07:04.623 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.622+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1294005880 con 0x7f12ac0940c0 2024-09-21T07:07:04.625 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:07:04.625 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":53,"btime":"2024-09-21T07:07:03:112008+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:07:02.935964+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:07:04.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.623+0000 7f12b1600640 1 -- 172.21.15.114:0/4062794483 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+11994 (secure 0 0 0) 0x7f12a800d040 con 0x7f12ac0940c0 2024-09-21T07:07:04.625 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 53 2024-09-21T07:07:04.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f129c0b6880 msgr2=0x7f129c0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:04.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f129c0b6880 0x7f129c0b8d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f12ac0055a0 tx=0x7f12a0009110 comp rx=0 tx=0).stop 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 msgr2=0x7f12ac0a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0a0b40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f12a800b450 tx=0x7f12a800b910 comp rx=0 tx=0).stop 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 shutdown_connections 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12ac0a25b0 0x7f12ac13eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac094aa0 0x7f12ac0a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f129c0b6880 0x7f129c0b8d90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 --2- 172.21.15.114:0/4062794483 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac0940c0 0x7f12ac0a0b40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 >> 172.21.15.114:0/4062794483 conn(0x7f12ac00c060 msgr2=0x7f12ac0a2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 shutdown_connections 2024-09-21T07:07:04.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:04.626+0000 7f12ba000640 1 -- 172.21.15.114:0/4062794483 wait complete. 2024-09-21T07:07:05.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:05 smithi114 ceph-mon[30654]: pgmap v794: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.2 MiB/s wr, 217 op/s 2024-09-21T07:07:05.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:05 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/4062794483' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:05 smithi193 ceph-mon[34601]: pgmap v794: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.2 MiB/s wr, 217 op/s 2024-09-21T07:07:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:05 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/4062794483' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:05 smithi183 ceph-mon[34219]: pgmap v794: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 1.2 MiB/s wr, 217 op/s 2024-09-21T07:07:05.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:05 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/4062794483' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:06.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:06 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:07:06.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:06 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:07:06.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:06 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:07:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:06 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:07:06.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:06 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:07:06.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:06 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:07:06.670 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0' 2024-09-21T07:07:06.671 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 137.8 --await-for 45.0 2024-09-21T07:07:07.308 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:07.504 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:07 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:07:07.504 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:07 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:07:07.504 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:07 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:07:07.505 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:07 smithi114 ceph-mon[30654]: pgmap v795: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-09-21T07:07:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:07 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:07:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:07 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:07:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:07 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:07:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:07 smithi183 ceph-mon[34219]: pgmap v795: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-09-21T07:07:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:07 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:07:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:07 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:07:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:07 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:07:07.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:07 smithi193 ceph-mon[34601]: pgmap v795: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.8 MiB/s wr, 291 op/s 2024-09-21T07:07:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 750.0 s, 444.4 tps, lat 254.322 ms stddev 348.793 2024-09-21T07:07:09.305 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:09 smithi114 ceph-mon[30654]: pgmap v796: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.7 MiB/s wr, 342 op/s 2024-09-21T07:07:09.306 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:09 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:07:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:09 smithi183 ceph-mon[34219]: pgmap v796: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.7 MiB/s wr, 342 op/s 2024-09-21T07:07:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:09 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:07:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:09 smithi193 ceph-mon[34601]: pgmap v796: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 2.7 MiB/s wr, 342 op/s 2024-09-21T07:07:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:09 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 -- 172.21.15.114:0/2307153854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3024076100 msgr2=0x7f302410fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 --2- 172.21.15.114:0/2307153854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3024076100 0x7f302410fba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3010009900 tx=0x7f301002a410 comp rx=0 tx=0).stop 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 -- 172.21.15.114:0/2307153854 shutdown_connections 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 --2- 172.21.15.114:0/2307153854 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30241100e0 0x7f3024112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 --2- 172.21.15.114:0/2307153854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3024076100 0x7f302410fba0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 --2- 172.21.15.114:0/2307153854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30240757b0 0x7f3024075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.978+0000 7f3029200640 1 -- 172.21.15.114:0/2307153854 >> 172.21.15.114:0/2307153854 conn(0x7f30240ff5d0 msgr2=0x7f3024101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:09.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.979+0000 7f3029200640 1 -- 172.21.15.114:0/2307153854 shutdown_connections 2024-09-21T07:07:09.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.979+0000 7f3029200640 1 -- 172.21.15.114:0/2307153854 wait complete. 2024-09-21T07:07:09.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.980+0000 7f3029200640 1 Processor -- start 2024-09-21T07:07:09.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 -- start start 2024-09-21T07:07:09.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240757b0 0x7f302419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30241100e0 0x7f302419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 0x7f30241a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3024115ce0 con 0x7f30241100e0 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3024115b60 con 0x7f30240757b0 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3029200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3024115e60 con 0x7f30241a1f50 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3022800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 0x7f30241a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3022800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 0x7f30241a2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59202/0 (socket says 172.21.15.114:59202) 2024-09-21T07:07:09.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3022800640 1 -- 172.21.15.114:0/1966977763 learned_addr learned my addr 172.21.15.114:0/1966977763 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.981+0000 7f3021400640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30241100e0 0x7f302419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3022800640 1 -- 172.21.15.114:0/1966977763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240757b0 msgr2=0x7f302419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3022800640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240757b0 0x7f302419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3022800640 1 -- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30241100e0 msgr2=0x7f302419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:09.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3022800640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30241100e0 0x7f302419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3022800640 1 -- 172.21.15.114:0/1966977763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30241a2b30 con 0x7f30241a1f50 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3022800640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 0x7f30241a2360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f301c00eb10 tx=0x7f301c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f301aa00640 1 -- 172.21.15.114:0/1966977763 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c004b20 con 0x7f30241a1f50 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30241a90d0 con 0x7f30241a1f50 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f301aa00640 1 -- 172.21.15.114:0/1966977763 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301c004cc0 con 0x7f30241a1f50 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.982+0000 7f301aa00640 1 -- 172.21.15.114:0/1966977763 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c010670 con 0x7f30241a1f50 2024-09-21T07:07:09.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.983+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241a94c0 con 0x7f30241a1f50 2024-09-21T07:07:09.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.990+0000 7f301aa00640 1 -- 172.21.15.114:0/1966977763 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f301c005020 con 0x7f30241a1f50 2024-09-21T07:07:09.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.991+0000 7f301aa00640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30080b6680 0x7f30080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:09.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.992+0000 7f301aa00640 1 -- 172.21.15.114:0/1966977763 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f301c0e8230 con 0x7f30241a1f50 2024-09-21T07:07:09.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.992+0000 7f3021e00640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30080b6680 0x7f30080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:09.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:09.993+0000 7f3021e00640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30080b6680 0x7f30080b8b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3014005d00 tx=0x7f3014005c50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:10.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.011+0000 7f302ffbc740 1 Processor -- start 2024-09-21T07:07:10.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 -- start start 2024-09-21T07:07:10.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x559462580350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x559462580890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x559462580dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594624f5ca0 con 0x5594621b7b60 2024-09-21T07:07:10.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5594624e50d0 con 0x5594621d8840 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.012+0000 7f302ffbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594624e8ee0 con 0x5594621af980 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.013+0000 7f3021e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x559462580350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.013+0000 7f3022800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x559462580dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.013+0000 7f3022800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x559462580dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44630/0 (socket says 172.21.15.114:44630) 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.013+0000 7f3022800640 1 -- 172.21.15.114:0/4274790597 learned_addr learned my addr 172.21.15.114:0/4274790597 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.013+0000 7f3021400640 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x559462580890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3022800640 1 -- 172.21.15.114:0/4274790597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 msgr2=0x559462580890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3022800640 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x559462580890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3022800640 1 -- 172.21.15.114:0/4274790597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 msgr2=0x559462580350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3022800640 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x559462580350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3022800640 1 -- 172.21.15.114:0/4274790597 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559462497400 con 0x5594621b7b60 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3021e00640 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x559462580350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f3022800640 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x559462580dd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f301c0aa220 tx=0x7f301c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05204726fffa93e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:10.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.014+0000 7f300f400640 1 -- 172.21.15.114:0/4274790597 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0950b0 con 0x5594621b7b60 2024-09-21T07:07:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.015+0000 7f300f400640 1 -- 172.21.15.114:0/4274790597 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301c0ad760 con 0x5594621b7b60 2024-09-21T07:07:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.015+0000 7f300f400640 1 -- 172.21.15.114:0/4274790597 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0d10e0 con 0x5594621b7b60 2024-09-21T07:07:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 -- 172.21.15.114:0/4274790597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 msgr2=0x559462580dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x559462580dd0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f301c0aa220 tx=0x7f301c00cf70 comp rx=0 tx=0).stop 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 -- 172.21.15.114:0/4274790597 shutdown_connections 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x559462580dd0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x559462580890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 --2- 172.21.15.114:0/4274790597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x559462580350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 -- 172.21.15.114:0/4274790597 >> 172.21.15.114:0/4274790597 conn(0x5594621cc990 msgr2=0x55946257e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 -- 172.21.15.114:0/4274790597 shutdown_connections 2024-09-21T07:07:10.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.016+0000 7f302ffbc740 1 -- 172.21.15.114:0/4274790597 wait complete. 2024-09-21T07:07:10.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.017+0000 7f302ffbc740 1 -- start start 2024-09-21T07:07:10.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.017+0000 7f302ffbc740 1 Processor -- start 2024-09-21T07:07:10.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.018+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x5594625a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.018+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x5594625a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.020+0000 7f3021e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x5594625a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.020+0000 7f3021e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x5594625a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59214/0 (socket says 172.21.15.114:59214) 2024-09-21T07:07:10.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.021+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x5594625ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.021+0000 7f302ffbc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594623467a0 con 0x5594621b7b60 2024-09-21T07:07:10.023 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.021+0000 7f302ffbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55946248aa10 con 0x5594621d8840 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.021+0000 7f302ffbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55946259fae0 con 0x5594621af980 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.021+0000 7f302ffbc740 10 client.0 _ll_register_callbacks cb 0x55946249abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.021+0000 7f3021e00640 1 -- 172.21.15.114:0/321772375 learned_addr learned my addr 172.21.15.114:0/321772375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021400640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x5594625a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3022800640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x5594625ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021e00640 1 -- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 msgr2=0x5594625ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021e00640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x5594625ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021e00640 1 -- 172.21.15.114:0/321772375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 msgr2=0x5594625a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021e00640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x5594625a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021e00640 1 -- 172.21.15.114:0/321772375 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559462252b90 con 0x5594621af980 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3022800640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x5594625ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.022+0000 7f3021400640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x5594625a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.023+0000 7f3021e00640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x5594625a5b90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f301401d7b0 tx=0x7f30140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12fa7f412b8760e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.023+0000 7f3003e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.023+0000 7f3003e00640 1 -- 172.21.15.114:0/321772375 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014010dc0 con 0x5594621af980 2024-09-21T07:07:10.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.023+0000 7f3003e00640 1 -- 172.21.15.114:0/321772375 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f3014005020 con 0x5594621af980 2024-09-21T07:07:10.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.023+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301c0ad760 con 0x5594621af980 2024-09-21T07:07:10.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f3003e00640 1 -- 172.21.15.114:0/321772375 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3014030050 con 0x5594621af980 2024-09-21T07:07:10.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f302ffbc740 10 client.28687 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:10.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5594625b4140 con 0x5594621af980 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f3003e00640 1 -- 172.21.15.114:0/321772375 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f301c0ad760 con 0x5594621af980 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f3003e00640 1 -- 172.21.15.114:0/321772375 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f3014024030 con 0x5594621af980 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f302ffbc740 10 client.28687 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f302ffbc740 10 client.28687 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f302ffbc740 20 client.28687 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.024+0000 7f302ffbc740 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594621971c0 0x5594625b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.025+0000 7f302ffbc740 4 client.28687 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.025+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594624f7bf0 con 0x5594621971c0 2024-09-21T07:07:10.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.025+0000 7f3021400640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594621971c0 0x5594625b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.026+0000 7f3021400640 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594621971c0 0x5594625b77b0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b044fe8a42353589 server_cookie=28fc6aa10387e2e1 in_seq=0 out_seq=0 2024-09-21T07:07:10.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.026+0000 7f3003e00640 10 client.28687 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:10.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.026+0000 7f3003e00640 1 -- 172.21.15.114:0/321772375 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594624f7bf0 con 0x5594621971c0 2024-09-21T07:07:10.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.026+0000 7f3003e00640 10 client.28687 handle_command_reply: tid=0 2024-09-21T07:07:10.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.026+0000 7f302ffbc740 1 client.28687 shutdown 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 msgr2=0x5594625a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x5594625a5b90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f301401d7b0 tx=0x7f30140046c0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 shutdown_connections 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621d8840 0x5594625ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621b7b60 0x5594625a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594621971c0 0x5594625b77b0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 --2- 172.21.15.114:0/321772375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621af980 0x5594625a5b90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f3003e00640 0 client.28687 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:10.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.027+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 >> 172.21.15.114:0/321772375 conn(0x5594621cc990 msgr2=0x55946257c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:10.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.028+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 shutdown_connections 2024-09-21T07:07:10.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.028+0000 7f302ffbc740 1 -- 172.21.15.114:0/321772375 wait complete. 2024-09-21T07:07:10.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.028+0000 7f302ffbc740 20 client.28687 trim_cache size 0 max 16384 2024-09-21T07:07:10.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 Processor -- start 2024-09-21T07:07:10.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 -- start start 2024-09-21T07:07:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621af980 0x55946257e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621b7b60 0x5594625b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621d8840 0x5594625a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3024072a60 con 0x5594621d8840 2024-09-21T07:07:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3024074690 con 0x5594621af980 2024-09-21T07:07:10.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.046+0000 7f302ffbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3024073130 con 0x5594621b7b60 2024-09-21T07:07:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621af980 0x55946257e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621af980 0x55946257e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42386/0 (socket says 172.21.15.114:42386) 2024-09-21T07:07:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621b7b60 0x5594625b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021e00640 1 -- 172.21.15.114:0/231282108 learned_addr learned my addr 172.21.15.114:0/231282108 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3022800640 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621d8840 0x5594625a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021400640 1 -- 172.21.15.114:0/231282108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621af980 msgr2=0x55946257e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021400640 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621af980 0x55946257e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021400640 1 -- 172.21.15.114:0/231282108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621d8840 msgr2=0x5594625a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021400640 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621d8840 0x5594625a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3021400640 1 -- 172.21.15.114:0/231282108 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946257e760 con 0x5594621b7b60 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.047+0000 7f3022800640 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621d8840 0x5594625a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.048+0000 7f3021400640 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621b7b60 0x5594625b1f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3024076a40 tx=0x7f3010000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9bc087048b6b07 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.048+0000 7f300ea00640 1 -- 172.21.15.114:0/231282108 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010035040 con 0x5594621b7b60 2024-09-21T07:07:10.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.048+0000 7f300ea00640 1 -- 172.21.15.114:0/231282108 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3010004480 con 0x5594621b7b60 2024-09-21T07:07:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.048+0000 7f300ea00640 1 -- 172.21.15.114:0/231282108 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010004d50 con 0x5594621b7b60 2024-09-21T07:07:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.049+0000 7f302ffbc740 1 -- 172.21.15.114:0/231282108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621b7b60 msgr2=0x5594625b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.049+0000 7f302ffbc740 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621b7b60 0x5594625b1f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f3024076a40 tx=0x7f3010000bc0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.049+0000 7f302ffbc740 1 -- 172.21.15.114:0/231282108 shutdown_connections 2024-09-21T07:07:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.049+0000 7f302ffbc740 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621d8840 0x5594625a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.050+0000 7f302ffbc740 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621b7b60 0x5594625b1f80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.050+0000 7f302ffbc740 1 --2- 172.21.15.114:0/231282108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621af980 0x55946257e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.050+0000 7f302ffbc740 1 -- 172.21.15.114:0/231282108 >> 172.21.15.114:0/231282108 conn(0x5594621cc990 msgr2=0x55946257db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:10.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.050+0000 7f302ffbc740 1 -- 172.21.15.114:0/231282108 shutdown_connections 2024-09-21T07:07:10.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.050+0000 7f302ffbc740 1 -- 172.21.15.114:0/231282108 wait complete. 2024-09-21T07:07:10.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.050+0000 7f302ffbc740 1 -- start start 2024-09-21T07:07:10.052 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.051+0000 7f302ffbc740 1 Processor -- start 2024-09-21T07:07:10.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621af980 0x5594625abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621b7b60 0x5594625ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 0x5594625b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3024073f60 con 0x5594621af980 2024-09-21T07:07:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3024073800 con 0x5594621b7b60 2024-09-21T07:07:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3024070aa0 con 0x5594621d8840 2024-09-21T07:07:10.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.052+0000 7f302ffbc740 10 client.0 _ll_register_callbacks cb 0x7f301401d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.054+0000 7f3022800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 0x5594625b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.054+0000 7f3022800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 0x5594625b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59240/0 (socket says 172.21.15.114:59240) 2024-09-21T07:07:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.054+0000 7f3022800640 1 -- 172.21.15.114:0/380605516 learned_addr learned my addr 172.21.15.114:0/380605516 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.055+0000 7f3021400640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621b7b60 0x5594625ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.055+0000 7f3021e00640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621af980 0x5594625abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.058+0000 7f3022800640 1 -- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621b7b60 msgr2=0x5594625ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.058+0000 7f3022800640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621b7b60 0x5594625ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.058+0000 7f3022800640 1 -- 172.21.15.114:0/380605516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621af980 msgr2=0x5594625abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:10.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.058+0000 7f3022800640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621af980 0x5594625abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.058+0000 7f3022800640 1 -- 172.21.15.114:0/380605516 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594625b1840 con 0x5594621d8840 2024-09-21T07:07:10.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.061+0000 7f3003e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:10.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.060+0000 7f3022800640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 0x5594625b1250 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f301c00cf20 tx=0x7f301c0aa150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87507d51cd674466 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:10.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.064+0000 7f3003e00640 1 -- 172.21.15.114:0/380605516 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0d5070 con 0x5594621d8840 2024-09-21T07:07:10.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.064+0000 7f3003e00640 1 -- 172.21.15.114:0/380605516 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9490+0+0 (secure 0 0 0) 0x7f301c0aceb0 con 0x5594621d8840 2024-09-21T07:07:10.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.065+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3010004480 con 0x5594621d8840 2024-09-21T07:07:10.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.071+0000 7f3003e00640 1 -- 172.21.15.114:0/380605516 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f301c0d5210 con 0x5594621d8840 2024-09-21T07:07:10.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.071+0000 7f302ffbc740 10 client.28693 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:10.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.071+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5594626229c0 con 0x5594621d8840 2024-09-21T07:07:10.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f3003e00640 1 -- 172.21.15.114:0/380605516 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3010004480 con 0x5594621d8840 2024-09-21T07:07:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f3003e00640 1 -- 172.21.15.114:0/380605516 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f301c0a95a0 con 0x5594621d8840 2024-09-21T07:07:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f302ffbc740 10 client.28693 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f302ffbc740 10 client.28693 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:07:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f302ffbc740 20 client.28693 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:10.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f302ffbc740 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55946216c570 0x559462624e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f302ffbc740 4 client.28693 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}] 2024-09-21T07:07:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.073+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.8, "await_for": 45.0}) -- 0x5594624f36a0 con 0x55946216c570 2024-09-21T07:07:10.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.075+0000 7f3021e00640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55946216c570 0x559462624e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.077+0000 7f3003e00640 10 client.28693 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:10.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.077+0000 7f3021e00640 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55946216c570 0x559462624e20 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6780f0f16692ecb9 server_cookie=6e615b81b9f97166 in_seq=0 out_seq=0 2024-09-21T07:07:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f3003e00640 1 -- 172.21.15.114:0/380605516 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5594624f36a0 con 0x55946216c570 2024-09-21T07:07:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f3003e00640 10 client.28693 handle_command_reply: tid=0 2024-09-21T07:07:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 client.28693 shutdown 2024-09-21T07:07:10.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 msgr2=0x5594625b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 0x5594625b1250 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f301c00cf20 tx=0x7f301c0aa150 comp rx=0 tx=0).stop 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 shutdown_connections 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55946216c570 0x559462624e20 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594621d8840 0x5594625b1250 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594621b7b60 0x5594625ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f3003e00640 0 client.28693 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 --2- 172.21.15.114:0/380605516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594621af980 0x5594625abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 >> 172.21.15.114:0/380605516 conn(0x5594621cc990 msgr2=0x5594625a9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 shutdown_connections 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.448+0000 7f302ffbc740 1 -- 172.21.15.114:0/380605516 wait complete. 2024-09-21T07:07:10.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f302ffbc740 20 client.28693 trim_cache size 0 max 16384 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 54, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 50, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "2feeda4": { 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 50, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 137.8, 2024-09-21T07:07:10.452 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:10.453 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30080b6680 msgr2=0x7f30080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30080b6680 0x7f30080b8b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3014005d00 tx=0x7f3014005c50 comp rx=0 tx=0).stop 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 msgr2=0x7f30241a2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 0x7f30241a2360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f301c00eb10 tx=0x7f301c00c2d0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 shutdown_connections 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30241a1f50 0x7f30241a2360 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f30241100e0 0x7f302419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f30080b6680 0x7f30080b8b90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.449+0000 7f3029200640 1 --2- 172.21.15.114:0/1966977763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30240757b0 0x7f302419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:10.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.450+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 >> 172.21.15.114:0/1966977763 conn(0x7f30240ff5d0 msgr2=0x7f3024109ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:10.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.450+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 shutdown_connections 2024-09-21T07:07:10.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:10.450+0000 7f3029200640 1 -- 172.21.15.114:0/1966977763 wait complete. 2024-09-21T07:07:11.365 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2feeda4', quiesce took 11.2 seconds. Will release after: 6.600000000000001 2024-09-21T07:07:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:11 smithi114 ceph-mon[30654]: pgmap v797: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-09-21T07:07:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:11 smithi183 ceph-mon[34219]: pgmap v797: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-09-21T07:07:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:11 smithi193 ceph-mon[34601]: pgmap v797: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.0 MiB/s wr, 338 op/s 2024-09-21T07:07:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:12 smithi114 ceph-mon[30654]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T07:07:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:12 smithi183 ceph-mon[34219]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T07:07:12.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:12 smithi193 ceph-mon[34601]: osdmap e100: 12 total, 12 up, 12 in 2024-09-21T07:07:12.835 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 755.0 s, 230.3 tps, lat 206.777 ms stddev 286.803 2024-09-21T07:07:13.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:13 smithi193 ceph-mon[34601]: pgmap v799: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 4.2 MiB/s wr, 440 op/s 2024-09-21T07:07:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:13 smithi114 ceph-mon[30654]: pgmap v799: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 4.2 MiB/s wr, 440 op/s 2024-09-21T07:07:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:13 smithi183 ceph-mon[34219]: pgmap v799: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 4.2 MiB/s wr, 440 op/s 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 -- 172.21.15.114:0/190739625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 msgr2=0x7f7c5810b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/190739625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5810b850 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009900 tx=0x7f7c4402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 -- 172.21.15.114:0/190739625 shutdown_connections 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/190739625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c5810bd90 0x7f7c581125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/190739625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5810b850 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/190739625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c58105b10 0x7f7c58105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 -- 172.21.15.114:0/190739625 >> 172.21.15.114:0/190739625 conn(0x7f7c58101550 msgr2=0x7f7c581039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 -- 172.21.15.114:0/190739625 shutdown_connections 2024-09-21T07:07:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.735+0000 7f7c5ea00640 1 -- 172.21.15.114:0/190739625 wait complete. 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.736+0000 7f7c5ea00640 1 Processor -- start 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.736+0000 7f7c5ea00640 1 -- start start 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.736+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c58105b10 0x7f7c5819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c5810bd90 0x7f7c581a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5ea00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58115aa0 con 0x7f7c5810b440 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58115920 con 0x7f7c58105b10 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58115c20 con 0x7f7c5810bd90 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44690/0 (socket says 172.21.15.114:44690) 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 -- 172.21.15.114:0/483205962 learned_addr learned my addr 172.21.15.114:0/483205962 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5e000640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c5810bd90 0x7f7c581a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.740 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5d600640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c58105b10 0x7f7c5819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 -- 172.21.15.114:0/483205962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c5810bd90 msgr2=0x7f7c581a1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c5810bd90 0x7f7c581a1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 -- 172.21.15.114:0/483205962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c58105b10 msgr2=0x7f7c5819c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c58105b10 0x7f7c5819c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.737+0000 7f7c5cc00640 1 -- 172.21.15.114:0/483205962 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c581a2410 con 0x7f7c5810b440 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.738+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5819ca80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005ab0 tx=0x7f7c44003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:13.741 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.738+0000 7f7c56000640 1 -- 172.21.15.114:0/483205962 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4403f070 con 0x7f7c5810b440 2024-09-21T07:07:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.738+0000 7f7c56000640 1 -- 172.21.15.114:0/483205962 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c44031b50 con 0x7f7c5810b440 2024-09-21T07:07:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.738+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c581a26a0 con 0x7f7c5810b440 2024-09-21T07:07:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.738+0000 7f7c56000640 1 -- 172.21.15.114:0/483205962 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44035030 con 0x7f7c5810b440 2024-09-21T07:07:13.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.738+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c581a8e90 con 0x7f7c5810b440 2024-09-21T07:07:13.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.741+0000 7f7c67206740 1 Processor -- start 2024-09-21T07:07:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.741+0000 7f7c67206740 1 -- start start 2024-09-21T07:07:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c67206740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c56000640 1 -- 172.21.15.114:0/483205962 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7c44042070 con 0x7f7c5810b440 2024-09-21T07:07:13.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c67206740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c67206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d87a4d0 0x559f7dc40890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c67206740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559f7dbb56b0 con 0x559f7d8722f0 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c67206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f7dba4ae0 con 0x559f7d87a4d0 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c67206740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f7dba88f0 con 0x559f7d89b1b0 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc3fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59274/0 (socket says 172.21.15.114:59274) 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d87a4d0 0x559f7dc40890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.745 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 -- 172.21.15.114:0/2410399424 learned_addr learned my addr 172.21.15.114:0/2410399424 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 -- 172.21.15.114:0/2410399424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d87a4d0 msgr2=0x559f7dc40890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d87a4d0 0x559f7dc40890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 -- 172.21.15.114:0/2410399424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 msgr2=0x559f7dc40350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc40350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.742+0000 7f7c5e000640 1 -- 172.21.15.114:0/2410399424 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7db71780 con 0x559f7d89b1b0 2024-09-21T07:07:13.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.743+0000 7f7c5e000640 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc3fe10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7c50011470 tx=0x7f7c50011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd6efd81ce09220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.743+0000 7f7c4a600640 1 -- 172.21.15.114:0/2410399424 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5001e070 con 0x559f7d89b1b0 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.743+0000 7f7c4a600640 1 -- 172.21.15.114:0/2410399424 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c50003c30 con 0x559f7d89b1b0 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.743+0000 7f7c4a600640 1 -- 172.21.15.114:0/2410399424 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000f900 con 0x559f7d89b1b0 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.743+0000 7f7c56000640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c400b6680 0x7f7c400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c5d600640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c400b6680 0x7f7c400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c56000640 1 -- 172.21.15.114:0/483205962 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9578+0+0 (secure 0 0 0) 0x7f7c44108090 con 0x7f7c5810b440 2024-09-21T07:07:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 -- 172.21.15.114:0/2410399424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 msgr2=0x559f7dc3fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc3fe10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7c50011470 tx=0x7f7c50011930 comp rx=0 tx=0).stop 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 -- 172.21.15.114:0/2410399424 shutdown_connections 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d87a4d0 0x559f7dc40890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc40350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 --2- 172.21.15.114:0/2410399424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc3fe10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.744+0000 7f7c67206740 1 -- 172.21.15.114:0/2410399424 >> 172.21.15.114:0/2410399424 conn(0x559f7d88f300 msgr2=0x559f7dc3dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.745+0000 7f7c67206740 1 -- 172.21.15.114:0/2410399424 shutdown_connections 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.745+0000 7f7c5d600640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c400b6680 0x7f7c400b8b90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x559f7dc3ea90 tx=0x7f7c4c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.745+0000 7f7c67206740 1 -- 172.21.15.114:0/2410399424 wait complete. 2024-09-21T07:07:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 -- start start 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 Processor -- start 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 0x559f7dc4b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc4bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559f7dadcab0 con 0x559f7d87a4d0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559f7db4b570 con 0x559f7d8722f0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559f7dc4e590 con 0x559f7d89b1b0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.746+0000 7f7c67206740 10 client.0 _ll_register_callbacks cb 0x559f7db5a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 0x559f7dc4b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 0x559f7dc4b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44704/0 (socket says 172.21.15.114:44704) 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 -- 172.21.15.114:0/2364390786 learned_addr learned my addr 172.21.15.114:0/2364390786 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5e000640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc4bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5d600640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 -- 172.21.15.114:0/2364390786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 msgr2=0x559f7dc4bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc4bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 -- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 msgr2=0x559f7dc4b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc4b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 -- 172.21.15.114:0/2364390786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7d906790 con 0x559f7d87a4d0 2024-09-21T07:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 0x559f7dc4b980 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7c44006010 tx=0x7f7c44047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9b6f01ccc0263 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c3ea00640 1 -- 172.21.15.114:0/2364390786 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44049070 con 0x559f7d87a4d0 2024-09-21T07:07:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c3ea00640 1 -- 172.21.15.114:0/2364390786 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9578+0+0 (secure 0 0 0) 0x7f7c44105640 con 0x559f7d87a4d0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.747+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c50003c30 con 0x559f7d87a4d0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c3ea00640 1 -- 172.21.15.114:0/2364390786 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7c440ca110 con 0x559f7d87a4d0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 10 client.18165 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559f7d97f620 con 0x559f7d87a4d0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c3ea00640 1 -- 172.21.15.114:0/2364390786 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c50003c30 con 0x559f7d87a4d0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c3ea00640 1 -- 172.21.15.114:0/2364390786 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f7c440d21e0 con 0x559f7d87a4d0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 10 client.18165 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 10 client.18165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 20 client.18165 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d859d80 0x559f7dc5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 4 client.18165 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.748+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f7dbb7600 con 0x559f7d859d80 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.749+0000 7f7c5d600640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d859d80 0x559f7dc5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.749+0000 7f7c3ea00640 10 client.18165 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.749+0000 7f7c5d600640 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d859d80 0x559f7dc5d710 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde4742a6b7eb778 server_cookie=38c6abc51354337b in_seq=0 out_seq=0 2024-09-21T07:07:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c3ea00640 1 -- 172.21.15.114:0/2364390786 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f7dbb7600 con 0x559f7d859d80 2024-09-21T07:07:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c3ea00640 10 client.18165 handle_command_reply: tid=0 2024-09-21T07:07:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 client.18165 shutdown 2024-09-21T07:07:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 msgr2=0x559f7dc4b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 0x559f7dc4b980 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7c44006010 tx=0x7f7c44047000 comp rx=0 tx=0).stop 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 shutdown_connections 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d859d80 0x559f7dc5d710 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d89b1b0 0x559f7dc4bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c3ea00640 0 client.18165 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d87a4d0 0x559f7dc4b980 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 --2- 172.21.15.114:0/2364390786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc4b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.750+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 >> 172.21.15.114:0/2364390786 conn(0x559f7d88f300 msgr2=0x559f7dc40e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.751+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 shutdown_connections 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.751+0000 7f7c67206740 1 -- 172.21.15.114:0/2364390786 wait complete. 2024-09-21T07:07:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.751+0000 7f7c67206740 20 client.18165 trim_cache size 0 max 16384 2024-09-21T07:07:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 Processor -- start 2024-09-21T07:07:13.768 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 -- start start 2024-09-21T07:07:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc3ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc41800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d89b1b0 0x559f7dc4d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581b3070 con 0x559f7d8722f0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581b4ca0 con 0x559f7d89b1b0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c67206740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581b3740 con 0x559f7d87a4d0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c5d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc3ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c5d600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc3ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44714/0 (socket says 172.21.15.114:44714) 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c5d600640 1 -- 172.21.15.114:0/2302152621 learned_addr learned my addr 172.21.15.114:0/2302152621 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c5e000640 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d89b1b0 0x559f7dc4d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.766+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c5d600640 1 -- 172.21.15.114:0/2302152621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 msgr2=0x559f7dc41800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c5d600640 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc41800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c5d600640 1 -- 172.21.15.114:0/2302152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d89b1b0 msgr2=0x559f7dc4d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c5d600640 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d89b1b0 0x559f7dc4d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c5d600640 1 -- 172.21.15.114:0/2302152621 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7dc3e600 con 0x559f7d8722f0 2024-09-21T07:07:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c5d600640 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc3ce30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x559f7dc5de70 tx=0x7f7c4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9d3821432e2ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c49c00640 1 -- 172.21.15.114:0/2302152621 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c036070 con 0x559f7d8722f0 2024-09-21T07:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c49c00640 1 -- 172.21.15.114:0/2302152621 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4c003c40 con 0x559f7d8722f0 2024-09-21T07:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.767+0000 7f7c49c00640 1 -- 172.21.15.114:0/2302152621 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c01d670 con 0x559f7d8722f0 2024-09-21T07:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 -- 172.21.15.114:0/2302152621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 msgr2=0x559f7dc3ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc3ce30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x559f7dc5de70 tx=0x7f7c4c019000 comp rx=0 tx=0).stop 2024-09-21T07:07:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 -- 172.21.15.114:0/2302152621 shutdown_connections 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d89b1b0 0x559f7dc4d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc41800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 --2- 172.21.15.114:0/2302152621 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d8722f0 0x559f7dc3ce30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 -- 172.21.15.114:0/2302152621 >> 172.21.15.114:0/2302152621 conn(0x559f7d88f300 msgr2=0x559f7dc4c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 -- 172.21.15.114:0/2302152621 shutdown_connections 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 -- 172.21.15.114:0/2302152621 wait complete. 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.768+0000 7f7c67206740 1 -- start start 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 Processor -- start 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc51ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 0x559f7dc52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581b4570 con 0x559f7d89b1b0 2024-09-21T07:07:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581b3e10 con 0x559f7d8722f0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581b1070 con 0x559f7d87a4d0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c67206740 10 client.0 _ll_register_callbacks cb 0x7f7c44005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c5e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 0x559f7dc52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c5d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c5e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 0x559f7dc52420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44720/0 (socket says 172.21.15.114:44720) 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c5d600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42464/0 (socket says 172.21.15.114:42464) 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c5e000640 1 -- 172.21.15.114:0/3433322530 learned_addr learned my addr 172.21.15.114:0/3433322530 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.769+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc51ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5d600640 1 -- 172.21.15.114:0/3433322530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 msgr2=0x559f7dc51ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5d600640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc51ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5d600640 1 -- 172.21.15.114:0/3433322530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 msgr2=0x559f7dc52420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5d600640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 0x559f7dc52420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5d600640 1 -- 172.21.15.114:0/3433322530 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7dcc7290 con 0x559f7d8722f0 2024-09-21T07:07:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5e000640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 0x559f7dc52420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c5d600640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc519a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x559f7dc3e960 tx=0x7f7c4c0043a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1909580a84c12dde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c3ea00640 1 -- 172.21.15.114:0/3433322530 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c040060 con 0x559f7d8722f0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c4c003c40 con 0x559f7d8722f0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.770+0000 7f7c3ea00640 1 -- 172.21.15.114:0/3433322530 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9578+0+0 (secure 0 0 0) 0x7f7c4c005030 con 0x559f7d8722f0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c3ea00640 1 -- 172.21.15.114:0/3433322530 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7c4c047050 con 0x559f7d8722f0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 10 client.28730 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559f7dcc8520 con 0x559f7d8722f0 2024-09-21T07:07:13.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c3ea00640 1 -- 172.21.15.114:0/3433322530 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c4c003c40 con 0x559f7d8722f0 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c3ea00640 1 -- 172.21.15.114:0/3433322530 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f7c4c0046c0 con 0x559f7d8722f0 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 10 client.28730 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 10 client.28730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 20 client.28730 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d82f1c0 0x559f7dccacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 4 client.28730 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.771+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f7dbb30b0 con 0x559f7d82f1c0 2024-09-21T07:07:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.772+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d82f1c0 0x559f7dccacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.772+0000 7f7c5cc00640 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d82f1c0 0x559f7dccacc0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ca263469fca119 server_cookie=30d35d3f45d1283d in_seq=0 out_seq=0 2024-09-21T07:07:13.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.772+0000 7f7c3ea00640 10 client.28730 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c3ea00640 1 -- 172.21.15.114:0/3433322530 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f7dbb30b0 con 0x559f7d82f1c0 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c3ea00640 10 client.28730 handle_command_reply: tid=0 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 client.28730 shutdown 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 msgr2=0x559f7dc519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc519a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x559f7dc3e960 tx=0x7f7c4c0043a0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 shutdown_connections 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559f7d89b1b0 0x559f7dc52420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559f7d82f1c0 0x559f7dccacc0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559f7d87a4d0 0x559f7dc51ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 --2- 172.21.15.114:0/3433322530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559f7d8722f0 0x559f7dc519a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c3ea00640 0 client.28730 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 >> 172.21.15.114:0/3433322530 conn(0x559f7d88f300 msgr2=0x559f7dc415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 shutdown_connections 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 1 -- 172.21.15.114:0/3433322530 wait complete. 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.782+0000 7f7c67206740 20 client.28730 trim_cache size 0 max 16384 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:07:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c400b6680 msgr2=0x7f7c400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c400b6680 0x7f7c400b8b90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x559f7dc3ea90 tx=0x7f7c4c005f90 comp rx=0 tx=0).stop 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 msgr2=0x7f7c5819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5819ca80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7c44005ab0 tx=0x7f7c44003b90 comp rx=0 tx=0).stop 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 shutdown_connections 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c5810bd90 0x7f7c581a1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7c5810b440 0x7f7c5819ca80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7c400b6680 0x7f7c400b8b90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 --2- 172.21.15.114:0/483205962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c58105b10 0x7f7c5819c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 >> 172.21.15.114:0/483205962 conn(0x7f7c58101550 msgr2=0x7f7c58103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 shutdown_connections 2024-09-21T07:07:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:13.783+0000 7f7c5ea00640 1 -- 172.21.15.114:0/483205962 wait complete. 2024-09-21T07:07:15.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:15 smithi114 ceph-mon[30654]: pgmap v800: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-09-21T07:07:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:15 smithi183 ceph-mon[34219]: pgmap v800: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-09-21T07:07:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:15 smithi193 ceph-mon[34601]: pgmap v800: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 3.9 MiB/s wr, 412 op/s 2024-09-21T07:07:17.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:17 smithi114 ceph-mon[30654]: pgmap v801: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-09-21T07:07:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:17 smithi183 ceph-mon[34219]: pgmap v801: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-09-21T07:07:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:17 smithi193 ceph-mon[34601]: pgmap v801: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-09-21T07:07:17.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 760.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:07:17.967 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:07:18.350 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:19.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 -- 172.21.15.114:0/3341586293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac1101a0 msgr2=0x7fe2ac1125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:19.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/3341586293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1125c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009be0 tx=0x7fe2a402a750 comp rx=0 tx=0).stop 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 -- 172.21.15.114:0/3341586293 shutdown_connections 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/3341586293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1125c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/3341586293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac102f40 0x7fe2ac10fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/3341586293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1025f0 0x7fe2ac102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 -- 172.21.15.114:0/3341586293 >> 172.21.15.114:0/3341586293 conn(0x7fe2ac079740 msgr2=0x7fe2ac079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 -- 172.21.15.114:0/3341586293 shutdown_connections 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.069+0000 7fe2b0c00640 1 -- 172.21.15.114:0/3341586293 wait complete. 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 Processor -- start 2024-09-21T07:07:19.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 -- start start 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac1025f0 0x7fe2ac19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 0x7fe2ac19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac115c90 con 0x7fe2ac102f40 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac115b10 con 0x7fe2ac1025f0 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.070+0000 7fe2b0c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac115e10 con 0x7fe2ac1101a0 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 0x7fe2ac19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac1025f0 0x7fe2ac19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 0x7fe2ac19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49268/0 (socket says 172.21.15.114:49268) 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 -- 172.21.15.114:0/591705725 learned_addr learned my addr 172.21.15.114:0/591705725 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:19.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2aa800640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:19.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 -- 172.21.15.114:0/591705725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1101a0 msgr2=0x7fe2ac1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 -- 172.21.15.114:0/591705725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac1025f0 msgr2=0x7fe2ac19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac1025f0 0x7fe2ac19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2a9400640 1 -- 172.21.15.114:0/591705725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac1a2590 con 0x7fe2ac102f40 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.071+0000 7fe2aa800640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.072+0000 7fe2a9400640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 0x7fe2ac19cbc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9f0 tx=0x7fe29800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.072+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800cd80 con 0x7fe2ac102f40 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.072+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac1a2880 con 0x7fe2ac102f40 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.072+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2980108e0 con 0x7fe2ac102f40 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.072+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298018980 con 0x7fe2ac102f40 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.072+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1a8fe0 con 0x7fe2ac102f40 2024-09-21T07:07:19.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.073+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac06c240 con 0x7fe2ac102f40 2024-09-21T07:07:19.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.076+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe298010430 con 0x7fe2ac102f40 2024-09-21T07:07:19.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.078+0000 7fe2a2a00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2900b6680 0x7fe2900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:19.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.078+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9578+0+0 (secure 0 0 0) 0x7fe2980e7ed0 con 0x7fe2ac102f40 2024-09-21T07:07:19.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.078+0000 7fe2a9e00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2900b6680 0x7fe2900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:19.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.079+0000 7fe2a9e00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2900b6680 0x7fe2900b8b90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009530 tx=0x7fe29c009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:19.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.082+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe2980b2380 con 0x7fe2ac102f40 2024-09-21T07:07:19.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.380+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2ac19d750 con 0x7fe2ac102f40 2024-09-21T07:07:19.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.381+0000 7fe2a2a00640 1 -- 172.21.15.114:0/591705725 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7fe2ac19d750 con 0x7fe2ac102f40 2024-09-21T07:07:19.384 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:07:19.384 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":54,"btime":"2024-09-21T07:07:08:168462+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:07:07.936407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:07:19.384 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 54 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2900b6680 msgr2=0x7fe2900b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2900b6680 0x7fe2900b8b90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009530 tx=0x7fe29c009260 comp rx=0 tx=0).stop 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 msgr2=0x7fe2ac19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 0x7fe2ac19cbc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9f0 tx=0x7fe29800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 shutdown_connections 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2ac1101a0 0x7fe2ac1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe2ac102f40 0x7fe2ac19cbc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe2900b6680 0x7fe2900b8b90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 --2- 172.21.15.114:0/591705725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac1025f0 0x7fe2ac19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:19.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 >> 172.21.15.114:0/591705725 conn(0x7fe2ac079740 msgr2=0x7fe2ac109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:19.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 shutdown_connections 2024-09-21T07:07:19.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:19.384+0000 7fe2b0c00640 1 -- 172.21.15.114:0/591705725 wait complete. 2024-09-21T07:07:19.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:19 smithi114 ceph-mon[30654]: pgmap v802: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-09-21T07:07:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:19 smithi183 ceph-mon[34219]: pgmap v802: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-09-21T07:07:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:19 smithi193 ceph-mon[34601]: pgmap v802: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-09-21T07:07:20.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:20 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/591705725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:20.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:20 smithi114 ceph-mon[30654]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T07:07:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:20 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/591705725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:20.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:20 smithi183 ceph-mon[34219]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T07:07:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:20 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/591705725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:20.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:20 smithi193 ceph-mon[34601]: osdmap e101: 12 total, 12 up, 12 in 2024-09-21T07:07:20.703 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 2feeda4 --release --await' 2024-09-21T07:07:20.703 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 2feeda4 --release --await 2024-09-21T07:07:21.077 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:21.409 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:21 smithi114 ceph-mon[30654]: pgmap v804: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-21T07:07:21.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:21 smithi183 ceph-mon[34219]: pgmap v804: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-21T07:07:21.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:21 smithi193 ceph-mon[34601]: pgmap v804: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 -- 172.21.15.114:0/565980282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2701083c0 msgr2=0x7fb2701087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 --2- 172.21.15.114:0/565980282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2701083c0 0x7fb2701087d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb258009900 tx=0x7fb25802a410 comp rx=0 tx=0).stop 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 -- 172.21.15.114:0/565980282 shutdown_connections 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 --2- 172.21.15.114:0/565980282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb270108d10 0x7fb27010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 --2- 172.21.15.114:0/565980282 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb2701083c0 0x7fb2701087d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 --2- 172.21.15.114:0/565980282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb270102a80 0x7fb270102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 -- 172.21.15.114:0/565980282 >> 172.21.15.114:0/565980282 conn(0x7fb270079740 msgr2=0x7fb270079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:21.772 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 -- 172.21.15.114:0/565980282 shutdown_connections 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.769+0000 7fb275400640 1 -- 172.21.15.114:0/565980282 wait complete. 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.770+0000 7fb275400640 1 Processor -- start 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.770+0000 7fb275400640 1 -- start start 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb275400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 0x7fb2700ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb275400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2701083c0 0x7fb2700ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb275400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb270108d10 0x7fb270100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb275400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb270112980 con 0x7fb270102a80 2024-09-21T07:07:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb275400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb270112800 con 0x7fb2701083c0 2024-09-21T07:07:21.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb275400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb270112b00 con 0x7fb270108d10 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb26e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 0x7fb2700ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb265e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2701083c0 0x7fb2700ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb26e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 0x7fb2700ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49290/0 (socket says 172.21.15.114:49290) 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb26e800640 1 -- 172.21.15.114:0/1022192239 learned_addr learned my addr 172.21.15.114:0/1022192239 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.771+0000 7fb26f200640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb270108d10 0x7fb270100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb26e800640 1 -- 172.21.15.114:0/1022192239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb270108d10 msgr2=0x7fb270100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb26e800640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb270108d10 0x7fb270100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb26e800640 1 -- 172.21.15.114:0/1022192239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2701083c0 msgr2=0x7fb2700ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb26e800640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2701083c0 0x7fb2700ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb26e800640 1 -- 172.21.15.114:0/1022192239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2701a8740 con 0x7fb270102a80 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb26e800640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 0x7fb2700ff690 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb26000db80 tx=0x7fb26000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb267e00640 1 -- 172.21.15.114:0/1022192239 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260017070 con 0x7fb270102a80 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb267e00640 1 -- 172.21.15.114:0/1022192239 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb260003b70 con 0x7fb270102a80 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2701a8a30 con 0x7fb270102a80 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb267e00640 1 -- 172.21.15.114:0/1022192239 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600106c0 con 0x7fb270102a80 2024-09-21T07:07:21.776 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.772+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2701a8f40 con 0x7fb270102a80 2024-09-21T07:07:21.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.776+0000 7fb27c08e740 1 Processor -- start 2024-09-21T07:07:21.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 -- start start 2024-09-21T07:07:21.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 0x55d479cdb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cdbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d479c52290 con 0x55d479934520 2024-09-21T07:07:21.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d479c416c0 con 0x55d47990b660 2024-09-21T07:07:21.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.777+0000 7fb27c08e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d479c454d0 con 0x55d479913840 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb26f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 0x55d479cdb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb267e00640 1 -- 172.21.15.114:0/1022192239 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb260004020 con 0x7fb270102a80 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb26f200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 0x55d479cdb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49302/0 (socket says 172.21.15.114:49302) 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb26f200640 1 -- 172.21.15.114:0/1120159731 learned_addr learned my addr 172.21.15.114:0/1120159731 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb265e00640 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cdbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb26e800640 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb265e00640 1 -- 172.21.15.114:0/1120159731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 msgr2=0x55d479cdb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb265e00640 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb265e00640 1 -- 172.21.15.114:0/1120159731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 msgr2=0x55d479cdb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.781 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb265e00640 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 0x55d479cdb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb265e00640 1 -- 172.21.15.114:0/1120159731 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d479bce460 con 0x55d479913840 2024-09-21T07:07:21.782 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.778+0000 7fb26f200640 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 0x55d479cdb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.779+0000 7fb265e00640 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cdbc40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb258006010 tx=0x7fb258032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ae99fdeb494670 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.779+0000 7fb25f400640 1 -- 172.21.15.114:0/1120159731 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258040070 con 0x55d479913840 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.779+0000 7fb25f400640 1 -- 172.21.15.114:0/1120159731 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb258004030 con 0x55d479913840 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.779+0000 7fb25f400640 1 -- 172.21.15.114:0/1120159731 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258035040 con 0x55d479913840 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb267e00640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2540b6680 0x7fb2540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb267e00640 1 -- 172.21.15.114:0/1022192239 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fb2600e7c00 con 0x7fb270102a80 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb26f200640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2540b6680 0x7fb2540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 -- 172.21.15.114:0/1120159731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 msgr2=0x55d479cdbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cdbc40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb258006010 tx=0x7fb258032bd0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 -- 172.21.15.114:0/1120159731 shutdown_connections 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cdbc40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1120159731 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479934520 0x55d479cdb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 -- 172.21.15.114:0/1120159731 >> 172.21.15.114:0/1120159731 conn(0x55d479928910 msgr2=0x55d479cd90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 -- 172.21.15.114:0/1120159731 shutdown_connections 2024-09-21T07:07:21.784 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.780+0000 7fb27c08e740 1 -- 172.21.15.114:0/1120159731 wait complete. 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.781+0000 7fb26f200640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2540b6680 0x7fb2540b8b90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2680099a0 tx=0x7fb268005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.781+0000 7fb27c08e740 1 -- start start 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.781+0000 7fb27c08e740 1 Processor -- start 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479ce6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479913840 0x55d479ce73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479cf2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d479aa2bb0 con 0x55d47990b660 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d479bdc6c0 con 0x55d479913840 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d479ce9940 con 0x55d479934520 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb27c08e740 10 client.0 _ll_register_callbacks cb 0x55d479bf7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479ce6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479ce6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49316/0 (socket says 172.21.15.114:49316) 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 -- 172.21.15.114:0/1972517028 learned_addr learned my addr 172.21.15.114:0/1972517028 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb265e00640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479913840 0x55d479ce73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26f200640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479cf2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 -- 172.21.15.114:0/1972517028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 msgr2=0x55d479cf2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479cf2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 -- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479913840 msgr2=0x55d479ce73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479913840 0x55d479ce73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.782+0000 7fb26e800640 1 -- 172.21.15.114:0/1972517028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47999fb00 con 0x55d47990b660 2024-09-21T07:07:21.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb26f200640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479cf2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb26e800640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479ce6e20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb260000f80 tx=0x7fb260004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706e3600e58229c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb253e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb253e00640 1 -- 172.21.15.114:0/1972517028 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600acf70 con 0x55d47990b660 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb253e00640 1 -- 172.21.15.114:0/1972517028 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fb2600e51a0 con 0x55d47990b660 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb258004030 con 0x55d47990b660 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb253e00640 1 -- 172.21.15.114:0/1972517028 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb26000be60 con 0x55d47990b660 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb27c08e740 10 client.18189 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.783+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d479a16960 con 0x55d47990b660 2024-09-21T07:07:21.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb253e00640 1 -- 172.21.15.114:0/1972517028 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb258004030 con 0x55d47990b660 2024-09-21T07:07:21.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb253e00640 1 -- 172.21.15.114:0/1972517028 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7fb2600ee020 con 0x55d47990b660 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb27c08e740 10 client.18189 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb27c08e740 10 client.18189 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb27c08e740 20 client.18189 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798f2ea0 0x55d479cf8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb27c08e740 4 client.18189 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.784+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d479c541e0 con 0x55d4798f2ea0 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.785+0000 7fb265e00640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798f2ea0 0x55d479cf8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.785+0000 7fb265e00640 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798f2ea0 0x55d479cf8df0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462c699b82e26807 server_cookie=a3d177b60ad171d6 in_seq=0 out_seq=0 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.785+0000 7fb253e00640 10 client.18189 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb253e00640 1 -- 172.21.15.114:0/1972517028 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d479c541e0 con 0x55d4798f2ea0 2024-09-21T07:07:21.790 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb253e00640 10 client.18189 handle_command_reply: tid=0 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 client.18189 shutdown 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 msgr2=0x55d479ce6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479ce6e20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb260000f80 tx=0x7fb260004940 comp rx=0 tx=0).stop 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 shutdown_connections 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798f2ea0 0x55d479cf8df0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479cf2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479913840 0x55d479ce73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1972517028 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479ce6e20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb253e00640 0 client.18189 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:21.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.786+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 >> 172.21.15.114:0/1972517028 conn(0x55d479928910 msgr2=0x55d479cde3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:21.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.787+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 shutdown_connections 2024-09-21T07:07:21.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.787+0000 7fb27c08e740 1 -- 172.21.15.114:0/1972517028 wait complete. 2024-09-21T07:07:21.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.787+0000 7fb27c08e740 20 client.18189 trim_cache size 0 max 16384 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 Processor -- start 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 -- start start 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 0x55d479ce02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479ce8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b3160 con 0x55d479913840 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b4d90 con 0x55d47990b660 2024-09-21T07:07:21.808 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.805+0000 7fb27c08e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b3830 con 0x55d479934520 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 0x55d479ce02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb26e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 0x55d479ce02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49326/0 (socket says 172.21.15.114:49326) 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 -- 172.21.15.114:0/2026145335 learned_addr learned my addr 172.21.15.114:0/2026145335 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb26f200640 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479ce8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 -- 172.21.15.114:0/2026145335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 msgr2=0x55d479ce8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479ce8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.809 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 -- 172.21.15.114:0/2026145335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 msgr2=0x55d479cdc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb265e00640 1 -- 172.21.15.114:0/2026145335 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d479ce7ee0 con 0x55d479913840 2024-09-21T07:07:21.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.806+0000 7fb26e800640 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:21.810 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.807+0000 7fb265e00640 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 0x55d479ce02d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb2580095d0 tx=0x7fb258000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b11b299a19c91a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.807+0000 7fb25ea00640 1 -- 172.21.15.114:0/2026145335 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258040040 con 0x55d479913840 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.807+0000 7fb25ea00640 1 -- 172.21.15.114:0/2026145335 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb258032cc0 con 0x55d479913840 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.807+0000 7fb25ea00640 1 -- 172.21.15.114:0/2026145335 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258035040 con 0x55d479913840 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 -- 172.21.15.114:0/2026145335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 msgr2=0x55d479ce02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 0x55d479ce02d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb2580095d0 tx=0x7fb258000bc0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 -- 172.21.15.114:0/2026145335 shutdown_connections 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479934520 0x55d479ce8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d479913840 0x55d479ce02d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 --2- 172.21.15.114:0/2026145335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d47990b660 0x55d479cdc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 -- 172.21.15.114:0/2026145335 >> 172.21.15.114:0/2026145335 conn(0x55d479928910 msgr2=0x55d479ce54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:21.811 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 -- 172.21.15.114:0/2026145335 shutdown_connections 2024-09-21T07:07:21.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 -- 172.21.15.114:0/2026145335 wait complete. 2024-09-21T07:07:21.812 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.808+0000 7fb27c08e740 1 -- start start 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 Processor -- start 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479cebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479934520 0x55d479cec730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b4660 con 0x55d47990b660 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b3f00 con 0x55d479934520 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb2701b1160 con 0x55d479913840 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb27c08e740 10 client.0 _ll_register_callbacks cb 0x7fb26000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb26e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479cebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb26e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479cebcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49330/0 (socket says 172.21.15.114:49330) 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.809+0000 7fb26e800640 1 -- 172.21.15.114:0/1257903992 learned_addr learned my addr 172.21.15.114:0/1257903992 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb265e00640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26f200640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479934520 0x55d479cec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.813 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26e800640 1 -- 172.21.15.114:0/1257903992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 msgr2=0x55d479cec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26e800640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26e800640 1 -- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479934520 msgr2=0x55d479cec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:21.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26e800640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479934520 0x55d479cec730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26e800640 1 -- 172.21.15.114:0/1257903992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d479d638f0 con 0x55d47990b660 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb26e800640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479cebcb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2600b7160 tx=0x7fb2600a9530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eebb4310d57cc2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.810+0000 7fb253e00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb253e00640 1 -- 172.21.15.114:0/1257903992 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260014070 con 0x55d47990b660 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb253e00640 1 -- 172.21.15.114:0/1257903992 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fb2600eb340 con 0x55d47990b660 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb258032cc0 con 0x55d47990b660 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb253e00640 1 -- 172.21.15.114:0/1257903992 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb2600acd90 con 0x55d47990b660 2024-09-21T07:07:21.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb27c08e740 10 client.18195 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d479d64bb0 con 0x55d47990b660 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb253e00640 1 -- 172.21.15.114:0/1257903992 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb258032cc0 con 0x55d47990b660 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.811+0000 7fb253e00640 1 -- 172.21.15.114:0/1257903992 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7fb2600ee050 con 0x55d47990b660 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb27c08e740 10 client.18195 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb27c08e740 10 client.18195 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb27c08e740 20 client.18195 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798c8250 0x55d479d672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb27c08e740 4 client.18195 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2feeda4", "release": true, "await": true}] 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2feeda4", "release": true, "await": true}) -- 0x55d479c4fc90 con 0x55d4798c8250 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.812+0000 7fb265e00640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798c8250 0x55d479d672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.813+0000 7fb265e00640 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798c8250 0x55d479d672b0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3cfdd5a9884eb1 server_cookie=16d5ab1a70525e3c in_seq=0 out_seq=0 2024-09-21T07:07:21.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.813+0000 7fb253e00640 10 client.18195 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:21.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.842+0000 7fb253e00640 1 -- 172.21.15.114:0/1257903992 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d479c4fc90 con 0x55d4798c8250 2024-09-21T07:07:21.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.842+0000 7fb253e00640 10 client.18195 handle_command_reply: tid=0 2024-09-21T07:07:21.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.842+0000 7fb27c08e740 1 client.18195 shutdown 2024-09-21T07:07:21.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 msgr2=0x55d479cebcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479cebcb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb2600b7160 tx=0x7fb2600a9530 comp rx=0 tx=0).stop 2024-09-21T07:07:21.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 shutdown_connections 2024-09-21T07:07:21.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d479934520 0x55d479cec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d4798c8250 0x55d479d672b0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d479913840 0x55d479cec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 --2- 172.21.15.114:0/1257903992 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d47990b660 0x55d479cebcb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 >> 172.21.15.114:0/1257903992 conn(0x55d479928910 msgr2=0x55d479ce5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.843+0000 7fb253e00640 0 client.18195 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.844+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 shutdown_connections 2024-09-21T07:07:21.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.844+0000 7fb27c08e740 1 -- 172.21.15.114:0/1257903992 wait complete. 2024-09-21T07:07:21.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.845+0000 7fb27c08e740 20 client.18195 trim_cache size 0 max 16384 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 54, 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 52, 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "2feeda4": { 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 52, 2024-09-21T07:07:21.849 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 137.8, 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:21.850 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.846+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2540b6680 msgr2=0x7fb2540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2540b6680 0x7fb2540b8b90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb2680099a0 tx=0x7fb268005f90 comp rx=0 tx=0).stop 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 msgr2=0x7fb2700ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 0x7fb2700ff690 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb26000db80 tx=0x7fb26000b630 comp rx=0 tx=0).stop 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 shutdown_connections 2024-09-21T07:07:21.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb270108d10 0x7fb270100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb2540b6680 0x7fb2540b8b90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2701083c0 0x7fb2700ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 --2- 172.21.15.114:0/1022192239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb270102a80 0x7fb2700ff690 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:21.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.847+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 >> 172.21.15.114:0/1022192239 conn(0x7fb270079740 msgr2=0x7fb27006bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:21.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.849+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 shutdown_connections 2024-09-21T07:07:21.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:21.849+0000 7fb275400640 1 -- 172.21.15.114:0/1022192239 wait complete. 2024-09-21T07:07:22.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 765.0 s, 151.6 tps, lat 1682.303 ms stddev 4024.328 2024-09-21T07:07:23.025 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2feeda4', total seconds elapsed: 22.9 2024-09-21T07:07:23.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:23 smithi114 ceph-mon[30654]: pgmap v805: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-21T07:07:23.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:23 smithi183 ceph-mon[34219]: pgmap v805: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-21T07:07:23.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:23 smithi193 ceph-mon[34601]: pgmap v805: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-21T07:07:25.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:25 smithi114 ceph-mon[30654]: pgmap v806: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 518 KiB/s wr, 54 op/s 2024-09-21T07:07:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:25 smithi183 ceph-mon[34219]: pgmap v806: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 518 KiB/s wr, 54 op/s 2024-09-21T07:07:25.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:25 smithi193 ceph-mon[34601]: pgmap v806: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 518 KiB/s wr, 54 op/s 2024-09-21T07:07:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 -- 172.21.15.114:0/4276836173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97401101b0 msgr2=0x7f97401169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 --2- 172.21.15.114:0/4276836173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97401101b0 0x7f97401169e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9738009860 tx=0x7f973802a340 comp rx=0 tx=0).stop 2024-09-21T07:07:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 -- 172.21.15.114:0/4276836173 shutdown_connections 2024-09-21T07:07:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 --2- 172.21.15.114:0/4276836173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f97401101b0 0x7f97401169e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 --2- 172.21.15.114:0/4276836173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f974010f860 0x7f974010fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.108 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 --2- 172.21.15.114:0/4276836173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740109f60 0x7f974010a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.106+0000 7f9746800640 1 -- 172.21.15.114:0/4276836173 >> 172.21.15.114:0/4276836173 conn(0x7f9740079680 msgr2=0x7f97401081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.107+0000 7f9746800640 1 -- 172.21.15.114:0/4276836173 shutdown_connections 2024-09-21T07:07:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.107+0000 7f9746800640 1 -- 172.21.15.114:0/4276836173 wait complete. 2024-09-21T07:07:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.108+0000 7f9746800640 1 Processor -- start 2024-09-21T07:07:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 -- start start 2024-09-21T07:07:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740109f60 0x7f9740074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974010f860 0x7f974006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 0x7f974006e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9740119e70 con 0x7f974010f860 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9740119cf0 con 0x7f97401101b0 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.109+0000 7f9746800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9740119ff0 con 0x7f9740109f60 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.110+0000 7f9745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 0x7f974006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.110+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740109f60 0x7f9740074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 0x7f974006e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36920/0 (socket says 172.21.15.114:36920) 2024-09-21T07:07:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 -- 172.21.15.114:0/3784157550 learned_addr learned my addr 172.21.15.114:0/3784157550 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 -- 172.21.15.114:0/3784157550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740109f60 msgr2=0x7f9740074e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740109f60 0x7f9740074e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 -- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974010f860 msgr2=0x7f974006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974010f860 0x7f974006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.112+0000 7f9745e00640 1 -- 172.21.15.114:0/3784157550 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974006eda0 con 0x7f97401101b0 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.113+0000 7f9745e00640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 0x7f974006e610 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f973802a930 tx=0x7f9738032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.113+0000 7f973e000640 1 -- 172.21.15.114:0/3784157550 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738040020 con 0x7f97401101b0 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.114+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9740073a70 con 0x7f97401101b0 2024-09-21T07:07:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.114+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9740073fb0 con 0x7f97401101b0 2024-09-21T07:07:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.114+0000 7f973e000640 1 -- 172.21.15.114:0/3784157550 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9738030dd0 con 0x7f97401101b0 2024-09-21T07:07:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.114+0000 7f973e000640 1 -- 172.21.15.114:0/3784157550 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738037040 con 0x7f97401101b0 2024-09-21T07:07:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.123+0000 7f974ee7c740 1 Processor -- start 2024-09-21T07:07:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.124+0000 7f973e000640 1 -- 172.21.15.114:0/3784157550 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9738043020 con 0x7f97401101b0 2024-09-21T07:07:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.126+0000 7f973e000640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97280b6700 0x7f97280b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.126+0000 7f973e000640 1 -- 172.21.15.114:0/3784157550 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f9738107c30 con 0x7f97401101b0 2024-09-21T07:07:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.127+0000 7f9744a00640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97280b6700 0x7f97280b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.127+0000 7f974ee7c740 1 -- start start 2024-09-21T07:07:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.128+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372147e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.128+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d7a2f0 0x55b372148350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.128+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.128+0000 7f974ee7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b3720bd6b0 con 0x55b371d7a2f0 2024-09-21T07:07:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.128+0000 7f974ee7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b3720acae0 con 0x55b371d824d0 2024-09-21T07:07:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.128+0000 7f974ee7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b3720b08f0 con 0x55b371da31b0 2024-09-21T07:07:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.129+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.129+0000 7f9745e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.129+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41110/0 (socket says 172.21.15.114:41110) 2024-09-21T07:07:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.129+0000 7f9745400640 1 -- 172.21.15.114:0/3617662798 learned_addr learned my addr 172.21.15.114:0/3617662798 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.130+0000 7f9745400640 1 -- 172.21.15.114:0/3617662798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 msgr2=0x55b372148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.130+0000 7f9745400640 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.130+0000 7f9745400640 1 -- 172.21.15.114:0/3617662798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d7a2f0 msgr2=0x55b372148350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.130+0000 7f9744a00640 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d7a2f0 0x55b372148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.130+0000 7f9745400640 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d7a2f0 0x55b372148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.130+0000 7f9745400640 1 -- 172.21.15.114:0/3617662798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b372079780 con 0x55b371da31b0 2024-09-21T07:07:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.131+0000 7f9745400640 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372147e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9734007fc0 tx=0x7f973400f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c65096d09083158 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.131+0000 7f9732600640 1 -- 172.21.15.114:0/3617662798 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734004d20 con 0x55b371da31b0 2024-09-21T07:07:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.131+0000 7f9732600640 1 -- 172.21.15.114:0/3617662798 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9734010bb0 con 0x55b371da31b0 2024-09-21T07:07:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.131+0000 7f9732600640 1 -- 172.21.15.114:0/3617662798 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97340057b0 con 0x55b371da31b0 2024-09-21T07:07:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.133+0000 7f974ee7c740 1 -- 172.21.15.114:0/3617662798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 msgr2=0x55b372147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.133+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372147e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9734007fc0 tx=0x7f973400f850 comp rx=0 tx=0).stop 2024-09-21T07:07:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.133+0000 7f974ee7c740 1 -- 172.21.15.114:0/3617662798 shutdown_connections 2024-09-21T07:07:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.134+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.134+0000 7f9744a00640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97280b6700 0x7f97280b8c10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b372146a90 tx=0x7f972c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.134+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d7a2f0 0x55b372148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.134+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3617662798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372147e10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.134+0000 7f974ee7c740 1 -- 172.21.15.114:0/3617662798 >> 172.21.15.114:0/3617662798 conn(0x55b371d97300 msgr2=0x55b372145d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.135+0000 7f974ee7c740 1 -- 172.21.15.114:0/3617662798 shutdown_connections 2024-09-21T07:07:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.136+0000 7f974ee7c740 1 -- 172.21.15.114:0/3617662798 wait complete. 2024-09-21T07:07:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.137+0000 7f974ee7c740 1 -- start start 2024-09-21T07:07:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.137+0000 7f974ee7c740 1 Processor -- start 2024-09-21T07:07:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.140+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d7a2f0 0x55b372153400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.140+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372153940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.140+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372153e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.141+0000 7f9744a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.141+0000 7f9744a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36938/0 (socket says 172.21.15.114:36938) 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.141+0000 7f9745e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.141+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d7a2f0 0x55b372153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.141+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d7a2f0 0x55b372153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41124/0 (socket says 172.21.15.114:41124) 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.141+0000 7f9745e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:41722/0 (socket says 172.21.15.114:41722) 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f974ee7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b371fe4ab0 con 0x55b371da31b0 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f974ee7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b372053570 con 0x55b371d824d0 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f974ee7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b372156590 con 0x55b371d7a2f0 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f974ee7c740 10 client.0 _ll_register_callbacks cb 0x55b372062100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f9745400640 1 -- 172.21.15.114:0/3311727892 learned_addr learned my addr 172.21.15.114:0/3311727892 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f9744a00640 1 -- 172.21.15.114:0/3311727892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d7a2f0 msgr2=0x55b372153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f9744a00640 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d7a2f0 0x55b372153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.142+0000 7f9744a00640 1 -- 172.21.15.114:0/3311727892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 msgr2=0x55b372153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.143+0000 7f9744a00640 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.143+0000 7f9744a00640 1 -- 172.21.15.114:0/3311727892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b371e0e790 con 0x55b371d824d0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.143+0000 7f9745e00640 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.143+0000 7f9744a00640 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372153940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f972c0058c0 tx=0x7f972c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9192de1a3c1e2407 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.143+0000 7f9726a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.144+0000 7f9726a00640 1 -- 172.21.15.114:0/3311727892 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c022070 con 0x55b371d824d0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.144+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9734010bb0 con 0x55b371d824d0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.144+0000 7f9726a00640 1 -- 172.21.15.114:0/3311727892 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f972c00ec90 con 0x55b371d824d0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.146+0000 7f9726a00640 1 -- 172.21.15.114:0/3311727892 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f972c02b050 con 0x55b371d824d0 2024-09-21T07:07:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.146+0000 7f974ee7c740 10 client.28763 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.146+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b371e87620 con 0x55b371d824d0 2024-09-21T07:07:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.147+0000 7f9726a00640 1 -- 172.21.15.114:0/3311727892 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9734010bb0 con 0x55b371d824d0 2024-09-21T07:07:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.147+0000 7f9726a00640 1 -- 172.21.15.114:0/3311727892 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f972c02e020 con 0x55b371d824d0 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.148+0000 7f974ee7c740 10 client.28763 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.148+0000 7f974ee7c740 10 client.28763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.148+0000 7f974ee7c740 20 client.28763 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.148+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b371d61d80 0x55b372165630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.148+0000 7f974ee7c740 4 client.28763 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.148+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3720bf600 con 0x55b371d61d80 2024-09-21T07:07:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.149+0000 7f9745400640 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b371d61d80 0x55b372165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.150+0000 7f9745400640 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b371d61d80 0x55b372165630 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1d8bb3750799da server_cookie=aba232829cbe1e77 in_seq=0 out_seq=0 2024-09-21T07:07:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.150+0000 7f9726a00640 10 client.28763 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.152+0000 7f9726a00640 1 -- 172.21.15.114:0/3311727892 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3720bf600 con 0x55b371d61d80 2024-09-21T07:07:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.152+0000 7f9726a00640 10 client.28763 handle_command_reply: tid=0 2024-09-21T07:07:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.152+0000 7f974ee7c740 1 client.28763 shutdown 2024-09-21T07:07:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.153+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 msgr2=0x55b372153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.153+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372153940 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f972c0058c0 tx=0x7f972c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 shutdown_connections 2024-09-21T07:07:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d824d0 0x55b372153940 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b371d61d80 0x55b372165630 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 --2- 172.21.15.114:0/3311727892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d7a2f0 0x55b372153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 >> 172.21.15.114:0/3311727892 conn(0x55b371d97300 msgr2=0x55b372148e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f9726a00640 0 client.28763 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 shutdown_connections 2024-09-21T07:07:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.154+0000 7f974ee7c740 1 -- 172.21.15.114:0/3311727892 wait complete. 2024-09-21T07:07:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.155+0000 7f974ee7c740 20 client.28763 trim_cache size 0 max 16384 2024-09-21T07:07:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.184+0000 7f974ee7c740 1 Processor -- start 2024-09-21T07:07:27.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.184+0000 7f974ee7c740 1 -- start start 2024-09-21T07:07:27.186 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 0x55b372149550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d824d0 0x55b3721505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372150b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f974ee7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f97401c83e0 con 0x55b371d824d0 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f974ee7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97401ca010 con 0x55b371d7a2f0 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f974ee7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97401c8ab0 con 0x55b371da31b0 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.185+0000 7f9745400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 0x55b372149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.186+0000 7f9745400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 0x55b372149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36954/0 (socket says 172.21.15.114:36954) 2024-09-21T07:07:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.186+0000 7f9745400640 1 -- 172.21.15.114:0/742444287 learned_addr learned my addr 172.21.15.114:0/742444287 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.186+0000 7f9745e00640 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.188 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.186+0000 7f9744a00640 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d824d0 0x55b3721505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.188+0000 7f9745400640 1 -- 172.21.15.114:0/742444287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 msgr2=0x55b372150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.188+0000 7f9745400640 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.188+0000 7f9745400640 1 -- 172.21.15.114:0/742444287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d824d0 msgr2=0x55b3721505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.188+0000 7f9745400640 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d824d0 0x55b3721505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.188+0000 7f9745400640 1 -- 172.21.15.114:0/742444287 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3721518e0 con 0x55b371d7a2f0 2024-09-21T07:07:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.189+0000 7f9744a00640 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d824d0 0x55b3721505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.189+0000 7f9745400640 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 0x55b372149550 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9734007b90 tx=0x7f9734005310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ba0bc2bdcfa86c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.190+0000 7f9731c00640 1 -- 172.21.15.114:0/742444287 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734012ea0 con 0x55b371d7a2f0 2024-09-21T07:07:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.190+0000 7f9731c00640 1 -- 172.21.15.114:0/742444287 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97340058f0 con 0x55b371d7a2f0 2024-09-21T07:07:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.190+0000 7f9731c00640 1 -- 172.21.15.114:0/742444287 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97340049e0 con 0x55b371d7a2f0 2024-09-21T07:07:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.191+0000 7f974ee7c740 1 -- 172.21.15.114:0/742444287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 msgr2=0x55b372149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.191+0000 7f974ee7c740 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 0x55b372149550 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9734007b90 tx=0x7f9734005310 comp rx=0 tx=0).stop 2024-09-21T07:07:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 -- 172.21.15.114:0/742444287 shutdown_connections 2024-09-21T07:07:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371da31b0 0x55b372150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371d824d0 0x55b3721505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 --2- 172.21.15.114:0/742444287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d7a2f0 0x55b372149550 secure :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9734007b90 tx=0x7f9734005310 comp rx=0 tx=0).stop 2024-09-21T07:07:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 -- 172.21.15.114:0/742444287 >> 172.21.15.114:0/742444287 conn(0x55b371d97300 msgr2=0x55b372148e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 -- 172.21.15.114:0/742444287 shutdown_connections 2024-09-21T07:07:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.192+0000 7f974ee7c740 1 -- 172.21.15.114:0/742444287 wait complete. 2024-09-21T07:07:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.193+0000 7f974ee7c740 1 -- start start 2024-09-21T07:07:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.193+0000 7f974ee7c740 1 Processor -- start 2024-09-21T07:07:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.194+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 0x55b372159a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f974ee7c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d371c0 0x55b37215a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 0x55b372159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f9745400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 0x55b372159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41128/0 (socket says 172.21.15.114:41128) 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f974ee7c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f97401c98e0 con 0x55b371da31b0 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f974ee7c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97401c9180 con 0x55b371d371c0 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.195+0000 7f974ee7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f974006f3d0 con 0x55b371d824d0 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.196+0000 7f974ee7c740 10 client.0 _ll_register_callbacks cb 0x7f972c009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.196+0000 7f9745400640 1 -- 172.21.15.114:0/2436198870 learned_addr learned my addr 172.21.15.114:0/2436198870 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.196+0000 7f9745e00640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d371c0 0x55b37215a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.196+0000 7f9744a00640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.198+0000 7f9744a00640 1 -- 172.21.15.114:0/2436198870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 msgr2=0x55b372159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.198+0000 7f9744a00640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 0x55b372159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.198+0000 7f9744a00640 1 -- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d371c0 msgr2=0x55b37215a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:27.201 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.198+0000 7f9744a00640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d371c0 0x55b37215a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.198+0000 7f9744a00640 1 -- 172.21.15.114:0/2436198870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3721cf270 con 0x55b371da31b0 2024-09-21T07:07:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.200+0000 7f9745400640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 0x55b372159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:07:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.200+0000 7f9744a00640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372159f80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f972c01e670 tx=0x7f972c01e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d94e3a7bd89268f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:27.202 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.200+0000 7f9726a00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.202+0000 7f9726a00640 1 -- 172.21.15.114:0/2436198870 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c022040 con 0x55b371da31b0 2024-09-21T07:07:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.202+0000 7f9726a00640 1 -- 172.21.15.114:0/2436198870 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f972c00ea90 con 0x55b371da31b0 2024-09-21T07:07:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.202+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97340058f0 con 0x55b371da31b0 2024-09-21T07:07:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.203+0000 7f9726a00640 1 -- 172.21.15.114:0/2436198870 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f972c03e050 con 0x55b371da31b0 2024-09-21T07:07:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.203+0000 7f974ee7c740 10 client.18207 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.203+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b3721d0500 con 0x55b371da31b0 2024-09-21T07:07:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.206+0000 7f9726a00640 1 -- 172.21.15.114:0/2436198870 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97340058f0 con 0x55b371da31b0 2024-09-21T07:07:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.206+0000 7f9726a00640 1 -- 172.21.15.114:0/2436198870 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f972c02e040 con 0x55b371da31b0 2024-09-21T07:07:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.206+0000 7f974ee7c740 10 client.18207 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.206+0000 7f974ee7c740 10 client.18207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.206+0000 7f974ee7c740 20 client.18207 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.207+0000 7f974ee7c740 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b3721d0c70 0x55b3721d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.207+0000 7f974ee7c740 4 client.18207 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:07:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.207+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3720bb0b0 con 0x55b3721d0c70 2024-09-21T07:07:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.207+0000 7f9745400640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b3721d0c70 0x55b3721d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.208+0000 7f9726a00640 10 client.18207 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.209+0000 7f9745400640 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b3721d0c70 0x55b3721d30c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e73d845353a37b server_cookie=4644a18eee842f03 in_seq=0 out_seq=0 2024-09-21T07:07:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.320+0000 7f9726a00640 1 -- 172.21.15.114:0/2436198870 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3720bb0b0 con 0x55b3721d0c70 2024-09-21T07:07:27.322 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.320+0000 7f9726a00640 10 client.18207 handle_command_reply: tid=0 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.320+0000 7f974ee7c740 1 client.18207 shutdown 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.320+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 msgr2=0x55b372159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.320+0000 7f974ee7c740 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372159f80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f972c01e670 tx=0x7f972c01e9a0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 shutdown_connections 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f974ee7c740 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b371d371c0 0x55b37215a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f974ee7c740 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b371da31b0 0x55b372159f80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f974ee7c740 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55b3721d0c70 0x55b3721d30c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f974ee7c740 1 --2- 172.21.15.114:0/2436198870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b371d824d0 0x55b372159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 >> 172.21.15.114:0/2436198870 conn(0x55b371d97300 msgr2=0x55b372144360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.321+0000 7f9726a00640 0 client.18207 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.323+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 shutdown_connections 2024-09-21T07:07:27.324 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.323+0000 7f974ee7c740 1 -- 172.21.15.114:0/2436198870 wait complete. 2024-09-21T07:07:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.323+0000 7f974ee7c740 20 client.18207 trim_cache size 0 max 16384 2024-09-21T07:07:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:07:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:07:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:07:27.325 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:07:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97280b6700 msgr2=0x7f97280b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97280b6700 0x7f97280b8c10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b372146a90 tx=0x7f972c005850 comp rx=0 tx=0).stop 2024-09-21T07:07:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 msgr2=0x7f974006e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 0x7f974006e610 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f973802a930 tx=0x7f9738032d50 comp rx=0 tx=0).stop 2024-09-21T07:07:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 shutdown_connections 2024-09-21T07:07:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f97280b6700 0x7f97280b8c10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97401101b0 0x7f974006e610 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f974010f860 0x7f974006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 --2- 172.21.15.114:0/3784157550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9740109f60 0x7f9740074e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.325+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 >> 172.21.15.114:0/3784157550 conn(0x7f9740079680 msgr2=0x7f974010baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.326+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 shutdown_connections 2024-09-21T07:07:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:27.326+0000 7f9746800640 1 -- 172.21.15.114:0/3784157550 wait complete. 2024-09-21T07:07:27.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:27 smithi114 ceph-mon[30654]: pgmap v807: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 914 KiB/s wr, 113 op/s 2024-09-21T07:07:27.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:27 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:27 smithi183 ceph-mon[34219]: pgmap v807: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 914 KiB/s wr, 113 op/s 2024-09-21T07:07:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:27 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:27 smithi193 ceph-mon[34601]: pgmap v807: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 914 KiB/s wr, 113 op/s 2024-09-21T07:07:27.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:27 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:07:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 770.0 s, 534.8 tps, lat 189.231 ms stddev 206.211 2024-09-21T07:07:28.249 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:07:28.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:28.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:28 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:28.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:28 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:07:28.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:28.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:28 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:28.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:28 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:07:28.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:07:28.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:28 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:07:28.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:28 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:07:29.001 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:29.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:29 smithi114 ceph-mon[30654]: pgmap v808: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 2.5 MiB/s wr, 234 op/s 2024-09-21T07:07:29.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:29 smithi183 ceph-mon[34219]: pgmap v808: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 2.5 MiB/s wr, 234 op/s 2024-09-21T07:07:29.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:29 smithi193 ceph-mon[34601]: pgmap v808: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 2.5 MiB/s wr, 234 op/s 2024-09-21T07:07:31.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:31 smithi183 ceph-mon[34219]: pgmap v809: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-09-21T07:07:31.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.633+0000 7f2a0d400640 1 -- 172.21.15.114:0/2239884813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 msgr2=0x7f2a0810a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.633+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2239884813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a0810a5c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f29f8009900 tx=0x7f29f802a440 comp rx=0 tx=0).stop 2024-09-21T07:07:31.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 -- 172.21.15.114:0/2239884813 shutdown_connections 2024-09-21T07:07:31.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2239884813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a08110440 0x7f2a08116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2239884813 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a0810faf0 0x7f2a0810ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2239884813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a0810a5c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 -- 172.21.15.114:0/2239884813 >> 172.21.15.114:0/2239884813 conn(0x7f2a080780e0 msgr2=0x7f2a081081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:31.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 -- 172.21.15.114:0/2239884813 shutdown_connections 2024-09-21T07:07:31.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.634+0000 7f2a0d400640 1 -- 172.21.15.114:0/2239884813 wait complete. 2024-09-21T07:07:31.637 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:31 smithi114 ceph-mon[30654]: pgmap v809: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-09-21T07:07:31.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.635+0000 7f2a0d400640 1 Processor -- start 2024-09-21T07:07:31.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 -- start start 2024-09-21T07:07:31.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a081a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a0810faf0 0x7f2a081a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a08110440 0x7f2a081a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0811a140 con 0x7f2a08110440 2024-09-21T07:07:31.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a08119fc0 con 0x7f2a0810a1b0 2024-09-21T07:07:31.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.638+0000 7f2a0d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0811a2c0 con 0x7f2a0810faf0 2024-09-21T07:07:31.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.639+0000 7f2a06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a081a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.639+0000 7f2a06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a081a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36972/0 (socket says 172.21.15.114:36972) 2024-09-21T07:07:31.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.639+0000 7f2a06800640 1 -- 172.21.15.114:0/2781884906 learned_addr learned my addr 172.21.15.114:0/2781884906 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:31.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.639+0000 7f2a05e00640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a0810faf0 0x7f2a081a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.641+0000 7f2a06800640 1 -- 172.21.15.114:0/2781884906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a0810faf0 msgr2=0x7f2a081a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.641+0000 7f2a06800640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a0810faf0 0x7f2a081a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.641+0000 7f2a06800640 1 -- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a08110440 msgr2=0x7f2a081a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.641+0000 7f2a06800640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a08110440 0x7f2a081a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.641+0000 7f2a06800640 1 -- 172.21.15.114:0/2781884906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a081a6a20 con 0x7f2a0810a1b0 2024-09-21T07:07:31.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.641+0000 7f2a06800640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a081a0ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005ab0 tx=0x7f29f8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:31.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.642+0000 7f29ff400640 1 -- 172.21.15.114:0/2781884906 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8040030 con 0x7f2a0810a1b0 2024-09-21T07:07:31.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.642+0000 7f29ff400640 1 -- 172.21.15.114:0/2781884906 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f803c8f0 con 0x7f2a0810a1b0 2024-09-21T07:07:31.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.642+0000 7f29ff400640 1 -- 172.21.15.114:0/2781884906 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8031900 con 0x7f2a0810a1b0 2024-09-21T07:07:31.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.642+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a081a6cb0 con 0x7f2a0810a1b0 2024-09-21T07:07:31.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.643+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a081ad470 con 0x7f2a0810a1b0 2024-09-21T07:07:31.650 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.649+0000 7f29ff400640 1 -- 172.21.15.114:0/2781884906 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f29f8043030 con 0x7f2a0810a1b0 2024-09-21T07:07:31.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.652+0000 7f29ff400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29ec0b6700 0x7f29ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.652+0000 7f29ff400640 1 -- 172.21.15.114:0/2781884906 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f29f81081f0 con 0x7f2a0810a1b0 2024-09-21T07:07:31.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.655+0000 7f2a05e00640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29ec0b6700 0x7f29ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.656+0000 7f2a05e00640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29ec0b6700 0x7f29ec0b8c10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2a0806b110 tx=0x7f29f4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:31.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:31 smithi193 ceph-mon[34601]: pgmap v809: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 2.9 MiB/s wr, 287 op/s 2024-09-21T07:07:31.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.669+0000 7f2a14042740 1 Processor -- start 2024-09-21T07:07:31.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.672+0000 7f2a14042740 1 -- start start 2024-09-21T07:07:31.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.672+0000 7f2a14042740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca22eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.672+0000 7f2a14042740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca22f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.672+0000 7f2a14042740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca22f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca22f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca22f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36978/0 (socket says 172.21.15.114:36978) 2024-09-21T07:07:31.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a14042740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575ca1a5f70 con 0x5575c9e67390 2024-09-21T07:07:31.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 -- 172.21.15.114:0/1732052224 learned_addr learned my addr 172.21.15.114:0/1732052224 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:31.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575ca1953a0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575ca1991b0 con 0x5575c9e88070 2024-09-21T07:07:31.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a06800640 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca22eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.678 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 -- 172.21.15.114:0/1732052224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 msgr2=0x5575ca22eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca22eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 -- 172.21.15.114:0/1732052224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 msgr2=0x5575ca22f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca22f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.673+0000 7f2a05e00640 1 -- 172.21.15.114:0/1732052224 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ca15fc90 con 0x5575c9e5f1b0 2024-09-21T07:07:31.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.674+0000 7f2a05e00640 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca22f0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29f400e600 tx=0x7f29f4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617128f792ae2a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:31.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.677+0000 7f29f3400640 1 -- 172.21.15.114:0/1732052224 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004020 con 0x5575c9e5f1b0 2024-09-21T07:07:31.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.677+0000 7f29f3400640 1 -- 172.21.15.114:0/1732052224 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f40041c0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.680 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.677+0000 7f29f3400640 1 -- 172.21.15.114:0/1732052224 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f401c650 con 0x5575c9e5f1b0 2024-09-21T07:07:31.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.679+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 msgr2=0x5575ca22f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.681 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.679+0000 7f2a14042740 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca22f0a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29f400e600 tx=0x7f29f4014000 comp rx=0 tx=0).stop 2024-09-21T07:07:31.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.684+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 shutdown_connections 2024-09-21T07:07:31.685 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.684+0000 7f2a14042740 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca22f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.684+0000 7f2a14042740 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca22f0a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.684+0000 7f2a14042740 1 --2- 172.21.15.114:0/1732052224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca22eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.684+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 >> 172.21.15.114:0/1732052224 conn(0x5575c9e7c460 msgr2=0x5575ca22ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:31.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.685+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 shutdown_connections 2024-09-21T07:07:31.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.685+0000 7f2a14042740 1 -- 172.21.15.114:0/1732052224 wait complete. 2024-09-21T07:07:31.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.685+0000 7f2a14042740 1 -- start start 2024-09-21T07:07:31.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.686+0000 7f2a14042740 1 Processor -- start 2024-09-21T07:07:31.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.687+0000 7f2a14042740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e5f1b0 0x5575ca23a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.687+0000 7f2a14042740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e67390 0x5575ca23ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.687+0000 7f2a14042740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca246690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.688+0000 7f2a14042740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5575c9ff6700 con 0x5575c9e5f1b0 2024-09-21T07:07:31.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.688+0000 7f2a14042740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5575ca130540 con 0x5575c9e67390 2024-09-21T07:07:31.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.688+0000 7f2a14042740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5575ca23d340 con 0x5575c9e88070 2024-09-21T07:07:31.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.688+0000 7f2a14042740 10 client.0 _ll_register_callbacks cb 0x5575ca140430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca246690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca246690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:41156/0 (socket says 172.21.15.114:41156) 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 -- 172.21.15.114:0/3188812668 learned_addr learned my addr 172.21.15.114:0/3188812668 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a05e00640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e67390 0x5575ca23ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a06800640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e5f1b0 0x5575ca23a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 -- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e67390 msgr2=0x5575ca23ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e67390 0x5575ca23ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 -- 172.21.15.114:0/3188812668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e5f1b0 msgr2=0x5575ca23a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e5f1b0 0x5575ca23a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.690+0000 7f2a07200640 1 -- 172.21.15.114:0/3188812668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c9ef3650 con 0x5575c9e88070 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.691+0000 7f2a07200640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca246690 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000e570 tx=0x7f2a0000ea30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0acdc33e6a9b9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:31.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.691+0000 7f29e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.691+0000 7f29e7e00640 1 -- 172.21.15.114:0/3188812668 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00004d20 con 0x5575c9e88070 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.691+0000 7f29e7e00640 1 -- 172.21.15.114:0/3188812668 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f2a000106a0 con 0x5575c9e88070 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.691+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575ca132b50 con 0x5575c9e88070 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.692+0000 7f29e7e00640 1 -- 172.21.15.114:0/3188812668 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2a00022050 con 0x5575c9e88070 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.692+0000 7f2a14042740 10 client.28759 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.692+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5575ca14d910 con 0x5575c9e88070 2024-09-21T07:07:31.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.692+0000 7f29e7e00640 1 -- 172.21.15.114:0/3188812668 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5575ca132b50 con 0x5575c9e88070 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.693+0000 7f29e7e00640 1 -- 172.21.15.114:0/3188812668 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f2a0000f040 con 0x5575c9e88070 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a14042740 10 client.28759 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a14042740 10 client.28759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a14042740 20 client.28759 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a14042740 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e469f0 0x5575ca24c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a14042740 4 client.28759 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ca1a7ec0 con 0x5575c9e469f0 2024-09-21T07:07:31.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.695+0000 7f2a06800640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e469f0 0x5575ca24c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.696+0000 7f2a06800640 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e469f0 0x5575ca24c7b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f7595080366a31 server_cookie=8079ae146742dd1b in_seq=0 out_seq=0 2024-09-21T07:07:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.696+0000 7f29e7e00640 10 client.28759 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.697+0000 7f29e7e00640 1 -- 172.21.15.114:0/3188812668 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5575ca1a7ec0 con 0x5575c9e469f0 2024-09-21T07:07:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.697+0000 7f29e7e00640 10 client.28759 handle_command_reply: tid=0 2024-09-21T07:07:31.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.697+0000 7f2a14042740 1 client.28759 shutdown 2024-09-21T07:07:31.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 msgr2=0x5575ca246690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca246690 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000e570 tx=0x7f2a0000ea30 comp rx=0 tx=0).stop 2024-09-21T07:07:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 shutdown_connections 2024-09-21T07:07:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e469f0 0x5575ca24c7b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca246690 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e67390 0x5575ca23ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 --2- 172.21.15.114:0/3188812668 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e5f1b0 0x5575ca23a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 >> 172.21.15.114:0/3188812668 conn(0x5575c9e7c460 msgr2=0x5575ca231cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f29e7e00640 0 client.28759 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.700+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 shutdown_connections 2024-09-21T07:07:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.701+0000 7f2a14042740 1 -- 172.21.15.114:0/3188812668 wait complete. 2024-09-21T07:07:31.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.701+0000 7f2a14042740 20 client.28759 trim_cache size 0 max 16384 2024-09-21T07:07:31.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.738+0000 7f2a14042740 1 Processor -- start 2024-09-21T07:07:31.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.741+0000 7f2a14042740 1 -- start start 2024-09-21T07:07:31.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.742+0000 7f2a14042740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.742+0000 7f2a14042740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca232620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.742+0000 7f2a14042740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca23c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.742+0000 7f2a14042740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081b74e0 con 0x5575c9e67390 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.742+0000 7f2a14042740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081b9110 con 0x5575c9e5f1b0 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.742+0000 7f2a14042740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081b7bb0 con 0x5575c9e88070 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a05e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca232620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37008/0 (socket says 172.21.15.114:37008) 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 -- 172.21.15.114:0/1921673889 learned_addr learned my addr 172.21.15.114:0/1921673889 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a07200640 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca23c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 -- 172.21.15.114:0/1921673889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 msgr2=0x5575ca23c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca23c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 -- 172.21.15.114:0/1921673889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 msgr2=0x5575ca232620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca232620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.743+0000 7f2a06800640 1 -- 172.21.15.114:0/1921673889 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ca235150 con 0x5575c9e5f1b0 2024-09-21T07:07:31.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.744+0000 7f2a06800640 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ae80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005f70 tx=0x7f29f80b5040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0dcb1b9aec0a270 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:31.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.744+0000 7f29f2a00640 1 -- 172.21.15.114:0/1921673889 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f804a040 con 0x5575c9e5f1b0 2024-09-21T07:07:31.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.744+0000 7f29f2a00640 1 -- 172.21.15.114:0/1921673889 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f80d2750 con 0x5575c9e5f1b0 2024-09-21T07:07:31.746 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.744+0000 7f29f2a00640 1 -- 172.21.15.114:0/1921673889 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80f10c0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 -- 172.21.15.114:0/1921673889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 msgr2=0x5575ca23ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ae80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005f70 tx=0x7f29f80b5040 comp rx=0 tx=0).stop 2024-09-21T07:07:31.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 -- 172.21.15.114:0/1921673889 shutdown_connections 2024-09-21T07:07:31.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e88070 0x5575ca23c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca232620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 --2- 172.21.15.114:0/1921673889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ae80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 -- 172.21.15.114:0/1921673889 >> 172.21.15.114:0/1921673889 conn(0x5575c9e7c460 msgr2=0x5575ca23b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:31.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 -- 172.21.15.114:0/1921673889 shutdown_connections 2024-09-21T07:07:31.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.746+0000 7f2a14042740 1 -- 172.21.15.114:0/1921673889 wait complete. 2024-09-21T07:07:31.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.747+0000 7f2a14042740 1 -- start start 2024-09-21T07:07:31.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.747+0000 7f2a14042740 1 Processor -- start 2024-09-21T07:07:31.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.748+0000 7f2a14042740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.748+0000 7f2a14042740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca23f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.750 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.748+0000 7f2a14042740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e88070 0x5575ca2b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.749+0000 7f2a14042740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081b89e0 con 0x5575c9e67390 2024-09-21T07:07:31.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.749+0000 7f2a14042740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081b8280 con 0x5575c9e88070 2024-09-21T07:07:31.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.749+0000 7f2a14042740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081b54e0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.749+0000 7f2a14042740 10 client.0 _ll_register_callbacks cb 0x7f2a00009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:31.751 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.749+0000 7f2a06800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.749+0000 7f2a07200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e88070 0x5575ca2b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a07200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e88070 0x5575ca2b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37016/0 (socket says 172.21.15.114:37016) 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a07200640 1 -- 172.21.15.114:0/1160536412 learned_addr learned my addr 172.21.15.114:0/1160536412 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a05e00640 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca23f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a06800640 1 -- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e88070 msgr2=0x5575ca2b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a06800640 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e88070 0x5575ca2b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a06800640 1 -- 172.21.15.114:0/1160536412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 msgr2=0x5575ca23f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:31.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a06800640 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca23f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.750+0000 7f2a06800640 1 -- 172.21.15.114:0/1160536412 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ca2b66b0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.751+0000 7f29e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:31.753 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.751+0000 7f2a06800640 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ec60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5575ca24cdf0 tx=0x7f29f802a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8deecf763da4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:31.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.751+0000 7f29e7e00640 1 -- 172.21.15.114:0/1160536412 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f804a040 con 0x5575c9e5f1b0 2024-09-21T07:07:31.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.751+0000 7f29e7e00640 1 -- 172.21.15.114:0/1160536412 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f29f80f5170 con 0x5575c9e5f1b0 2024-09-21T07:07:31.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.752+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29f80d2750 con 0x5575c9e5f1b0 2024-09-21T07:07:31.754 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.752+0000 7f29e7e00640 1 -- 172.21.15.114:0/1160536412 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f29f80f10c0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.754+0000 7f2a14042740 10 client.28765 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:31.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.754+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5575ca2b78a0 con 0x5575c9e5f1b0 2024-09-21T07:07:31.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.755+0000 7f29e7e00640 1 -- 172.21.15.114:0/1160536412 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f80d2750 con 0x5575c9e5f1b0 2024-09-21T07:07:31.756 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.755+0000 7f29e7e00640 1 -- 172.21.15.114:0/1160536412 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f29f80d7200 con 0x5575c9e5f1b0 2024-09-21T07:07:31.758 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.755+0000 7f2a14042740 10 client.28765 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:31.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.757+0000 7f2a14042740 10 client.28765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:31.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.757+0000 7f2a14042740 20 client.28765 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:31.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.758+0000 7f2a14042740 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e1bda0 0x5575ca2b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:31.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.758+0000 7f2a14042740 4 client.28765 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:07:31.759 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.758+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5575ca1a3970 con 0x5575c9e1bda0 2024-09-21T07:07:31.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.758+0000 7f2a05e00640 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e1bda0 0x5575ca2b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:31.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.759+0000 7f29e7e00640 10 client.28765 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:31.760 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.759+0000 7f2a05e00640 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e1bda0 0x5575ca2b9e20 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607ee01ed15e0f8a server_cookie=4bfc79efa9622245 in_seq=0 out_seq=0 2024-09-21T07:07:31.765 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:07:31.765 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:07:31.765 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:07:31.765 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:06:54.030598+0000", 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.766 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.768 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.769 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:07:31.770 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.771 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.759+0000 7f29e7e00640 1 -- 172.21.15.114:0/1160536412 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x5575ca1a3970 con 0x5575c9e1bda0 2024-09-21T07:07:31.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.759+0000 7f29e7e00640 10 client.28765 handle_command_reply: tid=0 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.759+0000 7f2a14042740 1 client.28765 shutdown 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e5f1b0 msgr2=0x5575ca23ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ec60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x5575ca24cdf0 tx=0x7f29f802a440 comp rx=0 tx=0).stop 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 shutdown_connections 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5575c9e88070 0x5575ca2b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5575c9e67390 0x5575ca23f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5575c9e1bda0 0x5575ca2b9e20 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 --2- 172.21.15.114:0/1160536412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5575c9e5f1b0 0x5575ca23ec60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f29e7e00640 0 client.28765 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 >> 172.21.15.114:0/1160536412 conn(0x5575c9e7c460 msgr2=0x5575ca2323f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.760+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 shutdown_connections 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.761+0000 7f2a14042740 1 -- 172.21.15.114:0/1160536412 wait complete. 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.761+0000 7f2a14042740 20 client.28765 trim_cache size 0 max 16384 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29ec0b6700 msgr2=0x7f29ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29ec0b6700 0x7f29ec0b8c10 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2a0806b110 tx=0x7f29f4005f90 comp rx=0 tx=0).stop 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 msgr2=0x7f2a081a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a081a0ab0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f29f8005ab0 tx=0x7f29f8003f10 comp rx=0 tx=0).stop 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 shutdown_connections 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2a08110440 0x7f2a081a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a0810faf0 0x7f2a081a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29ec0b6700 0x7f29ec0b8c10 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 --2- 172.21.15.114:0/2781884906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a0810a1b0 0x7f2a081a0ab0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 >> 172.21.15.114:0/2781884906 conn(0x7f2a080780e0 msgr2=0x7f2a080788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 shutdown_connections 2024-09-21T07:07:31.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:31.762+0000 7f2a0d400640 1 -- 172.21.15.114:0/2781884906 wait complete. 2024-09-21T07:07:32.796 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:07:32.796 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:07:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 775.0 s, 390.0 tps, lat 251.121 ms stddev 279.294 2024-09-21T07:07:33.438 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:33.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:33 smithi183 ceph-mon[34219]: pgmap v810: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-09-21T07:07:33.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:33 smithi193 ceph-mon[34601]: pgmap v810: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-09-21T07:07:33.673 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:33 smithi114 ceph-mon[30654]: pgmap v810: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-09-21T07:07:34.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 -- 172.21.15.114:0/3280192538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 msgr2=0x7fcc1c109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:34.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 --2- 172.21.15.114:0/3280192538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 0x7fcc1c109700 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009900 tx=0x7fcc0802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 -- 172.21.15.114:0/3280192538 shutdown_connections 2024-09-21T07:07:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 --2- 172.21.15.114:0/3280192538 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc1c109c40 0x7fcc1c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 --2- 172.21.15.114:0/3280192538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 0x7fcc1c109700 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 --2- 172.21.15.114:0/3280192538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c1039b0 0x7fcc1c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 -- 172.21.15.114:0/3280192538 >> 172.21.15.114:0/3280192538 conn(0x7fcc1c0ff440 msgr2=0x7fcc1c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 -- 172.21.15.114:0/3280192538 shutdown_connections 2024-09-21T07:07:34.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.311+0000 7fcc23000640 1 -- 172.21.15.114:0/3280192538 wait complete. 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.313+0000 7fcc23000640 1 Processor -- start 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.313+0000 7fcc23000640 1 -- start start 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.314+0000 7fcc23000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc1c1039b0 0x7fcc1c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.314+0000 7fcc23000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 0x7fcc1c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.315+0000 7fcc23000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 0x7fcc1c1a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.315+0000 7fcc23000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c113950 con 0x7fcc1c1039b0 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.315+0000 7fcc23000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c1137d0 con 0x7fcc1c109c40 2024-09-21T07:07:34.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.315+0000 7fcc23000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c113ad0 con 0x7fcc1c1092f0 2024-09-21T07:07:34.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.316+0000 7fcc22600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 0x7fcc1c1a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc21200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 0x7fcc1c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 0x7fcc1c1a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37028/0 (socket says 172.21.15.114:37028) 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 -- 172.21.15.114:0/1105551392 learned_addr learned my addr 172.21.15.114:0/1105551392 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc21c00640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc1c1039b0 0x7fcc1c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 -- 172.21.15.114:0/1105551392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 msgr2=0x7fcc1c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 0x7fcc1c19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.319 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 -- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc1c1039b0 msgr2=0x7fcc1c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc1c1039b0 0x7fcc1c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.317+0000 7fcc22600640 1 -- 172.21.15.114:0/1105551392 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c1a24b0 con 0x7fcc1c109c40 2024-09-21T07:07:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.318+0000 7fcc22600640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 0x7fcc1c1a1d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009fd0 tx=0x7fcc1400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.318+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14017070 con 0x7fcc1c109c40 2024-09-21T07:07:34.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.318+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c1a27a0 con 0x7fcc1c109c40 2024-09-21T07:07:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.318+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc14003e20 con 0x7fcc1c109c40 2024-09-21T07:07:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.318+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc140106c0 con 0x7fcc1c109c40 2024-09-21T07:07:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.318+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1a8f00 con 0x7fcc1c109c40 2024-09-21T07:07:34.321 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.320+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c103dc0 con 0x7fcc1c109c40 2024-09-21T07:07:34.324 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.322+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fcc14003970 con 0x7fcc1c109c40 2024-09-21T07:07:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.323+0000 7fcc1a000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcc040b6700 0x7fcc040b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.324+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fcc140e7cc0 con 0x7fcc1c109c40 2024-09-21T07:07:34.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.324+0000 7fcc21c00640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcc040b6700 0x7fcc040b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:34.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.325+0000 7fcc21c00640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcc040b6700 0x7fcc040b8c10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009b60 tx=0x7fcc10005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:34.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.327+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc140ee080 con 0x7fcc1c109c40 2024-09-21T07:07:34.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.685+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc1c1a90d0 con 0x7fcc1c109c40 2024-09-21T07:07:34.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.687+0000 7fcc1a000640 1 -- 172.21.15.114:0/1105551392 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11998 (secure 0 0 0) 0x7fcc140adde0 con 0x7fcc1c109c40 2024-09-21T07:07:34.694 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:07:34.694 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":54,"btime":"2024-09-21T07:07:08:168462+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:07:07.936407+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:07:34.694 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 54 2024-09-21T07:07:34.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.692+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcc040b6700 msgr2=0x7fcc040b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:34.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.692+0000 7fcc23000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcc040b6700 0x7fcc040b8c10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009b60 tx=0x7fcc10005f90 comp rx=0 tx=0).stop 2024-09-21T07:07:34.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.692+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 msgr2=0x7fcc1c1a1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:34.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.692+0000 7fcc23000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 0x7fcc1c1a1d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009fd0 tx=0x7fcc1400b260 comp rx=0 tx=0).stop 2024-09-21T07:07:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 shutdown_connections 2024-09-21T07:07:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcc040b6700 0x7fcc040b8c10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc1c109c40 0x7fcc1c1a1d80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc1c1092f0 0x7fcc1c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 --2- 172.21.15.114:0/1105551392 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcc1c1039b0 0x7fcc1c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:34.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 >> 172.21.15.114:0/1105551392 conn(0x7fcc1c0ff440 msgr2=0x7fcc1c106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:34.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 shutdown_connections 2024-09-21T07:07:34.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:34.695+0000 7fcc23000640 1 -- 172.21.15.114:0/1105551392 wait complete. 2024-09-21T07:07:35.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:35 smithi183 ceph-mon[34219]: pgmap v811: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 4.4 MiB/s wr, 451 op/s 2024-09-21T07:07:35.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:35 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1105551392' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:35.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:35 smithi114 ceph-mon[30654]: pgmap v811: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 4.4 MiB/s wr, 451 op/s 2024-09-21T07:07:35.627 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:35 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1105551392' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:35.632 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:07:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:35 smithi193 ceph-mon[34601]: pgmap v811: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 4.4 MiB/s wr, 451 op/s 2024-09-21T07:07:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:35 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1105551392' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:07:36.381 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:37.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.384+0000 7f927f400640 1 -- 172.21.15.114:0/787192100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9278105300 msgr2=0x7f927810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.384+0000 7f927f400640 1 --2- 172.21.15.114:0/787192100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9278105300 0x7f927810d850 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9264009900 tx=0x7f926402a410 comp rx=0 tx=0).stop 2024-09-21T07:07:37.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.384+0000 7f927f400640 1 -- 172.21.15.114:0/787192100 shutdown_connections 2024-09-21T07:07:37.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.385+0000 7f927f400640 1 --2- 172.21.15.114:0/787192100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f927810de10 0x7f9278110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.385+0000 7f927f400640 1 --2- 172.21.15.114:0/787192100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9278105300 0x7f927810d850 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.385+0000 7f927f400640 1 --2- 172.21.15.114:0/787192100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92781049b0 0x7f9278104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.385+0000 7f927f400640 1 -- 172.21.15.114:0/787192100 >> 172.21.15.114:0/787192100 conn(0x7f9278079730 msgr2=0x7f9278079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:37.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.389+0000 7f927f400640 1 -- 172.21.15.114:0/787192100 shutdown_connections 2024-09-21T07:07:37.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.389+0000 7f927f400640 1 -- 172.21.15.114:0/787192100 wait complete. 2024-09-21T07:07:37.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.395+0000 7f927f400640 1 Processor -- start 2024-09-21T07:07:37.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.395+0000 7f927f400640 1 -- start start 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 0x7f92781026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9278105300 0x7f9278102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f927810de10 0x7f9278103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927f400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9278113b70 con 0x7f927810de10 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927f400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f92781139f0 con 0x7f9278105300 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927f400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9278113cf0 con 0x7f92781049b0 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.396+0000 7f927ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 0x7f92781026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.397+0000 7f9277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9278105300 0x7f9278102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.397+0000 7f927ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 0x7f92781026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58240/0 (socket says 172.21.15.114:58240) 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.397+0000 7f927ca00640 1 -- 172.21.15.114:0/1179906798 learned_addr learned my addr 172.21.15.114:0/1179906798 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.397+0000 7f927ca00640 1 -- 172.21.15.114:0/1179906798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9278105300 msgr2=0x7f9278102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.397+0000 7f927ca00640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9278105300 0x7f9278102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.397+0000 7f927ca00640 1 -- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f927810de10 msgr2=0x7f9278103170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.398+0000 7f927ca00640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f927810de10 0x7f9278103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.398+0000 7f927ca00640 1 -- 172.21.15.114:0/1179906798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92781a89a0 con 0x7f92781049b0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.398+0000 7f927ca00640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 0x7f92781026f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f926c01c560 tx=0x7f926c01ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.398+0000 7f9275600640 1 -- 172.21.15.114:0/1179906798 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c020860 con 0x7f92781049b0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.399+0000 7f9275600640 1 -- 172.21.15.114:0/1179906798 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c028400 con 0x7f92781049b0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.399+0000 7f9275600640 1 -- 172.21.15.114:0/1179906798 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c01e650 con 0x7f92781049b0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.399+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927806ce40 con 0x7f92781049b0 2024-09-21T07:07:37.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.399+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927806d300 con 0x7f92781049b0 2024-09-21T07:07:37.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.405+0000 7f9286030740 1 Processor -- start 2024-09-21T07:07:37.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 -- start start 2024-09-21T07:07:37.414 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 0x55e98d6415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d641af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d642030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e98d5affb0 con 0x55e98d2691b0 2024-09-21T07:07:37.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e98d59f3e0 con 0x55e98d271390 2024-09-21T07:07:37.415 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.410+0000 7f9286030740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e98d5a31f0 con 0x55e98d292070 2024-09-21T07:07:37.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.413+0000 7f927d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 0x55e98d6415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.413+0000 7f927d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 0x55e98d6415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58252/0 (socket says 172.21.15.114:58252) 2024-09-21T07:07:37.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.413+0000 7f927d400640 1 -- 172.21.15.114:0/180522618 learned_addr learned my addr 172.21.15.114:0/180522618 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:37.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.414+0000 7f9277e00640 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d642030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.414+0000 7f927d400640 1 -- 172.21.15.114:0/180522618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 msgr2=0x55e98d642030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.416+0000 7f927d400640 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d642030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.416+0000 7f927d400640 1 -- 172.21.15.114:0/180522618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 msgr2=0x55e98d641af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:07:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.416+0000 7f927ca00640 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d641af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.416+0000 7f927d400640 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d641af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.416+0000 7f927d400640 1 -- 172.21.15.114:0/180522618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98d569cd0 con 0x55e98d292070 2024-09-21T07:07:37.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.417+0000 7f927ca00640 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d641af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:37.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.417+0000 7f9275600640 1 -- 172.21.15.114:0/1179906798 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f926c01e040 con 0x7f92781049b0 2024-09-21T07:07:37.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.421+0000 7f927d400640 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 0x55e98d6415b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9270009d20 tx=0x7f927000b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61950cdd7aacbcde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:37.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.421+0000 7f9269c00640 1 -- 172.21.15.114:0/180522618 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927000bd40 con 0x55e98d292070 2024-09-21T07:07:37.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.423+0000 7f9269c00640 1 -- 172.21.15.114:0/180522618 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270017070 con 0x55e98d292070 2024-09-21T07:07:37.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.423+0000 7f9269c00640 1 -- 172.21.15.114:0/180522618 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270011700 con 0x55e98d292070 2024-09-21T07:07:37.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.424+0000 7f9286030740 1 -- 172.21.15.114:0/180522618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 msgr2=0x55e98d6415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.424+0000 7f9286030740 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 0x55e98d6415b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9270009d20 tx=0x7f927000b420 comp rx=0 tx=0).stop 2024-09-21T07:07:37.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.425+0000 7f9286030740 1 -- 172.21.15.114:0/180522618 shutdown_connections 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.425+0000 7f9286030740 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d642030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.425+0000 7f9286030740 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d641af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.425+0000 7f9286030740 1 --2- 172.21.15.114:0/180522618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d292070 0x55e98d6415b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.425+0000 7f9286030740 1 -- 172.21.15.114:0/180522618 >> 172.21.15.114:0/180522618 conn(0x55e98d286460 msgr2=0x55e98d63f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.426+0000 7f9286030740 1 -- 172.21.15.114:0/180522618 shutdown_connections 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.426+0000 7f9275600640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92600d86c0 0x7f92600dabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.426+0000 7f9275600640 1 -- 172.21.15.114:0/1179906798 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f926c0e7560 con 0x7f92781049b0 2024-09-21T07:07:37.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.429+0000 7f9286030740 1 -- 172.21.15.114:0/180522618 wait complete. 2024-09-21T07:07:37.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.429+0000 7f9286030740 1 -- start start 2024-09-21T07:07:37.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.430+0000 7f9286030740 1 Processor -- start 2024-09-21T07:07:37.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9286030740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d64cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9286030740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d64d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9286030740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d292070 0x55e98d659080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9286030740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e98d400700 con 0x55e98d2691b0 2024-09-21T07:07:37.434 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:37 smithi114 ceph-mon[30654]: pgmap v812: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9286030740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e98d53a580 con 0x55e98d292070 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9286030740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e98d64fd90 con 0x55e98d271390 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9277e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d64d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f9277e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d64d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58264/0 (socket says 172.21.15.114:58264) 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9277e00640 1 -- 172.21.15.114:0/2537843749 learned_addr learned my addr 172.21.15.114:0/2537843749 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f927d400640 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d292070 0x55e98d659080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9286030740 10 client.0 _ll_register_callbacks cb 0x55e98d54a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.431+0000 7f927ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d64cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9277e00640 1 -- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d292070 msgr2=0x55e98d659080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9277e00640 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d292070 0x55e98d659080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9277e00640 1 -- 172.21.15.114:0/2537843749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 msgr2=0x55e98d64cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9277e00640 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d64cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.432+0000 7f9277e00640 1 -- 172.21.15.114:0/2537843749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98d2fd650 con 0x55e98d271390 2024-09-21T07:07:37.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.433+0000 7f9277e00640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92600d86c0 0x7f92600dabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.433+0000 7f9277e00640 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d64d340 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9264009900 tx=0x7f92640316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177fa4469bf5e9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:37.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.433+0000 7f925e000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:37.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.434+0000 7f925e000640 1 -- 172.21.15.114:0/2537843749 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926404e060 con 0x55e98d271390 2024-09-21T07:07:37.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.434+0000 7f925e000640 1 -- 172.21.15.114:0/2537843749 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f92640035f0 con 0x55e98d271390 2024-09-21T07:07:37.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.434+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e98d53cb90 con 0x55e98d271390 2024-09-21T07:07:37.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.434+0000 7f9277e00640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92600d86c0 0x7f92600dabd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f926404b000 tx=0x7f9264048000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:37.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.435+0000 7f925e000640 1 -- 172.21.15.114:0/2537843749 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9264035040 con 0x55e98d271390 2024-09-21T07:07:37.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.435+0000 7f9286030740 10 client.28777 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:37.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.435+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e98d557950 con 0x55e98d271390 2024-09-21T07:07:37.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.435+0000 7f925e000640 1 -- 172.21.15.114:0/2537843749 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e98d53cb90 con 0x55e98d271390 2024-09-21T07:07:37.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f925e000640 1 -- 172.21.15.114:0/2537843749 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f926403cb40 con 0x55e98d271390 2024-09-21T07:07:37.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f9286030740 10 client.28777 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:37.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f9286030740 10 client.28777 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:07:37.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f9286030740 20 client.28777 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:37.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f9286030740 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d2509f0 0x55e98d6398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f9286030740 4 client.28777 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:37.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e98d5b1f00 con 0x55e98d2509f0 2024-09-21T07:07:37.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.439+0000 7f927ca00640 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d2509f0 0x55e98d6398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.440+0000 7f927ca00640 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d2509f0 0x55e98d6398b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db07a3e8b9190d83 server_cookie=7d5a71bb84599b86 in_seq=0 out_seq=0 2024-09-21T07:07:37.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.440+0000 7f925e000640 10 client.28777 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:07:37.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.445+0000 7f925e000640 1 -- 172.21.15.114:0/2537843749 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e98d5b1f00 con 0x55e98d2509f0 2024-09-21T07:07:37.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.445+0000 7f925e000640 10 client.28777 handle_command_reply: tid=0 2024-09-21T07:07:37.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.445+0000 7f9286030740 1 client.28777 shutdown 2024-09-21T07:07:37.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.445+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 msgr2=0x55e98d64d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.445+0000 7f9286030740 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d64d340 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9264009900 tx=0x7f92640316d0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 shutdown_connections 2024-09-21T07:07:37.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d2509f0 0x55e98d6398b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d292070 0x55e98d659080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d64d340 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 --2- 172.21.15.114:0/2537843749 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d2691b0 0x55e98d64cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 >> 172.21.15.114:0/2537843749 conn(0x55e98d286460 msgr2=0x55e98d644740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f925e000640 0 client.28777 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:07:37.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.446+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 shutdown_connections 2024-09-21T07:07:37.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.448+0000 7f9286030740 1 -- 172.21.15.114:0/2537843749 wait complete. 2024-09-21T07:07:37.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.449+0000 7f9286030740 20 client.28777 trim_cache size 0 max 16384 2024-09-21T07:07:37.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.485+0000 7f9286030740 1 Processor -- start 2024-09-21T07:07:37.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.485+0000 7f9286030740 1 -- start start 2024-09-21T07:07:37.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9286030740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 0x55e98d644dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9286030740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d658ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9286030740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d659530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9286030740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9278073150 con 0x55e98d292070 2024-09-21T07:07:37.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9286030740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9278074d80 con 0x55e98d2691b0 2024-09-21T07:07:37.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9286030740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9278073820 con 0x55e98d271390 2024-09-21T07:07:37.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9277e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d658ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 0x55e98d644dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 0x55e98d644dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50802/0 (socket says 172.21.15.114:50802) 2024-09-21T07:07:37.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 -- 172.21.15.114:0/2744554931 learned_addr learned my addr 172.21.15.114:0/2744554931 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:37.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.486+0000 7f9277e00640 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d658ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58274/0 (socket says 172.21.15.114:58274) 2024-09-21T07:07:37.489 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927d400640 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d659530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 -- 172.21.15.114:0/2744554931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 msgr2=0x55e98d658ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d658ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 -- 172.21.15.114:0/2744554931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 msgr2=0x55e98d659530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d659530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927ca00640 1 -- 172.21.15.114:0/2744554931 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e98d64e080 con 0x55e98d2691b0 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.487+0000 7f927d400640 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d659530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.488+0000 7f927ca00640 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 0x55e98d644dc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f926c00d000 tx=0x7f926c0e99f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ef13092fdb6d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.488+0000 7f9269200640 1 -- 172.21.15.114:0/2744554931 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0a91f0 con 0x55e98d2691b0 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.488+0000 7f9269200640 1 -- 172.21.15.114:0/2744554931 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c0288f0 con 0x55e98d2691b0 2024-09-21T07:07:37.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.488+0000 7f9269200640 1 -- 172.21.15.114:0/2744554931 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c0ac1e0 con 0x55e98d2691b0 2024-09-21T07:07:37.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 -- 172.21.15.114:0/2744554931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 msgr2=0x55e98d644dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 0x55e98d644dc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f926c00d000 tx=0x7f926c0e99f0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 -- 172.21.15.114:0/2744554931 shutdown_connections 2024-09-21T07:07:37.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d659530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d271390 0x55e98d658ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 --2- 172.21.15.114:0/2744554931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d2691b0 0x55e98d644dc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.492+0000 7f9286030740 1 -- 172.21.15.114:0/2744554931 >> 172.21.15.114:0/2744554931 conn(0x55e98d286460 msgr2=0x55e98d6446b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:37.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.493+0000 7f9286030740 1 -- 172.21.15.114:0/2744554931 shutdown_connections 2024-09-21T07:07:37.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.493+0000 7f9286030740 1 -- 172.21.15.114:0/2744554931 wait complete. 2024-09-21T07:07:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.494+0000 7f9286030740 1 -- start start 2024-09-21T07:07:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.494+0000 7f9286030740 1 Processor -- start 2024-09-21T07:07:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d2691b0 0x55e98d65e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d65eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d6b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9278074650 con 0x55e98d292070 2024-09-21T07:07:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9278073ef0 con 0x55e98d271390 2024-09-21T07:07:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9278071150 con 0x55e98d2691b0 2024-09-21T07:07:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9286030740 10 client.0 _ll_register_callbacks cb 0x7f9264032d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d65eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.495+0000 7f9277e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d65eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50816/0 (socket says 172.21.15.114:50816) 2024-09-21T07:07:37.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.496+0000 7f9277e00640 1 -- 172.21.15.114:0/4115684640 learned_addr learned my addr 172.21.15.114:0/4115684640 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:37.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.496+0000 7f927d400640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d6b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.498 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.496+0000 7f927ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d2691b0 0x55e98d65e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.497+0000 7f9277e00640 1 -- 172.21.15.114:0/4115684640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d2691b0 msgr2=0x55e98d65e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.497+0000 7f9277e00640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d2691b0 0x55e98d65e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.497+0000 7f9277e00640 1 -- 172.21.15.114:0/4115684640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 msgr2=0x55e98d6b9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.497+0000 7f9277e00640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d6b9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.497+0000 7f9277e00640 1 -- 172.21.15.114:0/4115684640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e98d6ba3b0 con 0x55e98d271390 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.497+0000 7f927ca00640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d2691b0 0x55e98d65e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.498+0000 7f9277e00640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d65eb20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f926403cfd0 tx=0x7f9264032e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306bfb8bc0f00f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:37.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.498+0000 7f925e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:37.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.500+0000 7f925e000640 1 -- 172.21.15.114:0/4115684640 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264005850 con 0x55e98d271390 2024-09-21T07:07:37.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.500+0000 7f925e000640 1 -- 172.21.15.114:0/4115684640 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f9264004250 con 0x55e98d271390 2024-09-21T07:07:37.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.500+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c0288f0 con 0x55e98d271390 2024-09-21T07:07:37.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.502+0000 7f925e000640 1 -- 172.21.15.114:0/4115684640 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f926403d950 con 0x55e98d271390 2024-09-21T07:07:37.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.502+0000 7f9286030740 10 client.28808 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:37.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.502+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e98d6c16f0 con 0x55e98d271390 2024-09-21T07:07:37.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.503+0000 7f925e000640 1 -- 172.21.15.114:0/4115684640 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c0288f0 con 0x55e98d271390 2024-09-21T07:07:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.504+0000 7f925e000640 1 -- 172.21.15.114:0/4115684640 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f926403c600 con 0x55e98d271390 2024-09-21T07:07:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.505+0000 7f9286030740 10 client.28808 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.505+0000 7f9286030740 10 client.28808 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:07:37.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.505+0000 7f9286030740 20 client.28808 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.505+0000 7f9286030740 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d225da0 0x55e98d6c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.505+0000 7f9286030740 4 client.28808 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:07:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.505+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e98d5ad9b0 con 0x55e98d225da0 2024-09-21T07:07:37.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.506+0000 7f927ca00640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d225da0 0x55e98d6c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.507+0000 7f925e000640 10 client.28808 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:07:37.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.507+0000 7f927ca00640 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d225da0 0x55e98d6c3ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8df5d5ead5897417 server_cookie=339fc7f0af809545 in_seq=0 out_seq=0 2024-09-21T07:07:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.507+0000 7f925e000640 1 -- 172.21.15.114:0/4115684640 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e98d5ad9b0 con 0x55e98d225da0 2024-09-21T07:07:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.507+0000 7f925e000640 10 client.28808 handle_command_reply: tid=0 2024-09-21T07:07:37.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.507+0000 7f9286030740 1 client.28808 shutdown 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 msgr2=0x55e98d65eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d65eb20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f926403cfd0 tx=0x7f9264032e30 comp rx=0 tx=0).stop 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 shutdown_connections 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e98d292070 0x55e98d6b9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55e98d225da0 0x55e98d6c3ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e98d271390 0x55e98d65eb20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 --2- 172.21.15.114:0/4115684640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e98d2691b0 0x55e98d65e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f925e000640 0 client.28808 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 >> 172.21.15.114:0/4115684640 conn(0x55e98d286460 msgr2=0x55e98d6425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:37.510 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 shutdown_connections 2024-09-21T07:07:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.508+0000 7f9286030740 1 -- 172.21.15.114:0/4115684640 wait complete. 2024-09-21T07:07:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.509+0000 7f9286030740 20 client.28808 trim_cache size 0 max 16384 2024-09-21T07:07:37.511 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:07:37.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92600d86c0 msgr2=0x7f92600dabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92600d86c0 0x7f92600dabd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f926404b000 tx=0x7f9264048000 comp rx=0 tx=0).stop 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 msgr2=0x7f92781026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 0x7f92781026f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f926c01c560 tx=0x7f926c01ca20 comp rx=0 tx=0).stop 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 shutdown_connections 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f927810de10 0x7f9278103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f92600d86c0 0x7f92600dabd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9278105300 0x7f9278102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 --2- 172.21.15.114:0/1179906798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92781049b0 0x7f92781026f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:37.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 >> 172.21.15.114:0/1179906798 conn(0x7f9278079730 msgr2=0x7f9278100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.510+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 shutdown_connections 2024-09-21T07:07:37.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:37.513+0000 7f927f400640 1 -- 172.21.15.114:0/1179906798 wait complete. 2024-09-21T07:07:37.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:37 smithi183 ceph-mon[34219]: pgmap v812: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-09-21T07:07:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:37 smithi193 ceph-mon[34601]: pgmap v812: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-09-21T07:07:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 780.0 s, 422.6 tps, lat 232.767 ms stddev 278.927 2024-09-21T07:07:38.496 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:07:39.296 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:39.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:39 smithi193 ceph-mon[34601]: pgmap v813: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.2 MiB/s wr, 531 op/s 2024-09-21T07:07:39.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:39 smithi114 ceph-mon[30654]: pgmap v813: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.2 MiB/s wr, 531 op/s 2024-09-21T07:07:39.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:39 smithi183 ceph-mon[34219]: pgmap v813: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.2 MiB/s wr, 531 op/s 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 -- 172.21.15.114:0/4128068911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 msgr2=0x7f6d0c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 --2- 172.21.15.114:0/4128068911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c103d20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009900 tx=0x7f6d0002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 -- 172.21.15.114:0/4128068911 shutdown_connections 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 --2- 172.21.15.114:0/4128068911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0c109b90 0x7f6d0c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 --2- 172.21.15.114:0/4128068911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d0c109240 0x7f6d0c109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 --2- 172.21.15.114:0/4128068911 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c103d20 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 -- 172.21.15.114:0/4128068911 >> 172.21.15.114:0/4128068911 conn(0x7f6d0c0ff330 msgr2=0x7f6d0c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 -- 172.21.15.114:0/4128068911 shutdown_connections 2024-09-21T07:07:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.702+0000 7f6d12000640 1 -- 172.21.15.114:0/4128068911 wait complete. 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.703+0000 7f6d12000640 1 Processor -- start 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.703+0000 7f6d12000640 1 -- start start 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d12000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d12000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0c109240 0x7f6d0c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d12000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d0c109b90 0x7f6d0c19fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d12000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c113850 con 0x7f6d0c103910 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d12000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1136d0 con 0x7f6d0c109240 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d12000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1139d0 con 0x7f6d0c109b90 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c19a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55212/0 (socket says 172.21.15.114:55212) 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 -- 172.21.15.114:0/3161506274 learned_addr learned my addr 172.21.15.114:0/3161506274 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d0be00640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0c109240 0x7f6d0c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 -- 172.21.15.114:0/3161506274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d0c109b90 msgr2=0x7f6d0c19fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d0c109b90 0x7f6d0c19fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 -- 172.21.15.114:0/3161506274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0c109240 msgr2=0x7f6d0c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0c109240 0x7f6d0c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.704+0000 7f6d10c00640 1 -- 172.21.15.114:0/3161506274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c1a0200 con 0x7f6d0c103910 2024-09-21T07:07:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.705+0000 7f6d10c00640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c19a2d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005940 tx=0x7f6d000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.705+0000 7f6d09600640 1 -- 172.21.15.114:0/3161506274 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0003f030 con 0x7f6d0c103910 2024-09-21T07:07:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.705+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c1a0490 con 0x7f6d0c103910 2024-09-21T07:07:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.705+0000 7f6d09600640 1 -- 172.21.15.114:0/3161506274 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d000044c0 con 0x7f6d0c103910 2024-09-21T07:07:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.705+0000 7f6d09600640 1 -- 172.21.15.114:0/3161506274 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00035070 con 0x7f6d0c103910 2024-09-21T07:07:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.705+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c1a6c80 con 0x7f6d0c103910 2024-09-21T07:07:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.708+0000 7f6d1a7e7740 1 Processor -- start 2024-09-21T07:07:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.708+0000 7f6d1a7e7740 1 -- start start 2024-09-21T07:07:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.708+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 0x5594c81ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c8200350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e3a4d0 0x5594c8200890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594c81756b0 con 0x5594c7e322f0 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5594c8164ae0 con 0x5594c7e3a4d0 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594c81688f0 con 0x5594c7e5b1b0 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d09600640 1 -- 172.21.15.114:0/3161506274 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6d00042030 con 0x7f6d0c103910 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 0x5594c81ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 0x5594c81ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58326/0 (socket says 172.21.15.114:58326) 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 -- 172.21.15.114:0/2572344027 learned_addr learned my addr 172.21.15.114:0/2572344027 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d0be00640 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e3a4d0 0x5594c8200890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d10c00640 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c8200350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 -- 172.21.15.114:0/2572344027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e3a4d0 msgr2=0x5594c8200890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e3a4d0 0x5594c8200890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 -- 172.21.15.114:0/2572344027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 msgr2=0x5594c8200350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c8200350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.709+0000 7f6d11600640 1 -- 172.21.15.114:0/2572344027 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c8131780 con 0x5594c7e5b1b0 2024-09-21T07:07:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.711+0000 7f6d09600640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cf40b6680 0x7f6cf40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.711+0000 7f6d09600640 1 -- 172.21.15.114:0/3161506274 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f6d00107fe0 con 0x7f6d0c103910 2024-09-21T07:07:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.711+0000 7f6d10c00640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cf40b6680 0x7f6cf40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.712+0000 7f6d10c00640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cf40b6680 0x7f6cf40b8b90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5594c81fea90 tx=0x7f6d0010bab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.717+0000 7f6d11600640 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 0x5594c81ffe10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009ed0 tx=0x7f6d0400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a644feed5e737c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.717+0000 7f6cfdc00640 1 -- 172.21.15.114:0/2572344027 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04004d20 con 0x5594c7e5b1b0 2024-09-21T07:07:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.717+0000 7f6cfdc00640 1 -- 172.21.15.114:0/2572344027 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d04013c80 con 0x5594c7e5b1b0 2024-09-21T07:07:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.717+0000 7f6cfdc00640 1 -- 172.21.15.114:0/2572344027 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04005730 con 0x5594c7e5b1b0 2024-09-21T07:07:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2572344027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 msgr2=0x5594c81ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 0x5594c81ffe10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009ed0 tx=0x7f6d0400e720 comp rx=0 tx=0).stop 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2572344027 shutdown_connections 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e3a4d0 0x5594c8200890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c8200350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2572344027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e5b1b0 0x5594c81ffe10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2572344027 >> 172.21.15.114:0/2572344027 conn(0x5594c7e4f300 msgr2=0x5594c81fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2572344027 shutdown_connections 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.718+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2572344027 wait complete. 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.719+0000 7f6d1a7e7740 1 -- start start 2024-09-21T07:07:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.719+0000 7f6d1a7e7740 1 Processor -- start 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.719+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c820b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c820b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c820be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5594c809cab0 con 0x5594c7e322f0 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5594c810b570 con 0x5594c7e5b1b0 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5594c820e590 con 0x5594c7e3a4d0 2024-09-21T07:07:40.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d1a7e7740 10 client.0 _ll_register_callbacks cb 0x5594c811a100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d11600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c820b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d11600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c820b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55234/0 (socket says 172.21.15.114:55234) 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d11600640 1 -- 172.21.15.114:0/2685515033 learned_addr learned my addr 172.21.15.114:0/2685515033 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d0be00640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c820b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d11600640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c820be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d0be00640 1 -- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 msgr2=0x5594c820be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d0be00640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c820be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d0be00640 1 -- 172.21.15.114:0/2685515033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 msgr2=0x5594c820b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.720+0000 7f6d0be00640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c820b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6d0be00640 1 -- 172.21.15.114:0/2685515033 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c7ec6790 con 0x5594c7e3a4d0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6d11600640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c820b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6d0be00640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c820b940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c06c400 tx=0x7f6cf8012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a03a0dff85f47be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6cf2000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6cf2000640 1 -- 172.21.15.114:0/2685515033 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8004020 con 0x5594c7e3a4d0 2024-09-21T07:07:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6cf2000640 1 -- 172.21.15.114:0/2685515033 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f6cf80041c0 con 0x5594c7e3a4d0 2024-09-21T07:07:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.721+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d04013c80 con 0x5594c7e3a4d0 2024-09-21T07:07:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6cf2000640 1 -- 172.21.15.114:0/2685515033 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6cf8025050 con 0x5594c7e3a4d0 2024-09-21T07:07:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 10 client.28792 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5594c7f3f620 con 0x5594c7e3a4d0 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6cf2000640 1 -- 172.21.15.114:0/2685515033 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d04013c80 con 0x5594c7e3a4d0 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6cf2000640 1 -- 172.21.15.114:0/2685515033 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f6cf8003a40 con 0x5594c7e3a4d0 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 10 client.28792 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 10 client.28792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 20 client.28792 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7e19d80 0x5594c821d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 4 client.28792 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.722+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c8177600 con 0x5594c7e19d80 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.723+0000 7f6d11600640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7e19d80 0x5594c821d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.723+0000 7f6cf2000640 10 client.28792 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.723+0000 7f6d11600640 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7e19d80 0x5594c821d660 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9646b71da04de40 server_cookie=7ffe84b9bd7f574d in_seq=0 out_seq=0 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.724+0000 7f6cf2000640 1 -- 172.21.15.114:0/2685515033 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5594c8177600 con 0x5594c7e19d80 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.724+0000 7f6cf2000640 10 client.28792 handle_command_reply: tid=0 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.724+0000 7f6d1a7e7740 1 client.28792 shutdown 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 msgr2=0x5594c820b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c820b940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c06c400 tx=0x7f6cf8012b60 comp rx=0 tx=0).stop 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 shutdown_connections 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c820be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7e19d80 0x5594c821d660 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c820b940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6cf2000640 0 client.28792 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/2685515033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e322f0 0x5594c820b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 >> 172.21.15.114:0/2685515033 conn(0x5594c7e4f300 msgr2=0x5594c8200e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 shutdown_connections 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/2685515033 wait complete. 2024-09-21T07:07:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.725+0000 7f6d1a7e7740 20 client.28792 trim_cache size 0 max 16384 2024-09-21T07:07:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.740+0000 7f6d1a7e7740 1 Processor -- start 2024-09-21T07:07:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.740+0000 7f6d1a7e7740 1 -- start start 2024-09-21T07:07:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e322f0 0x5594c81fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 0x5594c820d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c82085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1aebc0 con 0x5594c7e3a4d0 2024-09-21T07:07:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1b07f0 con 0x5594c7e5b1b0 2024-09-21T07:07:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1af290 con 0x5594c7e322f0 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 0x5594c820d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 0x5594c820d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55240/0 (socket says 172.21.15.114:55240) 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d11600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e322f0 0x5594c81fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 -- 172.21.15.114:0/3178990997 learned_addr learned my addr 172.21.15.114:0/3178990997 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d11600640 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c82085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 -- 172.21.15.114:0/3178990997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e322f0 msgr2=0x5594c81fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e322f0 0x5594c81fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 -- 172.21.15.114:0/3178990997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 msgr2=0x5594c82085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c82085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.741+0000 7f6d0be00640 1 -- 172.21.15.114:0/3178990997 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c82098e0 con 0x5594c7e3a4d0 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.742+0000 7f6d0be00640 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 0x5594c820d030 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800a010 tx=0x7f6cf80096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7674355f34e90ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.742+0000 7f6cfd200640 1 -- 172.21.15.114:0/3178990997 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf802e070 con 0x5594c7e3a4d0 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.742+0000 7f6cfd200640 1 -- 172.21.15.114:0/3178990997 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf801c470 con 0x5594c7e3a4d0 2024-09-21T07:07:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.742+0000 7f6cfd200640 1 -- 172.21.15.114:0/3178990997 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8022040 con 0x5594c7e3a4d0 2024-09-21T07:07:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3178990997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 msgr2=0x5594c820d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 0x5594c820d030 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800a010 tx=0x7f6cf80096a0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3178990997 shutdown_connections 2024-09-21T07:07:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e5b1b0 0x5594c82085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e3a4d0 0x5594c820d030 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3178990997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e322f0 0x5594c81fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3178990997 >> 172.21.15.114:0/3178990997 conn(0x5594c7e4f300 msgr2=0x5594c820c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3178990997 shutdown_connections 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3178990997 wait complete. 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 -- start start 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.743+0000 7f6d1a7e7740 1 Processor -- start 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e322f0 0x5594c8213e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c8211320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 0x5594c8211860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1b00c0 con 0x5594c7e5b1b0 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1af960 con 0x5594c7e322f0 2024-09-21T07:07:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1acbc0 con 0x5594c7e3a4d0 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d1a7e7740 10 client.0 _ll_register_callbacks cb 0x7f6cf8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d11600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e322f0 0x5594c8213e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d11600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 0x5594c8211860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c8211320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d11600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 0x5594c8211860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55254/0 (socket says 172.21.15.114:55254) 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d11600640 1 -- 172.21.15.114:0/3803421628 learned_addr learned my addr 172.21.15.114:0/3803421628 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.744+0000 7f6d0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c8211320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58358/0 (socket says 172.21.15.114:58358) 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d11600640 1 -- 172.21.15.114:0/3803421628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 msgr2=0x5594c8211320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d11600640 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c8211320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d11600640 1 -- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e322f0 msgr2=0x5594c8213e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d11600640 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e322f0 0x5594c8213e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d11600640 1 -- 172.21.15.114:0/3803421628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594c8211ea0 con 0x5594c7e5b1b0 2024-09-21T07:07:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d11600640 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 0x5594c8211860 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x5594c8209bf0 tx=0x7f6d0400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a56726ed9911d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6cf2000640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6cf2000640 1 -- 172.21.15.114:0/3803421628 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04003970 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6cf2000640 1 -- 172.21.15.114:0/3803421628 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f6d04021040 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cf801c470 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6cf2000640 1 -- 172.21.15.114:0/3803421628 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6d04003b10 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d1a7e7740 10 client.18252 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.745+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5594c8287e10 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6cf2000640 1 -- 172.21.15.114:0/3803421628 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf801c470 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6cf2000640 1 -- 172.21.15.114:0/3803421628 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f6d04004030 con 0x5594c7e5b1b0 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d1a7e7740 10 client.18252 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d1a7e7740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d1a7e7740 20 client.18252 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7def1c0 0x5594c828a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d1a7e7740 4 client.18252 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:07:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594c81730b0 con 0x5594c7def1c0 2024-09-21T07:07:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.746+0000 7f6d0be00640 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7def1c0 0x5594c828a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.747+0000 7f6d0be00640 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7def1c0 0x5594c828a410 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a417240946264ba server_cookie=75148b08db178bf1 in_seq=0 out_seq=0 2024-09-21T07:07:40.753 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.747+0000 7f6cf2000640 10 client.18252 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.942+0000 7f6cf2000640 1 -- 172.21.15.114:0/3803421628 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594c81730b0 con 0x5594c7def1c0 2024-09-21T07:07:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.942+0000 7f6cf2000640 10 client.18252 handle_command_reply: tid=0 2024-09-21T07:07:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.942+0000 7f6d1a7e7740 1 client.18252 shutdown 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 msgr2=0x5594c8211860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 0x5594c8211860 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x5594c8209bf0 tx=0x7f6d0400e720 comp rx=0 tx=0).stop 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 shutdown_connections 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5594c7e5b1b0 0x5594c8211860 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5594c7def1c0 0x5594c828a410 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5594c7e3a4d0 0x5594c8211320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 --2- 172.21.15.114:0/3803421628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5594c7e322f0 0x5594c8213e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 >> 172.21.15.114:0/3803421628 conn(0x5594c7e4f300 msgr2=0x5594c81fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.943+0000 7f6cf2000640 0 client.18252 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.944+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 shutdown_connections 2024-09-21T07:07:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.944+0000 7f6d1a7e7740 1 -- 172.21.15.114:0/3803421628 wait complete. 2024-09-21T07:07:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.944+0000 7f6d1a7e7740 20 client.18252 trim_cache size 0 max 16384 2024-09-21T07:07:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:07:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cf40b6680 msgr2=0x7f6cf40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cf40b6680 0x7f6cf40b8b90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x5594c81fea90 tx=0x7f6d0010bab0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 msgr2=0x7f6d0c19a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c19a2d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f6d00005940 tx=0x7f6d000040c0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 shutdown_connections 2024-09-21T07:07:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d0c109b90 0x7f6d0c19fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6cf40b6680 0x7f6cf40b8b90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d0c109240 0x7f6d0c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 --2- 172.21.15.114:0/3161506274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6d0c103910 0x7f6d0c19a2d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.945+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 >> 172.21.15.114:0/3161506274 conn(0x7f6d0c0ff330 msgr2=0x7f6d0c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.947+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 shutdown_connections 2024-09-21T07:07:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:40.948+0000 7f6d12000640 1 -- 172.21.15.114:0/3161506274 wait complete. 2024-09-21T07:07:41.343 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.341+0000 7fa635800640 1 -- 172.21.15.114:0/3727343673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 msgr2=0x7fa630110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.341+0000 7fa635800640 1 --2- 172.21.15.114:0/3727343673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa630110300 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa628009ca0 tx=0x7fa62802a8e0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.342+0000 7fa635800640 1 -- 172.21.15.114:0/3727343673 shutdown_connections 2024-09-21T07:07:41.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.342+0000 7fa635800640 1 --2- 172.21.15.114:0/3727343673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa630110300 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.342+0000 7fa635800640 1 --2- 172.21.15.114:0/3727343673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.342+0000 7fa635800640 1 --2- 172.21.15.114:0/3727343673 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 0x7fa630104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.343+0000 7fa635800640 1 -- 172.21.15.114:0/3727343673 >> 172.21.15.114:0/3727343673 conn(0x7fa630079730 msgr2=0x7fa630079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.343+0000 7fa635800640 1 -- 172.21.15.114:0/3727343673 shutdown_connections 2024-09-21T07:07:41.345 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.343+0000 7fa635800640 1 -- 172.21.15.114:0/3727343673 wait complete. 2024-09-21T07:07:41.346 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.345+0000 7fa635800640 1 Processor -- start 2024-09-21T07:07:41.349 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 -- start start 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 0x7fa63010ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa63010b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa630104140 con 0x7fa6301049b0 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa630103fc0 con 0x7fa630105300 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.348+0000 7fa635800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301042c0 con 0x7fa63010de10 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.349+0000 7fa62ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 0x7fa63010ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.349+0000 7fa62f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa63010b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.351 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.349+0000 7fa62f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa63010b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58384/0 (socket says 172.21.15.114:58384) 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.349+0000 7fa62e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.349+0000 7fa62e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50866/0 (socket says 172.21.15.114:50866) 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.349+0000 7fa62e000640 1 -- 172.21.15.114:0/4035333123 learned_addr learned my addr 172.21.15.114:0/4035333123 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.350+0000 7fa62e000640 1 -- 172.21.15.114:0/4035333123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 msgr2=0x7fa63010b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.350+0000 7fa62e000640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa63010b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.350+0000 7fa62e000640 1 -- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 msgr2=0x7fa63010ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.352 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.350+0000 7fa62e000640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 0x7fa63010ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.351+0000 7fa62e000640 1 -- 172.21.15.114:0/4035333123 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6301acb50 con 0x7fa630105300 2024-09-21T07:07:41.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.351+0000 7fa62f400640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa63010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:41.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.351+0000 7fa62ea00640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 0x7fa63010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:41.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.351+0000 7fa62e000640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010b360 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00d740 tx=0x7fa61c00dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.352+0000 7fa627400640 1 -- 172.21.15.114:0/4035333123 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c017070 con 0x7fa630105300 2024-09-21T07:07:41.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.352+0000 7fa627400640 1 -- 172.21.15.114:0/4035333123 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c00de50 con 0x7fa630105300 2024-09-21T07:07:41.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.352+0000 7fa627400640 1 -- 172.21.15.114:0/4035333123 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00f620 con 0x7fa630105300 2024-09-21T07:07:41.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.352+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6301ace40 con 0x7fa630105300 2024-09-21T07:07:41.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.352+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6301ad240 con 0x7fa630105300 2024-09-21T07:07:41.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.357+0000 7fa63c407740 1 Processor -- start 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.358+0000 7fa63c407740 1 -- start start 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.358+0000 7fa627400640 1 -- 172.21.15.114:0/4035333123 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa61c01a070 con 0x7fa630105300 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.359+0000 7fa63c407740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df00da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.359+0000 7fa63c407740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db311b0 0x55f32df012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa62f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df00da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa62f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df00da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50868/0 (socket says 172.21.15.114:50868) 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa63c407740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa63c407740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f32de77fb0 con 0x55f32db39390 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa63c407740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f32de673e0 con 0x55f32db5a070 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa63c407740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f32de6b1f0 con 0x55f32db311b0 2024-09-21T07:07:41.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.360+0000 7fa62f400640 1 -- 172.21.15.114:0/1518172096 learned_addr learned my addr 172.21.15.114:0/1518172096 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:41.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.361+0000 7fa62f400640 1 -- 172.21.15.114:0/1518172096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db311b0 msgr2=0x55f32df012e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:07:41.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.361+0000 7fa62ea00640 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db311b0 0x55f32df012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.363+0000 7fa62f400640 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db311b0 0x55f32df012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.363+0000 7fa62f400640 1 -- 172.21.15.114:0/1518172096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 msgr2=0x55f32df01820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.363+0000 7fa62f400640 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.363+0000 7fa62f400640 1 -- 172.21.15.114:0/1518172096 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32de31cd0 con 0x55f32db5a070 2024-09-21T07:07:41.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.363+0000 7fa62ea00640 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db311b0 0x55f32df012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:41.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.364+0000 7fa62f400640 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df00da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa628009ca0 tx=0x7fa628009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48931030bbccd08f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.364+0000 7fa61be00640 1 -- 172.21.15.114:0/1518172096 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628031c50 con 0x55f32db5a070 2024-09-21T07:07:41.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.364+0000 7fa61be00640 1 -- 172.21.15.114:0/1518172096 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa628036040 con 0x55f32db5a070 2024-09-21T07:07:41.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.364+0000 7fa61be00640 1 -- 172.21.15.114:0/1518172096 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280305e0 con 0x55f32db5a070 2024-09-21T07:07:41.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa627400640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6140b6730 0x7fa6140b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 -- 172.21.15.114:0/1518172096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 msgr2=0x55f32df00da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df00da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa628009ca0 tx=0x7fa628009a20 comp rx=0 tx=0).stop 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 -- 172.21.15.114:0/1518172096 shutdown_connections 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db311b0 0x55f32df012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 --2- 172.21.15.114:0/1518172096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df00da0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.366+0000 7fa63c407740 1 -- 172.21.15.114:0/1518172096 >> 172.21.15.114:0/1518172096 conn(0x55f32db4e460 msgr2=0x55f32defec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.367+0000 7fa63c407740 1 -- 172.21.15.114:0/1518172096 shutdown_connections 2024-09-21T07:07:41.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.367+0000 7fa627400640 1 -- 172.21.15.114:0/4035333123 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fa61c0e7350 con 0x7fa630105300 2024-09-21T07:07:41.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.367+0000 7fa63c407740 1 -- 172.21.15.114:0/1518172096 wait complete. 2024-09-21T07:07:41.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.368+0000 7fa62ea00640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6140b6730 0x7fa6140b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.368+0000 7fa63c407740 1 -- start start 2024-09-21T07:07:41.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.368+0000 7fa63c407740 1 Processor -- start 2024-09-21T07:07:41.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.369+0000 7fa63c407740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.369+0000 7fa63c407740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db39390 0x55f32df0cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa63c407740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df18840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa63c407740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f32dcc8700 con 0x55f32db311b0 2024-09-21T07:07:41.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa63c407740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f32de02580 con 0x55f32db5a070 2024-09-21T07:07:41.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa63c407740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f32df0f520 con 0x55f32db39390 2024-09-21T07:07:41.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa63c407740 10 client.0 _ll_register_callbacks cb 0x55f32de12470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:41.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa62f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df18840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.371+0000 7fa62f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df18840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50874/0 (socket says 172.21.15.114:50874) 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.371+0000 7fa62f400640 1 -- 172.21.15.114:0/3525573195 learned_addr learned my addr 172.21.15.114:0/3525573195 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.371+0000 7fa62ea00640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.371+0000 7fa62f400640 1 -- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db39390 msgr2=0x55f32df0cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.370+0000 7fa62e000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db39390 0x55f32df0cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.372+0000 7fa62f400640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db39390 0x55f32df0cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.372+0000 7fa62f400640 1 -- 172.21.15.114:0/3525573195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 msgr2=0x55f32df0c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.372+0000 7fa62f400640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.372+0000 7fa62f400640 1 -- 172.21.15.114:0/3525573195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32dbc5650 con 0x55f32db5a070 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.373+0000 7fa62ea00640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6140b6730 0x7fa6140b8c40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa6200095f0 tx=0x7fa620005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.373+0000 7fa62e000640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db39390 0x55f32df0cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.373+0000 7fa62f400640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df18840 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa628004ac0 tx=0x7fa628031d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c2b6ea8de59681 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.373+0000 7fa618c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:41.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.373+0000 7fa618c00640 1 -- 172.21.15.114:0/3525573195 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62804e070 con 0x55f32db5a070 2024-09-21T07:07:41.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.374+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f32de04b90 con 0x55f32db5a070 2024-09-21T07:07:41.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.374+0000 7fa618c00640 1 -- 172.21.15.114:0/3525573195 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fa628003c40 con 0x55f32db5a070 2024-09-21T07:07:41.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.375+0000 7fa618c00640 1 -- 172.21.15.114:0/3525573195 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa628058050 con 0x55f32db5a070 2024-09-21T07:07:41.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.375+0000 7fa63c407740 10 client.28838 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:41.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.375+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f32de1f950 con 0x55f32db5a070 2024-09-21T07:07:41.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.377+0000 7fa618c00640 1 -- 172.21.15.114:0/3525573195 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f32de04b90 con 0x55f32db5a070 2024-09-21T07:07:41.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.377+0000 7fa618c00640 1 -- 172.21.15.114:0/3525573195 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7fa628053440 con 0x55f32db5a070 2024-09-21T07:07:41.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.378+0000 7fa63c407740 10 client.28838 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:41.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.378+0000 7fa63c407740 10 client.28838 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:07:41.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.378+0000 7fa63c407740 20 client.28838 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:41.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.378+0000 7fa63c407740 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32db189f0 0x55f32df1e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.378+0000 7fa63c407740 4 client.28838 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:41.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.378+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f32de79f00 con 0x55f32db189f0 2024-09-21T07:07:41.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.381+0000 7fa62ea00640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32db189f0 0x55f32df1e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.383+0000 7fa62ea00640 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32db189f0 0x55f32df1e4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1cac1247d7187fe0 server_cookie=ed69f9abdb68f562 in_seq=0 out_seq=0 2024-09-21T07:07:41.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.383+0000 7fa618c00640 10 client.28838 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:07:41.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.384+0000 7fa618c00640 1 -- 172.21.15.114:0/3525573195 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f32de79f00 con 0x55f32db189f0 2024-09-21T07:07:41.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.384+0000 7fa618c00640 10 client.28838 handle_command_reply: tid=0 2024-09-21T07:07:41.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.384+0000 7fa63c407740 1 client.28838 shutdown 2024-09-21T07:07:41.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.384+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 msgr2=0x55f32df18840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.384+0000 7fa63c407740 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df18840 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa628004ac0 tx=0x7fa628031d00 comp rx=0 tx=0).stop 2024-09-21T07:07:41.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 shutdown_connections 2024-09-21T07:07:41.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32db189f0 0x55f32df1e4c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db5a070 0x55f32df18840 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db39390 0x55f32df0cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 --2- 172.21.15.114:0/3525573195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 >> 172.21.15.114:0/3525573195 conn(0x55f32db4e460 msgr2=0x55f32defd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa618c00640 0 client.28838 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 shutdown_connections 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 1 -- 172.21.15.114:0/3525573195 wait complete. 2024-09-21T07:07:41.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.385+0000 7fa63c407740 20 client.28838 trim_cache size 0 max 16384 2024-09-21T07:07:41.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.414+0000 7fa63c407740 1 Processor -- start 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.414+0000 7fa63c407740 1 -- start start 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.415+0000 7fa63c407740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.415+0000 7fa63c407740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db39390 0x55f32df09810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.415+0000 7fa63c407740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df09d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.415+0000 7fa63c407740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b7570 con 0x55f32db311b0 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.415+0000 7fa63c407740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b91a0 con 0x55f32db39390 2024-09-21T07:07:41.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.415+0000 7fa63c407740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b7c40 con 0x55f32db5a070 2024-09-21T07:07:41.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.416+0000 7fa62e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db39390 0x55f32df09810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.416+0000 7fa62ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.416+0000 7fa62f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df09d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.416+0000 7fa62ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:55314/0 (socket says 172.21.15.114:55314) 2024-09-21T07:07:41.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.416+0000 7fa62f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df09d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58398/0 (socket says 172.21.15.114:58398) 2024-09-21T07:07:41.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.416+0000 7fa62ea00640 1 -- 172.21.15.114:0/3523828370 learned_addr learned my addr 172.21.15.114:0/3523828370 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:41.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.418+0000 7fa62f400640 1 -- 172.21.15.114:0/3523828370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db39390 msgr2=0x55f32df09810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.419 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.418+0000 7fa62f400640 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db39390 0x55f32df09810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.418+0000 7fa62f400640 1 -- 172.21.15.114:0/3523828370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 msgr2=0x55f32df0e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.418+0000 7fa62f400640 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.420 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.418+0000 7fa62f400640 1 -- 172.21.15.114:0/3523828370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f32df04eb0 con 0x55f32db5a070 2024-09-21T07:07:41.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.419+0000 7fa62f400640 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df09d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa628053a00 tx=0x7fa628009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c87bc8cf4cffa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.419+0000 7fa61b400640 1 -- 172.21.15.114:0/3523828370 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628054e60 con 0x55f32db5a070 2024-09-21T07:07:41.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.420+0000 7fa61b400640 1 -- 172.21.15.114:0/3523828370 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa628036050 con 0x55f32db5a070 2024-09-21T07:07:41.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.420+0000 7fa61b400640 1 -- 172.21.15.114:0/3523828370 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62804e070 con 0x55f32db5a070 2024-09-21T07:07:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.423+0000 7fa63c407740 1 -- 172.21.15.114:0/3523828370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 msgr2=0x55f32df09d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.423+0000 7fa63c407740 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df09d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa628053a00 tx=0x7fa628009a20 comp rx=0 tx=0).stop 2024-09-21T07:07:41.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.424+0000 7fa63c407740 1 -- 172.21.15.114:0/3523828370 shutdown_connections 2024-09-21T07:07:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.424+0000 7fa63c407740 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df09d50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.424+0000 7fa63c407740 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db39390 0x55f32df09810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.424+0000 7fa63c407740 1 --2- 172.21.15.114:0/3523828370 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db311b0 0x55f32df0e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.424+0000 7fa63c407740 1 -- 172.21.15.114:0/3523828370 >> 172.21.15.114:0/3523828370 conn(0x55f32db4e460 msgr2=0x55f32df0db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.425+0000 7fa63c407740 1 -- 172.21.15.114:0/3523828370 shutdown_connections 2024-09-21T07:07:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.425+0000 7fa63c407740 1 -- 172.21.15.114:0/3523828370 wait complete. 2024-09-21T07:07:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.426+0000 7fa63c407740 1 -- start start 2024-09-21T07:07:41.428 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.426+0000 7fa63c407740 1 Processor -- start 2024-09-21T07:07:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db311b0 0x55f32df21d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df02010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b8a70 con 0x55f32db39390 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b8310 con 0x55f32db311b0 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301b5570 con 0x55f32db5a070 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.429+0000 7fa63c407740 10 client.0 _ll_register_callbacks cb 0x7fa628004930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.430+0000 7fa62ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db311b0 0x55f32df21d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.430+0000 7fa62e000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.430+0000 7fa62f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df02010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.430+0000 7fa62f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df02010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:58404/0 (socket says 172.21.15.114:58404) 2024-09-21T07:07:41.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.430+0000 7fa62f400640 1 -- 172.21.15.114:0/3402927303 learned_addr learned my addr 172.21.15.114:0/3402927303 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:41.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.431+0000 7fa62f400640 1 -- 172.21.15.114:0/3402927303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db311b0 msgr2=0x55f32df21d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.431+0000 7fa62f400640 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db311b0 0x55f32df21d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.431+0000 7fa62f400640 1 -- 172.21.15.114:0/3402927303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 msgr2=0x55f32df01a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:41.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.431+0000 7fa62f400640 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.431+0000 7fa62f400640 1 -- 172.21.15.114:0/3402927303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f32df027a0 con 0x55f32db5a070 2024-09-21T07:07:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.432+0000 7fa62ea00640 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db311b0 0x55f32df21d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.434+0000 7fa62f400640 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df02010 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa62803e040 tx=0x7fa6280095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db25ec3fc0de7bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:41.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.434+0000 7fa618c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:41.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.437+0000 7fa618c00640 1 -- 172.21.15.114:0/3402927303 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62804e040 con 0x55f32db5a070 2024-09-21T07:07:41.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.437+0000 7fa618c00640 1 -- 172.21.15.114:0/3402927303 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fa628004050 con 0x55f32db5a070 2024-09-21T07:07:41.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.438+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa628036050 con 0x55f32db5a070 2024-09-21T07:07:41.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.438+0000 7fa618c00640 1 -- 172.21.15.114:0/3402927303 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa62806b050 con 0x55f32db5a070 2024-09-21T07:07:41.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.439+0000 7fa63c407740 10 client.28810 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:41.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.439+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f32df99c40 con 0x55f32db5a070 2024-09-21T07:07:41.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.439+0000 7fa618c00640 1 -- 172.21.15.114:0/3402927303 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa628036050 con 0x55f32db5a070 2024-09-21T07:07:41.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa618c00640 1 -- 172.21.15.114:0/3402927303 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7fa62805d440 con 0x55f32db5a070 2024-09-21T07:07:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa63c407740 10 client.28810 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa63c407740 10 client.28810 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:07:41.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa63c407740 20 client.28810 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa63c407740 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32daedda0 0x55f32df9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa63c407740 4 client.28810 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:07:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.440+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f32de759b0 con 0x55f32daedda0 2024-09-21T07:07:41.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.441+0000 7fa62ea00640 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32daedda0 0x55f32df9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.442+0000 7fa62ea00640 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32daedda0 0x55f32df9c370 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21afae0222d3f86e server_cookie=40c4b238600f7538 in_seq=0 out_seq=0 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.442+0000 7fa618c00640 10 client.28810 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.442+0000 7fa618c00640 1 -- 172.21.15.114:0/3402927303 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f32de759b0 con 0x55f32daedda0 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.442+0000 7fa618c00640 10 client.28810 handle_command_reply: tid=0 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.442+0000 7fa63c407740 1 client.28810 shutdown 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 msgr2=0x55f32df02010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df02010 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa62803e040 tx=0x7fa6280095c0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 shutdown_connections 2024-09-21T07:07:41.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55f32daedda0 0x55f32df9c370 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f32db5a070 0x55f32df02010 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f32db39390 0x55f32df01a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 --2- 172.21.15.114:0/3402927303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f32db311b0 0x55f32df21d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 >> 172.21.15.114:0/3402927303 conn(0x55f32db4e460 msgr2=0x55f32df03e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.443+0000 7fa618c00640 0 client.28810 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.444+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 shutdown_connections 2024-09-21T07:07:41.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.444+0000 7fa63c407740 1 -- 172.21.15.114:0/3402927303 wait complete. 2024-09-21T07:07:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.445+0000 7fa63c407740 20 client.28810 trim_cache size 0 max 16384 2024-09-21T07:07:41.447 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:07:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6140b6730 msgr2=0x7fa6140b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6140b6730 0x7fa6140b8c40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa6200095f0 tx=0x7fa620005f90 comp rx=0 tx=0).stop 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 msgr2=0x7fa63010b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010b360 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00d740 tx=0x7fa61c00dc00 comp rx=0 tx=0).stop 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 shutdown_connections 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa63010de10 0x7fa63010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6140b6730 0x7fa6140b8c40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa630105300 0x7fa63010b360 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 --2- 172.21.15.114:0/4035333123 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa6301049b0 0x7fa63010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 >> 172.21.15.114:0/4035333123 conn(0x7fa630079730 msgr2=0x7fa630114150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:41.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 shutdown_connections 2024-09-21T07:07:41.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:41.446+0000 7fa635800640 1 -- 172.21.15.114:0/4035333123 wait complete. 2024-09-21T07:07:41.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:41 smithi193 ceph-mon[34601]: pgmap v814: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 4.1 MiB/s wr, 475 op/s 2024-09-21T07:07:41.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:41 smithi114 ceph-mon[30654]: pgmap v814: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 4.1 MiB/s wr, 475 op/s 2024-09-21T07:07:41.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:41 smithi183 ceph-mon[34219]: pgmap v814: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 4.1 MiB/s wr, 475 op/s 2024-09-21T07:07:42.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 785.0 s, 466.2 tps, lat 217.492 ms stddev 232.302 2024-09-21T07:07:43.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:43 smithi114 ceph-mon[30654]: pgmap v815: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-09-21T07:07:43.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:43 smithi183 ceph-mon[34219]: pgmap v815: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-09-21T07:07:43.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:43 smithi193 ceph-mon[34601]: pgmap v815: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-09-21T07:07:44.625 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:07:45.276 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:45.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:45 smithi114 ceph-mon[30654]: pgmap v816: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 5.2 MiB/s wr, 551 op/s 2024-09-21T07:07:45.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:45 smithi183 ceph-mon[34219]: pgmap v816: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 5.2 MiB/s wr, 551 op/s 2024-09-21T07:07:45.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:45 smithi193 ceph-mon[34601]: pgmap v816: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 5.2 MiB/s wr, 551 op/s 2024-09-21T07:07:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:47 smithi114 ceph-mon[30654]: pgmap v817: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-09-21T07:07:47.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 790.0 s, 431.4 tps, lat 227.019 ms stddev 243.302 2024-09-21T07:07:47.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:47 smithi183 ceph-mon[34219]: pgmap v817: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.896+0000 7f5a77200640 1 -- 172.21.15.114:0/252299157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 msgr2=0x7f5a78110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.896+0000 7f5a77200640 1 --2- 172.21.15.114:0/252299157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 0x7f5a78110470 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009ca0 tx=0x7f5a7002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.897+0000 7f5a77200640 1 -- 172.21.15.114:0/252299157 shutdown_connections 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.897+0000 7f5a77200640 1 --2- 172.21.15.114:0/252299157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 0x7f5a78110470 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.897+0000 7f5a77200640 1 --2- 172.21.15.114:0/252299157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781092f0 0x7f5a78109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.897+0000 7f5a77200640 1 --2- 172.21.15.114:0/252299157 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a781039b0 0x7f5a78103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.899 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.897+0000 7f5a77200640 1 -- 172.21.15.114:0/252299157 >> 172.21.15.114:0/252299157 conn(0x7f5a780ff440 msgr2=0x7f5a78101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:47.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.900+0000 7f5a77200640 1 -- 172.21.15.114:0/252299157 shutdown_connections 2024-09-21T07:07:47.902 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.900+0000 7f5a77200640 1 -- 172.21.15.114:0/252299157 wait complete. 2024-09-21T07:07:47.903 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.902+0000 7f5a77200640 1 Processor -- start 2024-09-21T07:07:47.904 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.902+0000 7f5a77200640 1 -- start start 2024-09-21T07:07:47.905 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.903+0000 7f5a77200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 0x7f5a78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.906 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.903+0000 7f5a77200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a781092f0 0x7f5a78075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.903+0000 7f5a77200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 0x7f5a78076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.903+0000 7f5a77200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78113950 con 0x7f5a781092f0 2024-09-21T07:07:47.908 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.903+0000 7f5a77200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781137d0 con 0x7f5a781039b0 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.903+0000 7f5a77200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a78113ad0 con 0x7f5a78109c40 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.904+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 0x7f5a78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.904+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 0x7f5a78075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36280/0 (socket says 172.21.15.114:36280) 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.904+0000 7f5a75e00640 1 -- 172.21.15.114:0/651352473 learned_addr learned my addr 172.21.15.114:0/651352473 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.904+0000 7f5a76800640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 0x7f5a78076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.904+0000 7f5a75400640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a781092f0 0x7f5a78075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.905+0000 7f5a75e00640 1 -- 172.21.15.114:0/651352473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 msgr2=0x7f5a78076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.905+0000 7f5a75e00640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 0x7f5a78076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.905+0000 7f5a75e00640 1 -- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a781092f0 msgr2=0x7f5a78075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:47.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.905+0000 7f5a75e00640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a781092f0 0x7f5a78075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.905+0000 7f5a75e00640 1 -- 172.21.15.114:0/651352473 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a781a8370 con 0x7f5a781039b0 2024-09-21T07:07:47.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.908+0000 7f5a75e00640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 0x7f5a78075710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c890 tx=0x7f5a6800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:47.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.908+0000 7f5a6ea00640 1 -- 172.21.15.114:0/651352473 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004220 con 0x7f5a781039b0 2024-09-21T07:07:47.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.908+0000 7f5a6ea00640 1 -- 172.21.15.114:0/651352473 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a680043c0 con 0x7f5a781039b0 2024-09-21T07:07:47.911 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.908+0000 7f5a6ea00640 1 -- 172.21.15.114:0/651352473 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004a00 con 0x7f5a781039b0 2024-09-21T07:07:47.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:47 smithi193 ceph-mon[34601]: pgmap v817: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 4.4 MiB/s wr, 479 op/s 2024-09-21T07:07:47.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.911+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a781a8660 con 0x7f5a781039b0 2024-09-21T07:07:47.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.911+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a781a8b20 con 0x7f5a781039b0 2024-09-21T07:07:47.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.920+0000 7f5a6ea00640 1 -- 172.21.15.114:0/651352473 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5a680079b0 con 0x7f5a781039b0 2024-09-21T07:07:47.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.927+0000 7f5a6ea00640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a5c0b6780 0x7f5a5c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.931 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.928+0000 7f5a6ea00640 1 -- 172.21.15.114:0/651352473 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f5a680e7b60 con 0x7f5a781039b0 2024-09-21T07:07:47.932 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.928+0000 7f5a75400640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a5c0b6780 0x7f5a5c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.932+0000 7f5a75400640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a5c0b6780 0x7f5a5c0b8c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5a78102c90 tx=0x7f5a6400a4c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:47.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.940+0000 7f5a83cb8740 1 Processor -- start 2024-09-21T07:07:47.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 -- start start 2024-09-21T07:07:47.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x5643625f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643625f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643625f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56436256ffb0 con 0x5643622291b0 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56436255f3e0 con 0x564362231390 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.943+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5643625631f0 con 0x564362252070 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x5643625f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x5643625f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56068/0 (socket says 172.21.15.114:56068) 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a75e00640 1 -- 172.21.15.114:0/477942100 learned_addr learned my addr 172.21.15.114:0/477942100 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a76800640 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643625f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a75400640 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643625f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a76800640 1 -- 172.21.15.114:0/477942100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 msgr2=0x5643625f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a76800640 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x5643625f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.948 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a76800640 1 -- 172.21.15.114:0/477942100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 msgr2=0x5643625f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a76800640 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643625f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a76800640 1 -- 172.21.15.114:0/477942100 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564362529cd0 con 0x564362231390 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.944+0000 7f5a75400640 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643625f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.945+0000 7f5a76800640 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643625f99f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5a7002aea0 tx=0x7f5a70001140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5fd17265e04d09e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.945+0000 7f5a63400640 1 -- 172.21.15.114:0/477942100 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70030b80 con 0x564362231390 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.945+0000 7f5a63400640 1 -- 172.21.15.114:0/477942100 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70036030 con 0x564362231390 2024-09-21T07:07:47.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.945+0000 7f5a63400640 1 -- 172.21.15.114:0/477942100 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a700045a0 con 0x564362231390 2024-09-21T07:07:47.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.947+0000 7f5a83cb8740 1 -- 172.21.15.114:0/477942100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 msgr2=0x5643625f99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:47.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.947+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643625f99f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5a7002aea0 tx=0x7f5a70001140 comp rx=0 tx=0).stop 2024-09-21T07:07:47.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.949+0000 7f5a83cb8740 1 -- 172.21.15.114:0/477942100 shutdown_connections 2024-09-21T07:07:47.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.949+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643625f99f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.949+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643625f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.949+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/477942100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x5643625f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.949+0000 7f5a83cb8740 1 -- 172.21.15.114:0/477942100 >> 172.21.15.114:0/477942100 conn(0x564362246460 msgr2=0x5643625f6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:47.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.949+0000 7f5a83cb8740 1 -- 172.21.15.114:0/477942100 shutdown_connections 2024-09-21T07:07:47.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.950+0000 7f5a83cb8740 1 -- 172.21.15.114:0/477942100 wait complete. 2024-09-21T07:07:47.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.950+0000 7f5a83cb8740 1 -- start start 2024-09-21T07:07:47.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.951+0000 7f5a83cb8740 1 Processor -- start 2024-09-21T07:07:47.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x564362604c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x564362619220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5643623c0700 con 0x5643622291b0 2024-09-21T07:07:47.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5643624fa580 con 0x564362231390 2024-09-21T07:07:47.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56436260ff00 con 0x564362252070 2024-09-21T07:07:47.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.952+0000 7f5a83cb8740 10 client.0 _ll_register_callbacks cb 0x56436250a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:47.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a76800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x564362619220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.955 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x564362604c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:37234/0 (socket says 172.21.15.114:37234) 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75e00640 1 -- 172.21.15.114:0/417159071 learned_addr learned my addr 172.21.15.114:0/417159071 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75400640 1 -- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 msgr2=0x564362619220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75400640 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x564362619220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.953+0000 7f5a75400640 1 -- 172.21.15.114:0/417159071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 msgr2=0x5643626046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a75400640 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a75400640 1 -- 172.21.15.114:0/417159071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643622bd650 con 0x564362231390 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a76800640 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x564362619220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a75400640 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x564362604c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5643625f7bf0 tx=0x7f5a640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a5e2c99d00c0f37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:47.956 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a57e00640 1 -- 172.21.15.114:0/417159071 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400fa10 con 0x564362231390 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.954+0000 7f5a57e00640 1 -- 172.21.15.114:0/417159071 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f5a6400fbb0 con 0x564362231390 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.955+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5643624fcb90 con 0x564362231390 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.955+0000 7f5a57e00640 1 -- 172.21.15.114:0/417159071 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5a6400fdb0 con 0x564362231390 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.955+0000 7f5a83cb8740 10 client.28853 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.955+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564362517950 con 0x564362231390 2024-09-21T07:07:47.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.955+0000 7f5a57e00640 1 -- 172.21.15.114:0/417159071 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5643624fcb90 con 0x564362231390 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a57e00640 1 -- 172.21.15.114:0/417159071 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f5a6402f420 con 0x564362231390 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a83cb8740 10 client.28853 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a83cb8740 10 client.28853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a83cb8740 20 client.28853 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5643622109f0 0x56436261f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a83cb8740 4 client.28853 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:47.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.957+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564362571f00 con 0x5643622109f0 2024-09-21T07:07:47.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.958+0000 7f5a75e00640 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5643622109f0 0x56436261f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.961+0000 7f5a57e00640 10 client.28853 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:07:47.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.961+0000 7f5a75e00640 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5643622109f0 0x56436261f410 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9bcaf4c563ee3cf0 server_cookie=b8052be6d6f8afb9 in_seq=0 out_seq=0 2024-09-21T07:07:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.962+0000 7f5a57e00640 1 -- 172.21.15.114:0/417159071 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564362571f00 con 0x5643622109f0 2024-09-21T07:07:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.962+0000 7f5a57e00640 10 client.28853 handle_command_reply: tid=0 2024-09-21T07:07:47.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.962+0000 7f5a83cb8740 1 client.28853 shutdown 2024-09-21T07:07:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 msgr2=0x564362604c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x564362604c20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5643625f7bf0 tx=0x7f5a640046c0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 shutdown_connections 2024-09-21T07:07:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x5643622109f0 0x56436261f410 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a57e00640 0 client.28853 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:07:47.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362252070 0x564362619220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x564362604c20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/417159071 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.963+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 >> 172.21.15.114:0/417159071 conn(0x564362246460 msgr2=0x5643625fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.964+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 shutdown_connections 2024-09-21T07:07:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.964+0000 7f5a83cb8740 1 -- 172.21.15.114:0/417159071 wait complete. 2024-09-21T07:07:47.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.964+0000 7f5a83cb8740 20 client.28853 trim_cache size 0 max 16384 2024-09-21T07:07:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 Processor -- start 2024-09-21T07:07:47.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 -- start start 2024-09-21T07:07:47.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622109f0 0x564362604ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643622291b0 0x564362619120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643626018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781aeb40 con 0x5643622109f0 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781b0770 con 0x564362231390 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.996+0000 7f5a83cb8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781af210 con 0x5643622291b0 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.997+0000 7f5a76800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643626018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.997+0000 7f5a76800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643626018d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36306/0 (socket says 172.21.15.114:36306) 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.997+0000 7f5a76800640 1 -- 172.21.15.114:0/1441093025 learned_addr learned my addr 172.21.15.114:0/1441093025 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.997+0000 7f5a75400640 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643622291b0 0x564362619120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.997+0000 7f5a76800640 1 -- 172.21.15.114:0/1441093025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643622291b0 msgr2=0x564362619120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:48.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.997+0000 7f5a76800640 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643622291b0 0x564362619120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.998+0000 7f5a76800640 1 -- 172.21.15.114:0/1441093025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622109f0 msgr2=0x564362604ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.998+0000 7f5a76800640 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622109f0 0x564362604ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.998+0000 7f5a76800640 1 -- 172.21.15.114:0/1441093025 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436261f9e0 con 0x564362231390 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.998+0000 7f5a76800640 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643626018d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009ca0 tx=0x7f5a70005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd20692c41340225 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.999+0000 7f5a62a00640 1 -- 172.21.15.114:0/1441093025 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7004a040 con 0x564362231390 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.999+0000 7f5a62a00640 1 -- 172.21.15.114:0/1441093025 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70036040 con 0x564362231390 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:47.999+0000 7f5a62a00640 1 -- 172.21.15.114:0/1441093025 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a700045e0 con 0x564362231390 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.000+0000 7f5a83cb8740 1 -- 172.21.15.114:0/1441093025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 msgr2=0x5643626018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:48.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.000+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643626018d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009ca0 tx=0x7f5a70005fd0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 -- 172.21.15.114:0/1441093025 shutdown_connections 2024-09-21T07:07:48.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564362231390 0x5643626018d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5643622291b0 0x564362619120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/1441093025 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622109f0 0x564362604ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 -- 172.21.15.114:0/1441093025 >> 172.21.15.114:0/1441093025 conn(0x564362246460 msgr2=0x564362604490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:48.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 -- 172.21.15.114:0/1441093025 shutdown_connections 2024-09-21T07:07:48.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.001+0000 7f5a83cb8740 1 -- 172.21.15.114:0/1441093025 wait complete. 2024-09-21T07:07:48.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.002+0000 7f5a83cb8740 1 -- start start 2024-09-21T07:07:48.004 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.002+0000 7f5a83cb8740 1 Processor -- start 2024-09-21T07:07:48.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.006+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 0x564362615190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.006+0000 7f5a83cb8740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.006+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 0x564362615190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:48.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.006+0000 7f5a75e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 0x564362615190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36314/0 (socket says 172.21.15.114:36314) 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a75e00640 1 -- 172.21.15.114:0/3629491033 learned_addr learned my addr 172.21.15.114:0/3629491033 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362231390 0x564362610140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781b0040 con 0x5643622291b0 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781af8e0 con 0x5643622109f0 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a781acb60 con 0x564362231390 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a83cb8740 10 client.0 _ll_register_callbacks cb 0x7f5a64010450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a76800640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362231390 0x564362610140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a75400640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:48.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a75e00640 1 -- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362231390 msgr2=0x564362610140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:48.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.007+0000 7f5a75e00640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362231390 0x564362610140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a75e00640 1 -- 172.21.15.114:0/3629491033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 msgr2=0x5643626156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:48.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a75e00640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a75e00640 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564362615c40 con 0x5643622109f0 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a75400640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a75e00640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 0x564362615190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5a68005fd0 tx=0x7f5a680a9420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d5245f813a0c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a57e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a57e00640 1 -- 172.21.15.114:0/3629491033 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a680e5310 con 0x5643622109f0 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.008+0000 7f5a57e00640 1 -- 172.21.15.114:0/3629491033 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f5a680a90d0 con 0x5643622109f0 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.009+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a70036040 con 0x5643622109f0 2024-09-21T07:07:48.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.009+0000 7f5a57e00640 1 -- 172.21.15.114:0/3629491033 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5a68005290 con 0x5643622109f0 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.009+0000 7f5a83cb8740 10 client.28859 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.009+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56436260d700 con 0x5643622109f0 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.009+0000 7f5a57e00640 1 -- 172.21.15.114:0/3629491033 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70036040 con 0x5643622109f0 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.009+0000 7f5a57e00640 1 -- 172.21.15.114:0/3629491033 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f5a680c8bd0 con 0x5643622109f0 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a83cb8740 10 client.28859 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a83cb8740 10 client.28859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a83cb8740 20 client.28859 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564362252070 0x56436260de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a83cb8740 4 client.28859 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:07:48.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56436256d9b0 con 0x564362252070 2024-09-21T07:07:48.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.010+0000 7f5a75400640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564362252070 0x56436260de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:48.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a75400640 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564362252070 0x56436260de90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42f36cef6b1d155b server_cookie=c5ed0f0185377672 in_seq=0 out_seq=0 2024-09-21T07:07:48.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a57e00640 10 client.28859 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:07:48.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a57e00640 1 -- 172.21.15.114:0/3629491033 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56436256d9b0 con 0x564362252070 2024-09-21T07:07:48.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a57e00640 10 client.28859 handle_command_reply: tid=0 2024-09-21T07:07:48.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a83cb8740 1 client.28859 shutdown 2024-09-21T07:07:48.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 msgr2=0x564362615190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:48.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.011+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 0x564362615190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5a68005fd0 tx=0x7f5a680a9420 comp rx=0 tx=0).stop 2024-09-21T07:07:48.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 shutdown_connections 2024-09-21T07:07:48.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x564362252070 0x56436260de90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564362231390 0x564362610140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a57e00640 0 client.28859 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5643622291b0 0x5643626156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 --2- 172.21.15.114:0/3629491033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5643622109f0 0x564362615190 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 >> 172.21.15.114:0/3629491033 conn(0x564362246460 msgr2=0x5643625f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 shutdown_connections 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 1 -- 172.21.15.114:0/3629491033 wait complete. 2024-09-21T07:07:48.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.013+0000 7f5a83cb8740 20 client.28859 trim_cache size 0 max 16384 2024-09-21T07:07:48.016 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a5c0b6780 msgr2=0x7f5a5c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a5c0b6780 0x7f5a5c0b8c90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5a78102c90 tx=0x7f5a6400a4c0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 msgr2=0x7f5a78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 0x7f5a78075710 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800c890 tx=0x7f5a6800cd50 comp rx=0 tx=0).stop 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 shutdown_connections 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a78109c40 0x7f5a78076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a781092f0 0x7f5a78075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a5c0b6780 0x7f5a5c0b8c90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 --2- 172.21.15.114:0/651352473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a781039b0 0x7f5a78075710 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 >> 172.21.15.114:0/651352473 conn(0x7f5a780ff440 msgr2=0x7f5a78107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 shutdown_connections 2024-09-21T07:07:48.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:48.015+0000 7f5a77200640 1 -- 172.21.15.114:0/651352473 wait complete. 2024-09-21T07:07:49.046 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:07:49.743 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:49.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:49 smithi114 ceph-mon[30654]: pgmap v818: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 5.0 MiB/s wr, 508 op/s 2024-09-21T07:07:49.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:49 smithi183 ceph-mon[34219]: pgmap v818: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 5.0 MiB/s wr, 508 op/s 2024-09-21T07:07:49.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:49 smithi193 ceph-mon[34601]: pgmap v818: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 5.0 MiB/s wr, 508 op/s 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.568+0000 7f5e24800640 1 -- 172.21.15.114:0/1175264622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c116640 msgr2=0x7f5e1c118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.568+0000 7f5e24800640 1 --2- 172.21.15.114:0/1175264622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c116640 0x7f5e1c118b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009ca0 tx=0x7f5e1802a8c0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.569+0000 7f5e24800640 1 -- 172.21.15.114:0/1175264622 shutdown_connections 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.569+0000 7f5e24800640 1 --2- 172.21.15.114:0/1175264622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c116640 0x7f5e1c118b50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.569+0000 7f5e24800640 1 --2- 172.21.15.114:0/1175264622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10aee0 0x7f5e1c115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.569+0000 7f5e24800640 1 --2- 172.21.15.114:0/1175264622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e1c10a590 0x7f5e1c10a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.569+0000 7f5e24800640 1 -- 172.21.15.114:0/1175264622 >> 172.21.15.114:0/1175264622 conn(0x7f5e1c070ec0 msgr2=0x7f5e1c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.569+0000 7f5e24800640 1 -- 172.21.15.114:0/1175264622 shutdown_connections 2024-09-21T07:07:50.573 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.570+0000 7f5e24800640 1 -- 172.21.15.114:0/1175264622 wait complete. 2024-09-21T07:07:50.574 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.572+0000 7f5e24800640 1 Processor -- start 2024-09-21T07:07:50.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.577+0000 7f5e24800640 1 -- start start 2024-09-21T07:07:50.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.577+0000 7f5e24800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10a590 0x7f5e1c10be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.578+0000 7f5e24800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c10aee0 0x7f5e1c10c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.578+0000 7f5e24800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e1c116640 0x7f5e1c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.578+0000 7f5e24800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c11c330 con 0x7f5e1c116640 2024-09-21T07:07:50.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.578+0000 7f5e24800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c11c1b0 con 0x7f5e1c10aee0 2024-09-21T07:07:50.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.578+0000 7f5e24800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c11c4b0 con 0x7f5e1c10a590 2024-09-21T07:07:50.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.580+0000 7f5e21e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10a590 0x7f5e1c10be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.580+0000 7f5e21e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10a590 0x7f5e1c10be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56112/0 (socket says 172.21.15.114:56112) 2024-09-21T07:07:50.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.580+0000 7f5e21e00640 1 -- 172.21.15.114:0/3945146524 learned_addr learned my addr 172.21.15.114:0/3945146524 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:50.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.580+0000 7f5e21400640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c10aee0 0x7f5e1c10c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.581+0000 7f5e22800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e1c116640 0x7f5e1c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.581+0000 7f5e21400640 1 -- 172.21.15.114:0/3945146524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10a590 msgr2=0x7f5e1c10be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.581+0000 7f5e21400640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10a590 0x7f5e1c10be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.581+0000 7f5e21400640 1 -- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e1c116640 msgr2=0x7f5e1c10c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.581+0000 7f5e21400640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e1c116640 0x7f5e1c10c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.581+0000 7f5e21400640 1 -- 172.21.15.114:0/3945146524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c1b0f00 con 0x7f5e1c10aee0 2024-09-21T07:07:50.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.582+0000 7f5e21400640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c10aee0 0x7f5e1c10c3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c10bd30 tx=0x7f5e0c00cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:50.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.582+0000 7f5e16a00640 1 -- 172.21.15.114:0/3945146524 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c007bb0 con 0x7f5e1c10aee0 2024-09-21T07:07:50.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.583+0000 7f5e16a00640 1 -- 172.21.15.114:0/3945146524 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0c007d50 con 0x7f5e1c10aee0 2024-09-21T07:07:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.583+0000 7f5e16a00640 1 -- 172.21.15.114:0/3945146524 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c004d80 con 0x7f5e1c10aee0 2024-09-21T07:07:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.583+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c1b11f0 con 0x7f5e1c10aee0 2024-09-21T07:07:50.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.583+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c1b16e0 con 0x7f5e1c10aee0 2024-09-21T07:07:50.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.590+0000 7f5e16a00640 1 -- 172.21.15.114:0/3945146524 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5e0c004f20 con 0x7f5e1c10aee0 2024-09-21T07:07:50.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.591+0000 7f5e2b476740 1 Processor -- start 2024-09-21T07:07:50.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.591+0000 7f5e2b476740 1 -- start start 2024-09-21T07:07:50.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 0x5589b17b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.595 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13e3390 0x5589b17b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e2b476740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5589b1721fb0 con 0x5589b1404070 2024-09-21T07:07:50.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e2b476740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5589b17113e0 con 0x5589b13db1b0 2024-09-21T07:07:50.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e2b476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589b17151f0 con 0x5589b13e3390 2024-09-21T07:07:50.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e21e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 0x5589b17b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e21e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 0x5589b17b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36344/0 (socket says 172.21.15.114:36344) 2024-09-21T07:07:50.596 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.592+0000 7f5e21e00640 1 -- 172.21.15.114:0/3016278337 learned_addr learned my addr 172.21.15.114:0/3016278337 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:50.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e21400640 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13e3390 0x5589b17b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e22800640 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e21e00640 1 -- 172.21.15.114:0/3016278337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13e3390 msgr2=0x5589b17b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e21e00640 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13e3390 0x5589b17b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e21e00640 1 -- 172.21.15.114:0/3016278337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 msgr2=0x5589b17b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e21e00640 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e21e00640 1 -- 172.21.15.114:0/3016278337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b16dbcd0 con 0x5589b13db1b0 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.593+0000 7f5e22800640 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.594+0000 7f5e21e00640 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 0x5589b17b3af0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000b860 tx=0x7f5e1000bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5fc8cd83716fc7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:50.598 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.594+0000 7f5e16a00640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e040b67d0 0x7f5e040b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.595+0000 7f5e16a00640 1 -- 172.21.15.114:0/3945146524 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f5e0c0e77a0 con 0x7f5e1c10aee0 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.595+0000 7f5e22800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e040b67d0 0x7f5e040b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.595+0000 7f5e0b000640 1 -- 172.21.15.114:0/3016278337 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10003dd0 con 0x5589b13db1b0 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.595+0000 7f5e0b000640 1 -- 172.21.15.114:0/3016278337 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e10015070 con 0x5589b13db1b0 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.595+0000 7f5e0b000640 1 -- 172.21.15.114:0/3016278337 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100117b0 con 0x5589b13db1b0 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.596+0000 7f5e2b476740 1 -- 172.21.15.114:0/3016278337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 msgr2=0x5589b17b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 0x5589b17b3af0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000b860 tx=0x7f5e1000bd20 comp rx=0 tx=0).stop 2024-09-21T07:07:50.599 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e22800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e040b67d0 0x7f5e040b8ce0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e18006010 tx=0x7f5e18031f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 -- 172.21.15.114:0/3016278337 shutdown_connections 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13e3390 0x5589b17b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13db1b0 0x5589b17b3af0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 --2- 172.21.15.114:0/3016278337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 -- 172.21.15.114:0/3016278337 >> 172.21.15.114:0/3016278337 conn(0x5589b13f8460 msgr2=0x5589b17b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 -- 172.21.15.114:0/3016278337 shutdown_connections 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.597+0000 7f5e2b476740 1 -- 172.21.15.114:0/3016278337 wait complete. 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.598+0000 7f5e2b476740 1 -- start start 2024-09-21T07:07:50.600 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.598+0000 7f5e2b476740 1 Processor -- start 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13db1b0 0x5589b17ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 0x5589b17add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5589b1572700 con 0x5589b1404070 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5589b16ac580 con 0x5589b13e3390 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589b17b08c0 con 0x5589b13db1b0 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e2b476740 10 client.0 _ll_register_callbacks cb 0x5589b16bc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:50.601 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e21400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 0x5589b17add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e21400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 0x5589b17add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36356/0 (socket says 172.21.15.114:36356) 2024-09-21T07:07:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.599+0000 7f5e21400640 1 -- 172.21.15.114:0/1318429192 learned_addr learned my addr 172.21.15.114:0/1318429192 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e22800640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.602 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13db1b0 0x5589b17ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21400640 1 -- 172.21.15.114:0/1318429192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13db1b0 msgr2=0x5589b17ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21400640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13db1b0 0x5589b17ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21400640 1 -- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 msgr2=0x5589b17ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.603 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21400640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21400640 1 -- 172.21.15.114:0/1318429192 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b146f650 con 0x5589b13e3390 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13db1b0 0x5589b17ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5e21400640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 0x5589b17add30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ad1d0 tx=0x7f5e0c00a460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24686d957643d65c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.600+0000 7f5dff400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.601+0000 7f5dff400640 1 -- 172.21.15.114:0/1318429192 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c0ad010 con 0x5589b13e3390 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.601+0000 7f5dff400640 1 -- 172.21.15.114:0/1318429192 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f5e0c005210 con 0x5589b13e3390 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.601+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589b16aeb90 con 0x5589b13e3390 2024-09-21T07:07:50.604 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.602+0000 7f5dff400640 1 -- 172.21.15.114:0/1318429192 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5e0c0ac9b0 con 0x5589b13e3390 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.602+0000 7f5e2b476740 10 client.28871 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.602+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5589b16c9950 con 0x5589b13e3390 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.602+0000 7f5dff400640 1 -- 172.21.15.114:0/1318429192 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5589b16aeb90 con 0x5589b13e3390 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.602+0000 7f5dff400640 1 -- 172.21.15.114:0/1318429192 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f5e0c0a8b90 con 0x5589b13e3390 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e2b476740 10 client.28871 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e2b476740 10 client.28871 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e2b476740 20 client.28871 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b13c29f0 0x5589b17d9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e2b476740 4 client.28871 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:50.605 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b1723f00 con 0x5589b13c29f0 2024-09-21T07:07:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.603+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b13c29f0 0x5589b17d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.604+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b13c29f0 0x5589b17d9770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1d62904bee8d22d server_cookie=c32a764d24d2f1f0 in_seq=0 out_seq=0 2024-09-21T07:07:50.606 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.604+0000 7f5dff400640 10 client.28871 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:07:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.605+0000 7f5dff400640 1 -- 172.21.15.114:0/1318429192 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589b1723f00 con 0x5589b13c29f0 2024-09-21T07:07:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.605+0000 7f5dff400640 10 client.28871 handle_command_reply: tid=0 2024-09-21T07:07:50.607 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 client.28871 shutdown 2024-09-21T07:07:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 msgr2=0x5589b17add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 0x5589b17add30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ad1d0 tx=0x7f5e0c00a460 comp rx=0 tx=0).stop 2024-09-21T07:07:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 shutdown_connections 2024-09-21T07:07:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b13c29f0 0x5589b17d9770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.608 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b1404070 0x5589b17ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b13e3390 0x5589b17add30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1318429192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b13db1b0 0x5589b17ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.606+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 >> 172.21.15.114:0/1318429192 conn(0x5589b13f8460 msgr2=0x5589b17b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.607+0000 7f5dff400640 0 client.28871 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.607+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 shutdown_connections 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.607+0000 7f5e2b476740 1 -- 172.21.15.114:0/1318429192 wait complete. 2024-09-21T07:07:50.609 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.607+0000 7f5e2b476740 20 client.28871 trim_cache size 0 max 16384 2024-09-21T07:07:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.624+0000 7f5e2b476740 1 Processor -- start 2024-09-21T07:07:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.624+0000 7f5e2b476740 1 -- start start 2024-09-21T07:07:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1404070 0x5589b17aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1397da0 0x5589b17aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e2b476740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0000ff80 con 0x5589b13db1b0 2024-09-21T07:07:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e2b476740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00011bb0 con 0x5589b1397da0 2024-09-21T07:07:50.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e2b476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00010650 con 0x5589b1404070 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e22800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1397da0 0x5589b17aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e21400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1404070 0x5589b17aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e22800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1397da0 0x5589b17aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36368/0 (socket says 172.21.15.114:36368) 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e22800640 1 -- 172.21.15.114:0/1410664837 learned_addr learned my addr 172.21.15.114:0/1410664837 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.625+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e21400640 1 -- 172.21.15.114:0/1410664837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1397da0 msgr2=0x5589b17aaf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e21400640 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1397da0 0x5589b17aaf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e21400640 1 -- 172.21.15.114:0/1410664837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 msgr2=0x5589b17b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e21400640 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e21400640 1 -- 172.21.15.114:0/1410664837 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b17a2f90 con 0x5589b1404070 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e21400640 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1404070 0x5589b17aa9e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0a97c0 tx=0x7f5e0c004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aced51c56d8f5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e0a600640 1 -- 172.21.15.114:0/1410664837 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c0acb90 con 0x5589b1404070 2024-09-21T07:07:50.628 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e0a600640 1 -- 172.21.15.114:0/1410664837 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0c0acd50 con 0x5589b1404070 2024-09-21T07:07:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.626+0000 7f5e0a600640 1 -- 172.21.15.114:0/1410664837 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0c0b6c60 con 0x5589b1404070 2024-09-21T07:07:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 -- 172.21.15.114:0/1410664837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1404070 msgr2=0x5589b17aa9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1404070 0x5589b17aa9e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0a97c0 tx=0x7f5e0c004920 comp rx=0 tx=0).stop 2024-09-21T07:07:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 -- 172.21.15.114:0/1410664837 shutdown_connections 2024-09-21T07:07:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1397da0 0x5589b17aaf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1404070 0x5589b17aa9e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1410664837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.627+0000 7f5e2b476740 1 -- 172.21.15.114:0/1410664837 >> 172.21.15.114:0/1410664837 conn(0x5589b13f8460 msgr2=0x5589b17b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.628+0000 7f5e2b476740 1 -- 172.21.15.114:0/1410664837 shutdown_connections 2024-09-21T07:07:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.628+0000 7f5e2b476740 1 -- 172.21.15.114:0/1410664837 wait complete. 2024-09-21T07:07:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.628+0000 7f5e2b476740 1 -- start start 2024-09-21T07:07:50.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.628+0000 7f5e2b476740 1 Processor -- start 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 0x5589b17cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1404070 0x5589b1843170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00011480 con 0x5589b13db1b0 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00010d20 con 0x5589b1404070 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0000df80 con 0x5589b1397da0 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e2b476740 10 client.0 _ll_register_callbacks cb 0x7f5e0c00cf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:50.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e21e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 0x5589b17cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e21e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 0x5589b17cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56158/0 (socket says 172.21.15.114:56158) 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e21e00640 1 -- 172.21.15.114:0/1916763204 learned_addr learned my addr 172.21.15.114:0/1916763204 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e21400640 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.629+0000 7f5e22800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1404070 0x5589b1843170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5e21e00640 1 -- 172.21.15.114:0/1916763204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1404070 msgr2=0x5589b1843170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1404070 0x5589b1843170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5e21e00640 1 -- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 msgr2=0x5589b17cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5e21e00640 1 -- 172.21.15.114:0/1916763204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b18436b0 con 0x5589b1397da0 2024-09-21T07:07:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5e21e00640 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 0x5589b17cbfd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5e1001afd0 tx=0x7f5e10003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b9e213966b8724a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.630+0000 7f5dff400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.631+0000 7f5dff400640 1 -- 172.21.15.114:0/1916763204 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10030060 con 0x5589b1397da0 2024-09-21T07:07:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.631+0000 7f5dff400640 1 -- 172.21.15.114:0/1916763204 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f5e10005240 con 0x5589b1397da0 2024-09-21T07:07:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.631+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e0c0acd50 con 0x5589b1397da0 2024-09-21T07:07:50.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.631+0000 7f5dff400640 1 -- 172.21.15.114:0/1916763204 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5e10037050 con 0x5589b1397da0 2024-09-21T07:07:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.631+0000 7f5e2b476740 10 client.28840 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:50.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.631+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5589b18448f0 con 0x5589b1397da0 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.633+0000 7f5dff400640 1 -- 172.21.15.114:0/1916763204 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0c0acd50 con 0x5589b1397da0 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5dff400640 1 -- 172.21.15.114:0/1916763204 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f5e100045e0 con 0x5589b1397da0 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5e2b476740 10 client.28840 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5e2b476740 10 client.28840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5e2b476740 20 client.28840 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b1844d90 0x5589b18471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5e2b476740 4 client.28840 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:07:50.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.634+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589b171f9b0 con 0x5589b1844d90 2024-09-21T07:07:50.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.637+0000 7f5e21400640 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b1844d90 0x5589b18471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:50.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.638+0000 7f5e21400640 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b1844d90 0x5589b18471b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf4292cafa7854 server_cookie=4bd1108be5e7c0d6 in_seq=0 out_seq=0 2024-09-21T07:07:50.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.638+0000 7f5dff400640 10 client.28840 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:07:50.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5dff400640 1 -- 172.21.15.114:0/1916763204 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589b171f9b0 con 0x5589b1844d90 2024-09-21T07:07:50.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5dff400640 10 client.28840 handle_command_reply: tid=0 2024-09-21T07:07:50.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 client.28840 shutdown 2024-09-21T07:07:50.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 msgr2=0x5589b17cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 0x5589b17cbfd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5e1001afd0 tx=0x7f5e10003cf0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 shutdown_connections 2024-09-21T07:07:50.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x5589b1844d90 0x5589b18471b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589b1404070 0x5589b1843170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5589b13db1b0 0x5589b17cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 --2- 172.21.15.114:0/1916763204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589b1397da0 0x5589b17cbfd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 >> 172.21.15.114:0/1916763204 conn(0x5589b13f8460 msgr2=0x5589b17aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.639+0000 7f5dff400640 0 client.28840 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.640+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 shutdown_connections 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.640+0000 7f5e2b476740 1 -- 172.21.15.114:0/1916763204 wait complete. 2024-09-21T07:07:50.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.640+0000 7f5e2b476740 20 client.28840 trim_cache size 0 max 16384 2024-09-21T07:07:50.643 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:07:50.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e040b67d0 msgr2=0x7f5e040b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e040b67d0 0x7f5e040b8ce0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5e18006010 tx=0x7f5e18031f90 comp rx=0 tx=0).stop 2024-09-21T07:07:50.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c10aee0 msgr2=0x7f5e1c10c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c10aee0 0x7f5e1c10c3c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c10bd30 tx=0x7f5e0c00cdd0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 shutdown_connections 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5e1c116640 0x7f5e1c10c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5e040b67d0 0x7f5e040b8ce0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e1c10aee0 0x7f5e1c10c3c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 --2- 172.21.15.114:0/3945146524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e1c10a590 0x7f5e1c10be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 >> 172.21.15.114:0/3945146524 conn(0x7f5e1c070ec0 msgr2=0x7f5e1c073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.641+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 shutdown_connections 2024-09-21T07:07:50.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:50.642+0000 7f5e24800640 1 -- 172.21.15.114:0/3945146524 wait complete. 2024-09-21T07:07:51.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:51 smithi114 ceph-mon[30654]: pgmap v819: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 4.4 MiB/s wr, 455 op/s 2024-09-21T07:07:51.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:51 smithi183 ceph-mon[34219]: pgmap v819: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 4.4 MiB/s wr, 455 op/s 2024-09-21T07:07:51.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:51 smithi193 ceph-mon[34601]: pgmap v819: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 4.4 MiB/s wr, 455 op/s 2024-09-21T07:07:52.146 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:07:52.831 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:52.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 795.0 s, 453.6 tps, lat 226.635 ms stddev 278.110 2024-09-21T07:07:53.810 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:53 smithi114 ceph-mon[30654]: pgmap v820: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 5.1 MiB/s wr, 487 op/s 2024-09-21T07:07:53.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:53 smithi183 ceph-mon[34219]: pgmap v820: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 5.1 MiB/s wr, 487 op/s 2024-09-21T07:07:53.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:53 smithi193 ceph-mon[34601]: pgmap v820: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 5.1 MiB/s wr, 487 op/s 2024-09-21T07:07:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 -- 172.21.15.114:0/860375435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980114680 msgr2=0x7f7980116aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 --2- 172.21.15.114:0/860375435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980114680 0x7f7980116aa0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7978009ca0 tx=0x7f797802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 -- 172.21.15.114:0/860375435 shutdown_connections 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 --2- 172.21.15.114:0/860375435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980114680 0x7f7980116aa0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 --2- 172.21.15.114:0/860375435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980078290 0x7f7980114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 --2- 172.21.15.114:0/860375435 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7980077940 0x7f7980077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.349+0000 7f7986e00640 1 -- 172.21.15.114:0/860375435 >> 172.21.15.114:0/860375435 conn(0x7f79801016f0 msgr2=0x7f7980103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.350+0000 7f7986e00640 1 -- 172.21.15.114:0/860375435 shutdown_connections 2024-09-21T07:07:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.350+0000 7f7986e00640 1 -- 172.21.15.114:0/860375435 wait complete. 2024-09-21T07:07:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.351+0000 7f7986e00640 1 Processor -- start 2024-09-21T07:07:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.352+0000 7f7986e00640 1 -- start start 2024-09-21T07:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.352+0000 7f7986e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 0x7f79801a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7986e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980078290 0x7f79801a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7986e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7980114680 0x7f79801a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7986e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f798011a010 con 0x7f7980114680 2024-09-21T07:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7986e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7980119e90 con 0x7f7980077940 2024-09-21T07:07:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7986e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f798011a190 con 0x7f7980078290 2024-09-21T07:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f797fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 0x7f79801a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f797fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 0x7f79801a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36396/0 (socket says 172.21.15.114:36396) 2024-09-21T07:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f797fe00640 1 -- 172.21.15.114:0/2037957003 learned_addr learned my addr 172.21.15.114:0/2037957003 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:54.355 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7984e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7980114680 0x7f79801a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.353+0000 7f7977e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980078290 0x7f79801a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f797fe00640 1 -- 172.21.15.114:0/2037957003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980078290 msgr2=0x7f79801a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f797fe00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980078290 0x7f79801a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f797fe00640 1 -- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7980114680 msgr2=0x7f79801a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f797fe00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7980114680 0x7f79801a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f797fe00640 1 -- 172.21.15.114:0/2037957003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79801a6920 con 0x7f7980077940 2024-09-21T07:07:54.356 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f7977e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980078290 0x7f79801a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.354+0000 7f797fe00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 0x7f79801a09f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f797000e9f0 tx=0x7f797000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.355+0000 7f797d600640 1 -- 172.21.15.114:0/2037957003 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000cd80 con 0x7f7980077940 2024-09-21T07:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.355+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79801a6c10 con 0x7f7980077940 2024-09-21T07:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.355+0000 7f797d600640 1 -- 172.21.15.114:0/2037957003 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79700108e0 con 0x7f7980077940 2024-09-21T07:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.355+0000 7f797d600640 1 -- 172.21.15.114:0/2037957003 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970018980 con 0x7f7980077940 2024-09-21T07:07:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.355+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801ad3a0 con 0x7f7980077940 2024-09-21T07:07:54.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.361+0000 7f798dec1740 1 Processor -- start 2024-09-21T07:07:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.361+0000 7f798dec1740 1 -- start start 2024-09-21T07:07:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.362+0000 7f797d600640 1 -- 172.21.15.114:0/2037957003 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7970010430 con 0x7f7980077940 2024-09-21T07:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.363+0000 7f798dec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 0x55781e120c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.363+0000 7f798dec1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e121170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.363+0000 7f798dec1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e1216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.363+0000 7f798dec1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55781e0966b0 con 0x55781dd532f0 2024-09-21T07:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.363+0000 7f798dec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55781e085ae0 con 0x55781dd7c1b0 2024-09-21T07:07:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.363+0000 7f798dec1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55781e0898f0 con 0x55781dd5b4d0 2024-09-21T07:07:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.364+0000 7f7984e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 0x55781e120c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.364+0000 7f7984e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 0x55781e120c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36400/0 (socket says 172.21.15.114:36400) 2024-09-21T07:07:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.364+0000 7f7984e00640 1 -- 172.21.15.114:0/2102333786 learned_addr learned my addr 172.21.15.114:0/2102333786 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.364+0000 7f797fe00640 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e121170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 -- 172.21.15.114:0/2102333786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 msgr2=0x55781e1216b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e1216b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f797d600640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79640b6780 0x7f79640b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 -- 172.21.15.114:0/2102333786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 msgr2=0x55781e121170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e121170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 -- 172.21.15.114:0/2102333786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781e052780 con 0x55781dd7c1b0 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f797fe00640 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e121170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:07:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f797d600640 1 -- 172.21.15.114:0/2037957003 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f79700e7e00 con 0x7f7980077940 2024-09-21T07:07:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79640b6780 0x7f79640b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.365+0000 7f7984e00640 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 0x55781e120c30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7978009830 tx=0x7f7978009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=262d60fc41c1314 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.366+0000 7f7975600640 1 -- 172.21.15.114:0/2102333786 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978030800 con 0x55781dd7c1b0 2024-09-21T07:07:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.366+0000 7f7975600640 1 -- 172.21.15.114:0/2102333786 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978036070 con 0x55781dd7c1b0 2024-09-21T07:07:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.366+0000 7f7975600640 1 -- 172.21.15.114:0/2102333786 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978030800 con 0x55781dd7c1b0 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 -- 172.21.15.114:0/2102333786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 msgr2=0x55781e120c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f7984e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79640b6780 0x7f79640b8c90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7978009a60 tx=0x7f7978004460 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 0x55781e120c30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7978009830 tx=0x7f7978009860 comp rx=0 tx=0).stop 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 -- 172.21.15.114:0/2102333786 shutdown_connections 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e1216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e121170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 --2- 172.21.15.114:0/2102333786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd7c1b0 0x55781e120c30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 -- 172.21.15.114:0/2102333786 >> 172.21.15.114:0/2102333786 conn(0x55781dd70300 msgr2=0x55781e11eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 -- 172.21.15.114:0/2102333786 shutdown_connections 2024-09-21T07:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.367+0000 7f798dec1740 1 -- 172.21.15.114:0/2102333786 wait complete. 2024-09-21T07:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.368+0000 7f798dec1740 1 -- start start 2024-09-21T07:07:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.368+0000 7f798dec1740 1 Processor -- start 2024-09-21T07:07:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e12c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e12c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd7c1b0 0x55781e12cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55781dfbdab0 con 0x55781dd532f0 2024-09-21T07:07:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55781e02c570 con 0x55781dd5b4d0 2024-09-21T07:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55781e12f3b0 con 0x55781dd7c1b0 2024-09-21T07:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.370+0000 7f798dec1740 10 client.0 _ll_register_callbacks cb 0x55781e03b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7977e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e12c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7977e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e12c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:36402/0 (socket says 172.21.15.114:36402) 2024-09-21T07:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7977e00640 1 -- 172.21.15.114:0/2338358733 learned_addr learned my addr 172.21.15.114:0/2338358733 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7984e00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd7c1b0 0x55781e12cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f797fe00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e12c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7977e00640 1 -- 172.21.15.114:0/2338358733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd7c1b0 msgr2=0x55781e12cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7977e00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd7c1b0 0x55781e12cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.371+0000 7f7977e00640 1 -- 172.21.15.114:0/2338358733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 msgr2=0x55781e12c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f7977e00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e12c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f7977e00640 1 -- 172.21.15.114:0/2338358733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781dde7790 con 0x55781dd5b4d0 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f797fe00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e12c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f7977e00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e12c720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7968011970 tx=0x7f7968011e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52ac9413b5aa652f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f796e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f796e000640 1 -- 172.21.15.114:0/2338358733 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796801e060 con 0x55781dd5b4d0 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f796e000640 1 -- 172.21.15.114:0/2338358733 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f796800f9e0 con 0x55781dd5b4d0 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.372+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7978036070 con 0x55781dd5b4d0 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f796e000640 1 -- 172.21.15.114:0/2338358733 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7968025050 con 0x55781dd5b4d0 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 10 client.28889 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55781de60620 con 0x55781dd5b4d0 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f796e000640 1 -- 172.21.15.114:0/2338358733 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978036070 con 0x55781dd5b4d0 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f796e000640 1 -- 172.21.15.114:0/2338358733 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f796800e3b0 con 0x55781dd5b4d0 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 10 client.28889 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 10 client.28889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 20 client.28889 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd3ad80 0x55781e13e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 4 client.28889 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.373+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55781e098600 con 0x55781dd3ad80 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.374+0000 7f797fe00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd3ad80 0x55781e13e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.374+0000 7f797fe00640 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd3ad80 0x55781e13e4c0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390d960af1b5be00 server_cookie=9985807cb1598427 in_seq=0 out_seq=0 2024-09-21T07:07:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.374+0000 7f796e000640 10 client.28889 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.375+0000 7f796e000640 1 -- 172.21.15.114:0/2338358733 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55781e098600 con 0x55781dd3ad80 2024-09-21T07:07:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.375+0000 7f796e000640 10 client.28889 handle_command_reply: tid=0 2024-09-21T07:07:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.375+0000 7f798dec1740 1 client.28889 shutdown 2024-09-21T07:07:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 msgr2=0x55781e12c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e12c720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7968011970 tx=0x7f7968011e30 comp rx=0 tx=0).stop 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 shutdown_connections 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd3ad80 0x55781e13e4c0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f796e000640 0 client.28889 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd7c1b0 0x55781e12cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e12c720 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 --2- 172.21.15.114:0/2338358733 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e12c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.376+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 >> 172.21.15.114:0/2338358733 conn(0x55781dd70300 msgr2=0x55781e121c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.377+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 shutdown_connections 2024-09-21T07:07:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.377+0000 7f798dec1740 1 -- 172.21.15.114:0/2338358733 wait complete. 2024-09-21T07:07:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.377+0000 7f798dec1740 20 client.28889 trim_cache size 0 max 16384 2024-09-21T07:07:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 Processor -- start 2024-09-21T07:07:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 -- start start 2024-09-21T07:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd3ad80 0x55781e122370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e144200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b74a0 con 0x55781dd532f0 2024-09-21T07:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b90d0 con 0x55781dd3ad80 2024-09-21T07:07:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.404+0000 7f798dec1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b7b70 con 0x55781dd5b4d0 2024-09-21T07:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7977e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f797fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd3ad80 0x55781e122370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e144200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56198/0 (socket says 172.21.15.114:56198) 2024-09-21T07:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 -- 172.21.15.114:0/692354631 learned_addr learned my addr 172.21.15.114:0/692354631 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 -- 172.21.15.114:0/692354631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd3ad80 msgr2=0x55781e122370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd3ad80 0x55781e122370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 -- 172.21.15.114:0/692354631 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 msgr2=0x55781e143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.405+0000 7f7984e00640 1 -- 172.21.15.114:0/692354631 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781e123a40 con 0x55781dd5b4d0 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.406+0000 7f7984e00640 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e144200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7978006010 tx=0x7f7978051000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5012db444ba8c809 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.406+0000 7f7974c00640 1 -- 172.21.15.114:0/692354631 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978057050 con 0x55781dd5b4d0 2024-09-21T07:07:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.406+0000 7f7974c00640 1 -- 172.21.15.114:0/692354631 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978058040 con 0x55781dd5b4d0 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.406+0000 7f7974c00640 1 -- 172.21.15.114:0/692354631 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978030a20 con 0x55781dd5b4d0 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 -- 172.21.15.114:0/692354631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 msgr2=0x55781e144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e144200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7978006010 tx=0x7f7978051000 comp rx=0 tx=0).stop 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 -- 172.21.15.114:0/692354631 shutdown_connections 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd5b4d0 0x55781e144200 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 --2- 172.21.15.114:0/692354631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd3ad80 0x55781e122370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 -- 172.21.15.114:0/692354631 >> 172.21.15.114:0/692354631 conn(0x55781dd70300 msgr2=0x55781e121c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 -- 172.21.15.114:0/692354631 shutdown_connections 2024-09-21T07:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.407+0000 7f798dec1740 1 -- 172.21.15.114:0/692354631 wait complete. 2024-09-21T07:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.408+0000 7f798dec1740 1 -- start start 2024-09-21T07:07:54.410 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.408+0000 7f798dec1740 1 Processor -- start 2024-09-21T07:07:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 0x55781e132640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e132b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.411 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e1330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b89a0 con 0x55781dd532f0 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b8240 con 0x55781dd5b4d0 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b54a0 con 0x55781dd3ad80 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f798dec1740 10 client.0 _ll_register_callbacks cb 0x7f7968009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f797fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 0x55781e132640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f7984e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e1330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f797fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 0x55781e132640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:56204/0 (socket says 172.21.15.114:56204) 2024-09-21T07:07:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.410+0000 7f797fe00640 1 -- 172.21.15.114:0/444763530 learned_addr learned my addr 172.21.15.114:0/444763530 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f7977e00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e132b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f797fe00640 1 -- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 msgr2=0x55781e1330c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f797fe00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e1330c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f797fe00640 1 -- 172.21.15.114:0/444763530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 msgr2=0x55781e132b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f797fe00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e132b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f797fe00640 1 -- 172.21.15.114:0/444763530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781e1a7f00 con 0x55781dd3ad80 2024-09-21T07:07:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.411+0000 7f7984e00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e1330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f797fe00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 0x55781e132640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f79700ad5d0 tx=0x7f7970018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38117ce8a7fa132a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f796e000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f796e000640 1 -- 172.21.15.114:0/444763530 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970010720 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f796e000640 1 -- 172.21.15.114:0/444763530 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f7970018b20 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7978058040 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f796e000640 1 -- 172.21.15.114:0/444763530 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f79700ad3e0 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f798dec1740 10 client.28855 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.412+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55781e1a9190 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f796e000640 1 -- 172.21.15.114:0/444763530 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978058040 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f796e000640 1 -- 172.21.15.114:0/444763530 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f79700ac5a0 con 0x55781dd3ad80 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f798dec1740 10 client.28855 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f798dec1740 10 client.28855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f798dec1740 20 client.28855 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f798dec1740 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd7c1b0 0x55781e1ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f798dec1740 4 client.28855 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:07:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.413+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55781e0940b0 con 0x55781dd7c1b0 2024-09-21T07:07:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.414+0000 7f7977e00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd7c1b0 0x55781e1ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.414+0000 7f7977e00640 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd7c1b0 0x55781e1ab9e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3d96814dd548bf server_cookie=545c2f788ef1bf76 in_seq=0 out_seq=0 2024-09-21T07:07:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.414+0000 7f796e000640 10 client.28855 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f796e000640 1 -- 172.21.15.114:0/444763530 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55781e0940b0 con 0x55781dd7c1b0 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f796e000640 10 client.28855 handle_command_reply: tid=0 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 client.28855 shutdown 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 msgr2=0x55781e132640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 0x55781e132640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f79700ad5d0 tx=0x7f7970018420 comp rx=0 tx=0).stop 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 shutdown_connections 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55781dd5b4d0 0x55781e1330c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55781dd532f0 0x55781e132b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55781dd7c1b0 0x55781e1ab9e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 --2- 172.21.15.114:0/444763530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55781dd3ad80 0x55781e132640 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 >> 172.21.15.114:0/444763530 conn(0x55781dd70300 msgr2=0x55781e121c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f796e000640 0 client.28855 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 shutdown_connections 2024-09-21T07:07:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.425+0000 7f798dec1740 1 -- 172.21.15.114:0/444763530 wait complete. 2024-09-21T07:07:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.426+0000 7f798dec1740 20 client.28855 trim_cache size 0 max 16384 2024-09-21T07:07:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:07:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:07:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:07:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:07:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.426+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79640b6780 msgr2=0x7f79640b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.426+0000 7f7986e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79640b6780 0x7f79640b8c90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7978009a60 tx=0x7f7978004460 comp rx=0 tx=0).stop 2024-09-21T07:07:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.426+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 msgr2=0x7f79801a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.426+0000 7f7986e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 0x7f79801a09f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f797000e9f0 tx=0x7f797000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 shutdown_connections 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7980114680 0x7f79801a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7980078290 0x7f79801a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f79640b6780 0x7f79640b8c90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 --2- 172.21.15.114:0/2037957003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7980077940 0x7f79801a09f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 >> 172.21.15.114:0/2037957003 conn(0x7f79801016f0 msgr2=0x7f7980116700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 shutdown_connections 2024-09-21T07:07:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:07:54.427+0000 7f7986e00640 1 -- 172.21.15.114:0/2037957003 wait complete. 2024-09-21T07:07:55.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.413+0000 7f3735800640 1 -- 172.21.15.114:0/794954337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3730116640 msgr2=0x7f3730118b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.413+0000 7f3735800640 1 --2- 172.21.15.114:0/794954337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3730116640 0x7f3730118b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3728009900 tx=0x7f372802a440 comp rx=0 tx=0).stop 2024-09-21T07:07:55.416 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 -- 172.21.15.114:0/794954337 shutdown_connections 2024-09-21T07:07:55.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 --2- 172.21.15.114:0/794954337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3730116640 0x7f3730118b50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 --2- 172.21.15.114:0/794954337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f3730115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 --2- 172.21.15.114:0/794954337 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f373010a590 0x7f373010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 -- 172.21.15.114:0/794954337 >> 172.21.15.114:0/794954337 conn(0x7f3730070ec0 msgr2=0x7f37300712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:55.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 -- 172.21.15.114:0/794954337 shutdown_connections 2024-09-21T07:07:55.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.414+0000 7f3735800640 1 -- 172.21.15.114:0/794954337 wait complete. 2024-09-21T07:07:55.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.416+0000 7f3735800640 1 Processor -- start 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.417+0000 7f3735800640 1 -- start start 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.418+0000 7f3735800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373010a590 0x7f37301aa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.418+0000 7f3735800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f37301aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.418+0000 7f372ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373010a590 0x7f37301aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.418+0000 7f372ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373010a590 0x7f37301aa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54022/0 (socket says 172.21.15.114:54022) 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f3726000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f37301aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f3726000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f37301aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57740/0 (socket says 172.21.15.114:57740) 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f372ea00640 1 -- 172.21.15.114:0/3543319618 learned_addr learned my addr 172.21.15.114:0/3543319618 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3730116640 0x7f37301aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f373011c370 con 0x7f3730116640 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f373011c1f0 con 0x7f373010a590 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f373011c4f0 con 0x7f373010aee0 2024-09-21T07:07:55.422 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.419+0000 7f372f400640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3730116640 0x7f37301aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.421+0000 7f3726000640 1 -- 172.21.15.114:0/3543319618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373010a590 msgr2=0x7f37301aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.421+0000 7f3726000640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373010a590 0x7f37301aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.421+0000 7f3726000640 1 -- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3730116640 msgr2=0x7f37301aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.421+0000 7f3726000640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3730116640 0x7f37301aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.421+0000 7f3726000640 1 -- 172.21.15.114:0/3543319618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37301b5360 con 0x7f373010aee0 2024-09-21T07:07:55.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.421+0000 7f3726000640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f37301aa590 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f373010bd30 tx=0x7f371800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:55.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.422+0000 7f3727e00640 1 -- 172.21.15.114:0/3543319618 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718015020 con 0x7f373010aee0 2024-09-21T07:07:55.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.423+0000 7f3727e00640 1 -- 172.21.15.114:0/3543319618 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3718011ce0 con 0x7f373010aee0 2024-09-21T07:07:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.423+0000 7f3727e00640 1 -- 172.21.15.114:0/3543319618 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718010da0 con 0x7f373010aee0 2024-09-21T07:07:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.423+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37301b5650 con 0x7f373010aee0 2024-09-21T07:07:55.425 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.423+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37301b5c10 con 0x7f373010aee0 2024-09-21T07:07:55.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.434+0000 7f3727e00640 1 -- 172.21.15.114:0/3543319618 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f3718018020 con 0x7f373010aee0 2024-09-21T07:07:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.436+0000 7f3727e00640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37140b6700 0x7f37140b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.437+0000 7f3727e00640 1 -- 172.21.15.114:0/3543319618 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f37180e8160 con 0x7f373010aee0 2024-09-21T07:07:55.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.437+0000 7f372ea00640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37140b6700 0x7f37140b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.454+0000 7f372ea00640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37140b6700 0x7f37140b8c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3730116010 tx=0x7f3720009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:55.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 Processor -- start 2024-09-21T07:07:55.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 -- start start 2024-09-21T07:07:55.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de7fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab8390 0x562b4de80620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562b4ddf6fb0 con 0x562b4dab8390 2024-09-21T07:07:55.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b4dde63e0 con 0x562b4dab01b0 2024-09-21T07:07:55.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.456+0000 7f373c46c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b4ddea1f0 con 0x562b4dad9070 2024-09-21T07:07:55.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.457+0000 7f3726000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.457+0000 7f3726000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de800e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54026/0 (socket says 172.21.15.114:54026) 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.457+0000 7f3726000640 1 -- 172.21.15.114:0/3663855705 learned_addr learned my addr 172.21.15.114:0/3663855705 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.457+0000 7f372ea00640 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de7fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.457+0000 7f372f400640 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab8390 0x562b4de80620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.458+0000 7f372ea00640 1 -- 172.21.15.114:0/3663855705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 msgr2=0x562b4de800e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.458+0000 7f372ea00640 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de800e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.458+0000 7f372ea00640 1 -- 172.21.15.114:0/3663855705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab8390 msgr2=0x562b4de80620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.458+0000 7f372ea00640 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab8390 0x562b4de80620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.458+0000 7f372ea00640 1 -- 172.21.15.114:0/3663855705 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4ddb0cd0 con 0x562b4dad9070 2024-09-21T07:07:55.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.460+0000 7f372ea00640 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de7fba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3720009d90 tx=0x7f3720014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8279f3cc1f042e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:55.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.461+0000 7f371fe00640 1 -- 172.21.15.114:0/3663855705 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372000f810 con 0x562b4dad9070 2024-09-21T07:07:55.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.461+0000 7f371fe00640 1 -- 172.21.15.114:0/3663855705 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372000fe60 con 0x562b4dad9070 2024-09-21T07:07:55.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.461+0000 7f371fe00640 1 -- 172.21.15.114:0/3663855705 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372001d3b0 con 0x562b4dad9070 2024-09-21T07:07:55.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.463+0000 7f373c46c740 1 -- 172.21.15.114:0/3663855705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 msgr2=0x562b4de7fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.463+0000 7f373c46c740 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de7fba0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3720009d90 tx=0x7f3720014000 comp rx=0 tx=0).stop 2024-09-21T07:07:55.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 -- 172.21.15.114:0/3663855705 shutdown_connections 2024-09-21T07:07:55.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab8390 0x562b4de80620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de800e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 --2- 172.21.15.114:0/3663855705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de7fba0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 -- 172.21.15.114:0/3663855705 >> 172.21.15.114:0/3663855705 conn(0x562b4dacd460 msgr2=0x562b4de7da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:55.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 -- 172.21.15.114:0/3663855705 shutdown_connections 2024-09-21T07:07:55.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.466+0000 7f373c46c740 1 -- 172.21.15.114:0/3663855705 wait complete. 2024-09-21T07:07:55.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.467+0000 7f373c46c740 1 -- start start 2024-09-21T07:07:55.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.467+0000 7f373c46c740 1 Processor -- start 2024-09-21T07:07:55.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.468+0000 7f373c46c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de8b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f373c46c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dab8390 0x562b4de8bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f373c46c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dad9070 0x562b4de976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f372ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de8b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f372ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de8b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54040/0 (socket says 172.21.15.114:54040) 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f373c46c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x562b4dc47700 con 0x562b4dad9070 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f373c46c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b4dd81580 con 0x562b4dab01b0 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f373c46c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562b4de8e3d0 con 0x562b4dab8390 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.469+0000 7f373c46c740 10 client.0 _ll_register_callbacks cb 0x562b4dd91470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.470+0000 7f372f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dad9070 0x562b4de976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.471+0000 7f372ea00640 1 -- 172.21.15.114:0/2498378960 learned_addr learned my addr 172.21.15.114:0/2498378960 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:55.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.471+0000 7f3726000640 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dab8390 0x562b4de8bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f372f400640 1 -- 172.21.15.114:0/2498378960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dab8390 msgr2=0x562b4de8bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f372f400640 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dab8390 0x562b4de8bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f372f400640 1 -- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 msgr2=0x562b4de8b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f372f400640 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de8b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f372f400640 1 -- 172.21.15.114:0/2498378960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4db44650 con 0x562b4dad9070 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f372f400640 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dad9070 0x562b4de976c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f372802a960 tx=0x7f3728004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77825a2f3fd62310 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:55.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.472+0000 7f371cc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:07:55.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.473+0000 7f371cc00640 1 -- 172.21.15.114:0/2498378960 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728035040 con 0x562b4dad9070 2024-09-21T07:07:55.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.473+0000 7f371cc00640 1 -- 172.21.15.114:0/2498378960 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f3728005510 con 0x562b4dad9070 2024-09-21T07:07:55.475 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.474+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b4dd83b90 con 0x562b4dad9070 2024-09-21T07:07:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.478+0000 7f371cc00640 1 -- 172.21.15.114:0/2498378960 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f37280046c0 con 0x562b4dad9070 2024-09-21T07:07:55.480 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.478+0000 7f373c46c740 10 client.18312 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:55.481 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.478+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562b4dd9e950 con 0x562b4dad9070 2024-09-21T07:07:55.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f371cc00640 1 -- 172.21.15.114:0/2498378960 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562b4dd83b90 con 0x562b4dad9070 2024-09-21T07:07:55.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f371cc00640 1 -- 172.21.15.114:0/2498378960 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f3728003a60 con 0x562b4dad9070 2024-09-21T07:07:55.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f373c46c740 10 client.18312 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:55.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f373c46c740 10 client.18312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:55.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f373c46c740 20 client.18312 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:55.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f373c46c740 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da979f0 0x562b4de9d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f373c46c740 4 client.18312 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:07:55.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.481+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4ddf8f00 con 0x562b4da979f0 2024-09-21T07:07:55.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.483+0000 7f372ea00640 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da979f0 0x562b4de9d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.484+0000 7f372ea00640 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da979f0 0x562b4de9d770 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c31b8977800871 server_cookie=d25a3e22fbe54760 in_seq=0 out_seq=0 2024-09-21T07:07:55.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.484+0000 7f371cc00640 10 client.18312 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:55.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.485+0000 7f371cc00640 1 -- 172.21.15.114:0/2498378960 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b4ddf8f00 con 0x562b4da979f0 2024-09-21T07:07:55.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.485+0000 7f371cc00640 10 client.18312 handle_command_reply: tid=0 2024-09-21T07:07:55.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.485+0000 7f373c46c740 1 client.18312 shutdown 2024-09-21T07:07:55.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dad9070 msgr2=0x562b4de976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dad9070 0x562b4de976c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f372802a960 tx=0x7f3728004510 comp rx=0 tx=0).stop 2024-09-21T07:07:55.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 shutdown_connections 2024-09-21T07:07:55.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dad9070 0x562b4de976c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da979f0 0x562b4de9d770 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dab8390 0x562b4de8bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 --2- 172.21.15.114:0/2498378960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab01b0 0x562b4de8b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f371cc00640 0 client.18312 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:55.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 >> 172.21.15.114:0/2498378960 conn(0x562b4dacd460 msgr2=0x562b4de82d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:55.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.488+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 shutdown_connections 2024-09-21T07:07:55.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.489+0000 7f373c46c740 1 -- 172.21.15.114:0/2498378960 wait complete. 2024-09-21T07:07:55.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.489+0000 7f373c46c740 20 client.18312 trim_cache size 0 max 16384 2024-09-21T07:07:55.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.518+0000 7f373c46c740 1 Processor -- start 2024-09-21T07:07:55.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.518+0000 7f373c46c740 1 -- start start 2024-09-21T07:07:55.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.519+0000 7f373c46c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de8be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.520+0000 7f373c46c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de975c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.520+0000 7f373c46c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de84e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.520+0000 7f373c46c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37301bfe60 con 0x562b4dab01b0 2024-09-21T07:07:55.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.520+0000 7f373c46c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37301c1a90 con 0x562b4dab8390 2024-09-21T07:07:55.522 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.520+0000 7f373c46c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37301c0530 con 0x562b4dad9070 2024-09-21T07:07:55.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.520+0000 7f3726000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54054/0 (socket says 172.21.15.114:54054) 2024-09-21T07:07:55.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 -- 172.21.15.114:0/2295975927 learned_addr learned my addr 172.21.15.114:0/2295975927 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f372f400640 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de84e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 -- 172.21.15.114:0/2295975927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 msgr2=0x562b4de84e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f372ea00640 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de8be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de84e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 -- 172.21.15.114:0/2295975927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 msgr2=0x562b4de8be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de8be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.521+0000 7f3726000640 1 -- 172.21.15.114:0/2295975927 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4de82430 con 0x562b4dab8390 2024-09-21T07:07:55.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.522+0000 7f372ea00640 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de8be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:07:55.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.522+0000 7f3726000640 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de975c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f37180aa3a0 tx=0x7f37180ad410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73df3a7a39755603 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:55.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.522+0000 7f371f400640 1 -- 172.21.15.114:0/2295975927 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37180b2020 con 0x562b4dab8390 2024-09-21T07:07:55.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.522+0000 7f371f400640 1 -- 172.21.15.114:0/2295975927 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37180ad6f0 con 0x562b4dab8390 2024-09-21T07:07:55.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.522+0000 7f371f400640 1 -- 172.21.15.114:0/2295975927 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37180b7600 con 0x562b4dab8390 2024-09-21T07:07:55.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 -- 172.21.15.114:0/2295975927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 msgr2=0x562b4de975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de975c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f37180aa3a0 tx=0x7f37180ad410 comp rx=0 tx=0).stop 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 -- 172.21.15.114:0/2295975927 shutdown_connections 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de84e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de975c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 --2- 172.21.15.114:0/2295975927 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de8be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 -- 172.21.15.114:0/2295975927 >> 172.21.15.114:0/2295975927 conn(0x562b4dacd460 msgr2=0x562b4de8b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 -- 172.21.15.114:0/2295975927 shutdown_connections 2024-09-21T07:07:55.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.524+0000 7f373c46c740 1 -- 172.21.15.114:0/2295975927 wait complete. 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.525+0000 7f373c46c740 1 -- start start 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.525+0000 7f373c46c740 1 Processor -- start 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de93b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de8e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f37301c1360 con 0x562b4dab01b0 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37301c0c00 con 0x562b4dab8390 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37301bde60 con 0x562b4dad9070 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.526+0000 7f373c46c740 10 client.0 _ll_register_callbacks cb 0x7f37280041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.527+0000 7f372f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de8e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.527+0000 7f372f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de8e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57768/0 (socket says 172.21.15.114:57768) 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.527+0000 7f372f400640 1 -- 172.21.15.114:0/3878482437 learned_addr learned my addr 172.21.15.114:0/3878482437 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:07:55.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.527+0000 7f3726000640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de93b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f372f400640 1 -- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 msgr2=0x562b4de93b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f372f400640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de93b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f372f400640 1 -- 172.21.15.114:0/3878482437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 msgr2=0x562b4de93630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f372f400640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f372f400640 1 -- 172.21.15.114:0/3878482437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4de94260 con 0x562b4dad9070 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f3726000640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de93b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f372f400640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de8e5e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3728003e30 tx=0x7f3728004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db143d10af1ccb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:07:55.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.528+0000 7f371cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.529+0000 7f371cc00640 1 -- 172.21.15.114:0/3878482437 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728035030 con 0x562b4dad9070 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.529+0000 7f371cc00640 1 -- 172.21.15.114:0/3878482437 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f37280037f0 con 0x562b4dad9070 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.529+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37180ad6f0 con 0x562b4dad9070 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.529+0000 7f371cc00640 1 -- 172.21.15.114:0/3878482437 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f372805f050 con 0x562b4dad9070 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.529+0000 7f373c46c740 10 client.28867 fetch_fsmap learned FSMap version 54 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.530+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562b4df08560 con 0x562b4dad9070 2024-09-21T07:07:55.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.530+0000 7f371cc00640 1 -- 172.21.15.114:0/3878482437 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37180ad6f0 con 0x562b4dad9070 2024-09-21T07:07:55.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.531+0000 7f371cc00640 1 -- 172.21.15.114:0/3878482437 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f3728057020 con 0x562b4dad9070 2024-09-21T07:07:55.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.532+0000 7f373c46c740 10 client.28867 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:07:55.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.532+0000 7f373c46c740 10 client.28867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:07:55.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.532+0000 7f373c46c740 20 client.28867 populate_metadata read hostname 'smithi114' 2024-09-21T07:07:55.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.532+0000 7f373c46c740 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da6cda0 0x562b4df0ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:07:55.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.532+0000 7f373c46c740 4 client.28867 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:07:55.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.532+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b4ddf49b0 con 0x562b4da6cda0 2024-09-21T07:07:55.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.535+0000 7f372ea00640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da6cda0 0x562b4df0ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:07:55.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.538+0000 7f372ea00640 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da6cda0 0x562b4df0ad20 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8aaf098ba97a58 server_cookie=3dcb179849c89e44 in_seq=0 out_seq=0 2024-09-21T07:07:55.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.538+0000 7f371cc00640 10 client.28867 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:55.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.538+0000 7f371cc00640 1 -- 172.21.15.114:0/3878482437 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b4ddf49b0 con 0x562b4da6cda0 2024-09-21T07:07:55.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.538+0000 7f371cc00640 10 client.28867 handle_command_reply: tid=0 2024-09-21T07:07:55.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.538+0000 7f373c46c740 1 client.28867 shutdown 2024-09-21T07:07:55.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.539+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 msgr2=0x562b4de8e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.539+0000 7f373c46c740 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de8e5e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3728003e30 tx=0x7f3728004a90 comp rx=0 tx=0).stop 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 shutdown_connections 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f373c46c740 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x562b4da6cda0 0x562b4df0ad20 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f373c46c740 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562b4dad9070 0x562b4de8e5e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f373c46c740 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4dab8390 0x562b4de93b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f373c46c740 1 --2- 172.21.15.114:0/3878482437 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x562b4dab01b0 0x562b4de93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f371cc00640 0 client.28867 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:07:55.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.540+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 >> 172.21.15.114:0/3878482437 conn(0x562b4dacd460 msgr2=0x562b4de80bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:55.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.541+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 shutdown_connections 2024-09-21T07:07:55.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.541+0000 7f373c46c740 1 -- 172.21.15.114:0/3878482437 wait complete. 2024-09-21T07:07:55.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.542+0000 7f373c46c740 20 client.28867 trim_cache size 0 max 16384 2024-09-21T07:07:55.545 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.544+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37140b6700 msgr2=0x7f37140b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.544+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37140b6700 0x7f37140b8c10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3730116010 tx=0x7f3720009500 comp rx=0 tx=0).stop 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.544+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 msgr2=0x7f37301aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.544+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f37301aa590 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f373010bd30 tx=0x7f371800b850 comp rx=0 tx=0).stop 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 shutdown_connections 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3730116640 0x7f37301aaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f373010aee0 0x7f37301aa590 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f37140b6700 0x7f37140b8c10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 --2- 172.21.15.114:0/3543319618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f373010a590 0x7f37301aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:07:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 >> 172.21.15.114:0/3543319618 conn(0x7f3730070ec0 msgr2=0x7f373010edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:07:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 shutdown_connections 2024-09-21T07:07:55.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:07:55.545+0000 7f3735800640 1 -- 172.21.15.114:0/3543319618 wait complete. 2024-09-21T07:07:55.817 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:55 smithi114 ceph-mon[30654]: pgmap v821: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-09-21T07:07:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:55 smithi183 ceph-mon[34219]: pgmap v821: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-09-21T07:07:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:55 smithi193 ceph-mon[34601]: pgmap v821: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 4.8 MiB/s wr, 507 op/s 2024-09-21T07:07:57.501 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:07:57.501 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:07:57.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:57 smithi114 ceph-mon[30654]: pgmap v822: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 4.8 MiB/s wr, 475 op/s 2024-09-21T07:07:57.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 800.0 s, 390.4 tps, lat 238.988 ms stddev 308.625 2024-09-21T07:07:57.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:57 smithi183 ceph-mon[34219]: pgmap v822: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 4.8 MiB/s wr, 475 op/s 2024-09-21T07:07:57.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:57 smithi193 ceph-mon[34601]: pgmap v822: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 4.8 MiB/s wr, 475 op/s 2024-09-21T07:07:58.157 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:07:58.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:07:58 smithi114 ceph-mon[30654]: pgmap v823: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-21T07:07:58.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:07:58 smithi183 ceph-mon[34219]: pgmap v823: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-21T07:07:58.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:07:58 smithi193 ceph-mon[34601]: pgmap v823: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 4.6 MiB/s wr, 473 op/s 2024-09-21T07:08:00.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.198+0000 7f0f11e00640 1 -- 172.21.15.114:0/3064280844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 msgr2=0x7f0f0c10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.202 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.198+0000 7f0f11e00640 1 --2- 172.21.15.114:0/3064280844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c10e000 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009860 tx=0x7f0efc02a360 comp rx=0 tx=0).stop 2024-09-21T07:08:00.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 -- 172.21.15.114:0/3064280844 shutdown_connections 2024-09-21T07:08:00.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 --2- 172.21.15.114:0/3064280844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 0x7f0f0c11efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 --2- 172.21.15.114:0/3064280844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c10e000 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.203 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 --2- 172.21.15.114:0/3064280844 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c10d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.204 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 -- 172.21.15.114:0/3064280844 >> 172.21.15.114:0/3064280844 conn(0x7f0f0c06e9f0 msgr2=0x7f0f0c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:00.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 -- 172.21.15.114:0/3064280844 shutdown_connections 2024-09-21T07:08:00.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.201+0000 7f0f11e00640 1 -- 172.21.15.114:0/3064280844 wait complete. 2024-09-21T07:08:00.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.202+0000 7f0f11e00640 1 Processor -- start 2024-09-21T07:08:00.205 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.203+0000 7f0f11e00640 1 -- start start 2024-09-21T07:08:00.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.204+0000 7f0f11e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c1a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.207 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.204+0000 7f0f11e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c1a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.205+0000 7f0f11e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 0x7f0f0c1ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.205+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c1a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.205+0000 7f0f10a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c1a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.205+0000 7f0f10a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c1a91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57778/0 (socket says 172.21.15.114:57778) 2024-09-21T07:08:00.208 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.206+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c1a9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60046/0 (socket says 172.21.15.114:60046) 2024-09-21T07:08:00.209 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.206+0000 7f0f0be00640 1 -- 172.21.15.114:0/2919341852 learned_addr learned my addr 172.21.15.114:0/2919341852 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:00.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.208+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c122720 con 0x7f0f0c111cd0 2024-09-21T07:08:00.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.208+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1225a0 con 0x7f0f0c11cba0 2024-09-21T07:08:00.210 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.208+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0c1228a0 con 0x7f0f0c10dbf0 2024-09-21T07:08:00.211 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.208+0000 7f0f11400640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 0x7f0f0c1ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.209+0000 7f0f11400640 1 -- 172.21.15.114:0/2919341852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 msgr2=0x7f0f0c1a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.212 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.209+0000 7f0f11400640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c1a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.209+0000 7f0f11400640 1 -- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 msgr2=0x7f0f0c1a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.209+0000 7f0f11400640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c1a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.210+0000 7f0f11400640 1 -- 172.21.15.114:0/2919341852 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0c1af120 con 0x7f0f0c11cba0 2024-09-21T07:08:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.210+0000 7f0f10a00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c1a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.210+0000 7f0f0be00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c1a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:00.213 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.210+0000 7f0f11400640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 0x7f0f0c1ae9f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d680 tx=0x7f0ef800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.211+0000 7f0f09600640 1 -- 172.21.15.114:0/2919341852 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8017070 con 0x7f0f0c11cba0 2024-09-21T07:08:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.211+0000 7f0f09600640 1 -- 172.21.15.114:0/2919341852 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef8009820 con 0x7f0f0c11cba0 2024-09-21T07:08:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.211+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0c1af410 con 0x7f0f0c11cba0 2024-09-21T07:08:00.214 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.211+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f0c1b5b70 con 0x7f0f0c11cba0 2024-09-21T07:08:00.215 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.211+0000 7f0f09600640 1 -- 172.21.15.114:0/2919341852 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004bc0 con 0x7f0f0c11cba0 2024-09-21T07:08:00.218 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.215+0000 7f0f1a1d3740 1 Processor -- start 2024-09-21T07:08:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 -- start start 2024-09-21T07:08:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 0x56208b030f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.219 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 0x56208b031440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b031980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56208afa6e90 con 0x56208ac60490 2024-09-21T07:08:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56208af962c0 con 0x56208ac68670 2024-09-21T07:08:00.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.216+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56208af9a0d0 con 0x56208ac89350 2024-09-21T07:08:00.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.217+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 0x56208b030f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 0x56208b030f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57790/0 (socket says 172.21.15.114:57790) 2024-09-21T07:08:00.222 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 -- 172.21.15.114:0/419115752 learned_addr learned my addr 172.21.15.114:0/419115752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b031980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 -- 172.21.15.114:0/419115752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 msgr2=0x56208b031980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b031980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.223 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 -- 172.21.15.114:0/419115752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 msgr2=0x56208b031440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 0x56208b031440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.220+0000 7f0f0be00640 1 -- 172.21.15.114:0/419115752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208acf4930 con 0x56208ac89350 2024-09-21T07:08:00.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.221+0000 7f0f0be00640 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 0x56208b030f00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0efc02a880 tx=0x7f0efc0094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d70fc4153bfd5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.222+0000 7f0f01c00640 1 -- 172.21.15.114:0/419115752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc047070 con 0x56208ac89350 2024-09-21T07:08:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.222+0000 7f0f01c00640 1 -- 172.21.15.114:0/419115752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc038040 con 0x56208ac89350 2024-09-21T07:08:00.225 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.222+0000 7f0f01c00640 1 -- 172.21.15.114:0/419115752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc037030 con 0x56208ac89350 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.223+0000 7f0f09600640 1 -- 172.21.15.114:0/2919341852 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0ef8005220 con 0x7f0f0c11cba0 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.223+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/419115752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 msgr2=0x56208b030f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.223+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 0x56208b030f00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0efc02a880 tx=0x7f0efc0094e0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.224+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/419115752 shutdown_connections 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.224+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b031980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.224+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 0x56208b031440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.227 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.224+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/419115752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac89350 0x56208b030f00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.224+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/419115752 >> 172.21.15.114:0/419115752 conn(0x56208ac7d740 msgr2=0x56208b02edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.225+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/419115752 shutdown_connections 2024-09-21T07:08:00.228 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.225+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/419115752 wait complete. 2024-09-21T07:08:00.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.226+0000 7f0f1a1d3740 1 -- start start 2024-09-21T07:08:00.229 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.226+0000 7f0f1a1d3740 1 Processor -- start 2024-09-21T07:08:00.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.227+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 0x56208b03cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.230 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b03d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac89350 0x56208b048a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.231 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56208adf7890 con 0x56208ac60490 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56208af31160 con 0x56208ac89350 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56208b03f680 con 0x56208ac68670 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f1a1d3740 10 client.0 _ll_register_callbacks cb 0x56208af4da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f10a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b03d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f10a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b03d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57802/0 (socket says 172.21.15.114:57802) 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.228+0000 7f0f10a00640 1 -- 172.21.15.114:0/2252596598 learned_addr learned my addr 172.21.15.114:0/2252596598 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:00.233 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f10a00640 1 -- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac89350 msgr2=0x56208b048a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f10a00640 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac89350 0x56208b048a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f10a00640 1 -- 172.21.15.114:0/2252596598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 msgr2=0x56208b03cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f10a00640 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 0x56208b03cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f10a00640 1 -- 172.21.15.114:0/2252596598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208b048f70 con 0x56208ac68670 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f09600640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0ef00b6610 0x7f0ef00b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f10a00640 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b03d000 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009fd0 tx=0x7f0f0400af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a163113658993bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0f09600640 1 -- 172.21.15.114:0/2919341852 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f0ef80e72b0 con 0x7f0f0c11cba0 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.229+0000 7f0ef6000640 1 -- 172.21.15.114:0/2252596598 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04018020 con 0x56208ac68670 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.230+0000 7f0ef6000640 1 -- 172.21.15.114:0/2252596598 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f0f04014400 con 0x56208ac68670 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.230+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0efc038040 con 0x56208ac68670 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.230+0000 7f0ef6000640 1 -- 172.21.15.114:0/2252596598 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0f04014600 con 0x56208ac68670 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.230+0000 7f0f1a1d3740 10 client.28879 fetch_fsmap learned FSMap version 54 2024-09-21T07:08:00.234 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.231+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56208b04b8d0 con 0x56208ac68670 2024-09-21T07:08:00.235 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.231+0000 7f0ef6000640 1 -- 172.21.15.114:0/2252596598 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc038040 con 0x56208ac68670 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.231+0000 7f0ef6000640 1 -- 172.21.15.114:0/2252596598 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f0f0401c900 con 0x56208ac68670 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.232+0000 7f0f0be00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0ef00b6610 0x7f0ef00b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f1a1d3740 10 client.28879 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f1a1d3740 10 client.28879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f1a1d3740 20 client.28879 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac47cd0 0x56208b04ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f1a1d3740 4 client.28879 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56208afa8de0 con 0x56208ac47cd0 2024-09-21T07:08:00.236 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.233+0000 7f0f0be00640 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac47cd0 0x56208b04ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.236+0000 7f0f0be00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0ef00b6610 0x7f0ef00b8b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0efc000c00 tx=0x7f0efc005320 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:00.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.236+0000 7f0f0be00640 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac47cd0 0x56208b04ef60 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0682bfb2ca1dfe0 server_cookie=326da09d2ce17b68 in_seq=0 out_seq=0 2024-09-21T07:08:00.239 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.236+0000 7f0ef6000640 10 client.28879 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:00.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.237+0000 7f0ef6000640 1 -- 172.21.15.114:0/2252596598 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56208afa8de0 con 0x56208ac47cd0 2024-09-21T07:08:00.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.237+0000 7f0ef6000640 10 client.28879 handle_command_reply: tid=0 2024-09-21T07:08:00.240 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.237+0000 7f0f1a1d3740 1 client.28879 shutdown 2024-09-21T07:08:00.243 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.238+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 msgr2=0x56208b03d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.238+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b03d000 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009fd0 tx=0x7f0f0400af90 comp rx=0 tx=0).stop 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 shutdown_connections 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac89350 0x56208b048a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac47cd0 0x56208b04ef60 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b03d000 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/2252596598 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac60490 0x56208b03cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0ef6000640 0 client.28879 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:00.244 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 >> 172.21.15.114:0/2252596598 conn(0x56208ac7d740 msgr2=0x56208b034090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.241+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 shutdown_connections 2024-09-21T07:08:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.242+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/2252596598 wait complete. 2024-09-21T07:08:00.245 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.242+0000 7f0f1a1d3740 20 client.28879 trim_cache size 0 max 16384 2024-09-21T07:08:00.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.283+0000 7f0f1a1d3740 1 Processor -- start 2024-09-21T07:08:00.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 -- start start 2024-09-21T07:08:00.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac60490 0x56208b02e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b02e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 0x56208b039ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec010040 con 0x56208ac89350 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec011c70 con 0x56208ac60490 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.285+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec010710 con 0x56208ac68670 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.286+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b02e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.286+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b02e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:57812/0 (socket says 172.21.15.114:57812) 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.286+0000 7f0f0be00640 1 -- 172.21.15.114:0/4166998264 learned_addr learned my addr 172.21.15.114:0/4166998264 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.286+0000 7f0f10a00640 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac60490 0x56208b02e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.286+0000 7f0f0be00640 1 -- 172.21.15.114:0/4166998264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac60490 msgr2=0x56208b02e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.286+0000 7f0f0be00640 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac60490 0x56208b02e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.287+0000 7f0f0be00640 1 -- 172.21.15.114:0/4166998264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 msgr2=0x56208b039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.287+0000 7f0f0be00640 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 0x56208b039ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.287+0000 7f0f0be00640 1 -- 172.21.15.114:0/4166998264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56208b03db80 con 0x56208ac68670 2024-09-21T07:08:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.287+0000 7f0f0be00640 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b02e8e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0052f0 tx=0x7f0efc053030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce009b6fdbbaa22c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.288+0000 7f0f01200640 1 -- 172.21.15.114:0/4166998264 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc047040 con 0x56208ac68670 2024-09-21T07:08:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.288+0000 7f0f01200640 1 -- 172.21.15.114:0/4166998264 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc0309a0 con 0x56208ac68670 2024-09-21T07:08:00.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.288+0000 7f0f01200640 1 -- 172.21.15.114:0/4166998264 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc037040 con 0x56208ac68670 2024-09-21T07:08:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.289+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/4166998264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 msgr2=0x56208b02e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.289+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b02e8e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0052f0 tx=0x7f0efc053030 comp rx=0 tx=0).stop 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/4166998264 shutdown_connections 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 0x56208b039ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac68670 0x56208b02e8e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/4166998264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac60490 0x56208b02e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/4166998264 >> 172.21.15.114:0/4166998264 conn(0x56208ac7d740 msgr2=0x56208b03b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/4166998264 shutdown_connections 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.290+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/4166998264 wait complete. 2024-09-21T07:08:00.294 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.291+0000 7f0f1a1d3740 1 -- start start 2024-09-21T07:08:00.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.293+0000 7f0f1a1d3740 1 Processor -- start 2024-09-21T07:08:00.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac60490 0x56208b044960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b044ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 0x56208b0b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec011540 con 0x56208ac89350 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec010de0 con 0x56208ac68670 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec00e040 con 0x56208ac60490 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f1a1d3740 10 client.0 _ll_register_callbacks cb 0x7f0f04009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b044ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b044ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54106/0 (socket says 172.21.15.114:54106) 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.296+0000 7f0f0be00640 1 -- 172.21.15.114:0/1216821952 learned_addr learned my addr 172.21.15.114:0/1216821952 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.297+0000 7f0f10a00640 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac60490 0x56208b044960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.297+0000 7f0f10a00640 1 -- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 msgr2=0x56208b044ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.297+0000 7f0f10a00640 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b044ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.297+0000 7f0f10a00640 1 -- 172.21.15.114:0/1216821952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 msgr2=0x56208b0b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.297+0000 7f0f10a00640 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 0x56208b0b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.297+0000 7f0f10a00640 1 -- 172.21.15.114:0/1216821952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56208b045670 con 0x56208ac60490 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.298+0000 7f0f10a00640 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac60490 0x56208b044960 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f0401f040 tx=0x7f0f04014f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84647ff0b287605d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:00.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.298+0000 7f0ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.298+0000 7f0ef6000640 1 -- 172.21.15.114:0/1216821952 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04018040 con 0x56208ac60490 2024-09-21T07:08:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.298+0000 7f0ef6000640 1 -- 172.21.15.114:0/1216821952 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f0f0401c5a0 con 0x56208ac60490 2024-09-21T07:08:00.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.298+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0efc0309a0 con 0x56208ac60490 2024-09-21T07:08:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.300+0000 7f0ef6000640 1 -- 172.21.15.114:0/1216821952 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0f04037050 con 0x56208ac60490 2024-09-21T07:08:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.300+0000 7f0f1a1d3740 10 client.28885 fetch_fsmap learned FSMap version 54 2024-09-21T07:08:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.300+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56208b0caf50 con 0x56208ac60490 2024-09-21T07:08:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.300+0000 7f0ef6000640 1 -- 172.21.15.114:0/1216821952 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc0309a0 con 0x56208ac60490 2024-09-21T07:08:00.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0ef6000640 1 -- 172.21.15.114:0/1216821952 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 54) ==== 8394+0+0 (secure 0 0 0) 0x7f0f04026a10 con 0x56208ac60490 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f1a1d3740 10 client.28885 fetch_fsmap finished waiting for FSMap version 54 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f1a1d3740 10 client.28885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f1a1d3740 20 client.28885 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac1d080 0x56208b0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f1a1d3740 4 client.28885 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56208afa4890 con 0x56208ac1d080 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.301+0000 7f0f0be00640 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac1d080 0x56208b0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.302+0000 7f0f0be00640 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac1d080 0x56208b0cd7a0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b43f89a49f15ad server_cookie=8a9855cf03f4148b in_seq=0 out_seq=0 2024-09-21T07:08:00.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.302+0000 7f0ef6000640 10 client.28885 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:00.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.306+0000 7f0ef6000640 1 -- 172.21.15.114:0/1216821952 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56208afa4890 con 0x56208ac1d080 2024-09-21T07:08:00.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.306+0000 7f0ef6000640 10 client.28885 handle_command_reply: tid=0 2024-09-21T07:08:00.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.306+0000 7f0f1a1d3740 1 client.28885 shutdown 2024-09-21T07:08:00.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac60490 msgr2=0x56208b044960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac60490 0x56208b044960 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f0401f040 tx=0x7f0f04014f30 comp rx=0 tx=0).stop 2024-09-21T07:08:00.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 shutdown_connections 2024-09-21T07:08:00.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56208ac89350 0x56208b0b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56208ac68670 0x56208b044ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56208ac1d080 0x56208b0cd7a0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 --2- 172.21.15.114:0/1216821952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56208ac60490 0x56208b044960 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 >> 172.21.15.114:0/1216821952 conn(0x56208ac7d740 msgr2=0x56208b034000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0ef6000640 0 client.28885 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 shutdown_connections 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 1 -- 172.21.15.114:0/1216821952 wait complete. 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.307+0000 7f0f1a1d3740 20 client.28885 trim_cache size 0 max 16384 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "68277b18-d8ea-49d2-baab-e2089ee24e2b", 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:08:00.311 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:08:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.310+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0ef00b6610 msgr2=0x7f0ef00b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.310+0000 7f0f11e00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0ef00b6610 0x7f0ef00b8b20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0efc000c00 tx=0x7f0efc005320 comp rx=0 tx=0).stop 2024-09-21T07:08:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.310+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 msgr2=0x7f0f0c1ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:00.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.310+0000 7f0f11e00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 0x7f0f0c1ae9f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800d680 tx=0x7f0ef800db40 comp rx=0 tx=0).stop 2024-09-21T07:08:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 shutdown_connections 2024-09-21T07:08:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0ef00b6610 0x7f0ef00b8b20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c11cba0 0x7f0f0c1ae9f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.314 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0f0c111cd0 0x7f0f0c1a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 --2- 172.21.15.114:0/2919341852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f0c10dbf0 0x7f0f0c1a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 >> 172.21.15.114:0/2919341852 conn(0x7f0f0c06e9f0 msgr2=0x7f0f0c110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 shutdown_connections 2024-09-21T07:08:00.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:00.311+0000 7f0f11e00640 1 -- 172.21.15.114:0/2919341852 wait complete. 2024-09-21T07:08:01.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: pgmap v824: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:08:01.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:01 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: pgmap v824: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:08:01.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:01 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: pgmap v824: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 3.7 MiB/s wr, 409 op/s 2024-09-21T07:08:01.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:08:01.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:01 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.061 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:08:02.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:08:02.318 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:08:02.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:02.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:02 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:02.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:02 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:08:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:08:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:02.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:02 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:08:02.708 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:02.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 805.0 s, 360.6 tps, lat 285.489 ms stddev 341.377 2024-09-21T07:08:03.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:03 smithi114 ceph-mon[30654]: pgmap v825: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 4.2 MiB/s wr, 463 op/s 2024-09-21T07:08:03.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:03 smithi193 ceph-mon[34601]: pgmap v825: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 4.2 MiB/s wr, 463 op/s 2024-09-21T07:08:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:03 smithi183 ceph-mon[34219]: pgmap v825: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 4.2 MiB/s wr, 463 op/s 2024-09-21T07:08:05.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:05 smithi193 ceph-mon[34601]: pgmap v826: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-09-21T07:08:05.509 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:05 smithi114 ceph-mon[30654]: pgmap v826: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-09-21T07:08:05.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:05 smithi183 ceph-mon[34219]: pgmap v826: 129 pgs: 129 active+clean; 3.0 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 -- 172.21.15.114:0/4255902268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 msgr2=0x7f12cc104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 --2- 172.21.15.114:0/4255902268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 0x7f12cc104bd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009860 tx=0x7f12bc02a340 comp rx=0 tx=0).stop 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 -- 172.21.15.114:0/4255902268 shutdown_connections 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 --2- 172.21.15.114:0/4255902268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc10dc20 0x7f12cc110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 --2- 172.21.15.114:0/4255902268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc105110 0x7f12cc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 --2- 172.21.15.114:0/4255902268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 0x7f12cc104bd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.407 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.403+0000 7f12d1200640 1 -- 172.21.15.114:0/4255902268 >> 172.21.15.114:0/4255902268 conn(0x7f12cc0fe2c0 msgr2=0x7f12cc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:06.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.404+0000 7f12d1200640 1 -- 172.21.15.114:0/4255902268 shutdown_connections 2024-09-21T07:08:06.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.404+0000 7f12d1200640 1 -- 172.21.15.114:0/4255902268 wait complete. 2024-09-21T07:08:06.408 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.406+0000 7f12d1200640 1 Processor -- start 2024-09-21T07:08:06.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.406+0000 7f12d1200640 1 -- start start 2024-09-21T07:08:06.409 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12d1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 0x7f12cc19c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12d1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 0x7f12cc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12d1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 0x7f12cc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 0x7f12cc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 0x7f12cc19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44978/0 (socket says 172.21.15.114:44978) 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 0x7f12cc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 0x7f12cc1a1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59754/0 (socket says 172.21.15.114:59754) 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12caa00640 1 -- 172.21.15.114:0/377735265 learned_addr learned my addr 172.21.15.114:0/377735265 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc113880 con 0x7f12cc1047c0 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc113700 con 0x7f12cc10dc20 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.407+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc113a00 con 0x7f12cc105110 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12cbe00640 1 -- 172.21.15.114:0/377735265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 msgr2=0x7f12cc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12cbe00640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 0x7f12cc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12cbe00640 1 -- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 msgr2=0x7f12cc19c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12cbe00640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 0x7f12cc19c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12cbe00640 1 -- 172.21.15.114:0/377735265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1a24c0 con 0x7f12cc10dc20 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12caa00640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 0x7f12cc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:06.412 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.408+0000 7f12cbe00640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 0x7f12cc1a1d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f12c400ef00 tx=0x7f12c400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:06.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.409+0000 7f12c3e00640 1 -- 172.21.15.114:0/377735265 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4004b20 con 0x7f12cc10dc20 2024-09-21T07:08:06.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.409+0000 7f12c3e00640 1 -- 172.21.15.114:0/377735265 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12c4003e20 con 0x7f12cc10dc20 2024-09-21T07:08:06.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.409+0000 7f12c3e00640 1 -- 172.21.15.114:0/377735265 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c400b570 con 0x7f12cc10dc20 2024-09-21T07:08:06.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.409+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc1a27b0 con 0x7f12cc10dc20 2024-09-21T07:08:06.413 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.409+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc105600 con 0x7f12cc10dc20 2024-09-21T07:08:06.421 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.418+0000 7f12c3e00640 1 -- 172.21.15.114:0/377735265 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f12c401d020 con 0x7f12cc10dc20 2024-09-21T07:08:06.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.420+0000 7f12c3e00640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12b00b6680 0x7f12b00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.423 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.421+0000 7f12c3e00640 1 -- 172.21.15.114:0/377735265 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f12c40e8920 con 0x7f12cc10dc20 2024-09-21T07:08:06.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.424+0000 7f12cb400640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12b00b6680 0x7f12b00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.453+0000 7f12cb400640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12b00b6680 0x7f12b00b8b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0095f0 tx=0x7f12bc009580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:06.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.453+0000 7f12d93e5740 1 Processor -- start 2024-09-21T07:08:06.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 -- start start 2024-09-21T07:08:06.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b75dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b75e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b75e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56414b6d4f70 con 0x56414b396390 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56414b6c43a0 con 0x56414b38e1b0 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.455+0000 7f12d93e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56414b6c81b0 con 0x56414b3b7070 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b75dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b75e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12cb400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b75dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:44990/0 (socket says 172.21.15.114:44990) 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b75e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59770/0 (socket says 172.21.15.114:59770) 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12cb400640 1 -- 172.21.15.114:0/1908412711 learned_addr learned my addr 172.21.15.114:0/1908412711 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12cbe00640 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b75e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12caa00640 1 -- 172.21.15.114:0/1908412711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 msgr2=0x56414b75dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12caa00640 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b75dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12caa00640 1 -- 172.21.15.114:0/1908412711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 msgr2=0x56414b75e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.456+0000 7f12caa00640 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b75e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.457+0000 7f12caa00640 1 -- 172.21.15.114:0/1908412711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414b68ec90 con 0x56414b38e1b0 2024-09-21T07:08:06.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.459+0000 7f12caa00640 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b75e230 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009a70 tx=0x7f12b8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a3295cbf55ea53 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.459+0000 7f12c0c00640 1 -- 172.21.15.114:0/1908412711 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8022020 con 0x56414b38e1b0 2024-09-21T07:08:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.459+0000 7f12c0c00640 1 -- 172.21.15.114:0/1908412711 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b800bbc0 con 0x56414b38e1b0 2024-09-21T07:08:06.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.459+0000 7f12c0c00640 1 -- 172.21.15.114:0/1908412711 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b801d5c0 con 0x56414b38e1b0 2024-09-21T07:08:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 -- 172.21.15.114:0/1908412711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 msgr2=0x56414b75e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b75e230 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009a70 tx=0x7f12b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 -- 172.21.15.114:0/1908412711 shutdown_connections 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b75e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b75e230 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 --2- 172.21.15.114:0/1908412711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b75dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 -- 172.21.15.114:0/1908412711 >> 172.21.15.114:0/1908412711 conn(0x56414b3ab460 msgr2=0x56414b75bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 -- 172.21.15.114:0/1908412711 shutdown_connections 2024-09-21T07:08:06.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.461+0000 7f12d93e5740 1 -- 172.21.15.114:0/1908412711 wait complete. 2024-09-21T07:08:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.462+0000 7f12d93e5740 1 -- start start 2024-09-21T07:08:06.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.462+0000 7f12d93e5740 1 Processor -- start 2024-09-21T07:08:06.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.464+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b38e1b0 0x56414b7697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.464+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 0x56414b769d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.464+0000 7f12caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 0x56414b769d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b7867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12caa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 0x56414b769d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59786/0 (socket says 172.21.15.114:59786) 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12cb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b38e1b0 0x56414b7697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12cb400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b38e1b0 0x56414b7697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:58306/0 (socket says 172.21.15.114:58306) 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12cb400640 1 -- 172.21.15.114:0/2659722316 learned_addr learned my addr 172.21.15.114:0/2659722316 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56414b525700 con 0x56414b38e1b0 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56414b65f540 con 0x56414b396390 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56414b77d4e0 con 0x56414b3b7070 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.465+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b7867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12cbe00640 1 -- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 msgr2=0x56414b769d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 0x56414b769d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12d93e5740 10 client.0 _ll_register_callbacks cb 0x56414b66f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12cbe00640 1 -- 172.21.15.114:0/2659722316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b38e1b0 msgr2=0x56414b7697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b38e1b0 0x56414b7697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12cbe00640 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414b422650 con 0x56414b3b7070 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12caa00640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 0x56414b769d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:06.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.466+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56414b661b50 con 0x56414b3b7070 2024-09-21T07:08:06.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.467+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b7867d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f12c40e5ca0 tx=0x7f12c40adcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcc35c3088dc550 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:06.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.467+0000 7f12b5600640 10 client.28897 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:06.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.470+0000 7f12b5600640 1 -- 172.21.15.114:0/2659722316 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4095050 con 0x56414b3b7070 2024-09-21T07:08:06.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.470+0000 7f12b5600640 1 -- 172.21.15.114:0/2659722316 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f12c40995d0 con 0x56414b3b7070 2024-09-21T07:08:06.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.470+0000 7f12b5600640 1 -- 172.21.15.114:0/2659722316 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12c401a040 con 0x56414b3b7070 2024-09-21T07:08:06.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.470+0000 7f12d93e5740 10 client.28897 fetch_fsmap learned FSMap version 55 2024-09-21T07:08:06.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.470+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=55}) -- 0x56414b67c910 con 0x56414b3b7070 2024-09-21T07:08:06.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12b5600640 1 -- 172.21.15.114:0/2659722316 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7f12c400b040 con 0x56414b3b7070 2024-09-21T07:08:06.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12d93e5740 10 client.28897 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T07:08:06.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12d93e5740 10 client.28897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:06.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12d93e5740 20 client.28897 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:06.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b3759f0 0x56414b78caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12d93e5740 4 client.28897 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:06.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.471+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56414b6d6ec0 con 0x56414b3759f0 2024-09-21T07:08:06.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.475+0000 7f12cb400640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b3759f0 0x56414b78caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.480+0000 7f12b5600640 10 client.28897 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:06.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.480+0000 7f12cb400640 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b3759f0 0x56414b78caf0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba65bc75f106bd7 server_cookie=d2c4d539a60577ce in_seq=0 out_seq=0 2024-09-21T07:08:06.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12b5600640 1 -- 172.21.15.114:0/2659722316 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56414b6d6ec0 con 0x56414b3759f0 2024-09-21T07:08:06.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12b5600640 10 client.28897 handle_command_reply: tid=0 2024-09-21T07:08:06.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12d93e5740 1 client.28897 shutdown 2024-09-21T07:08:06.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 msgr2=0x56414b7867d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b7867d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f12c40e5ca0 tx=0x7f12c40adcf0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 shutdown_connections 2024-09-21T07:08:06.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b3759f0 0x56414b78caf0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.482+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b3b7070 0x56414b7867d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b396390 0x56414b769d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2659722316 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b38e1b0 0x56414b7697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 >> 172.21.15.114:0/2659722316 conn(0x56414b3ab460 msgr2=0x56414b760e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:06.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12b5600640 0 client.28897 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:06.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 shutdown_connections 2024-09-21T07:08:06.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12d93e5740 1 -- 172.21.15.114:0/2659722316 wait complete. 2024-09-21T07:08:06.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.483+0000 7f12d93e5740 20 client.28897 trim_cache size 0 max 16384 2024-09-21T07:08:06.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.520+0000 7f12d93e5740 1 Processor -- start 2024-09-21T07:08:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.522+0000 7f12d93e5740 1 -- start start 2024-09-21T07:08:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.523+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b38e1b0 0x56414b75f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.525 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.523+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b7866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.523+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 0x56414b786c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.523+0000 7f12d93e5740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc072c80 con 0x56414b396390 2024-09-21T07:08:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.523+0000 7f12d93e5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc0748b0 con 0x56414b3b7070 2024-09-21T07:08:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.523+0000 7f12d93e5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc073350 con 0x56414b38e1b0 2024-09-21T07:08:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.524+0000 7f12cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 0x56414b786c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.526 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.524+0000 7f12cbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 0x56414b786c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59798/0 (socket says 172.21.15.114:59798) 2024-09-21T07:08:06.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.524+0000 7f12cbe00640 1 -- 172.21.15.114:0/2044459135 learned_addr learned my addr 172.21.15.114:0/2044459135 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:06.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.524+0000 7f12cb400640 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b38e1b0 0x56414b75f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.525+0000 7f12caa00640 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b7866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.525+0000 7f12cbe00640 1 -- 172.21.15.114:0/2044459135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b38e1b0 msgr2=0x56414b75f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.525+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b38e1b0 0x56414b75f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.525+0000 7f12cbe00640 1 -- 172.21.15.114:0/2044459135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 msgr2=0x56414b7866d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.525+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b7866d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.525+0000 7f12cbe00640 1 -- 172.21.15.114:0/2044459135 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56414b76a8b0 con 0x56414b3b7070 2024-09-21T07:08:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.526+0000 7f12cbe00640 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 0x56414b786c10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f12c40aa450 tx=0x7f12c40ade20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=239fe5131e719855 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:06.529 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.527+0000 7f12b7e00640 1 -- 172.21.15.114:0/2044459135 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4095070 con 0x56414b3b7070 2024-09-21T07:08:06.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.527+0000 7f12b7e00640 1 -- 172.21.15.114:0/2044459135 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56414b661b50 con 0x56414b3b7070 2024-09-21T07:08:06.530 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.527+0000 7f12b7e00640 1 -- 172.21.15.114:0/2044459135 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4095ce0 con 0x56414b3b7070 2024-09-21T07:08:06.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.529+0000 7f12d93e5740 1 -- 172.21.15.114:0/2044459135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 msgr2=0x56414b786c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 0x56414b786c10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f12c40aa450 tx=0x7f12c40ade20 comp rx=0 tx=0).stop 2024-09-21T07:08:06.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 -- 172.21.15.114:0/2044459135 shutdown_connections 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b3b7070 0x56414b786c10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b396390 0x56414b7866d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 --2- 172.21.15.114:0/2044459135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b38e1b0 0x56414b75f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 -- 172.21.15.114:0/2044459135 >> 172.21.15.114:0/2044459135 conn(0x56414b3ab460 msgr2=0x56414b75ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.530+0000 7f12d93e5740 1 -- 172.21.15.114:0/2044459135 shutdown_connections 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.531+0000 7f12d93e5740 1 -- 172.21.15.114:0/2044459135 wait complete. 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.532+0000 7f12d93e5740 1 -- start start 2024-09-21T07:08:06.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.532+0000 7f12d93e5740 1 Processor -- start 2024-09-21T07:08:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.534+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b782740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.534+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b396390 0x56414b782ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.534+0000 7f12d93e5740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b3b7070 0x56414b77d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.534+0000 7f12cb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b782740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.535+0000 7f12cb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b782740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59810/0 (socket says 172.21.15.114:59810) 2024-09-21T07:08:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.535+0000 7f12caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b396390 0x56414b782ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.535+0000 7f12caa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b396390 0x56414b782ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45018/0 (socket says 172.21.15.114:45018) 2024-09-21T07:08:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.535+0000 7f12caa00640 1 -- 172.21.15.114:0/224616888 learned_addr learned my addr 172.21.15.114:0/224616888 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.535+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc074180 con 0x56414b3b7070 2024-09-21T07:08:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.535+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc073a20 con 0x56414b38e1b0 2024-09-21T07:08:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.536+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc070ce0 con 0x56414b396390 2024-09-21T07:08:06.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.536+0000 7f12d93e5740 10 client.0 _ll_register_callbacks cb 0x7f12c40aa260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.536+0000 7f12cb400640 1 -- 172.21.15.114:0/224616888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b396390 msgr2=0x56414b782ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.536+0000 7f12cb400640 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b396390 0x56414b782ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.536+0000 7f12cb400640 1 -- 172.21.15.114:0/224616888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b3b7070 msgr2=0x56414b77d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:06.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.536+0000 7f12cb400640 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b3b7070 0x56414b77d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.537+0000 7f12cb400640 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56414b77dc30 con 0x56414b38e1b0 2024-09-21T07:08:06.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.538+0000 7f12cb400640 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b782740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x56414b78d050 tx=0x7f12bc039000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894a385aaa4cf207 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:06.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.538+0000 7f12b5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:06.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.538+0000 7f12b5600640 1 -- 172.21.15.114:0/224616888 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc037040 con 0x56414b38e1b0 2024-09-21T07:08:06.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.539+0000 7f12b5600640 1 -- 172.21.15.114:0/224616888 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7f12bc030850 con 0x56414b38e1b0 2024-09-21T07:08:06.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.540+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56414b661b50 con 0x56414b38e1b0 2024-09-21T07:08:06.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.540+0000 7f12b5600640 1 -- 172.21.15.114:0/224616888 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12bc059050 con 0x56414b38e1b0 2024-09-21T07:08:06.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.540+0000 7f12d93e5740 10 client.28943 fetch_fsmap learned FSMap version 55 2024-09-21T07:08:06.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.540+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56414b7f7ca0 con 0x56414b38e1b0 2024-09-21T07:08:06.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.541+0000 7f12b5600640 1 -- 172.21.15.114:0/224616888 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56414b661b50 con 0x56414b38e1b0 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.542+0000 7f12b5600640 1 -- 172.21.15.114:0/224616888 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7f12bc04a030 con 0x56414b38e1b0 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12d93e5740 10 client.28943 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12d93e5740 10 client.28943 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12d93e5740 20 client.28943 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12d93e5740 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b34ada0 0x56414b7fa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12d93e5740 4 client.28943 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56414b6d2970 con 0x56414b34ada0 2024-09-21T07:08:06.546 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.543+0000 7f12caa00640 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b34ada0 0x56414b7fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.544+0000 7f12caa00640 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b34ada0 0x56414b7fa380 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a753fa2a5763181 server_cookie=270e51f01e122abc in_seq=0 out_seq=0 2024-09-21T07:08:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.544+0000 7f12b5600640 10 client.28943 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.544+0000 7f12b5600640 1 -- 172.21.15.114:0/224616888 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x56414b6d2970 con 0x56414b34ada0 2024-09-21T07:08:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.544+0000 7f12b5600640 10 client.28943 handle_command_reply: tid=0 2024-09-21T07:08:06.547 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.544+0000 7f12d93e5740 1 client.28943 shutdown 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 msgr2=0x56414b782740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b782740 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x56414b78d050 tx=0x7f12bc039000 comp rx=0 tx=0).stop 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 shutdown_connections 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56414b3b7070 0x56414b77d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56414b34ada0 0x56414b7fa380 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56414b396390 0x56414b782ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.548 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 --2- 172.21.15.114:0/224616888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56414b38e1b0 0x56414b782740 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12b5600640 0 client.28943 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.545+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 >> 172.21.15.114:0/224616888 conn(0x56414b3ab460 msgr2=0x56414b76aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.546+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 shutdown_connections 2024-09-21T07:08:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.546+0000 7f12d93e5740 1 -- 172.21.15.114:0/224616888 wait complete. 2024-09-21T07:08:06.549 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.546+0000 7f12d93e5740 20 client.28943 trim_cache size 0 max 16384 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "68277b18-d8ea-49d2-baab-e2089ee24e2b": { 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "68277b18-d8ea-49d2-baab-e2089ee24e2b", 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.552 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "68277b18-d8ea-49d2-baab-e2089ee24e2b": { 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:08:00.304805+0000", 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.553 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.554 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.555 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:08:06.556 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.558 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.559 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:06:54.030598+0000", 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:06.560 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12b00b6680 msgr2=0x7f12b00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12b00b6680 0x7f12b00b8b90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0095f0 tx=0x7f12bc009580 comp rx=0 tx=0).stop 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 msgr2=0x7f12cc1a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 0x7f12cc1a1d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f12c400ef00 tx=0x7f12c400c740 comp rx=0 tx=0).stop 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 shutdown_connections 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f12b00b6680 0x7f12b00b8b90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12cc10dc20 0x7f12cc1a1d90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12cc105110 0x7f12cc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 --2- 172.21.15.114:0/377735265 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f12cc1047c0 0x7f12cc19c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.547+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 >> 172.21.15.114:0/377735265 conn(0x7f12cc0fe2c0 msgr2=0x7f12cc102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.548+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 shutdown_connections 2024-09-21T07:08:06.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:06.548+0000 7f12d1200640 1 -- 172.21.15.114:0/377735265 wait complete. 2024-09-21T07:08:07.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:07 smithi114 ceph-mon[30654]: pgmap v827: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-09-21T07:08:07.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:07 smithi193 ceph-mon[34601]: pgmap v827: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-09-21T07:08:07.523 INFO:tasks.cephfs.filesystem:scrub status for tag:68277b18-d8ea-49d2-baab-e2089ee24e2b - {'path': '/', 'tag': '68277b18-d8ea-49d2-baab-e2089ee24e2b', 'options': 'recursive,force'} 2024-09-21T07:08:07.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:07 smithi183 ceph-mon[34219]: pgmap v827: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 3.8 MiB/s wr, 510 op/s 2024-09-21T07:08:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 -- 172.21.15.114:0/91493708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0757b0 msgr2=0x7fc93c075bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 --2- 172.21.15.114:0/91493708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0757b0 0x7fc93c075bc0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009900 tx=0x7fc92c02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 -- 172.21.15.114:0/91493708 shutdown_connections 2024-09-21T07:08:07.754 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 --2- 172.21.15.114:0/91493708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c10fe60 0x7fc93c1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 --2- 172.21.15.114:0/91493708 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc93c076100 0x7fc93c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 --2- 172.21.15.114:0/91493708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0757b0 0x7fc93c075bc0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 -- 172.21.15.114:0/91493708 >> 172.21.15.114:0/91493708 conn(0x7fc93c0ff330 msgr2=0x7fc93c101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 -- 172.21.15.114:0/91493708 shutdown_connections 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.750+0000 7fc942e00640 1 -- 172.21.15.114:0/91493708 wait complete. 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.752+0000 7fc942e00640 1 Processor -- start 2024-09-21T07:08:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.752+0000 7fc942e00640 1 -- start start 2024-09-21T07:08:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.753+0000 7fc942e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0757b0 0x7fc93c1b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.753+0000 7fc942e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c076100 0x7fc93c1b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc942e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc93c10fe60 0x7fc93c1ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0757b0 0x7fc93c1b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc942e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c06d520 con 0x7fc93c10fe60 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc942e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c06d3a0 con 0x7fc93c0757b0 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0757b0 0x7fc93c1b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59836/0 (socket says 172.21.15.114:59836) 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc941a00640 1 -- 172.21.15.114:0/1957883154 learned_addr learned my addr 172.21.15.114:0/1957883154 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c06d6a0 con 0x7fc93c076100 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.754+0000 7fc941000640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c076100 0x7fc93c1b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.755+0000 7fc942400640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc93c10fe60 0x7fc93c1ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.755+0000 7fc941000640 1 -- 172.21.15.114:0/1957883154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0757b0 msgr2=0x7fc93c1b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.755+0000 7fc941000640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0757b0 0x7fc93c1b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.755+0000 7fc941000640 1 -- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc93c10fe60 msgr2=0x7fc93c1ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.755+0000 7fc941000640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc93c10fe60 0x7fc93c1ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.755+0000 7fc941000640 1 -- 172.21.15.114:0/1957883154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1ad9c0 con 0x7fc93c076100 2024-09-21T07:08:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.756+0000 7fc941000640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c076100 0x7fc93c1b3310 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0777d0 tx=0x7fc93000cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.757+0000 7fc93a000640 1 -- 172.21.15.114:0/1957883154 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc930007bb0 con 0x7fc93c076100 2024-09-21T07:08:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.757+0000 7fc93a000640 1 -- 172.21.15.114:0/1957883154 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc930007d50 con 0x7fc93c076100 2024-09-21T07:08:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.757+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c1adcb0 con 0x7fc93c076100 2024-09-21T07:08:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.757+0000 7fc93a000640 1 -- 172.21.15.114:0/1957883154 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9300186f0 con 0x7fc93c076100 2024-09-21T07:08:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.757+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c109710 con 0x7fc93c076100 2024-09-21T07:08:07.770 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.764+0000 7fc94b640740 1 Processor -- start 2024-09-21T07:08:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.768+0000 7fc94b640740 1 -- start start 2024-09-21T07:08:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.768+0000 7fc94b640740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.769+0000 7fc93a000640 1 -- 172.21.15.114:0/1957883154 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc93000f450 con 0x7fc93c076100 2024-09-21T07:08:07.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc94b640740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396228e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc942400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc942400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45056/0 (socket says 172.21.15.114:45056) 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc942400640 1 -- 172.21.15.114:0/676591352 learned_addr learned my addr 172.21.15.114:0/676591352 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc94b640740 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec84d0 0x56396228e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5639622036b0 con 0x563961ec02f0 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5639621f2ae0 con 0x563961ec84d0 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.770+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5639621f68f0 con 0x563961ee91b0 2024-09-21T07:08:07.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.771+0000 7fc941a00640 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396228e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.771+0000 7fc942400640 1 -- 172.21.15.114:0/676591352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec84d0 msgr2=0x56396228e890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:08:07.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.772+0000 7fc941000640 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec84d0 0x56396228e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc942400640 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec84d0 0x56396228e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc942400640 1 -- 172.21.15.114:0/676591352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 msgr2=0x56396228e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc942400640 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396228e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc942400640 1 -- 172.21.15.114:0/676591352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639621bf780 con 0x563961ee91b0 2024-09-21T07:08:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc941000640 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec84d0 0x56396228e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc942400640 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228de10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc934009e20 tx=0x7fc93400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b779e09a67537e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.773+0000 7fc93a000640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc9240b6730 0x7fc9240b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.775+0000 7fc92aa00640 1 -- 172.21.15.114:0/676591352 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934003b70 con 0x563961ee91b0 2024-09-21T07:08:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.775+0000 7fc92aa00640 1 -- 172.21.15.114:0/676591352 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc934003d10 con 0x563961ee91b0 2024-09-21T07:08:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.775+0000 7fc92aa00640 1 -- 172.21.15.114:0/676591352 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9340118b0 con 0x563961ee91b0 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.775+0000 7fc93a000640 1 -- 172.21.15.114:0/1957883154 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fc9300e7bb0 con 0x7fc93c076100 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.776+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 msgr2=0x56396228de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.776+0000 7fc94b640740 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228de10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc934009e20 tx=0x7fc93400f730 comp rx=0 tx=0).stop 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.776+0000 7fc941a00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc9240b6730 0x7fc9240b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.777+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 shutdown_connections 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.777+0000 7fc94b640740 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec84d0 0x56396228e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.777+0000 7fc94b640740 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396228e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.777+0000 7fc94b640740 1 --2- 172.21.15.114:0/676591352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228de10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.777+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 >> 172.21.15.114:0/676591352 conn(0x563961edd300 msgr2=0x56396228bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.778+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 shutdown_connections 2024-09-21T07:08:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.778+0000 7fc94b640740 1 -- 172.21.15.114:0/676591352 wait complete. 2024-09-21T07:08:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.778+0000 7fc941a00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc9240b6730 0x7fc9240b8c40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005f70 tx=0x7fc92c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.779+0000 7fc94b640740 1 -- start start 2024-09-21T07:08:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.780+0000 7fc94b640740 1 Processor -- start 2024-09-21T07:08:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.781+0000 7fc94b640740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x563962299490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.781+0000 7fc94b640740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 0x5639622999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.781+0000 7fc94b640740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ee91b0 0x5639622a5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.782+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x563962299490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.782+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x563962299490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59860/0 (socket says 172.21.15.114:59860) 2024-09-21T07:08:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.783+0000 7fc941000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 0x5639622999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.784+0000 7fc941000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 0x5639622999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45064/0 (socket says 172.21.15.114:45064) 2024-09-21T07:08:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.784+0000 7fc942400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ee91b0 0x5639622a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.784+0000 7fc94b640740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56396212aab0 con 0x563961ee91b0 2024-09-21T07:08:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.784+0000 7fc94b640740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563962199570 con 0x563961ec02f0 2024-09-21T07:08:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.784+0000 7fc94b640740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56396229c590 con 0x563961ec84d0 2024-09-21T07:08:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.784+0000 7fc94b640740 10 client.0 _ll_register_callbacks cb 0x5639621a8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:07.788 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.785+0000 7fc941a00640 1 -- 172.21.15.114:0/2435959809 learned_addr learned my addr 172.21.15.114:0/2435959809 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.786+0000 7fc941000640 1 -- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 msgr2=0x563962299490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.786+0000 7fc941000640 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x563962299490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.786+0000 7fc941000640 1 -- 172.21.15.114:0/2435959809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ee91b0 msgr2=0x5639622a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.786+0000 7fc941000640 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ee91b0 0x5639622a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.786+0000 7fc941000640 1 -- 172.21.15.114:0/2435959809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563961f54790 con 0x563961ec84d0 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.786+0000 7fc942400640 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ee91b0 0x5639622a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.787+0000 7fc941000640 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 0x5639622999d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc9300e4f30 tx=0x7fc9300acf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b682d8f036d88f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.787+0000 7fc91f400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.787+0000 7fc91f400640 1 -- 172.21.15.114:0/2435959809 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9300add30 con 0x563961ec84d0 2024-09-21T07:08:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.788+0000 7fc91f400640 1 -- 172.21.15.114:0/2435959809 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fc9300a98a0 con 0x563961ec84d0 2024-09-21T07:08:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.788+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc934003d10 con 0x563961ec84d0 2024-09-21T07:08:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.789+0000 7fc91f400640 1 -- 172.21.15.114:0/2435959809 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc9300b2110 con 0x563961ec84d0 2024-09-21T07:08:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.789+0000 7fc94b640740 10 client.28915 fetch_fsmap learned FSMap version 55 2024-09-21T07:08:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.789+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563961fcd620 con 0x563961ec84d0 2024-09-21T07:08:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.790+0000 7fc91f400640 1 -- 172.21.15.114:0/2435959809 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9300b7050 con 0x563961ec84d0 2024-09-21T07:08:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.790+0000 7fc91f400640 1 -- 172.21.15.114:0/2435959809 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7fc9300ac300 con 0x563961ec84d0 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.790+0000 7fc94b640740 10 client.28915 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.790+0000 7fc94b640740 10 client.28915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.790+0000 7fc94b640740 20 client.28915 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.790+0000 7fc94b640740 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563961ea7d80 0x5639622ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.791+0000 7fc94b640740 4 client.28915 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.791+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc934003d10 con 0x563961ea7d80 2024-09-21T07:08:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.791+0000 7fc941a00640 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563961ea7d80 0x5639622ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.792+0000 7fc941a00640 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563961ea7d80 0x5639622ab590 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab7733498fdd691 server_cookie=4afb29f233124976 in_seq=0 out_seq=0 2024-09-21T07:08:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.792+0000 7fc91f400640 10 client.28915 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.793+0000 7fc91f400640 1 -- 172.21.15.114:0/2435959809 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc934003d10 con 0x563961ea7d80 2024-09-21T07:08:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.793+0000 7fc91f400640 10 client.28915 handle_command_reply: tid=0 2024-09-21T07:08:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.793+0000 7fc94b640740 1 client.28915 shutdown 2024-09-21T07:08:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 msgr2=0x5639622999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 0x5639622999d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc9300e4f30 tx=0x7fc9300acf80 comp rx=0 tx=0).stop 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 shutdown_connections 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ee91b0 0x5639622a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563961ea7d80 0x5639622ab590 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ec84d0 0x5639622999d0 secure :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc9300e4f30 tx=0x7fc9300acf80 comp rx=0 tx=0).stop 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 --2- 172.21.15.114:0/2435959809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x563962299490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc91f400640 0 client.28915 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.794+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 >> 172.21.15.114:0/2435959809 conn(0x563961edd300 msgr2=0x56396228ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.795+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 shutdown_connections 2024-09-21T07:08:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.795+0000 7fc94b640740 1 -- 172.21.15.114:0/2435959809 wait complete. 2024-09-21T07:08:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.795+0000 7fc94b640740 20 client.28915 trim_cache size 0 max 16384 2024-09-21T07:08:07.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 810.0 s, 401.6 tps, lat 245.498 ms stddev 288.978 2024-09-21T07:08:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 Processor -- start 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 -- start start 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x56396228ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961e7d1c0 0x563962296660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1c8300 con 0x563961e7d1c0 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1c9f30 con 0x563961ec02f0 2024-09-21T07:08:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.837+0000 7fc94b640740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1c89d0 con 0x563961ee91b0 2024-09-21T07:08:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.838+0000 7fc941000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.838+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x56396228ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.839+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x56396228ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59866/0 (socket says 172.21.15.114:59866) 2024-09-21T07:08:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.839+0000 7fc941000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45078/0 (socket says 172.21.15.114:45078) 2024-09-21T07:08:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.839+0000 7fc941a00640 1 -- 172.21.15.114:0/1766060395 learned_addr learned my addr 172.21.15.114:0/1766060395 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.840+0000 7fc942400640 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961e7d1c0 0x563962296660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.840+0000 7fc941a00640 1 -- 172.21.15.114:0/1766060395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 msgr2=0x56396228f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.840+0000 7fc941a00640 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.840+0000 7fc941a00640 1 -- 172.21.15.114:0/1766060395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961e7d1c0 msgr2=0x563962296660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.840+0000 7fc941a00640 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961e7d1c0 0x563962296660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.840+0000 7fc941a00640 1 -- 172.21.15.114:0/1766060395 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563962286d30 con 0x563961ec02f0 2024-09-21T07:08:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.841+0000 7fc941a00640 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x56396228ee40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc92c000c00 tx=0x7fc92c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa38b61639a9241 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.841+0000 7fc92a000640 1 -- 172.21.15.114:0/1766060395 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c035050 con 0x563961ec02f0 2024-09-21T07:08:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.842+0000 7fc92a000640 1 -- 172.21.15.114:0/1766060395 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c0054d0 con 0x563961ec02f0 2024-09-21T07:08:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.842+0000 7fc92a000640 1 -- 172.21.15.114:0/1766060395 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c003e00 con 0x563961ec02f0 2024-09-21T07:08:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.843+0000 7fc94b640740 1 -- 172.21.15.114:0/1766060395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 msgr2=0x56396228ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.844+0000 7fc94b640740 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x56396228ee40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc92c000c00 tx=0x7fc92c03f000 comp rx=0 tx=0).stop 2024-09-21T07:08:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.844+0000 7fc94b640740 1 -- 172.21.15.114:0/1766060395 shutdown_connections 2024-09-21T07:08:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.844+0000 7fc94b640740 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961e7d1c0 0x563962296660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.844+0000 7fc94b640740 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396228f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.844+0000 7fc94b640740 1 --2- 172.21.15.114:0/1766060395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961ec02f0 0x56396228ee40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.844+0000 7fc94b640740 1 -- 172.21.15.114:0/1766060395 >> 172.21.15.114:0/1766060395 conn(0x563961edd300 msgr2=0x563962297cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.845+0000 7fc94b640740 1 -- 172.21.15.114:0/1766060395 shutdown_connections 2024-09-21T07:08:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.845+0000 7fc94b640740 1 -- 172.21.15.114:0/1766060395 wait complete. 2024-09-21T07:08:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.845+0000 7fc94b640740 1 -- start start 2024-09-21T07:08:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.846+0000 7fc94b640740 1 Processor -- start 2024-09-21T07:08:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.847+0000 7fc94b640740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961e7d1c0 0x5639622a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.847+0000 7fc94b640740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396229e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961e7d1c0 0x5639622a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc941a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961e7d1c0 0x5639622a20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59876/0 (socket says 172.21.15.114:59876) 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc94b640740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396229ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc94b640740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1c9800 con 0x563961ec02f0 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc94b640740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1c90a0 con 0x563961e7d1c0 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc94b640740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1c63c0 con 0x563961ee91b0 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc942400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396229ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc942400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396229ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:45088/0 (socket says 172.21.15.114:45088) 2024-09-21T07:08:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.848+0000 7fc942400640 1 -- 172.21.15.114:0/2724409439 learned_addr learned my addr 172.21.15.114:0/2724409439 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc942400640 1 -- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961e7d1c0 msgr2=0x5639622a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc942400640 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961e7d1c0 0x5639622a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc942400640 1 -- 172.21.15.114:0/2724409439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 msgr2=0x56396229e560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc942400640 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396229e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc942400640 1 -- 172.21.15.114:0/2724409439 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56396229f320 con 0x563961ee91b0 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc94b640740 10 client.0 _ll_register_callbacks cb 0x7fc93000fc60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.849+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c0054d0 con 0x563961ee91b0 2024-09-21T07:08:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.850+0000 7fc942400640 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396229ebb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc934004f40 tx=0x7fc934004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a266777060bf8a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.850+0000 7fc91f400640 10 client.28921 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.850+0000 7fc91f400640 1 -- 172.21.15.114:0/2724409439 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93402c060 con 0x563961ee91b0 2024-09-21T07:08:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.851+0000 7fc91f400640 1 -- 172.21.15.114:0/2724409439 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9490+0+0 (secure 0 0 0) 0x7fc934005450 con 0x563961ee91b0 2024-09-21T07:08:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.851+0000 7fc91f400640 1 -- 172.21.15.114:0/2724409439 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc934005670 con 0x563961ee91b0 2024-09-21T07:08:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.851+0000 7fc94b640740 10 client.28921 fetch_fsmap learned FSMap version 55 2024-09-21T07:08:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.852+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=55}) -- 0x563962316e00 con 0x563961ee91b0 2024-09-21T07:08:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc91f400640 1 -- 172.21.15.114:0/2724409439 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 55) ==== 8390+0+0 (secure 0 0 0) 0x7fc934026070 con 0x563961ee91b0 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc94b640740 10 client.28921 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc94b640740 10 client.28921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc94b640740 20 client.28921 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc94b640740 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563962317520 0x563962319970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc94b640740 4 client.28921 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.853+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563962205600 con 0x563962317520 2024-09-21T07:08:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.854+0000 7fc941a00640 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563962317520 0x563962319970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.855+0000 7fc91f400640 10 client.28921 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:07.855+0000 7fc941a00640 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563962317520 0x563962319970 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee53a11dafdc663 server_cookie=852a779db97e802a in_seq=0 out_seq=0 2024-09-21T07:08:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.122+0000 7fc91f400640 1 -- 172.21.15.114:0/2724409439 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563962205600 con 0x563962317520 2024-09-21T07:08:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.122+0000 7fc91f400640 10 client.28921 handle_command_reply: tid=0 2024-09-21T07:08:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.122+0000 7fc94b640740 1 client.28921 shutdown 2024-09-21T07:08:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.122+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 msgr2=0x56396229ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.122+0000 7fc94b640740 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396229ebb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc934004f40 tx=0x7fc934004f70 comp rx=0 tx=0).stop 2024-09-21T07:08:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 shutdown_connections 2024-09-21T07:08:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x563962317520 0x563962319970 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc91f400640 0 client.28921 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563961ee91b0 0x56396229ebb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563961ec02f0 0x56396229e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 --2- 172.21.15.114:0/2724409439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563961e7d1c0 0x5639622a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 >> 172.21.15.114:0/2724409439 conn(0x563961edd300 msgr2=0x56396228a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 shutdown_connections 2024-09-21T07:08:08.126 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.123+0000 7fc94b640740 1 -- 172.21.15.114:0/2724409439 wait complete. 2024-09-21T07:08:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.124+0000 7fc94b640740 20 client.28921 trim_cache size 0 max 16384 2024-09-21T07:08:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:08:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:08:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:08:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:08:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.125+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc9240b6730 msgr2=0x7fc9240b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.125+0000 7fc942e00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc9240b6730 0x7fc9240b8c40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc92c005f70 tx=0x7fc92c031400 comp rx=0 tx=0).stop 2024-09-21T07:08:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.125+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c076100 msgr2=0x7fc93c1b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.125+0000 7fc942e00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c076100 0x7fc93c1b3310 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0777d0 tx=0x7fc93000cdd0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 shutdown_connections 2024-09-21T07:08:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc93c10fe60 0x7fc93c1ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c076100 0x7fc93c1b3310 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc9240b6730 0x7fc9240b8c40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 --2- 172.21.15.114:0/1957883154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc93c0757b0 0x7fc93c1b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 >> 172.21.15.114:0/1957883154 conn(0x7fc93c0ff330 msgr2=0x7fc93c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.126+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 shutdown_connections 2024-09-21T07:08:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:08.127+0000 7fc942e00640 1 -- 172.21.15.114:0/1957883154 wait complete. 2024-09-21T07:08:09.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:09 smithi193 ceph-mon[34601]: pgmap v828: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 3.7 MiB/s wr, 542 op/s 2024-09-21T07:08:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:09 smithi114 ceph-mon[30654]: pgmap v828: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 3.7 MiB/s wr, 542 op/s 2024-09-21T07:08:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:09 smithi183 ceph-mon[34219]: pgmap v828: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 3.7 MiB/s wr, 542 op/s 2024-09-21T07:08:11.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:11 smithi114 ceph-mon[30654]: pgmap v829: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 3.9 MiB/s wr, 517 op/s 2024-09-21T07:08:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:08:11.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:11 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:08:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:11 smithi183 ceph-mon[34219]: pgmap v829: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 3.9 MiB/s wr, 517 op/s 2024-09-21T07:08:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:08:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:11 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:08:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:11 smithi193 ceph-mon[34601]: pgmap v829: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 3.9 MiB/s wr, 517 op/s 2024-09-21T07:08:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:08:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:11 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:08:12.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:12 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:08:12.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:12 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:08:12.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:12 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:08:12.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:12 smithi114 ceph-mon[30654]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T07:08:12.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:12 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:12 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:08:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:12 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:08:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:12 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:08:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:12 smithi183 ceph-mon[34219]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T07:08:12.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:12 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:12 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:08:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:12 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:08:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:12 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:08:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:12 smithi193 ceph-mon[34601]: osdmap e102: 12 total, 12 up, 12 in 2024-09-21T07:08:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:12 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:12.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 815.0 s, 364.4 tps, lat 284.970 ms stddev 338.049 2024-09-21T07:08:13.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:13 smithi193 ceph-mon[34601]: pgmap v831: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.8 MiB/s wr, 517 op/s 2024-09-21T07:08:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:13 smithi114 ceph-mon[30654]: pgmap v831: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.8 MiB/s wr, 517 op/s 2024-09-21T07:08:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:13 smithi183 ceph-mon[34219]: pgmap v831: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 3.8 MiB/s wr, 517 op/s 2024-09-21T07:08:15.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:15 smithi114 ceph-mon[30654]: pgmap v832: 129 pgs: 3 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-09-21T07:08:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:15 smithi183 ceph-mon[34219]: pgmap v832: 129 pgs: 3 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-09-21T07:08:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:15 smithi193 ceph-mon[34601]: pgmap v832: 129 pgs: 3 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-09-21T07:08:17.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:17 smithi114 ceph-mon[30654]: pgmap v833: 129 pgs: 3 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 4.3 MiB/s wr, 458 op/s 2024-09-21T07:08:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:17 smithi183 ceph-mon[34219]: pgmap v833: 129 pgs: 3 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 4.3 MiB/s wr, 458 op/s 2024-09-21T07:08:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:17 smithi193 ceph-mon[34601]: pgmap v833: 129 pgs: 3 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 4.3 MiB/s wr, 458 op/s 2024-09-21T07:08:17.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 820.0 s, 410.2 tps, lat 232.054 ms stddev 242.386 2024-09-21T07:08:19.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:19 smithi114 ceph-mon[30654]: pgmap v834: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-09-21T07:08:19.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:19 smithi183 ceph-mon[34219]: pgmap v834: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-09-21T07:08:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:19 smithi193 ceph-mon[34601]: pgmap v834: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.1 MiB/s wr, 457 op/s 2024-09-21T07:08:21.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:20 smithi114 ceph-mon[30654]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T07:08:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:20 smithi114 ceph-mon[30654]: pgmap v836: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 3.8 MiB/s wr, 449 op/s 2024-09-21T07:08:21.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:20 smithi183 ceph-mon[34219]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T07:08:21.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:20 smithi183 ceph-mon[34219]: pgmap v836: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 3.8 MiB/s wr, 449 op/s 2024-09-21T07:08:21.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:20 smithi193 ceph-mon[34601]: osdmap e103: 12 total, 12 up, 12 in 2024-09-21T07:08:21.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:20 smithi193 ceph-mon[34601]: pgmap v836: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 3.8 MiB/s wr, 449 op/s 2024-09-21T07:08:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 -- 172.21.15.114:0/2328591628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010b440 msgr2=0x7f171010b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 --2- 172.21.15.114:0/2328591628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010b440 0x7f171010b850 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009900 tx=0x7f16fc02a410 comp rx=0 tx=0).stop 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 -- 172.21.15.114:0/2328591628 shutdown_connections 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 --2- 172.21.15.114:0/2328591628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f171010bd90 0x7f17101125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 --2- 172.21.15.114:0/2328591628 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010b440 0x7f171010b850 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 --2- 172.21.15.114:0/2328591628 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 0x7f1710105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 -- 172.21.15.114:0/2328591628 >> 172.21.15.114:0/2328591628 conn(0x7f17101014f0 msgr2=0x7f1710103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 -- 172.21.15.114:0/2328591628 shutdown_connections 2024-09-21T07:08:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.443+0000 7f1715000640 1 -- 172.21.15.114:0/2328591628 wait complete. 2024-09-21T07:08:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.444+0000 7f1715000640 1 Processor -- start 2024-09-21T07:08:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 -- start start 2024-09-21T07:08:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 0x7f171007b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f171010b440 0x7f1710077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 0x7f1710078050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1710115aa0 con 0x7f1710105b10 2024-09-21T07:08:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1710115920 con 0x7f171010b440 2024-09-21T07:08:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.445+0000 7f1715000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1710115c20 con 0x7f171010bd90 2024-09-21T07:08:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 0x7f1710078050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 0x7f1710078050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50732/0 (socket says 172.21.15.114:50732) 2024-09-21T07:08:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 -- 172.21.15.114:0/672539634 learned_addr learned my addr 172.21.15.114:0/672539634 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170ea00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f171010b440 0x7f1710077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170f400640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 0x7f171007b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 -- 172.21.15.114:0/672539634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f171010b440 msgr2=0x7f1710077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f171010b440 0x7f1710077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 -- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 msgr2=0x7f171007b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 0x7f171007b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170fe00640 1 -- 172.21.15.114:0/672539634 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17101aca00 con 0x7f171010bd90 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.446+0000 7f170f400640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 0x7f171007b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.447+0000 7f170fe00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 0x7f1710078050 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f170800efd0 tx=0x7f170800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.447+0000 7f1707e00640 1 -- 172.21.15.114:0/672539634 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708003b70 con 0x7f171010bd90 2024-09-21T07:08:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.447+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17101acc30 con 0x7f171010bd90 2024-09-21T07:08:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.447+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101ad1a0 con 0x7f171010bd90 2024-09-21T07:08:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.447+0000 7f1707e00640 1 -- 172.21.15.114:0/672539634 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1708003d10 con 0x7f171010bd90 2024-09-21T07:08:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.447+0000 7f1707e00640 1 -- 172.21.15.114:0/672539634 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17080108f0 con 0x7f171010bd90 2024-09-21T07:08:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.452+0000 7f1707e00640 1 -- 172.21.15.114:0/672539634 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1708004b40 con 0x7f171010bd90 2024-09-21T07:08:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.452+0000 7f171d6b7740 1 Processor -- start 2024-09-21T07:08:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.455+0000 7f1707e00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b67b0 0x7f16f40b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.456+0000 7f1707e00640 1 -- 172.21.15.114:0/672539634 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f17080e7da0 con 0x7f171010bd90 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.456+0000 7f171d6b7740 1 -- start start 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.456+0000 7f170ea00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b67b0 0x7f16f40b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.456+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed2ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.457+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.457+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.457+0000 7f171d6b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56138eca06b0 con 0x56138e9861b0 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.457+0000 7f171d6b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56138ec8fae0 con 0x56138e9654d0 2024-09-21T07:08:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.457+0000 7f171d6b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56138ec938f0 con 0x56138e95d2f0 2024-09-21T07:08:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.458+0000 7f170fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.458+0000 7f170fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed2b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:56698/0 (socket says 172.21.15.114:56698) 2024-09-21T07:08:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.458+0000 7f170fe00640 1 -- 172.21.15.114:0/3836855845 learned_addr learned my addr 172.21.15.114:0/3836855845 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.458+0000 7f170f400640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed2ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.458+0000 7f170fe00640 1 -- 172.21.15.114:0/3836855845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 msgr2=0x56138ed2b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170ea00640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170fe00640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170fe00640 1 -- 172.21.15.114:0/3836855845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 msgr2=0x56138ed2ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170fe00640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed2ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170fe00640 1 -- 172.21.15.114:0/3836855845 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138ec5c780 con 0x56138e9654d0 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170f400640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed2ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.459+0000 7f170ea00640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.460+0000 7f170fe00640 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed2b890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f17080ad4c0 tx=0x7f17080a9f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef3763a67e3812e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.460+0000 7f170ea00640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b67b0 0x7f16f40b8cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16fc02aa00 tx=0x7f16fc000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.460+0000 7f1704c00640 1 -- 172.21.15.114:0/3836855845 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1708004e30 con 0x56138e9654d0 2024-09-21T07:08:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.460+0000 7f1704c00640 1 -- 172.21.15.114:0/3836855845 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17080ad2d0 con 0x56138e9654d0 2024-09-21T07:08:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.460+0000 7f1704c00640 1 -- 172.21.15.114:0/3836855845 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170809a570 con 0x56138e9654d0 2024-09-21T07:08:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.461+0000 7f171d6b7740 1 -- 172.21.15.114:0/3836855845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 msgr2=0x56138ed2b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.461+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed2b890 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f17080ad4c0 tx=0x7f17080a9f40 comp rx=0 tx=0).stop 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.461+0000 7f171d6b7740 1 -- 172.21.15.114:0/3836855845 shutdown_connections 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.462+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed2b890 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.462+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.462+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3836855845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed2ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.462+0000 7f171d6b7740 1 -- 172.21.15.114:0/3836855845 >> 172.21.15.114:0/3836855845 conn(0x56138e97a300 msgr2=0x56138ed28d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.462+0000 7f171d6b7740 1 -- 172.21.15.114:0/3836855845 shutdown_connections 2024-09-21T07:08:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.462+0000 7f171d6b7740 1 -- 172.21.15.114:0/3836855845 wait complete. 2024-09-21T07:08:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 -- start start 2024-09-21T07:08:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 Processor -- start 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e95d2f0 0x56138ed364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 0x56138ed36a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x56138ebc7ab0 con 0x56138e9861b0 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56138ec36570 con 0x56138e95d2f0 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.463+0000 7f171d6b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56138ed39590 con 0x56138e9654d0 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 0x56138ed36a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f171d6b7740 10 client.0 _ll_register_callbacks cb 0x56138ec45100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 0x56138ed36a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50758/0 (socket says 172.21.15.114:50758) 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 -- 172.21.15.114:0/1065532638 learned_addr learned my addr 172.21.15.114:0/1065532638 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170f400640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e95d2f0 0x56138ed364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170fe00640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 -- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e95d2f0 msgr2=0x56138ed364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e95d2f0 0x56138ed364c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 -- 172.21.15.114:0/1065532638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 msgr2=0x56138ed36f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed36f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.464+0000 7f170ea00640 1 -- 172.21.15.114:0/1065532638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138e9f1790 con 0x56138e9654d0 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f170ea00640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 0x56138ed36a00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f16fc006010 tx=0x7f16fc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=731eb645fe284090 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f16f9600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f16f9600640 1 -- 172.21.15.114:0/1065532638 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc035040 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17080ad2d0 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f16f9600640 1 -- 172.21.15.114:0/1065532638 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f16fc004940 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f16f9600640 1 -- 172.21.15.114:0/1065532638 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f16fc059050 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f171d6b7740 10 client.28930 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56138ea6a620 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.465+0000 7f16f9600640 1 -- 172.21.15.114:0/1065532638 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17080ad2d0 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f16f9600640 1 -- 172.21.15.114:0/1065532638 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7f16fc0324b0 con 0x56138e9654d0 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f171d6b7740 10 client.28930 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f171d6b7740 10 client.28930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f171d6b7740 20 client.28930 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f171d6b7740 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e944d80 0x56138ed59530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f171d6b7740 4 client.28930 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.466+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56138eca2600 con 0x56138e944d80 2024-09-21T07:08:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.467+0000 7f170f400640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e944d80 0x56138ed59530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.467+0000 7f170f400640 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e944d80 0x56138ed59530 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b80478bd7b212cd server_cookie=b21583b6fca17bc2 in_seq=0 out_seq=0 2024-09-21T07:08:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.467+0000 7f16f9600640 10 client.28930 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.468+0000 7f16f9600640 1 -- 172.21.15.114:0/1065532638 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56138eca2600 con 0x56138e944d80 2024-09-21T07:08:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.468+0000 7f16f9600640 10 client.28930 handle_command_reply: tid=0 2024-09-21T07:08:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.468+0000 7f171d6b7740 1 client.28930 shutdown 2024-09-21T07:08:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 msgr2=0x56138ed36a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 0x56138ed36a00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f16fc006010 tx=0x7f16fc009260 comp rx=0 tx=0).stop 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 shutdown_connections 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed36f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e944d80 0x56138ed59530 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e9654d0 0x56138ed36a00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 --2- 172.21.15.114:0/1065532638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e95d2f0 0x56138ed364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 >> 172.21.15.114:0/1065532638 conn(0x56138e97a300 msgr2=0x56138ed2be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f16f9600640 0 client.28930 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.469+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 shutdown_connections 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.470+0000 7f171d6b7740 1 -- 172.21.15.114:0/1065532638 wait complete. 2024-09-21T07:08:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.470+0000 7f171d6b7740 20 client.28930 trim_cache size 0 max 16384 2024-09-21T07:08:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.491+0000 7f171d6b7740 1 Processor -- start 2024-09-21T07:08:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.491+0000 7f171d6b7740 1 -- start start 2024-09-21T07:08:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9654d0 0x56138ed33690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9861b0 0x56138ed33bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f171d6b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16f000ffe0 con 0x56138e9654d0 2024-09-21T07:08:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f171d6b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0011be0 con 0x56138e9861b0 2024-09-21T07:08:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f171d6b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f00106b0 con 0x56138e95d2f0 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f170f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f170f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50760/0 (socket says 172.21.15.114:50760) 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.492+0000 7f170f400640 1 -- 172.21.15.114:0/3723957980 learned_addr learned my addr 172.21.15.114:0/3723957980 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f170f400640 1 -- 172.21.15.114:0/3723957980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9861b0 msgr2=0x56138ed33bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f170f400640 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9861b0 0x56138ed33bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f170f400640 1 -- 172.21.15.114:0/3723957980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9654d0 msgr2=0x56138ed33690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f170f400640 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9654d0 0x56138ed33690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f170f400640 1 -- 172.21.15.114:0/3723957980 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e944fb0 con 0x56138e95d2f0 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f170f400640 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2c550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1700009b40 tx=0x7f1700011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53509b1b54ad4482 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f16fbe00640 1 -- 172.21.15.114:0/3723957980 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700004250 con 0x56138e95d2f0 2024-09-21T07:08:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f16fbe00640 1 -- 172.21.15.114:0/3723957980 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17000043f0 con 0x56138e95d2f0 2024-09-21T07:08:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.493+0000 7f16fbe00640 1 -- 172.21.15.114:0/3723957980 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170001c900 con 0x56138e95d2f0 2024-09-21T07:08:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 -- 172.21.15.114:0/3723957980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 msgr2=0x56138ed2c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2c550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1700009b40 tx=0x7f1700011810 comp rx=0 tx=0).stop 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 -- 172.21.15.114:0/3723957980 shutdown_connections 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9861b0 0x56138ed33bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9654d0 0x56138ed33690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 --2- 172.21.15.114:0/3723957980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed2c550 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.494+0000 7f171d6b7740 1 -- 172.21.15.114:0/3723957980 >> 172.21.15.114:0/3723957980 conn(0x56138e97a300 msgr2=0x56138ed2be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.495+0000 7f171d6b7740 1 -- 172.21.15.114:0/3723957980 shutdown_connections 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.495+0000 7f171d6b7740 1 -- 172.21.15.114:0/3723957980 wait complete. 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.495+0000 7f171d6b7740 1 -- start start 2024-09-21T07:08:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.495+0000 7f171d6b7740 1 Processor -- start 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed3ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed3c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f16f00114e0 con 0x56138e9861b0 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f16f0010d80 con 0x56138e9654d0 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f16f000dfe0 con 0x56138e95d2f0 2024-09-21T07:08:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f171d6b7740 10 client.0 _ll_register_callbacks cb 0x7f16fc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed3ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed3ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:50764/0 (socket says 172.21.15.114:50764) 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 -- 172.21.15.114:0/4261864420 learned_addr learned my addr 172.21.15.114:0/4261864420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170ea00640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed3c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170fe00640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 -- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 msgr2=0x56138ed3c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed3c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 -- 172.21.15.114:0/4261864420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 msgr2=0x56138ed3c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed3c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.496+0000 7f170f400640 1 -- 172.21.15.114:0/4261864420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138ed3ce70 con 0x56138e95d2f0 2024-09-21T07:08:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.497+0000 7f170f400640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed3ee10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1700014000 tx=0x7f170001c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6aed81ac72417fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.497+0000 7f16f9600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.497+0000 7f16f9600640 1 -- 172.21.15.114:0/4261864420 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1700005030 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.497+0000 7f16f9600640 1 -- 172.21.15.114:0/4261864420 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f17000051d0 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.497+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17000043f0 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f16f9600640 1 -- 172.21.15.114:0/4261864420 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f17000053d0 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 10 client.28936 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x56138edc3c20 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f16f9600640 1 -- 172.21.15.114:0/4261864420 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17000043f0 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f16f9600640 1 -- 172.21.15.114:0/4261864420 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7f170001c9b0 con 0x56138e95d2f0 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 10 client.28936 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 10 client.28936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 20 client.28936 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e91a1c0 0x56138edc62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 4 client.28936 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:08:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.498+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56138ec9e0b0 con 0x56138e91a1c0 2024-09-21T07:08:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.499+0000 7f170ea00640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e91a1c0 0x56138edc62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.500+0000 7f16f9600640 10 client.28936 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.500+0000 7f170ea00640 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e91a1c0 0x56138edc62e0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642007b5d535c9e9 server_cookie=f9fafb85b146194a in_seq=0 out_seq=0 2024-09-21T07:08:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.517+0000 7f16f9600640 1 -- 172.21.15.114:0/4261864420 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56138ec9e0b0 con 0x56138e91a1c0 2024-09-21T07:08:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.517+0000 7f16f9600640 10 client.28936 handle_command_reply: tid=0 2024-09-21T07:08:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.517+0000 7f171d6b7740 1 client.28936 shutdown 2024-09-21T07:08:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.517+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 msgr2=0x56138ed3ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.519 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.517+0000 7f171d6b7740 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed3ee10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1700014000 tx=0x7f170001c460 comp rx=0 tx=0).stop 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 shutdown_connections 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x56138e9861b0 0x56138ed3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56138e9654d0 0x56138ed3c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x56138e91a1c0 0x56138edc62e0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 --2- 172.21.15.114:0/4261864420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56138e95d2f0 0x56138ed3ee10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 >> 172.21.15.114:0/4261864420 conn(0x56138e97a300 msgr2=0x56138ed2be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f16f9600640 0 client.28936 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 shutdown_connections 2024-09-21T07:08:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 1 -- 172.21.15.114:0/4261864420 wait complete. 2024-09-21T07:08:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.518+0000 7f171d6b7740 20 client.28936 trim_cache size 0 max 16384 2024-09-21T07:08:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:08:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:08:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:08:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b67b0 msgr2=0x7f16f40b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b67b0 0x7f16f40b8cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16fc02aa00 tx=0x7f16fc000bc0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 msgr2=0x7f1710078050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 0x7f1710078050 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f170800efd0 tx=0x7f170800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 shutdown_connections 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f171010bd90 0x7f1710078050 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f16f40b67b0 0x7f16f40b8cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f171010b440 0x7f1710077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 --2- 172.21.15.114:0/672539634 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1710105b10 0x7f171007b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 >> 172.21.15.114:0/672539634 conn(0x7f17101014f0 msgr2=0x7f1710107b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.519+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 shutdown_connections 2024-09-21T07:08:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:21.520+0000 7f1715000640 1 -- 172.21.15.114:0/672539634 wait complete. 2024-09-21T07:08:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 825.0 s, 403.0 tps, lat 255.061 ms stddev 310.004 2024-09-21T07:08:23.372 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:23 smithi114 ceph-mon[30654]: pgmap v837: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-21T07:08:23.411 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:23 smithi193 ceph-mon[34601]: pgmap v837: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-21T07:08:23.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:23 smithi183 ceph-mon[34219]: pgmap v837: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-21T07:08:25.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:25 smithi114 ceph-mon[30654]: pgmap v838: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-21T07:08:25.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:25 smithi193 ceph-mon[34601]: pgmap v838: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-21T07:08:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:25 smithi183 ceph-mon[34219]: pgmap v838: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-21T07:08:27.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:27 smithi193 ceph-mon[34601]: pgmap v839: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 4.0 MiB/s wr, 464 op/s 2024-09-21T07:08:27.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:27 smithi114 ceph-mon[30654]: pgmap v839: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 4.0 MiB/s wr, 464 op/s 2024-09-21T07:08:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:27 smithi183 ceph-mon[34219]: pgmap v839: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 4.0 MiB/s wr, 464 op/s 2024-09-21T07:08:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 830.0 s, 432.2 tps, lat 228.660 ms stddev 285.995 2024-09-21T07:08:29.141 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-21T07:08:29.141 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-21T07:08:29.141 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:08:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:29 smithi193 ceph-mon[34601]: pgmap v840: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-09-21T07:08:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:29 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:29.569 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:29 smithi114 ceph-mon[30654]: pgmap v840: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-09-21T07:08:29.570 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:29 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:29.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:29 smithi183 ceph-mon[34219]: pgmap v840: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-09-21T07:08:29.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:29 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:08:29.785 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:30.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:30.244 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:30.245 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:30 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:08:30.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:30.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:30.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:30 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:08:30.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:08:30.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:08:30.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:30 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:08:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.040+0000 7f686ee00640 1 -- 172.21.15.114:0/4290775333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810fcf0 msgr2=0x7f6868110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:31.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.040+0000 7f686ee00640 1 --2- 172.21.15.114:0/4290775333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810fcf0 0x7f6868110100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c02a440 comp rx=0 tx=0).stop 2024-09-21T07:08:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 -- 172.21.15.114:0/4290775333 shutdown_connections 2024-09-21T07:08:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 --2- 172.21.15.114:0/4290775333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6868110640 0x7f6868116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 --2- 172.21.15.114:0/4290775333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810fcf0 0x7f6868110100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 --2- 172.21.15.114:0/4290775333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f686810a650 0x7f686810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.043 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 -- 172.21.15.114:0/4290775333 >> 172.21.15.114:0/4290775333 conn(0x7f686806ee50 msgr2=0x7f686806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:31.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 -- 172.21.15.114:0/4290775333 shutdown_connections 2024-09-21T07:08:31.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.041+0000 7f686ee00640 1 -- 172.21.15.114:0/4290775333 wait complete. 2024-09-21T07:08:31.045 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.043+0000 7f686ee00640 1 Processor -- start 2024-09-21T07:08:31.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.043+0000 7f686ee00640 1 -- start start 2024-09-21T07:08:31.046 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686ee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 0x7f68681a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686ee00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f686810fcf0 0x7f68681aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f686810fcf0 0x7f68681aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686ee00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6868110640 0x7f68681b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 0x7f68681a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 0x7f68681a9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:34558/0 (socket says 172.21.15.114:34558) 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.044+0000 7f686d000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f686810fcf0 0x7f68681aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:49824/0 (socket says 172.21.15.114:49824) 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.045+0000 7f686ee00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f686811c130 con 0x7f6868110640 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.045+0000 7f686ee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f686811bfb0 con 0x7f686810a650 2024-09-21T07:08:31.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.045+0000 7f686ee00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f686811c2b0 con 0x7f686810fcf0 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.045+0000 7f686da00640 1 -- 172.21.15.114:0/3631717027 learned_addr learned my addr 172.21.15.114:0/3631717027 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f686da00640 1 -- 172.21.15.114:0/3631717027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f686810fcf0 msgr2=0x7f68681aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f686da00640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f686810fcf0 0x7f68681aa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f686da00640 1 -- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6868110640 msgr2=0x7f68681b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f686da00640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6868110640 0x7f68681b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f686da00640 1 -- 172.21.15.114:0/3631717027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68681b5370 con 0x7f686810a650 2024-09-21T07:08:31.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f686da00640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 0x7f68681a9db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f686000eab0 tx=0x7f686000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:31.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.046+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686000ce40 con 0x7f686810a650 2024-09-21T07:08:31.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.047+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6860011920 con 0x7f686810a650 2024-09-21T07:08:31.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.047+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860010a30 con 0x7f686810a650 2024-09-21T07:08:31.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.047+0000 7f686ee00640 1 -- 172.21.15.114:0/3631717027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68681b5540 con 0x7f686810a650 2024-09-21T07:08:31.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.047+0000 7f686ee00640 1 -- 172.21.15.114:0/3631717027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68681b59a0 con 0x7f686810a650 2024-09-21T07:08:31.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.050+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686810aa60 con 0x7f686810a650 2024-09-21T07:08:31.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.053+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6860010bd0 con 0x7f686810a650 2024-09-21T07:08:31.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.055+0000 7f6866000640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68500b6700 0x7f68500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:31.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.056+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f68600e80b0 con 0x7f686810a650 2024-09-21T07:08:31.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.056+0000 7f686d000640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68500b6700 0x7f68500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:31.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.058+0000 7f686d000640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68500b6700 0x7f68500b8c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f685c02a960 tx=0x7f685c000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:31.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.059+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6860016030 con 0x7f686810a650 2024-09-21T07:08:31.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:31 smithi114 ceph-mon[30654]: pgmap v841: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.8 MiB/s wr, 453 op/s 2024-09-21T07:08:31.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:31 smithi193 ceph-mon[34601]: pgmap v841: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.8 MiB/s wr, 453 op/s 2024-09-21T07:08:31.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.461+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6868116850 con 0x7f686810a650 2024-09-21T07:08:31.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.464+0000 7f6866000640 1 -- 172.21.15.114:0/3631717027 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+11995 (secure 0 0 0) 0x7f68600ae230 con 0x7f686810a650 2024-09-21T07:08:31.468 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:08:31.469 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":56,"btime":"2024-09-21T07:08:11:184042+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":56,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:08:10.627823+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:08:31.469 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 56 2024-09-21T07:08:31.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.472+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68500b6700 msgr2=0x7f68500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:31.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.472+0000 7f6857400640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68500b6700 0x7f68500b8c10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f685c02a960 tx=0x7f685c000bc0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.472+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 msgr2=0x7f68681a9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:31.474 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.472+0000 7f6857400640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 0x7f68681a9db0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f686000eab0 tx=0x7f686000ef70 comp rx=0 tx=0).stop 2024-09-21T07:08:31.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.475+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 shutdown_connections 2024-09-21T07:08:31.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.475+0000 7f6857400640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6868110640 0x7f68681b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.475+0000 7f6857400640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f686810fcf0 0x7f68681aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.475+0000 7f6857400640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f68500b6700 0x7f68500b8c10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.475+0000 7f6857400640 1 --2- 172.21.15.114:0/3631717027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686810a650 0x7f68681a9db0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:31.477 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.475+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 >> 172.21.15.114:0/3631717027 conn(0x7f686806ee50 msgr2=0x7f686810bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:31.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.476+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 shutdown_connections 2024-09-21T07:08:31.478 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:31.476+0000 7f6857400640 1 -- 172.21.15.114:0/3631717027 wait complete. 2024-09-21T07:08:31.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:31 smithi183 ceph-mon[34219]: pgmap v841: 129 pgs: 129 active+clean; 3.0 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 3.8 MiB/s wr, 453 op/s 2024-09-21T07:08:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:32 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3631717027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:32.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:32 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3631717027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:32.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:32 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3631717027' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 835.0 s, 345.6 tps, lat 276.300 ms stddev 330.208 2024-09-21T07:08:33.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:33 smithi114 ceph-mon[30654]: pgmap v842: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-09-21T07:08:33.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:33 smithi183 ceph-mon[34219]: pgmap v842: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-09-21T07:08:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:33 smithi193 ceph-mon[34601]: pgmap v842: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 3.9 MiB/s wr, 428 op/s 2024-09-21T07:08:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.848+0000 7f2bf3e00640 1 -- 172.21.15.114:0/2725456716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec096ed0 msgr2=0x7f2bec0972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.848+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/2725456716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec096ed0 0x7f2bec0972e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009860 tx=0x7f2be002a360 comp rx=0 tx=0).stop 2024-09-21T07:08:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 -- 172.21.15.114:0/2725456716 shutdown_connections 2024-09-21T07:08:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/2725456716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bec0a0480 0x7f2bec0a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/2725456716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bec097900 0x7f2bec09ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/2725456716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec096ed0 0x7f2bec0972e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 -- 172.21.15.114:0/2725456716 >> 172.21.15.114:0/2725456716 conn(0x7f2bec00cd90 msgr2=0x7f2bec00d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 -- 172.21.15.114:0/2725456716 shutdown_connections 2024-09-21T07:08:34.853 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.851+0000 7f2bf3e00640 1 -- 172.21.15.114:0/2725456716 wait complete. 2024-09-21T07:08:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.852+0000 7f2bf3e00640 1 Processor -- start 2024-09-21T07:08:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 -- start start 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bec096ed0 0x7f2bec12ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bec097900 0x7f2bec12f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 0x7f2bec134530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec0a60a0 con 0x7f2bec097900 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec0a5f20 con 0x7f2bec0a0480 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.853+0000 7f2bf3e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec0a6220 con 0x7f2bec096ed0 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 0x7f2bec134530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 0x7f2bec134530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46804/0 (socket says 172.21.15.114:46804) 2024-09-21T07:08:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 -- 172.21.15.114:0/1525011094 learned_addr learned my addr 172.21.15.114:0/1525011094 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bec096ed0 0x7f2bec12ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 -- 172.21.15.114:0/1525011094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bec096ed0 msgr2=0x7f2bec12ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bec096ed0 0x7f2bec12ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 -- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bec097900 msgr2=0x7f2bec12f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bec097900 0x7f2bec12f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.854+0000 7f2bf3400640 1 -- 172.21.15.114:0/1525011094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bec134c30 con 0x7f2bec0a0480 2024-09-21T07:08:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.855+0000 7f2bf3400640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 0x7f2bec134530 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4052910 tx=0x7f2bf407e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.855+0000 7f2beb400640 1 -- 172.21.15.114:0/1525011094 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4084020 con 0x7f2bec0a0480 2024-09-21T07:08:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.856+0000 7f2beb400640 1 -- 172.21.15.114:0/1525011094 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf4080af0 con 0x7f2bec0a0480 2024-09-21T07:08:34.857 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.856+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bec134f20 con 0x7f2bec0a0480 2024-09-21T07:08:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.856+0000 7f2beb400640 1 -- 172.21.15.114:0/1525011094 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4084020 con 0x7f2bec0a0480 2024-09-21T07:08:34.858 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.856+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bec13b6b0 con 0x7f2bec0a0480 2024-09-21T07:08:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.863+0000 7f2c006d3740 1 Processor -- start 2024-09-21T07:08:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.865+0000 7f2beb400640 1 -- 172.21.15.114:0/1525011094 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f2bf4087020 con 0x7f2bec0a0480 2024-09-21T07:08:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.869+0000 7f2c006d3740 1 -- start start 2024-09-21T07:08:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.870+0000 7f2beb400640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2bd80b6880 0x7f2bd80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.870+0000 7f2beb400640 1 -- 172.21.15.114:0/1525011094 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f2bf41553e0 con 0x7f2bec0a0480 2024-09-21T07:08:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2bf2000640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2bd80b6880 0x7f2bd80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd508e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd509350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd509890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2c006d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5606bd47e6b0 con 0x5606bd1434d0 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2c006d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5606bd46dae0 con 0x5606bd13b2f0 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.873+0000 7f2c006d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606bd4718f0 con 0x5606bd1641b0 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.874+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd508e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.874+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd508e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34304/0 (socket says 172.21.15.114:34304) 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.874+0000 7f2bf2a00640 1 -- 172.21.15.114:0/1328872825 learned_addr learned my addr 172.21.15.114:0/1328872825 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.874+0000 7f2bf2000640 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd509350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.875+0000 7f2bf2a00640 1 -- 172.21.15.114:0/1328872825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 msgr2=0x5606bd509350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.875+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd509350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.875+0000 7f2bf2a00640 1 -- 172.21.15.114:0/1328872825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 msgr2=0x5606bd509890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.875+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd509890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.875+0000 7f2bf2a00640 1 -- 172.21.15.114:0/1328872825 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bd43a780 con 0x5606bd1641b0 2024-09-21T07:08:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.876+0000 7f2bf2000640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2bd80b6880 0x7f2bd80b8d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2be40058c0 tx=0x7f2be4009f20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.877+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd508e10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2be002a880 tx=0x7f2be0005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25a11c88d22cc2da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.878+0000 7f2bdf400640 1 -- 172.21.15.114:0/1328872825 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0037040 con 0x5606bd1641b0 2024-09-21T07:08:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.878+0000 7f2bdf400640 1 -- 172.21.15.114:0/1328872825 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be0038040 con 0x5606bd1641b0 2024-09-21T07:08:34.880 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.879+0000 7f2bdf400640 1 -- 172.21.15.114:0/1328872825 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0037040 con 0x5606bd1641b0 2024-09-21T07:08:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.880+0000 7f2c006d3740 1 -- 172.21.15.114:0/1328872825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 msgr2=0x5606bd508e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.880+0000 7f2c006d3740 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd508e10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2be002a880 tx=0x7f2be0005c50 comp rx=0 tx=0).stop 2024-09-21T07:08:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.880+0000 7f2c006d3740 1 -- 172.21.15.114:0/1328872825 shutdown_connections 2024-09-21T07:08:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.880+0000 7f2c006d3740 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd509890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.880+0000 7f2c006d3740 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd509350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.880+0000 7f2c006d3740 1 --2- 172.21.15.114:0/1328872825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd508e10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.882 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.881+0000 7f2c006d3740 1 -- 172.21.15.114:0/1328872825 >> 172.21.15.114:0/1328872825 conn(0x5606bd158300 msgr2=0x5606bd506d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.881+0000 7f2c006d3740 1 -- 172.21.15.114:0/1328872825 shutdown_connections 2024-09-21T07:08:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.881+0000 7f2c006d3740 1 -- 172.21.15.114:0/1328872825 wait complete. 2024-09-21T07:08:34.885 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.882+0000 7f2c006d3740 1 -- start start 2024-09-21T07:08:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.884+0000 7f2c006d3740 1 Processor -- start 2024-09-21T07:08:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.886+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd51d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.886+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1434d0 0x5606bd51da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.887+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1641b0 0x5606bd522aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.887+0000 7f2c006d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5606bd3a5ab0 con 0x5606bd1641b0 2024-09-21T07:08:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.887+0000 7f2c006d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5606bd414570 con 0x5606bd13b2f0 2024-09-21T07:08:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.887+0000 7f2c006d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5606bd517580 con 0x5606bd1434d0 2024-09-21T07:08:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.887+0000 7f2c006d3740 10 client.0 _ll_register_callbacks cb 0x5606bd423100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.888+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd51d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.888+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd51d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46822/0 (socket says 172.21.15.114:46822) 2024-09-21T07:08:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.888+0000 7f2bf2a00640 1 -- 172.21.15.114:0/3867858033 learned_addr learned my addr 172.21.15.114:0/3867858033 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.888+0000 7f2bf2000640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1434d0 0x5606bd51da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.888+0000 7f2bf3400640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1641b0 0x5606bd522aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.889+0000 7f2bf2a00640 1 -- 172.21.15.114:0/3867858033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1434d0 msgr2=0x5606bd51da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.889+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1434d0 0x5606bd51da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.889+0000 7f2bf2a00640 1 -- 172.21.15.114:0/3867858033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1641b0 msgr2=0x5606bd522aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.889+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1641b0 0x5606bd522aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.889+0000 7f2bf2a00640 1 -- 172.21.15.114:0/3867858033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bd1cf790 con 0x5606bd13b2f0 2024-09-21T07:08:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.890+0000 7f2bd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.890+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd51d4b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2be0030490 tx=0x7f2be00307a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a1a485f7eb7f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.891+0000 7f2bd3e00640 1 -- 172.21.15.114:0/3867858033 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be004c030 con 0x5606bd13b2f0 2024-09-21T07:08:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.891+0000 7f2bd3e00640 1 -- 172.21.15.114:0/3867858033 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f2be0005250 con 0x5606bd13b2f0 2024-09-21T07:08:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.891+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be0038040 con 0x5606bd13b2f0 2024-09-21T07:08:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.892+0000 7f2bd3e00640 1 -- 172.21.15.114:0/3867858033 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f2be0037040 con 0x5606bd13b2f0 2024-09-21T07:08:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.892+0000 7f2c006d3740 10 client.28991 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.892+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5606bd248620 con 0x5606bd13b2f0 2024-09-21T07:08:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.893+0000 7f2bd3e00640 1 -- 172.21.15.114:0/3867858033 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be0038040 con 0x5606bd13b2f0 2024-09-21T07:08:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.893+0000 7f2bd3e00640 1 -- 172.21.15.114:0/3867858033 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7f2be003e5a0 con 0x5606bd13b2f0 2024-09-21T07:08:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.893+0000 7f2c006d3740 10 client.28991 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.893+0000 7f2c006d3740 10 client.28991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.894+0000 7f2c006d3740 20 client.28991 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.894+0000 7f2c006d3740 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd122d80 0x5606bd52eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.894+0000 7f2c006d3740 4 client.28991 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.894+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606bd480600 con 0x5606bd122d80 2024-09-21T07:08:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.895+0000 7f2bf2000640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd122d80 0x5606bd52eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.896+0000 7f2bf2000640 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd122d80 0x5606bd52eb80 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484070965c3e9e11 server_cookie=645c664b4b0fa1de in_seq=0 out_seq=0 2024-09-21T07:08:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.896+0000 7f2bd3e00640 10 client.28991 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.897+0000 7f2bd3e00640 1 -- 172.21.15.114:0/3867858033 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606bd480600 con 0x5606bd122d80 2024-09-21T07:08:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.897+0000 7f2bd3e00640 10 client.28991 handle_command_reply: tid=0 2024-09-21T07:08:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.898+0000 7f2c006d3740 1 client.28991 shutdown 2024-09-21T07:08:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 msgr2=0x5606bd51d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd51d4b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2be0030490 tx=0x7f2be00307a0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 shutdown_connections 2024-09-21T07:08:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1641b0 0x5606bd522aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd122d80 0x5606bd52eb80 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1434d0 0x5606bd51da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 --2- 172.21.15.114:0/3867858033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd13b2f0 0x5606bd51d4b0 secure :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2be0030490 tx=0x7f2be00307a0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2bd3e00640 0 client.28991 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.899+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 >> 172.21.15.114:0/3867858033 conn(0x5606bd158300 msgr2=0x5606bd509e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.901+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 shutdown_connections 2024-09-21T07:08:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.901+0000 7f2c006d3740 1 -- 172.21.15.114:0/3867858033 wait complete. 2024-09-21T07:08:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.901+0000 7f2c006d3740 20 client.28991 trim_cache size 0 max 16384 2024-09-21T07:08:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.930+0000 7f2c006d3740 1 Processor -- start 2024-09-21T07:08:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.930+0000 7f2c006d3740 1 -- start start 2024-09-21T07:08:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd51d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd516910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd0f81c0 0x5606bd51a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2c006d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec141400 con 0x5606bd1434d0 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2c006d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec143000 con 0x5606bd0f81c0 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2c006d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec141ad0 con 0x5606bd1641b0 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd516910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.933 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd516910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34322/0 (socket says 172.21.15.114:34322) 2024-09-21T07:08:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2bf2000640 1 -- 172.21.15.114:0/4215073381 learned_addr learned my addr 172.21.15.114:0/4215073381 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.931+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd51d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.932+0000 7f2bf3400640 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd0f81c0 0x5606bd51a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.932+0000 7f2bf2a00640 1 -- 172.21.15.114:0/4215073381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 msgr2=0x5606bd516910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.932+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd516910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.932+0000 7f2bf2a00640 1 -- 172.21.15.114:0/4215073381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd0f81c0 msgr2=0x5606bd51a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.932+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd0f81c0 0x5606bd51a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.932+0000 7f2bf2a00640 1 -- 172.21.15.114:0/4215073381 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bd517470 con 0x5606bd1434d0 2024-09-21T07:08:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.933+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd51d200 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2be0030890 tx=0x7f2be002a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a544c2bbf59ebf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.933+0000 7f2bdea00640 1 -- 172.21.15.114:0/4215073381 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be004c070 con 0x5606bd1434d0 2024-09-21T07:08:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.933+0000 7f2bdea00640 1 -- 172.21.15.114:0/4215073381 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be003ec00 con 0x5606bd1434d0 2024-09-21T07:08:34.935 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.933+0000 7f2bdea00640 1 -- 172.21.15.114:0/4215073381 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004040 con 0x5606bd1434d0 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 -- 172.21.15.114:0/4215073381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 msgr2=0x5606bd51d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd51d200 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2be0030890 tx=0x7f2be002a360 comp rx=0 tx=0).stop 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 -- 172.21.15.114:0/4215073381 shutdown_connections 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd0f81c0 0x5606bd51a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd516910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 --2- 172.21.15.114:0/4215073381 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd1434d0 0x5606bd51d200 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.934+0000 7f2c006d3740 1 -- 172.21.15.114:0/4215073381 >> 172.21.15.114:0/4215073381 conn(0x5606bd158300 msgr2=0x5606bd51caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.935+0000 7f2c006d3740 1 -- 172.21.15.114:0/4215073381 shutdown_connections 2024-09-21T07:08:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.935+0000 7f2c006d3740 1 -- 172.21.15.114:0/4215073381 wait complete. 2024-09-21T07:08:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.936+0000 7f2c006d3740 1 -- start start 2024-09-21T07:08:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.936+0000 7f2c006d3740 1 Processor -- start 2024-09-21T07:08:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.937+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd0f81c0 0x5606bd5298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.939 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.938+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 0x5606bd526d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.938+0000 7f2c006d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd527400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.938+0000 7f2c006d3740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec142900 con 0x5606bd0f81c0 2024-09-21T07:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.938+0000 7f2c006d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec1421a0 con 0x5606bd1434d0 2024-09-21T07:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.938+0000 7f2c006d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2bec13f460 con 0x5606bd1641b0 2024-09-21T07:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.938+0000 7f2c006d3740 10 client.0 _ll_register_callbacks cb 0x7f2be4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.939+0000 7f2bf2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd0f81c0 0x5606bd5298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.939+0000 7f2bf3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd527400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.939+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 0x5606bd526d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.939+0000 7f2bf2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 0x5606bd526d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46844/0 (socket says 172.21.15.114:46844) 2024-09-21T07:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.939+0000 7f2bf2000640 1 -- 172.21.15.114:0/174741752 learned_addr learned my addr 172.21.15.114:0/174741752 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.940+0000 7f2bf3400640 1 -- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 msgr2=0x5606bd526d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.940+0000 7f2bf3400640 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 0x5606bd526d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.940+0000 7f2bf3400640 1 -- 172.21.15.114:0/174741752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd0f81c0 msgr2=0x5606bd5298a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.940+0000 7f2bf3400640 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd0f81c0 0x5606bd5298a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.940+0000 7f2bf3400640 1 -- 172.21.15.114:0/174741752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bd527b70 con 0x5606bd1641b0 2024-09-21T07:08:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.941+0000 7f2bf2000640 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 0x5606bd526d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.942+0000 7f2bd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.942+0000 7f2bf3400640 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd527400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4152760 tx=0x7f2bf4080ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=badfceb40b7aa8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.942+0000 7f2bd3e00640 1 -- 172.21.15.114:0/174741752 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf41245f0 con 0x5606bd1641b0 2024-09-21T07:08:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.942+0000 7f2bd3e00640 1 -- 172.21.15.114:0/174741752 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f2bf4142170 con 0x5606bd1641b0 2024-09-21T07:08:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.943+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be003ec00 con 0x5606bd1641b0 2024-09-21T07:08:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.944+0000 7f2bd3e00640 1 -- 172.21.15.114:0/174741752 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f2bf411a970 con 0x5606bd1641b0 2024-09-21T07:08:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.944+0000 7f2c006d3740 10 client.28960 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:34.945 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.944+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5606bd5993d0 con 0x5606bd1641b0 2024-09-21T07:08:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.944+0000 7f2bd3e00640 1 -- 172.21.15.114:0/174741752 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be003ec00 con 0x5606bd1641b0 2024-09-21T07:08:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2bd3e00640 1 -- 172.21.15.114:0/174741752 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7f2bf411a0c0 con 0x5606bd1641b0 2024-09-21T07:08:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2c006d3740 10 client.28960 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2c006d3740 10 client.28960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2c006d3740 20 client.28960 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2c006d3740 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd599b00 0x5606bd59bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2c006d3740 4 client.28960 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.945+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606bd47c0b0 con 0x5606bd599b00 2024-09-21T07:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.946+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd599b00 0x5606bd59bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.947+0000 7f2bf2a00640 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd599b00 0x5606bd59bf20 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10b36c3ea47ea0b server_cookie=644a9a09aa65125d in_seq=0 out_seq=0 2024-09-21T07:08:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:34.947+0000 7f2bd3e00640 10 client.28960 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:35.026 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-21T07:08:35.026 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.024+0000 7f2bd3e00640 1 -- 172.21.15.114:0/174741752 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606bd47c0b0 con 0x5606bd599b00 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.024+0000 7f2bd3e00640 10 client.28960 handle_command_reply: tid=0 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.024+0000 7f2c006d3740 1 client.28960 shutdown 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 msgr2=0x5606bd527400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd527400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4152760 tx=0x7f2bf4080ed0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 shutdown_connections 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5606bd599b00 0x5606bd59bf20 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5606bd1641b0 0x5606bd527400 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606bd1434d0 0x5606bd526d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 --2- 172.21.15.114:0/174741752 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5606bd0f81c0 0x5606bd5298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 >> 172.21.15.114:0/174741752 conn(0x5606bd158300 msgr2=0x5606bd509e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2bd3e00640 0 client.28960 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 shutdown_connections 2024-09-21T07:08:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 1 -- 172.21.15.114:0/174741752 wait complete. 2024-09-21T07:08:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.025+0000 7f2c006d3740 20 client.28960 trim_cache size 0 max 16384 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.026+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2bd80b6880 msgr2=0x7f2bd80b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.026+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2bd80b6880 0x7f2bd80b8d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2be40058c0 tx=0x7f2be4009f20 comp rx=0 tx=0).stop 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.026+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 msgr2=0x7f2bec134530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.026+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 0x7f2bec134530 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4052910 tx=0x7f2bf407e7e0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.027+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 shutdown_connections 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.027+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f2bd80b6880 0x7f2bd80b8d90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.027+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2bec0a0480 0x7f2bec134530 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.027+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f2bec097900 0x7f2bec12f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.027+0000 7f2bf3e00640 1 --2- 172.21.15.114:0/1525011094 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bec096ed0 0x7f2bec12ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.027+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 >> 172.21.15.114:0/1525011094 conn(0x7f2bec00cd90 msgr2=0x7f2bec094d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.028+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 shutdown_connections 2024-09-21T07:08:35.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:35.028+0000 7f2bf3e00640 1 -- 172.21.15.114:0/1525011094 wait complete. 2024-09-21T07:08:35.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:35 smithi114 ceph-mon[30654]: pgmap v843: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-09-21T07:08:35.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:35 smithi183 ceph-mon[34219]: pgmap v843: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-09-21T07:08:35.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:35 smithi193 ceph-mon[34601]: pgmap v843: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 4.3 MiB/s wr, 497 op/s 2024-09-21T07:08:35.925 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:37.524 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:08:37.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:37 smithi114 ceph-mon[30654]: pgmap v844: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.7 MiB/s wr, 415 op/s 2024-09-21T07:08:37.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:37 smithi183 ceph-mon[34219]: pgmap v844: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.7 MiB/s wr, 415 op/s 2024-09-21T07:08:37.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:37 smithi193 ceph-mon[34601]: pgmap v844: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 3.7 MiB/s wr, 415 op/s 2024-09-21T07:08:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 840.0 s, 359.4 tps, lat 294.613 ms stddev 393.464 2024-09-21T07:08:38.876 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:39.463 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:39 smithi114 ceph-mon[30654]: pgmap v845: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 457 op/s 2024-09-21T07:08:39.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:39 smithi183 ceph-mon[34219]: pgmap v845: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 457 op/s 2024-09-21T07:08:39.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:39 smithi193 ceph-mon[34601]: pgmap v845: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 457 op/s 2024-09-21T07:08:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 -- 172.21.15.114:0/1664999016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 msgr2=0x7f0988077d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 --2- 172.21.15.114:0/1664999016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988077d50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f097c01a910 tx=0x7f097c03b420 comp rx=0 tx=0).stop 2024-09-21T07:08:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 -- 172.21.15.114:0/1664999016 shutdown_connections 2024-09-21T07:08:39.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 --2- 172.21.15.114:0/1664999016 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 0x7f09881169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 --2- 172.21.15.114:0/1664999016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 0x7f0988114070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 --2- 172.21.15.114:0/1664999016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988077d50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 -- 172.21.15.114:0/1664999016 >> 172.21.15.114:0/1664999016 conn(0x7f0988101680 msgr2=0x7f0988103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 -- 172.21.15.114:0/1664999016 shutdown_connections 2024-09-21T07:08:39.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.828+0000 7f098e800640 1 -- 172.21.15.114:0/1664999016 wait complete. 2024-09-21T07:08:39.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.830+0000 7f098e800640 1 Processor -- start 2024-09-21T07:08:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.830+0000 7f098e800640 1 -- start start 2024-09-21T07:08:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988108870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098e800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 0x7f0988108db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098e800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 0x7f09881092f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098e800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09881080b0 con 0x7f09881145b0 2024-09-21T07:08:39.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098e800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0988107f30 con 0x7f0988077940 2024-09-21T07:08:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098e800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0988108230 con 0x7f0988078290 2024-09-21T07:08:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46868/0 (socket says 172.21.15.114:46868) 2024-09-21T07:08:39.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098d400640 1 -- 172.21.15.114:0/2146512572 learned_addr learned my addr 172.21.15.114:0/2146512572 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.831+0000 7f098ca00640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 0x7f0988108db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.832+0000 7f098d400640 1 -- 172.21.15.114:0/2146512572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 msgr2=0x7f0988108db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.833+0000 7f098de00640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 0x7f09881092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.833+0000 7f098d400640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 0x7f0988108db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.833+0000 7f098d400640 1 -- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 msgr2=0x7f09881092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.833+0000 7f098d400640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 0x7f09881092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.833+0000 7f098d400640 1 -- 172.21.15.114:0/2146512572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0988113af0 con 0x7f0988077940 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.834+0000 7f098ca00640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 0x7f0988108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.834+0000 7f098de00640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 0x7f09881092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:39.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.835+0000 7f098d400640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988108870 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f097c01aa30 tx=0x7f097c004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.835+0000 7f0986000640 1 -- 172.21.15.114:0/2146512572 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c04f020 con 0x7f0988077940 2024-09-21T07:08:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.836+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0988113d80 con 0x7f0988077940 2024-09-21T07:08:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.836+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09881142b0 con 0x7f0988077940 2024-09-21T07:08:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.836+0000 7f0986000640 1 -- 172.21.15.114:0/2146512572 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f097c004260 con 0x7f0988077940 2024-09-21T07:08:39.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.836+0000 7f0986000640 1 -- 172.21.15.114:0/2146512572 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c046070 con 0x7f0988077940 2024-09-21T07:08:39.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.842+0000 7f0986000640 1 -- 172.21.15.114:0/2146512572 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f097c052020 con 0x7f0988077940 2024-09-21T07:08:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.845+0000 7f0986000640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09700b66f0 0x7f09700b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.846+0000 7f0986000640 1 -- 172.21.15.114:0/2146512572 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f097c11a1a0 con 0x7f0988077940 2024-09-21T07:08:39.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.846+0000 7f098ca00640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09700b66f0 0x7f09700b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.856+0000 7f098ca00640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09700b66f0 0x7f09700b8c00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0988079a30 tx=0x7f0974005d30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 Processor -- start 2024-09-21T07:08:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 -- start start 2024-09-21T07:08:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e1abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5634c3d90a90 con 0x5634c3a52b70 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.858+0000 7f0996a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5634c3d7fec0 con 0x5634c3a4a990 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.859+0000 7f0996a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634c3d83cd0 con 0x5634c3a73850 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.859+0000 7f098d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e1abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.859+0000 7f098ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.859+0000 7f098ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e1b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46874/0 (socket says 172.21.15.114:46874) 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.859+0000 7f098ca00640 1 -- 172.21.15.114:0/3300525933 learned_addr learned my addr 172.21.15.114:0/3300525933 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.859+0000 7f098de00640 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098d400640 1 -- 172.21.15.114:0/3300525933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 msgr2=0x5634c3e1b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098d400640 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e1b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098d400640 1 -- 172.21.15.114:0/3300525933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 msgr2=0x5634c3e1b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098d400640 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e1b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098d400640 1 -- 172.21.15.114:0/3300525933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c3d259d0 con 0x5634c3a73850 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098ca00640 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:39.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.861+0000 7f098d400640 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e1abb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f097c117520 tx=0x7f097c01a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3ed5b51d87beab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.862+0000 7f097a600640 1 -- 172.21.15.114:0/3300525933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0db390 con 0x5634c3a73850 2024-09-21T07:08:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.862+0000 7f097a600640 1 -- 172.21.15.114:0/3300525933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f097c0df730 con 0x5634c3a73850 2024-09-21T07:08:39.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.862+0000 7f097a600640 1 -- 172.21.15.114:0/3300525933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0dbff0 con 0x5634c3a73850 2024-09-21T07:08:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.864+0000 7f0996a7a740 1 -- 172.21.15.114:0/3300525933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 msgr2=0x5634c3e1abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.864+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e1abb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f097c117520 tx=0x7f097c01a270 comp rx=0 tx=0).stop 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.865+0000 7f0996a7a740 1 -- 172.21.15.114:0/3300525933 shutdown_connections 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.865+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.865+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.865+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3300525933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e1abb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.865+0000 7f0996a7a740 1 -- 172.21.15.114:0/3300525933 >> 172.21.15.114:0/3300525933 conn(0x5634c3a67c40 msgr2=0x5634c3e18aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.866+0000 7f0996a7a740 1 -- 172.21.15.114:0/3300525933 shutdown_connections 2024-09-21T07:08:39.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.866+0000 7f0996a7a740 1 -- 172.21.15.114:0/3300525933 wait complete. 2024-09-21T07:08:39.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.868+0000 7f0996a7a740 1 -- start start 2024-09-21T07:08:39.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.868+0000 7f0996a7a740 1 Processor -- start 2024-09-21T07:08:39.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.869+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e26730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.869+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e26c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.869+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f098de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f098de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e323b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34372/0 (socket says 172.21.15.114:34372) 2024-09-21T07:08:39.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f0996a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5634c3be17b0 con 0x5634c3a52b70 2024-09-21T07:08:39.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f0996a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5634c3d24650 con 0x5634c3a4a990 2024-09-21T07:08:39.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f0996a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5634c3e29150 con 0x5634c3a73850 2024-09-21T07:08:39.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f0996a7a740 10 client.0 _ll_register_callbacks cb 0x5634c3a8b0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:39.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f098d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e26730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.870+0000 7f098de00640 1 -- 172.21.15.114:0/3711475375 learned_addr learned my addr 172.21.15.114:0/3711475375 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.871+0000 7f098ca00640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e26c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.871+0000 7f098d400640 1 -- 172.21.15.114:0/3711475375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 msgr2=0x5634c3e323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.871+0000 7f098d400640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.871+0000 7f098d400640 1 -- 172.21.15.114:0/3711475375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 msgr2=0x5634c3e26c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.871+0000 7f098d400640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e26c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.871+0000 7f098d400640 1 -- 172.21.15.114:0/3711475375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c3aedba0 con 0x5634c3a4a990 2024-09-21T07:08:39.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.872+0000 7f098de00640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:39.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.873+0000 7f098d400640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e26730 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f097c0df920 tx=0x7f097c0dc0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ae3b1ec49f4e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.874 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.873+0000 7f096ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:39.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.873+0000 7f096ea00640 1 -- 172.21.15.114:0/3711475375 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c107070 con 0x5634c3a4a990 2024-09-21T07:08:39.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.873+0000 7f096ea00640 1 -- 172.21.15.114:0/3711475375 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f097c0df080 con 0x5634c3a4a990 2024-09-21T07:08:39.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.874+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f097c0df730 con 0x5634c3a4a990 2024-09-21T07:08:39.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.874+0000 7f096ea00640 1 -- 172.21.15.114:0/3711475375 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f097c0db390 con 0x5634c3a4a990 2024-09-21T07:08:39.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.874+0000 7f0996a7a740 10 client.29009 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:39.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.874+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5634c3e35030 con 0x5634c3a4a990 2024-09-21T07:08:39.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.874+0000 7f096ea00640 1 -- 172.21.15.114:0/3711475375 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f097c0df730 con 0x5634c3a4a990 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f096ea00640 1 -- 172.21.15.114:0/3711475375 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7f097c0de8f0 con 0x5634c3a4a990 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f0996a7a740 10 client.29009 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f0996a7a740 10 client.29009 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f0996a7a740 20 client.29009 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a321d0 0x5634c3e38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f0996a7a740 4 client.29009 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.875+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c3d929e0 con 0x5634c3a321d0 2024-09-21T07:08:39.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.876+0000 7f098ca00640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a321d0 0x5634c3e38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.876+0000 7f098ca00640 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a321d0 0x5634c3e38390 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7325959b855fa4 server_cookie=8d1881c75380f671 in_seq=0 out_seq=0 2024-09-21T07:08:39.878 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.876+0000 7f096ea00640 10 client.29009 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:39.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.877+0000 7f096ea00640 1 -- 172.21.15.114:0/3711475375 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5634c3d929e0 con 0x5634c3a321d0 2024-09-21T07:08:39.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.877+0000 7f096ea00640 10 client.29009 handle_command_reply: tid=0 2024-09-21T07:08:39.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.877+0000 7f0996a7a740 1 client.29009 shutdown 2024-09-21T07:08:39.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 msgr2=0x5634c3e26730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.879 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e26730 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f097c0df920 tx=0x7f097c0dc0c0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 shutdown_connections 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a321d0 0x5634c3e38390 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e26c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 --2- 172.21.15.114:0/3711475375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e26730 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 >> 172.21.15.114:0/3711475375 conn(0x5634c3a67c40 msgr2=0x5634c3e1dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.880 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.878+0000 7f096ea00640 0 client.29009 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:39.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.879+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 shutdown_connections 2024-09-21T07:08:39.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.879+0000 7f0996a7a740 1 -- 172.21.15.114:0/3711475375 wait complete. 2024-09-21T07:08:39.881 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.879+0000 7f0996a7a740 20 client.29009 trim_cache size 0 max 16384 2024-09-21T07:08:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.910+0000 7f0996a7a740 1 Processor -- start 2024-09-21T07:08:39.912 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 -- start start 2024-09-21T07:08:39.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e322b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e23720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0968010060 con 0x5634c3a52b70 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0968011c60 con 0x5634c3a4a990 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.911+0000 7f0996a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0968010730 con 0x5634c3a73850 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e23720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e23720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34380/0 (socket says 172.21.15.114:34380) 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 -- 172.21.15.114:0/454680145 learned_addr learned my addr 172.21.15.114:0/454680145 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098ca00640 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098d400640 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 -- 172.21.15.114:0/454680145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 msgr2=0x5634c3e322b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.915 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e322b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 -- 172.21.15.114:0/454680145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 msgr2=0x5634c3e327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e327f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.912+0000 7f098de00640 1 -- 172.21.15.114:0/454680145 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c3e18fe0 con 0x5634c3a73850 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.913+0000 7f098de00640 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e23720 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0980008f50 tx=0x7f09800108c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9265aa2e577b5ea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.913+0000 7f0979c00640 1 -- 172.21.15.114:0/454680145 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098001e070 con 0x5634c3a73850 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.913+0000 7f0979c00640 1 -- 172.21.15.114:0/454680145 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098000eb30 con 0x5634c3a73850 2024-09-21T07:08:39.916 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.913+0000 7f0979c00640 1 -- 172.21.15.114:0/454680145 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980019c70 con 0x5634c3a73850 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.914+0000 7f0996a7a740 1 -- 172.21.15.114:0/454680145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 msgr2=0x5634c3e23720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.914+0000 7f0996a7a740 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e23720 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0980008f50 tx=0x7f09800108c0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 -- 172.21.15.114:0/454680145 shutdown_connections 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a73850 0x5634c3e23720 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 --2- 172.21.15.114:0/454680145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a4a990 0x5634c3e322b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 -- 172.21.15.114:0/454680145 >> 172.21.15.114:0/454680145 conn(0x5634c3a67c40 msgr2=0x5634c3e183c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 -- 172.21.15.114:0/454680145 shutdown_connections 2024-09-21T07:08:39.917 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.915+0000 7f0996a7a740 1 -- 172.21.15.114:0/454680145 wait complete. 2024-09-21T07:08:39.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.916+0000 7f0996a7a740 1 -- start start 2024-09-21T07:08:39.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.916+0000 7f0996a7a740 1 Processor -- start 2024-09-21T07:08:39.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.916+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a4a990 0x5634c3e293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.918 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.916+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e298e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.916+0000 7f0996a7a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a73850 0x5634c3e29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f0996a7a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0968011560 con 0x5634c3a52b70 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f0996a7a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0968010e00 con 0x5634c3a73850 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f0996a7a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f096800e060 con 0x5634c3a4a990 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f0996a7a740 10 client.0 _ll_register_callbacks cb 0x7f097c0e0f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f098ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e298e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f098ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e298e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49808/0 (socket says 172.21.15.114:49808) 2024-09-21T07:08:39.919 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f098ca00640 1 -- 172.21.15.114:0/1937795207 learned_addr learned my addr 172.21.15.114:0/1937795207 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.917+0000 7f098de00640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a73850 0x5634c3e29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098d400640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a4a990 0x5634c3e293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098ca00640 1 -- 172.21.15.114:0/1937795207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a4a990 msgr2=0x5634c3e293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098ca00640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a4a990 0x5634c3e293a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098ca00640 1 -- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a73850 msgr2=0x5634c3e29e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098ca00640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a73850 0x5634c3e29e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098ca00640 1 -- 172.21.15.114:0/1937795207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c3e21e30 con 0x5634c3a52b70 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098de00640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a73850 0x5634c3e29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f098ca00640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e298e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f097400bfd0 tx=0x7f0974004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f286532c57679b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.920 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.918+0000 7f096ea00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:08:39.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.919+0000 7f096ea00640 1 -- 172.21.15.114:0/1937795207 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097400fd70 con 0x5634c3a52b70 2024-09-21T07:08:39.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.919+0000 7f096ea00640 1 -- 172.21.15.114:0/1937795207 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f097402f420 con 0x5634c3a52b70 2024-09-21T07:08:39.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.919+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f098000eb30 con 0x5634c3a52b70 2024-09-21T07:08:39.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.920+0000 7f096ea00640 1 -- 172.21.15.114:0/1937795207 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f097402f620 con 0x5634c3a52b70 2024-09-21T07:08:39.921 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.920+0000 7f0996a7a740 10 client.18360 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.920+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5634c3ea3db0 con 0x5634c3a52b70 2024-09-21T07:08:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.920+0000 7f096ea00640 1 -- 172.21.15.114:0/1937795207 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098000eb30 con 0x5634c3a52b70 2024-09-21T07:08:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.920+0000 7f096ea00640 1 -- 172.21.15.114:0/1937795207 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7f0974039a00 con 0x5634c3a52b70 2024-09-21T07:08:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f0996a7a740 10 client.18360 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f0996a7a740 10 client.18360 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:08:39.922 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f0996a7a740 20 client.18360 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f0996a7a740 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a07580 0x5634c3ea6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f0996a7a740 4 client.18360 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-21T07:08:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x5634c3d8e490 con 0x5634c3a07580 2024-09-21T07:08:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f098d400640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a07580 0x5634c3ea6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.921+0000 7f096ea00640 10 client.18360 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:39.923 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.922+0000 7f098d400640 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a07580 0x5634c3ea6690 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f20c87c8612af30 server_cookie=a3b441f88fdff071 in_seq=0 out_seq=0 2024-09-21T07:08:39.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 -- 172.21.15.114:0/1531627319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9811cc30 msgr2=0x7fbf9811f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 --2- 172.21.15.114:0/1531627319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9811cc30 0x7fbf9811f140 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009ca0 tx=0x7fbf9002a8d0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 -- 172.21.15.114:0/1531627319 shutdown_connections 2024-09-21T07:08:39.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 --2- 172.21.15.114:0/1531627319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9811cc30 0x7fbf9811f140 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 --2- 172.21.15.114:0/1531627319 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9810a9e0 0x7fbf9810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 --2- 172.21.15.114:0/1531627319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf98111f40 0x7fbf9810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.947+0000 7fbf9da00640 1 -- 172.21.15.114:0/1531627319 >> 172.21.15.114:0/1531627319 conn(0x7fbf98070ec0 msgr2=0x7fbf980712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.948+0000 7fbf9da00640 1 -- 172.21.15.114:0/1531627319 shutdown_connections 2024-09-21T07:08:39.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.948+0000 7fbf9da00640 1 -- 172.21.15.114:0/1531627319 wait complete. 2024-09-21T07:08:39.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.949+0000 7fbf9da00640 1 Processor -- start 2024-09-21T07:08:39.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.949+0000 7fbf9da00640 1 -- start start 2024-09-21T07:08:39.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf9da00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf9810a9e0 0x7fbf981a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf9da00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 0x7fbf981a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf9da00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9811cc30 0x7fbf981aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf9da00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98122980 con 0x7fbf9811cc30 2024-09-21T07:08:39.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf9da00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98122800 con 0x7fbf98111f40 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf9da00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98122b00 con 0x7fbf9810a9e0 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf96000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 0x7fbf981a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf9810a9e0 0x7fbf981a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf96000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 0x7fbf981a9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46918/0 (socket says 172.21.15.114:46918) 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.950+0000 7fbf96000640 1 -- 172.21.15.114:0/4059464643 learned_addr learned my addr 172.21.15.114:0/4059464643 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf96000640 1 -- 172.21.15.114:0/4059464643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf9810a9e0 msgr2=0x7fbf981a92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf96000640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf9810a9e0 0x7fbf981a92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf96000640 1 -- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9811cc30 msgr2=0x7fbf981aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf96000640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9811cc30 0x7fbf981aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf96000640 1 -- 172.21.15.114:0/4059464643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf981af260 con 0x7fbf98111f40 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf96000640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 0x7fbf981a9890 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005fd0 tx=0x7fbf8400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.953 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf8f400640 1 -- 172.21.15.114:0/4059464643 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84017070 con 0x7fbf98111f40 2024-09-21T07:08:39.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf8f400640 1 -- 172.21.15.114:0/4059464643 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf84009c30 con 0x7fbf98111f40 2024-09-21T07:08:39.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.951+0000 7fbf8f400640 1 -- 172.21.15.114:0/4059464643 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84003d10 con 0x7fbf98111f40 2024-09-21T07:08:39.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.952+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf981af550 con 0x7fbf98111f40 2024-09-21T07:08:39.954 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.952+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf981b5cb0 con 0x7fbf98111f40 2024-09-21T07:08:39.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.956+0000 7fbfa47ee740 1 Processor -- start 2024-09-21T07:08:39.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.956+0000 7fbfa47ee740 1 -- start start 2024-09-21T07:08:39.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbf8f400640 1 -- 172.21.15.114:0/4059464643 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fbf84003eb0 con 0x7fbf98111f40 2024-09-21T07:08:39.958 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757007390 0x5567573cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556757345f70 con 0x556757007390 2024-09-21T07:08:39.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567573353a0 con 0x556757028070 2024-09-21T07:08:39.959 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567573391b0 con 0x556756fff1b0 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbf97400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbf97400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46928/0 (socket says 172.21.15.114:46928) 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.957+0000 7fbf97400640 1 -- 172.21.15.114:0/2342585579 learned_addr learned my addr 172.21.15.114:0/2342585579 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf97400640 1 -- 172.21.15.114:0/2342585579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 msgr2=0x5567573cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf97400640 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf97400640 1 -- 172.21.15.114:0/2342585579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757007390 msgr2=0x5567573cf5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf97400640 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757007390 0x5567573cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf97400640 1 -- 172.21.15.114:0/2342585579 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567572ffc90 con 0x556757028070 2024-09-21T07:08:39.960 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf97400640 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573ceb60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbf9002f460 tx=0x7fbf9002f490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86516d3c7ea83bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf83e00640 1 -- 172.21.15.114:0/2342585579 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90031b00 con 0x556757028070 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf83e00640 1 -- 172.21.15.114:0/2342585579 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9003f070 con 0x556757028070 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.958+0000 7fbf83e00640 1 -- 172.21.15.114:0/2342585579 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90004040 con 0x556757028070 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.959+0000 7fbf8f400640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf7c0b67d0 0x7fbf7c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.959+0000 7fbf8f400640 1 -- 172.21.15.114:0/4059464643 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fbf840e76c0 con 0x7fbf98111f40 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2342585579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 msgr2=0x5567573ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573ceb60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbf9002f460 tx=0x7fbf9002f490 comp rx=0 tx=0).stop 2024-09-21T07:08:39.961 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbf96a00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf7c0b67d0 0x7fbf7c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2342585579 shutdown_connections 2024-09-21T07:08:39.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757007390 0x5567573cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2342585579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573ceb60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2342585579 >> 172.21.15.114:0/2342585579 conn(0x55675701c460 msgr2=0x5567573cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.962 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2342585579 shutdown_connections 2024-09-21T07:08:39.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.960+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2342585579 wait complete. 2024-09-21T07:08:39.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.961+0000 7fbf96a00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf7c0b67d0 0x7fbf7c0b8ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5567573cd7e0 tx=0x7fbf88004750 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.961+0000 7fbfa47ee740 1 -- start start 2024-09-21T07:08:39.963 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.961+0000 7fbfa47ee740 1 Processor -- start 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573da720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x556757196700 con 0x556757028070 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567572d0540 con 0x556757007390 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5567573dd2e0 con 0x556756fff1b0 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbfa47ee740 10 client.0 _ll_register_callbacks cb 0x5567572e0430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573da720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573da720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34410/0 (socket says 172.21.15.114:34410) 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbf96a00640 1 -- 172.21.15.114:0/1837502185 learned_addr learned my addr 172.21.15.114:0/1837502185 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbf96000640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.964 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.962+0000 7fbf97400640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.965 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96000640 1 -- 172.21.15.114:0/1837502185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 msgr2=0x5567573da720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96000640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573da720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96000640 1 -- 172.21.15.114:0/1837502185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 msgr2=0x5567573e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96000640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96000640 1 -- 172.21.15.114:0/1837502185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556757093650 con 0x556757007390 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf97400640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96a00640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573da720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf96000640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573dac60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbf840e4a40 tx=0x7fbf840ad040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd8182bc8d5a57ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf80c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.963+0000 7fbf80c00640 1 -- 172.21.15.114:0/1837502185 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840e7470 con 0x556757007390 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbf80c00640 1 -- 172.21.15.114:0/1837502185 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fbf840e7b70 con 0x556757007390 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567572d2b50 con 0x556757007390 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbf80c00640 1 -- 172.21.15.114:0/1837502185 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbf840acc50 con 0x556757007390 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbfa47ee740 10 client.29024 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:39.966 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5567572ed910 con 0x556757007390 2024-09-21T07:08:39.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbf80c00640 1 -- 172.21.15.114:0/1837502185 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567572d2b50 con 0x556757007390 2024-09-21T07:08:39.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.964+0000 7fbf80c00640 1 -- 172.21.15.114:0/1837502185 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7fbf840a8a90 con 0x556757007390 2024-09-21T07:08:39.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbfa47ee740 10 client.29024 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:39.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbfa47ee740 10 client.29024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:39.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbfa47ee740 20 client.29024 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:39.967 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fe69f0 0x5567573ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbfa47ee740 4 client.29024 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:39.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556757347ec0 con 0x556756fe69f0 2024-09-21T07:08:39.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbf96a00640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fe69f0 0x5567573ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbf80c00640 10 client.29024 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:39.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.965+0000 7fbf96a00640 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fe69f0 0x5567573ecad0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8ddf08ae25dda9 server_cookie=1d9e45e20d837b02 in_seq=0 out_seq=0 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.966+0000 7fbf80c00640 1 -- 172.21.15.114:0/1837502185 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556757347ec0 con 0x556756fe69f0 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.966+0000 7fbf80c00640 10 client.29024 handle_command_reply: tid=0 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.966+0000 7fbfa47ee740 1 client.29024 shutdown 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.966+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 msgr2=0x5567573dac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.966+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573dac60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbf840e4a40 tx=0x7fbf840ad040 comp rx=0 tx=0).stop 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 shutdown_connections 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573dac60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fe69f0 0x5567573ecad0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/1837502185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573da720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 >> 172.21.15.114:0/1837502185 conn(0x55675701c460 msgr2=0x5567573cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbf80c00640 0 client.29024 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:39.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 shutdown_connections 2024-09-21T07:08:39.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 1 -- 172.21.15.114:0/1837502185 wait complete. 2024-09-21T07:08:39.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.967+0000 7fbfa47ee740 20 client.29024 trim_cache size 0 max 16384 2024-09-21T07:08:39.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 Processor -- start 2024-09-21T07:08:39.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 -- start start 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556756fff1b0 0x5567573e6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556757007390 0x5567573cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981bfdc0 con 0x556756fff1b0 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981c19c0 con 0x556757028070 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.990+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981c0490 con 0x556757007390 2024-09-21T07:08:39.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf96000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556757007390 0x5567573cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf97400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf97400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:46940/0 (socket says 172.21.15.114:46940) 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf97400640 1 -- 172.21.15.114:0/2141837099 learned_addr learned my addr 172.21.15.114:0/2141837099 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf96000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556757007390 0x5567573cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34422/0 (socket says 172.21.15.114:34422) 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf96a00640 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556756fff1b0 0x5567573e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf96a00640 1 -- 172.21.15.114:0/2141837099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556757007390 msgr2=0x5567573cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.991+0000 7fbf96a00640 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556757007390 0x5567573cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf96a00640 1 -- 172.21.15.114:0/2141837099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 msgr2=0x5567573cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf96a00640 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf96a00640 1 -- 172.21.15.114:0/2141837099 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567573d0e50 con 0x556756fff1b0 2024-09-21T07:08:39.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf96a00640 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556756fff1b0 0x5567573e6500 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x5567573ed1d0 tx=0x7fbf88015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c0dbb6243100f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf83400640 1 -- 172.21.15.114:0/2141837099 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8802c030 con 0x556756fff1b0 2024-09-21T07:08:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf83400640 1 -- 172.21.15.114:0/2141837099 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf88015d60 con 0x556756fff1b0 2024-09-21T07:08:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.992+0000 7fbf83400640 1 -- 172.21.15.114:0/2141837099 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8801b040 con 0x556756fff1b0 2024-09-21T07:08:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2141837099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556756fff1b0 msgr2=0x5567573e6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:39.995 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556756fff1b0 0x5567573e6500 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x5567573ed1d0 tx=0x7fbf88015000 comp rx=0 tx=0).stop 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2141837099 shutdown_connections 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757028070 0x5567573cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556757007390 0x5567573cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/2141837099 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556756fff1b0 0x5567573e6500 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2141837099 >> 172.21.15.114:0/2141837099 conn(0x55675701c460 msgr2=0x5567573d8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2141837099 shutdown_connections 2024-09-21T07:08:39.996 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.994+0000 7fbfa47ee740 1 -- 172.21.15.114:0/2141837099 wait complete. 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 -- start start 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 Processor -- start 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981c12c0 con 0x556757028070 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981c0b60 con 0x556757007390 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.995+0000 7fbfa47ee740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbf981bddc0 con 0x556756fff1b0 2024-09-21T07:08:39.997 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbfa47ee740 10 client.0 _ll_register_callbacks cb 0x7fbf840a9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34436/0 (socket says 172.21.15.114:34436) 2024-09-21T07:08:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 -- 172.21.15.114:0/4273348640 learned_addr learned my addr 172.21.15.114:0/4273348640 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf97400640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.998 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96000640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 -- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 msgr2=0x5567573e2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573e2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 -- 172.21.15.114:0/4273348640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 msgr2=0x5567573e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96a00640 1 -- 172.21.15.114:0/4273348640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567573e3250 con 0x556756fff1b0 2024-09-21T07:08:39.999 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.996+0000 7fbf96000640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573e2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.997+0000 7fbf96a00640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573f2ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800ee60 tx=0x7fbf88009170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7afbce42678a339 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.997+0000 7fbf80c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.997+0000 7fbf80c00640 1 -- 172.21.15.114:0/4273348640 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88035070 con 0x556756fff1b0 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.997+0000 7fbf80c00640 1 -- 172.21.15.114:0/4273348640 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fbf880136c0 con 0x556756fff1b0 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.997+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf88015d60 con 0x556756fff1b0 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.997+0000 7fbf80c00640 1 -- 172.21.15.114:0/4273348640 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fbf8801b040 con 0x556756fff1b0 2024-09-21T07:08:40.000 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 10 client.28984 fetch_fsmap learned FSMap version 56 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x556757457490 con 0x556756fff1b0 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbf80c00640 1 -- 172.21.15.114:0/4273348640 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf88015d60 con 0x556756fff1b0 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbf80c00640 1 -- 172.21.15.114:0/4273348640 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 56) ==== 8386+0+0 (secure 0 0 0) 0x7fbf88021c20 con 0x556756fff1b0 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 10 client.28984 fetch_fsmap finished waiting for FSMap version 56 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 10 client.28984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 20 client.28984 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fbbda0 0x556757459c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 4 client.28984 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556757343970 con 0x556756fbbda0 2024-09-21T07:08:40.001 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.998+0000 7fbf96000640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fbbda0 0x556757459c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.999+0000 7fbf96000640 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fbbda0 0x556757459c20 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2498b0ecb6acd2b3 server_cookie=9dc3f617619596aa in_seq=0 out_seq=0 2024-09-21T07:08:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.999+0000 7fbf80c00640 10 client.28984 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.999+0000 7fbf80c00640 1 -- 172.21.15.114:0/4273348640 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x556757343970 con 0x556756fbbda0 2024-09-21T07:08:40.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.999+0000 7fbf80c00640 10 client.28984 handle_command_reply: tid=0 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:39.999+0000 7fbfa47ee740 1 client.28984 shutdown 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 msgr2=0x5567573f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573f2ab0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800ee60 tx=0x7fbf88009170 comp rx=0 tx=0).stop 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 shutdown_connections 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x556757028070 0x5567573e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556757007390 0x5567573e2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x556756fbbda0 0x556757459c20 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 --2- 172.21.15.114:0/4273348640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556756fff1b0 0x5567573f2ab0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbf80c00640 0 client.28984 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 >> 172.21.15.114:0/4273348640 conn(0x55675701c460 msgr2=0x5567573d8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 shutdown_connections 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 1 -- 172.21.15.114:0/4273348640 wait complete. 2024-09-21T07:08:40.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.000+0000 7fbfa47ee740 20 client.28984 trim_cache size 0 max 16384 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "68277b18-d8ea-49d2-baab-e2089ee24e2b": { 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:08:00.304805+0000", 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.004 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:08:40.005 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.006 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.007 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:08:40.008 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.009 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:08:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:08:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.010 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:08:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:08:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.011 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.012 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:08:40.013 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.014 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:08:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:08:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.016 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:08:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:08:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.017 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:06:54.030598+0000", 2024-09-21T07:08:40.018 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.019 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:08:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf7c0b67d0 msgr2=0x7fbf7c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf7c0b67d0 0x7fbf7c0b8ce0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x5567573cd7e0 tx=0x7fbf88004750 comp rx=0 tx=0).stop 2024-09-21T07:08:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 msgr2=0x7fbf981a9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 0x7fbf981a9890 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005fd0 tx=0x7fbf8400dde0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 shutdown_connections 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbf9811cc30 0x7fbf981aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbf7c0b67d0 0x7fbf7c0b8ce0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf98111f40 0x7fbf981a9890 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 --2- 172.21.15.114:0/4059464643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf9810a9e0 0x7fbf981a92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 >> 172.21.15.114:0/4059464643 conn(0x7fbf98070ec0 msgr2=0x7fbf9811e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 shutdown_connections 2024-09-21T07:08:40.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.001+0000 7fbf9da00640 1 -- 172.21.15.114:0/4059464643 wait complete. 2024-09-21T07:08:40.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f096ea00640 1 -- 172.21.15.114:0/1937795207 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5634c3d8e490 con 0x5634c3a07580 2024-09-21T07:08:40.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f096ea00640 10 client.18360 handle_command_reply: tid=0 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 client.18360 shutdown 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 msgr2=0x5634c3e298e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e298e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f097400bfd0 tx=0x7f0974004730 comp rx=0 tx=0).stop 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 shutdown_connections 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5634c3a73850 0x5634c3e29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5634c3a52b70 0x5634c3e298e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5634c3a07580 0x5634c3ea6690 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 --2- 172.21.15.114:0/1937795207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5634c3a4a990 0x5634c3e293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f096ea00640 0 client.18360 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 >> 172.21.15.114:0/1937795207 conn(0x5634c3a67c40 msgr2=0x5634c3e182b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 shutdown_connections 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.127+0000 7f0996a7a740 1 -- 172.21.15.114:0/1937795207 wait complete. 2024-09-21T07:08:40.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.128+0000 7f0996a7a740 20 client.18360 trim_cache size 0 max 16384 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 56, 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 54, 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "38fafa95": { 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 54, 2024-09-21T07:08:40.131 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 138.8, 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.132 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.133 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:40.133 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.128+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09700b66f0 msgr2=0x7f09700b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.128+0000 7f098e800640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09700b66f0 0x7f09700b8c00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0988079a30 tx=0x7f0974005d30 comp rx=0 tx=0).stop 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.128+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 msgr2=0x7f0988108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.128+0000 7f098e800640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988108870 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f097c01aa30 tx=0x7f097c004a90 comp rx=0 tx=0).stop 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 shutdown_connections 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f09881145b0 0x7f09881092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0988078290 0x7f0988108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09700b66f0 0x7f09700b8c00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 --2- 172.21.15.114:0/2146512572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0988077940 0x7f0988108870 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 >> 172.21.15.114:0/2146512572 conn(0x7f0988101680 msgr2=0x7f0988116620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 shutdown_connections 2024-09-21T07:08:40.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:40.129+0000 7f098e800640 1 -- 172.21.15.114:0/2146512572 wait complete. 2024-09-21T07:08:41.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:41 smithi114 ceph-mon[30654]: pgmap v846: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-21T07:08:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:41 smithi183 ceph-mon[34219]: pgmap v846: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-21T07:08:41.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:41 smithi193 ceph-mon[34601]: pgmap v846: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 3.7 MiB/s wr, 394 op/s 2024-09-21T07:08:42.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:42 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:42.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:42 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:42 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:42.834 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 845.0 s, 156.5 tps, lat 287.026 ms stddev 281.401 2024-09-21T07:08:43.504 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:43 smithi114 ceph-mon[30654]: pgmap v847: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-09-21T07:08:43.549 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '38fafa95', quiesce took 14.4 seconds. Will release after: 4.450000000000001 2024-09-21T07:08:43.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:43 smithi193 ceph-mon[34601]: pgmap v847: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-09-21T07:08:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:43 smithi183 ceph-mon[34219]: pgmap v847: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-09-21T07:08:44.245 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:08:44.245 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:08:44.630 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:45.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 -- 172.21.15.114:0/3354540183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb448109c40 msgr2=0x7fb448110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:45.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 --2- 172.21.15.114:0/3354540183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb448109c40 0x7fb448110470 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb440009ca0 tx=0x7fb44002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 -- 172.21.15.114:0/3354540183 shutdown_connections 2024-09-21T07:08:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 --2- 172.21.15.114:0/3354540183 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb448109c40 0x7fb448110470 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 --2- 172.21.15.114:0/3354540183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4481092f0 0x7fb448109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 --2- 172.21.15.114:0/3354540183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 0x7fb448103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 -- 172.21.15.114:0/3354540183 >> 172.21.15.114:0/3354540183 conn(0x7fb4480ff440 msgr2=0x7fb448101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 -- 172.21.15.114:0/3354540183 shutdown_connections 2024-09-21T07:08:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.305+0000 7fb447e00640 1 -- 172.21.15.114:0/3354540183 wait complete. 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.306+0000 7fb447e00640 1 Processor -- start 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 -- start start 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 0x7fb44819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4481092f0 0x7fb44819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb448109c40 0x7fb44819fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb448113950 con 0x7fb4481092f0 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4481137d0 con 0x7fb4481039b0 2024-09-21T07:08:45.310 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb447e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb448113ad0 con 0x7fb448109c40 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.307+0000 7fb446000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4481092f0 0x7fb44819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 0x7fb44819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4481092f0 0x7fb44819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46008/0 (socket says 172.21.15.114:46008) 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446000640 1 -- 172.21.15.114:0/2048599831 learned_addr learned my addr 172.21.15.114:0/2048599831 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb447400640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb448109c40 0x7fb44819fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 -- 172.21.15.114:0/2048599831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb448109c40 msgr2=0x7fb44819fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb448109c40 0x7fb44819fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 -- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4481092f0 msgr2=0x7fb44819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4481092f0 0x7fb44819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 -- 172.21.15.114:0/2048599831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4481a0210 con 0x7fb4481039b0 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.308+0000 7fb446a00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 0x7fb44819a2a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9f0 tx=0x7fb43800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.309+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43800ccc0 con 0x7fb4481039b0 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.309+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4380108e0 con 0x7fb4481039b0 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.309+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4481a0500 con 0x7fb4481039b0 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.309+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438018880 con 0x7fb4481039b0 2024-09-21T07:08:45.312 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.309+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4481a6c60 con 0x7fb4481039b0 2024-09-21T07:08:45.313 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.310+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb448103dc0 con 0x7fb4481039b0 2024-09-21T07:08:45.315 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.313+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb438010430 con 0x7fb4481039b0 2024-09-21T07:08:45.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.314+0000 7fb43f400640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb42c0b6680 0x7fb42c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:45.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.314+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fb4380e7fd0 con 0x7fb4481039b0 2024-09-21T07:08:45.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.314+0000 7fb446000640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb42c0b6680 0x7fb42c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:45.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.315+0000 7fb446000640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb42c0b6680 0x7fb42c0b8b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb4340095f0 tx=0x7fb434005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:45.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.316+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4380b2530 con 0x7fb4481039b0 2024-09-21T07:08:45.550 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.548+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb44802e640 con 0x7fb4481039b0 2024-09-21T07:08:45.551 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:45 smithi114 ceph-mon[30654]: pgmap v848: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 397 op/s 2024-09-21T07:08:45.551 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.549+0000 7fb43f400640 1 -- 172.21.15.114:0/2048599831 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11994 (secure 0 0 0) 0x7fb4380b7470 con 0x7fb4481039b0 2024-09-21T07:08:45.552 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:08:45.552 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 57 2024-09-21T07:08:45.552 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":57,"btime":"2024-09-21T07:08:41:220727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:08:40.627012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb42c0b6680 msgr2=0x7fb42c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb42c0b6680 0x7fb42c0b8b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb4340095f0 tx=0x7fb434005f90 comp rx=0 tx=0).stop 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 msgr2=0x7fb44819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 0x7fb44819a2a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9f0 tx=0x7fb43800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 shutdown_connections 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb448109c40 0x7fb44819fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb4481092f0 0x7fb44819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.554 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb42c0b6680 0x7fb42c0b8b90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 --2- 172.21.15.114:0/2048599831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4481039b0 0x7fb44819a2a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:45.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.551+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 >> 172.21.15.114:0/2048599831 conn(0x7fb4480ff440 msgr2=0x7fb448101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:45.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.552+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 shutdown_connections 2024-09-21T07:08:45.555 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:45.552+0000 7fb447e00640 1 -- 172.21.15.114:0/2048599831 wait complete. 2024-09-21T07:08:45.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:45 smithi183 ceph-mon[34219]: pgmap v848: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 397 op/s 2024-09-21T07:08:45.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:45 smithi193 ceph-mon[34601]: pgmap v848: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 397 op/s 2024-09-21T07:08:46.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:46 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2048599831' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:46.470 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:08:46.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:46 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2048599831' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:46 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2048599831' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:46.862 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:47.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:47 smithi114 ceph-mon[30654]: pgmap v849: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-09-21T07:08:47.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:47 smithi183 ceph-mon[34219]: pgmap v849: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-09-21T07:08:47.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 -- 172.21.15.114:0/3296165870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb0105320 msgr2=0x7f9eb010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/3296165870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb0105320 0x7f9eb010d870 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009900 tx=0x7f9e9802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 -- 172.21.15.114:0/3296165870 shutdown_connections 2024-09-21T07:08:47.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/3296165870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb010de30 0x7f9eb0110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/3296165870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb0105320 0x7f9eb010d870 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/3296165870 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9eb01049d0 0x7f9eb0104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 -- 172.21.15.114:0/3296165870 >> 172.21.15.114:0/3296165870 conn(0x7f9eb0079740 msgr2=0x7f9eb0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 -- 172.21.15.114:0/3296165870 shutdown_connections 2024-09-21T07:08:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.532+0000 7f9eb5e00640 1 -- 172.21.15.114:0/3296165870 wait complete. 2024-09-21T07:08:47.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.533+0000 7f9eb5e00640 1 Processor -- start 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 -- start start 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 0x7f9eb0102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9eb0105320 0x7f9eb0102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb010de30 0x7f9eb0103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0113a90 con 0x7f9eb0105320 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0113910 con 0x7f9eb010de30 2024-09-21T07:08:47.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eb5e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0113c10 con 0x7f9eb01049d0 2024-09-21T07:08:47.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9eb0105320 0x7f9eb0102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 0x7f9eb0102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb010de30 0x7f9eb0103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 0x7f9eb0102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55792/0 (socket says 172.21.15.114:55792) 2024-09-21T07:08:47.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.534+0000 7f9eaea00640 1 -- 172.21.15.114:0/1976511205 learned_addr learned my addr 172.21.15.114:0/1976511205 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eaea00640 1 -- 172.21.15.114:0/1976511205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb010de30 msgr2=0x7f9eb0103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eaea00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb010de30 0x7f9eb0103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eaea00640 1 -- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9eb0105320 msgr2=0x7f9eb0102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eaea00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9eb0105320 0x7f9eb0102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eaea00640 1 -- 172.21.15.114:0/1976511205 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb01a8800 con 0x7f9eb01049d0 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eaea00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 0x7f9eb0102710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e9c0 tx=0x7f9ea000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eb4a00640 1 -- 172.21.15.114:0/1976511205 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cde0 con 0x7f9eb01049d0 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eb4a00640 1 -- 172.21.15.114:0/1976511205 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea00108e0 con 0x7f9eb01049d0 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb01a8af0 con 0x7f9eb01049d0 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.535+0000 7f9eb4a00640 1 -- 172.21.15.114:0/1976511205 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00188d0 con 0x7f9eb01049d0 2024-09-21T07:08:47.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.536+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a9000 con 0x7f9eb01049d0 2024-09-21T07:08:47.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.539+0000 7f9eb4a00640 1 -- 172.21.15.114:0/1976511205 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9ea001f020 con 0x7f9eb01049d0 2024-09-21T07:08:47.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.541+0000 7f9eb4a00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e940b6680 0x7f9e940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.541+0000 7f9eb4a00640 1 -- 172.21.15.114:0/1976511205 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f9ea00e7ce0 con 0x7f9eb01049d0 2024-09-21T07:08:47.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.541+0000 7f9ea6000640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e940b6680 0x7f9e940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.544 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.542+0000 7f9ea6000640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e940b6680 0x7f9e940b8b90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009230 tx=0x7f9e98031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:47.557 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 Processor -- start 2024-09-21T07:08:47.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 -- start start 2024-09-21T07:08:47.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 0x55d9476bbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.558 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 0x55d9476bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d947632fb0 con 0x55d947315070 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9476223e0 con 0x55d9472ec1b0 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.555+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9476261f0 con 0x55d9472f4390 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 0x55d9476bbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 0x55d9476bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 0x55d9476bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37650/0 (socket says 172.21.15.114:37650) 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9ea6000640 1 -- 172.21.15.114:0/1920077535 learned_addr learned my addr 172.21.15.114:0/1920077535 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaea00640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 0x55d9476bbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46036/0 (socket says 172.21.15.114:46036) 2024-09-21T07:08:47.559 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaf400640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaf400640 1 -- 172.21.15.114:0/1920077535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 msgr2=0x55d9476bc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaf400640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 0x55d9476bc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaf400640 1 -- 172.21.15.114:0/1920077535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 msgr2=0x55d9476bbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaf400640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 0x55d9476bbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaf400640 1 -- 172.21.15.114:0/1920077535 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9475eccd0 con 0x55d9472f4390 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9ea6000640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 0x55d9476bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:47.560 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.556+0000 7f9eaea00640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 0x55d9476bbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:47.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.557+0000 7f9eaf400640 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bc620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009f00 tx=0x7f9ea8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee71eb6c386a17f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:47.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.557+0000 7f9e9fe00640 1 -- 172.21.15.114:0/1920077535 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8004d20 con 0x55d9472f4390 2024-09-21T07:08:47.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.557+0000 7f9e9fe00640 1 -- 172.21.15.114:0/1920077535 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea8004050 con 0x55d9472f4390 2024-09-21T07:08:47.561 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.557+0000 7f9e9fe00640 1 -- 172.21.15.114:0/1920077535 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8005280 con 0x55d9472f4390 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 -- 172.21.15.114:0/1920077535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 msgr2=0x55d9476bc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bc620 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009f00 tx=0x7f9ea8011810 comp rx=0 tx=0).stop 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 -- 172.21.15.114:0/1920077535 shutdown_connections 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bc620 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9472ec1b0 0x55d9476bc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/1920077535 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d947315070 0x55d9476bbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 -- 172.21.15.114:0/1920077535 >> 172.21.15.114:0/1920077535 conn(0x55d947309460 msgr2=0x55d9476b9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 -- 172.21.15.114:0/1920077535 shutdown_connections 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.558+0000 7f9ebca6e740 1 -- 172.21.15.114:0/1920077535 wait complete. 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.559+0000 7f9ebca6e740 1 -- start start 2024-09-21T07:08:47.562 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.559+0000 7f9ebca6e740 1 Processor -- start 2024-09-21T07:08:47.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.559+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.564 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d947483700 con 0x55d9472f4390 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9475bd580 con 0x55d947315070 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d9476ca380 con 0x55d9472ec1b0 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ebca6e740 10 client.0 _ll_register_callbacks cb 0x55d9475cd470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaf400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55798/0 (socket says 172.21.15.114:55798) 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 -- 172.21.15.114:0/3592036390 learned_addr learned my addr 172.21.15.114:0/3592036390 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 -- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 msgr2=0x55d9476d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 -- 172.21.15.114:0/3592036390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 msgr2=0x55d9476c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.560+0000 7f9eaea00640 1 -- 172.21.15.114:0/3592036390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d947380650 con 0x55d9472ec1b0 2024-09-21T07:08:47.565 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9eaea00640 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476c7740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00ad690 tx=0x7f9ea0010cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977708bab1e29df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9e9cc00640 1 -- 172.21.15.114:0/3592036390 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00d4030 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9475bfb90 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9e9cc00640 1 -- 172.21.15.114:0/3592036390 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f9ea0010430 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9e9cc00640 1 -- 172.21.15.114:0/3592036390 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0010630 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9ebca6e740 10 client.29002 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d9475da950 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.561+0000 7f9e9cc00640 1 -- 172.21.15.114:0/3592036390 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d9475bfb90 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9e9cc00640 1 -- 172.21.15.114:0/3592036390 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f9ea0018ab0 con 0x55d9472ec1b0 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ebca6e740 10 client.29002 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ebca6e740 10 client.29002 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ebca6e740 20 client.29002 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472d39f0 0x55d9476d95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.566 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ebca6e740 4 client.29002 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:47.567 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d947634f00 con 0x55d9472d39f0 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.562+0000 7f9ea6000640 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472d39f0 0x55d9476d95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.563+0000 7f9ea6000640 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472d39f0 0x55d9476d95e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=694f2e7ef5b021fa server_cookie=416b27cebe5976e4 in_seq=0 out_seq=0 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.563+0000 7f9e9cc00640 10 client.29002 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9e9cc00640 1 -- 172.21.15.114:0/3592036390 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d947634f00 con 0x55d9472d39f0 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9e9cc00640 10 client.29002 handle_command_reply: tid=0 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 client.29002 shutdown 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 msgr2=0x55d9476c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476c7740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00ad690 tx=0x7f9ea0010cf0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 shutdown_connections 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472d39f0 0x55d9476d95e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.568 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3592036390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476c7740 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 >> 172.21.15.114:0/3592036390 conn(0x55d947309460 msgr2=0x55d9476bed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:47.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9e9cc00640 0 client.29002 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:08:47.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 shutdown_connections 2024-09-21T07:08:47.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3592036390 wait complete. 2024-09-21T07:08:47.569 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.564+0000 7f9ebca6e740 20 client.29002 trim_cache size 0 max 16384 2024-09-21T07:08:47.580 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 Processor -- start 2024-09-21T07:08:47.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 -- start start 2024-09-21T07:08:47.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472ec1b0 0x55d9476bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b3190 con 0x55d9472ec1b0 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b4d60 con 0x55d947315070 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.578+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b3860 con 0x55d9472f4390 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9eaea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472ec1b0 0x55d9476bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55802/0 (socket says 172.21.15.114:55802) 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 -- 172.21.15.114:0/3822118771 learned_addr learned my addr 172.21.15.114:0/3822118771 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:47.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9eaf400640 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 -- 172.21.15.114:0/3822118771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 msgr2=0x55d9476d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 -- 172.21.15.114:0/3822118771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472ec1b0 msgr2=0x55d9476bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472ec1b0 0x55d9476bd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9ea6000640 1 -- 172.21.15.114:0/3822118771 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9476c8800 con 0x55d9472f4390 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.579+0000 7f9eaea00640 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472ec1b0 0x55d9476bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.580+0000 7f9ea6000640 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bf3b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9e980057c0 tx=0x7f9e9803d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8e920bcbf36dec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.580+0000 7f9e9f400640 1 -- 172.21.15.114:0/3822118771 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98035040 con 0x55d9472f4390 2024-09-21T07:08:47.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.580+0000 7f9e9f400640 1 -- 172.21.15.114:0/3822118771 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e98031860 con 0x55d9472f4390 2024-09-21T07:08:47.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.580+0000 7f9e9f400640 1 -- 172.21.15.114:0/3822118771 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98004b80 con 0x55d9472f4390 2024-09-21T07:08:47.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3822118771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 msgr2=0x55d9476bf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.584 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bf3b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9e980057c0 tx=0x7f9e9803d000 comp rx=0 tx=0).stop 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3822118771 shutdown_connections 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d9476d3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472f4390 0x55d9476bf3b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/3822118771 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472ec1b0 0x55d9476bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3822118771 >> 172.21.15.114:0/3822118771 conn(0x55d947309460 msgr2=0x55d9476beca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3822118771 shutdown_connections 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 -- 172.21.15.114:0/3822118771 wait complete. 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.581+0000 7f9ebca6e740 1 -- start start 2024-09-21T07:08:47.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 Processor -- start 2024-09-21T07:08:47.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d94774b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b4690 con 0x55d9472f4390 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b3f30 con 0x55d947315070 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b1190 con 0x55d9472ec1b0 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.582+0000 7f9ebca6e740 10 client.0 _ll_register_callbacks cb 0x7f9ea00ad330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46062/0 (socket says 172.21.15.114:46062) 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 -- 172.21.15.114:0/2876456146 learned_addr learned my addr 172.21.15.114:0/2876456146 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9eaea00640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9eaf400640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d94774b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 -- 172.21.15.114:0/2876456146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 msgr2=0x55d9476cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.587 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 -- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 msgr2=0x55d94774b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d94774b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 -- 172.21.15.114:0/2876456146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94774bcf0 con 0x55d9472f4390 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9ea6000640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476cbec0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e98054040 tx=0x7f9e98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf5dffcb0b1957e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.583+0000 7f9e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9e9cc00640 1 -- 172.21.15.114:0/2876456146 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98035050 con 0x55d9472f4390 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9e9cc00640 1 -- 172.21.15.114:0/2876456146 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f9e98032c50 con 0x55d9472f4390 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e98031860 con 0x55d9472f4390 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9e9cc00640 1 -- 172.21.15.114:0/2876456146 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f9e980035f0 con 0x55d9472f4390 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9ebca6e740 10 client.18396 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:47.588 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d94774cdc0 con 0x55d9472f4390 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.584+0000 7f9e9cc00640 1 -- 172.21.15.114:0/2876456146 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e98031860 con 0x55d9472f4390 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9e9cc00640 1 -- 172.21.15.114:0/2876456146 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f9e9805b9a0 con 0x55d9472f4390 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9ebca6e740 10 client.18396 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9ebca6e740 10 client.18396 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9ebca6e740 20 client.18396 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472a8da0 0x55d94774f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9ebca6e740 4 client.18396 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9476309b0 con 0x55d9472a8da0 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.585+0000 7f9eaea00640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472a8da0 0x55d94774f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9eaea00640 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472a8da0 0x55d94774f4e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f179811c7a34b191 server_cookie=35a31955771ccd88 in_seq=0 out_seq=0 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9e9cc00640 10 client.18396 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9e9cc00640 1 -- 172.21.15.114:0/2876456146 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9476309b0 con 0x55d9472a8da0 2024-09-21T07:08:47.590 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9e9cc00640 10 client.18396 handle_command_reply: tid=0 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 client.18396 shutdown 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 msgr2=0x55d9476cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476cbec0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e98054040 tx=0x7f9e98009260 comp rx=0 tx=0).stop 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 shutdown_connections 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x55d9472a8da0 0x55d94774f4e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d947315070 0x55d94774b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d9472f4390 0x55d9476cbec0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 --2- 172.21.15.114:0/2876456146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d9472ec1b0 0x55d9476cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 >> 172.21.15.114:0/2876456146 conn(0x55d947309460 msgr2=0x55d9476bd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9e9cc00640 0 client.18396 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 shutdown_connections 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 1 -- 172.21.15.114:0/2876456146 wait complete. 2024-09-21T07:08:47.591 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.586+0000 7f9ebca6e740 20 client.18396 trim_cache size 0 max 16384 2024-09-21T07:08:47.592 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:08:47.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e940b6680 msgr2=0x7f9e940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e940b6680 0x7f9e940b8b90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009230 tx=0x7f9e98031400 comp rx=0 tx=0).stop 2024-09-21T07:08:47.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 msgr2=0x7f9eb0102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:47.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 0x7f9eb0102710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e9c0 tx=0x7f9ea000ee80 comp rx=0 tx=0).stop 2024-09-21T07:08:47.592 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 shutdown_connections 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9e940b6680 0x7f9e940b8b90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9eb010de30 0x7f9eb0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9eb0105320 0x7f9eb0102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 --2- 172.21.15.114:0/1976511205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb01049d0 0x7f9eb0102710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 >> 172.21.15.114:0/1976511205 conn(0x7f9eb0079740 msgr2=0x7f9eb01068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 shutdown_connections 2024-09-21T07:08:47.593 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:47.587+0000 7f9eb5e00640 1 -- 172.21.15.114:0/1976511205 wait complete. 2024-09-21T07:08:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:47 smithi193 ceph-mon[34601]: pgmap v849: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 2.8 MiB/s wr, 290 op/s 2024-09-21T07:08:47.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 850.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:08:48.000 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.258+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3384309873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 msgr2=0x7f8a68109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.258+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3384309873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a68109650 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009900 tx=0x7f8a5002a400 comp rx=0 tx=0).stop 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.258+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3384309873 shutdown_connections 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.258+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3384309873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a68109b90 0x7f8a681103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.259+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3384309873 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a68109650 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.259+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3384309873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a68103910 0x7f8a68103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.259+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3384309873 >> 172.21.15.114:0/3384309873 conn(0x7f8a680ff330 msgr2=0x7f8a68101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.259+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3384309873 shutdown_connections 2024-09-21T07:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.259+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3384309873 wait complete. 2024-09-21T07:08:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 Processor -- start 2024-09-21T07:08:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 -- start start 2024-09-21T07:08:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a68103910 0x7f8a6819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a6819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a68109b90 0x7f8a681a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68113850 con 0x7f8a68109240 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681136d0 con 0x7f8a68109b90 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.260+0000 7f8a6ca00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681139d0 con 0x7f8a68103910 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a6819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a6819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46090/0 (socket says 172.21.15.114:46090) 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3617089268 learned_addr learned my addr 172.21.15.114:0/3617089268 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a67e00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a68109b90 0x7f8a681a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a67400640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a68103910 0x7f8a6819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3617089268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a68103910 msgr2=0x7f8a6819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a68103910 0x7f8a6819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3617089268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a68109b90 msgr2=0x7f8a681a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a68109b90 0x7f8a681a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3617089268 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a681a23c0 con 0x7f8a68109240 2024-09-21T07:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.261+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a6819ca30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a920 tx=0x7f8a50031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.262+0000 7f8a64c00640 1 -- 172.21.15.114:0/3617089268 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5003f030 con 0x7f8a68109240 2024-09-21T07:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.262+0000 7f8a64c00640 1 -- 172.21.15.114:0/3617089268 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a50004490 con 0x7f8a68109240 2024-09-21T07:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.262+0000 7f8a64c00640 1 -- 172.21.15.114:0/3617089268 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50035070 con 0x7f8a68109240 2024-09-21T07:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.262+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a681a2650 con 0x7f8a68109240 2024-09-21T07:08:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.262+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a681a8e40 con 0x7f8a68109240 2024-09-21T07:08:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 Processor -- start 2024-09-21T07:08:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 -- start start 2024-09-21T07:08:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a64c00640 1 -- 172.21.15.114:0/3617089268 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8a50042030 con 0x7f8a68109240 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf1ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf1b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d93ce906b0 con 0x55d93cb554d0 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d93ce7fae0 con 0x55d93cb4d2f0 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.266+0000 7f8a7506e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d93ce838f0 con 0x55d93cb761b0 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf1b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf1b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46104/0 (socket says 172.21.15.114:46104) 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a5fe00640 1 -- 172.21.15.114:0/4023346665 learned_addr learned my addr 172.21.15.114:0/4023346665 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf1ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67400640 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 -- 172.21.15.114:0/4023346665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 msgr2=0x55d93cf1b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 -- 172.21.15.114:0/4023346665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 msgr2=0x55d93cf1b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf1b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 -- 172.21.15.114:0/4023346665 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93ce4c780 con 0x55d93cb761b0 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf1b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.267+0000 7f8a67e00640 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf1ae10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009f30 tx=0x7f8a600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=903dda03d87370a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.268+0000 7f8a64c00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8a4c0b6680 0x7f8a4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.268+0000 7f8a57e00640 1 -- 172.21.15.114:0/4023346665 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60004d20 con 0x55d93cb761b0 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.268+0000 7f8a57e00640 1 -- 172.21.15.114:0/4023346665 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a60004050 con 0x55d93cb761b0 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.268+0000 7f8a57e00640 1 -- 172.21.15.114:0/4023346665 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60005280 con 0x55d93cb761b0 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.268+0000 7f8a64c00640 1 -- 172.21.15.114:0/3617089268 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f8a50107e10 con 0x7f8a68109240 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.268+0000 7f8a67400640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8a4c0b6680 0x7f8a4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 -- 172.21.15.114:0/4023346665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 msgr2=0x55d93cf1ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a67400640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8a4c0b6680 0x7f8a4c0b8b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55d93cf19a90 tx=0x7f8a58005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf1ae10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009f30 tx=0x7f8a600118d0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 -- 172.21.15.114:0/4023346665 shutdown_connections 2024-09-21T07:08:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf1b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 --2- 172.21.15.114:0/4023346665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf1ae10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 -- 172.21.15.114:0/4023346665 >> 172.21.15.114:0/4023346665 conn(0x55d93cb6a300 msgr2=0x55d93cf18d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 -- 172.21.15.114:0/4023346665 shutdown_connections 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.269+0000 7f8a7506e740 1 -- 172.21.15.114:0/4023346665 wait complete. 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 -- start start 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 Processor -- start 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb554d0 0x55d93cf26960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 0x55d93cf26ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55d93cdb7ab0 con 0x55d93cb761b0 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d93ce26570 con 0x55d93cb4d2f0 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.270+0000 7f8a7506e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d93cf29590 con 0x55d93cb554d0 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a7506e740 10 client.0 _ll_register_callbacks cb 0x55d93ce35100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 0x55d93cf26ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 0x55d93cf26ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46108/0 (socket says 172.21.15.114:46108) 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 -- 172.21.15.114:0/1665497233 learned_addr learned my addr 172.21.15.114:0/1665497233 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb554d0 0x55d93cf26960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67400640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 -- 172.21.15.114:0/1665497233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb554d0 msgr2=0x55d93cf26960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb554d0 0x55d93cf26960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 -- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 msgr2=0x55d93cf26420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf26420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.271+0000 7f8a67e00640 1 -- 172.21.15.114:0/1665497233 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93cbe1790 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a67e00640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 0x55d93cf26ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8a60004920 tx=0x7f8a600042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39310ad020b68fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a54c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a54c00640 1 -- 172.21.15.114:0/1665497233 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60026020 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a60004050 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a54c00640 1 -- 172.21.15.114:0/1665497233 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f8a60022660 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a54c00640 1 -- 172.21.15.114:0/1665497233 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f8a60031050 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a7506e740 10 client.18408 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d93cc5a620 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.272+0000 7f8a54c00640 1 -- 172.21.15.114:0/1665497233 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a60004050 con 0x55d93cb761b0 2024-09-21T07:08:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a54c00640 1 -- 172.21.15.114:0/1665497233 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f8a6000f980 con 0x55d93cb761b0 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a7506e740 10 client.18408 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a7506e740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a7506e740 20 client.18408 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb34d80 0x55d93cf38460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a7506e740 4 client.18408 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d93ce92600 con 0x55d93cb34d80 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.273+0000 7f8a67400640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb34d80 0x55d93cf38460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.274+0000 7f8a67400640 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb34d80 0x55d93cf38460 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad33e2bf1e0a0710 server_cookie=d953cad7b30899da in_seq=0 out_seq=0 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.274+0000 7f8a54c00640 10 client.18408 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a54c00640 1 -- 172.21.15.114:0/1665497233 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d93ce92600 con 0x55d93cb34d80 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a54c00640 10 client.18408 handle_command_reply: tid=0 2024-09-21T07:08:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 client.18408 shutdown 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 msgr2=0x55d93cf26ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 0x55d93cf26ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8a60004920 tx=0x7f8a600042a0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 shutdown_connections 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb761b0 0x55d93cf26ec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb34d80 0x55d93cf38460 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb554d0 0x55d93cf26960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1665497233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf26420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 >> 172.21.15.114:0/1665497233 conn(0x55d93cb6a300 msgr2=0x55d93cf1be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a54c00640 0 client.18408 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 shutdown_connections 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 1 -- 172.21.15.114:0/1665497233 wait complete. 2024-09-21T07:08:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.275+0000 7f8a7506e740 20 client.18408 trim_cache size 0 max 16384 2024-09-21T07:08:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.290+0000 7f8a7506e740 1 Processor -- start 2024-09-21T07:08:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.290+0000 7f8a7506e740 1 -- start start 2024-09-21T07:08:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf23b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a7506e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b2f60 con 0x55d93cb554d0 2024-09-21T07:08:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a7506e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b4b30 con 0x55d93cb4d2f0 2024-09-21T07:08:48.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a7506e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b3630 con 0x55d93cb761b0 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a67400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a67e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf23b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46124/0 (socket says 172.21.15.114:46124) 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 -- 172.21.15.114:0/1717660730 learned_addr learned my addr 172.21.15.114:0/1717660730 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 -- 172.21.15.114:0/1717660730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 msgr2=0x55d93cf23b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf23b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 -- 172.21.15.114:0/1717660730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 msgr2=0x55d93cf1c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a5fe00640 1 -- 172.21.15.114:0/1717660730 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d93cf32780 con 0x55d93cb554d0 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.291+0000 7f8a67e00640 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf23b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.292+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf235f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8a500c9f10 tx=0x7f8a5010b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c99bc68eff57d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:48.295 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.292+0000 7f8a57400640 1 -- 172.21.15.114:0/1717660730 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50049040 con 0x55d93cb554d0 2024-09-21T07:08:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.292+0000 7f8a57400640 1 -- 172.21.15.114:0/1717660730 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a500cd450 con 0x55d93cb554d0 2024-09-21T07:08:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.292+0000 7f8a57400640 1 -- 172.21.15.114:0/1717660730 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a500c90b0 con 0x55d93cb554d0 2024-09-21T07:08:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 -- 172.21.15.114:0/1717660730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 msgr2=0x55d93cf235f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf235f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8a500c9f10 tx=0x7f8a5010b960 comp rx=0 tx=0).stop 2024-09-21T07:08:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 -- 172.21.15.114:0/1717660730 shutdown_connections 2024-09-21T07:08:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb761b0 0x55d93cf23b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf235f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 --2- 172.21.15.114:0/1717660730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb4d2f0 0x55d93cf1c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 -- 172.21.15.114:0/1717660730 >> 172.21.15.114:0/1717660730 conn(0x55d93cb6a300 msgr2=0x55d93cf1be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 -- 172.21.15.114:0/1717660730 shutdown_connections 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 -- 172.21.15.114:0/1717660730 wait complete. 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 -- start start 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.293+0000 7f8a7506e740 1 Processor -- start 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb4d2f0 0x55d93cf2c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf2c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb761b0 0x55d93cf9ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b4460 con 0x55d93cb554d0 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b3d00 con 0x55d93cb761b0 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a681b0f60 con 0x55d93cb4d2f0 2024-09-21T07:08:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a7506e740 10 client.0 _ll_register_callbacks cb 0x7f8a5010b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf2c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf2c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46130/0 (socket says 172.21.15.114:46130) 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3729892353 learned_addr learned my addr 172.21.15.114:0/3729892353 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a67400640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb4d2f0 0x55d93cf2c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a67e00640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb761b0 0x55d93cf9ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3729892353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb4d2f0 msgr2=0x55d93cf2c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb4d2f0 0x55d93cf2c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb761b0 msgr2=0x55d93cf9ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.294+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb761b0 0x55d93cf9ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a5fe00640 1 -- 172.21.15.114:0/3729892353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d93cf9b3b0 con 0x55d93cb554d0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a67e00640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb761b0 0x55d93cf9ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a5fe00640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf2c6c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a440 tx=0x7f8a50105640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcb1109fa6c6356 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a54c00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a54c00640 1 -- 172.21.15.114:0/3729892353 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50049070 con 0x55d93cb554d0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a54c00640 1 -- 172.21.15.114:0/3729892353 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f8a500d6e20 con 0x55d93cb554d0 2024-09-21T07:08:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a500cd450 con 0x55d93cb554d0 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a54c00640 1 -- 172.21.15.114:0/3729892353 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f8a500c90b0 con 0x55d93cb554d0 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a7506e740 10 client.18414 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d93cfa26f0 con 0x55d93cb554d0 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.295+0000 7f8a54c00640 1 -- 172.21.15.114:0/3729892353 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a500cd450 con 0x55d93cb554d0 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a54c00640 1 -- 172.21.15.114:0/3729892353 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f8a500c9290 con 0x55d93cb554d0 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a7506e740 10 client.18414 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a7506e740 10 client.18414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a7506e740 20 client.18414 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a7506e740 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb0a1c0 0x55d93cfa4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a7506e740 4 client.18414 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:08:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d93ce8e0b0 con 0x55d93cb0a1c0 2024-09-21T07:08:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.296+0000 7f8a67400640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb0a1c0 0x55d93cfa4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.297+0000 7f8a67400640 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb0a1c0 0x55d93cfa4e00 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d15a13559c1563a server_cookie=4c8524bec04b6fa7 in_seq=0 out_seq=0 2024-09-21T07:08:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.297+0000 7f8a54c00640 10 client.18414 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.307+0000 7f8a54c00640 1 -- 172.21.15.114:0/3729892353 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d93ce8e0b0 con 0x55d93cb0a1c0 2024-09-21T07:08:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.307+0000 7f8a54c00640 10 client.18414 handle_command_reply: tid=0 2024-09-21T07:08:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.307+0000 7f8a7506e740 1 client.18414 shutdown 2024-09-21T07:08:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.307+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 msgr2=0x55d93cf2c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.307+0000 7f8a7506e740 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf2c6c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a440 tx=0x7f8a50105640 comp rx=0 tx=0).stop 2024-09-21T07:08:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 shutdown_connections 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d93cb761b0 0x55d93cf9ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55d93cb554d0 0x55d93cf2c6c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55d93cb0a1c0 0x55d93cfa4e00 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 --2- 172.21.15.114:0/3729892353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d93cb4d2f0 0x55d93cf2c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 >> 172.21.15.114:0/3729892353 conn(0x55d93cb6a300 msgr2=0x55d93cf17360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a54c00640 0 client.18414 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 shutdown_connections 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 1 -- 172.21.15.114:0/3729892353 wait complete. 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a7506e740 20 client.18414 trim_cache size 0 max 16384 2024-09-21T07:08:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8a4c0b6680 msgr2=0x7f8a4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8a4c0b6680 0x7f8a4c0b8b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x55d93cf19a90 tx=0x7f8a58005f90 comp rx=0 tx=0).stop 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 msgr2=0x7f8a6819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.308+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a6819ca30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8a5002a920 tx=0x7f8a50031a20 comp rx=0 tx=0).stop 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 shutdown_connections 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8a4c0b6680 0x7f8a4c0b8b90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a68109b90 0x7f8a681a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8a68109240 0x7f8a6819ca30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 --2- 172.21.15.114:0/3617089268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a68103910 0x7f8a6819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 >> 172.21.15.114:0/3617089268 conn(0x7f8a680ff330 msgr2=0x7f8a6810e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 shutdown_connections 2024-09-21T07:08:48.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:08:48.309+0000 7f8a6ca00640 1 -- 172.21.15.114:0/3617089268 wait complete. 2024-09-21T07:08:48.529 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:08:48.620 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:49.267 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:49.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.278+0000 7f7576e00640 1 -- 172.21.15.114:0/2025917485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75701094b0 msgr2=0x7f75701098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.278+0000 7f7576e00640 1 --2- 172.21.15.114:0/2025917485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75701094b0 0x7f75701098c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7558009900 tx=0x7f755802a410 comp rx=0 tx=0).stop 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 -- 172.21.15.114:0/2025917485 shutdown_connections 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 --2- 172.21.15.114:0/2025917485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7570109e00 0x7f7570110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 --2- 172.21.15.114:0/2025917485 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f75701094b0 0x7f75701098c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 --2- 172.21.15.114:0/2025917485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7570103b70 0x7f7570103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 -- 172.21.15.114:0/2025917485 >> 172.21.15.114:0/2025917485 conn(0x7f75700ff5e0 msgr2=0x7f7570101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 -- 172.21.15.114:0/2025917485 shutdown_connections 2024-09-21T07:08:49.282 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.279+0000 7f7576e00640 1 -- 172.21.15.114:0/2025917485 wait complete. 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 Processor -- start 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 -- start start 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 0x7f757019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75701094b0 0x7f757019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7570109e00 0x7f75701a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7570113b10 con 0x7f7570103b70 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7570113990 con 0x7f7570109e00 2024-09-21T07:08:49.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.280+0000 7f7576e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7570113c90 con 0x7f75701094b0 2024-09-21T07:08:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 0x7f757019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 0x7f757019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46162/0 (socket says 172.21.15.114:46162) 2024-09-21T07:08:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 -- 172.21.15.114:0/988582214 learned_addr learned my addr 172.21.15.114:0/988582214 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f7574e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7570109e00 0x7f75701a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f7567e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75701094b0 0x7f757019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 -- 172.21.15.114:0/988582214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75701094b0 msgr2=0x7f757019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75701094b0 0x7f757019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 -- 172.21.15.114:0/988582214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7570109e00 msgr2=0x7f75701a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7570109e00 0x7f75701a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.281+0000 7f756fe00640 1 -- 172.21.15.114:0/988582214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75701a2630 con 0x7f7570103b70 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.282+0000 7f756fe00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 0x7f757019c6c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f756000e9f0 tx=0x7f756000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.282+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756000cd80 con 0x7f7570103b70 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.282+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75600108e0 con 0x7f7570103b70 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.282+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75701a2920 con 0x7f7570103b70 2024-09-21T07:08:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.282+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560018980 con 0x7f7570103b70 2024-09-21T07:08:49.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.283+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a9080 con 0x7f7570103b70 2024-09-21T07:08:49.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.284+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7570103f80 con 0x7f7570103b70 2024-09-21T07:08:49.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.286+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7560010430 con 0x7f7570103b70 2024-09-21T07:08:49.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.288+0000 7f756d600640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75540b6700 0x7f75540b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:49.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.288+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f75600e8220 con 0x7f7570103b70 2024-09-21T07:08:49.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.288+0000 7f7567e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75540b6700 0x7f75540b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:49.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.289+0000 7f7567e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75540b6700 0x7f75540b8c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f755802aa00 tx=0x7f7558005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:49.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.290+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f75600b2780 con 0x7f7570103b70 2024-09-21T07:08:49.408 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:49 smithi114 ceph-mon[30654]: pgmap v850: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-09-21T07:08:49.523 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.521+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75701a93f0 con 0x7f7570103b70 2024-09-21T07:08:49.524 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.522+0000 7f756d600640 1 -- 172.21.15.114:0/988582214 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11994 (secure 0 0 0) 0x7f75600b76c0 con 0x7f7570103b70 2024-09-21T07:08:49.525 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:08:49.525 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":57,"btime":"2024-09-21T07:08:41:220727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:08:40.627012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:08:49.525 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 57 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75540b6700 msgr2=0x7f75540b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75540b6700 0x7f75540b8c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f755802aa00 tx=0x7f7558005a20 comp rx=0 tx=0).stop 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 msgr2=0x7f757019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 0x7f757019c6c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f756000e9f0 tx=0x7f756000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 shutdown_connections 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f75540b6700 0x7f75540b8c10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.527 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7570109e00 0x7f75701a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75701094b0 0x7f757019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 --2- 172.21.15.114:0/988582214 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7570103b70 0x7f757019c6c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 >> 172.21.15.114:0/988582214 conn(0x7f75700ff5e0 msgr2=0x7f757010e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.524+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 shutdown_connections 2024-09-21T07:08:49.528 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:49.525+0000 7f7576e00640 1 -- 172.21.15.114:0/988582214 wait complete. 2024-09-21T07:08:49.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:49 smithi183 ceph-mon[34219]: pgmap v850: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-09-21T07:08:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:49 smithi193 ceph-mon[34601]: pgmap v850: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-09-21T07:08:50.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 -- 172.21.15.114:0/424419639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 msgr2=0x7fe32c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 --2- 172.21.15.114:0/424419639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c103dc0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe320009ca0 tx=0x7fe32002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 -- 172.21.15.114:0/424419639 shutdown_connections 2024-09-21T07:08:50.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 --2- 172.21.15.114:0/424419639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe32c109c40 0x7fe32c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 --2- 172.21.15.114:0/424419639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 0x7fe32c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 --2- 172.21.15.114:0/424419639 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c103dc0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 -- 172.21.15.114:0/424419639 >> 172.21.15.114:0/424419639 conn(0x7fe32c0ff440 msgr2=0x7fe32c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.064+0000 7fe333000640 1 -- 172.21.15.114:0/424419639 shutdown_connections 2024-09-21T07:08:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.065+0000 7fe333000640 1 -- 172.21.15.114:0/424419639 wait complete. 2024-09-21T07:08:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 Processor -- start 2024-09-21T07:08:50.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 -- start start 2024-09-21T07:08:50.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 0x7fe32c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe32c109c40 0x7fe32c1a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c113950 con 0x7fe32c1039b0 2024-09-21T07:08:50.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1137d0 con 0x7fe32c1092f0 2024-09-21T07:08:50.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe333000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c113ad0 con 0x7fe32c109c40 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe331c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe331c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46180/0 (socket says 172.21.15.114:46180) 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.066+0000 7fe331c00640 1 -- 172.21.15.114:0/3304682609 learned_addr learned my addr 172.21.15.114:0/3304682609 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331200640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 0x7fe32c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe332600640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe32c109c40 0x7fe32c1a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331c00640 1 -- 172.21.15.114:0/3304682609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe32c109c40 msgr2=0x7fe32c1a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331c00640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe32c109c40 0x7fe32c1a1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331c00640 1 -- 172.21.15.114:0/3304682609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 msgr2=0x7fe32c19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331c00640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 0x7fe32c19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331c00640 1 -- 172.21.15.114:0/3304682609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c1a2490 con 0x7fe32c1039b0 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331200640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 0x7fe32c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.067+0000 7fe331c00640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c19c520 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe320005e30 tx=0x7fe320031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.068+0000 7fe32a000640 1 -- 172.21.15.114:0/3304682609 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe320003b70 con 0x7fe32c1039b0 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.068+0000 7fe32a000640 1 -- 172.21.15.114:0/3304682609 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe320003d10 con 0x7fe32c1039b0 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.068+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c1a2720 con 0x7fe32c1039b0 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.068+0000 7fe32a000640 1 -- 172.21.15.114:0/3304682609 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32002fa30 con 0x7fe32c1039b0 2024-09-21T07:08:50.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.068+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1a8ee0 con 0x7fe32c1039b0 2024-09-21T07:08:50.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 Processor -- start 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 -- start start 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 0x55fe6f962f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fe6f8d9fb0 con 0x55fe6f5bc070 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fe6f8c93e0 con 0x55fe6f59b390 2024-09-21T07:08:50.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.071+0000 7fe33b277740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe6f8cd1f0 con 0x55fe6f5931b0 2024-09-21T07:08:50.075 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 0x55fe6f962f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 0x55fe6f962f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46186/0 (socket says 172.21.15.114:46186) 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 -- 172.21.15.114:0/220907132 learned_addr learned my addr 172.21.15.114:0/220907132 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe331200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe32a000640 1 -- 172.21.15.114:0/3304682609 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe32002fbd0 con 0x7fe32c1039b0 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 -- 172.21.15.114:0/220907132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 msgr2=0x55fe6f9634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 -- 172.21.15.114:0/220907132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 msgr2=0x55fe6f9639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.072+0000 7fe332600640 1 -- 172.21.15.114:0/220907132 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe6f893cd0 con 0x55fe6f5bc070 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.073+0000 7fe332600640 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 0x55fe6f962f70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe324009ca0 tx=0x7fe324011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dae6fbe97a12f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.073+0000 7fe31ea00640 1 -- 172.21.15.114:0/220907132 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32401e070 con 0x55fe6f5bc070 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.073+0000 7fe31ea00640 1 -- 172.21.15.114:0/220907132 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe324009e50 con 0x55fe6f5bc070 2024-09-21T07:08:50.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.073+0000 7fe31ea00640 1 -- 172.21.15.114:0/220907132 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400ea50 con 0x55fe6f5bc070 2024-09-21T07:08:50.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe32a000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe3140b6680 0x7fe3140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 -- 172.21.15.114:0/220907132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 msgr2=0x55fe6f962f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 0x55fe6f962f70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe324009ca0 tx=0x7fe324011810 comp rx=0 tx=0).stop 2024-09-21T07:08:50.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 -- 172.21.15.114:0/220907132 shutdown_connections 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 --2- 172.21.15.114:0/220907132 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5bc070 0x55fe6f962f70 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 -- 172.21.15.114:0/220907132 >> 172.21.15.114:0/220907132 conn(0x55fe6f5b0460 msgr2=0x55fe6f960e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe32a000640 1 -- 172.21.15.114:0/3304682609 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fe320108bf0 con 0x7fe32c1039b0 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe331c00640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe3140b6680 0x7fe3140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 -- 172.21.15.114:0/220907132 shutdown_connections 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.074+0000 7fe33b277740 1 -- 172.21.15.114:0/220907132 wait complete. 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.075+0000 7fe331c00640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe3140b6680 0x7fe3140b8b90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55fe6f961bf0 tx=0x7fe32002a8c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:50.078 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.075+0000 7fe33b277740 1 -- start start 2024-09-21T07:08:50.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.075+0000 7fe33b277740 1 Processor -- start 2024-09-21T07:08:50.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f96e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f96ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f97aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fe6f72a700 con 0x55fe6f5931b0 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fe6f864580 con 0x55fe6f59b390 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fe6f971750 con 0x55fe6f5bc070 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe33b277740 10 client.0 _ll_register_callbacks cb 0x55fe6f874470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f96ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe332600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f96e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f96ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37790/0 (socket says 172.21.15.114:37790) 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 -- 172.21.15.114:0/1824486730 learned_addr learned my addr 172.21.15.114:0/1824486730 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe332600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f96e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46196/0 (socket says 172.21.15.114:46196) 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe332600640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f97aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 -- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 msgr2=0x55fe6f97aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.081 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f97aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 -- 172.21.15.114:0/1824486730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 msgr2=0x55fe6f96e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f96e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe331200640 1 -- 172.21.15.114:0/1824486730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe6f627650 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.076+0000 7fe332600640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f96e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe331200640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f96ecb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe318012d20 tx=0x7fe31800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5171bbb13ded6ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe313400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe313400640 1 -- 172.21.15.114:0/1824486730 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318012110 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe6f866b90 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe313400640 1 -- 172.21.15.114:0/1824486730 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fe318004050 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe313400640 1 -- 172.21.15.114:0/1824486730 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe318025050 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe33b277740 10 client.29087 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.077+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe6f881950 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe313400640 1 -- 172.21.15.114:0/1824486730 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fe6f866b90 con 0x55fe6f59b390 2024-09-21T07:08:50.082 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe313400640 1 -- 172.21.15.114:0/1824486730 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fe318010430 con 0x55fe6f59b390 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe33b277740 10 client.29087 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe33b277740 10 client.29087 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe33b277740 20 client.29087 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe33b277740 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f57a9f0 0x55fe6f980990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe33b277740 4 client.29087 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.078+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe6f8dbf00 con 0x55fe6f57a9f0 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.079+0000 7fe332600640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f57a9f0 0x55fe6f980990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.079+0000 7fe332600640 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f57a9f0 0x55fe6f980990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8dba3cd9452f0d5 server_cookie=e6e0817b9f2b5f28 in_seq=0 out_seq=0 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.079+0000 7fe313400640 10 client.29087 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.080+0000 7fe313400640 1 -- 172.21.15.114:0/1824486730 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe6f8dbf00 con 0x55fe6f57a9f0 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.080+0000 7fe313400640 10 client.29087 handle_command_reply: tid=0 2024-09-21T07:08:50.084 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.080+0000 7fe33b277740 1 client.29087 shutdown 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 msgr2=0x55fe6f96ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f96ecb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe318012d20 tx=0x7fe31800f790 comp rx=0 tx=0).stop 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 shutdown_connections 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f57a9f0 0x55fe6f980990 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f97aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f96ecb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 --2- 172.21.15.114:0/1824486730 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f96e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe313400640 0 client.29087 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 >> 172.21.15.114:0/1824486730 conn(0x55fe6f5b0460 msgr2=0x55fe6f966150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 shutdown_connections 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 1 -- 172.21.15.114:0/1824486730 wait complete. 2024-09-21T07:08:50.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.081+0000 7fe33b277740 20 client.29087 trim_cache size 0 max 16384 2024-09-21T07:08:50.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.095+0000 7fe33b277740 1 Processor -- start 2024-09-21T07:08:50.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.095+0000 7fe33b277740 1 -- start start 2024-09-21T07:08:50.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.095+0000 7fe33b277740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe33b277740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 0x55fe6f960610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe33b277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f5bc070 0x55fe6f960b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe33b277740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b2fb0 con 0x55fe6f59b390 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe33b277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b4b80 con 0x55fe6f5bc070 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe33b277740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b3680 con 0x55fe6f5931b0 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 0x55fe6f960610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 0x55fe6f960610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46212/0 (socket says 172.21.15.114:46212) 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 -- 172.21.15.114:0/4093446097 learned_addr learned my addr 172.21.15.114:0/4093446097 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe332600640 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe332600640 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f5bc070 0x55fe6f960b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 -- 172.21.15.114:0/4093446097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 msgr2=0x55fe6f9703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 -- 172.21.15.114:0/4093446097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f5bc070 msgr2=0x55fe6f960b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f5bc070 0x55fe6f960b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.096+0000 7fe331200640 1 -- 172.21.15.114:0/4093446097 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe6f965dd0 con 0x55fe6f59b390 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.097+0000 7fe331200640 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 0x55fe6f960610 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe318001eb0 tx=0x7fe318012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bbbcce8fe5a9e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.097+0000 7fe31e000640 1 -- 172.21.15.114:0/4093446097 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318000e60 con 0x55fe6f59b390 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.097+0000 7fe31e000640 1 -- 172.21.15.114:0/4093446097 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31800fe80 con 0x55fe6f59b390 2024-09-21T07:08:50.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.097+0000 7fe31e000640 1 -- 172.21.15.114:0/4093446097 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318010ba0 con 0x55fe6f59b390 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 -- 172.21.15.114:0/4093446097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 msgr2=0x55fe6f960610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 0x55fe6f960610 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe318001eb0 tx=0x7fe318012320 comp rx=0 tx=0).stop 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 -- 172.21.15.114:0/4093446097 shutdown_connections 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f5bc070 0x55fe6f960b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f59b390 0x55fe6f960610 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 --2- 172.21.15.114:0/4093446097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5931b0 0x55fe6f9703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 -- 172.21.15.114:0/4093446097 >> 172.21.15.114:0/4093446097 conn(0x55fe6f5b0460 msgr2=0x55fe6f96fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:50.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 -- 172.21.15.114:0/4093446097 shutdown_connections 2024-09-21T07:08:50.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.098+0000 7fe33b277740 1 -- 172.21.15.114:0/4093446097 wait complete. 2024-09-21T07:08:50.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 -- start start 2024-09-21T07:08:50.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 Processor -- start 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f9730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f9ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b44b0 con 0x55fe6f5931b0 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b3d50 con 0x55fe6f59b390 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1b0fb0 con 0x55fe6f5bc070 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.099+0000 7fe33b277740 10 client.0 _ll_register_callbacks cb 0x7fe318012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f9730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.103 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f9730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46226/0 (socket says 172.21.15.114:46226) 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 -- 172.21.15.114:0/1294520146 learned_addr learned my addr 172.21.15.114:0/1294520146 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe331200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f9ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 -- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 msgr2=0x55fe6f9ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f9ea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 -- 172.21.15.114:0/1294520146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 msgr2=0x55fe6f9735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 -- 172.21.15.114:0/1294520146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe6f9ea8d0 con 0x55fe6f5931b0 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe331200640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe332600640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f9730b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe324000c00 tx=0x7fe3240097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0bce0ae3da87dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:50.104 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.100+0000 7fe313400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:08:50.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe313400640 1 -- 172.21.15.114:0/1294520146 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32402b070 con 0x55fe6f5931b0 2024-09-21T07:08:50.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe313400640 1 -- 172.21.15.114:0/1294520146 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fe32400e900 con 0x55fe6f5931b0 2024-09-21T07:08:50.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe6f8d79b0 con 0x55fe6f5931b0 2024-09-21T07:08:50.105 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe313400640 1 -- 172.21.15.114:0/1294520146 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe32401e040 con 0x55fe6f5931b0 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe33b277740 10 client.18438 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55fe6f9ebb20 con 0x55fe6f5931b0 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.101+0000 7fe313400640 1 -- 172.21.15.114:0/1294520146 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fe6f8d79b0 con 0x55fe6f5931b0 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe313400640 1 -- 172.21.15.114:0/1294520146 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fe324024040 con 0x55fe6f5931b0 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe33b277740 10 client.18438 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe33b277740 10 client.18438 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe33b277740 20 client.18438 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe33b277740 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f54fda0 0x55fe6f9ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe33b277740 4 client.18438 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe6f8d4de0 con 0x55fe6f54fda0 2024-09-21T07:08:50.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.102+0000 7fe331200640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f54fda0 0x55fe6f9ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:50.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.103+0000 7fe331200640 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f54fda0 0x55fe6f9ee140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=728514b0340fc896 server_cookie=ecfb20320214a9ac in_seq=0 out_seq=0 2024-09-21T07:08:50.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.103+0000 7fe313400640 10 client.18438 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:08:50.107 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.103+0000 7fe313400640 1 -- 172.21.15.114:0/1294520146 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe6f8d4de0 con 0x55fe6f54fda0 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.103+0000 7fe313400640 10 client.18438 handle_command_reply: tid=0 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.103+0000 7fe33b277740 1 client.18438 shutdown 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 msgr2=0x55fe6f9730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f9730b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe324000c00 tx=0x7fe3240097c0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 shutdown_connections 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55fe6f54fda0 0x55fe6f9ee140 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fe6f5bc070 0x55fe6f9ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe6f59b390 0x55fe6f9735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 --2- 172.21.15.114:0/1294520146 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fe6f5931b0 0x55fe6f9730b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 >> 172.21.15.114:0/1294520146 conn(0x55fe6f5b0460 msgr2=0x55fe6f964960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe313400640 0 client.18438 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 shutdown_connections 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 1 -- 172.21.15.114:0/1294520146 wait complete. 2024-09-21T07:08:50.108 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.104+0000 7fe33b277740 20 client.18438 trim_cache size 0 max 16384 2024-09-21T07:08:50.109 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:08:50.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe3140b6680 msgr2=0x7fe3140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe3140b6680 0x7fe3140b8b90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55fe6f961bf0 tx=0x7fe32002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 msgr2=0x7fe32c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:50.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c19c520 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe320005e30 tx=0x7fe320031840 comp rx=0 tx=0).stop 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 shutdown_connections 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe32c109c40 0x7fe32c1a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe3140b6680 0x7fe3140b8b90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe32c1092f0 0x7fe32c19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 --2- 172.21.15.114:0/3304682609 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe32c1039b0 0x7fe32c19c520 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 >> 172.21.15.114:0/3304682609 conn(0x7fe32c0ff440 msgr2=0x7fe32c1089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 shutdown_connections 2024-09-21T07:08:50.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:50.105+0000 7fe333000640 1 -- 172.21.15.114:0/3304682609 wait complete. 2024-09-21T07:08:50.497 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:50 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/988582214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:50 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/988582214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:50 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/988582214' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:51.238 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id 38fafa95 --release --await' 2024-09-21T07:08:51.238 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id 38fafa95 --release --await 2024-09-21T07:08:51.444 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:08:51.493 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:51 smithi114 ceph-mon[30654]: pgmap v851: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-21T07:08:51.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:51 smithi183 ceph-mon[34219]: pgmap v851: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-21T07:08:51.621 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:51.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:51 smithi193 ceph-mon[34601]: pgmap v851: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 -- 172.21.15.114:0/2624008815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954103b70 msgr2=0x7fd954103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 --2- 172.21.15.114:0/2624008815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954103b70 0x7fd954103f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd944009900 tx=0x7fd94402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 -- 172.21.15.114:0/2624008815 shutdown_connections 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 --2- 172.21.15.114:0/2624008815 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954109e00 0x7fd954110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 --2- 172.21.15.114:0/2624008815 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541094b0 0x7fd9541098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 --2- 172.21.15.114:0/2624008815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954103b70 0x7fd954103f80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 -- 172.21.15.114:0/2624008815 >> 172.21.15.114:0/2624008815 conn(0x7fd9540ff5e0 msgr2=0x7fd954101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:52.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 -- 172.21.15.114:0/2624008815 shutdown_connections 2024-09-21T07:08:52.290 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.286+0000 7fd95ac00640 1 -- 172.21.15.114:0/2624008815 wait complete. 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 Processor -- start 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 -- start start 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 0x7fd954075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541094b0 0x7fd954075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954109e00 0x7fd954076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd954113b10 con 0x7fd9541094b0 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd954113990 con 0x7fd954109e00 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.287+0000 7fd95ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd954113c90 con 0x7fd954103b70 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 0x7fd954075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 0x7fd954075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55976/0 (socket says 172.21.15.114:55976) 2024-09-21T07:08:52.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 -- 172.21.15.114:0/3890048698 learned_addr learned my addr 172.21.15.114:0/3890048698 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953400640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541094b0 0x7fd954075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd958c00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954109e00 0x7fd954076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 -- 172.21.15.114:0/3890048698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954109e00 msgr2=0x7fd954076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954109e00 0x7fd954076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 -- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541094b0 msgr2=0x7fd954075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541094b0 0x7fd954075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.288+0000 7fd953e00640 1 -- 172.21.15.114:0/3890048698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9541a8740 con 0x7fd954103b70 2024-09-21T07:08:52.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.289+0000 7fd953e00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 0x7fd954075710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd944005ab0 tx=0x7fd944003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:52.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.289+0000 7fd950c00640 1 -- 172.21.15.114:0/3890048698 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94403f070 con 0x7fd954103b70 2024-09-21T07:08:52.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.289+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9541a89d0 con 0x7fd954103b70 2024-09-21T07:08:52.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.289+0000 7fd950c00640 1 -- 172.21.15.114:0/3890048698 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd944031b60 con 0x7fd954103b70 2024-09-21T07:08:52.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.289+0000 7fd950c00640 1 -- 172.21.15.114:0/3890048698 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944035030 con 0x7fd954103b70 2024-09-21T07:08:52.293 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.289+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541a8f00 con 0x7fd954103b70 2024-09-21T07:08:52.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 Processor -- start 2024-09-21T07:08:52.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd950c00640 1 -- 172.21.15.114:0/3890048698 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd944042070 con 0x7fd954103b70 2024-09-21T07:08:52.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 -- start start 2024-09-21T07:08:52.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfcf520 0x5596bc3761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc376700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 0x5596bc376c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5596bc2ed290 con 0x5596bbfae840 2024-09-21T07:08:52.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596bc2dc6c0 con 0x5596bbfcf520 2024-09-21T07:08:52.296 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.293+0000 7fd961895740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5596bc2e04d0 con 0x5596bbfa6660 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 0x5596bc376c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd958c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfcf520 0x5596bc3761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 0x5596bc376c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46256/0 (socket says 172.21.15.114:46256) 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 -- 172.21.15.114:0/2091719222 learned_addr learned my addr 172.21.15.114:0/2091719222 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953e00640 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc376700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 -- 172.21.15.114:0/2091719222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 msgr2=0x5596bc376700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc376700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 -- 172.21.15.114:0/2091719222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfcf520 msgr2=0x5596bc3761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfcf520 0x5596bc3761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd953400640 1 -- 172.21.15.114:0/2091719222 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596bc269460 con 0x5596bbfae840 2024-09-21T07:08:52.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.294+0000 7fd958c00640 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfcf520 0x5596bc3761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:52.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd953400640 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 0x5596bc376c40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009c20 tx=0x7fd93c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3aa2a0b62ee64a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:52.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd950c00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9380b6700 0x7fd9380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd950c00640 1 -- 172.21.15.114:0/3890048698 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fd9441081b0 con 0x7fd954103b70 2024-09-21T07:08:52.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd958c00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9380b6700 0x7fd9380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd949200640 1 -- 172.21.15.114:0/2091719222 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c004230 con 0x5596bbfae840 2024-09-21T07:08:52.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd949200640 1 -- 172.21.15.114:0/2091719222 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd93c0043d0 con 0x5596bbfae840 2024-09-21T07:08:52.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.295+0000 7fd949200640 1 -- 172.21.15.114:0/2091719222 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c004d50 con 0x5596bbfae840 2024-09-21T07:08:52.299 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 -- 172.21.15.114:0/2091719222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 msgr2=0x5596bc376c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 0x5596bc376c40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009c20 tx=0x7fd93c011810 comp rx=0 tx=0).stop 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd958c00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9380b6700 0x7fd9380b8c10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009fd0 tx=0x7fd94c009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 -- 172.21.15.114:0/2091719222 shutdown_connections 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfae840 0x5596bc376c40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc376700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 --2- 172.21.15.114:0/2091719222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfcf520 0x5596bc3761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 -- 172.21.15.114:0/2091719222 >> 172.21.15.114:0/2091719222 conn(0x5596bbfc3910 msgr2=0x5596bc3740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.296+0000 7fd961895740 1 -- 172.21.15.114:0/2091719222 shutdown_connections 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.297+0000 7fd961895740 1 -- 172.21.15.114:0/2091719222 wait complete. 2024-09-21T07:08:52.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.297+0000 7fd961895740 1 -- start start 2024-09-21T07:08:52.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 Processor -- start 2024-09-21T07:08:52.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc381db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc382330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc38dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5596bc13dbb0 con 0x5596bbfcf520 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596bc2776c0 con 0x5596bbfa6660 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5596bc3849a0 con 0x5596bbfae840 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd961895740 10 client.0 _ll_register_callbacks cb 0x5596bc292490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd953400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc382330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd958c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc38dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd953400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc382330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:55988/0 (socket says 172.21.15.114:55988) 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953400640 1 -- 172.21.15.114:0/2511790498 learned_addr learned my addr 172.21.15.114:0/2511790498 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:52.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.298+0000 7fd953e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc381db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953e00640 1 -- 172.21.15.114:0/2511790498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 msgr2=0x5596bc382330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953e00640 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc382330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953e00640 1 -- 172.21.15.114:0/2511790498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 msgr2=0x5596bc38dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953e00640 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc38dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953e00640 1 -- 172.21.15.114:0/2511790498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596bc03ab00 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd953e00640 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc381db0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd9441054d0 tx=0x7fd94403b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9806e75b7a8c985 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd941600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.299+0000 7fd941600640 1 -- 172.21.15.114:0/2511790498 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944049070 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd941600640 1 -- 172.21.15.114:0/2511790498 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fd9440f10b0 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd93c0043d0 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd941600640 1 -- 172.21.15.114:0/2511790498 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd9440c9b20 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 10 client.29105 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5596bc0b1960 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd941600640 1 -- 172.21.15.114:0/2511790498 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd93c0043d0 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd941600640 1 -- 172.21.15.114:0/2511790498 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fd944105750 con 0x5596bbfa6660 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 10 client.29105 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:52.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 10 client.29105 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:08:52.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 20 client.29105 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:52.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf8dea0 0x5596bc393d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 4 client.29105 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:52.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.300+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596bc2ef1e0 con 0x5596bbf8dea0 2024-09-21T07:08:52.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.301+0000 7fd953400640 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf8dea0 0x5596bc393d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.301+0000 7fd953400640 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf8dea0 0x5596bc393d60 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55742a22a52533b9 server_cookie=1c7f3972d72cd320 in_seq=0 out_seq=0 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.301+0000 7fd941600640 10 client.29105 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.302+0000 7fd941600640 1 -- 172.21.15.114:0/2511790498 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5596bc2ef1e0 con 0x5596bbf8dea0 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.302+0000 7fd941600640 10 client.29105 handle_command_reply: tid=0 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.302+0000 7fd961895740 1 client.29105 shutdown 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 msgr2=0x5596bc381db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc381db0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd9441054d0 tx=0x7fd94403b800 comp rx=0 tx=0).stop 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 shutdown_connections 2024-09-21T07:08:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc38dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf8dea0 0x5596bc393d60 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc382330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 --2- 172.21.15.114:0/2511790498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc381db0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 >> 172.21.15.114:0/2511790498 conn(0x5596bbfc3910 msgr2=0x5596bc379350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd941600640 0 client.29105 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 shutdown_connections 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 1 -- 172.21.15.114:0/2511790498 wait complete. 2024-09-21T07:08:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.303+0000 7fd961895740 20 client.29105 trim_cache size 0 max 16384 2024-09-21T07:08:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 Processor -- start 2024-09-21T07:08:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 -- start start 2024-09-21T07:08:52.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc38dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc37eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc37f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541b3050 con 0x5596bbfcf520 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541b4c20 con 0x5596bbfa6660 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.322+0000 7fd961895740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541b3720 con 0x5596bbfae840 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd958c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc37f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd958c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc37f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46274/0 (socket says 172.21.15.114:46274) 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd958c00640 1 -- 172.21.15.114:0/940657559 learned_addr learned my addr 172.21.15.114:0/940657559 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953e00640 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc38dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953400640 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc37eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953e00640 1 -- 172.21.15.114:0/940657559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 msgr2=0x5596bc37eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953e00640 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc37eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953e00640 1 -- 172.21.15.114:0/940657559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 msgr2=0x5596bc37f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953e00640 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc37f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd953e00640 1 -- 172.21.15.114:0/940657559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596bc374a20 con 0x5596bbfa6660 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.323+0000 7fd958c00640 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc37f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.324+0000 7fd953e00640 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc38dbc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd9440ca250 tx=0x7fd94402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05ff1ccd47b327d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.324+0000 7fd943e00640 1 -- 172.21.15.114:0/940657559 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944049070 con 0x5596bbfa6660 2024-09-21T07:08:52.327 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.324+0000 7fd943e00640 1 -- 172.21.15.114:0/940657559 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440cd790 con 0x5596bbfa6660 2024-09-21T07:08:52.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.324+0000 7fd943e00640 1 -- 172.21.15.114:0/940657559 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9440cd3f0 con 0x5596bbfa6660 2024-09-21T07:08:52.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 -- 172.21.15.114:0/940657559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 msgr2=0x5596bc38dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc38dbc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd9440ca250 tx=0x7fd94402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 -- 172.21.15.114:0/940657559 shutdown_connections 2024-09-21T07:08:52.328 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc37f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfae840 0x5596bc37eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 --2- 172.21.15.114:0/940657559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfa6660 0x5596bc38dbc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 -- 172.21.15.114:0/940657559 >> 172.21.15.114:0/940657559 conn(0x5596bbfc3910 msgr2=0x5596bc380430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 -- 172.21.15.114:0/940657559 shutdown_connections 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.325+0000 7fd961895740 1 -- 172.21.15.114:0/940657559 wait complete. 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.326+0000 7fd961895740 1 -- start start 2024-09-21T07:08:52.329 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.326+0000 7fd961895740 1 Processor -- start 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.326+0000 7fd961895740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc386650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.326+0000 7fd961895740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfae840 0x5596bc386c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.326+0000 7fd961895740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc3f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd961895740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541b4550 con 0x5596bbfcf520 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd961895740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541b3df0 con 0x5596bbfae840 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd961895740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541b1050 con 0x5596bbfa6660 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd961895740 10 client.0 _ll_register_callbacks cb 0x7fd94410b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd958c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc3f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc386650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfae840 0x5596bc386c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd958c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc3f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46282/0 (socket says 172.21.15.114:46282) 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd958c00640 1 -- 172.21.15.114:0/849261908 learned_addr learned my addr 172.21.15.114:0/849261908 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953400640 1 -- 172.21.15.114:0/849261908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 msgr2=0x5596bc386650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953400640 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc386650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953400640 1 -- 172.21.15.114:0/849261908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 msgr2=0x5596bc3f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953400640 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc3f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.331 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.327+0000 7fd953400640 1 -- 172.21.15.114:0/849261908 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596bc3f7df0 con 0x5596bbfae840 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd953400640 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfae840 0x5596bc386c10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x5596bc36f6c0 tx=0x7fd93c004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed5a1705b5ef5f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd941600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd941600640 1 -- 172.21.15.114:0/849261908 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c01e040 con 0x5596bbfae840 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd941600640 1 -- 172.21.15.114:0/849261908 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fd93c004250 con 0x5596bbfae840 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9440cd790 con 0x5596bbfae840 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd941600640 1 -- 172.21.15.114:0/849261908 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd93c039050 con 0x5596bbfae840 2024-09-21T07:08:52.332 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd961895740 10 client.29111 fetch_fsmap learned FSMap version 57 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.328+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5596bc3ff220 con 0x5596bbfae840 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd941600640 1 -- 172.21.15.114:0/849261908 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440cd790 con 0x5596bbfae840 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd941600640 1 -- 172.21.15.114:0/849261908 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fd93c024040 con 0x5596bbfae840 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd961895740 10 client.29111 fetch_fsmap finished waiting for FSMap version 57 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd961895740 10 client.29111 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd961895740 20 client.29111 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd961895740 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf63250 0x5596bc401760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd961895740 4 client.29111 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "38fafa95", "release": true, "await": true}] 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.329+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "38fafa95", "release": true, "await": true}) -- 0x5596bc2eac90 con 0x5596bbf63250 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.330+0000 7fd953e00640 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf63250 0x5596bc401760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.330+0000 7fd953e00640 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf63250 0x5596bc401760 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125e0aedeec7474a server_cookie=d98a1c570e7025c7 in_seq=0 out_seq=0 2024-09-21T07:08:52.333 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.330+0000 7fd941600640 10 client.29111 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:52.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd941600640 1 -- 172.21.15.114:0/849261908 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5596bc2eac90 con 0x5596bbf63250 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd941600640 10 client.29111 handle_command_reply: tid=0 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 client.29111 shutdown 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfae840 msgr2=0x5596bc386c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfae840 0x5596bc386c10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x5596bc36f6c0 tx=0x7fd93c004950 comp rx=0 tx=0).stop 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 shutdown_connections 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5596bbfcf520 0x5596bc3f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596bbfae840 0x5596bc386c10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5596bbf63250 0x5596bc401760 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.355+0000 7fd961895740 1 --2- 172.21.15.114:0/849261908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5596bbfa6660 0x5596bc386650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.356+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 >> 172.21.15.114:0/849261908 conn(0x5596bbfc3910 msgr2=0x5596bc3771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:52.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.356+0000 7fd941600640 0 client.29111 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:52.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.357+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 shutdown_connections 2024-09-21T07:08:52.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.357+0000 7fd961895740 1 -- 172.21.15.114:0/849261908 wait complete. 2024-09-21T07:08:52.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.357+0000 7fd961895740 20 client.29111 trim_cache size 0 max 16384 2024-09-21T07:08:52.361 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:08:52.361 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 57, 2024-09-21T07:08:52.361 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:08:52.361 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 56, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "38fafa95": { 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 56, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 138.8, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:08:52.362 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:52.363 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:08:52.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.360+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9380b6700 msgr2=0x7fd9380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.360+0000 7fd95ac00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9380b6700 0x7fd9380b8c10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009fd0 tx=0x7fd94c009500 comp rx=0 tx=0).stop 2024-09-21T07:08:52.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.360+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 msgr2=0x7fd954075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:52.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.360+0000 7fd95ac00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 0x7fd954075710 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd944005ab0 tx=0x7fd944003c10 comp rx=0 tx=0).stop 2024-09-21T07:08:52.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.361+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 shutdown_connections 2024-09-21T07:08:52.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.362+0000 7fd95ac00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd9380b6700 0x7fd9380b8c10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.362+0000 7fd95ac00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd954109e00 0x7fd954076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.362+0000 7fd95ac00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd9541094b0 0x7fd954075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.362+0000 7fd95ac00640 1 --2- 172.21.15.114:0/3890048698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd954103b70 0x7fd954075710 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:52.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.362+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 >> 172.21.15.114:0/3890048698 conn(0x7fd9540ff5e0 msgr2=0x7fd954101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:52.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.366+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 shutdown_connections 2024-09-21T07:08:52.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:52.369+0000 7fd95ac00640 1 -- 172.21.15.114:0/3890048698 wait complete. 2024-09-21T07:08:52.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 855.0 s, 69.3 tps, lat 3408.075 ms stddev 5561.302 2024-09-21T07:08:53.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:53 smithi114 ceph-mon[30654]: pgmap v852: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 779 KiB/s wr, 101 op/s 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 -- 172.21.15.114:0/582786394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711809ba50 msgr2=0x7f711809be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 --2- 172.21.15.114:0/582786394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711809ba50 0x7f711809be60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f710c009860 tx=0x7f710c02a360 comp rx=0 tx=0).stop 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 -- 172.21.15.114:0/582786394 shutdown_connections 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 --2- 172.21.15.114:0/582786394 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 0x7f71180a2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 --2- 172.21.15.114:0/582786394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711809ba50 0x7f711809be60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 --2- 172.21.15.114:0/582786394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7118096110 0x7f7118096520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.345+0000 7f7126200640 1 -- 172.21.15.114:0/582786394 >> 172.21.15.114:0/582786394 conn(0x7f7118091b60 msgr2=0x7f7118093fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.347+0000 7f7126200640 1 -- 172.21.15.114:0/582786394 shutdown_connections 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.349+0000 7f7126200640 1 -- 172.21.15.114:0/582786394 wait complete. 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.350+0000 7f7126200640 1 Processor -- start 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.351+0000 7f7126200640 1 -- start start 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.352+0000 7f7126200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 0x7f711809f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.352+0000 7f7126200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711809f5c0 0x7f711809f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.352+0000 7f7126200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 0x7f711813f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.352+0000 7f7126200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f71180074e0 con 0x7f711809c3a0 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.353+0000 7f7125800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 0x7f711813f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.353+0000 7f7126200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7118007360 con 0x7f711813d110 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.353+0000 7f7126200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7118007660 con 0x7f711809f5c0 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.353+0000 7f7125800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 0x7f711813f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37910/0 (socket says 172.21.15.114:37910) 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.353+0000 7f7125800640 1 -- 172.21.15.114:0/496551848 learned_addr learned my addr 172.21.15.114:0/496551848 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.354+0000 7f7125800640 1 -- 172.21.15.114:0/496551848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711809f5c0 msgr2=0x7f711809f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.354+0000 7f7125800640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711809f5c0 0x7f711809f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.354+0000 7f7125800640 1 -- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 msgr2=0x7f711809f080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.354+0000 7f7124e00640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 0x7f711809f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.357+0000 7f7125800640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 0x7f711809f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.357+0000 7f7125800640 1 -- 172.21.15.114:0/496551848 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711813fc30 con 0x7f711813d110 2024-09-21T07:08:53.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.358+0000 7f7125800640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 0x7f711813f530 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7120051d10 tx=0x7f712006b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.359+0000 7f711d600640 1 -- 172.21.15.114:0/496551848 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120084070 con 0x7f711813d110 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.359+0000 7f711d600640 1 -- 172.21.15.114:0/496551848 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7120080ae0 con 0x7f711813d110 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.359+0000 7f711d600640 1 -- 172.21.15.114:0/496551848 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712007fb20 con 0x7f711813d110 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.360+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711813ff20 con 0x7f711813d110 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.360+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7118140380 con 0x7f711813d110 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.360+0000 7f7124e00640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 0x7f711809f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:53.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.368+0000 7f712e5c4740 1 Processor -- start 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.370+0000 7f712e5c4740 1 -- start start 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f711d600640 1 -- 172.21.15.114:0/496551848 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7120087070 con 0x7f711813d110 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bfd070 0x55c352fa3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bd41b0 0x55c352fa42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fa47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f712e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c352f1afb0 con 0x55c352bfd070 2024-09-21T07:08:53.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f712e5c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c352f0a3e0 con 0x55c352bdc390 2024-09-21T07:08:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.371+0000 7f712e5c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c352f0e1f0 con 0x55c352bd41b0 2024-09-21T07:08:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.373+0000 7f7124e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bd41b0 0x55c352fa42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.373+0000 7f711fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bfd070 0x55c352fa3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.373+0000 7f711fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bfd070 0x55c352fa3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:46300/0 (socket says 172.21.15.114:46300) 2024-09-21T07:08:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.373+0000 7f711fe00640 1 -- 172.21.15.114:0/874933617 learned_addr learned my addr 172.21.15.114:0/874933617 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:53.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.374+0000 7f711d600640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71080b6780 0x7f71080b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.374+0000 7f711fe00640 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fa47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.374+0000 7f711d600640 1 -- 172.21.15.114:0/496551848 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f7120155e10 con 0x7f711813d110 2024-09-21T07:08:53.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.375+0000 7f7124e00640 1 -- 172.21.15.114:0/874933617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 msgr2=0x55c352fa47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.375+0000 7f7124e00640 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fa47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.375+0000 7f7124e00640 1 -- 172.21.15.114:0/874933617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bfd070 msgr2=0x55c352fa3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.375+0000 7f7124e00640 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bfd070 0x55c352fa3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.375+0000 7f7124e00640 1 -- 172.21.15.114:0/874933617 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c352ed4cd0 con 0x55c352bd41b0 2024-09-21T07:08:53.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.376+0000 7f7124e00640 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bd41b0 0x55c352fa42a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f71180a2af0 tx=0x7f711400ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b02e4ea69fbe14c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:53.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.376+0000 7f7112000640 1 -- 172.21.15.114:0/874933617 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114015070 con 0x55c352bd41b0 2024-09-21T07:08:53.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.376+0000 7f7112000640 1 -- 172.21.15.114:0/874933617 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71140113f0 con 0x55c352bd41b0 2024-09-21T07:08:53.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.376+0000 7f7112000640 1 -- 172.21.15.114:0/874933617 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71140185c0 con 0x55c352bd41b0 2024-09-21T07:08:53.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.377+0000 7f712e5c4740 1 -- 172.21.15.114:0/874933617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bd41b0 msgr2=0x55c352fa42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.380 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.377+0000 7f712e5c4740 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bd41b0 0x55c352fa42a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f71180a2af0 tx=0x7f711400ad30 comp rx=0 tx=0).stop 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 -- 172.21.15.114:0/874933617 shutdown_connections 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fa47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bd41b0 0x55c352fa42a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 --2- 172.21.15.114:0/874933617 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bfd070 0x55c352fa3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 -- 172.21.15.114:0/874933617 >> 172.21.15.114:0/874933617 conn(0x55c352bf1460 msgr2=0x55c352fa1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 -- 172.21.15.114:0/874933617 shutdown_connections 2024-09-21T07:08:53.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.378+0000 7f712e5c4740 1 -- 172.21.15.114:0/874933617 wait complete. 2024-09-21T07:08:53.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.379+0000 7f7125800640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71080b6780 0x7f71080b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.379+0000 7f712e5c4740 1 -- start start 2024-09-21T07:08:53.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.379+0000 7f712e5c4740 1 Processor -- start 2024-09-21T07:08:53.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f7125800640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71080b6780 0x7f71080b8c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7120052860 tx=0x7f7120069e10 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:53.386 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 0x55c352faf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bdc390 0x55c352fafe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fcc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f712e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c352d6b700 con 0x55c352bdc390 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f712e5c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c352ea5580 con 0x55c352bd41b0 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.380+0000 7f712e5c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c352fc3550 con 0x55c352bfd070 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.381+0000 7f712e5c4740 10 client.0 _ll_register_callbacks cb 0x55c352eb5470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.381+0000 7f711fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fcc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 0x55c352faf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 0x55c352faf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37928/0 (socket says 172.21.15.114:37928) 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 -- 172.21.15.114:0/3413179622 learned_addr learned my addr 172.21.15.114:0/3413179622 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 -- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 msgr2=0x55c352fcc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fcc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 -- 172.21.15.114:0/3413179622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bdc390 msgr2=0x55c352fafe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bdc390 0x55c352fafe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.382+0000 7f711fe00640 1 -- 172.21.15.114:0/3413179622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c352c68650 con 0x55c352bd41b0 2024-09-21T07:08:53.389 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.386+0000 7f711fe00640 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 0x55c352faf8f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f71180058d0 tx=0x7f710c005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126ae99f88747b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:53.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.389+0000 7f7106a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:53.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.389+0000 7f7106a00640 1 -- 172.21.15.114:0/3413179622 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c037030 con 0x55c352bd41b0 2024-09-21T07:08:53.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.391+0000 7f7106a00640 1 -- 172.21.15.114:0/3413179622 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f710c003e40 con 0x55c352bd41b0 2024-09-21T07:08:53.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.392+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c352ea7b90 con 0x55c352bd41b0 2024-09-21T07:08:53.396 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.394+0000 7f7106a00640 1 -- 172.21.15.114:0/3413179622 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f710c037030 con 0x55c352bd41b0 2024-09-21T07:08:53.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.394+0000 7f712e5c4740 10 client.29123 fetch_fsmap learned FSMap version 58 2024-09-21T07:08:53.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.394+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c352ec2950 con 0x55c352bd41b0 2024-09-21T07:08:53.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.395+0000 7f7106a00640 1 -- 172.21.15.114:0/3413179622 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c352ea7b90 con 0x55c352bd41b0 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.395+0000 7f7106a00640 1 -- 172.21.15.114:0/3413179622 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 58) ==== 8378+0+0 (secure 0 0 0) 0x7f710c0306d0 con 0x55c352bd41b0 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.397+0000 7f712e5c4740 10 client.29123 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.397+0000 7f712e5c4740 10 client.29123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.397+0000 7f712e5c4740 20 client.29123 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.397+0000 7f712e5c4740 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352bbb9f0 0x55c352fb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.397+0000 7f712e5c4740 4 client.29123 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:53.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.397+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c352f1cf00 con 0x55c352bbb9f0 2024-09-21T07:08:53.401 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.398+0000 7f7124e00640 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352bbb9f0 0x55c352fb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.401+0000 7f7106a00640 10 client.29123 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:08:53.403 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.401+0000 7f7124e00640 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352bbb9f0 0x55c352fb54b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd8a0336698ca0dd server_cookie=8f3d69f554fb9ff0 in_seq=0 out_seq=0 2024-09-21T07:08:53.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.401+0000 7f7106a00640 1 -- 172.21.15.114:0/3413179622 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c352f1cf00 con 0x55c352bbb9f0 2024-09-21T07:08:53.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.401+0000 7f7106a00640 10 client.29123 handle_command_reply: tid=0 2024-09-21T07:08:53.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 client.29123 shutdown 2024-09-21T07:08:53.404 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 msgr2=0x55c352faf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 0x55c352faf8f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f71180058d0 tx=0x7f710c005760 comp rx=0 tx=0).stop 2024-09-21T07:08:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 shutdown_connections 2024-09-21T07:08:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352bbb9f0 0x55c352fb54b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fcc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bdc390 0x55c352fafe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.405 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 --2- 172.21.15.114:0/3413179622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bd41b0 0x55c352faf8f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 >> 172.21.15.114:0/3413179622 conn(0x55c352bf1460 msgr2=0x55c352fa4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.402+0000 7f7106a00640 0 client.29123 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:08:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.403+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 shutdown_connections 2024-09-21T07:08:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.403+0000 7f712e5c4740 1 -- 172.21.15.114:0/3413179622 wait complete. 2024-09-21T07:08:53.406 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.403+0000 7f712e5c4740 20 client.29123 trim_cache size 0 max 16384 2024-09-21T07:08:53.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.430+0000 7f712e5c4740 1 Processor -- start 2024-09-21T07:08:53.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 -- start start 2024-09-21T07:08:53.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fa1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fa5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f711814a6a0 con 0x55c352bd41b0 2024-09-21T07:08:53.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f711814c240 con 0x55c352bdc390 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.431+0000 7f712e5c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f711814ad70 con 0x55c352bfd070 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.432+0000 7f7124e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.432+0000 7f7124e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37936/0 (socket says 172.21.15.114:37936) 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.432+0000 7f7124e00640 1 -- 172.21.15.114:0/2456747907 learned_addr learned my addr 172.21.15.114:0/2456747907 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.432+0000 7f711fe00640 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fa5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f7124e00640 1 -- 172.21.15.114:0/2456747907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 msgr2=0x55c352fa5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f7124e00640 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fa5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f7124e00640 1 -- 172.21.15.114:0/2456747907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 msgr2=0x55c352fa1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f711fe00640 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fa1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f7124e00640 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fa1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f7124e00640 1 -- 172.21.15.114:0/2456747907 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c352fb09b0 con 0x55c352bdc390 2024-09-21T07:08:53.436 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.433+0000 7f711fe00640 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fa1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:53.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.436+0000 7f7124e00640 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc2b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7114011d50 tx=0x7f7114008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40ca46583cb424f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:53.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.436+0000 7f7111600640 1 -- 172.21.15.114:0/2456747907 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114015040 con 0x55c352bdc390 2024-09-21T07:08:53.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.436+0000 7f7111600640 1 -- 172.21.15.114:0/2456747907 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711400a150 con 0x55c352bdc390 2024-09-21T07:08:53.440 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.436+0000 7f7111600640 1 -- 172.21.15.114:0/2456747907 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114011700 con 0x55c352bdc390 2024-09-21T07:08:53.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.438+0000 7f712e5c4740 1 -- 172.21.15.114:0/2456747907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 msgr2=0x55c352fc2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.438+0000 7f712e5c4740 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc2b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7114011d50 tx=0x7f7114008040 comp rx=0 tx=0).stop 2024-09-21T07:08:53.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 -- 172.21.15.114:0/2456747907 shutdown_connections 2024-09-21T07:08:53.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fa5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc2b50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 --2- 172.21.15.114:0/2456747907 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fa1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 -- 172.21.15.114:0/2456747907 >> 172.21.15.114:0/2456747907 conn(0x55c352bf1460 msgr2=0x55c352fb0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:53.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 -- 172.21.15.114:0/2456747907 shutdown_connections 2024-09-21T07:08:53.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.440+0000 7f712e5c4740 1 -- 172.21.15.114:0/2456747907 wait complete. 2024-09-21T07:08:53.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.441+0000 7f712e5c4740 1 -- start start 2024-09-21T07:08:53.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.441+0000 7f712e5c4740 1 Processor -- start 2024-09-21T07:08:53.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fc8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.446 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fb9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f711814bb70 con 0x55c352bd41b0 2024-09-21T07:08:53.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f711814b440 con 0x55c352bdc390 2024-09-21T07:08:53.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71181486a0 con 0x55c352bfd070 2024-09-21T07:08:53.447 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.443+0000 7f712e5c4740 10 client.0 _ll_register_callbacks cb 0x7f710c003ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.444+0000 7f7124e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.444+0000 7f7124e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:37946/0 (socket says 172.21.15.114:37946) 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.444+0000 7f7124e00640 1 -- 172.21.15.114:0/520193593 learned_addr learned my addr 172.21.15.114:0/520193593 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.444+0000 7f711fe00640 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fb9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.445+0000 7f7124e00640 1 -- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 msgr2=0x55c352fb9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.445+0000 7f7124e00640 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fb9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.445+0000 7f7124e00640 1 -- 172.21.15.114:0/520193593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 msgr2=0x55c352fc8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.445+0000 7f7124e00640 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fc8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.445+0000 7f7124e00640 1 -- 172.21.15.114:0/520193593 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c352fc9320 con 0x55c352bdc390 2024-09-21T07:08:53.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.446+0000 7f7106a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:53.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.446+0000 7f7124e00640 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc8d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f711400ad70 tx=0x7f7114006ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d567451b8aec8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:53.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.447+0000 7f7106a00640 1 -- 172.21.15.114:0/520193593 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114015040 con 0x55c352bdc390 2024-09-21T07:08:53.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.447+0000 7f7106a00640 1 -- 172.21.15.114:0/520193593 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f7114011460 con 0x55c352bdc390 2024-09-21T07:08:53.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.447+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711400a150 con 0x55c352bdc390 2024-09-21T07:08:53.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.448+0000 7f7106a00640 1 -- 172.21.15.114:0/520193593 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7114011d80 con 0x55c352bdc390 2024-09-21T07:08:53.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.449+0000 7f712e5c4740 10 client.29129 fetch_fsmap learned FSMap version 58 2024-09-21T07:08:53.451 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.449+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c35302c9c0 con 0x55c352bdc390 2024-09-21T07:08:53.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f7106a00640 1 -- 172.21.15.114:0/520193593 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711400a150 con 0x55c352bdc390 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f7106a00640 1 -- 172.21.15.114:0/520193593 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 58) ==== 8378+0+0 (secure 0 0 0) 0x7f7114004370 con 0x55c352bdc390 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f712e5c4740 10 client.29129 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f712e5c4740 10 client.29129 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f712e5c4740 20 client.29129 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f712e5c4740 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352b90da0 0x55c35302ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f712e5c4740 4 client.29129 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c352f189b0 con 0x55c352b90da0 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.450+0000 7f711fe00640 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352b90da0 0x55c35302ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.452+0000 7f711fe00640 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352b90da0 0x55c35302ee10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80e2258cfa980fdb server_cookie=1b33a14ec8786b64 in_seq=0 out_seq=0 2024-09-21T07:08:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.452+0000 7f7106a00640 10 client.29129 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:08:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.452+0000 7f7106a00640 1 -- 172.21.15.114:0/520193593 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c352f189b0 con 0x55c352b90da0 2024-09-21T07:08:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.453+0000 7f7106a00640 10 client.29129 handle_command_reply: tid=0 2024-09-21T07:08:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.453+0000 7f712e5c4740 1 client.29129 shutdown 2024-09-21T07:08:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.453+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 msgr2=0x55c352fc8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.453+0000 7f712e5c4740 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc8d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f711400ad70 tx=0x7f7114006ec0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.453+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 shutdown_connections 2024-09-21T07:08:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.453+0000 7f712e5c4740 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55c352b90da0 0x55c35302ee10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.454+0000 7f712e5c4740 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c352bfd070 0x55c352fb9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.454+0000 7f712e5c4740 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c352bdc390 0x55c352fc8d60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.454+0000 7f712e5c4740 1 --2- 172.21.15.114:0/520193593 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c352bd41b0 0x55c352fc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.454+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 >> 172.21.15.114:0/520193593 conn(0x55c352bf1460 msgr2=0x55c352fa1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.455+0000 7f7106a00640 0 client.29129 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:08:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.455+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 shutdown_connections 2024-09-21T07:08:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.456+0000 7f712e5c4740 1 -- 172.21.15.114:0/520193593 wait complete. 2024-09-21T07:08:53.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.456+0000 7f712e5c4740 20 client.29129 trim_cache size 0 max 16384 2024-09-21T07:08:53.459 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.457+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71080b6780 msgr2=0x7f71080b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.457+0000 7f7126200640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71080b6780 0x7f71080b8c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7120052860 tx=0x7f7120069e10 comp rx=0 tx=0).stop 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.457+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 msgr2=0x7f711813f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.457+0000 7f7126200640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 0x7f711813f530 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7120051d10 tx=0x7f712006b9b0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.458+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 shutdown_connections 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.458+0000 7f7126200640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f71080b6780 0x7f71080b8c90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.458+0000 7f7126200640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711813d110 0x7f711813f530 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.458+0000 7f7126200640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f711809f5c0 0x7f711809f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.458+0000 7f7126200640 1 --2- 172.21.15.114:0/496551848 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f711809c3a0 0x7f711809f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.458+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 >> 172.21.15.114:0/496551848 conn(0x7f7118091b60 msgr2=0x7f7118007ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.460+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 shutdown_connections 2024-09-21T07:08:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:53.460+0000 7f7126200640 1 -- 172.21.15.114:0/496551848 wait complete. 2024-09-21T07:08:53.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:53 smithi183 ceph-mon[34219]: pgmap v852: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 779 KiB/s wr, 101 op/s 2024-09-21T07:08:53.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:53 smithi193 ceph-mon[34601]: pgmap v852: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 779 KiB/s wr, 101 op/s 2024-09-21T07:08:53.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '38fafa95', total seconds elapsed: 24.6 2024-09-21T07:08:53.772 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-21T07:08:53.772 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:08:54.549 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:54 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:54.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:54 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:54.662 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:08:54.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:54 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:54.795 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:55.545 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:55 smithi114 ceph-mon[30654]: pgmap v853: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 448 KiB/s wr, 79 op/s 2024-09-21T07:08:55.683 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:55.686 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.683+0000 7fdbba400640 1 -- 172.21.15.114:0/2013754734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4103b70 msgr2=0x7fdbb4103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:55.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.683+0000 7fdbba400640 1 --2- 172.21.15.114:0/2013754734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4103b70 0x7fdbb4103f80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009900 tx=0x7fdba402a440 comp rx=0 tx=0).stop 2024-09-21T07:08:55.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 -- 172.21.15.114:0/2013754734 shutdown_connections 2024-09-21T07:08:55.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 --2- 172.21.15.114:0/2013754734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4109e00 0x7fdbb4110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:55.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 --2- 172.21.15.114:0/2013754734 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41094b0 0x7fdbb41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:55.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 --2- 172.21.15.114:0/2013754734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4103b70 0x7fdbb4103f80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:55.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 -- 172.21.15.114:0/2013754734 >> 172.21.15.114:0/2013754734 conn(0x7fdbb40ff5e0 msgr2=0x7fdbb4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:55.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 -- 172.21.15.114:0/2013754734 shutdown_connections 2024-09-21T07:08:55.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.685+0000 7fdbba400640 1 -- 172.21.15.114:0/2013754734 wait complete. 2024-09-21T07:08:55.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.686+0000 7fdbba400640 1 Processor -- start 2024-09-21T07:08:55.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.687+0000 7fdbba400640 1 -- start start 2024-09-21T07:08:55.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.688+0000 7fdbba400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 0x7fdbb419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:55.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.688+0000 7fdbba400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41094b0 0x7fdbb419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:55.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.688+0000 7fdbba400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4109e00 0x7fdbb419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:55.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.688+0000 7fdbba400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4113b50 con 0x7fdbb41094b0 2024-09-21T07:08:55.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.688+0000 7fdbba400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41139d0 con 0x7fdbb4103b70 2024-09-21T07:08:55.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.688+0000 7fdbba400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4113cd0 con 0x7fdbb4109e00 2024-09-21T07:08:55.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.689+0000 7fdbb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 0x7fdbb419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:55.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.689+0000 7fdbb2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41094b0 0x7fdbb419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.689+0000 7fdbb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 0x7fdbb419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54088/0 (socket says 172.21.15.114:54088) 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.689+0000 7fdbb3400640 1 -- 172.21.15.114:0/58774325 learned_addr learned my addr 172.21.15.114:0/58774325 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.690+0000 7fdbb3400640 1 -- 172.21.15.114:0/58774325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4109e00 msgr2=0x7fdbb419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.690+0000 7fdbb3400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4109e00 0x7fdbb419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.690+0000 7fdbb3400640 1 -- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41094b0 msgr2=0x7fdbb419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.690+0000 7fdbb3400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41094b0 0x7fdbb419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:55.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.690+0000 7fdbb3400640 1 -- 172.21.15.114:0/58774325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb41a0450 con 0x7fdbb4103b70 2024-09-21T07:08:55.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.692+0000 7fdbb3400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 0x7fdbb419a4e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdba4004770 tx=0x7fdba40044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.694+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4035030 con 0x7fdbb4103b70 2024-09-21T07:08:55.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.694+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba4041070 con 0x7fdbb4103b70 2024-09-21T07:08:55.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.694+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4003cb0 con 0x7fdbb4103b70 2024-09-21T07:08:55.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.695+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb41a06e0 con 0x7fdbb4103b70 2024-09-21T07:08:55.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.695+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb40755f0 con 0x7fdbb4103b70 2024-09-21T07:08:55.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.697+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb4104df0 con 0x7fdbb4103b70 2024-09-21T07:08:55.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.703+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fdba403d760 con 0x7fdbb4103b70 2024-09-21T07:08:55.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.710+0000 7fdbabe00640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdb980b67d0 0x7fdb980b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:55.713 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.710+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fdba410c4a0 con 0x7fdbb4103b70 2024-09-21T07:08:55.716 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.713+0000 7fdbb2a00640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdb980b67d0 0x7fdb980b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:55.717 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.715+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdba40d26c0 con 0x7fdbb4103b70 2024-09-21T07:08:55.722 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:55.719+0000 7fdbb2a00640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdb980b67d0 0x7fdb980b8ce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4108f00 tx=0x7fdba0008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:55.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:55 smithi183 ceph-mon[34219]: pgmap v853: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 448 KiB/s wr, 79 op/s 2024-09-21T07:08:55.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:55 smithi193 ceph-mon[34601]: pgmap v853: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 448 KiB/s wr, 79 op/s 2024-09-21T07:08:56.180 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.177+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb410bc30 con 0x7fdbb4103b70 2024-09-21T07:08:56.184 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.181+0000 7fdbabe00640 1 -- 172.21.15.114:0/58774325 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11992 (secure 0 0 0) 0x7fdba40d9e20 con 0x7fdbb4103b70 2024-09-21T07:08:56.184 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:08:56.186 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 58 2024-09-21T07:08:56.186 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":58,"btime":"2024-09-21T07:08:53:258625+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:08:52.945721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:08:56.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdb980b67d0 msgr2=0x7fdb980b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdb980b67d0 0x7fdb980b8ce0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4108f00 tx=0x7fdba0008040 comp rx=0 tx=0).stop 2024-09-21T07:08:56.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 msgr2=0x7fdbb419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 0x7fdbb419a4e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdba4004770 tx=0x7fdba40044f0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 shutdown_connections 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbb4109e00 0x7fdbb419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fdbb41094b0 0x7fdbb419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fdb980b67d0 0x7fdb980b8ce0 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4108f00 tx=0x7fdba0008040 comp rx=0 tx=0).stop 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 --2- 172.21.15.114:0/58774325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb4103b70 0x7fdbb419a4e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 >> 172.21.15.114:0/58774325 conn(0x7fdbb40ff5e0 msgr2=0x7fdbb4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 shutdown_connections 2024-09-21T07:08:56.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.187+0000 7fdbba400640 1 -- 172.21.15.114:0/58774325 wait complete. 2024-09-21T07:08:56.681 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:56 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/58774325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:56.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.582+0000 7fa6a1200640 1 -- 172.21.15.114:0/809487973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c1047c0 msgr2=0x7fa69c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.687 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.582+0000 7fa6a1200640 1 --2- 172.21.15.114:0/809487973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c1047c0 0x7fa69c104bd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009900 tx=0x7fa68c02a440 comp rx=0 tx=0).stop 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 -- 172.21.15.114:0/809487973 shutdown_connections 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 --2- 172.21.15.114:0/809487973 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c10dc20 0x7fa69c110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 --2- 172.21.15.114:0/809487973 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 --2- 172.21.15.114:0/809487973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c1047c0 0x7fa69c104bd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 -- 172.21.15.114:0/809487973 >> 172.21.15.114:0/809487973 conn(0x7fa69c0fe2c0 msgr2=0x7fa69c100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 -- 172.21.15.114:0/809487973 shutdown_connections 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.584+0000 7fa6a1200640 1 -- 172.21.15.114:0/809487973 wait complete. 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.586+0000 7fa6a1200640 1 Processor -- start 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.586+0000 7fa6a1200640 1 -- start start 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.587+0000 7fa6a1200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c1047c0 0x7fa69c1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.587+0000 7fa6a1200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.688 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.587+0000 7fa6a1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c10dc20 0x7fa69c1a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.587+0000 7fa6a1200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c119f50 con 0x7fa69c1047c0 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.587+0000 7fa6a1200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c119dd0 con 0x7fa69c10dc20 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.587+0000 7fa6a1200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa69c11a0d0 con 0x7fa69c105110 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.588+0000 7fa69b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c1047c0 0x7fa69c1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.588+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.588+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c1a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59680/0 (socket says 172.21.15.114:59680) 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.588+0000 7fa69aa00640 1 -- 172.21.15.114:0/3792271989 learned_addr learned my addr 172.21.15.114:0/3792271989 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.588+0000 7fa69be00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c10dc20 0x7fa69c1a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69be00640 1 -- 172.21.15.114:0/3792271989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 msgr2=0x7fa69c1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69be00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69be00640 1 -- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c1047c0 msgr2=0x7fa69c1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69be00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c1047c0 0x7fa69c1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69be00640 1 -- 172.21.15.114:0/3792271989 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69c1a6970 con 0x7fa69c10dc20 2024-09-21T07:08:56.689 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69aa00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.589+0000 7fa69b400640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c1047c0 0x7fa69c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.590+0000 7fa69be00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c10dc20 0x7fa69c1a6240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa694009fd0 tx=0x7fa69400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.595+0000 7fa693e00640 1 -- 172.21.15.114:0/3792271989 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400ce20 con 0x7fa69c10dc20 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.595+0000 7fa693e00640 1 -- 172.21.15.114:0/3792271989 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa694005020 con 0x7fa69c10dc20 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.595+0000 7fa693e00640 1 -- 172.21.15.114:0/3792271989 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694010690 con 0x7fa69c10dc20 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.595+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69c1a6c60 con 0x7fa69c10dc20 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.595+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69c1ad3c0 con 0x7fa69c10dc20 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.605+0000 7fa6a946a740 1 Processor -- start 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.607+0000 7fa693e00640 1 -- 172.21.15.114:0/3792271989 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa694004a40 con 0x7fa69c10dc20 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 -- start start 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0e1070 0x55fcce487d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 0x55fcce488270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.690 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0c0390 0x55fcce4887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3fefb0 con 0x55fcce0e1070 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3ee3e0 con 0x55fcce0c0390 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.611+0000 7fa6a946a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcce3f21f0 con 0x55fcce0b81b0 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.612+0000 7fa69b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 0x55fcce488270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.612+0000 7fa69b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 0x55fcce488270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59690/0 (socket says 172.21.15.114:59690) 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.612+0000 7fa69b400640 1 -- 172.21.15.114:0/3081811520 learned_addr learned my addr 172.21.15.114:0/3081811520 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.612+0000 7fa69aa00640 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0c0390 0x55fcce4887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa69b400640 1 -- 172.21.15.114:0/3081811520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0c0390 msgr2=0x55fcce4887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa69b400640 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0c0390 0x55fcce4887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.691 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa69b400640 1 -- 172.21.15.114:0/3081811520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0e1070 msgr2=0x55fcce487d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa69b400640 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0e1070 0x55fcce487d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa69b400640 1 -- 172.21.15.114:0/3081811520 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce3b8cd0 con 0x55fcce0b81b0 2024-09-21T07:08:56.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa69aa00640 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0c0390 0x55fcce4887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:56.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa693e00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6800b6840 0x7fa6800b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.613+0000 7fa693e00640 1 -- 172.21.15.114:0/3792271989 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fa6940e7f80 con 0x7fa69c10dc20 2024-09-21T07:08:56.692 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.614+0000 7fa69b400640 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 0x55fcce488270 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009900 tx=0x7fa68c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf62798d56adf09e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.614+0000 7fa69aa00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6800b6840 0x7fa6800b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.614+0000 7fa690c00640 1 -- 172.21.15.114:0/3081811520 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c035040 con 0x55fcce0b81b0 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.614+0000 7fa690c00640 1 -- 172.21.15.114:0/3081811520 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa68c004bd0 con 0x55fcce0b81b0 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.614+0000 7fa690c00640 1 -- 172.21.15.114:0/3081811520 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68c003b20 con 0x55fcce0b81b0 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.615+0000 7fa6a946a740 1 -- 172.21.15.114:0/3081811520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 msgr2=0x55fcce488270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.615+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 0x55fcce488270 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009900 tx=0x7fa68c004610 comp rx=0 tx=0).stop 2024-09-21T07:08:56.693 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.616+0000 7fa6a946a740 1 -- 172.21.15.114:0/3081811520 shutdown_connections 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.616+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0c0390 0x55fcce4887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.616+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0b81b0 0x55fcce488270 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.616+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3081811520 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0e1070 0x55fcce487d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.616+0000 7fa6a946a740 1 -- 172.21.15.114:0/3081811520 >> 172.21.15.114:0/3081811520 conn(0x55fcce0d5460 msgr2=0x55fcce485bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.617+0000 7fa6a946a740 1 -- 172.21.15.114:0/3081811520 shutdown_connections 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.617+0000 7fa6a946a740 1 -- 172.21.15.114:0/3081811520 wait complete. 2024-09-21T07:08:56.694 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.617+0000 7fa6a946a740 1 -- start start 2024-09-21T07:08:56.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.619+0000 7fa6a946a740 1 Processor -- start 2024-09-21T07:08:56.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.621+0000 7fa69aa00640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6800b6840 0x7fa6800b8d50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa688006e70 tx=0x7fa6880047d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:56.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce493900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 0x55fcce493e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.695 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55fcce24f700 con 0x55fcce0c0390 2024-09-21T07:08:56.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fcce389580 con 0x55fcce0b81b0 2024-09-21T07:08:56.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fcce4a74c0 con 0x55fcce0e1070 2024-09-21T07:08:56.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.622+0000 7fa6a946a740 10 client.0 _ll_register_callbacks cb 0x55fcce399470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:56.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.623+0000 7fa69b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce493900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.623+0000 7fa69b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce493900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54134/0 (socket says 172.21.15.114:54134) 2024-09-21T07:08:56.696 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.623+0000 7fa69b400640 1 -- 172.21.15.114:0/3058617875 learned_addr learned my addr 172.21.15.114:0/3058617875 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.623+0000 7fa69be00640 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.624+0000 7fa69be00640 1 -- 172.21.15.114:0/3058617875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 msgr2=0x55fcce493900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.624+0000 7fa69be00640 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce493900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.624+0000 7fa69be00640 1 -- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 msgr2=0x55fcce493e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.624+0000 7fa69be00640 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 0x55fcce493e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.624+0000 7fa69be00640 1 -- 172.21.15.114:0/3058617875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce14c650 con 0x55fcce0e1070 2024-09-21T07:08:56.697 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.626+0000 7fa69be00640 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4b0840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa6940e52a0 tx=0x7fa694004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a383fe594f51c5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.626+0000 7fa685600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.626+0000 7fa685600640 1 -- 172.21.15.114:0/3058617875 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940ace00 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.626+0000 7fa685600640 1 -- 172.21.15.114:0/3058617875 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fa6940a92f0 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.628+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fcce38bb90 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.629+0000 7fa685600640 1 -- 172.21.15.114:0/3058617875 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa6940ae0a0 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.629+0000 7fa6a946a740 10 client.29089 fetch_fsmap learned FSMap version 58 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.629+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fcce3a6950 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.630+0000 7fa685600640 1 -- 172.21.15.114:0/3058617875 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fcce38bb90 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.631+0000 7fa685600640 1 -- 172.21.15.114:0/3058617875 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 58) ==== 8378+0+0 (secure 0 0 0) 0x7fa6940b0ce0 con 0x55fcce0e1070 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.632+0000 7fa6a946a740 10 client.29089 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.632+0000 7fa6a946a740 10 client.29089 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.632+0000 7fa6a946a740 20 client.29089 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:56.698 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.632+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce09f9f0 0x55fcce4b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.632+0000 7fa6a946a740 4 client.29089 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.632+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcce400f00 con 0x55fcce09f9f0 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.633+0000 7fa69b400640 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce09f9f0 0x55fcce4b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.635+0000 7fa69b400640 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce09f9f0 0x55fcce4b6a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d4d611eb11266ad server_cookie=3e926da0846d9560 in_seq=0 out_seq=0 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.635+0000 7fa685600640 10 client.29089 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.637+0000 7fa685600640 1 -- 172.21.15.114:0/3058617875 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fcce400f00 con 0x55fcce09f9f0 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.637+0000 7fa685600640 10 client.29089 handle_command_reply: tid=0 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 client.29089 shutdown 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 msgr2=0x55fcce4b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4b0840 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa6940e52a0 tx=0x7fa694004f60 comp rx=0 tx=0).stop 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 shutdown_connections 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce09f9f0 0x55fcce4b6a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.699 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4b0840 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 0x55fcce493e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 --2- 172.21.15.114:0/3058617875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce493900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 >> 172.21.15.114:0/3058617875 conn(0x55fcce0d5460 msgr2=0x55fcce48aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.638+0000 7fa685600640 0 client.29089 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.640+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 shutdown_connections 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.640+0000 7fa6a946a740 1 -- 172.21.15.114:0/3058617875 wait complete. 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.640+0000 7fa6a946a740 20 client.29089 trim_cache size 0 max 16384 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.666+0000 7fa6a946a740 1 Processor -- start 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 -- start start 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce485530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 0x55fcce485a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c00fff0 con 0x55fcce0c0390 2024-09-21T07:08:56.700 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c011b90 con 0x55fcce0b81b0 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.670+0000 7fa6a946a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c0106c0 con 0x55fcce0e1070 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.671+0000 7fa69be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.671+0000 7fa69be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4908f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59696/0 (socket says 172.21.15.114:59696) 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.671+0000 7fa69be00640 1 -- 172.21.15.114:0/2861282209 learned_addr learned my addr 172.21.15.114:0/2861282209 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.671+0000 7fa69b400640 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce485530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.672+0000 7fa69be00640 1 -- 172.21.15.114:0/2861282209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 msgr2=0x55fcce485530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.672+0000 7fa69be00640 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce485530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.672+0000 7fa69be00640 1 -- 172.21.15.114:0/2861282209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 msgr2=0x55fcce485a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.672+0000 7fa69be00640 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 0x55fcce485a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.672+0000 7fa69be00640 1 -- 172.21.15.114:0/2861282209 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcce4b33d0 con 0x55fcce0e1070 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.672+0000 7fa69be00640 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4908f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c760 tx=0x7fa6940d1190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb58bf3891acc35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.673+0000 7fa687e00640 1 -- 172.21.15.114:0/2861282209 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940ae0a0 con 0x55fcce0e1070 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.673+0000 7fa687e00640 1 -- 172.21.15.114:0/2861282209 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6940b73c0 con 0x55fcce0e1070 2024-09-21T07:08:56.701 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.673+0000 7fa687e00640 1 -- 172.21.15.114:0/2861282209 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940ac670 con 0x55fcce0e1070 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 -- 172.21.15.114:0/2861282209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 msgr2=0x55fcce4908f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4908f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c760 tx=0x7fa6940d1190 comp rx=0 tx=0).stop 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 -- 172.21.15.114:0/2861282209 shutdown_connections 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0e1070 0x55fcce4908f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0c0390 0x55fcce485a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2861282209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0b81b0 0x55fcce485530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 -- 172.21.15.114:0/2861282209 >> 172.21.15.114:0/2861282209 conn(0x55fcce0d5460 msgr2=0x55fcce48e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 -- 172.21.15.114:0/2861282209 shutdown_connections 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.675+0000 7fa6a946a740 1 -- 172.21.15.114:0/2861282209 wait complete. 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.676+0000 7fa6a946a740 1 -- start start 2024-09-21T07:08:56.702 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.676+0000 7fa6a946a740 1 Processor -- start 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0b81b0 0x55fcce4ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0c0390 0x55fcce4accf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 0x55fcce4a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c0114c0 con 0x55fcce0b81b0 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c010d90 con 0x55fcce0e1070 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa67c00dfb0 con 0x55fcce0c0390 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.677+0000 7fa6a946a740 10 client.0 _ll_register_callbacks cb 0x7fa6940ad1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.678+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0c0390 0x55fcce4accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.681+0000 7fa69be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 0x55fcce4a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.681+0000 7fa69be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 0x55fcce4a7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54158/0 (socket says 172.21.15.114:54158) 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.681+0000 7fa69be00640 1 -- 172.21.15.114:0/2433365575 learned_addr learned my addr 172.21.15.114:0/2433365575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:56.703 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.681+0000 7fa69b400640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0b81b0 0x55fcce4ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.682+0000 7fa69be00640 1 -- 172.21.15.114:0/2433365575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0c0390 msgr2=0x55fcce4accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.682+0000 7fa69be00640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0c0390 0x55fcce4accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.682+0000 7fa69be00640 1 -- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0b81b0 msgr2=0x55fcce4ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.682+0000 7fa69be00640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0b81b0 0x55fcce4ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.682+0000 7fa69be00640 1 -- 172.21.15.114:0/2433365575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcce4a7ca0 con 0x55fcce0e1070 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.684+0000 7fa69aa00640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0c0390 0x55fcce4accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:56.704 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.684+0000 7fa685600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:56.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.685+0000 7fa69be00640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 0x55fcce4a7760 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa6940a97e0 tx=0x7fa6940a9520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be6c04f4d1c59e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:56.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.686+0000 7fa685600640 1 -- 172.21.15.114:0/2433365575 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6940a9840 con 0x55fcce0e1070 2024-09-21T07:08:56.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.686+0000 7fa685600640 1 -- 172.21.15.114:0/2433365575 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fa6940c3d60 con 0x55fcce0e1070 2024-09-21T07:08:56.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.686+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6940b73c0 con 0x55fcce0e1070 2024-09-21T07:08:56.706 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.686+0000 7fa685600640 1 -- 172.21.15.114:0/2433365575 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa6940ae0a0 con 0x55fcce0e1070 2024-09-21T07:08:56.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.686+0000 7fa6a946a740 10 client.29153 fetch_fsmap learned FSMap version 58 2024-09-21T07:08:56.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.686+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fcce521b20 con 0x55fcce0e1070 2024-09-21T07:08:56.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa685600640 1 -- 172.21.15.114:0/2433365575 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6940b73c0 con 0x55fcce0e1070 2024-09-21T07:08:56.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa685600640 1 -- 172.21.15.114:0/2433365575 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 58) ==== 8378+0+0 (secure 0 0 0) 0x7fa6940bf030 con 0x55fcce0e1070 2024-09-21T07:08:56.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa6a946a740 10 client.29153 fetch_fsmap finished waiting for FSMap version 58 2024-09-21T07:08:56.707 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa6a946a740 10 client.29153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa6a946a740 20 client.29153 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce074da0 0x55fcce524110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa6a946a740 4 client.29153 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.687+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fcce3fc9b0 con 0x55fcce074da0 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.688+0000 7fa69b400640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce074da0 0x55fcce524110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.689+0000 7fa69b400640 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce074da0 0x55fcce524110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6de919f58dabda73 server_cookie=89818bf20a11a28b in_seq=0 out_seq=0 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.690+0000 7fa685600640 10 client.29153 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.695+0000 7fa685600640 1 -- 172.21.15.114:0/2433365575 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fcce3fc9b0 con 0x55fcce074da0 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.695+0000 7fa685600640 10 client.29153 handle_command_reply: tid=0 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.695+0000 7fa6a946a740 1 client.29153 shutdown 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 msgr2=0x55fcce4a7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.708 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 0x55fcce4a7760 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa6940a97e0 tx=0x7fa6940a9520 comp rx=0 tx=0).stop 2024-09-21T07:08:56.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 shutdown_connections 2024-09-21T07:08:56.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55fcce074da0 0x55fcce524110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fcce0e1070 0x55fcce4a7760 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcce0c0390 0x55fcce4accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 --2- 172.21.15.114:0/2433365575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55fcce0b81b0 0x55fcce4ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.709 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.696+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 >> 172.21.15.114:0/2433365575 conn(0x55fcce0d5460 msgr2=0x55fcce48e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.697+0000 7fa685600640 0 client.29153 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:08:56.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.697+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 shutdown_connections 2024-09-21T07:08:56.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.698+0000 7fa6a946a740 1 -- 172.21.15.114:0/2433365575 wait complete. 2024-09-21T07:08:56.710 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.698+0000 7fa6a946a740 20 client.29153 trim_cache size 0 max 16384 2024-09-21T07:08:56.710 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6800b6840 msgr2=0x7fa6800b8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6800b6840 0x7fa6800b8d50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa688006e70 tx=0x7fa6880047d0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c10dc20 msgr2=0x7fa69c1a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c10dc20 0x7fa69c1a6240 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa694009fd0 tx=0x7fa69400ef10 comp rx=0 tx=0).stop 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 shutdown_connections 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa6800b6840 0x7fa6800b8d50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.711 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa69c10dc20 0x7fa69c1a6240 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa69c105110 0x7fa69c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 --2- 172.21.15.114:0/3792271989 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa69c1047c0 0x7fa69c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.700+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 >> 172.21.15.114:0/3792271989 conn(0x7fa69c0fe2c0 msgr2=0x7fa69c107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.701+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 shutdown_connections 2024-09-21T07:08:56.712 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:56.701+0000 7fa6a1200640 1 -- 172.21.15.114:0/3792271989 wait complete. 2024-09-21T07:08:56.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:56 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/58774325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:56.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:56 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/58774325' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:08:57.732 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:57 smithi114 ceph-mon[30654]: pgmap v854: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 235 KiB/s wr, 46 op/s 2024-09-21T07:08:57.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-21T07:08:57.791 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-21T07:08:57.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 860.0 s, 402.4 tps, lat 303.764 ms stddev 941.739 2024-09-21T07:08:57.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:57 smithi183 ceph-mon[34219]: pgmap v854: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 235 KiB/s wr, 46 op/s 2024-09-21T07:08:57.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:57 smithi193 ceph-mon[34601]: pgmap v854: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 235 KiB/s wr, 46 op/s 2024-09-21T07:08:58.189 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:08:58.515 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:59.407 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:08:59.459 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.457+0000 7fe15ac00640 1 -- 172.21.15.114:0/1644732174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09c390 msgr2=0x7fe14c0a2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.457+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1644732174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09c390 0x7fe14c0a2bc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe154066480 tx=0x7fe154098170 comp rx=0 tx=0).stop 2024-09-21T07:08:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.459+0000 7fe15ac00640 1 -- 172.21.15.114:0/1644732174 shutdown_connections 2024-09-21T07:08:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.459+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1644732174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09c390 0x7fe14c0a2bc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.459+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1644732174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c09ba40 0x7fe14c09be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.459+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1644732174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c096100 0x7fe14c096510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.459+0000 7fe15ac00640 1 -- 172.21.15.114:0/1644732174 >> 172.21.15.114:0/1644732174 conn(0x7fe14c00ddf0 msgr2=0x7fe14c00e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.460+0000 7fe15ac00640 1 -- 172.21.15.114:0/1644732174 shutdown_connections 2024-09-21T07:08:59.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.460+0000 7fe15ac00640 1 -- 172.21.15.114:0/1644732174 wait complete. 2024-09-21T07:08:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.463+0000 7fe15ac00640 1 Processor -- start 2024-09-21T07:08:59.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.464+0000 7fe15ac00640 1 -- start start 2024-09-21T07:08:59.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.465+0000 7fe15ac00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 0x7fe14c12ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.465+0000 7fe15ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09ba40 0x7fe14c12f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.466+0000 7fe15ac00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c09c390 0x7fe14c134510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.466+0000 7fe159800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 0x7fe14c12ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.466+0000 7fe15ac00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0a5fe0 con 0x7fe14c09c390 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.466+0000 7fe15ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0a5e60 con 0x7fe14c09ba40 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.466+0000 7fe15ac00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c0a6160 con 0x7fe14c096100 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.466+0000 7fe159800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 0x7fe14c12ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59714/0 (socket says 172.21.15.114:59714) 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.467+0000 7fe159800640 1 -- 172.21.15.114:0/1607599411 learned_addr learned my addr 172.21.15.114:0/1607599411 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.467+0000 7fe15a200640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c09c390 0x7fe14c134510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.468+0000 7fe159800640 1 -- 172.21.15.114:0/1607599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09ba40 msgr2=0x7fe14c12f270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:08:59.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.468+0000 7fe159800640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09ba40 0x7fe14c12f270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.468+0000 7fe159800640 1 -- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c09c390 msgr2=0x7fe14c134510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.468+0000 7fe159800640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c09c390 0x7fe14c134510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.468+0000 7fe159800640 1 -- 172.21.15.114:0/1607599411 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c134c40 con 0x7fe14c096100 2024-09-21T07:08:59.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.468+0000 7fe15a200640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c09c390 0x7fe14c134510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:08:59.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.469+0000 7fe159800640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 0x7fe14c12ecd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe14800b4a0 tx=0x7fe14800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:59.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.470+0000 7fe152000640 1 -- 172.21.15.114:0/1607599411 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148014070 con 0x7fe14c096100 2024-09-21T07:08:59.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.470+0000 7fe152000640 1 -- 172.21.15.114:0/1607599411 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe148010ac0 con 0x7fe14c096100 2024-09-21T07:08:59.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.470+0000 7fe152000640 1 -- 172.21.15.114:0/1607599411 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148017c30 con 0x7fe14c096100 2024-09-21T07:08:59.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.470+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c134f30 con 0x7fe14c096100 2024-09-21T07:08:59.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.470+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c13b690 con 0x7fe14c096100 2024-09-21T07:08:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.480+0000 7fe162e89740 1 Processor -- start 2024-09-21T07:08:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.480+0000 7fe152000640 1 -- 172.21.15.114:0/1607599411 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe148017070 con 0x7fe14c096100 2024-09-21T07:08:59.483 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 -- start start 2024-09-21T07:08:59.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc90653c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9065900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9065e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558cc8fd2a90 con 0x558cc8c8c990 2024-09-21T07:08:59.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558cc8fc1ec0 con 0x558cc8c94b70 2024-09-21T07:08:59.484 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.481+0000 7fe162e89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558cc8fc5cd0 con 0x558cc8cb5850 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.482+0000 7fe15a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc90653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.482+0000 7fe15a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc90653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59722/0 (socket says 172.21.15.114:59722) 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.482+0000 7fe15a200640 1 -- 172.21.15.114:0/2568994932 learned_addr learned my addr 172.21.15.114:0/2568994932 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.483+0000 7fe15a200640 1 -- 172.21.15.114:0/2568994932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 msgr2=0x558cc9065e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.483+0000 7fe15a200640 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9065e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.483+0000 7fe159800640 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9065900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.483+0000 7fe15a200640 1 -- 172.21.15.114:0/2568994932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 msgr2=0x558cc9065900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.484+0000 7fe15a200640 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9065900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.486 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.484+0000 7fe15a200640 1 -- 172.21.15.114:0/2568994932 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc8f679d0 con 0x558cc8cb5850 2024-09-21T07:08:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.484+0000 7fe152000640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe13c0b67b0 0x7fe13c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.484+0000 7fe159800640 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:08:59.487 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.485+0000 7fe159800640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe13c0b67b0 0x7fe13c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.488 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.486+0000 7fe152000640 1 -- 172.21.15.114:0/1607599411 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fe1480e7070 con 0x7fe14c096100 2024-09-21T07:08:59.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.488+0000 7fe15a200640 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc90653c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe1540692a0 tx=0x7fe15406aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=516cb6ebab08bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:59.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.489+0000 7fe159800640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe13c0b67b0 0x7fe13c0b8cc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x558cc9064040 tx=0x7fe1480ec040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:59.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.489+0000 7fe146a00640 1 -- 172.21.15.114:0/2568994932 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540b0070 con 0x558cc8cb5850 2024-09-21T07:08:59.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.489+0000 7fe146a00640 1 -- 172.21.15.114:0/2568994932 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15409c040 con 0x558cc8cb5850 2024-09-21T07:08:59.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.489+0000 7fe146a00640 1 -- 172.21.15.114:0/2568994932 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540ab6e0 con 0x558cc8cb5850 2024-09-21T07:08:59.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.490+0000 7fe162e89740 1 -- 172.21.15.114:0/2568994932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 msgr2=0x558cc90653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.490+0000 7fe162e89740 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc90653c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe1540692a0 tx=0x7fe15406aea0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.491+0000 7fe162e89740 1 -- 172.21.15.114:0/2568994932 shutdown_connections 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.491+0000 7fe162e89740 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9065e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.491+0000 7fe162e89740 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9065900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.491+0000 7fe162e89740 1 --2- 172.21.15.114:0/2568994932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc90653c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.491+0000 7fe162e89740 1 -- 172.21.15.114:0/2568994932 >> 172.21.15.114:0/2568994932 conn(0x558cc8ca9c40 msgr2=0x558cc90632b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.492+0000 7fe162e89740 1 -- 172.21.15.114:0/2568994932 shutdown_connections 2024-09-21T07:08:59.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.492+0000 7fe162e89740 1 -- 172.21.15.114:0/2568994932 wait complete. 2024-09-21T07:08:59.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.493+0000 7fe162e89740 1 -- start start 2024-09-21T07:08:59.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.493+0000 7fe162e89740 1 Processor -- start 2024-09-21T07:08:59.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc90713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc907cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x558cc8e237b0 con 0x558cc8c8c990 2024-09-21T07:08:59.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558cc8f66650 con 0x558cc8c94b70 2024-09-21T07:08:59.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558cc9073b90 con 0x558cc8cb5850 2024-09-21T07:08:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.494+0000 7fe162e89740 10 client.0 _ll_register_callbacks cb 0x558cc8ccd0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc90713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe15a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc90713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54174/0 (socket says 172.21.15.114:54174) 2024-09-21T07:08:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 -- 172.21.15.114:0/3721144573 learned_addr learned my addr 172.21.15.114:0/3721144573 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:59.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe15a200640 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc907cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 -- 172.21.15.114:0/3721144573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 msgr2=0x558cc907cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc907cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 -- 172.21.15.114:0/3721144573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 msgr2=0x558cc9070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.495+0000 7fe158e00640 1 -- 172.21.15.114:0/3721144573 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc8d2fba0 con 0x558cc8c94b70 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe158e00640 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc90713f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe14000e380 tx=0x7fe14000e840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182a67f9af7756b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe13b400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe13b400640 1 -- 172.21.15.114:0/3721144573 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140005220 con 0x558cc8c94b70 2024-09-21T07:08:59.499 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15409c040 con 0x558cc8c94b70 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe13b400640 1 -- 172.21.15.114:0/3721144573 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fe1400053c0 con 0x558cc8c94b70 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe13b400640 1 -- 172.21.15.114:0/3721144573 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe1400055c0 con 0x558cc8c94b70 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe162e89740 10 client.29159 fetch_fsmap learned FSMap version 59 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.496+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558cc907fb50 con 0x558cc8c94b70 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe13b400640 1 -- 172.21.15.114:0/3721144573 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15409c040 con 0x558cc8c94b70 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe13b400640 1 -- 172.21.15.114:0/3721144573 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fe140010bc0 con 0x558cc8c94b70 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe162e89740 10 client.29159 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe162e89740 10 client.29159 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe162e89740 20 client.29159 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe162e89740 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8c741d0 0x558cc90830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe162e89740 4 client.29159 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:08:59.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.497+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc8fd49e0 con 0x558cc8c741d0 2024-09-21T07:08:59.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.498+0000 7fe15a200640 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8c741d0 0x558cc90830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.498+0000 7fe13b400640 10 client.29159 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:59.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.499+0000 7fe15a200640 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8c741d0 0x558cc90830a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461a4cd2b8eb3ada server_cookie=db439872789019ff in_seq=0 out_seq=0 2024-09-21T07:08:59.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe13b400640 1 -- 172.21.15.114:0/3721144573 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558cc8fd49e0 con 0x558cc8c741d0 2024-09-21T07:08:59.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe13b400640 10 client.29159 handle_command_reply: tid=0 2024-09-21T07:08:59.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 client.29159 shutdown 2024-09-21T07:08:59.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 msgr2=0x558cc90713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc90713f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe14000e380 tx=0x7fe14000e840 comp rx=0 tx=0).stop 2024-09-21T07:08:59.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 shutdown_connections 2024-09-21T07:08:59.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8c741d0 0x558cc90830a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8cb5850 0x558cc907cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe13b400640 0 client.29159 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:59.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc90713f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 --2- 172.21.15.114:0/3721144573 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc9070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.500+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 >> 172.21.15.114:0/3721144573 conn(0x558cc8ca9c40 msgr2=0x558cc9068550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.501+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 shutdown_connections 2024-09-21T07:08:59.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.501+0000 7fe162e89740 1 -- 172.21.15.114:0/3721144573 wait complete. 2024-09-21T07:08:59.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.501+0000 7fe162e89740 20 client.29159 trim_cache size 0 max 16384 2024-09-21T07:08:59.506 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:59 smithi114 ceph-mon[30654]: pgmap v855: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-09-21T07:08:59.506 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:08:59 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:59.531 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 Processor -- start 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 -- start start 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 0x558cc9068b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc90728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc907ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe134010030 con 0x558cc8c8c990 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe134011ba0 con 0x558cc8c94b70 2024-09-21T07:08:59.532 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.530+0000 7fe162e89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe134010700 con 0x558cc8c741d0 2024-09-21T07:08:59.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 0x558cc9068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 0x558cc9068b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59734/0 (socket says 172.21.15.114:59734) 2024-09-21T07:08:59.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 -- 172.21.15.114:0/1383452481 learned_addr learned my addr 172.21.15.114:0/1383452481 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:59.533 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe158e00640 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc90728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc907ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 -- 172.21.15.114:0/1383452481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 msgr2=0x558cc907ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc907ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 -- 172.21.15.114:0/1383452481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 msgr2=0x558cc90728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc90728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.531+0000 7fe15a200640 1 -- 172.21.15.114:0/1383452481 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc906f1b0 con 0x558cc8c741d0 2024-09-21T07:08:59.534 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.532+0000 7fe15a200640 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 0x558cc9068b30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe154052020 tx=0x7fe1540a3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=503c3618011743d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:59.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.532+0000 7fe146000640 1 -- 172.21.15.114:0/1383452481 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540b0040 con 0x558cc8c741d0 2024-09-21T07:08:59.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.532+0000 7fe146000640 1 -- 172.21.15.114:0/1383452481 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15409c070 con 0x558cc8c741d0 2024-09-21T07:08:59.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.532+0000 7fe146000640 1 -- 172.21.15.114:0/1383452481 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540ab480 con 0x558cc8c741d0 2024-09-21T07:08:59.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.533+0000 7fe162e89740 1 -- 172.21.15.114:0/1383452481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 msgr2=0x558cc9068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.535 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.533+0000 7fe162e89740 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 0x558cc9068b30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe154052020 tx=0x7fe1540a3040 comp rx=0 tx=0).stop 2024-09-21T07:08:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 -- 172.21.15.114:0/1383452481 shutdown_connections 2024-09-21T07:08:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc907ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.536 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c8c990 0x558cc90728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 --2- 172.21.15.114:0/1383452481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c741d0 0x558cc9068b30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 -- 172.21.15.114:0/1383452481 >> 172.21.15.114:0/1383452481 conn(0x558cc8ca9c40 msgr2=0x558cc90721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 -- 172.21.15.114:0/1383452481 shutdown_connections 2024-09-21T07:08:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.534+0000 7fe162e89740 1 -- 172.21.15.114:0/1383452481 wait complete. 2024-09-21T07:08:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.535+0000 7fe162e89740 1 -- start start 2024-09-21T07:08:59.537 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.535+0000 7fe162e89740 1 Processor -- start 2024-09-21T07:08:59.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 0x558cc9078ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c8c990 0x558cc90793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9073e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.538 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe1340114d0 con 0x558cc8c741d0 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe134010dd0 con 0x558cc8c94b70 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe13400e030 con 0x558cc8c8c990 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe162e89740 10 client.0 _ll_register_callbacks cb 0x7fe140009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.536+0000 7fe15a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 0x558cc9078ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 0x558cc9078ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49550/0 (socket says 172.21.15.114:49550) 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 -- 172.21.15.114:0/2570061421 learned_addr learned my addr 172.21.15.114:0/2570061421 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.539 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe158e00640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c8c990 0x558cc90793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 -- 172.21.15.114:0/2570061421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c8c990 msgr2=0x558cc90793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c8c990 0x558cc90793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 -- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 msgr2=0x558cc9073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe15a200640 1 -- 172.21.15.114:0/2570061421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc9074390 con 0x558cc8c741d0 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.537+0000 7fe158e00640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c8c990 0x558cc90793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe15a200640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 0x558cc9078ea0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe154069050 tx=0x7fe154069570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e0c6211a5edccd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:08:59.540 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe13b400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:08:59.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe13b400640 1 -- 172.21.15.114:0/2570061421 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540b0040 con 0x558cc8c741d0 2024-09-21T07:08:59.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15409c070 con 0x558cc8c741d0 2024-09-21T07:08:59.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe13b400640 1 -- 172.21.15.114:0/2570061421 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fe15406c390 con 0x558cc8c741d0 2024-09-21T07:08:59.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe13b400640 1 -- 172.21.15.114:0/2570061421 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fe1540ab480 con 0x558cc8c741d0 2024-09-21T07:08:59.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe162e89740 10 client.18483 fetch_fsmap learned FSMap version 59 2024-09-21T07:08:59.541 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.538+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558cc90ee0c0 con 0x558cc8c741d0 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe13b400640 1 -- 172.21.15.114:0/2570061421 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15409c070 con 0x558cc8c741d0 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe13b400640 1 -- 172.21.15.114:0/2570061421 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fe1540b6070 con 0x558cc8c741d0 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe162e89740 10 client.18483 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe162e89740 10 client.18483 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe162e89740 20 client.18483 populate_metadata read hostname 'smithi114' 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe162e89740 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8cb5850 0x558cc90f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe162e89740 4 client.18483 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-21T07:08:59.542 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.539+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x558cc8fd0490 con 0x558cc8cb5850 2024-09-21T07:08:59.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.540+0000 7fe158e00640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8cb5850 0x558cc90f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:08:59.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.541+0000 7fe158e00640 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8cb5850 0x558cc90f02f0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6621b530c5abaf9 server_cookie=39d65c5f693c50bb in_seq=0 out_seq=0 2024-09-21T07:08:59.543 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.541+0000 7fe13b400640 10 client.18483 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.766+0000 7fe13b400640 1 -- 172.21.15.114:0/2570061421 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558cc8fd0490 con 0x558cc8cb5850 2024-09-21T07:08:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.766+0000 7fe13b400640 10 client.18483 handle_command_reply: tid=0 2024-09-21T07:08:59.771 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.766+0000 7fe162e89740 1 client.18483 shutdown 2024-09-21T07:08:59.773 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 msgr2=0x558cc9078ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 0x558cc9078ea0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe154069050 tx=0x7fe154069570 comp rx=0 tx=0).stop 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 shutdown_connections 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cc8c94b70 0x558cc9073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x558cc8cb5850 0x558cc90f02f0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558cc8c8c990 0x558cc90793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 --2- 172.21.15.114:0/2570061421 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x558cc8c741d0 0x558cc9078ea0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.774 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 >> 172.21.15.114:0/2570061421 conn(0x558cc8ca9c40 msgr2=0x558cc9063290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.772+0000 7fe13b400640 0 client.18483 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:08:59.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.773+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 shutdown_connections 2024-09-21T07:08:59.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.773+0000 7fe162e89740 1 -- 172.21.15.114:0/2570061421 wait complete. 2024-09-21T07:08:59.775 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.773+0000 7fe162e89740 20 client.18483 trim_cache size 0 max 16384 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 59, 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 58, 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "a95b3f56": { 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 58, 2024-09-21T07:08:59.776 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 138.8, 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "QUIESCED", 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:59.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:59.778 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:08:59.778 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:08:59.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe13c0b67b0 msgr2=0x7fe13c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.778 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe13c0b67b0 0x7fe13c0b8cc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x558cc9064040 tx=0x7fe1480ec040 comp rx=0 tx=0).stop 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 msgr2=0x7fe14c12ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 0x7fe14c12ecd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe14800b4a0 tx=0x7fe14800b960 comp rx=0 tx=0).stop 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 shutdown_connections 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe14c09c390 0x7fe14c134510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe13c0b67b0 0x7fe13c0b8cc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe14c09ba40 0x7fe14c12f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 --2- 172.21.15.114:0/1607599411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe14c096100 0x7fe14c12ecd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:08:59.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.777+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 >> 172.21.15.114:0/1607599411 conn(0x7fe14c00ddf0 msgr2=0x7fe14c099950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:08:59.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.778+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 shutdown_connections 2024-09-21T07:08:59.780 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:08:59.778+0000 7fe15ac00640 1 -- 172.21.15.114:0/1607599411 wait complete. 2024-09-21T07:08:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:59 smithi183 ceph-mon[34219]: pgmap v855: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-09-21T07:08:59.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:08:59 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:08:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:59 smithi193 ceph-mon[34601]: pgmap v855: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-09-21T07:08:59.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:08:59 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:00.711 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a95b3f56', quiesce took 6.9 seconds. Will release after: 11.950000000000001 2024-09-21T07:09:01.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 -- 172.21.15.114:0/459191408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f0105300 msgr2=0x7f29f010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 --2- 172.21.15.114:0/459191408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f0105300 0x7f29f010d850 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009900 tx=0x7f29d802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 -- 172.21.15.114:0/459191408 shutdown_connections 2024-09-21T07:09:01.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 --2- 172.21.15.114:0/459191408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f010de10 0x7f29f0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 --2- 172.21.15.114:0/459191408 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f0105300 0x7f29f010d850 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 --2- 172.21.15.114:0/459191408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f01049b0 0x7f29f0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 -- 172.21.15.114:0/459191408 >> 172.21.15.114:0/459191408 conn(0x7f29f0079760 msgr2=0x7f29f0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.015+0000 7f29f4c00640 1 -- 172.21.15.114:0/459191408 shutdown_connections 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.016+0000 7f29f4c00640 1 -- 172.21.15.114:0/459191408 wait complete. 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 Processor -- start 2024-09-21T07:09:01.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 -- start start 2024-09-21T07:09:01.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 0x7f29f01026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f0105300 0x7f29f0102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f010de10 0x7f29f0103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0113ac0 con 0x7f29f01049b0 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0113940 con 0x7f29f0105300 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.017+0000 7f29f4c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0113c40 con 0x7f29f010de10 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 0x7f29f01026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 0x7f29f01026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49574/0 (socket says 172.21.15.114:49574) 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 -- 172.21.15.114:0/2056409246 learned_addr learned my addr 172.21.15.114:0/2056409246 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29e5400640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f0105300 0x7f29f0102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ee800640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f010de10 0x7f29f0103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 -- 172.21.15.114:0/2056409246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f010de10 msgr2=0x7f29f0103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f010de10 0x7f29f0103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 -- 172.21.15.114:0/2056409246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f0105300 msgr2=0x7f29f0102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f0105300 0x7f29f0102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 -- 172.21.15.114:0/2056409246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f01a8830 con 0x7f29f01049b0 2024-09-21T07:09:01.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29e5400640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f0105300 0x7f29f0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:01.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.018+0000 7f29ede00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 0x7f29f01026f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f29e000ea60 tx=0x7f29e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.019+0000 7f29e7400640 1 -- 172.21.15.114:0/2056409246 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e000cde0 con 0x7f29f01049b0 2024-09-21T07:09:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.019+0000 7f29e7400640 1 -- 172.21.15.114:0/2056409246 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e00035f0 con 0x7f29f01049b0 2024-09-21T07:09:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.019+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f01a8b20 con 0x7f29f01049b0 2024-09-21T07:09:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.019+0000 7f29e7400640 1 -- 172.21.15.114:0/2056409246 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0010690 con 0x7f29f01049b0 2024-09-21T07:09:01.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.019+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01a9030 con 0x7f29f01049b0 2024-09-21T07:09:01.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.022+0000 7f29fb9be740 1 Processor -- start 2024-09-21T07:09:01.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.022+0000 7f29fb9be740 1 -- start start 2024-09-21T07:09:01.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.022+0000 7f29e7400640 1 -- 172.21.15.114:0/2056409246 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f29e00037a0 con 0x7f29f01049b0 2024-09-21T07:09:01.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.022+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a184bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.022+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a184c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.022+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a184c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29fb9be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5583a17c2fb0 con 0x5583a147c1b0 2024-09-21T07:09:01.025 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29fb9be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5583a17b23e0 con 0x5583a14a5070 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29fb9be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583a17b61f0 con 0x5583a1484390 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ede00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a184c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a184bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ede00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a184c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49584/0 (socket says 172.21.15.114:49584) 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ede00640 1 -- 172.21.15.114:0/1822948468 learned_addr learned my addr 172.21.15.114:0/1822948468 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29e5400640 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a184c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ee800640 1 -- 172.21.15.114:0/1822948468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 msgr2=0x5583a184c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ee800640 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a184c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ee800640 1 -- 172.21.15.114:0/1822948468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 msgr2=0x5583a184c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ee800640 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a184c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.023+0000 7f29ee800640 1 -- 172.21.15.114:0/1822948468 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a177ccd0 con 0x5583a14a5070 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.024+0000 7f29ede00640 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a184c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:01.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.024+0000 7f29ee800640 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a184bda0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009e50 tx=0x7f29e80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8063bee8a78877f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.024+0000 7f29df000640 1 -- 172.21.15.114:0/1822948468 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8003b70 con 0x5583a14a5070 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.024+0000 7f29df000640 1 -- 172.21.15.114:0/1822948468 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29e8015070 con 0x5583a14a5070 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.024+0000 7f29df000640 1 -- 172.21.15.114:0/1822948468 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e8011740 con 0x5583a14a5070 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29e7400640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29d00b6680 0x7f29d00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29e7400640 1 -- 172.21.15.114:0/2056409246 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f29e00e7ee0 con 0x7f29f01049b0 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29ede00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29d00b6680 0x7f29d00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 -- 172.21.15.114:0/1822948468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 msgr2=0x5583a184bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a184bda0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009e50 tx=0x7f29e80107a0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 -- 172.21.15.114:0/1822948468 shutdown_connections 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a184c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a184c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 --2- 172.21.15.114:0/1822948468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a184bda0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 -- 172.21.15.114:0/1822948468 >> 172.21.15.114:0/1822948468 conn(0x5583a1499460 msgr2=0x5583a1849c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.025+0000 7f29fb9be740 1 -- 172.21.15.114:0/1822948468 shutdown_connections 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.026+0000 7f29fb9be740 1 -- 172.21.15.114:0/1822948468 wait complete. 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.026+0000 7f29ede00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29d00b6680 0x7f29d00b8b90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f29e00aab70 tx=0x7f29e00eb990 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.026+0000 7f29fb9be740 1 -- start start 2024-09-21T07:09:01.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.026+0000 7f29fb9be740 1 Processor -- start 2024-09-21T07:09:01.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a147c1b0 0x5583a18575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.031 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a1857b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a18638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5583a1613700 con 0x5583a14a5070 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5583a174d580 con 0x5583a147c1b0 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5583a185a580 con 0x5583a1484390 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29fb9be740 10 client.0 _ll_register_callbacks cb 0x5583a175d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a147c1b0 0x5583a18575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29ee800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a18638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29e5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a1857b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29e5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a1857b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59786/0 (socket says 172.21.15.114:59786) 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29ee800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a147c1b0 0x5583a18575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54216/0 (socket says 172.21.15.114:54216) 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.027+0000 7f29e5400640 1 -- 172.21.15.114:0/373667280 learned_addr learned my addr 172.21.15.114:0/373667280 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29e5400640 1 -- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a147c1b0 msgr2=0x5583a18575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29e5400640 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a147c1b0 0x5583a18575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29e5400640 1 -- 172.21.15.114:0/373667280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 msgr2=0x5583a18638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29e5400640 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a18638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29e5400640 1 -- 172.21.15.114:0/373667280 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a1510650 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29e5400640 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a1857b00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f29d8006010 tx=0x7f29d8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41f22cb17ce10a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29d7400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29d7400640 1 -- 172.21.15.114:0/373667280 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8035040 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583a174fb90 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.028+0000 7f29d7400640 1 -- 172.21.15.114:0/373667280 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f29d8004970 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29d7400640 1 -- 172.21.15.114:0/373667280 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f29d8004b70 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 10 client.29125 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5583a176a950 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29d7400640 1 -- 172.21.15.114:0/373667280 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5583a174fb90 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29d7400640 1 -- 172.21.15.114:0/373667280 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7f29d8044020 con 0x5583a1484390 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 10 client.29125 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 10 client.29125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 20 client.29125 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:01.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a14639f0 0x5583a1869630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 4 client.29125 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:01.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a17c4f00 con 0x5583a14639f0 2024-09-21T07:09:01.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.029+0000 7f29ee800640 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a14639f0 0x5583a1869630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.030+0000 7f29ee800640 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a14639f0 0x5583a1869630 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1a5562c5522a4c server_cookie=317467a5dcb09e03 in_seq=0 out_seq=0 2024-09-21T07:09:01.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.030+0000 7f29d7400640 10 client.29125 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29d7400640 1 -- 172.21.15.114:0/373667280 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583a17c4f00 con 0x5583a14639f0 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29d7400640 10 client.29125 handle_command_reply: tid=0 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 client.29125 shutdown 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 msgr2=0x5583a1857b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a1857b00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f29d8006010 tx=0x7f29d8005270 comp rx=0 tx=0).stop 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 shutdown_connections 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a18638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a14639f0 0x5583a1869630 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a1857b00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 --2- 172.21.15.114:0/373667280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a147c1b0 0x5583a18575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.035 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29d7400640 0 client.29125 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 >> 172.21.15.114:0/373667280 conn(0x5583a1499460 msgr2=0x5583a184ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 shutdown_connections 2024-09-21T07:09:01.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 1 -- 172.21.15.114:0/373667280 wait complete. 2024-09-21T07:09:01.036 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.031+0000 7f29fb9be740 20 client.29125 trim_cache size 0 max 16384 2024-09-21T07:09:01.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.045+0000 7f29fb9be740 1 Processor -- start 2024-09-21T07:09:01.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.045+0000 7f29fb9be740 1 -- start start 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a147c1b0 0x5583a1849cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 0x5583a18637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a184eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29fb9be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b3190 con 0x5583a14a5070 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29fb9be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b4d00 con 0x5583a1484390 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29fb9be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b3860 con 0x5583a147c1b0 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29ee800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a147c1b0 0x5583a1849cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 0x5583a18637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29ee800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a184eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 0x5583a18637a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54222/0 (socket says 172.21.15.114:54222) 2024-09-21T07:09:01.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 -- 172.21.15.114:0/4266901956 learned_addr learned my addr 172.21.15.114:0/4266901956 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.049 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 -- 172.21.15.114:0/4266901956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a147c1b0 msgr2=0x5583a1849cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a147c1b0 0x5583a1849cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 -- 172.21.15.114:0/4266901956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 msgr2=0x5583a184eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a184eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.046+0000 7f29e5400640 1 -- 172.21.15.114:0/4266901956 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a1858880 con 0x5583a1484390 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.047+0000 7f29e5400640 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 0x5583a18637a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009850 tx=0x7f29d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6830e008a0c6cf13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.047+0000 7f29de600640 1 -- 172.21.15.114:0/4266901956 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8000e60 con 0x5583a1484390 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.047+0000 7f29de600640 1 -- 172.21.15.114:0/4266901956 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d8031520 con 0x5583a1484390 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.047+0000 7f29de600640 1 -- 172.21.15.114:0/4266901956 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8035030 con 0x5583a1484390 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 -- 172.21.15.114:0/4266901956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 msgr2=0x5583a18637a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 0x5583a18637a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009850 tx=0x7f29d8000bc0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 -- 172.21.15.114:0/4266901956 shutdown_connections 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a14a5070 0x5583a184eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a1484390 0x5583a18637a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.050 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 --2- 172.21.15.114:0/4266901956 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a147c1b0 0x5583a1849cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 -- 172.21.15.114:0/4266901956 >> 172.21.15.114:0/4266901956 conn(0x5583a1499460 msgr2=0x5583a18495b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 -- 172.21.15.114:0/4266901956 shutdown_connections 2024-09-21T07:09:01.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 -- 172.21.15.114:0/4266901956 wait complete. 2024-09-21T07:09:01.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.048+0000 7f29fb9be740 1 -- start start 2024-09-21T07:09:01.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 Processor -- start 2024-09-21T07:09:01.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a185bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.053 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a185c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a18d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b4630 con 0x5583a147c1b0 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b3f30 con 0x5583a14a5070 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29f01b1190 con 0x5583a1484390 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29fb9be740 10 client.0 _ll_register_callbacks cb 0x7f29d8004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29ee800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a185bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29e5400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a185c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29ee800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a185bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49606/0 (socket says 172.21.15.114:49606) 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.049+0000 7f29ee800640 1 -- 172.21.15.114:0/2465981972 learned_addr learned my addr 172.21.15.114:0/2465981972 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29ee800640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a18d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29e5400640 1 -- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 msgr2=0x5583a18d2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29e5400640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a18d2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29e5400640 1 -- 172.21.15.114:0/2465981972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 msgr2=0x5583a185bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.054 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29e5400640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a185bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29e5400640 1 -- 172.21.15.114:0/2465981972 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a18d3520 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29ee800640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a18d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29e5400640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a185c0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009900 tx=0x7f29d80058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900aabf69917380c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.050+0000 7f29d7400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29d7400640 1 -- 172.21.15.114:0/2465981972 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8035070 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29d7400640 1 -- 172.21.15.114:0/2465981972 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7f29d80037f0 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d8031520 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29d7400640 1 -- 172.21.15.114:0/2465981972 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f29d80039f0 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 10 client.29131 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5583a18d4610 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29d7400640 1 -- 172.21.15.114:0/2465981972 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d8031520 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29d7400640 1 -- 172.21.15.114:0/2465981972 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7f29d8044040 con 0x5583a1484390 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 10 client.29131 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 10 client.29131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:01.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 20 client.29131 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:01.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a1438da0 0x5583a18d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 4 client.29131 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:09:01.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.051+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583a17c09b0 con 0x5583a1438da0 2024-09-21T07:09:01.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.052+0000 7f29ee800640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a1438da0 0x5583a18d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.052+0000 7f29ee800640 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a1438da0 0x5583a18d6cb0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bdfd6aa99a6495 server_cookie=3f0f557f132fb957 in_seq=0 out_seq=0 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.052+0000 7f29d7400640 10 client.29131 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.052+0000 7f29d7400640 1 -- 172.21.15.114:0/2465981972 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583a17c09b0 con 0x5583a1438da0 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.052+0000 7f29d7400640 10 client.29131 handle_command_reply: tid=0 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.052+0000 7f29fb9be740 1 client.29131 shutdown 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 msgr2=0x5583a185c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a185c0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009900 tx=0x7f29d80058c0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 shutdown_connections 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5583a14a5070 0x5583a18d2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5583a1438da0 0x5583a18d6cb0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5583a1484390 0x5583a185c0c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 --2- 172.21.15.114:0/2465981972 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5583a147c1b0 0x5583a185bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 >> 172.21.15.114:0/2465981972 conn(0x5583a1499460 msgr2=0x5583a184eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29d7400640 0 client.29131 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 shutdown_connections 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 1 -- 172.21.15.114:0/2465981972 wait complete. 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.053+0000 7f29fb9be740 20 client.29131 trim_cache size 0 max 16384 2024-09-21T07:09:01.058 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29d00b6680 msgr2=0x7f29d00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29d00b6680 0x7f29d00b8b90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f29e00aab70 tx=0x7f29e00eb990 comp rx=0 tx=0).stop 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 msgr2=0x7f29f01026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 0x7f29f01026f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f29e000ea60 tx=0x7f29e000ef20 comp rx=0 tx=0).stop 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 shutdown_connections 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f010de10 0x7f29f0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f29d00b6680 0x7f29d00b8b90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29f0105300 0x7f29f0102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 --2- 172.21.15.114:0/2056409246 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f29f01049b0 0x7f29f01026f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 >> 172.21.15.114:0/2056409246 conn(0x7f29f0079760 msgr2=0x7f29f0106890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 shutdown_connections 2024-09-21T07:09:01.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:01.054+0000 7f29f4c00640 1 -- 172.21.15.114:0/2056409246 wait complete. 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 -- 172.21.15.114:0/4009616164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 msgr2=0x7fa9d41125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 --2- 172.21.15.114:0/4009616164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 0x7fa9d41125c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009be0 tx=0x7fa9cc02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 -- 172.21.15.114:0/4009616164 shutdown_connections 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 --2- 172.21.15.114:0/4009616164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 0x7fa9d41125c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 --2- 172.21.15.114:0/4009616164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d410b440 0x7fa9d410b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 --2- 172.21.15.114:0/4009616164 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9d4105b10 0x7fa9d4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 -- 172.21.15.114:0/4009616164 >> 172.21.15.114:0/4009616164 conn(0x7fa9d4101550 msgr2=0x7fa9d41039a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 -- 172.21.15.114:0/4009616164 shutdown_connections 2024-09-21T07:09:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.511+0000 7fa9daa00640 1 -- 172.21.15.114:0/4009616164 wait complete. 2024-09-21T07:09:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.512+0000 7fa9daa00640 1 Processor -- start 2024-09-21T07:09:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.512+0000 7fa9daa00640 1 -- start start 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9daa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 0x7fa9d419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9daa00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9d410b440 0x7fa9d419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9daa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 0x7fa9d41a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9daa00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4115a50 con 0x7fa9d410b440 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9daa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41158d0 con 0x7fa9d4105b10 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9daa00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4115bd0 con 0x7fa9d410bd90 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 0x7fa9d419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d8c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9d410b440 0x7fa9d419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 0x7fa9d419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54250/0 (socket says 172.21.15.114:54250) 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 -- 172.21.15.114:0/2532221509 learned_addr learned my addr 172.21.15.114:0/2532221509 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9da000640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 0x7fa9d41a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 -- 172.21.15.114:0/2532221509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 msgr2=0x7fa9d41a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 0x7fa9d41a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 -- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9d410b440 msgr2=0x7fa9d419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9d410b440 0x7fa9d419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.513+0000 7fa9d9600640 1 -- 172.21.15.114:0/2532221509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d41a23e0 con 0x7fa9d4105b10 2024-09-21T07:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.514+0000 7fa9d9600640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 0x7fa9d419c4b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800e9c0 tx=0x7fa9c800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.514+0000 7fa9d2000640 1 -- 172.21.15.114:0/2532221509 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c800cde0 con 0x7fa9d4105b10 2024-09-21T07:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.514+0000 7fa9d2000640 1 -- 172.21.15.114:0/2532221509 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c80035f0 con 0x7fa9d4105b10 2024-09-21T07:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.514+0000 7fa9d2000640 1 -- 172.21.15.114:0/2532221509 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8010690 con 0x7fa9d4105b10 2024-09-21T07:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.514+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d41a26d0 con 0x7fa9d4105b10 2024-09-21T07:09:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.514+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41a8e60 con 0x7fa9d4105b10 2024-09-21T07:09:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.518+0000 7fa9e2fd0740 1 Processor -- start 2024-09-21T07:09:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.518+0000 7fa9e2fd0740 1 -- start start 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717e99e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e9a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559717e0f6b0 con 0x559717af51b0 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559717dfeae0 con 0x559717acc2f0 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559717e028f0 con 0x559717ad44d0 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9da000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717e99e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9da000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717e99e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49626/0 (socket says 172.21.15.114:49626) 2024-09-21T07:09:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9da000640 1 -- 172.21.15.114:0/316002975 learned_addr learned my addr 172.21.15.114:0/316002975 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.519+0000 7fa9d2000640 1 -- 172.21.15.114:0/2532221509 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fa9c8004000 con 0x7fa9d4105b10 2024-09-21T07:09:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9da000640 1 -- 172.21.15.114:0/316002975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 msgr2=0x559717e9a890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:09:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9da000640 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9da000640 1 -- 172.21.15.114:0/316002975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 msgr2=0x559717e9a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9da000640 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e9a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9da000640 1 -- 172.21.15.114:0/316002975 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559717dcb780 con 0x559717af51b0 2024-09-21T07:09:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9da000640 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717e99e10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009be0 tx=0x7fa9cc030430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5bbf00fb446fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9c6600640 1 -- 172.21.15.114:0/316002975 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc041030 con 0x559717af51b0 2024-09-21T07:09:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9c6600640 1 -- 172.21.15.114:0/316002975 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc003e20 con 0x559717af51b0 2024-09-21T07:09:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.520+0000 7fa9c6600640 1 -- 172.21.15.114:0/316002975 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc036040 con 0x559717af51b0 2024-09-21T07:09:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9d2000640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9d2000640 1 -- 172.21.15.114:0/2532221509 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fa9c80e7e10 con 0x7fa9d4105b10 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/316002975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 msgr2=0x559717e99e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717e99e10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009be0 tx=0x7fa9cc030430 comp rx=0 tx=0).stop 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/316002975 shutdown_connections 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9d9600640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e9a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/316002975 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717e99e10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.521+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/316002975 >> 172.21.15.114:0/316002975 conn(0x559717ae9300 msgr2=0x559717e97d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.522+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/316002975 shutdown_connections 2024-09-21T07:09:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.522+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/316002975 wait complete. 2024-09-21T07:09:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.522+0000 7fa9d9600640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x559717e98a90 tx=0x7fa9c80d1050 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.522+0000 7fa9e2fd0740 1 -- start start 2024-09-21T07:09:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 Processor -- start 2024-09-21T07:09:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 0x559717ea53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717ad44d0 0x559717ea5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559717d36ab0 con 0x559717af51b0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559717da5570 con 0x559717ad44d0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559717ea8530 con 0x559717acc2f0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9e2fd0740 10 client.0 _ll_register_callbacks cb 0x559717db4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9da000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9da000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 0x559717ea53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9da000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 0x559717ea53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59850/0 (socket says 172.21.15.114:59850) 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.523+0000 7fa9da000640 1 -- 172.21.15.114:0/2493206010 learned_addr learned my addr 172.21.15.114:0/2493206010 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717ad44d0 0x559717ea5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9da000640 1 -- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717ad44d0 msgr2=0x559717ea5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9da000640 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717ad44d0 0x559717ea5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9da000640 1 -- 172.21.15.114:0/2493206010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 msgr2=0x559717ea5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9da000640 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9da000640 1 -- 172.21.15.114:0/2493206010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559717b60790 con 0x559717acc2f0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9da000640 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 0x559717ea53d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x559717ea8160 tx=0x7fa9cc02a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73ed074e4d430759 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9baa00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9baa00640 1 -- 172.21.15.114:0/2493206010 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc041070 con 0x559717acc2f0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9baa00640 1 -- 172.21.15.114:0/2493206010 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fa9cc003b70 con 0x559717acc2f0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.524+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc003e20 con 0x559717acc2f0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9baa00640 1 -- 172.21.15.114:0/2493206010 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc036070 con 0x559717acc2f0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9e2fd0740 10 client.29140 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x559717bd9620 con 0x559717acc2f0 2024-09-21T07:09:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9baa00640 1 -- 172.21.15.114:0/2493206010 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc003e20 con 0x559717acc2f0 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9baa00640 1 -- 172.21.15.114:0/2493206010 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fa9cc03d760 con 0x559717acc2f0 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9e2fd0740 10 client.29140 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9e2fd0740 10 client.29140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9e2fd0740 20 client.29140 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.525+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717ab3d80 0x559717eb72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.526+0000 7fa9e2fd0740 4 client.29140 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.526+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559717e11600 con 0x559717ab3d80 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.526+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717ab3d80 0x559717eb72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.526+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717ab3d80 0x559717eb72f0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb7c1d8b553348f server_cookie=bff34d1b7ccc2e88 in_seq=0 out_seq=0 2024-09-21T07:09:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.526+0000 7fa9baa00640 10 client.29140 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9baa00640 1 -- 172.21.15.114:0/2493206010 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559717e11600 con 0x559717ab3d80 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9baa00640 10 client.29140 handle_command_reply: tid=0 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 client.29140 shutdown 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 msgr2=0x559717ea53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 0x559717ea53d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x559717ea8160 tx=0x7fa9cc02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 shutdown_connections 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717ad44d0 0x559717ea5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717ab3d80 0x559717eb72f0 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/2493206010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717acc2f0 0x559717ea53d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9baa00640 0 client.29140 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.527+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 >> 172.21.15.114:0/2493206010 conn(0x559717ae9300 msgr2=0x559717e9d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.528+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 shutdown_connections 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.528+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/2493206010 wait complete. 2024-09-21T07:09:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.528+0000 7fa9e2fd0740 20 client.29140 trim_cache size 0 max 16384 2024-09-21T07:09:01.544 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 Processor -- start 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 -- start start 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e969d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b2ed0 con 0x559717af51b0 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b4a40 con 0x559717acc2f0 2024-09-21T07:09:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b35a0 con 0x559717ad44d0 2024-09-21T07:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9da000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9da000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49652/0 (socket says 172.21.15.114:49652) 2024-09-21T07:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9da000640 1 -- 172.21.15.114:0/467780500 learned_addr learned my addr 172.21.15.114:0/467780500 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.543+0000 7fa9da000640 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9da000640 1 -- 172.21.15.114:0/467780500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 msgr2=0x559717e9d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9da000640 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9da000640 1 -- 172.21.15.114:0/467780500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 msgr2=0x559717e969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9da000640 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9da000640 1 -- 172.21.15.114:0/467780500 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559717e98600 con 0x559717af51b0 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9da000640 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea58d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0099a0 tx=0x7fa9cc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7fa3d2ebbbbc8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9c5c00640 1 -- 172.21.15.114:0/467780500 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc041070 con 0x559717af51b0 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9c5c00640 1 -- 172.21.15.114:0/467780500 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc030970 con 0x559717af51b0 2024-09-21T07:09:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.544+0000 7fa9c5c00640 1 -- 172.21.15.114:0/467780500 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc004090 con 0x559717af51b0 2024-09-21T07:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/467780500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 msgr2=0x559717ea58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea58d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0099a0 tx=0x7fa9cc03f040 comp rx=0 tx=0).stop 2024-09-21T07:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/467780500 shutdown_connections 2024-09-21T07:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717ea58d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717e9d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/467780500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717e969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/467780500 >> 172.21.15.114:0/467780500 conn(0x559717ae9300 msgr2=0x559717e9cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/467780500 shutdown_connections 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.545+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/467780500 wait complete. 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 -- start start 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 Processor -- start 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717eb8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717eb88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717eb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b4370 con 0x559717af51b0 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b3c70 con 0x559717acc2f0 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.546+0000 7fa9e2fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41b0ed0 con 0x559717ad44d0 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9e2fd0740 10 client.0 _ll_register_callbacks cb 0x7fa9c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717eb8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717eb8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54274/0 (socket says 172.21.15.114:54274) 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 -- 172.21.15.114:0/1376595293 learned_addr learned my addr 172.21.15.114:0/1376595293 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717eb88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 -- 172.21.15.114:0/1376595293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 msgr2=0x559717eb88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717eb88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 -- 172.21.15.114:0/1376595293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 msgr2=0x559717eb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717eb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9da000640 1 -- 172.21.15.114:0/1376595293 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559717e9db10 con 0x559717acc2f0 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.547+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717eb88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9da000640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717eb8390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009560 tx=0x7fa9cc004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8522697097e96020 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9baa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9baa00640 1 -- 172.21.15.114:0/1376595293 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc060030 con 0x559717acc2f0 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc030970 con 0x559717acc2f0 2024-09-21T07:09:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9baa00640 1 -- 172.21.15.114:0/1376595293 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fa9cc03d460 con 0x559717acc2f0 2024-09-21T07:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9baa00640 1 -- 172.21.15.114:0/1376595293 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc041040 con 0x559717acc2f0 2024-09-21T07:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9e2fd0740 10 client.29195 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.548+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x559717f21bb0 con 0x559717acc2f0 2024-09-21T07:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9baa00640 1 -- 172.21.15.114:0/1376595293 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc030970 con 0x559717acc2f0 2024-09-21T07:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9baa00640 1 -- 172.21.15.114:0/1376595293 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fa9cc05a040 con 0x559717acc2f0 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9e2fd0740 10 client.29195 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9e2fd0740 10 client.29195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9e2fd0740 20 client.29195 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717a891c0 0x559717f24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9e2fd0740 4 client.29195 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559717e0d0b0 con 0x559717a891c0 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.549+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717a891c0 0x559717f24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.550+0000 7fa9d8c00640 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717a891c0 0x559717f24100 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85421b59d91d089a server_cookie=af9bf493913eee75 in_seq=0 out_seq=0 2024-09-21T07:09:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:01.550+0000 7fa9baa00640 10 client.29195 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:01.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:01 smithi114 ceph-mon[30654]: pgmap v856: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-21T07:09:01.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:01 smithi183 ceph-mon[34219]: pgmap v856: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-21T07:09:01.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:01 smithi193 ceph-mon[34601]: pgmap v856: 129 pgs: 129 active+clean; 3.1 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-21T07:09:02.835 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 865.0 s, 142.7 tps, lat 250.999 ms stddev 271.650 2024-09-21T07:09:02.929 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T07:09:02.930 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T07:09:03.305 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:03.754 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:03 smithi114 ceph-mon[30654]: pgmap v857: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-21T07:09:03.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:03 smithi183 ceph-mon[34219]: pgmap v857: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-21T07:09:03.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:03 smithi193 ceph-mon[34601]: pgmap v857: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 -- 172.21.15.114:0/1731100652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 msgr2=0x7fb7f81096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 --2- 172.21.15.114:0/1731100652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f81096e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009900 tx=0x7fb7e802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 -- 172.21.15.114:0/1731100652 shutdown_connections 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 --2- 172.21.15.114:0/1731100652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 0x7fb7f8110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 --2- 172.21.15.114:0/1731100652 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f81096e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 --2- 172.21.15.114:0/1731100652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 0x7fb7f8103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 -- 172.21.15.114:0/1731100652 >> 172.21.15.114:0/1731100652 conn(0x7fb7f80ff3e0 msgr2=0x7fb7f8101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 -- 172.21.15.114:0/1731100652 shutdown_connections 2024-09-21T07:09:03.933 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.931+0000 7fb7ff200640 1 -- 172.21.15.114:0/1731100652 wait complete. 2024-09-21T07:09:03.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.932+0000 7fb7ff200640 1 Processor -- start 2024-09-21T07:09:03.934 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.932+0000 7fb7ff200640 1 -- start start 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7ff200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 0x7fb7f819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7ff200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7ff200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 0x7fb7f81a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7ff200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8113930 con 0x7fb7f81092d0 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7ff200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81137b0 con 0x7fb7f8109c20 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7ff200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8113ab0 con 0x7fb7f8103990 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7fd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7fd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49686/0 (socket says 172.21.15.114:49686) 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7fd400640 1 -- 172.21.15.114:0/704412403 learned_addr learned my addr 172.21.15.114:0/704412403 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 0x7fb7f819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.935 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.933+0000 7fb7fe800640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 0x7fb7f81a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fd400640 1 -- 172.21.15.114:0/704412403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 msgr2=0x7fb7f819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.936 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fd400640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 0x7fb7f819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fd400640 1 -- 172.21.15.114:0/704412403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 msgr2=0x7fb7f81a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fd400640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 0x7fb7f81a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fd400640 1 -- 172.21.15.114:0/704412403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f81a24b0 con 0x7fb7f81092d0 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fde00640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 0x7fb7f819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fe800640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 0x7fb7f81a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7fd400640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f819cae0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8005ab0 tx=0x7fb7e8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7f2a00640 1 -- 172.21.15.114:0/704412403 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e803f070 con 0x7fb7f81092d0 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7f2a00640 1 -- 172.21.15.114:0/704412403 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e8031b50 con 0x7fb7f81092d0 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f81a2740 con 0x7fb7f81092d0 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.934+0000 7fb7f2a00640 1 -- 172.21.15.114:0/704412403 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8035030 con 0x7fb7f81092d0 2024-09-21T07:09:03.937 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.935+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a8f00 con 0x7fb7f81092d0 2024-09-21T07:09:03.939 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.938+0000 7fb80747e740 1 Processor -- start 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.938+0000 7fb80747e740 1 -- start start 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.938+0000 7fb7f2a00640 1 -- 172.21.15.114:0/704412403 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fb7e8042070 con 0x7fb7f81092d0 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb80747e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc00df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb80747e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc00e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb80747e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc00e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb80747e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5616bbf83e90 con 0x5616bbc45670 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb80747e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616bbf732c0 con 0x5616bbc66350 2024-09-21T07:09:03.940 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb80747e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5616bbf770d0 con 0x5616bbc3d490 2024-09-21T07:09:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc00e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc00e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc00e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:59894/0 (socket says 172.21.15.114:59894) 2024-09-21T07:09:03.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc00e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49698/0 (socket says 172.21.15.114:49698) 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fde00640 1 -- 172.21.15.114:0/1188091413 learned_addr learned my addr 172.21.15.114:0/1188091413 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fe800640 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc00df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fde00640 1 -- 172.21.15.114:0/1188091413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 msgr2=0x5616bc00df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fde00640 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc00df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.939+0000 7fb7fde00640 1 -- 172.21.15.114:0/1188091413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 msgr2=0x5616bc00e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7fde00640 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc00e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7fde00640 1 -- 172.21.15.114:0/1188091413 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616bbcd1930 con 0x5616bbc3d490 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7fe800640 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc00df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:09:03.942 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7fde00640 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc00e440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0115c0 tx=0x7fb7ec011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c22a0ed1a68dba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7e7000640 1 -- 172.21.15.114:0/1188091413 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00f640 con 0x5616bbc3d490 2024-09-21T07:09:03.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7e7000640 1 -- 172.21.15.114:0/1188091413 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec00fc90 con 0x5616bbc3d490 2024-09-21T07:09:03.943 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7e7000640 1 -- 172.21.15.114:0/1188091413 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec019c60 con 0x5616bbc3d490 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.940+0000 7fb7f2a00640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6680 0x7fb7e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb7f2a00640 1 -- 172.21.15.114:0/704412403 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fb7e8108070 con 0x7fb7f81092d0 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb7fe800640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6680 0x7fb7e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 -- 172.21.15.114:0/1188091413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 msgr2=0x5616bc00e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc00e440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0115c0 tx=0x7fb7ec011a80 comp rx=0 tx=0).stop 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 -- 172.21.15.114:0/1188091413 shutdown_connections 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc00e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc00e440 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 --2- 172.21.15.114:0/1188091413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc00df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.941+0000 7fb80747e740 1 -- 172.21.15.114:0/1188091413 >> 172.21.15.114:0/1188091413 conn(0x5616bbc5a740 msgr2=0x5616bc00bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.942+0000 7fb7fe800640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6680 0x7fb7e00b8b90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005fd0 tx=0x7fb7f4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.942+0000 7fb80747e740 1 -- 172.21.15.114:0/1188091413 shutdown_connections 2024-09-21T07:09:03.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.942+0000 7fb80747e740 1 -- 172.21.15.114:0/1188091413 wait complete. 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.942+0000 7fb80747e740 1 -- start start 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 Processor -- start 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc0199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc019f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc025a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5616bbdd4890 con 0x5616bbc45670 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616bbf0e160 con 0x5616bbc66350 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5616bc01c6e0 con 0x5616bbc3d490 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb80747e740 10 client.0 _ll_register_callbacks cb 0x5616bbf2aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.943+0000 7fb7fd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc019f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.946 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc025a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc025a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:54320/0 (socket says 172.21.15.114:54320) 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 -- 172.21.15.114:0/78793284 learned_addr learned my addr 172.21.15.114:0/78793284 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fde00640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc0199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 -- 172.21.15.114:0/78793284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 msgr2=0x5616bc0199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc0199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 -- 172.21.15.114:0/78793284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 msgr2=0x5616bc019f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc019f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 -- 172.21.15.114:0/78793284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616bc025f70 con 0x5616bbc66350 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fd400640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc019f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7fe800640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc025a30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400ff70 tx=0x7fb7f4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4690abfc6e0d648 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.947 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.944+0000 7fb7db400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb7db400640 1 -- 172.21.15.114:0/78793284 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4003e00 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec00fc90 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb7db400640 1 -- 172.21.15.114:0/78793284 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fb7f40047d0 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb7db400640 1 -- 172.21.15.114:0/78793284 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb7f402e680 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb80747e740 10 client.29207 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5616bc0288e0 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.945+0000 7fb7db400640 1 -- 172.21.15.114:0/78793284 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec00fc90 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb7db400640 1 -- 172.21.15.114:0/78793284 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fb7f4027070 con 0x5616bbc66350 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb80747e740 10 client.29207 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb80747e740 10 client.29207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb80747e740 20 client.29207 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb80747e740 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbc24cd0 0x5616bc02beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb80747e740 4 client.29207 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:03.949 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616bbf85de0 con 0x5616bbc24cd0 2024-09-21T07:09:03.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.946+0000 7fb7fde00640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbc24cd0 0x5616bc02beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.947+0000 7fb7fde00640 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbc24cd0 0x5616bc02beb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51935d89fd78022a server_cookie=e8f81ca5ce64b971 in_seq=0 out_seq=0 2024-09-21T07:09:03.950 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.947+0000 7fb7db400640 10 client.29207 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb7db400640 1 -- 172.21.15.114:0/78793284 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616bbf85de0 con 0x5616bbc24cd0 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb7db400640 10 client.29207 handle_command_reply: tid=0 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 client.29207 shutdown 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 msgr2=0x5616bc025a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc025a30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400ff70 tx=0x7fb7f4004ab0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 shutdown_connections 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc025a30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc45670 0x5616bc019f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbc24cd0 0x5616bc02beb0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 --2- 172.21.15.114:0/78793284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc3d490 0x5616bc0199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb7db400640 0 client.29207 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:03.951 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 >> 172.21.15.114:0/78793284 conn(0x5616bbc5a740 msgr2=0x5616bc011090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 shutdown_connections 2024-09-21T07:09:03.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 1 -- 172.21.15.114:0/78793284 wait complete. 2024-09-21T07:09:03.952 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.948+0000 7fb80747e740 20 client.29207 trim_cache size 0 max 16384 2024-09-21T07:09:03.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 Processor -- start 2024-09-21T07:09:03.968 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 -- start start 2024-09-21T07:09:03.969 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc3d490 0x5616bc00be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc025930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 0x5616bc0169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b2f60 con 0x5616bbc66350 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b4ad0 con 0x5616bbc3d490 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb80747e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b3630 con 0x5616bbc45670 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.967+0000 7fb7fe800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 0x5616bc0169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 0x5616bc0169c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49718/0 (socket says 172.21.15.114:49718) 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 -- 172.21.15.114:0/381775348 learned_addr learned my addr 172.21.15.114:0/381775348 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fde00640 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc3d490 0x5616bc00be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fd400640 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc025930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 -- 172.21.15.114:0/381775348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 msgr2=0x5616bc025930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc025930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.970 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 -- 172.21.15.114:0/381775348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc3d490 msgr2=0x5616bc00be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc3d490 0x5616bc00be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 -- 172.21.15.114:0/381775348 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616bc010d60 con 0x5616bbc66350 2024-09-21T07:09:03.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fde00640 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc3d490 0x5616bc00be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:03.971 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7fe800640 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 0x5616bc0169c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4000f80 tx=0x7fb7f4029500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dd6190a6eee123 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.968+0000 7fb7e6600640 1 -- 172.21.15.114:0/381775348 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4029880 con 0x5616bbc66350 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb7e6600640 1 -- 172.21.15.114:0/381775348 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f4029a20 con 0x5616bbc66350 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb7e6600640 1 -- 172.21.15.114:0/381775348 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f401f430 con 0x5616bbc66350 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 -- 172.21.15.114:0/381775348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 msgr2=0x5616bc0169c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 0x5616bc0169c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4000f80 tx=0x7fb7f4029500 comp rx=0 tx=0).stop 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 -- 172.21.15.114:0/381775348 shutdown_connections 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc66350 0x5616bc0169c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc025930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.972 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 --2- 172.21.15.114:0/381775348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc3d490 0x5616bc00be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 -- 172.21.15.114:0/381775348 >> 172.21.15.114:0/381775348 conn(0x5616bbc5a740 msgr2=0x5616bc00b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.969+0000 7fb80747e740 1 -- 172.21.15.114:0/381775348 shutdown_connections 2024-09-21T07:09:03.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.970+0000 7fb80747e740 1 -- 172.21.15.114:0/381775348 wait complete. 2024-09-21T07:09:03.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.970+0000 7fb80747e740 1 -- start start 2024-09-21T07:09:03.973 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.970+0000 7fb80747e740 1 Processor -- start 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 0x5616bc0219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc021f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc01c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b4400 con 0x5616bbc3d490 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b3d00 con 0x5616bbc66350 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b0f60 con 0x5616bbc45670 2024-09-21T07:09:03.975 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb80747e740 10 client.0 _ll_register_callbacks cb 0x7fb7f4012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb7fde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 0x5616bc0219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb7fd400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc021f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb7fde00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 0x5616bc0219a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:49732/0 (socket says 172.21.15.114:49732) 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb7fde00640 1 -- 172.21.15.114:0/163328193 learned_addr learned my addr 172.21.15.114:0/163328193 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.971+0000 7fb7fe800640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc01c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fd400640 1 -- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 msgr2=0x5616bc01c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fd400640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc01c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fd400640 1 -- 172.21.15.114:0/163328193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 msgr2=0x5616bc0219a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fd400640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 0x5616bc0219a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fd400640 1 -- 172.21.15.114:0/163328193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616bc01ce90 con 0x5616bbc45670 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fde00640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 0x5616bc0219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fe800640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc01c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7fd400640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc021f20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb7e81053f0 tx=0x7fb7e802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29dc3aa1cbaa456b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:03.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7db400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7db400640 1 -- 172.21.15.114:0/163328193 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8049040 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f4029a20 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.972+0000 7fb7db400640 1 -- 172.21.15.114:0/163328193 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fb7e80c9e10 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb7db400640 1 -- 172.21.15.114:0/163328193 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb7e80c9a40 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 10 client.29164 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5616bc0970a0 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb7db400640 1 -- 172.21.15.114:0/163328193 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f4029a20 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb7db400640 1 -- 172.21.15.114:0/163328193 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fb7e810a0f0 con 0x5616bbc45670 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 10 client.29164 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 10 client.29164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 20 client.29164 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbbfa080 0x5616bc099860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 4 client.29164 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T07:09:03.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616bbf81890 con 0x5616bbbfa080 2024-09-21T07:09:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.973+0000 7fb7fde00640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbbfa080 0x5616bc099860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:03.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.974+0000 7fb7fde00640 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbbfa080 0x5616bc099860 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330c2e5f7d40fdc4 server_cookie=7bd893c220390501 in_seq=0 out_seq=0 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.974+0000 7fb7db400640 10 client.29164 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb7db400640 1 -- 172.21.15.114:0/163328193 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616bbf81890 con 0x5616bbbfa080 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb7db400640 10 client.29164 handle_command_reply: tid=0 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 client.29164 shutdown 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 msgr2=0x5616bc021f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc021f20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb7e81053f0 tx=0x7fb7e802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 shutdown_connections 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616bbc66350 0x5616bc01c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5616bbbfa080 0x5616bc099860 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616bbc45670 0x5616bc021f20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 --2- 172.21.15.114:0/163328193 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5616bbc3d490 0x5616bc0219a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb7db400640 0 client.29164 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 >> 172.21.15.114:0/163328193 conn(0x5616bbc5a740 msgr2=0x5616bc00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 shutdown_connections 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 1 -- 172.21.15.114:0/163328193 wait complete. 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.976+0000 7fb80747e740 20 client.29164 trim_cache size 0 max 16384 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "return_code": 0, 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_tag": "15610487-a10d-4566-a6d3-2ae5cb62bae1", 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": "asynchronous" 2024-09-21T07:09:03.980 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6680 msgr2=0x7fb7e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6680 0x7fb7e00b8b90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005fd0 tx=0x7fb7f4009500 comp rx=0 tx=0).stop 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 msgr2=0x7fb7f819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f819cae0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8005ab0 tx=0x7fb7e8003b90 comp rx=0 tx=0).stop 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 shutdown_connections 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fb7e00b6680 0x7fb7e00b8b90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f8109c20 0x7fb7f81a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fb7f81092d0 0x7fb7f819cae0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 --2- 172.21.15.114:0/704412403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7f8103990 0x7fb7f819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 >> 172.21.15.114:0/704412403 conn(0x7fb7f80ff3e0 msgr2=0x7fb7f81089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 shutdown_connections 2024-09-21T07:09:03.982 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:03.977+0000 7fb7ff200640 1 -- 172.21.15.114:0/704412403 wait complete. 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9baa00640 1 -- 172.21.15.114:0/1376595293 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559717e0d0b0 con 0x559717a891c0 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9baa00640 10 client.29195 handle_command_reply: tid=0 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 client.29195 shutdown 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 msgr2=0x559717eb8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717eb8390 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009560 tx=0x7fa9cc004a80 comp rx=0 tx=0).stop 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 shutdown_connections 2024-09-21T07:09:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x559717af51b0 0x559717eb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x559717a891c0 0x559717f24100 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559717ad44d0 0x559717eb88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 --2- 172.21.15.114:0/1376595293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559717acc2f0 0x559717eb8390 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 >> 172.21.15.114:0/1376595293 conn(0x559717ae9300 msgr2=0x559717ea6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.990+0000 7fa9baa00640 0 client.29195 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.991+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 shutdown_connections 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.991+0000 7fa9e2fd0740 1 -- 172.21.15.114:0/1376595293 wait complete. 2024-09-21T07:09:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.991+0000 7fa9e2fd0740 20 client.29195 trim_cache size 0 max 16384 2024-09-21T07:09:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:09:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:09:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:09:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9bc0b6680 msgr2=0x7fa9bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x559717e98a90 tx=0x7fa9c80d1050 comp rx=0 tx=0).stop 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 msgr2=0x7fa9d419c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 0x7fa9d419c4b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800e9c0 tx=0x7fa9c800ee80 comp rx=0 tx=0).stop 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 shutdown_connections 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9d410bd90 0x7fa9d41a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fa9d410b440 0x7fa9d419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 --2- 172.21.15.114:0/2532221509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9d4105b10 0x7fa9d419c4b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.992+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 >> 172.21.15.114:0/2532221509 conn(0x7fa9d4101550 msgr2=0x7fa9d4103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.993+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 shutdown_connections 2024-09-21T07:09:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:03.993+0000 7fa9daa00640 1 -- 172.21.15.114:0/2532221509 wait complete. 2024-09-21T07:09:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:04.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:04 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:04.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:04.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:04 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:04 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:04.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:04.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:04 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:04 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:04.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:04 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.034 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:09:05.403 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:05.746 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub queued for path: / 2024-09-21T07:09:05.746 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.746 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:05.746 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:05.746 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: active paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: pgmap v858: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.4 MiB/s wr, 306 op/s 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.747 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:05 smithi114 ceph-mon[30654]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub queued for path: / 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: active paths [/] 2024-09-21T07:09:05.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: pgmap v858: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.4 MiB/s wr, 306 op/s 2024-09-21T07:09:05.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.866 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:05 smithi183 ceph-mon[34219]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub queued for path: / 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: active paths [/] 2024-09-21T07:09:05.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: pgmap v858: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 2.4 MiB/s wr, 306 op/s 2024-09-21T07:09:05.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:05.913 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:05 smithi193 ceph-mon[34601]: scrub summary: idle+waiting paths [/] 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 -- 172.21.15.114:0/577550333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 msgr2=0x7fae60110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 --2- 172.21.15.114:0/577550333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 0x7fae60110470 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009ca0 tx=0x7fae5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 -- 172.21.15.114:0/577550333 shutdown_connections 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 --2- 172.21.15.114:0/577550333 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 0x7fae60110470 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 --2- 172.21.15.114:0/577550333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601092f0 0x7fae60109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 --2- 172.21.15.114:0/577550333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601039b0 0x7fae60103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 -- 172.21.15.114:0/577550333 >> 172.21.15.114:0/577550333 conn(0x7fae600ff440 msgr2=0x7fae60101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 -- 172.21.15.114:0/577550333 shutdown_connections 2024-09-21T07:09:06.085 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.083+0000 7fae67200640 1 -- 172.21.15.114:0/577550333 wait complete. 2024-09-21T07:09:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.084+0000 7fae67200640 1 Processor -- start 2024-09-21T07:09:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.084+0000 7fae67200640 1 -- start start 2024-09-21T07:09:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae67200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 0x7fae6019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae67200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601092f0 0x7fae6019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae67200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 0x7fae601a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae67200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fae60113900 con 0x7fae60109c40 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae67200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae60113780 con 0x7fae601092f0 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae67200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae60113a80 con 0x7fae601039b0 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae66800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 0x7fae601a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 0x7fae6019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 0x7fae6019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54186/0 (socket says 172.21.15.114:54186) 2024-09-21T07:09:06.087 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 -- 172.21.15.114:0/2397094050 learned_addr learned my addr 172.21.15.114:0/2397094050 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65400640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601092f0 0x7fae6019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 -- 172.21.15.114:0/2397094050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601092f0 msgr2=0x7fae6019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601092f0 0x7fae6019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 -- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 msgr2=0x7fae601a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 0x7fae601a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65e00640 1 -- 172.21.15.114:0/2397094050 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae601a2460 con 0x7fae601039b0 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.085+0000 7fae65400640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601092f0 0x7fae6019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:06.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.086+0000 7fae65e00640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 0x7fae6019c4f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.086+0000 7fae5aa00640 1 -- 172.21.15.114:0/2397094050 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5400cd80 con 0x7fae601039b0 2024-09-21T07:09:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.086+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae601a2750 con 0x7fae601039b0 2024-09-21T07:09:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.086+0000 7fae5aa00640 1 -- 172.21.15.114:0/2397094050 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae540108e0 con 0x7fae601039b0 2024-09-21T07:09:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.086+0000 7fae5aa00640 1 -- 172.21.15.114:0/2397094050 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54018980 con 0x7fae601039b0 2024-09-21T07:09:06.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.086+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601a8eb0 con 0x7fae601039b0 2024-09-21T07:09:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.089+0000 7fae6f4cf740 1 Processor -- start 2024-09-21T07:09:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.089+0000 7fae6f4cf740 1 -- start start 2024-09-21T07:09:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99818cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.091 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99819230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99451390 0x55ef99819770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ef9978ff70 con 0x55ef99472070 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ef9977f3a0 con 0x55ef99451390 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef997831b0 con 0x55ef994491b0 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae66800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99818cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae65400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99451390 0x55ef99819770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae66800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99818cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40176/0 (socket says 172.21.15.114:40176) 2024-09-21T07:09:06.092 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae66800640 1 -- 172.21.15.114:0/1145533944 learned_addr learned my addr 172.21.15.114:0/1145533944 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.090+0000 7fae5aa00640 1 -- 172.21.15.114:0/2397094050 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fae54010430 con 0x7fae601039b0 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae65400640 1 -- 172.21.15.114:0/1145533944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 msgr2=0x55ef99819230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae65400640 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99819230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae65400640 1 -- 172.21.15.114:0/1145533944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 msgr2=0x55ef99818cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae65400640 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99818cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae65400640 1 -- 172.21.15.114:0/1145533944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef99749c90 con 0x55ef99451390 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae66800640 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99818cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae65400640 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99451390 0x55ef99819770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fae6006b700 tx=0x7fae500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a30ecde2b5a51f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae4f000640 1 -- 172.21.15.114:0/1145533944 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50004310 con 0x55ef99451390 2024-09-21T07:09:06.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae4f000640 1 -- 172.21.15.114:0/1145533944 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae500044b0 con 0x55ef99451390 2024-09-21T07:09:06.094 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.091+0000 7fae4f000640 1 -- 172.21.15.114:0/1145533944 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae500035f0 con 0x55ef99451390 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae5aa00640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fae480b6680 0x7fae480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 -- 172.21.15.114:0/1145533944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99451390 msgr2=0x55ef99819770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99451390 0x55ef99819770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fae6006b700 tx=0x7fae500118d0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae5aa00640 1 -- 172.21.15.114:0/2397094050 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fae540e7e20 con 0x7fae601039b0 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 -- 172.21.15.114:0/1145533944 shutdown_connections 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99451390 0x55ef99819770 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99819230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/1145533944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99818cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae6f4cf740 1 -- 172.21.15.114:0/1145533944 >> 172.21.15.114:0/1145533944 conn(0x55ef99466460 msgr2=0x55ef99816b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.092+0000 7fae66800640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fae480b6680 0x7fae480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.093+0000 7fae6f4cf740 1 -- 172.21.15.114:0/1145533944 shutdown_connections 2024-09-21T07:09:06.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.093+0000 7fae6f4cf740 1 -- 172.21.15.114:0/1145533944 wait complete. 2024-09-21T07:09:06.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.093+0000 7fae6f4cf740 1 -- start start 2024-09-21T07:09:06.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.093+0000 7fae66800640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fae480b6680 0x7fae480b8b90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae5c035000 tx=0x7fae5c009920 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:06.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.093+0000 7fae6f4cf740 1 Processor -- start 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef99824990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef99824f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef998307f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55ef995e0700 con 0x55ef99472070 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ef9971a540 con 0x55ef994491b0 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ef998274d0 con 0x55ef99451390 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae6f4cf740 10 client.0 _ll_register_callbacks cb 0x55ef9972a430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae65e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef99824990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae65400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef99824f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae66800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef998307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae65400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef99824f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:54212/0 (socket says 172.21.15.114:54212) 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae65400640 1 -- 172.21.15.114:0/2961539266 learned_addr learned my addr 172.21.15.114:0/2961539266 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.094+0000 7fae65e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef99824990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58142/0 (socket says 172.21.15.114:58142) 2024-09-21T07:09:06.097 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae65e00640 1 -- 172.21.15.114:0/2961539266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 msgr2=0x55ef99824f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae65e00640 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef99824f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae65e00640 1 -- 172.21.15.114:0/2961539266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 msgr2=0x55ef998307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae65e00640 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef998307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae65e00640 1 -- 172.21.15.114:0/2961539266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef994dd650 con 0x55ef994491b0 2024-09-21T07:09:06.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae65e00640 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef99824990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fae540e51a0 tx=0x7fae540ad2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a0da72c648550e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:06.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.095+0000 7fae43400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:06.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.096+0000 7fae43400640 1 -- 172.21.15.114:0/2961539266 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae540ad460 con 0x55ef994491b0 2024-09-21T07:09:06.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.096+0000 7fae43400640 1 -- 172.21.15.114:0/2961539266 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fae540aebd0 con 0x55ef994491b0 2024-09-21T07:09:06.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.096+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef9971cb50 con 0x55ef994491b0 2024-09-21T07:09:06.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.096+0000 7fae43400640 1 -- 172.21.15.114:0/2961539266 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fae540aee50 con 0x55ef994491b0 2024-09-21T07:09:06.099 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.096+0000 7fae6f4cf740 10 client.29225 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.096+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ef99737910 con 0x55ef994491b0 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae43400640 1 -- 172.21.15.114:0/2961539266 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ef9971cb50 con 0x55ef994491b0 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae43400640 1 -- 172.21.15.114:0/2961539266 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fae540ac620 con 0x55ef994491b0 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae6f4cf740 10 client.29225 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae6f4cf740 10 client.29225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae6f4cf740 20 client.29225 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef994309f0 0x55ef99836890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae6f4cf740 4 client.29225 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef99791ec0 con 0x55ef994309f0 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.097+0000 7fae65400640 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef994309f0 0x55ef99836890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.098+0000 7fae43400640 10 client.29225 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:06.100 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.098+0000 7fae65400640 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef994309f0 0x55ef99836890 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98448e0118b205b server_cookie=d08593f2a15541fa in_seq=0 out_seq=0 2024-09-21T07:09:06.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae43400640 1 -- 172.21.15.114:0/2961539266 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef99791ec0 con 0x55ef994309f0 2024-09-21T07:09:06.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae43400640 10 client.29225 handle_command_reply: tid=0 2024-09-21T07:09:06.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 client.29225 shutdown 2024-09-21T07:09:06.101 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 msgr2=0x55ef99824990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef99824990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fae540e51a0 tx=0x7fae540ad2a0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 shutdown_connections 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef998307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef994309f0 0x55ef99836890 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef99824f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2961539266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef99824990 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae43400640 0 client.29225 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 >> 172.21.15.114:0/2961539266 conn(0x55ef99466460 msgr2=0x55ef9981bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 shutdown_connections 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2961539266 wait complete. 2024-09-21T07:09:06.102 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.099+0000 7fae6f4cf740 20 client.29225 trim_cache size 0 max 16384 2024-09-21T07:09:06.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 Processor -- start 2024-09-21T07:09:06.119 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 -- start start 2024-09-21T07:09:06.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef998306f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef9981f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99821980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fae601b2eb0 con 0x55ef99472070 2024-09-21T07:09:06.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae601b4a20 con 0x55ef994491b0 2024-09-21T07:09:06.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae601b3580 con 0x55ef99451390 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae65e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef998306f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae66800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99821980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae66800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99821980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40180/0 (socket says 172.21.15.114:40180) 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae65e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef998306f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58156/0 (socket says 172.21.15.114:58156) 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.118+0000 7fae66800640 1 -- 172.21.15.114:0/3796668567 learned_addr learned my addr 172.21.15.114:0/3796668567 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65400640 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef9981f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65e00640 1 -- 172.21.15.114:0/3796668567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 msgr2=0x55ef9981f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65e00640 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef9981f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65e00640 1 -- 172.21.15.114:0/3796668567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 msgr2=0x55ef99821980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.121 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65e00640 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99821980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65e00640 1 -- 172.21.15.114:0/3796668567 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef99825a50 con 0x55ef994491b0 2024-09-21T07:09:06.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae65e00640 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef998306f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fae540aabc0 tx=0x7fae540ad3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3994a3cbb4f1ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:06.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae4e600640 1 -- 172.21.15.114:0/3796668567 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54098070 con 0x55ef994491b0 2024-09-21T07:09:06.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae4e600640 1 -- 172.21.15.114:0/3796668567 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae540b2380 con 0x55ef994491b0 2024-09-21T07:09:06.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.119+0000 7fae4e600640 1 -- 172.21.15.114:0/3796668567 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae540aebd0 con 0x55ef994491b0 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 -- 172.21.15.114:0/3796668567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 msgr2=0x55ef998306f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef998306f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fae540aabc0 tx=0x7fae540ad3d0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 -- 172.21.15.114:0/3796668567 shutdown_connections 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99472070 0x55ef99821980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef99451390 0x55ef9981f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/3796668567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef994491b0 0x55ef998306f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 -- 172.21.15.114:0/3796668567 >> 172.21.15.114:0/3796668567 conn(0x55ef99466460 msgr2=0x55ef99823010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.120+0000 7fae6f4cf740 1 -- 172.21.15.114:0/3796668567 shutdown_connections 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.121+0000 7fae6f4cf740 1 -- 172.21.15.114:0/3796668567 wait complete. 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.121+0000 7fae6f4cf740 1 -- start start 2024-09-21T07:09:06.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.121+0000 7fae6f4cf740 1 Processor -- start 2024-09-21T07:09:06.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99829810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99451390 0x55ef99829d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99472070 0x55ef9982a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fae601b4350 con 0x55ef99451390 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae601b3c50 con 0x55ef99472070 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae601b0eb0 con 0x55ef994491b0 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae6f4cf740 10 client.0 _ll_register_callbacks cb 0x7fae540a9860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99451390 0x55ef99829d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99829810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99451390 0x55ef99829d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40184/0 (socket says 172.21.15.114:40184) 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65400640 1 -- 172.21.15.114:0/2039614896 learned_addr learned my addr 172.21.15.114:0/2039614896 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae66800640 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99472070 0x55ef9982a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65e00640 1 -- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99472070 msgr2=0x55ef9982a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65e00640 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99472070 0x55ef9982a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.122+0000 7fae65e00640 1 -- 172.21.15.114:0/2039614896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99451390 msgr2=0x55ef99829d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae65e00640 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99451390 0x55ef99829d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae65e00640 1 -- 172.21.15.114:0/2039614896 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef998a1370 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae65e00640 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99829810 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fae540e57a0 tx=0x7fae540aee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48523a6ffbad8528 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae43400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae43400640 1 -- 172.21.15.114:0/2039614896 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae54098030 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae43400640 1 -- 172.21.15.114:0/2039614896 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9490+0+0 (secure 0 0 0) 0x7fae540b3510 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae540b2380 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae43400640 1 -- 172.21.15.114:0/2039614896 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fae540b72c0 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae6f4cf740 10 client.29179 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55ef998a2600 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.123+0000 7fae43400640 1 -- 172.21.15.114:0/2039614896 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae540b2380 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae43400640 1 -- 172.21.15.114:0/2039614896 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7fae540b6870 con 0x55ef994491b0 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae6f4cf740 10 client.29179 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae6f4cf740 10 client.29179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae6f4cf740 20 client.29179 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef99405da0 0x55ef998a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:06.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae6f4cf740 4 client.29179 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:09:06.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ef9978d970 con 0x55ef99405da0 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.124+0000 7fae65400640 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef99405da0 0x55ef998a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.125+0000 7fae43400640 10 client.29179 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.125+0000 7fae65400640 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef99405da0 0x55ef998a49d0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea224155b5596583 server_cookie=74209e514273376c in_seq=0 out_seq=0 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.125+0000 7fae43400640 1 -- 172.21.15.114:0/2039614896 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x55ef9978d970 con 0x55ef99405da0 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.125+0000 7fae43400640 10 client.29179 handle_command_reply: tid=0 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.125+0000 7fae6f4cf740 1 client.29179 shutdown 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 msgr2=0x55ef99829810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99829810 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fae540e57a0 tx=0x7fae540aee80 comp rx=0 tx=0).stop 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 shutdown_connections 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ef99472070 0x55ef9982a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55ef99451390 0x55ef99829d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55ef99405da0 0x55ef998a49d0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 --2- 172.21.15.114:0/2039614896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ef994491b0 0x55ef99829810 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 >> 172.21.15.114:0/2039614896 conn(0x55ef99466460 msgr2=0x55ef9981de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae43400640 0 client.29179 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 shutdown_connections 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 1 -- 172.21.15.114:0/2039614896 wait complete. 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.126+0000 7fae6f4cf740 20 client.29179 trim_cache size 0 max 16384 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": { 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stdout: "15610487-a10d-4566-a6d3-2ae5cb62bae1": { 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tag": "15610487-a10d-4566-a6d3-2ae5cb62bae1", 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "options": "recursive,force" 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "15610487-a10d-4566-a6d3-2ae5cb62bae1": { 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:09:03.974081+0000", 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "68277b18-d8ea-49d2-baab-e2089ee24e2b": { 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:08:00.304805+0000", 2024-09-21T07:09:06.131 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:09:06.132 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.133 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.134 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.135 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.136 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.137 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:06:54.030598+0000", 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:06.138 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fae480b6680 msgr2=0x7fae480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fae480b6680 0x7fae480b8b90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae5c035000 tx=0x7fae5c009920 comp rx=0 tx=0).stop 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 msgr2=0x7fae6019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 0x7fae6019c4f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae5400e9f0 tx=0x7fae5400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 shutdown_connections 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fae60109c40 0x7fae601a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fae480b6680 0x7fae480b8b90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae601092f0 0x7fae6019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 --2- 172.21.15.114:0/2397094050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae601039b0 0x7fae6019c4f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:06.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 >> 172.21.15.114:0/2397094050 conn(0x7fae600ff440 msgr2=0x7fae60108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:06.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 shutdown_connections 2024-09-21T07:09:06.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:06.127+0000 7fae67200640 1 -- 172.21.15.114:0/2397094050 wait complete. 2024-09-21T07:09:07.054 INFO:tasks.cephfs.filesystem:scrub status for tag:15610487-a10d-4566-a6d3-2ae5cb62bae1 - {'path': '/', 'tag': '15610487-a10d-4566-a6d3-2ae5cb62bae1', 'options': 'recursive,force'} 2024-09-21T07:09:07.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:07 smithi114 ceph-mon[30654]: pgmap v859: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 2.3 MiB/s wr, 319 op/s 2024-09-21T07:09:07.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:07 smithi114 ceph-mon[30654]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T07:09:07.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 870.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:09:07.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:07 smithi183 ceph-mon[34219]: pgmap v859: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 2.3 MiB/s wr, 319 op/s 2024-09-21T07:09:07.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:07 smithi183 ceph-mon[34219]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T07:09:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:07 smithi193 ceph-mon[34601]: pgmap v859: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 2.3 MiB/s wr, 319 op/s 2024-09-21T07:09:07.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:07 smithi193 ceph-mon[34601]: osdmap e104: 12 total, 12 up, 12 in 2024-09-21T07:09:09.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:09 smithi114 ceph-mon[30654]: pgmap v861: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-09-21T07:09:09.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:09 smithi183 ceph-mon[34219]: pgmap v861: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-09-21T07:09:09.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:09 smithi193 ceph-mon[34601]: pgmap v861: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.2 MiB/s wr, 261 op/s 2024-09-21T07:09:11.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:11 smithi114 ceph-mon[30654]: pgmap v862: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 891 KiB/s wr, 227 op/s 2024-09-21T07:09:11.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:11 smithi183 ceph-mon[34219]: pgmap v862: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 891 KiB/s wr, 227 op/s 2024-09-21T07:09:11.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:11 smithi193 ceph-mon[34601]: pgmap v862: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 891 KiB/s wr, 227 op/s 2024-09-21T07:09:12.662 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:12.837 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 875.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-21T07:09:13.046 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:13.753 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:13 smithi114 ceph-mon[30654]: pgmap v863: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 649 KiB/s wr, 195 op/s 2024-09-21T07:09:13.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 -- 172.21.15.114:0/345838355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5490076100 msgr2=0x7f549010f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 --2- 172.21.15.114:0/345838355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5490076100 0x7f549010f9d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f547c009900 tx=0x7f547c02a410 comp rx=0 tx=0).stop 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 -- 172.21.15.114:0/345838355 shutdown_connections 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 --2- 172.21.15.114:0/345838355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f549010ff10 0x7f5490112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 --2- 172.21.15.114:0/345838355 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5490076100 0x7f549010f9d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 --2- 172.21.15.114:0/345838355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54900757b0 0x7f5490075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 -- 172.21.15.114:0/345838355 >> 172.21.15.114:0/345838355 conn(0x7f54900ff440 msgr2=0x7f5490101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.827+0000 7f5495600640 1 -- 172.21.15.114:0/345838355 shutdown_connections 2024-09-21T07:09:13.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.828+0000 7f5495600640 1 -- 172.21.15.114:0/345838355 wait complete. 2024-09-21T07:09:13.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 Processor -- start 2024-09-21T07:09:13.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 -- start start 2024-09-21T07:09:13.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54900757b0 0x7f549019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5490076100 0x7f549019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 0x7f54901a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5490115ac0 con 0x7f549010ff10 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5490115940 con 0x7f54900757b0 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.829+0000 7f5495600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5490115c40 con 0x7f5490076100 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 0x7f54901a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f548f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5490076100 0x7f549019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 0x7f54901a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40192/0 (socket says 172.21.15.114:40192) 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 -- 172.21.15.114:0/1129099096 learned_addr learned my addr 172.21.15.114:0/1129099096 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:13.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f548fe00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54900757b0 0x7f549019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 -- 172.21.15.114:0/1129099096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5490076100 msgr2=0x7f549019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5490076100 0x7f549019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 -- 172.21.15.114:0/1129099096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54900757b0 msgr2=0x7f549019c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54900757b0 0x7f549019c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.830+0000 7f5494c00640 1 -- 172.21.15.114:0/1129099096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54901a2420 con 0x7f549010ff10 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.831+0000 7f5494c00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 0x7f54901a1cf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f548800eb10 tx=0x7f548800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.831+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488004b20 con 0x7f549010ff10 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.831+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5488004cc0 con 0x7f549010ff10 2024-09-21T07:09:13.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.831+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488010670 con 0x7f549010ff10 2024-09-21T07:09:13.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.831+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54901a2710 con 0x7f549010ff10 2024-09-21T07:09:13.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.831+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54901a8e70 con 0x7f549010ff10 2024-09-21T07:09:13.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.832+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5490076cc0 con 0x7f549010ff10 2024-09-21T07:09:13.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.834+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5488005000 con 0x7f549010ff10 2024-09-21T07:09:13.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.837+0000 7f548cc00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54740b6680 0x7f54740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:13.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.837+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9578+0+0 (secure 0 0 0) 0x7f54880e82f0 con 0x7f549010ff10 2024-09-21T07:09:13.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.837+0000 7f548fe00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54740b6680 0x7f54740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:13.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.838+0000 7f548fe00640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54740b6680 0x7f54740b8b90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5480009bd0 tx=0x7f5480005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:13.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:13.839+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5488056710 con 0x7f549010ff10 2024-09-21T07:09:13.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:13 smithi183 ceph-mon[34219]: pgmap v863: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 649 KiB/s wr, 195 op/s 2024-09-21T07:09:13.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:13 smithi193 ceph-mon[34601]: pgmap v863: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 649 KiB/s wr, 195 op/s 2024-09-21T07:09:14.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.066+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54901a9040 con 0x7f549010ff10 2024-09-21T07:09:14.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.067+0000 7f548cc00640 1 -- 172.21.15.114:0/1129099096 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11990 (secure 0 0 0) 0x7f54880ae3c0 con 0x7f549010ff10 2024-09-21T07:09:14.069 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:14.069 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":59,"btime":"2024-09-21T07:08:58:432333+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:08:57.945367+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:09:14.069 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 59 2024-09-21T07:09:14.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.069+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54740b6680 msgr2=0x7f54740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:14.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.069+0000 7f5495600640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54740b6680 0x7f54740b8b90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5480009bd0 tx=0x7f5480005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:14.071 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.069+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 msgr2=0x7f54901a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.069+0000 7f5495600640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 0x7f54901a1cf0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f548800eb10 tx=0x7f548800c2d0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 shutdown_connections 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f549010ff10 0x7f54901a1cf0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5490076100 0x7f549019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f54740b6680 0x7f54740b8b90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 --2- 172.21.15.114:0/1129099096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54900757b0 0x7f549019c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 >> 172.21.15.114:0/1129099096 conn(0x7f54900ff440 msgr2=0x7f5490109ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 shutdown_connections 2024-09-21T07:09:14.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:14.070+0000 7f5495600640 1 -- 172.21.15.114:0/1129099096 wait complete. 2024-09-21T07:09:14.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:14 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1129099096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:14.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:14 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1129099096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:14.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:14 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1129099096' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:14.931 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --set-id a95b3f56 --release --await' 2024-09-21T07:09:14.931 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --set-id a95b3f56 --release --await 2024-09-21T07:09:15.306 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:15.788 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:15 smithi114 ceph-mon[30654]: pgmap v864: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 17 KiB/s wr, 114 op/s 2024-09-21T07:09:15.788 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:15 smithi114 ceph-mon[30654]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T07:09:15.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:15 smithi183 ceph-mon[34219]: pgmap v864: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 17 KiB/s wr, 114 op/s 2024-09-21T07:09:15.865 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:15 smithi183 ceph-mon[34219]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T07:09:15.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:15 smithi193 ceph-mon[34601]: pgmap v864: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 17 KiB/s wr, 114 op/s 2024-09-21T07:09:15.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:15 smithi193 ceph-mon[34601]: osdmap e105: 12 total, 12 up, 12 in 2024-09-21T07:09:15.976 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 -- 172.21.15.114:0/339937049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c110140 msgr2=0x7f812c112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 --2- 172.21.15.114:0/339937049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c110140 0x7f812c112560 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8124009ca0 tx=0x7f812402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 -- 172.21.15.114:0/339937049 shutdown_connections 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 --2- 172.21.15.114:0/339937049 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c110140 0x7f812c112560 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 --2- 172.21.15.114:0/339937049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812c076080 0x7f812c10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 --2- 172.21.15.114:0/339937049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.974+0000 7f8134600640 1 -- 172.21.15.114:0/339937049 >> 172.21.15.114:0/339937049 conn(0x7f812c07b850 msgr2=0x7f812c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.975+0000 7f8134600640 1 -- 172.21.15.114:0/339937049 shutdown_connections 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.975+0000 7f8134600640 1 -- 172.21.15.114:0/339937049 wait complete. 2024-09-21T07:09:15.977 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.976+0000 7f8134600640 1 Processor -- start 2024-09-21T07:09:15.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.976+0000 7f8134600640 1 -- start start 2024-09-21T07:09:15.978 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8134600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8134600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c076080 0x7f812c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8134600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812c110140 0x7f812c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8134600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f812c115ca0 con 0x7f812c076080 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8134600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f812c115b20 con 0x7f812c075730 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8134600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f812c115e20 con 0x7f812c110140 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c076080 0x7f812c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48618/0 (socket says 172.21.15.114:48618) 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 -- 172.21.15.114:0/218171388 learned_addr learned my addr 172.21.15.114:0/218171388 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:15.979 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8132600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812c110140 0x7f812c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 -- 172.21.15.114:0/218171388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812c110140 msgr2=0x7f812c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812c110140 0x7f812c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 -- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c076080 msgr2=0x7f812c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c076080 0x7f812c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.977+0000 7f8131c00640 1 -- 172.21.15.114:0/218171388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c1a2590 con 0x7f812c075730 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.978+0000 7f8131c00640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c19c620 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f812000e9f0 tx=0x7f812000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.978+0000 7f812a000640 1 -- 172.21.15.114:0/218171388 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812000cd80 con 0x7f812c075730 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.978+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c1a2880 con 0x7f812c075730 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.978+0000 7f812a000640 1 -- 172.21.15.114:0/218171388 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81200108e0 con 0x7f812c075730 2024-09-21T07:09:15.980 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.978+0000 7f812a000640 1 -- 172.21.15.114:0/218171388 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8120018980 con 0x7f812c075730 2024-09-21T07:09:15.981 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.978+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c1a8fe0 con 0x7f812c075730 2024-09-21T07:09:15.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.981+0000 7f813b22a740 1 Processor -- start 2024-09-21T07:09:15.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.981+0000 7f813b22a740 1 -- start start 2024-09-21T07:09:15.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f813b22a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3cff31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.983 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f813b22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc23660 0x557a3cff3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f813b22a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc2b840 0x557a3cff3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f813b22a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557a3cf6a290 con 0x557a3cc4c520 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f813b22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557a3cf596c0 con 0x557a3cc23660 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f813b22a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a3cf5d4d0 con 0x557a3cc2b840 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3cff31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3cff31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40104/0 (socket says 172.21.15.114:40104) 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 -- 172.21.15.114:0/990597773 learned_addr learned my addr 172.21.15.114:0/990597773 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8131200640 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc2b840 0x557a3cff3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.984 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 -- 172.21.15.114:0/990597773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc2b840 msgr2=0x557a3cff3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc2b840 0x557a3cff3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 -- 172.21.15.114:0/990597773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc23660 msgr2=0x557a3cff3700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc23660 0x557a3cff3700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.985 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.982+0000 7f8132600640 1 -- 172.21.15.114:0/990597773 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3cee6460 con 0x557a3cc4c520 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.983+0000 7f8132600640 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3cff31c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8124009800 tx=0x7f8124009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d641017532a65bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.983+0000 7f812a000640 1 -- 172.21.15.114:0/218171388 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8120010430 con 0x7f812c075730 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.983+0000 7f811ea00640 1 -- 172.21.15.114:0/990597773 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240305a0 con 0x557a3cc4c520 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.983+0000 7f811ea00640 1 -- 172.21.15.114:0/990597773 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124036030 con 0x557a3cc4c520 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.983+0000 7f811ea00640 1 -- 172.21.15.114:0/990597773 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124004240 con 0x557a3cc4c520 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 -- 172.21.15.114:0/990597773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 msgr2=0x557a3cff31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3cff31c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8124009800 tx=0x7f8124009830 comp rx=0 tx=0).stop 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 -- 172.21.15.114:0/990597773 shutdown_connections 2024-09-21T07:09:15.986 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc2b840 0x557a3cff3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc23660 0x557a3cff3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 --2- 172.21.15.114:0/990597773 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3cff31c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 -- 172.21.15.114:0/990597773 >> 172.21.15.114:0/990597773 conn(0x557a3cc40910 msgr2=0x557a3cff10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 -- 172.21.15.114:0/990597773 shutdown_connections 2024-09-21T07:09:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.984+0000 7f813b22a740 1 -- 172.21.15.114:0/990597773 wait complete. 2024-09-21T07:09:15.987 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.985+0000 7f812a000640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81140b6590 0x7f81140b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.985+0000 7f813b22a740 1 -- start start 2024-09-21T07:09:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.985+0000 7f812a000640 1 -- 172.21.15.114:0/218171388 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f81200e7e20 con 0x7f812c075730 2024-09-21T07:09:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.985+0000 7f8132600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81140b6590 0x7f81140b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.985+0000 7f813b22a740 1 Processor -- start 2024-09-21T07:09:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cffedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.988 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3cfff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x557a3cdbabb0 con 0x557a3cc4c520 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557a3cef46c0 con 0x557a3cc2b840 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x557a3d001940 con 0x557a3cc23660 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f813b22a740 10 client.0 _ll_register_callbacks cb 0x557a3cf0f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8132600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81140b6590 0x7f81140b8aa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8124003d60 tx=0x7f812402a8b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cffedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3cfff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cffedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37582/0 (socket says 172.21.15.114:37582) 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 -- 172.21.15.114:0/1932124225 learned_addr learned my addr 172.21.15.114:0/1932124225 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8132600640 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 -- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 msgr2=0x557a3cfff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3cfff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 -- 172.21.15.114:0/1932124225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 msgr2=0x557a3d00ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:15.989 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.986+0000 7f8131c00640 1 -- 172.21.15.114:0/1932124225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3ccb7b00 con 0x557a3cc23660 2024-09-21T07:09:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f8131c00640 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cffedb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f81200e57a0 tx=0x7f81200ad360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a9c30463de1202 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:15.990 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f8113400640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f8113400640 1 -- 172.21.15.114:0/1932124225 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200aedd0 con 0x557a3cc23660 2024-09-21T07:09:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8124036030 con 0x557a3cc23660 2024-09-21T07:09:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f8113400640 1 -- 172.21.15.114:0/1932124225 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f8120018b20 con 0x557a3cc23660 2024-09-21T07:09:15.991 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f8113400640 1 -- 172.21.15.114:0/1932124225 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8120010720 con 0x557a3cc23660 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f813b22a740 10 client.29197 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.987+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557a3cd2e960 con 0x557a3cc23660 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f8113400640 1 -- 172.21.15.114:0/1932124225 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124036030 con 0x557a3cc23660 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f8113400640 1 -- 172.21.15.114:0/1932124225 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7f81200ac620 con 0x557a3cc23660 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f813b22a740 10 client.29197 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f813b22a740 10 client.29197 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f813b22a740 20 client.29197 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f813b22a740 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cc0aea0 0x557a3d010be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f813b22a740 4 client.29197 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a3cf6c1e0 con 0x557a3cc0aea0 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.988+0000 7f8131200640 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cc0aea0 0x557a3d010be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.989+0000 7f8131200640 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cc0aea0 0x557a3d010be0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d7dc03adff7792 server_cookie=9fd1bf618792e7cf in_seq=0 out_seq=0 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.989+0000 7f8113400640 10 client.29197 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:15.992 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f8113400640 1 -- 172.21.15.114:0/1932124225 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a3cf6c1e0 con 0x557a3cc0aea0 2024-09-21T07:09:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f8113400640 10 client.29197 handle_command_reply: tid=0 2024-09-21T07:09:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 client.29197 shutdown 2024-09-21T07:09:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 msgr2=0x557a3cffedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:15.993 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cffedb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f81200e57a0 tx=0x7f81200ad360 comp rx=0 tx=0).stop 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 shutdown_connections 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3cfff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cc0aea0 0x557a3d010be0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 --2- 172.21.15.114:0/1932124225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cffedb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 >> 172.21.15.114:0/1932124225 conn(0x557a3cc40910 msgr2=0x557a3cff6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f8113400640 0 client.29197 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 shutdown_connections 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.990+0000 7f813b22a740 1 -- 172.21.15.114:0/1932124225 wait complete. 2024-09-21T07:09:15.994 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:15.991+0000 7f813b22a740 20 client.29197 trim_cache size 0 max 16384 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.008+0000 7f813b22a740 1 Processor -- start 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.008+0000 7f813b22a740 1 -- start start 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.008+0000 7f813b22a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cff4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.008+0000 7f813b22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d00ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f813b22a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f813b22a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1b3040 con 0x557a3cc4c520 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f813b22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1b4bb0 con 0x557a3cc2b840 2024-09-21T07:09:16.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f813b22a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1b3710 con 0x557a3cc23660 2024-09-21T07:09:16.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40124/0 (socket says 172.21.15.114:40124) 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 -- 172.21.15.114:0/2234979080 learned_addr learned my addr 172.21.15.114:0/2234979080 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8131c00640 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cff4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8131200640 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d00ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 -- 172.21.15.114:0/2234979080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 msgr2=0x557a3cff4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cff4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 -- 172.21.15.114:0/2234979080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 msgr2=0x557a3d00ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d00ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.009+0000 7f8132600640 1 -- 172.21.15.114:0/2234979080 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3d00b5b0 con 0x557a3cc4c520 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.010+0000 7f8132600640 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00b070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8124003ed0 tx=0x7f8124030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23289d34f22ff609 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.010+0000 7f811e000640 1 -- 172.21.15.114:0/2234979080 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124030800 con 0x557a3cc4c520 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.010+0000 7f811e000640 1 -- 172.21.15.114:0/2234979080 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124060070 con 0x557a3cc4c520 2024-09-21T07:09:16.012 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.010+0000 7f811e000640 1 -- 172.21.15.114:0/2234979080 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812404f460 con 0x557a3cc4c520 2024-09-21T07:09:16.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 -- 172.21.15.114:0/2234979080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 msgr2=0x557a3d00b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:16.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00b070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8124003ed0 tx=0x7f8124030fe0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 -- 172.21.15.114:0/2234979080 shutdown_connections 2024-09-21T07:09:16.013 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d00b070 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d00ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 --2- 172.21.15.114:0/2234979080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3cff4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 -- 172.21.15.114:0/2234979080 >> 172.21.15.114:0/2234979080 conn(0x557a3cc40910 msgr2=0x557a3cff6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 -- 172.21.15.114:0/2234979080 shutdown_connections 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 -- 172.21.15.114:0/2234979080 wait complete. 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.011+0000 7f813b22a740 1 -- start start 2024-09-21T07:09:16.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 Processor -- start 2024-09-21T07:09:16.015 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3d007200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1b44e0 con 0x557a3cc4c520 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1b3de0 con 0x557a3cc2b840 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f812c1b1040 con 0x557a3cc23660 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f813b22a740 10 client.0 _ll_register_callbacks cb 0x7f8120009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.012+0000 7f8132600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8131c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3d007200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d003ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:40134/0 (socket says 172.21.15.114:40134) 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 -- 172.21.15.114:0/3438799387 learned_addr learned my addr 172.21.15.114:0/3438799387 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8131200640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 -- 172.21.15.114:0/3438799387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 msgr2=0x557a3d007200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3d007200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 -- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 msgr2=0x557a3d003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 -- 172.21.15.114:0/3438799387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3d0042b0 con 0x557a3cc4c520 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8131c00640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3d007200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8132600640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d003ca0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f812405d040 tx=0x7f8124030c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8e252e121debf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.013+0000 7f8113400640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f8113400640 1 -- 172.21.15.114:0/3438799387 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124068070 con 0x557a3cc4c520 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8124060070 con 0x557a3cc4c520 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f8113400640 1 -- 172.21.15.114:0/3438799387 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f8124003f00 con 0x557a3cc4c520 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f8113400640 1 -- 172.21.15.114:0/3438799387 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8124074050 con 0x557a3cc4c520 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 10 client.18576 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x557a3d07c3c0 con 0x557a3cc4c520 2024-09-21T07:09:16.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f8113400640 1 -- 172.21.15.114:0/3438799387 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124060070 con 0x557a3cc4c520 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f8113400640 1 -- 172.21.15.114:0/3438799387 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7f8124043070 con 0x557a3cc4c520 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 10 client.18576 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 10 client.18576 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 20 client.18576 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cbe0250 0x557a3d07e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 4 client.18576 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a95b3f56", "release": true, "await": true}] 2024-09-21T07:09:16.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.014+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a95b3f56", "release": true, "await": true}) -- 0x557a3cf67c90 con 0x557a3cbe0250 2024-09-21T07:09:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.015+0000 7f8131c00640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cbe0250 0x557a3d07e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.015+0000 7f8131c00640 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cbe0250 0x557a3d07e700 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6086df686b507751 server_cookie=bfb4944152133144 in_seq=0 out_seq=0 2024-09-21T07:09:16.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.015+0000 7f8113400640 10 client.18576 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:16.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f8113400640 1 -- 172.21.15.114:0/3438799387 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557a3cf67c90 con 0x557a3cbe0250 2024-09-21T07:09:16.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f8113400640 10 client.18576 handle_command_reply: tid=0 2024-09-21T07:09:16.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 client.18576 shutdown 2024-09-21T07:09:16.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 msgr2=0x557a3d003ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:16.220 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d003ca0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f812405d040 tx=0x7f8124030c00 comp rx=0 tx=0).stop 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 shutdown_connections 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x557a3cc4c520 0x557a3d003ca0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557a3cc2b840 0x557a3d003760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x557a3cbe0250 0x557a3d07e700 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 --2- 172.21.15.114:0/3438799387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557a3cc23660 0x557a3d007200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 >> 172.21.15.114:0/3438799387 conn(0x557a3cc40910 msgr2=0x557a3d000080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.218+0000 7f8113400640 0 client.18576 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.219+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 shutdown_connections 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.219+0000 7f813b22a740 1 -- 172.21.15.114:0/3438799387 wait complete. 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.219+0000 7f813b22a740 20 client.18576 trim_cache size 0 max 16384 2024-09-21T07:09:16.221 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 59, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 60, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": { 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "a95b3f56": { 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "version": 60, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "age_ref": 0.0, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "timeout": 90.0, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "expiration": 138.8, 2024-09-21T07:09:16.222 INFO:teuthology.orchestra.run.smithi114.stdout: "members": { 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: "file:/": { 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: "excluded": false, 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: "state": { 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RELEASED", 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: "age": 0.0 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:16.223 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81140b6590 msgr2=0x7f81140b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81140b6590 0x7f81140b8aa0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8124003d60 tx=0x7f812402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 msgr2=0x7f812c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c19c620 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f812000e9f0 tx=0x7f812000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 shutdown_connections 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f812c110140 0x7f812c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f812c076080 0x7f812c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f81140b6590 0x7f81140b8aa0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 --2- 172.21.15.114:0/218171388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f812c075730 0x7f812c19c620 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 >> 172.21.15.114:0/218171388 conn(0x7f812c07b850 msgr2=0x7f812c109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 shutdown_connections 2024-09-21T07:09:16.224 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:16.220+0000 7f8134600640 1 -- 172.21.15.114:0/218171388 wait complete. 2024-09-21T07:09:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.304+0000 7f8846e00640 1 -- 172.21.15.114:0/1503405768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840109d60 msgr2=0x7f8840110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.304+0000 7f8846e00640 1 --2- 172.21.15.114:0/1503405768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840109d60 0x7f8840110590 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8838009ca0 tx=0x7f883802a8e0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.305+0000 7f8846e00640 1 -- 172.21.15.114:0/1503405768 shutdown_connections 2024-09-21T07:09:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.305+0000 7f8846e00640 1 --2- 172.21.15.114:0/1503405768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840109d60 0x7f8840110590 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.305+0000 7f8846e00640 1 --2- 172.21.15.114:0/1503405768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109410 0x7f8840109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.305+0000 7f8846e00640 1 --2- 172.21.15.114:0/1503405768 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8840103ae0 0x7f8840103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.305+0000 7f8846e00640 1 -- 172.21.15.114:0/1503405768 >> 172.21.15.114:0/1503405768 conn(0x7f88400ff520 msgr2=0x7f8840101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.305+0000 7f8846e00640 1 -- 172.21.15.114:0/1503405768 shutdown_connections 2024-09-21T07:09:17.307 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.306+0000 7f8846e00640 1 -- 172.21.15.114:0/1503405768 wait complete. 2024-09-21T07:09:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.307+0000 7f8846e00640 1 Processor -- start 2024-09-21T07:09:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.308+0000 7f8846e00640 1 -- start start 2024-09-21T07:09:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.309+0000 7f8846e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840103ae0 0x7f8840074e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.309+0000 7f8846e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8840109410 0x7f884006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.309+0000 7f8846e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 0x7f884006e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.309+0000 7f8846e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8840113ab0 con 0x7f8840109410 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.309+0000 7f8846e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8840113930 con 0x7f8840103ae0 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.309+0000 7f8846e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8840113c30 con 0x7f8840109d60 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 0x7f884006e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 0x7f884006e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37616/0 (socket says 172.21.15.114:37616) 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 -- 172.21.15.114:0/3119639648 learned_addr learned my addr 172.21.15.114:0/3119639648 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f883fe00640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840103ae0 0x7f8840074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f883f400640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8840109410 0x7f884006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 -- 172.21.15.114:0/3119639648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840103ae0 msgr2=0x7f8840074e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840103ae0 0x7f8840074e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 -- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8840109410 msgr2=0x7f884006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8840109410 0x7f884006e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.310+0000 7f8844e00640 1 -- 172.21.15.114:0/3119639648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884006eeb0 con 0x7f8840109d60 2024-09-21T07:09:17.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.312+0000 7f8844e00640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 0x7f884006e690 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8838003bd0 tx=0x7f8838003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.316+0000 7f883cc00640 1 -- 172.21.15.114:0/3119639648 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883804b070 con 0x7f8840109d60 2024-09-21T07:09:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.316+0000 7f883cc00640 1 -- 172.21.15.114:0/3119639648 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8838031d80 con 0x7f8840109d60 2024-09-21T07:09:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.316+0000 7f883cc00640 1 -- 172.21.15.114:0/3119639648 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883804b070 con 0x7f8840109d60 2024-09-21T07:09:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.316+0000 7f8846e00640 1 -- 172.21.15.114:0/3119639648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8840073b10 con 0x7f8840109d60 2024-09-21T07:09:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.316+0000 7f8846e00640 1 -- 172.21.15.114:0/3119639648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8840074090 con 0x7f8840109d60 2024-09-21T07:09:17.323 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.321+0000 7f884e039740 1 Processor -- start 2024-09-21T07:09:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.326+0000 7f883cc00640 1 -- 172.21.15.114:0/3119639648 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f883804e070 con 0x7f8840109d60 2024-09-21T07:09:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.328+0000 7f883cc00640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f88240b6a80 0x7f88240b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.329+0000 7f884e039740 1 -- start start 2024-09-21T07:09:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.329+0000 7f883f400640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f88240b6a80 0x7f88240b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.330+0000 7f883cc00640 1 -- 172.21.15.114:0/3119639648 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f883803d790 con 0x7f8840109d60 2024-09-21T07:09:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f884e039740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003501b0 0x5598006f7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f883f400640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f88240b6a80 0x7f88240b8f90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f882c006fd0 tx=0x7f882c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f884e039740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x5598006f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f884e039740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006f5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f884e039740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55980066a6b0 con 0x5598003272f0 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f884e039740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559800659ae0 con 0x55980032f4d0 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f884e039740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55980065d8f0 con 0x5598003501b0 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f883fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003501b0 0x5598006f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f883fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003501b0 0x5598006f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37626/0 (socket says 172.21.15.114:37626) 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f883fe00640 1 -- 172.21.15.114:0/854511236 learned_addr learned my addr 172.21.15.114:0/854511236 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f8844e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.331+0000 7f883f400640 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x5598006f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.332+0000 7f8844e00640 1 -- 172.21.15.114:0/854511236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003501b0 msgr2=0x5598006f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.332+0000 7f8844e00640 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003501b0 0x5598006f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.332+0000 7f8844e00640 1 -- 172.21.15.114:0/854511236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 msgr2=0x5598006f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.332+0000 7f8844e00640 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x5598006f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.332+0000 7f8844e00640 1 -- 172.21.15.114:0/854511236 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559800626780 con 0x55980032f4d0 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.332+0000 7f883f400640 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x5598006f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.333+0000 7f8844e00640 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006f5510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8838109500 tx=0x7f88380d99f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf532dccdea15a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:17.335 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.333+0000 7f8846e00640 1 -- 172.21.15.114:0/854511236 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380ce3f0 con 0x55980032f4d0 2024-09-21T07:09:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.333+0000 7f8846e00640 1 -- 172.21.15.114:0/854511236 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8838036070 con 0x55980032f4d0 2024-09-21T07:09:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.333+0000 7f8846e00640 1 -- 172.21.15.114:0/854511236 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380ce590 con 0x55980032f4d0 2024-09-21T07:09:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.335+0000 7f884e039740 1 -- 172.21.15.114:0/854511236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 msgr2=0x5598006f5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.335+0000 7f884e039740 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006f5510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8838109500 tx=0x7f88380d99f0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.337+0000 7f884e039740 1 -- 172.21.15.114:0/854511236 shutdown_connections 2024-09-21T07:09:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.337+0000 7f884e039740 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006f5510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.337+0000 7f884e039740 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x5598006f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.342+0000 7f884e039740 1 --2- 172.21.15.114:0/854511236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003501b0 0x5598006f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.342+0000 7f884e039740 1 -- 172.21.15.114:0/854511236 >> 172.21.15.114:0/854511236 conn(0x559800344300 msgr2=0x5598006f2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.343+0000 7f884e039740 1 -- 172.21.15.114:0/854511236 shutdown_connections 2024-09-21T07:09:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.343+0000 7f884e039740 1 -- 172.21.15.114:0/854511236 wait complete. 2024-09-21T07:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.344+0000 7f884e039740 1 -- start start 2024-09-21T07:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.344+0000 7f884e039740 1 Processor -- start 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x559800700590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55980032f4d0 0x559800700ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x55980070c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x559800591ab0 con 0x5598003272f0 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559800600570 con 0x5598003501b0 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55980065d8f0 con 0x55980032f4d0 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f884e039740 10 client.0 _ll_register_callbacks cb 0x55980060f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f8844e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x55980070c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f8844e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x55980070c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48686/0 (socket says 172.21.15.114:48686) 2024-09-21T07:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f883fe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x559800700590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.345+0000 7f8844e00640 1 -- 172.21.15.114:0/740398116 learned_addr learned my addr 172.21.15.114:0/740398116 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.346+0000 7f8844e00640 1 -- 172.21.15.114:0/740398116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55980032f4d0 msgr2=0x559800700ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.346+0000 7f883f400640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55980032f4d0 0x559800700ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.347+0000 7f8844e00640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55980032f4d0 0x559800700ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.347+0000 7f8844e00640 1 -- 172.21.15.114:0/740398116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 msgr2=0x559800700590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.347+0000 7f8844e00640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x559800700590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.347+0000 7f8844e00640 1 -- 172.21.15.114:0/740398116 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598003bb790 con 0x5598003501b0 2024-09-21T07:09:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.347+0000 7f883fe00640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x559800700590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.348+0000 7f8844e00640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x55980070c480 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8838003fd0 tx=0x7f8838009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f4a3299c37cde8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.348+0000 7f882a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.350+0000 7f882a000640 1 -- 172.21.15.114:0/740398116 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883804b070 con 0x5598003501b0 2024-09-21T07:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.350+0000 7f882a000640 1 -- 172.21.15.114:0/740398116 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f8838109780 con 0x5598003501b0 2024-09-21T07:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.351+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8838036070 con 0x5598003501b0 2024-09-21T07:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.351+0000 7f882a000640 1 -- 172.21.15.114:0/740398116 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f88380ce760 con 0x5598003501b0 2024-09-21T07:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.351+0000 7f884e039740 10 client.29264 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.351+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x559800434620 con 0x5598003501b0 2024-09-21T07:09:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.355+0000 7f882a000640 1 -- 172.21.15.114:0/740398116 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8838036070 con 0x5598003501b0 2024-09-21T07:09:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.356+0000 7f882a000640 1 -- 172.21.15.114:0/740398116 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7f8838045030 con 0x5598003501b0 2024-09-21T07:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.359+0000 7f884e039740 10 client.29264 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.359+0000 7f884e039740 10 client.29264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.359+0000 7f884e039740 20 client.29264 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.359+0000 7f884e039740 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55980030ed80 0x559800722fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.359+0000 7f884e039740 4 client.29264 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.359+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980066c600 con 0x55980030ed80 2024-09-21T07:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.361+0000 7f883fe00640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55980030ed80 0x559800722fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.362+0000 7f883fe00640 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55980030ed80 0x559800722fe0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a84291f4b77ca server_cookie=32396728e46283ab in_seq=0 out_seq=0 2024-09-21T07:09:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.362+0000 7f882a000640 10 client.29264 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.365+0000 7f882a000640 1 -- 172.21.15.114:0/740398116 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55980066c600 con 0x55980030ed80 2024-09-21T07:09:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.365+0000 7f882a000640 10 client.29264 handle_command_reply: tid=0 2024-09-21T07:09:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.366+0000 7f884e039740 1 client.29264 shutdown 2024-09-21T07:09:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 msgr2=0x55980070c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x55980070c480 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8838003fd0 tx=0x7f8838009410 comp rx=0 tx=0).stop 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 shutdown_connections 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x55980070c480 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55980030ed80 0x559800722fe0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55980032f4d0 0x559800700ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 --2- 172.21.15.114:0/740398116 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003272f0 0x559800700590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 >> 172.21.15.114:0/740398116 conn(0x559800344300 msgr2=0x5598006f6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.367+0000 7f882a000640 0 client.29264 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.368+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 shutdown_connections 2024-09-21T07:09:17.369 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.368+0000 7f884e039740 1 -- 172.21.15.114:0/740398116 wait complete. 2024-09-21T07:09:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.369+0000 7f884e039740 20 client.29264 trim_cache size 0 max 16384 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.404+0000 7f884e039740 1 Processor -- start 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.404+0000 7f884e039740 1 -- start start 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.404+0000 7f884e039740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x559800700a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f884e039740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55980032f4d0 0x55980070c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f884e039740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x5598006f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f884e039740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b7b70 con 0x55980032f4d0 2024-09-21T07:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f884e039740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b8240 con 0x5598003501b0 2024-09-21T07:09:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f884e039740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b8940 con 0x5598003272f0 2024-09-21T07:09:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f883f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55980032f4d0 0x55980070c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f883fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x559800700a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f883fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x559800700a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37642/0 (socket says 172.21.15.114:37642) 2024-09-21T07:09:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f883fe00640 1 -- 172.21.15.114:0/461169923 learned_addr learned my addr 172.21.15.114:0/461169923 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.405+0000 7f8844e00640 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x5598006f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.406+0000 7f883fe00640 1 -- 172.21.15.114:0/461169923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 msgr2=0x5598006f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.406+0000 7f883fe00640 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x5598006f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.406+0000 7f883fe00640 1 -- 172.21.15.114:0/461169923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55980032f4d0 msgr2=0x55980070c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.406+0000 7f883fe00640 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55980032f4d0 0x55980070c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.406+0000 7f883fe00640 1 -- 172.21.15.114:0/461169923 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980030efb0 con 0x5598003272f0 2024-09-21T07:09:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.407+0000 7f883fe00640 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x559800700a70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8830000c00 tx=0x7f8830014840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c9c5b2ec801385 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.410+0000 7f8834c00640 1 -- 172.21.15.114:0/461169923 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88300141d0 con 0x5598003272f0 2024-09-21T07:09:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.410+0000 7f8834c00640 1 -- 172.21.15.114:0/461169923 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8830012900 con 0x5598003272f0 2024-09-21T07:09:17.412 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.410+0000 7f8834c00640 1 -- 172.21.15.114:0/461169923 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883001d880 con 0x5598003272f0 2024-09-21T07:09:17.412 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a95b3f56', total seconds elapsed: 23.6 2024-09-21T07:09:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.411+0000 7f884e039740 1 -- 172.21.15.114:0/461169923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 msgr2=0x559800700a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.411+0000 7f884e039740 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x559800700a70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8830000c00 tx=0x7f8830014840 comp rx=0 tx=0).stop 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 -- 172.21.15.114:0/461169923 shutdown_connections 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598003501b0 0x5598006f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55980032f4d0 0x55980070c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 --2- 172.21.15.114:0/461169923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x559800700a70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 -- 172.21.15.114:0/461169923 >> 172.21.15.114:0/461169923 conn(0x559800344300 msgr2=0x5598006f5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 -- 172.21.15.114:0/461169923 shutdown_connections 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.414+0000 7f884e039740 1 -- 172.21.15.114:0/461169923 wait complete. 2024-09-21T07:09:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.415+0000 7f884e039740 1 -- start start 2024-09-21T07:09:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.415+0000 7f884e039740 1 Processor -- start 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.416+0000 7f884e039740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x5598006fafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.416+0000 7f884e039740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.416+0000 7f883fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x5598006fafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006fb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48698/0 (socket says 172.21.15.114:48698) 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x5598006fafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:37656/0 (socket says 172.21.15.114:37656) 2024-09-21T07:09:17.418 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 -- 172.21.15.114:0/4173452201 learned_addr learned my addr 172.21.15.114:0/4173452201 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003501b0 0x55980078b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b5af0 con 0x5598003501b0 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88401b5460 con 0x55980032f4d0 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559800659ae0 con 0x5598003272f0 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f884e039740 10 client.0 _ll_register_callbacks cb 0x7f882c009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f8844e00640 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003501b0 0x55980078b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 -- 172.21.15.114:0/4173452201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 msgr2=0x5598006fafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.419 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x5598006fafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 -- 172.21.15.114:0/4173452201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003501b0 msgr2=0x55980078b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003501b0 0x55980078b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.420 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.417+0000 7f883f400640 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980078be50 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f883f400640 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006fb520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f884006c8d0 tx=0x7f882c0046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c302ed945e6ec14d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f882a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f882a000640 1 -- 172.21.15.114:0/4173452201 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c0126b0 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f882a000640 1 -- 172.21.15.114:0/4173452201 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f882c012850 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8830012900 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f882a000640 1 -- 172.21.15.114:0/4173452201 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f882c012a50 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.418+0000 7f884e039740 10 client.29270 fetch_fsmap learned FSMap version 59 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55980078d0e0 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f882a000640 1 -- 172.21.15.114:0/4173452201 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8830012900 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f882a000640 1 -- 172.21.15.114:0/4173452201 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 59) ==== 8374+0+0 (secure 0 0 0) 0x7f882c026070 con 0x55980032f4d0 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 10 client.29270 fetch_fsmap finished waiting for FSMap version 59 2024-09-21T07:09:17.422 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 10 client.29270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 20 client.29270 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5598002e41c0 0x55980078f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 4 client.29270 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.419+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598006680b0 con 0x5598002e41c0 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.420+0000 7f883fe00640 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5598002e41c0 0x55980078f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.421+0000 7f883fe00640 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5598002e41c0 0x55980078f810 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d922cf5b942ef09 server_cookie=c8f1c9d75030bbc1 in_seq=0 out_seq=0 2024-09-21T07:09:17.423 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.421+0000 7f882a000640 10 client.29270 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:17.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:17 smithi114 ceph-mon[30654]: pgmap v866: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 17 KiB/s wr, 87 op/s 2024-09-21T07:09:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.762+0000 7f882a000640 1 -- 172.21.15.114:0/4173452201 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598006680b0 con 0x5598002e41c0 2024-09-21T07:09:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.762+0000 7f882a000640 10 client.29270 handle_command_reply: tid=0 2024-09-21T07:09:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.762+0000 7f884e039740 1 client.29270 shutdown 2024-09-21T07:09:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.762+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 msgr2=0x5598006fb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.762+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006fb520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f884006c8d0 tx=0x7f882c0046e0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 shutdown_connections 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5598003501b0 0x55980078b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55980032f4d0 0x5598006fb520 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5598002e41c0 0x55980078f810 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f884e039740 1 --2- 172.21.15.114:0/4173452201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5598003272f0 0x5598006fafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 >> 172.21.15.114:0/4173452201 conn(0x559800344300 msgr2=0x5598006f66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:17.765 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.763+0000 7f882a000640 0 client.29270 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.764+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 shutdown_connections 2024-09-21T07:09:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.764+0000 7f884e039740 1 -- 172.21.15.114:0/4173452201 wait complete. 2024-09-21T07:09:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.764+0000 7f884e039740 20 client.29270 trim_cache size 0 max 16384 2024-09-21T07:09:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:09:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:09:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:09:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.768+0000 7f8836000640 1 -- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f88240b6a80 msgr2=0x7f88240b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.768+0000 7f8836000640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f88240b6a80 0x7f88240b8f90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f882c006fd0 tx=0x7f882c008040 comp rx=0 tx=0).stop 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.769+0000 7f8836000640 1 -- 172.21.15.114:0/3119639648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 msgr2=0x7f884006e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.769+0000 7f8836000640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 0x7f884006e690 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8838003bd0 tx=0x7f8838003c00 comp rx=0 tx=0).stop 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 -- 172.21.15.114:0/3119639648 shutdown_connections 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8840109d60 0x7f884006e690 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8840109410 0x7f884006e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f88240b6a80 0x7f88240b8f90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 --2- 172.21.15.114:0/3119639648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8840103ae0 0x7f8840074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:17.773 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 -- 172.21.15.114:0/3119639648 >> 172.21.15.114:0/3119639648 conn(0x7f88400ff520 msgr2=0x7f8840108a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 -- 172.21.15.114:0/3119639648 shutdown_connections 2024-09-21T07:09:17.774 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:17.771+0000 7f8836000640 1 -- 172.21.15.114:0/3119639648 wait complete. 2024-09-21T07:09:17.833 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 880.0 s, 103.3 tps, lat 3470.541 ms stddev 6472.384 2024-09-21T07:09:17.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:17 smithi183 ceph-mon[34219]: pgmap v866: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 17 KiB/s wr, 87 op/s 2024-09-21T07:09:17.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:17 smithi193 ceph-mon[34601]: pgmap v866: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 17 KiB/s wr, 87 op/s 2024-09-21T07:09:19.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:19 smithi114 ceph-mon[30654]: pgmap v867: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 254 KiB/s wr, 50 op/s 2024-09-21T07:09:19.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:19 smithi183 ceph-mon[34219]: pgmap v867: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 254 KiB/s wr, 50 op/s 2024-09-21T07:09:19.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:19 smithi193 ceph-mon[34601]: pgmap v867: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 254 KiB/s wr, 50 op/s 2024-09-21T07:09:21.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:21 smithi114 ceph-mon[30654]: pgmap v868: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 528 KiB/s wr, 61 op/s 2024-09-21T07:09:21.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:21 smithi183 ceph-mon[34219]: pgmap v868: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 528 KiB/s wr, 61 op/s 2024-09-21T07:09:21.911 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:21 smithi193 ceph-mon[34601]: pgmap v868: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 528 KiB/s wr, 61 op/s 2024-09-21T07:09:22.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 885.0 s, 428.3 tps, lat 239.029 ms stddev 308.206 2024-09-21T07:09:23.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:23 smithi114 ceph-mon[30654]: pgmap v869: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 852 KiB/s wr, 109 op/s 2024-09-21T07:09:23.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:23 smithi183 ceph-mon[34219]: pgmap v869: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 852 KiB/s wr, 109 op/s 2024-09-21T07:09:23.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:23 smithi193 ceph-mon[34601]: pgmap v869: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 852 KiB/s wr, 109 op/s 2024-09-21T07:09:24.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:24 smithi114 ceph-mon[30654]: pgmap v870: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-09-21T07:09:24.864 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:24 smithi183 ceph-mon[34219]: pgmap v870: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-09-21T07:09:24.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:24 smithi193 ceph-mon[34601]: pgmap v870: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 2.3 MiB/s wr, 273 op/s 2024-09-21T07:09:26.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:25 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:09:26.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:25 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:09:26.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:25 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:09:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:09:27.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:09:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:09:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:26 smithi183 ceph-mon[34219]: scrub summary: idle 2024-09-21T07:09:27.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:26 smithi183 ceph-mon[34219]: pgmap v871: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-09-21T07:09:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:09:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:09:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:09:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:26 smithi193 ceph-mon[34601]: scrub summary: idle 2024-09-21T07:09:27.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:26 smithi193 ceph-mon[34601]: pgmap v871: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-09-21T07:09:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:09:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:09:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:09:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:26 smithi114 ceph-mon[30654]: scrub summary: idle 2024-09-21T07:09:27.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:26 smithi114 ceph-mon[30654]: pgmap v871: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 2.2 MiB/s wr, 269 op/s 2024-09-21T07:09:27.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 890.0 s, 456.0 tps, lat 218.274 ms stddev 256.745 2024-09-21T07:09:29.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:29 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:29.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:29 smithi114 ceph-mon[30654]: pgmap v872: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-09-21T07:09:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:29 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:29.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:29 smithi183 ceph-mon[34219]: pgmap v872: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-09-21T07:09:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:29 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:29 smithi193 ceph-mon[34601]: pgmap v872: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 3.3 MiB/s wr, 370 op/s 2024-09-21T07:09:30.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:30 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:30.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:30 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:30.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:30 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:09:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 -- 172.21.15.114:0/910965619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9010be80 msgr2=0x7f5a9010fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 --2- 172.21.15.114:0/910965619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9010be80 0x7f5a9010fa00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b9f0 tx=0x7f5a8002c0a0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.115 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 -- 172.21.15.114:0/910965619 shutdown_connections 2024-09-21T07:09:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 --2- 172.21.15.114:0/910965619 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a90079680 0x7f5a9007baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 --2- 172.21.15.114:0/910965619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010c370 0x7f5a9010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 --2- 172.21.15.114:0/910965619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9010be80 0x7f5a9010fa00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.112+0000 7f5a96600640 1 -- 172.21.15.114:0/910965619 >> 172.21.15.114:0/910965619 conn(0x7f5a90070dd0 msgr2=0x7f5a900711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.113+0000 7f5a96600640 1 -- 172.21.15.114:0/910965619 shutdown_connections 2024-09-21T07:09:31.116 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.113+0000 7f5a96600640 1 -- 172.21.15.114:0/910965619 wait complete. 2024-09-21T07:09:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.115+0000 7f5a96600640 1 Processor -- start 2024-09-21T07:09:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.115+0000 7f5a96600640 1 -- start start 2024-09-21T07:09:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.115+0000 7f5a96600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 0x7f5a90080650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.115+0000 7f5a96600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010be80 0x7f5a90080b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.115+0000 7f5a96600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a9010c370 0x7f5a900810f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a8f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 0x7f5a90080650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a8f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 0x7f5a90080650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:50386/0 (socket says 172.21.15.114:50386) 2024-09-21T07:09:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a96600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9007ead0 con 0x7f5a9010c370 2024-09-21T07:09:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a96600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9007e950 con 0x7f5a90079680 2024-09-21T07:09:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a96600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9007ec50 con 0x7f5a9010be80 2024-09-21T07:09:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a8f400640 1 -- 172.21.15.114:0/3691552420 learned_addr learned my addr 172.21.15.114:0/3691552420 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a8ea00640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010be80 0x7f5a90080b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.116+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a9010c370 0x7f5a900810f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.117+0000 7f5a8ea00640 1 -- 172.21.15.114:0/3691552420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 msgr2=0x7f5a90080650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.117+0000 7f5a8ea00640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 0x7f5a90080650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.117+0000 7f5a8ea00640 1 -- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a9010c370 msgr2=0x7f5a900810f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.117+0000 7f5a8ea00640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a9010c370 0x7f5a900810f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.117+0000 7f5a8ea00640 1 -- 172.21.15.114:0/3691552420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a901c6320 con 0x7f5a9010be80 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.117+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 0x7f5a90080650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.118+0000 7f5a8ea00640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010be80 0x7f5a90080b90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006c850 tx=0x7f5a7c00ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.118+0000 7f5a87e00640 1 -- 172.21.15.114:0/3691552420 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c01e070 con 0x7f5a9010be80 2024-09-21T07:09:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.118+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a901c6610 con 0x7f5a9010be80 2024-09-21T07:09:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.119+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a901c6bb0 con 0x7f5a9010be80 2024-09-21T07:09:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.122+0000 7f5a87e00640 1 -- 172.21.15.114:0/3691552420 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a7c00dd70 con 0x7f5a9010be80 2024-09-21T07:09:31.124 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.122+0000 7f5a87e00640 1 -- 172.21.15.114:0/3691552420 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c01e070 con 0x7f5a9010be80 2024-09-21T07:09:31.133 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.131+0000 7f5a9d66b740 1 Processor -- start 2024-09-21T07:09:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.131+0000 7f5a9d66b740 1 -- start start 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.132+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c375574c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.132+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c375575170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.132+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751af4d0 0x55c3755756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.132+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c3754ea6b0 con 0x55c3751af4d0 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.132+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c3754d9ae0 con 0x55c3751a72f0 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.132+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c3754dd8f0 con 0x55c3751d01b0 2024-09-21T07:09:31.135 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.133+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751af4d0 0x55c3755756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.133+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751af4d0 0x55c3755756b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:52998/0 (socket says 172.21.15.114:52998) 2024-09-21T07:09:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.133+0000 7f5a8ea00640 1 -- 172.21.15.114:0/4167344565 learned_addr learned my addr 172.21.15.114:0/4167344565 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:31.139 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.133+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c375574c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.137+0000 7f5a8fe00640 1 -- 172.21.15.114:0/4167344565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 msgr2=0x55c375575170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.137+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c375575170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.137+0000 7f5a8fe00640 1 -- 172.21.15.114:0/4167344565 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751af4d0 msgr2=0x55c3755756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.137+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751af4d0 0x55c3755756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.140 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.137+0000 7f5a8fe00640 1 -- 172.21.15.114:0/4167344565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3754a6780 con 0x55c3751d01b0 2024-09-21T07:09:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.139+0000 7f5a87e00640 1 -- 172.21.15.114:0/3691552420 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f5a7c021070 con 0x7f5a9010be80 2024-09-21T07:09:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.141+0000 7f5a87e00640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a740b6980 0x7f5a740b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.141+0000 7f5a87e00640 1 -- 172.21.15.114:0/3691552420 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f5a7c0ec880 con 0x7f5a9010be80 2024-09-21T07:09:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.142+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a740b6980 0x7f5a740b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.143+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c375574c30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5a88005fd0 tx=0x7f5a88009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d2850094eb6248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.146 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.144+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a740b6980 0x7f5a740b8e90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55c3755738b0 tx=0x7f5a8002e040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.147+0000 7f5a84c00640 1 -- 172.21.15.114:0/4167344565 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88003b70 con 0x55c3751d01b0 2024-09-21T07:09:31.149 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.147+0000 7f5a84c00640 1 -- 172.21.15.114:0/4167344565 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88003d10 con 0x55c3751d01b0 2024-09-21T07:09:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.147+0000 7f5a84c00640 1 -- 172.21.15.114:0/4167344565 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88011920 con 0x55c3751d01b0 2024-09-21T07:09:31.150 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.148+0000 7f5a9d66b740 1 -- 172.21.15.114:0/4167344565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 msgr2=0x55c375574c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.151 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.148+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c375574c30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5a88005fd0 tx=0x7f5a88009f90 comp rx=0 tx=0).stop 2024-09-21T07:09:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.149+0000 7f5a9d66b740 1 -- 172.21.15.114:0/4167344565 shutdown_connections 2024-09-21T07:09:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.149+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751af4d0 0x55c3755756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.149+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c375575170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.149+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/4167344565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c375574c30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.152 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.149+0000 7f5a9d66b740 1 -- 172.21.15.114:0/4167344565 >> 172.21.15.114:0/4167344565 conn(0x55c3751c4300 msgr2=0x55c375572b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.153+0000 7f5a9d66b740 1 -- 172.21.15.114:0/4167344565 shutdown_connections 2024-09-21T07:09:31.155 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.153+0000 7f5a9d66b740 1 -- 172.21.15.114:0/4167344565 wait complete. 2024-09-21T07:09:31.156 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.154+0000 7f5a9d66b740 1 -- start start 2024-09-21T07:09:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.154+0000 7f5a9d66b740 1 Processor -- start 2024-09-21T07:09:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c3755802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751af4d0 0x55c3755807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c37558c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.158 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55c375411ab0 con 0x55c3751a72f0 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c375480570 con 0x55c3751af4d0 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c375583400 con 0x55c3751d01b0 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c37558c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c37558c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34672/0 (socket says 172.21.15.114:34672) 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a8fe00640 1 -- 172.21.15.114:0/1432976597 learned_addr learned my addr 172.21.15.114:0/1432976597 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.155+0000 7f5a9d66b740 10 client.0 _ll_register_callbacks cb 0x55c37548f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a8fe00640 1 -- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751af4d0 msgr2=0x55c3755807e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751af4d0 0x55c3755807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a8fe00640 1 -- 172.21.15.114:0/1432976597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 msgr2=0x55c3755802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c3755802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a8fe00640 1 -- 172.21.15.114:0/1432976597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c37523b790 con 0x55c3751d01b0 2024-09-21T07:09:31.159 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c37558c780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5a88011520 tx=0x7f5a88004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a9dde937aab91a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.160 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.156+0000 7f5a79600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.157+0000 7f5a79600640 1 -- 172.21.15.114:0/1432976597 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88009b90 con 0x55c3751d01b0 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.157+0000 7f5a79600640 1 -- 172.21.15.114:0/1432976597 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f5a880040d0 con 0x55c3751d01b0 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.157+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a88003d10 con 0x55c3751d01b0 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.158+0000 7f5a79600640 1 -- 172.21.15.114:0/1432976597 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f5a8802e050 con 0x55c3751d01b0 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.158+0000 7f5a9d66b740 10 client.29227 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.158+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55c3752b4620 con 0x55c3751d01b0 2024-09-21T07:09:31.162 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.158+0000 7f5a79600640 1 -- 172.21.15.114:0/1432976597 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88011660 con 0x55c3751d01b0 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.158+0000 7f5a79600640 1 -- 172.21.15.114:0/1432976597 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f5a88005040 con 0x55c3751d01b0 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a9d66b740 10 client.29227 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a9d66b740 10 client.29227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a9d66b740 20 client.29227 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c37518ed80 0x55c375592380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a9d66b740 4 client.29227 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a88003d10 con 0x55c37518ed80 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.159+0000 7f5a8f400640 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c37518ed80 0x55c375592380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.163 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.161+0000 7f5a8f400640 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c37518ed80 0x55c375592380 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80fb72ae74608024 server_cookie=2cad9be940f7fd7a in_seq=0 out_seq=0 2024-09-21T07:09:31.164 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.161+0000 7f5a79600640 10 client.29227 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.163+0000 7f5a79600640 1 -- 172.21.15.114:0/1432976597 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5a88003d10 con 0x55c37518ed80 2024-09-21T07:09:31.165 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.163+0000 7f5a79600640 10 client.29227 handle_command_reply: tid=0 2024-09-21T07:09:31.166 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.164+0000 7f5a9d66b740 1 client.29227 shutdown 2024-09-21T07:09:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.165+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 msgr2=0x55c37558c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.165+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c37558c780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5a88011520 tx=0x7f5a88004c70 comp rx=0 tx=0).stop 2024-09-21T07:09:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.165+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 shutdown_connections 2024-09-21T07:09:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.165+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c37518ed80 0x55c375592380 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.165+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751d01b0 0x55c37558c780 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.165+0000 7f5a79600640 0 client.29227 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.167+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751af4d0 0x55c3755807e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.167+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/1432976597 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c3755802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.167+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 >> 172.21.15.114:0/1432976597 conn(0x55c3751c4300 msgr2=0x55c375575c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.167+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 shutdown_connections 2024-09-21T07:09:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.167+0000 7f5a9d66b740 1 -- 172.21.15.114:0/1432976597 wait complete. 2024-09-21T07:09:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.167+0000 7f5a9d66b740 20 client.29227 trim_cache size 0 max 16384 2024-09-21T07:09:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.214+0000 7f5a9d66b740 1 Processor -- start 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.214+0000 7f5a9d66b740 1 -- start start 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.214+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c37518ed80 0x55c375582180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c37557d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c37557d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7000ffc0 con 0x55c3751a72f0 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70011b00 con 0x55c37518ed80 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70010690 con 0x55c3751af4d0 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c37557d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c37557d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34686/0 (socket says 172.21.15.114:34686) 2024-09-21T07:09:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a8fe00640 1 -- 172.21.15.114:0/3164702386 learned_addr learned my addr 172.21.15.114:0/3164702386 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c37518ed80 0x55c375582180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.215+0000 7f5a8ea00640 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c37557d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.216+0000 7f5a8f400640 1 -- 172.21.15.114:0/3164702386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 msgr2=0x55c37557d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.216+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c37557d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.219 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.216+0000 7f5a8f400640 1 -- 172.21.15.114:0/3164702386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 msgr2=0x55c37557d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.216+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c37557d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.216+0000 7f5a8f400640 1 -- 172.21.15.114:0/3164702386 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c37558cd90 con 0x55c37518ed80 2024-09-21T07:09:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.216+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c37518ed80 0x55c375582180 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5a900860e0 tx=0x7f5a800056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a83aafd8a918fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.217+0000 7f5a7be00640 1 -- 172.21.15.114:0/3164702386 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a800096e0 con 0x55c37518ed80 2024-09-21T07:09:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.218+0000 7f5a7be00640 1 -- 172.21.15.114:0/3164702386 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8003a040 con 0x55c37518ed80 2024-09-21T07:09:31.220 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.218+0000 7f5a7be00640 1 -- 172.21.15.114:0/3164702386 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8004c450 con 0x55c37518ed80 2024-09-21T07:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.221+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3164702386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c37518ed80 msgr2=0x55c375582180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.223 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.221+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c37518ed80 0x55c375582180 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f5a900860e0 tx=0x7f5a800056d0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.222+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3164702386 shutdown_connections 2024-09-21T07:09:31.224 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.222+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c37557d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.222+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c3751a72f0 0x55c37557d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.222+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3164702386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c37518ed80 0x55c375582180 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.222+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3164702386 >> 172.21.15.114:0/3164702386 conn(0x55c3751c4300 msgr2=0x55c37558b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.223+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3164702386 shutdown_connections 2024-09-21T07:09:31.226 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.224+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3164702386 wait complete. 2024-09-21T07:09:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.224+0000 7f5a9d66b740 1 -- start start 2024-09-21T07:09:31.227 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.225+0000 7f5a9d66b740 1 Processor -- start 2024-09-21T07:09:31.229 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.227+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c37518ed80 0x55c3755f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.227+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c3755f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.230 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a9d66b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c3755f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a9d66b740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70011430 con 0x55c37518ed80 2024-09-21T07:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a8ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c3755f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c3755f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a8fe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c3755f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34690/0 (socket says 172.21.15.114:34690) 2024-09-21T07:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a8fe00640 1 -- 172.21.15.114:0/3727351202 learned_addr learned my addr 172.21.15.114:0/3727351202 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:31.231 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70010d60 con 0x55c3751a72f0 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.228+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7000dfc0 con 0x55c3751af4d0 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a9d66b740 10 client.0 _ll_register_callbacks cb 0x7f5a88009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8fe00640 1 -- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 msgr2=0x55c3755f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c3755f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8fe00640 1 -- 172.21.15.114:0/3727351202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c37518ed80 msgr2=0x55c3755f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c37518ed80 0x55c3755f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8fe00640 1 -- 172.21.15.114:0/3727351202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c375586840 con 0x55c3751af4d0 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8ea00640 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c3755f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a8fe00640 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c3755f85f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5a88002230 tx=0x7f5a88003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3604d8b81888831 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:31.233 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.229+0000 7f5a79600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.230+0000 7f5a79600640 1 -- 172.21.15.114:0/3727351202 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88037060 con 0x55c3751af4d0 2024-09-21T07:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.230+0000 7f5a79600640 1 -- 172.21.15.114:0/3727351202 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f5a8802b070 con 0x55c3751af4d0 2024-09-21T07:09:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.231+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a8003a040 con 0x55c3751af4d0 2024-09-21T07:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.235+0000 7f5a79600640 1 -- 172.21.15.114:0/3727351202 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f5a880058b0 con 0x55c3751af4d0 2024-09-21T07:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.235+0000 7f5a9d66b740 10 client.29233 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:31.237 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.235+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55c37560dee0 con 0x55c3751af4d0 2024-09-21T07:09:31.238 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.235+0000 7f5a79600640 1 -- 172.21.15.114:0/3727351202 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8003a040 con 0x55c3751af4d0 2024-09-21T07:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a79600640 1 -- 172.21.15.114:0/3727351202 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f5a880046d0 con 0x55c3751af4d0 2024-09-21T07:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a9d66b740 10 client.29233 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a9d66b740 10 client.29233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a9d66b740 20 client.29233 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:31.239 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c3751d01b0 0x55c375610670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a9d66b740 4 client.29233 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:09:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3754ec600 con 0x55c3751d01b0 2024-09-21T07:09:31.240 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.237+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c3751d01b0 0x55c375610670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.238+0000 7f5a8f400640 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c3751d01b0 0x55c375610670 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f818fbd0823d15 server_cookie=99f2b2ceb1aa064b in_seq=0 out_seq=0 2024-09-21T07:09:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.238+0000 7f5a79600640 10 client.29233 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.280+0000 7f5a79600640 1 -- 172.21.15.114:0/3727351202 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3754ec600 con 0x55c3751d01b0 2024-09-21T07:09:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.280+0000 7f5a79600640 10 client.29233 handle_command_reply: tid=0 2024-09-21T07:09:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.280+0000 7f5a9d66b740 1 client.29233 shutdown 2024-09-21T07:09:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 msgr2=0x55c3755f85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c3755f85f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f5a88002230 tx=0x7f5a88003d70 comp rx=0 tx=0).stop 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 shutdown_connections 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55c3751d01b0 0x55c375610670 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c3751af4d0 0x55c3755f85f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3751a72f0 0x55c3755f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 --2- 172.21.15.114:0/3727351202 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55c37518ed80 0x55c3755f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 >> 172.21.15.114:0/3727351202 conn(0x55c3751c4300 msgr2=0x55c375581ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a79600640 0 client.29233 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.281+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 shutdown_connections 2024-09-21T07:09:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.282+0000 7f5a9d66b740 1 -- 172.21.15.114:0/3727351202 wait complete. 2024-09-21T07:09:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.282+0000 7f5a9d66b740 20 client.29233 trim_cache size 0 max 16384 2024-09-21T07:09:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:09:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:09:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:09:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:09:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a740b6980 msgr2=0x7f5a740b8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a740b6980 0x7f5a740b8e90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55c3755738b0 tx=0x7f5a8002e040 comp rx=0 tx=0).stop 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010be80 msgr2=0x7f5a90080b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010be80 0x7f5a90080b90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5a9006c850 tx=0x7f5a7c00ff70 comp rx=0 tx=0).stop 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 shutdown_connections 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f5a9010c370 0x7f5a900810f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a9010be80 0x7f5a90080b90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f5a740b6980 0x7f5a740b8e90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 --2- 172.21.15.114:0/3691552420 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a90079680 0x7f5a90080650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.283+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 >> 172.21.15.114:0/3691552420 conn(0x7f5a90070dd0 msgr2=0x7f5a9010c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.284+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 shutdown_connections 2024-09-21T07:09:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:31.284+0000 7f5a96600640 1 -- 172.21.15.114:0/3691552420 wait complete. 2024-09-21T07:09:31.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:31 smithi114 ceph-mon[30654]: pgmap v873: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 3.6 MiB/s wr, 421 op/s 2024-09-21T07:09:31.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:31 smithi183 ceph-mon[34219]: pgmap v873: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 3.6 MiB/s wr, 421 op/s 2024-09-21T07:09:31.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:31 smithi193 ceph-mon[34601]: pgmap v873: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 3.6 MiB/s wr, 421 op/s 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi114 to 176.4M 2024-09-21T07:09:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: Unable to set osd_memory_target on smithi114 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-21T07:09:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:32.317 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:09:32.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi114 to 176.4M 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: Unable to set osd_memory_target on smithi114 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:32.365 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T07:09:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi114 to 176.4M 2024-09-21T07:09:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: Unable to set osd_memory_target on smithi114 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-21T07:09:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:09:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:09:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:09:32.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 895.0 s, 393.4 tps, lat 260.898 ms stddev 289.927 2024-09-21T07:09:33.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:33 smithi183 ceph-mon[34219]: pgmap v874: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-09-21T07:09:33.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:33 smithi193 ceph-mon[34601]: pgmap v874: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-09-21T07:09:33.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:33 smithi114 ceph-mon[30654]: pgmap v874: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-09-21T07:09:35.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:35 smithi193 ceph-mon[34601]: pgmap v875: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 6.5 MiB/s wr, 594 op/s 2024-09-21T07:09:35.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:35 smithi114 ceph-mon[30654]: pgmap v875: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 6.5 MiB/s wr, 594 op/s 2024-09-21T07:09:35.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:35 smithi183 ceph-mon[34219]: pgmap v875: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 6.5 MiB/s wr, 594 op/s 2024-09-21T07:09:37.055 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T07:09:37.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:37 smithi114 ceph-mon[30654]: pgmap v876: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 5.4 MiB/s wr, 484 op/s 2024-09-21T07:09:37.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:37 smithi183 ceph-mon[34219]: pgmap v876: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 5.4 MiB/s wr, 484 op/s 2024-09-21T07:09:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:37 smithi193 ceph-mon[34601]: pgmap v876: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 5.4 MiB/s wr, 484 op/s 2024-09-21T07:09:37.756 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:37.832 INFO:teuthology.orchestra.run.smithi114.stderr:progress: 900.0 s, 423.0 tps, lat 231.992 ms stddev 305.174 2024-09-21T07:09:38.167 INFO:teuthology.orchestra.run.smithi114.stdout:transaction type: 2024-09-21T07:09:38.167 INFO:teuthology.orchestra.run.smithi114.stdout:scaling factor: 32 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:query mode: simple 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:number of clients: 100 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:number of threads: 4 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:duration: 900 s 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:number of transactions actually processed: 513734 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:latency average = 175.173 ms 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:latency stddev = 973.579 ms 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:tps = 570.603621 (including connections establishing) 2024-09-21T07:09:38.168 INFO:teuthology.orchestra.run.smithi114.stdout:tps = 570.607376 (excluding connections establishing) 2024-09-21T07:09:38.174 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 -- 172.21.15.114:0/1609464635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f38109c40 msgr2=0x7f4f38110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1609464635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f38109c40 0x7f4f38110470 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009be0 tx=0x7f4f3002a6e0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 -- 172.21.15.114:0/1609464635 shutdown_connections 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1609464635 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f38109c40 0x7f4f38110470 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1609464635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f381092f0 0x7f4f38109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1609464635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.002 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.998+0000 7f4f37e00640 1 -- 172.21.15.114:0/1609464635 >> 172.21.15.114:0/1609464635 conn(0x7f4f380ff440 msgr2=0x7f4f38101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:39.003 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.999+0000 7f4f37e00640 1 -- 172.21.15.114:0/1609464635 shutdown_connections 2024-09-21T07:09:39.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:38.999+0000 7f4f37e00640 1 -- 172.21.15.114:0/1609464635 wait complete. 2024-09-21T07:09:39.006 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.001+0000 7f4f37e00640 1 Processor -- start 2024-09-21T07:09:39.007 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.004+0000 7f4f37e00640 1 -- start start 2024-09-21T07:09:39.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.005+0000 7f4f37e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.008 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.005+0000 7f4f37e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f381092f0 0x7f4f381992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f37e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f38109c40 0x7f4f3819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59802/0 (socket says 172.21.15.114:59802) 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38113900 con 0x7f4f381092f0 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38113780 con 0x7f4f381039b0 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38113a80 con 0x7f4f38109c40 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36a00640 1 -- 172.21.15.114:0/1196265699 learned_addr learned my addr 172.21.15.114:0/1196265699 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f37400640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f38109c40 0x7f4f3819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.009 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36000640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f381092f0 0x7f4f381992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36a00640 1 -- 172.21.15.114:0/1196265699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f38109c40 msgr2=0x7f4f3819fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36a00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f38109c40 0x7f4f3819fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.010 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.006+0000 7f4f36a00640 1 -- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f381092f0 msgr2=0x7f4f381992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.008+0000 7f4f36a00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f381092f0 0x7f4f381992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.008+0000 7f4f36a00640 1 -- 172.21.15.114:0/1196265699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f38199bc0 con 0x7f4f381039b0 2024-09-21T07:09:39.011 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.008+0000 7f4f36000640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f381092f0 0x7f4f381992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:09:39.014 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.012+0000 7f4f36a00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38198db0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9f0 tx=0x7f4f2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:39.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.013+0000 7f4f2f400640 1 -- 172.21.15.114:0/1196265699 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800cd80 con 0x7f4f381039b0 2024-09-21T07:09:39.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.013+0000 7f4f2f400640 1 -- 172.21.15.114:0/1196265699 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f280108e0 con 0x7f4f381039b0 2024-09-21T07:09:39.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.013+0000 7f4f2f400640 1 -- 172.21.15.114:0/1196265699 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28018980 con 0x7f4f381039b0 2024-09-21T07:09:39.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.013+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f381a0470 con 0x7f4f381039b0 2024-09-21T07:09:39.016 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.013+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381a0930 con 0x7f4f381039b0 2024-09-21T07:09:39.030 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.028+0000 7f4f2f400640 1 -- 172.21.15.114:0/1196265699 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4f28018b20 con 0x7f4f381039b0 2024-09-21T07:09:39.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.031+0000 7f4f2f400640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f1c0b6700 0x7f4f1c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.032+0000 7f4f2f400640 1 -- 172.21.15.114:0/1196265699 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f4f280e8d90 con 0x7f4f381039b0 2024-09-21T07:09:39.034 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.032+0000 7f4f36000640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f1c0b6700 0x7f4f1c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.041+0000 7f4f36000640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f1c0b6700 0x7f4f1c0b8c10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4f38107df0 tx=0x7f4f24009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:39.051 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.048+0000 7f4f44671740 1 Processor -- start 2024-09-21T07:09:39.055 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 -- start start 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcce070 0x55f04c074cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 0x55f04c075230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c075770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f04bfebf70 con 0x55f04bcad390 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f04bfdb3a0 con 0x55f04bca51b0 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.051+0000 7f4f44671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f04bfdf1b0 con 0x55f04bcce070 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 0x55f04c075230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 0x55f04c075230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59808/0 (socket says 172.21.15.114:59808) 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 -- 172.21.15.114:0/2865467739 learned_addr learned my addr 172.21.15.114:0/2865467739 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 -- 172.21.15.114:0/2865467739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcce070 msgr2=0x55f04c074cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcce070 0x55f04c074cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.056 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 -- 172.21.15.114:0/2865467739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 msgr2=0x55f04c075770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.052+0000 7f4f36000640 1 -- 172.21.15.114:0/2865467739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f04bfa5c90 con 0x55f04bca51b0 2024-09-21T07:09:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.053+0000 7f4f36000640 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 0x55f04c075230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4f24010e60 tx=0x7f4f24013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb0673ba5d4ab1f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.053+0000 7f4f23e00640 1 -- 172.21.15.114:0/2865467739 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24023070 con 0x55f04bca51b0 2024-09-21T07:09:39.057 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.053+0000 7f4f23e00640 1 -- 172.21.15.114:0/2865467739 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f24010e90 con 0x55f04bca51b0 2024-09-21T07:09:39.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.055+0000 7f4f44671740 1 -- 172.21.15.114:0/2865467739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 msgr2=0x55f04c075230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.055+0000 7f4f23e00640 1 -- 172.21.15.114:0/2865467739 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2401f450 con 0x55f04bca51b0 2024-09-21T07:09:39.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.055+0000 7f4f44671740 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 0x55f04c075230 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4f24010e60 tx=0x7f4f24013000 comp rx=0 tx=0).stop 2024-09-21T07:09:39.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.055+0000 7f4f44671740 1 -- 172.21.15.114:0/2865467739 shutdown_connections 2024-09-21T07:09:39.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.055+0000 7f4f44671740 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.055+0000 7f4f44671740 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bca51b0 0x55f04c075230 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.056+0000 7f4f44671740 1 --2- 172.21.15.114:0/2865467739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcce070 0x55f04c074cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.056+0000 7f4f44671740 1 -- 172.21.15.114:0/2865467739 >> 172.21.15.114:0/2865467739 conn(0x55f04bcc2460 msgr2=0x55f04c072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:39.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.056+0000 7f4f44671740 1 -- 172.21.15.114:0/2865467739 shutdown_connections 2024-09-21T07:09:39.059 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.056+0000 7f4f44671740 1 -- 172.21.15.114:0/2865467739 wait complete. 2024-09-21T07:09:39.061 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.059+0000 7f4f44671740 1 -- start start 2024-09-21T07:09:39.062 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.060+0000 7f4f44671740 1 Processor -- start 2024-09-21T07:09:39.065 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.063+0000 7f4f44671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 0x55f04c0808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.063+0000 7f4f44671740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c080e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.066 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 0x55f04c0808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f36000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c080e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f44671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c09d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f44671740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55f04be3c700 con 0x55f04bcad390 2024-09-21T07:09:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f44671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f04bf76540 con 0x55f04bcce070 2024-09-21T07:09:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f44671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f04c094530 con 0x55f04bca51b0 2024-09-21T07:09:39.067 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.064+0000 7f4f44671740 10 client.0 _ll_register_callbacks cb 0x55f04bf86430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:39.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.065+0000 7f4f36000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c080e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47420/0 (socket says 172.21.15.114:47420) 2024-09-21T07:09:39.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.065+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 0x55f04c0808c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40072/0 (socket says 172.21.15.114:40072) 2024-09-21T07:09:39.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.065+0000 7f4f36a00640 1 -- 172.21.15.114:0/261404173 learned_addr learned my addr 172.21.15.114:0/261404173 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:39.068 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.065+0000 7f4f37400640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c09d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.066+0000 7f4f36a00640 1 -- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 msgr2=0x55f04c09d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.066+0000 7f4f36a00640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c09d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.066+0000 7f4f36a00640 1 -- 172.21.15.114:0/261404173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 msgr2=0x55f04c080e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.066+0000 7f4f36a00640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c080e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.066+0000 7f4f36a00640 1 -- 172.21.15.114:0/261404173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f04bd39650 con 0x55f04bca51b0 2024-09-21T07:09:39.069 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.066+0000 7f4f36000640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c080e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:09:39.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.067+0000 7f4f36a00640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 0x55f04c0808c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4f280aade0 tx=0x7f4f280aad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fde0b6dd60b0753 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:39.070 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.067+0000 7f4f20c00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:39.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.068+0000 7f4f20c00640 1 -- 172.21.15.114:0/261404173 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280aef10 con 0x55f04bca51b0 2024-09-21T07:09:39.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.068+0000 7f4f20c00640 1 -- 172.21.15.114:0/261404173 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f4f280aa160 con 0x55f04bca51b0 2024-09-21T07:09:39.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.068+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f04bf78b50 con 0x55f04bca51b0 2024-09-21T07:09:39.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.068+0000 7f4f20c00640 1 -- 172.21.15.114:0/261404173 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4f280b32f0 con 0x55f04bca51b0 2024-09-21T07:09:39.072 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.068+0000 7f4f44671740 10 client.29242 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.068+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f04bf93910 con 0x55f04bca51b0 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.069+0000 7f4f20c00640 1 -- 172.21.15.114:0/261404173 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f04bf78b50 con 0x55f04bca51b0 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.069+0000 7f4f20c00640 1 -- 172.21.15.114:0/261404173 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f4f280b4480 con 0x55f04bca51b0 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f44671740 10 client.29242 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f44671740 10 client.29242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f44671740 20 client.29242 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:39.073 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f44671740 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc8c9f0 0x55f04c0a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f44671740 4 client.29242 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:39.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f04bfedec0 con 0x55f04bc8c9f0 2024-09-21T07:09:39.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.070+0000 7f4f36000640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc8c9f0 0x55f04c0a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.072+0000 7f4f36000640 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc8c9f0 0x55f04c0a38f0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6eaf259b9f136e6 server_cookie=87ab716772991520 in_seq=0 out_seq=0 2024-09-21T07:09:39.074 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.072+0000 7f4f20c00640 10 client.29242 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.073+0000 7f4f20c00640 1 -- 172.21.15.114:0/261404173 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f04bfedec0 con 0x55f04bc8c9f0 2024-09-21T07:09:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.073+0000 7f4f20c00640 10 client.29242 handle_command_reply: tid=0 2024-09-21T07:09:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.073+0000 7f4f44671740 1 client.29242 shutdown 2024-09-21T07:09:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 msgr2=0x55f04c0808c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.076 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 0x55f04c0808c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4f280aade0 tx=0x7f4f280aad10 comp rx=0 tx=0).stop 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 shutdown_connections 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c09d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bcad390 0x55f04c080e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc8c9f0 0x55f04c0a38f0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 --2- 172.21.15.114:0/261404173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bca51b0 0x55f04c0808c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f20c00640 0 client.29242 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:39.077 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.074+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 >> 172.21.15.114:0/261404173 conn(0x55f04bcc2460 msgr2=0x55f04c077e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:39.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.077+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 shutdown_connections 2024-09-21T07:09:39.079 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.077+0000 7f4f44671740 1 -- 172.21.15.114:0/261404173 wait complete. 2024-09-21T07:09:39.080 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.077+0000 7f4f44671740 20 client.29242 trim_cache size 0 max 16384 2024-09-21T07:09:39.120 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.117+0000 7f4f44671740 1 Processor -- start 2024-09-21T07:09:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 -- start start 2024-09-21T07:09:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c0718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c07d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c07ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b7480 con 0x55f04bca51b0 2024-09-21T07:09:39.122 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b8fc0 con 0x55f04bcce070 2024-09-21T07:09:39.123 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.119+0000 7f4f44671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b7b50 con 0x55f04bcad390 2024-09-21T07:09:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c07d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c07d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40080/0 (socket says 172.21.15.114:40080) 2024-09-21T07:09:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f36000640 1 -- 172.21.15.114:0/1780605985 learned_addr learned my addr 172.21.15.114:0/1780605985 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:39.125 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f37400640 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c07ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f36a00640 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c0718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f37400640 1 -- 172.21.15.114:0/1780605985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 msgr2=0x55f04c07d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f37400640 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c07d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f37400640 1 -- 172.21.15.114:0/1780605985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 msgr2=0x55f04c0718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f37400640 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c0718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.120+0000 7f4f37400640 1 -- 172.21.15.114:0/1780605985 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f04c0730d0 con 0x55f04bcce070 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.121+0000 7f4f37400640 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c07ddf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4f30034040 tx=0x7f4f300305c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb54fe0da4956765 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.121+0000 7f4f23400640 1 -- 172.21.15.114:0/1780605985 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30042070 con 0x55f04bcce070 2024-09-21T07:09:39.126 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.122+0000 7f4f23400640 1 -- 172.21.15.114:0/1780605985 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30009800 con 0x55f04bcce070 2024-09-21T07:09:39.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.123+0000 7f4f23400640 1 -- 172.21.15.114:0/1780605985 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30036040 con 0x55f04bcce070 2024-09-21T07:09:39.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.123+0000 7f4f44671740 1 -- 172.21.15.114:0/1780605985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 msgr2=0x55f04c07ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.123+0000 7f4f44671740 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c07ddf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4f30034040 tx=0x7f4f300305c0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.127 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.124+0000 7f4f44671740 1 -- 172.21.15.114:0/1780605985 shutdown_connections 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.124+0000 7f4f44671740 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c07ddf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.124+0000 7f4f44671740 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c07d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.124+0000 7f4f44671740 1 --2- 172.21.15.114:0/1780605985 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c0718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.124+0000 7f4f44671740 1 -- 172.21.15.114:0/1780605985 >> 172.21.15.114:0/1780605985 conn(0x55f04bcc2460 msgr2=0x55f04c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.124+0000 7f4f44671740 1 -- 172.21.15.114:0/1780605985 shutdown_connections 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.125+0000 7f4f44671740 1 -- 172.21.15.114:0/1780605985 wait complete. 2024-09-21T07:09:39.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.126+0000 7f4f44671740 1 -- start start 2024-09-21T07:09:39.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.126+0000 7f4f44671740 1 Processor -- start 2024-09-21T07:09:39.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c099fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c094790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c094cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b88f0 con 0x55f04bca51b0 2024-09-21T07:09:39.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b8220 con 0x55f04bcce070 2024-09-21T07:09:39.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38103dc0 con 0x55f04bcad390 2024-09-21T07:09:39.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.128+0000 7f4f44671740 10 client.0 _ll_register_callbacks cb 0x7f4f280ae650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:39.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f36000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c094790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c094cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c094cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59834/0 (socket says 172.21.15.114:59834) 2024-09-21T07:09:39.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 -- 172.21.15.114:0/1522764009 learned_addr learned my addr 172.21.15.114:0/1522764009 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 -- 172.21.15.114:0/1522764009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 msgr2=0x55f04c094790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c094790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 -- 172.21.15.114:0/1522764009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 msgr2=0x55f04c099fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c099fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.129+0000 7f4f37400640 1 -- 172.21.15.114:0/1522764009 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f04c0a3c60 con 0x55f04bcce070 2024-09-21T07:09:39.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.130+0000 7f4f20c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:39.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.131+0000 7f4f37400640 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c094cd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009b30 tx=0x7f4f30030ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac890739fe5182ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:39.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.131+0000 7f4f20c00640 1 -- 172.21.15.114:0/1522764009 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30054060 con 0x55f04bcce070 2024-09-21T07:09:39.135 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.132+0000 7f4f20c00640 1 -- 172.21.15.114:0/1522764009 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f4f30031a90 con 0x55f04bcce070 2024-09-21T07:09:39.137 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.135+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f30009800 con 0x55f04bcce070 2024-09-21T07:09:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.136+0000 7f4f20c00640 1 -- 172.21.15.114:0/1522764009 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f4f30036040 con 0x55f04bcce070 2024-09-21T07:09:39.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.136+0000 7f4f44671740 10 client.29297 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:39.140 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.136+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55f04c10efa0 con 0x55f04bcce070 2024-09-21T07:09:39.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.138+0000 7f4f20c00640 1 -- 172.21.15.114:0/1522764009 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f30009800 con 0x55f04bcce070 2024-09-21T07:09:39.141 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f20c00640 1 -- 172.21.15.114:0/1522764009 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f4f30004540 con 0x55f04bcce070 2024-09-21T07:09:39.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f44671740 10 client.29297 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:39.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f44671740 10 client.29297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:39.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f44671740 20 client.29297 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:39.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f44671740 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc61da0 0x55f04c111690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:39.142 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f44671740 4 client.29297 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T07:09:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.139+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f04bfe9970 con 0x55f04bc61da0 2024-09-21T07:09:39.143 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.140+0000 7f4f36a00640 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc61da0 0x55f04c111690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:39.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.142+0000 7f4f36a00640 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc61da0 0x55f04c111690 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d665204db94976 server_cookie=8a874b1046873c43 in_seq=0 out_seq=0 2024-09-21T07:09:39.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.142+0000 7f4f20c00640 10 client.29297 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:39.145 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.143+0000 7f4f20c00640 1 -- 172.21.15.114:0/1522764009 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55f04bfe9970 con 0x55f04bc61da0 2024-09-21T07:09:39.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.143+0000 7f4f20c00640 10 client.29297 handle_command_reply: tid=0 2024-09-21T07:09:39.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.143+0000 7f4f44671740 1 client.29297 shutdown 2024-09-21T07:09:39.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 msgr2=0x55f04c094cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c094cd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009b30 tx=0x7f4f30030ba0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.146 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 shutdown_connections 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f04bcce070 0x55f04c094cd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55f04bc61da0 0x55f04c111690 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f04bcad390 0x55f04c094790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 --2- 172.21.15.114:0/1522764009 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55f04bca51b0 0x55f04c099fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.144+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 >> 172.21.15.114:0/1522764009 conn(0x55f04bcc2460 msgr2=0x55f04c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.145+0000 7f4f20c00640 0 client.29297 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:39.148 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.145+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 shutdown_connections 2024-09-21T07:09:39.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.146+0000 7f4f44671740 1 -- 172.21.15.114:0/1522764009 wait complete. 2024-09-21T07:09:39.149 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.146+0000 7f4f44671740 20 client.29297 trim_cache size 0 max 16384 2024-09-21T07:09:39.150 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:09:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "status": "no active scrubs running", 2024-09-21T07:09:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "scrubs": {}, 2024-09-21T07:09:39.150 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_stats": { 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "15610487-a10d-4566-a6d3-2ae5cb62bae1": { 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:09:03.974081+0000", 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "68277b18-d8ea-49d2-baab-e2089ee24e2b": { 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:08:00.304805+0000", 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "8394af23-1a6b-44ac-a980-7631419f7455": { 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:01.154944+0000", 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "4eb7c4dc-e1ac-4313-8502-e52283f23360": { 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:00:02.582798+0000", 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.152 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "4b612c18-5d84-41a3-bbbb-9a7ba6a4d506": { 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:02:13.140227+0000", 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "baea4e7c-934e-4b5b-8adf-2cbac8995b1e": { 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:54:57.794347+0000", 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.153 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "4fe4b590-9a0b-4529-8c30-4c8a3215ba37": { 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:56:03.809482+0000", 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "de17ea7f-f4c5-41e1-92ba-3fa9d189f02b": { 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:01.245862+0000", 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.154 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "01b042dd-9ca7-4bf9-833e-c06fc008677f": { 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:57:59.455223+0000", 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "4d99c123-582b-4c1f-afb2-2a0a31e59673": { 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:05:36.693688+0000", 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.155 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "7f551d89-35a0-4b4f-9844-0afa60437f35": { 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T06:59:01.354490+0000", 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "c60e65cc-74ed-40b1-b950-c790a95d6c3e": { 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:01:01.300199+0000", 2024-09-21T07:09:39.156 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "427a4e34-6fa9-4f02-b1be-7f0b6f8ee551": { 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:03:20.504388+0000", 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "73ddb0e8-eb68-4bca-bf13-582ff62ccebc": { 2024-09-21T07:09:39.157 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:04:28.403541+0000", 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "fcc7a1f8-deb5-484e-bbd2-c23096bc8d1e": { 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "start_time": "2024-09-21T07:06:54.030598+0000", 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "path": "/", 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_passed": 0, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_failed": 0, 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_skipped": 0 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:39.158 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f1c0b6700 msgr2=0x7f4f1c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f1c0b6700 0x7f4f1c0b8c10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4f38107df0 tx=0x7f4f24009260 comp rx=0 tx=0).stop 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 msgr2=0x7f4f38198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38198db0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800e9f0 tx=0x7f4f2800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 shutdown_connections 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f38109c40 0x7f4f3819fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4f381092f0 0x7f4f381992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4f1c0b6700 0x7f4f1c0b8c10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.159 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 --2- 172.21.15.114:0/1196265699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f381039b0 0x7f4f38198db0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:39.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 >> 172.21.15.114:0/1196265699 conn(0x7f4f380ff440 msgr2=0x7f4f380ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:39.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 shutdown_connections 2024-09-21T07:09:39.160 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:39.147+0000 7f4f37e00640 1 -- 172.21.15.114:0/1196265699 wait complete. 2024-09-21T07:09:39.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:39 smithi114 ceph-mon[30654]: pgmap v877: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 575 op/s 2024-09-21T07:09:39.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:39 smithi183 ceph-mon[34219]: pgmap v877: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 575 op/s 2024-09-21T07:09:39.661 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:39 smithi193 ceph-mon[34601]: pgmap v877: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 575 op/s 2024-09-21T07:09:40.388 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T07:09:40.388 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:41.177 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:41.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:41 smithi114 ceph-mon[30654]: pgmap v878: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 528 op/s 2024-09-21T07:09:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:41 smithi183 ceph-mon[34219]: pgmap v878: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 528 op/s 2024-09-21T07:09:41.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:41 smithi193 ceph-mon[34601]: pgmap v878: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 528 op/s 2024-09-21T07:09:43.463 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:43 smithi193 ceph-mon[34601]: pgmap v879: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 517 op/s 2024-09-21T07:09:43.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:43 smithi114 ceph-mon[30654]: pgmap v879: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 517 op/s 2024-09-21T07:09:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:43 smithi183 ceph-mon[34219]: pgmap v879: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.8 MiB/s wr, 517 op/s 2024-09-21T07:09:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 -- 172.21.15.114:0/3648288963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093410dba0 msgr2=0x7f0934110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 --2- 172.21.15.114:0/3648288963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093410dba0 0x7f0934110090 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f092c009ca0 tx=0x7f092c02a8d0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 -- 172.21.15.114:0/3648288963 shutdown_connections 2024-09-21T07:09:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 --2- 172.21.15.114:0/3648288963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f093410dba0 0x7f0934110090 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 --2- 172.21.15.114:0/3648288963 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0934105090 0x7f093410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 --2- 172.21.15.114:0/3648288963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f0934104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 -- 172.21.15.114:0/3648288963 >> 172.21.15.114:0/3648288963 conn(0x7f09340fe230 msgr2=0x7f0934100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 -- 172.21.15.114:0/3648288963 shutdown_connections 2024-09-21T07:09:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.605+0000 7f0933e00640 1 -- 172.21.15.114:0/3648288963 wait complete. 2024-09-21T07:09:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.607+0000 7f0933e00640 1 Processor -- start 2024-09-21T07:09:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.607+0000 7f0933e00640 1 -- start start 2024-09-21T07:09:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0933e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f093410aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0933e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0934105090 0x7f093410b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0933e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093410dba0 0x7f093410b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0933e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0934103f20 con 0x7f093410dba0 2024-09-21T07:09:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0933e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0934103da0 con 0x7f0934104740 2024-09-21T07:09:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0933e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09341040a0 con 0x7f0934105090 2024-09-21T07:09:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0932a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f093410aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0932a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f093410aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59846/0 (socket says 172.21.15.114:59846) 2024-09-21T07:09:44.610 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0932a00640 1 -- 172.21.15.114:0/126618654 learned_addr learned my addr 172.21.15.114:0/126618654 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.608+0000 7f0932000640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0934105090 0x7f093410b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.609+0000 7f0933400640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093410dba0 0x7f093410b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.609+0000 7f0932a00640 1 -- 172.21.15.114:0/126618654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0934105090 msgr2=0x7f093410b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.609+0000 7f0932a00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0934105090 0x7f093410b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.609+0000 7f0932a00640 1 -- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093410dba0 msgr2=0x7f093410b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.609+0000 7f0932a00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093410dba0 0x7f093410b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.609+0000 7f0932a00640 1 -- 172.21.15.114:0/126618654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09340740f0 con 0x7f0934104740 2024-09-21T07:09:44.611 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.610+0000 7f0932a00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f093410aba0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f092400d950 tx=0x7f092400dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.610+0000 7f0938a00640 1 -- 172.21.15.114:0/126618654 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924017070 con 0x7f0934104740 2024-09-21T07:09:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.610+0000 7f0938a00640 1 -- 172.21.15.114:0/126618654 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f092400de80 con 0x7f0934104740 2024-09-21T07:09:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.610+0000 7f0938a00640 1 -- 172.21.15.114:0/126618654 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092400f620 con 0x7f0934104740 2024-09-21T07:09:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.610+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09340743e0 con 0x7f0934104740 2024-09-21T07:09:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.610+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0934074960 con 0x7f0934104740 2024-09-21T07:09:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.616+0000 7f0940e80740 1 Processor -- start 2024-09-21T07:09:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.616+0000 7f0940e80740 1 -- start start 2024-09-21T07:09:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0938a00640 1 -- 172.21.15.114:0/126618654 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0924009c30 con 0x7f0934104740 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0940e80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef102e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0940e80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef103350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0940e80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef103890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0940e80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5563ef0786b0 con 0x5563eed3d4d0 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0940e80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563ef067ae0 con 0x5563eed352f0 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0940e80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563ef06b8f0 con 0x5563eed5e1b0 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0932000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef103890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0932000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef103890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:47448/0 (socket says 172.21.15.114:47448) 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.617+0000 7f0932000640 1 -- 172.21.15.114:0/2620514174 learned_addr learned my addr 172.21.15.114:0/2620514174 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.618+0000 7f0932000640 1 -- 172.21.15.114:0/2620514174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 msgr2=0x5563ef102e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.618+0000 7f0932000640 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef102e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.618+0000 7f0932000640 1 -- 172.21.15.114:0/2620514174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 msgr2=0x5563ef103350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:44.623 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.618+0000 7f0932000640 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef103350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.618+0000 7f0932000640 1 -- 172.21.15.114:0/2620514174 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ef034780 con 0x5563eed3d4d0 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.619+0000 7f0932000640 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef103890 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0920009840 tx=0x7f0920009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc9708384d1eaab server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.621+0000 7f091fe00640 1 -- 172.21.15.114:0/2620514174 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920039070 con 0x5563eed3d4d0 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.621+0000 7f091fe00640 1 -- 172.21.15.114:0/2620514174 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0920030e70 con 0x5563eed3d4d0 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.621+0000 7f091fe00640 1 -- 172.21.15.114:0/2620514174 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09200343e0 con 0x5563eed3d4d0 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.621+0000 7f0938a00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09180b6730 0x7f09180b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.624 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.622+0000 7f0933400640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09180b6730 0x7f09180b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.622+0000 7f0940e80740 1 -- 172.21.15.114:0/2620514174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 msgr2=0x5563ef103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:44.625 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.622+0000 7f0940e80740 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef103890 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0920009840 tx=0x7f0920009d00 comp rx=0 tx=0).stop 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0933400640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09180b6730 0x7f09180b8c40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f092c0094e0 tx=0x7f092c009470 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0940e80740 1 -- 172.21.15.114:0/2620514174 shutdown_connections 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0940e80740 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef103890 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0940e80740 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0940e80740 1 --2- 172.21.15.114:0/2620514174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0940e80740 1 -- 172.21.15.114:0/2620514174 >> 172.21.15.114:0/2620514174 conn(0x5563eed52300 msgr2=0x5563ef100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:44.626 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0940e80740 1 -- 172.21.15.114:0/2620514174 shutdown_connections 2024-09-21T07:09:44.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.625+0000 7f0940e80740 1 -- 172.21.15.114:0/2620514174 wait complete. 2024-09-21T07:09:44.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.624+0000 7f0938a00640 1 -- 172.21.15.114:0/126618654 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f09240e7720 con 0x7f0934104740 2024-09-21T07:09:44.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.625+0000 7f0940e80740 1 -- start start 2024-09-21T07:09:44.627 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.626+0000 7f0940e80740 1 Processor -- start 2024-09-21T07:09:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.626+0000 7f0940e80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef10ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0940e80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef10f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0940e80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef1141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0940e80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x5563eef9fab0 con 0x5563eed3d4d0 2024-09-21T07:09:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0940e80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5563ef00e570 con 0x5563eed352f0 2024-09-21T07:09:44.636 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0940e80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5563ef109380 con 0x5563eed5e1b0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0940e80740 10 client.0 _ll_register_callbacks cb 0x5563ef01d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef10ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef10ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59860/0 (socket says 172.21.15.114:59860) 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 -- 172.21.15.114:0/193067292 learned_addr learned my addr 172.21.15.114:0/193067292 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 -- 172.21.15.114:0/193067292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 msgr2=0x5563ef1141b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef1141b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 -- 172.21.15.114:0/193067292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 msgr2=0x5563ef10f180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef10f180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.627+0000 7f0932a00640 1 -- 172.21.15.114:0/193067292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563eedc9790 con 0x5563eed352f0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.628+0000 7f0932a00640 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef10ebc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f09240aa3b0 tx=0x7f09240acb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75353bc69efb9b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.628+0000 7f091cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.628+0000 7f091cc00640 1 -- 172.21.15.114:0/193067292 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924020070 con 0x5563eed352f0 2024-09-21T07:09:44.637 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.628+0000 7f091cc00640 1 -- 172.21.15.114:0/193067292 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f09240a8c90 con 0x5563eed352f0 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.628+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0920030e70 con 0x5563eed352f0 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.629+0000 7f091cc00640 1 -- 172.21.15.114:0/193067292 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f09240ad840 con 0x5563eed352f0 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.629+0000 7f0940e80740 10 client.29306 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.629+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5563eee42620 con 0x5563eed352f0 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.629+0000 7f091cc00640 1 -- 172.21.15.114:0/193067292 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0920030e70 con 0x5563eed352f0 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.630+0000 7f091cc00640 1 -- 172.21.15.114:0/193067292 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f09240abe10 con 0x5563eed352f0 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0940e80740 10 client.29306 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0940e80740 10 client.29306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0940e80740 20 client.29306 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0940e80740 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eed1cd80 0x5563ef1204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0940e80740 4 client.29306 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ef07a600 con 0x5563eed1cd80 2024-09-21T07:09:44.638 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.631+0000 7f0932000640 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eed1cd80 0x5563ef1204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.634+0000 7f0932000640 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eed1cd80 0x5563ef1204b0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c554dec178a53a5 server_cookie=b6bc7e4f4721fa1d in_seq=0 out_seq=0 2024-09-21T07:09:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.634+0000 7f091cc00640 10 client.29306 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.635+0000 7f091cc00640 1 -- 172.21.15.114:0/193067292 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563ef07a600 con 0x5563eed1cd80 2024-09-21T07:09:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.635+0000 7f091cc00640 10 client.29306 handle_command_reply: tid=0 2024-09-21T07:09:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.635+0000 7f0940e80740 1 client.29306 shutdown 2024-09-21T07:09:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.636+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 msgr2=0x5563ef10ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.636+0000 7f0940e80740 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef10ebc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f09240aa3b0 tx=0x7f09240acb50 comp rx=0 tx=0).stop 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 shutdown_connections 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f0940e80740 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eed1cd80 0x5563ef1204b0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f0940e80740 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef1141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f0940e80740 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed3d4d0 0x5563ef10f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f091cc00640 0 client.29306 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f0940e80740 1 --2- 172.21.15.114:0/193067292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed352f0 0x5563ef10ebc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.637+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 >> 172.21.15.114:0/193067292 conn(0x5563eed52300 msgr2=0x5563ef104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.638+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 shutdown_connections 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.638+0000 7f0940e80740 1 -- 172.21.15.114:0/193067292 wait complete. 2024-09-21T07:09:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.638+0000 7f0940e80740 20 client.29306 trim_cache size 0 max 16384 2024-09-21T07:09:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.696+0000 7f0940e80740 1 Processor -- start 2024-09-21T07:09:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.697+0000 7f0940e80740 1 -- start start 2024-09-21T07:09:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0940e80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 0x5563ef0fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0940e80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef113f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0940e80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed5e1b0 0x5563ef1005b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0940e80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b74f0 con 0x5563eed5e1b0 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0940e80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b9030 con 0x5563eed3d4d0 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0940e80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b7bc0 con 0x5563eed352f0 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0932a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 0x5563ef0fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0932000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0932a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 0x5563ef0fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40142/0 (socket says 172.21.15.114:40142) 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.701+0000 7f0932a00640 1 -- 172.21.15.114:0/3433731769 learned_addr learned my addr 172.21.15.114:0/3433731769 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932000640 1 -- 172.21.15.114:0/3433731769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 msgr2=0x5563ef0fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932000640 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 0x5563ef0fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932000640 1 -- 172.21.15.114:0/3433731769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed5e1b0 msgr2=0x5563ef1005b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932000640 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed5e1b0 0x5563ef1005b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932000640 1 -- 172.21.15.114:0/3433731769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ef10f5e0 con 0x5563eed3d4d0 2024-09-21T07:09:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932a00640 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 0x5563ef0fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.702+0000 7f0932000640 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef113f90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5563ef120c50 tx=0x7f0920035c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d5ba46d61a89ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.703+0000 7f091f400640 1 -- 172.21.15.114:0/3433731769 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920039040 con 0x5563eed3d4d0 2024-09-21T07:09:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.703+0000 7f091f400640 1 -- 172.21.15.114:0/3433731769 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0920035760 con 0x5563eed3d4d0 2024-09-21T07:09:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.703+0000 7f091f400640 1 -- 172.21.15.114:0/3433731769 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920047820 con 0x5563eed3d4d0 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.704+0000 7f0940e80740 1 -- 172.21.15.114:0/3433731769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 msgr2=0x5563ef113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.704+0000 7f0940e80740 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef113f90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5563ef120c50 tx=0x7f0920035c20 comp rx=0 tx=0).stop 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 -- 172.21.15.114:0/3433731769 shutdown_connections 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed5e1b0 0x5563ef1005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef113f90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 --2- 172.21.15.114:0/3433731769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed352f0 0x5563ef0fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.706 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 -- 172.21.15.114:0/3433731769 >> 172.21.15.114:0/3433731769 conn(0x5563eed52300 msgr2=0x5563ef10d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 -- 172.21.15.114:0/3433731769 shutdown_connections 2024-09-21T07:09:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.705+0000 7f0940e80740 1 -- 172.21.15.114:0/3433731769 wait complete. 2024-09-21T07:09:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.706+0000 7f0940e80740 1 -- start start 2024-09-21T07:09:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.706+0000 7f0940e80740 1 Processor -- start 2024-09-21T07:09:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.709+0000 7f0940e80740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed352f0 0x5563ef1153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.709+0000 7f0940e80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef1158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.709+0000 7f0940e80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef11a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.709+0000 7f0940e80740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b8960 con 0x5563eed352f0 2024-09-21T07:09:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.709+0000 7f0940e80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09341b8290 con 0x5563eed3d4d0 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.709+0000 7f0940e80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0934104b50 con 0x5563eed5e1b0 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.710+0000 7f0940e80740 10 client.0 _ll_register_callbacks cb 0x7f09240e4ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.710+0000 7f0933400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef11a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.710+0000 7f0933400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef11a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:40158/0 (socket says 172.21.15.114:40158) 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.710+0000 7f0933400640 1 -- 172.21.15.114:0/2293709886 learned_addr learned my addr 172.21.15.114:0/2293709886 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.711+0000 7f0933400640 1 -- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 msgr2=0x5563ef1158e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.711+0000 7f0933400640 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef1158e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.711+0000 7f0933400640 1 -- 172.21.15.114:0/2293709886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed352f0 msgr2=0x5563ef1153a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.712+0000 7f0932a00640 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed352f0 0x5563ef1153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.712+0000 7f0933400640 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed352f0 0x5563ef1153a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.712+0000 7f0933400640 1 -- 172.21.15.114:0/2293709886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ef11a7d0 con 0x5563eed5e1b0 2024-09-21T07:09:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.713+0000 7f0933400640 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef11a1e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f092c035000 tx=0x7f092c0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f74c210371a245 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:44.715 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.713+0000 7f091cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.716+0000 7f091cc00640 1 -- 172.21.15.114:0/2293709886 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c02f820 con 0x5563eed5e1b0 2024-09-21T07:09:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.716+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0920035760 con 0x5563eed5e1b0 2024-09-21T07:09:44.717 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.716+0000 7f091cc00640 1 -- 172.21.15.114:0/2293709886 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f092c02f620 con 0x5563eed5e1b0 2024-09-21T07:09:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.716+0000 7f091cc00640 1 -- 172.21.15.114:0/2293709886 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f092c02f820 con 0x5563eed5e1b0 2024-09-21T07:09:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.717+0000 7f0940e80740 10 client.29260 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.717+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x5563ef18a9c0 con 0x5563eed5e1b0 2024-09-21T07:09:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.717+0000 7f091cc00640 1 -- 172.21.15.114:0/2293709886 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0920035760 con 0x5563eed5e1b0 2024-09-21T07:09:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.718+0000 7f091cc00640 1 -- 172.21.15.114:0/2293709886 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f092c030ce0 con 0x5563eed5e1b0 2024-09-21T07:09:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.718+0000 7f0940e80740 10 client.29260 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.718+0000 7f0940e80740 10 client.29260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.718+0000 7f0940e80740 20 client.29260 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.718+0000 7f0940e80740 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eecf21c0 0x5563ef18d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.718+0000 7f0940e80740 4 client.29260 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:09:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.719+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563ef0760b0 con 0x5563eecf21c0 2024-09-21T07:09:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.719+0000 7f0932a00640 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eecf21c0 0x5563ef18d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:44.722 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.721+0000 7f0932a00640 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eecf21c0 0x5563ef18d1b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2dfcae99989ded9 server_cookie=b26b45c177b6f2a3 in_seq=0 out_seq=0 2024-09-21T07:09:44.723 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:44.721+0000 7f091cc00640 10 client.29260 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.295+0000 7f9b63a00640 1 -- 172.21.15.114:0/2663241226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 msgr2=0x7f9b540a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.295+0000 7f9b63a00640 1 --2- 172.21.15.114:0/2663241226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b540a4db0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0524b0 tx=0x7f9b5c098180 comp rx=0 tx=0).stop 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 -- 172.21.15.114:0/2663241226 shutdown_connections 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 --2- 172.21.15.114:0/2663241226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b540a4db0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 --2- 172.21.15.114:0/2663241226 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b5409dc30 0x7f9b5409e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 --2- 172.21.15.114:0/2663241226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b540982f0 0x7f9b54098700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 -- 172.21.15.114:0/2663241226 >> 172.21.15.114:0/2663241226 conn(0x7f9b54093d60 msgr2=0x7f9b540961b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:45.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 -- 172.21.15.114:0/2663241226 shutdown_connections 2024-09-21T07:09:45.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.296+0000 7f9b63a00640 1 -- 172.21.15.114:0/2663241226 wait complete. 2024-09-21T07:09:45.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.298+0000 7f9b63a00640 1 Processor -- start 2024-09-21T07:09:45.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.299+0000 7f9b63a00640 1 -- start start 2024-09-21T07:09:45.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.302+0000 7f9b63a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b540982f0 0x7f9b5413fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.302+0000 7f9b63a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b5409dc30 0x7f9b54140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.304 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.302+0000 7f9b63a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b541453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b63a00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540a8190 con 0x7f9b540982f0 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b63a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540a8010 con 0x7f9b5409dc30 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b63a00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540a8310 con 0x7f9b5409e580 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b61000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b540982f0 0x7f9b5413fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b5be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b5409dc30 0x7f9b54140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b5be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b5409dc30 0x7f9b54140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58966/0 (socket says 172.21.15.114:58966) 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b5be00640 1 -- 172.21.15.114:0/688795653 learned_addr learned my addr 172.21.15.114:0/688795653 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:45.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.303+0000 7f9b61a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b541453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.304+0000 7f9b61a00640 1 -- 172.21.15.114:0/688795653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b5409dc30 msgr2=0x7f9b54140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.304+0000 7f9b61a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b5409dc30 0x7f9b54140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.304+0000 7f9b61a00640 1 -- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b540982f0 msgr2=0x7f9b5413fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.304+0000 7f9b61a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b540982f0 0x7f9b5413fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.304+0000 7f9b61a00640 1 -- 172.21.15.114:0/688795653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b54145b20 con 0x7f9b5409e580 2024-09-21T07:09:45.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.305+0000 7f9b61a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b541453f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c052910 tx=0x7f9b5c06a7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:45.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.305+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0af070 con 0x7f9b5409e580 2024-09-21T07:09:45.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.306+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b5c06ab30 con 0x7f9b5409e580 2024-09-21T07:09:45.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.306+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c0aa670 con 0x7f9b5409e580 2024-09-21T07:09:45.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.306+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b54145db0 con 0x7f9b5409e580 2024-09-21T07:09:45.309 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.307+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b54146210 con 0x7f9b5409e580 2024-09-21T07:09:45.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.309+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b54098700 con 0x7f9b5409e580 2024-09-21T07:09:45.320 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.318+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9b5c0a1470 con 0x7f9b5409e580 2024-09-21T07:09:45.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.322+0000 7f9b59600640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b440b6700 0x7f9b440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:45.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.322+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f9b5c174d30 con 0x7f9b5409e580 2024-09-21T07:09:45.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.322+0000 7f9b61000640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b440b6700 0x7f9b440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:45.325 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.323+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9b5c175230 con 0x7f9b5409e580 2024-09-21T07:09:45.326 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.324+0000 7f9b61000640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b440b6700 0x7f9b440b8c10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009bd0 tx=0x7f9b50005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:45.569 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:45 smithi114 ceph-mon[30654]: pgmap v880: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 670 op/s 2024-09-21T07:09:45.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:45 smithi183 ceph-mon[34219]: pgmap v880: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 670 op/s 2024-09-21T07:09:45.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:45 smithi193 ceph-mon[34601]: pgmap v880: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 670 op/s 2024-09-21T07:09:45.779 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.778+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b54140ce0 con 0x7f9b5409e580 2024-09-21T07:09:45.783 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.779+0000 7f9b59600640 1 -- 172.21.15.114:0/688795653 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+11989 (secure 0 0 0) 0x7f9b54140ce0 con 0x7f9b5409e580 2024-09-21T07:09:45.784 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 60 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.782+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b440b6700 msgr2=0x7f9b440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.782+0000 7f9b63a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b440b6700 0x7f9b440b8c10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009bd0 tx=0x7f9b50005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.782+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 msgr2=0x7f9b541453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.782+0000 7f9b63a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b541453f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c052910 tx=0x7f9b5c06a7b0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.782+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 shutdown_connections 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b5409e580 0x7f9b541453f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9b440b6700 0x7f9b440b8c10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b5409dc30 0x7f9b54140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.785 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 --2- 172.21.15.114:0/688795653 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9b540982f0 0x7f9b5413fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:45.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 >> 172.21.15.114:0/688795653 conn(0x7f9b54093d60 msgr2=0x7f9b54096180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:45.786 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":60,"btime":"2024-09-21T07:09:28:002358+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:09:27.952184+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24725,24727,24742,24760,24761]},"id":1}]} 2024-09-21T07:09:45.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 shutdown_connections 2024-09-21T07:09:45.786 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:45.783+0000 7f9b63a00640 1 -- 172.21.15.114:0/688795653 wait complete. 2024-09-21T07:09:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.030+0000 7f091cc00640 1 -- 172.21.15.114:0/2293709886 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563ef0760b0 con 0x5563eecf21c0 2024-09-21T07:09:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.030+0000 7f091cc00640 10 client.29260 handle_command_reply: tid=0 2024-09-21T07:09:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.030+0000 7f0940e80740 1 client.29260 shutdown 2024-09-21T07:09:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.030+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 msgr2=0x5563ef11a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.030+0000 7f0940e80740 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef11a1e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f092c035000 tx=0x7f092c0095c0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 shutdown_connections 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x5563eecf21c0 0x5563ef18d1b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f091cc00640 0 client.29260 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5563eed5e1b0 0x5563ef11a1e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5563eed3d4d0 0x5563ef1158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 --2- 172.21.15.114:0/2293709886 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x5563eed352f0 0x5563ef1153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 >> 172.21.15.114:0/2293709886 conn(0x5563eed52300 msgr2=0x5563ef10ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.031+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 shutdown_connections 2024-09-21T07:09:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.032+0000 7f0940e80740 1 -- 172.21.15.114:0/2293709886 wait complete. 2024-09-21T07:09:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.032+0000 7f0940e80740 20 client.29260 trim_cache size 0 max 16384 2024-09-21T07:09:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:09:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:09:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:09:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:09:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.033+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09180b6730 msgr2=0x7f09180b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.033+0000 7f0933e00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09180b6730 0x7f09180b8c40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f092c0094e0 tx=0x7f092c009470 comp rx=0 tx=0).stop 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.033+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 msgr2=0x7f093410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.033+0000 7f0933e00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f093410aba0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f092400d950 tx=0x7f092400dd50 comp rx=0 tx=0).stop 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 shutdown_connections 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f093410dba0 0x7f093410b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0934105090 0x7f093410b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f09180b6730 0x7f09180b8c40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 --2- 172.21.15.114:0/126618654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0934104740 0x7f093410aba0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 >> 172.21.15.114:0/126618654 conn(0x7f09340fe230 msgr2=0x7f09340ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 shutdown_connections 2024-09-21T07:09:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:46.034+0000 7f0933e00640 1 -- 172.21.15.114:0/126618654 wait complete. 2024-09-21T07:09:46.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:46 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/688795653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:46.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:46 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/688795653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:46 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/688795653' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:46.789 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T07:09:47.421 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:47.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:47 smithi183 ceph-mon[34219]: pgmap v881: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 542 op/s 2024-09-21T07:09:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:47 smithi114 ceph-mon[30654]: pgmap v881: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 542 op/s 2024-09-21T07:09:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:47 smithi193 ceph-mon[34601]: pgmap v881: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 542 op/s 2024-09-21T07:09:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.389+0000 7f9afe800640 1 -- 172.21.15.114:0/2507615207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81047c0 msgr2=0x7f9af8104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.389+0000 7f9afe800640 1 --2- 172.21.15.114:0/2507615207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81047c0 0x7f9af8104bd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009900 tx=0x7f9aec02a400 comp rx=0 tx=0).stop 2024-09-21T07:09:48.392 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 -- 172.21.15.114:0/2507615207 shutdown_connections 2024-09-21T07:09:48.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 --2- 172.21.15.114:0/2507615207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af810dc20 0x7f9af8110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 --2- 172.21.15.114:0/2507615207 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9af8105110 0x7f9af810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 --2- 172.21.15.114:0/2507615207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af81047c0 0x7f9af8104bd0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 -- 172.21.15.114:0/2507615207 >> 172.21.15.114:0/2507615207 conn(0x7f9af80fe2c0 msgr2=0x7f9af8100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 -- 172.21.15.114:0/2507615207 shutdown_connections 2024-09-21T07:09:48.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.391+0000 7f9afe800640 1 -- 172.21.15.114:0/2507615207 wait complete. 2024-09-21T07:09:48.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.393+0000 7f9afe800640 1 Processor -- start 2024-09-21T07:09:48.395 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.393+0000 7f9afe800640 1 -- start start 2024-09-21T07:09:48.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.393+0000 7f9afe800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af81047c0 0x7f9af81a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afe800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9af8105110 0x7f9af81a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afe800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 0x7f9af81a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afe800640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8111730 con 0x7f9af8105110 2024-09-21T07:09:48.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afe800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81115b0 con 0x7f9af81047c0 2024-09-21T07:09:48.397 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afe800640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81118b0 con 0x7f9af810dc20 2024-09-21T07:09:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 0x7f9af81a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afde00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 0x7f9af81a61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51720/0 (socket says 172.21.15.114:51720) 2024-09-21T07:09:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.394+0000 7f9afde00640 1 -- 172.21.15.114:0/246405274 learned_addr learned my addr 172.21.15.114:0/246405274 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.395+0000 7f9afd400640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af81047c0 0x7f9af81a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.395+0000 7f9afca00640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9af8105110 0x7f9af81a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.398 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.396+0000 7f9afde00640 1 -- 172.21.15.114:0/246405274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af81047c0 msgr2=0x7f9af81a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.396+0000 7f9afde00640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af81047c0 0x7f9af81a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.397+0000 7f9afde00640 1 -- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9af8105110 msgr2=0x7f9af81a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.397+0000 7f9afde00640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9af8105110 0x7f9af81a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.397+0000 7f9afde00640 1 -- 172.21.15.114:0/246405274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af81a68f0 con 0x7f9af810dc20 2024-09-21T07:09:48.399 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.397+0000 7f9afde00640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 0x7f9af81a61f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b9a0 tx=0x7f9af000be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.398+0000 7f9af6000640 1 -- 172.21.15.114:0/246405274 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0019070 con 0x7f9af810dc20 2024-09-21T07:09:48.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.398+0000 7f9af6000640 1 -- 172.21.15.114:0/246405274 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af000cba0 con 0x7f9af810dc20 2024-09-21T07:09:48.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.398+0000 7f9af6000640 1 -- 172.21.15.114:0/246405274 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00075d0 con 0x7f9af810dc20 2024-09-21T07:09:48.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.398+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af81a6be0 con 0x7f9af810dc20 2024-09-21T07:09:48.400 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.398+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af8197eb0 con 0x7f9af810dc20 2024-09-21T07:09:48.411 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.409+0000 7f9af6000640 1 -- 172.21.15.114:0/246405274 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9af001c070 con 0x7f9af810dc20 2024-09-21T07:09:48.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.415+0000 7f9af6000640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9ae00b66f0 0x7f9ae00b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.417 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.415+0000 7f9af6000640 1 -- 172.21.15.114:0/246405274 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f9af00e7f80 con 0x7f9af810dc20 2024-09-21T07:09:48.418 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.416+0000 7f9afd400640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9ae00b66f0 0x7f9ae00b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.424+0000 7f9afd400640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9ae00b66f0 0x7f9ae00b8c00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9aec031440 tx=0x7f9aec009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.426 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.425+0000 7f9b06c10740 1 Processor -- start 2024-09-21T07:09:48.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.425+0000 7f9b06c10740 1 -- start start 2024-09-21T07:09:48.427 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.426+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.429 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.428+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba05241b0 0x563ba08f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.430 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.428+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba052c390 0x563ba08f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.428+0000 7f9afd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.429+0000 7f9afd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54584/0 (socket says 172.21.15.114:54584) 2024-09-21T07:09:48.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.429+0000 7f9b06c10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563ba086afb0 con 0x563ba054d070 2024-09-21T07:09:48.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.429+0000 7f9b06c10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563ba085a3e0 con 0x563ba052c390 2024-09-21T07:09:48.431 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.429+0000 7f9b06c10740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ba085e1f0 con 0x563ba05241b0 2024-09-21T07:09:48.432 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.431+0000 7f9afd400640 1 -- 172.21.15.114:0/4141370095 learned_addr learned my addr 172.21.15.114:0/4141370095 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:48.433 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.431+0000 7f9afca00640 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba05241b0 0x563ba08f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.432+0000 7f9afca00640 1 -- 172.21.15.114:0/4141370095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba052c390 msgr2=0x563ba08f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.432+0000 7f9afca00640 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba052c390 0x563ba08f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.432+0000 7f9afca00640 1 -- 172.21.15.114:0/4141370095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 msgr2=0x563ba08f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.432+0000 7f9afca00640 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.434 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.433+0000 7f9afca00640 1 -- 172.21.15.114:0/4141370095 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba0824cd0 con 0x563ba05241b0 2024-09-21T07:09:48.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.433+0000 7f9afca00640 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba05241b0 0x563ba08f4270 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9af806a800 tx=0x7f9ae4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efc6d19c0cc495d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.433+0000 7f9aea600640 1 -- 172.21.15.114:0/4141370095 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4004050 con 0x563ba05241b0 2024-09-21T07:09:48.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.434+0000 7f9aea600640 1 -- 172.21.15.114:0/4141370095 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae40041f0 con 0x563ba05241b0 2024-09-21T07:09:48.435 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.434+0000 7f9aea600640 1 -- 172.21.15.114:0/4141370095 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40197b0 con 0x563ba05241b0 2024-09-21T07:09:48.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.435+0000 7f9b06c10740 1 -- 172.21.15.114:0/4141370095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba05241b0 msgr2=0x563ba08f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.437 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.435+0000 7f9b06c10740 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba05241b0 0x563ba08f4270 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9af806a800 tx=0x7f9ae4011810 comp rx=0 tx=0).stop 2024-09-21T07:09:48.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.436+0000 7f9b06c10740 1 -- 172.21.15.114:0/4141370095 shutdown_connections 2024-09-21T07:09:48.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.436+0000 7f9b06c10740 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba052c390 0x563ba08f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.436+0000 7f9b06c10740 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba05241b0 0x563ba08f4270 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.436+0000 7f9b06c10740 1 --2- 172.21.15.114:0/4141370095 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.436+0000 7f9b06c10740 1 -- 172.21.15.114:0/4141370095 >> 172.21.15.114:0/4141370095 conn(0x563ba0541460 msgr2=0x563ba08f1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.438 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.436+0000 7f9b06c10740 1 -- 172.21.15.114:0/4141370095 shutdown_connections 2024-09-21T07:09:48.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.437+0000 7f9b06c10740 1 -- 172.21.15.114:0/4141370095 wait complete. 2024-09-21T07:09:48.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.437+0000 7f9b06c10740 1 -- start start 2024-09-21T07:09:48.439 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.438+0000 7f9b06c10740 1 Processor -- start 2024-09-21T07:09:48.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.439+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba08ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.441 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba052c390 0x563ba08fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba054d070 0x563ba09057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9afd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba08ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9afd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba08ffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58994/0 (socket says 172.21.15.114:58994) 2024-09-21T07:09:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9afca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba052c390 0x563ba08fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9afca00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba052c390 0x563ba08fffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54598/0 (socket says 172.21.15.114:54598) 2024-09-21T07:09:48.442 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x563ba06bb700 con 0x563ba052c390 2024-09-21T07:09:48.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563ba07f5580 con 0x563ba05241b0 2024-09-21T07:09:48.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.440+0000 7f9afd400640 1 -- 172.21.15.114:0/1491501264 learned_addr learned my addr 172.21.15.114:0/1491501264 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:48.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.441+0000 7f9afde00640 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba054d070 0x563ba09057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.441+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ba0902510 con 0x563ba054d070 2024-09-21T07:09:48.443 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.441+0000 7f9b06c10740 10 client.0 _ll_register_callbacks cb 0x563ba0805470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:48.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.442+0000 7f9afde00640 1 -- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 msgr2=0x563ba08ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.442+0000 7f9afde00640 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba08ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.442+0000 7f9afde00640 1 -- 172.21.15.114:0/1491501264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba052c390 msgr2=0x563ba08fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.442+0000 7f9afde00640 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba052c390 0x563ba08fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.444 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.442+0000 7f9afde00640 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba05b8650 con 0x563ba054d070 2024-09-21T07:09:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.443+0000 7f9afde00640 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba054d070 0x563ba09057d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9af00e5300 tx=0x7f9af00ad4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75831049893a1287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.444+0000 7f9adea00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.444+0000 7f9adea00640 1 -- 172.21.15.114:0/1491501264 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b24e0 con 0x563ba054d070 2024-09-21T07:09:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.444+0000 7f9adea00640 1 -- 172.21.15.114:0/1491501264 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f9af00a9400 con 0x563ba054d070 2024-09-21T07:09:48.445 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.444+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ba07f7b90 con 0x563ba054d070 2024-09-21T07:09:48.448 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.446+0000 7f9adea00640 1 -- 172.21.15.114:0/1491501264 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f9af00b7420 con 0x563ba054d070 2024-09-21T07:09:48.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.446+0000 7f9b06c10740 10 client.29278 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:48.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.446+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x563ba0812950 con 0x563ba054d070 2024-09-21T07:09:48.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.447+0000 7f9adea00640 1 -- 172.21.15.114:0/1491501264 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563ba07f7b90 con 0x563ba054d070 2024-09-21T07:09:48.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.447+0000 7f9adea00640 1 -- 172.21.15.114:0/1491501264 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f9af00b3670 con 0x563ba054d070 2024-09-21T07:09:48.449 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.447+0000 7f9b06c10740 10 client.29278 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:48.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.447+0000 7f9b06c10740 10 client.29278 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:09:48.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.447+0000 7f9b06c10740 20 client.29278 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:48.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.448+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba050b9f0 0x563ba09119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.448+0000 7f9b06c10740 4 client.29278 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:48.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.448+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ba086cf00 con 0x563ba050b9f0 2024-09-21T07:09:48.450 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.449+0000 7f9afd400640 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba050b9f0 0x563ba09119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.450+0000 7f9afd400640 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba050b9f0 0x563ba09119a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4de4266043c2a33 server_cookie=41a362ed94db6b14 in_seq=0 out_seq=0 2024-09-21T07:09:48.452 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.450+0000 7f9adea00640 10 client.29278 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:09:48.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9adea00640 1 -- 172.21.15.114:0/1491501264 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ba086cf00 con 0x563ba050b9f0 2024-09-21T07:09:48.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9adea00640 10 client.29278 handle_command_reply: tid=0 2024-09-21T07:09:48.453 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 client.29278 shutdown 2024-09-21T07:09:48.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba054d070 msgr2=0x563ba09057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.454 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba054d070 0x563ba09057d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9af00e5300 tx=0x7f9af00ad4c0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 shutdown_connections 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba054d070 0x563ba09057d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba052c390 0x563ba08fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.452+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba050b9f0 0x563ba09119a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.453+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1491501264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba08ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.453+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 >> 172.21.15.114:0/1491501264 conn(0x563ba0541460 msgr2=0x563ba08f6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.453+0000 7f9adea00640 0 client.29278 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.453+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 shutdown_connections 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.453+0000 7f9b06c10740 1 -- 172.21.15.114:0/1491501264 wait complete. 2024-09-21T07:09:48.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.453+0000 7f9b06c10740 20 client.29278 trim_cache size 0 max 16384 2024-09-21T07:09:48.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.488+0000 7f9b06c10740 1 Processor -- start 2024-09-21T07:09:48.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.489+0000 7f9b06c10740 1 -- start start 2024-09-21T07:09:48.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.489+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba0905740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba08fea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9b06c10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81bfcc0 con 0x563ba054d070 2024-09-21T07:09:48.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9b06c10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81c1800 con 0x563ba05241b0 2024-09-21T07:09:48.491 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9b06c10740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81c0390 con 0x563ba052c390 2024-09-21T07:09:48.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9afd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba0905740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9afd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba0905740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:58998/0 (socket says 172.21.15.114:58998) 2024-09-21T07:09:48.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9afd400640 1 -- 172.21.15.114:0/1348208850 learned_addr learned my addr 172.21.15.114:0/1348208850 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:48.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.490+0000 7f9afca00640 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba08fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.492 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.491+0000 7f9afd400640 1 -- 172.21.15.114:0/1348208850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 msgr2=0x563ba08fea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.491+0000 7f9afd400640 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba08fea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.491+0000 7f9afd400640 1 -- 172.21.15.114:0/1348208850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 msgr2=0x563ba08f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.491+0000 7f9afd400640 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.491+0000 7f9afd400640 1 -- 172.21.15.114:0/1348208850 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ba08f6020 con 0x563ba05241b0 2024-09-21T07:09:48.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.492+0000 7f9afd400640 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba0905740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9aec02a920 tx=0x7f9aec040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ada0679d456aa76 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.492+0000 7f9ae9c00640 1 -- 172.21.15.114:0/1348208850 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec035040 con 0x563ba05241b0 2024-09-21T07:09:48.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.492+0000 7f9ae9c00640 1 -- 172.21.15.114:0/1348208850 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aec0054e0 con 0x563ba05241b0 2024-09-21T07:09:48.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.492+0000 7f9ae9c00640 1 -- 172.21.15.114:0/1348208850 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec004e60 con 0x563ba05241b0 2024-09-21T07:09:48.495 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.494+0000 7f9b06c10740 1 -- 172.21.15.114:0/1348208850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 msgr2=0x563ba0905740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.494+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba0905740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9aec02a920 tx=0x7f9aec040000 comp rx=0 tx=0).stop 2024-09-21T07:09:48.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 -- 172.21.15.114:0/1348208850 shutdown_connections 2024-09-21T07:09:48.496 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba054d070 0x563ba08f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba08fea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1348208850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba05241b0 0x563ba0905740 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 -- 172.21.15.114:0/1348208850 >> 172.21.15.114:0/1348208850 conn(0x563ba0541460 msgr2=0x563ba08fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 -- 172.21.15.114:0/1348208850 shutdown_connections 2024-09-21T07:09:48.497 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.495+0000 7f9b06c10740 1 -- 172.21.15.114:0/1348208850 wait complete. 2024-09-21T07:09:48.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.496+0000 7f9b06c10740 1 -- start start 2024-09-21T07:09:48.500 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.499+0000 7f9b06c10740 1 Processor -- start 2024-09-21T07:09:48.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba05241b0 0x563ba090bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba090c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba054d070 0x563ba09063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81c1130 con 0x563ba05241b0 2024-09-21T07:09:48.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81c0a60 con 0x563ba054d070 2024-09-21T07:09:48.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9af806b560 con 0x563ba052c390 2024-09-21T07:09:48.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.500+0000 7f9b06c10740 10 client.0 _ll_register_callbacks cb 0x7f9af00ad880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.501+0000 7f9afca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba090c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.501+0000 7f9afca00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba090c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51748/0 (socket says 172.21.15.114:51748) 2024-09-21T07:09:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.501+0000 7f9afca00640 1 -- 172.21.15.114:0/1578056681 learned_addr learned my addr 172.21.15.114:0/1578056681 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.501+0000 7f9afde00640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba054d070 0x563ba09063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.501+0000 7f9afd400640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba05241b0 0x563ba090bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.502+0000 7f9afde00640 1 -- 172.21.15.114:0/1578056681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 msgr2=0x563ba090c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.502+0000 7f9afde00640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba090c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.502+0000 7f9afde00640 1 -- 172.21.15.114:0/1578056681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba05241b0 msgr2=0x563ba090bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.502+0000 7f9afde00640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba05241b0 0x563ba090bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.502+0000 7f9afde00640 1 -- 172.21.15.114:0/1578056681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ba0906b70 con 0x563ba054d070 2024-09-21T07:09:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.502+0000 7f9afca00640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba090c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:48.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.503+0000 7f9adea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.503+0000 7f9afde00640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba054d070 0x563ba09063e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9af00ae200 tx=0x7f9af00aa210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5c869e91a24a699 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.504+0000 7f9adea00640 1 -- 172.21.15.114:0/1578056681 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00b7420 con 0x563ba054d070 2024-09-21T07:09:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.504+0000 7f9adea00640 1 -- 172.21.15.114:0/1578056681 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f9af00acf20 con 0x563ba054d070 2024-09-21T07:09:48.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.504+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec0054e0 con 0x563ba054d070 2024-09-21T07:09:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.504+0000 7f9adea00640 1 -- 172.21.15.114:0/1578056681 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f9af00b7420 con 0x563ba054d070 2024-09-21T07:09:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.504+0000 7f9b06c10740 10 client.29327 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.504+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x563ba097cc10 con 0x563ba054d070 2024-09-21T07:09:48.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.505+0000 7f9adea00640 1 -- 172.21.15.114:0/1578056681 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af00e5a70 con 0x563ba054d070 2024-09-21T07:09:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.505+0000 7f9adea00640 1 -- 172.21.15.114:0/1578056681 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f9af00b3670 con 0x563ba054d070 2024-09-21T07:09:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.505+0000 7f9b06c10740 10 client.29327 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.505+0000 7f9b06c10740 10 client.29327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T07:09:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.505+0000 7f9b06c10740 20 client.29327 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.505+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba04e0da0 0x563ba097f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:48.507 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.506+0000 7f9b06c10740 4 client.29327 mds_command: new command op to 24725 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:09:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.506+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 --> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9aec0054e0 con 0x563ba04e0da0 2024-09-21T07:09:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.506+0000 7f9afd400640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba04e0da0 0x563ba097f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.507+0000 7f9adea00640 10 client.29327 ms_handle_connect on v2:172.21.15.183:6836/841299119 2024-09-21T07:09:48.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.507+0000 7f9afd400640 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba04e0da0 0x563ba097f3d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=250558a8efeac520 server_cookie=24c5017070afa509 in_seq=0 out_seq=0 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9adea00640 1 -- 172.21.15.114:0/1578056681 <== mds.4 v2:172.21.15.183:6836/841299119 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9aec0054e0 con 0x563ba04e0da0 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9adea00640 10 client.29327 handle_command_reply: tid=0 2024-09-21T07:09:48.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 client.29327 shutdown 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba054d070 msgr2=0x563ba09063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba054d070 0x563ba09063e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9af00ae200 tx=0x7f9af00aa210 comp rx=0 tx=0).stop 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 shutdown_connections 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:6836/841299119,v1:172.21.15.183:6837/841299119] conn(0x563ba04e0da0 0x563ba097f3d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ba054d070 0x563ba09063e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9adea00640 0 client.29327 ms_handle_reset on v2:172.21.15.183:6836/841299119 2024-09-21T07:09:48.515 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ba052c390 0x563ba090c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 --2- 172.21.15.114:0/1578056681 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x563ba05241b0 0x563ba090bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.513+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 >> 172.21.15.114:0/1578056681 conn(0x563ba0541460 msgr2=0x563ba08fa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.514+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 shutdown_connections 2024-09-21T07:09:48.516 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.514+0000 7f9b06c10740 1 -- 172.21.15.114:0/1578056681 wait complete. 2024-09-21T07:09:48.517 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.515+0000 7f9b06c10740 20 client.29327 trim_cache size 0 max 16384 2024-09-21T07:09:48.517 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.516+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9ae00b66f0 msgr2=0x7f9ae00b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.516+0000 7f9afe800640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9ae00b66f0 0x7f9ae00b8c00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9aec031440 tx=0x7f9aec009560 comp rx=0 tx=0).stop 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 msgr2=0x7f9af81a61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 0x7f9af81a61f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b9a0 tx=0x7f9af000be60 comp rx=0 tx=0).stop 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 shutdown_connections 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9af810dc20 0x7f9af81a61f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.519 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9af8105110 0x7f9af81a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9ae00b66f0 0x7f9ae00b8c00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 --2- 172.21.15.114:0/246405274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af81047c0 0x7f9af81a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 >> 172.21.15.114:0/246405274 conn(0x7f9af80fe2c0 msgr2=0x7f9af8107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 shutdown_connections 2024-09-21T07:09:48.520 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:48.517+0000 7f9afe800640 1 -- 172.21.15.114:0/246405274 wait complete. 2024-09-21T07:09:49.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:49 smithi114 ceph-mon[30654]: pgmap v882: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-09-21T07:09:49.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T07:09:49.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:49 smithi183 ceph-mon[34219]: pgmap v882: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-09-21T07:09:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:49 smithi193 ceph-mon[34601]: pgmap v882: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-09-21T07:09:50.343 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:51.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:51 smithi114 ceph-mon[30654]: pgmap v883: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 5.7 MiB/s wr, 591 op/s 2024-09-21T07:09:51.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:51 smithi183 ceph-mon[34219]: pgmap v883: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 5.7 MiB/s wr, 591 op/s 2024-09-21T07:09:51.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:51 smithi193 ceph-mon[34601]: pgmap v883: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 5.7 MiB/s wr, 591 op/s 2024-09-21T07:09:52.810 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-21T07:09:52.955 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/: 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 .. 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 data 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data: 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:total 36 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 . 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 .. 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:54 base 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 21 06:54 current_logfiles 2024-09-21T07:09:52.956 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 06:54 global 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 log 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_commit_ts 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_dynshmem 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 21 06:54 pg_hba.conf 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 21 06:54 pg_ident.conf 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 07:09 pg_logical 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 pg_multixact 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_notify 2024-09-21T07:09:52.957 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_replslot 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_serial 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_snapshots 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 07:09 pg_stat 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 07:09 pg_stat_tmp 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 07:07 pg_subtrans 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_tblspc 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pg_twophase 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 54 Sep 21 07:09 pg_wal 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 pg_xact 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 21 06:54 postgresql.auto.conf 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 21 06:54 postgresql.conf 2024-09-21T07:09:52.958 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 21 06:54 postmaster.opts 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base: 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:54 . 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 1 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 13493 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 07:01 13494 2024-09-21T07:09:52.959 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 pgsql_tmp 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:total 7777 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 . 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:54 .. 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 112 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 113 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 1247 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1247_fsm 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1247_vm 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:54 1249 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1249_fsm 2024-09-21T07:09:52.960 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1249_vm 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:54 1255 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1255_fsm 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1255_vm 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:54 1259 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1259_fsm 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1259_vm 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:54 13344 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13344_fsm 2024-09-21T07:09:52.961 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13344_vm 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13346 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13348 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13349_fsm 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349_vm 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13351 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13353 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13354_fsm 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354_vm 2024-09-21T07:09:52.962 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13356 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13358 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13359_fsm 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359_vm 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13361 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13363 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1417 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1418 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 174 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 175 2024-09-21T07:09:52.963 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2187 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2224 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2328 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2336 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2337 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2579 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2600 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2600_fsm 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2600_vm 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2601_fsm 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601_vm 2024-09-21T07:09:52.964 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2602 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2602_fsm 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2602_vm 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2603 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2603_fsm 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2603_vm 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2604 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2605 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2605_fsm 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2605_vm 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606 2024-09-21T07:09:52.965 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2606_fsm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606_vm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2607 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2607_fsm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2607_vm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:54 2608 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2608_fsm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2608_vm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:54 2609 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2609_fsm 2024-09-21T07:09:52.966 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2609_vm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2610 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2610_fsm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2610_vm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2611 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2612_fsm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612_vm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2613 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2615_fsm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615_vm 2024-09-21T07:09:52.967 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2616 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2616_fsm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2616_vm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:54 2617 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2617_fsm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2617_vm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:54 2618 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2618_fsm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2618_vm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:54 2619 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2619_fsm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2619_vm 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2620 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2650 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2651 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2652 2024-09-21T07:09:52.968 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2653 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2654 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2655 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2656 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2657 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:54 2658 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 2659 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2660 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2661 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2662 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2663 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2664 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2665 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2666 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2667 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2668 2024-09-21T07:09:52.969 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2669 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2670 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:54 2673 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:54 2674 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:54 2675 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2678 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2679 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2680 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2681 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2682 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2683 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2684 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2685 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2686 2024-09-21T07:09:52.970 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2687 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2688 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2689 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:54 2690 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:54 2691 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2692 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2693 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2696 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2699 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2701 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2702 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2703 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2704 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2753 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2753_fsm 2024-09-21T07:09:52.971 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2753_vm 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2754 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2755 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2756 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2757 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2830 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2831 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2832 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2833 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2834 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2835 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2836 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2837 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:54 2838 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2838_fsm 2024-09-21T07:09:52.972 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2838_vm 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2839 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840_fsm 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2840_vm 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2841 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2995 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2996 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3079_fsm 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079_vm 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3080 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3081 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3085 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3118 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3119 2024-09-21T07:09:52.973 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:54 3164 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3256 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3257 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3258 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3350 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3351 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3379 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3380 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3381 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3394 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3394_fsm 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3394_vm 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3395 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3429 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3430 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3431 2024-09-21T07:09:52.974 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3433 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3439 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3440 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3455 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:54 3456 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3456_fsm 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3456_vm 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3466 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3467 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3468 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3501 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3502 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3503 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3534 2024-09-21T07:09:52.975 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3541_fsm 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541_vm 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3542 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3574 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3575 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3576 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3596 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3597 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3598 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3599 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3600_fsm 2024-09-21T07:09:52.976 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600_vm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3601_fsm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601_vm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3602_fsm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602_vm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603_fsm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3603_vm 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3604 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3605 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3606 2024-09-21T07:09:52.977 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3607 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3608 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 3609 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3712 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3764_fsm 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764_vm 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3766 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3767 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3997 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4143 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4144 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4145 2024-09-21T07:09:52.978 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4146 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4147 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4148 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4149 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4150 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4151 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4152 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4153 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4154 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4155 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4156 2024-09-21T07:09:52.979 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4157 2024-09-21T07:09:52.980 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4158 2024-09-21T07:09:52.980 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4159 2024-09-21T07:09:52.980 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4160 2024-09-21T07:09:52.980 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4161 2024-09-21T07:09:52.980 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4162 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4163 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4164 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4165 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4166 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4167 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4168 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4169 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4170 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4171 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4172 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4173 2024-09-21T07:09:52.992 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4174 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 5002 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 548 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 549 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6102 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6104 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6106 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6110 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6111 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6112 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6113 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6117 2024-09-21T07:09:52.993 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 826 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 827 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 828 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:total 7777 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 21 06:54 . 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:54 .. 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 112 2024-09-21T07:09:52.994 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 113 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 1247 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1247_fsm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1247_vm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:54 1249 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1249_fsm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1249_vm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:54 1255 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1255_fsm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1255_vm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 06:54 1259 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1259_fsm 2024-09-21T07:09:52.995 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1259_vm 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:54 13344 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13344_fsm 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13344_vm 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13346 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13348 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13349_fsm 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349_vm 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13351 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13353 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354 2024-09-21T07:09:52.996 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13354_fsm 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354_vm 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13356 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13358 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13359_fsm 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359_vm 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13361 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13363 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1417 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1418 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 174 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 175 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2187 2024-09-21T07:09:52.997 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2224 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2328 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2336 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2337 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2579 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2600 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2600_fsm 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2600_vm 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2601_fsm 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601_vm 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2602 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2602_fsm 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2602_vm 2024-09-21T07:09:52.998 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2603 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2603_fsm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2603_vm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2604 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2605 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2605_fsm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2605_vm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2606_fsm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606_vm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2607 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2607_fsm 2024-09-21T07:09:52.999 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2607_vm 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:54 2608 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2608_fsm 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2608_vm 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:54 2609 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2609_fsm 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2609_vm 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2610 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2610_fsm 2024-09-21T07:09:53.000 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2610_vm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2611 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2612_fsm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612_vm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2613 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2615_fsm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615_vm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2616 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2616_fsm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2616_vm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:54 2617 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2617_fsm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2617_vm 2024-09-21T07:09:53.001 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:54 2618 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2618_fsm 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2618_vm 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 21 06:54 2619 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2619_fsm 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2619_vm 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2620 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2650 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2651 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2652 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2653 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2654 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2655 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2656 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2657 2024-09-21T07:09:53.002 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:54 2658 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 2659 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2660 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2661 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2662 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2663 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2664 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2665 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2666 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2667 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2668 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2669 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2670 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:54 2673 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:54 2674 2024-09-21T07:09:53.003 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:54 2675 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2678 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2679 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2680 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2681 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2682 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2683 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2684 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2685 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2686 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2687 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2688 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2689 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:54 2690 2024-09-21T07:09:53.004 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:54 2691 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2692 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2693 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2696 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2699 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2701 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2702 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2703 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2704 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2753 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2753_fsm 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2753_vm 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2754 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2755 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2756 2024-09-21T07:09:53.005 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2757 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2830 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2831 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2832 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2833 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2834 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2835 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2836 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2837 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:54 2838 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2838_fsm 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2838_vm 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2839 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840 2024-09-21T07:09:53.006 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840_fsm 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2840_vm 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2841 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2995 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2996 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3079_fsm 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079_vm 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3080 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3081 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3085 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3118 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3119 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:54 3164 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3256 2024-09-21T07:09:53.007 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3257 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3258 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3350 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3351 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3379 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3380 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3381 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3394 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3394_fsm 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3394_vm 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3395 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3429 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3430 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3431 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3433 2024-09-21T07:09:53.008 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3439 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3440 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3455 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:54 3456 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3456_fsm 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3456_vm 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3466 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3467 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3468 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3501 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3502 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3503 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3534 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3541_fsm 2024-09-21T07:09:53.009 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541_vm 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3542 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3574 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3575 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3576 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3596 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3597 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3598 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3599 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3600_fsm 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600_vm 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3601_fsm 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601_vm 2024-09-21T07:09:53.010 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3602_fsm 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602_vm 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603_fsm 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3603_vm 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3604 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3605 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3606 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3607 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3608 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 3609 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3712 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764 2024-09-21T07:09:53.011 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3764_fsm 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764_vm 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3766 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3767 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3997 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4143 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4144 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4145 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4146 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4147 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4148 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4149 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4150 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4151 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4152 2024-09-21T07:09:53.012 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4153 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4154 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4155 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4156 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4157 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4158 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4159 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4160 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4161 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4162 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4163 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4164 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4165 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4166 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4167 2024-09-21T07:09:53.013 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4168 2024-09-21T07:09:53.014 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4169 2024-09-21T07:09:53.014 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4170 2024-09-21T07:09:53.014 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4171 2024-09-21T07:09:53.014 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4172 2024-09-21T07:09:53.044 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4173 2024-09-21T07:09:53.044 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4174 2024-09-21T07:09:53.044 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 5002 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 548 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 549 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6102 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6104 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6106 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6110 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6111 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6112 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6113 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6117 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 826 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 827 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 828 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T07:09:53.045 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:total 535854 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 21 07:01 . 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:54 .. 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 112 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 113 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 1247 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1247_fsm 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1247_vm 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 21 06:54 1249 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1249_fsm 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1249_vm 2024-09-21T07:09:53.046 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 21 06:54 1255 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1255_fsm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1255_vm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 21 07:09 1259 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1259_fsm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1259_vm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 21 06:54 13344 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13344_fsm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13344_vm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13346 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13348 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13349_fsm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13349_vm 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13351 2024-09-21T07:09:53.047 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13353 2024-09-21T07:09:53.048 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354 2024-09-21T07:09:53.048 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13354_fsm 2024-09-21T07:09:53.048 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13354_vm 2024-09-21T07:09:53.048 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13356 2024-09-21T07:09:53.048 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13358 2024-09-21T07:09:53.048 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359 2024-09-21T07:09:53.049 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 13359_fsm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13359_vm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 13361 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 13363 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1417 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 1418 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439369728 Sep 21 07:09 16396 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 21 07:09 16396_fsm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 07:09 16396_vm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 368640 Sep 21 07:09 16397 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 07:09 16397_fsm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 07:09 16397_vm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 385024 Sep 21 07:09 16399 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 07:09 16399_fsm 2024-09-21T07:09:53.050 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 07:08 16399_vm 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 07:09 16400 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 07:09 16402 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 21 07:09 16404 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28229632 Sep 21 07:09 16406 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 07:09 16406_fsm 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 07:09 16406_vm 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 174 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 175 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2187 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2224 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2328 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2336 2024-09-21T07:09:53.051 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2337 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2579 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2600 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2600_fsm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2600_vm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2601_fsm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2601_vm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2602 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2602_fsm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2602_vm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2603 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2603_fsm 2024-09-21T07:09:53.052 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2603_vm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2604 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2605 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2605_fsm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2605_vm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2606_fsm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2606_vm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2607 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2607_fsm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2607_vm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 21 06:54 2608 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2608_fsm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2608_vm 2024-09-21T07:09:53.053 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 21 06:54 2609 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2609_fsm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2609_vm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2610 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2610_fsm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2610_vm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2611 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2612_fsm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2612_vm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2613 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2615_fsm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2615_vm 2024-09-21T07:09:53.054 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2616 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2616_fsm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2616_vm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 21 06:54 2617 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2617_fsm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2617_vm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 21 06:54 2618 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2618_fsm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2618_vm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Sep 21 07:09 2619 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 07:03 2619_fsm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2619_vm 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2620 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2650 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2651 2024-09-21T07:09:53.055 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2652 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2653 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2654 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2655 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2656 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2657 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 21 06:54 2658 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 21 06:54 2659 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2660 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2661 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2662 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2663 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2664 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2665 2024-09-21T07:09:53.056 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2666 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2667 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2668 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2669 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2670 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 21 06:54 2673 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 21 06:54 2674 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 21 06:54 2675 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2678 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2679 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2680 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2681 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2682 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2683 2024-09-21T07:09:53.057 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2684 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2685 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2686 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2687 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2688 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 21 06:54 2689 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 21 06:54 2690 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 21 06:54 2691 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2692 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2693 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 07:07 2696 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2699 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2701 2024-09-21T07:09:53.058 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2702 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2703 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 2704 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2753 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2753_fsm 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2753_vm 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2754 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2755 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2756 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 2757 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2830 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2831 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2832 2024-09-21T07:09:53.059 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2833 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2834 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2835 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2836 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2837 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 21 06:54 2838 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2838_fsm 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2838_vm 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2839 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2840_fsm 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2840_vm 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2841 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2995 2024-09-21T07:09:53.060 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2996 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3079_fsm 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3079_vm 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3080 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3081 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3085 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3118 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3119 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 21 06:54 3164 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3256 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3257 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3258 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3350 2024-09-21T07:09:53.061 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3351 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3379 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3380 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3381 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3394 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3394_fsm 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3394_vm 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3395 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3429 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3430 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3431 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3433 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3439 2024-09-21T07:09:53.062 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3440 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 21 06:54 3455 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 21 06:54 3456 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3456_fsm 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3456_vm 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3466 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3467 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3468 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3501 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3502 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3503 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3534 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541 2024-09-21T07:09:53.063 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3541_fsm 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3541_vm 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3542 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3574 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3575 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3576 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3596 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3597 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3598 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3599 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3600_fsm 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3600_vm 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601 2024-09-21T07:09:53.064 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3601_fsm 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3601_vm 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3602_fsm 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3602_vm 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3603_fsm 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3603_vm 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3604 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3605 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3606 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3607 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3608 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 21 06:54 3609 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3712 2024-09-21T07:09:53.065 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 3764_fsm 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3764_vm 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3766 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 3767 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3997 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4143 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4144 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4145 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4146 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4147 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4148 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4149 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4150 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4151 2024-09-21T07:09:53.066 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4152 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4153 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4154 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4155 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4156 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4157 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4158 2024-09-21T07:09:53.067 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4159 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4160 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4161 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4162 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4163 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4164 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4165 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4166 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4167 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4168 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4169 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4170 2024-09-21T07:09:53.068 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4171 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4172 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4173 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4174 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 5002 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 548 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 549 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6102 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6104 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6106 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6110 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6111 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6112 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6113 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6117 2024-09-21T07:09:53.069 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 826 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 827 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 828 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 21 06:54 pg_internal.init 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 06:54 PG_VERSION 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 21 06:54 .. 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/global: 2024-09-21T07:09:53.070 INFO:teuthology.orchestra.run.smithi114.stdout:total 560 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 21 06:54 . 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1213 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1213_fsm 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1213_vm 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1214 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1214_fsm 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1214_vm 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 1232 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 1233 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1260 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1260_fsm 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1260_vm 2024-09-21T07:09:53.071 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1261 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1261_fsm 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1261_vm 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1262 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 1262_fsm 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 1262_vm 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2396 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 21 06:54 2396_fsm 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2396_vm 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2397 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2671 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2672 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2676 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2677 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2694 2024-09-21T07:09:53.072 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2695 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2697 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 21 06:54 2698 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2846 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2847 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2964 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2965 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 2966 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 2967 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 3592 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 3593 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4060 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4061 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4175 2024-09-21T07:09:53.073 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4176 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4177 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4178 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4181 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4182 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4183 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4184 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 4185 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 4186 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6000 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6001 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6002 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 6100 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6114 2024-09-21T07:09:53.074 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 6115 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 07:09 pg_control 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 21 06:54 pg_filenode.map 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 21 06:54 pg_internal.init 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/log: 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:total 2 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1249 Sep 21 07:09 postgresql-Sat.log 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.075 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:total 1 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 07:09 . 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 mappings 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 21 07:09 replorigin_checkpoint 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 snapshots 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.076 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 07:09 .. 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 07:09 .. 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 . 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.077 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 members 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 offsets 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:total 8 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 .. 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:54 0000 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:total 8 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 21 06:54 .. 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 21 06:59 0000 2024-09-21T07:09:53.078 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.079 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.080 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:total 15 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 21 07:09 . 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 21 07:09 db_0.stat 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 21 07:09 db_13494.stat 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 21 07:09 global.stat 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 07:09 . 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-21T07:09:53.081 INFO:teuthology.orchestra.run.smithi114.stdout:total 224 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 07:07 . 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 229376 Sep 21 07:09 0007 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.082 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:total 868352 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 54 Sep 21 07:09 . 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:09 000000010000000000000058 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 000000010000000000000059 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 00000001000000000000005A 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 00000001000000000000005B 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 00000001000000000000005C 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 00000001000000000000005D 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 00000001000000000000005E 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:00 00000001000000000000005F 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:00 000000010000000000000060 2024-09-21T07:09:53.083 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:03 000000010000000000000061 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:02 000000010000000000000062 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 000000010000000000000063 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:02 000000010000000000000064 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 000000010000000000000065 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 000000010000000000000066 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:03 000000010000000000000067 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 000000010000000000000068 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:00 000000010000000000000069 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:00 00000001000000000000006A 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 00000001000000000000006B 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 00000001000000000000006C 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:00 00000001000000000000006D 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 00000001000000000000006E 2024-09-21T07:09:53.084 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:03 00000001000000000000006F 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 06:59 000000010000000000000070 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:02 000000010000000000000071 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:01 000000010000000000000072 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:00 000000010000000000000073 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:06 000000010000000000000074 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 000000010000000000000075 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:07 000000010000000000000076 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:04 000000010000000000000077 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:06 000000010000000000000078 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 000000010000000000000079 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:08 00000001000000000000007A 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 00000001000000000000007B 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 00000001000000000000007C 2024-09-21T07:09:53.085 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:08 00000001000000000000007D 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 00000001000000000000007E 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:07 00000001000000000000007F 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 000000010000000000000080 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 000000010000000000000081 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:06 000000010000000000000082 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 000000010000000000000083 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:06 000000010000000000000084 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:06 000000010000000000000085 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:07 000000010000000000000086 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:07 000000010000000000000087 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:08 000000010000000000000088 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 000000010000000000000089 2024-09-21T07:09:53.086 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 00000001000000000000008A 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:04 00000001000000000000008B 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 21 07:05 00000001000000000000008C 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 archive_status 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:total 0 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 21 06:54 . 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 54 Sep 21 07:09 .. 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:total 128 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 21 06:54 . 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 21 07:09 .. 2024-09-21T07:09:53.087 INFO:teuthology.orchestra.run.smithi114.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 21 07:09 0000 2024-09-21T07:09:53.088 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-21T07:09:53.099 INFO:tasks.quiescer:joining Quiescers 2024-09-21T07:09:53.099 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-21T07:09:53.099 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:09:53.369 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:53 smithi114 ceph-mon[30654]: pgmap v884: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.5 MiB/s wr, 619 op/s 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 -- 172.21.15.114:0/3730813457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 msgr2=0x7f1eb41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 --2- 172.21.15.114:0/3730813457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 0x7f1eb41098d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009900 tx=0x7f1ea002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 -- 172.21.15.114:0/3730813457 shutdown_connections 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 --2- 172.21.15.114:0/3730813457 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 0x7f1eb4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 --2- 172.21.15.114:0/3730813457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 0x7f1eb41098d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 --2- 172.21.15.114:0/3730813457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb4103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 -- 172.21.15.114:0/3730813457 >> 172.21.15.114:0/3730813457 conn(0x7f1eb40ff590 msgr2=0x7f1eb41019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 -- 172.21.15.114:0/3730813457 shutdown_connections 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.452+0000 7f1eba000640 1 -- 172.21.15.114:0/3730813457 wait complete. 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.453+0000 7f1eba000640 1 Processor -- start 2024-09-21T07:09:53.455 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.453+0000 7f1eba000640 1 -- start start 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.453+0000 7f1eba000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.453+0000 7f1eba000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 0x7f1eb419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.453+0000 7f1eba000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 0x7f1eb41a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eba000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4113ad0 con 0x7f1eb4109e10 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eba000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4113950 con 0x7f1eb4103b80 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eba000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4113c50 con 0x7f1eb41094c0 2024-09-21T07:09:53.456 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb2a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 0x7f1eb419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59036/0 (socket says 172.21.15.114:59036) 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 -- 172.21.15.114:0/4111125944 learned_addr learned my addr 172.21.15.114:0/4111125944 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 0x7f1eb41a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 -- 172.21.15.114:0/4111125944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 msgr2=0x7f1eb419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 0x7f1eb419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 -- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 msgr2=0x7f1eb41a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 0x7f1eb41a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3400640 1 -- 172.21.15.114:0/4111125944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb41a2650 con 0x7f1eb4103b80 2024-09-21T07:09:53.457 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.454+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 0x7f1eb41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:09:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.455+0000 7f1eb3400640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb419c6e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e9f0 tx=0x7f1ea400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.455+0000 7f1eabe00640 1 -- 172.21.15.114:0/4111125944 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400cd80 con 0x7f1eb4103b80 2024-09-21T07:09:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.455+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb41a2940 con 0x7f1eb4103b80 2024-09-21T07:09:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.455+0000 7f1eabe00640 1 -- 172.21.15.114:0/4111125944 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea40108e0 con 0x7f1eb4103b80 2024-09-21T07:09:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.455+0000 7f1eabe00640 1 -- 172.21.15.114:0/4111125944 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4018980 con 0x7f1eb4103b80 2024-09-21T07:09:53.458 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.455+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb41a90a0 con 0x7f1eb4103b80 2024-09-21T07:09:53.460 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 Processor -- start 2024-09-21T07:09:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 -- start start 2024-09-21T07:09:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1eabe00640 1 -- 172.21.15.114:0/4111125944 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1ea4018b20 con 0x7f1eb4103b80 2024-09-21T07:09:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1df390 0x55851d5a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.461 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55851d51dfb0 con 0x55851d200070 2024-09-21T07:09:53.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55851d50d3e0 con 0x55851d1d71b0 2024-09-21T07:09:53.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.459+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55851d5111f0 con 0x55851d1df390 2024-09-21T07:09:53.462 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.460+0000 7f1eb3e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.460+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.460+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59038/0 (socket says 172.21.15.114:59038) 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.460+0000 7f1eb3400640 1 -- 172.21.15.114:0/3725856397 learned_addr learned my addr 172.21.15.114:0/3725856397 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.461+0000 7f1eb3400640 1 -- 172.21.15.114:0/3725856397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1df390 msgr2=0x55851d5a7620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.461+0000 7f1eb3400640 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1df390 0x55851d5a7620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.461+0000 7f1eb3400640 1 -- 172.21.15.114:0/3725856397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 msgr2=0x55851d5a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.461+0000 7f1eb3400640 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.461+0000 7f1eb3400640 1 -- 172.21.15.114:0/3725856397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851d4d7cd0 con 0x55851d1d71b0 2024-09-21T07:09:53.463 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1eabe00640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e980b6680 0x7f1e980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1eb3400640 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5a70e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40e51d0 tx=0x7f1ea40f05f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7ca25159195d27 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1eabe00640 1 -- 172.21.15.114:0/4111125944 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f1ea40e7f70 con 0x7f1eb4103b80 2024-09-21T07:09:53.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e980b6680 0x7f1e980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1e9fe00640 1 -- 172.21.15.114:0/3725856397 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4026030 con 0x55851d1d71b0 2024-09-21T07:09:53.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1e9fe00640 1 -- 172.21.15.114:0/3725856397 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea40ad4a0 con 0x55851d1d71b0 2024-09-21T07:09:53.464 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.462+0000 7f1e9fe00640 1 -- 172.21.15.114:0/3725856397 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40a9830 con 0x55851d1d71b0 2024-09-21T07:09:53.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.463+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/3725856397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 msgr2=0x55851d5a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.463+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5a70e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40e51d0 tx=0x7f1ea40f05f0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.463+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e980b6680 0x7f1e980b8b90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005fd0 tx=0x7f1eac009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.465 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/3725856397 shutdown_connections 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1df390 0x55851d5a7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5a70e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/3725856397 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/3725856397 >> 172.21.15.114:0/3725856397 conn(0x55851d1f4460 msgr2=0x55851d5a4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/3725856397 shutdown_connections 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/3725856397 wait complete. 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.464+0000 7f1ec0c0d740 1 -- start start 2024-09-21T07:09:53.466 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 Processor -- start 2024-09-21T07:09:53.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5baf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1df390 0x55851d5bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55851d36e700 con 0x55851d200070 2024-09-21T07:09:53.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55851d4a8580 con 0x55851d1df390 2024-09-21T07:09:53.467 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55851d5bdab0 con 0x55851d1d71b0 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.465+0000 7f1ec0c0d740 10 client.0 _ll_register_callbacks cb 0x55851d4b8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5baf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51768/0 (socket says 172.21.15.114:51768) 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 -- 172.21.15.114:0/2663624401 learned_addr learned my addr 172.21.15.114:0/2663624401 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1df390 0x55851d5bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.468 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 -- 172.21.15.114:0/2663624401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1df390 msgr2=0x55851d5bb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1df390 0x55851d5bb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 -- 172.21.15.114:0/2663624401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 msgr2=0x55851d5c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 -- 172.21.15.114:0/2663624401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851d26b650 con 0x55851d1d71b0 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1eb3400640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5baf70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40aac00 tx=0x7f1ea4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6929143f6b3e0cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.466+0000 7f1e9cc00640 1 -- 172.21.15.114:0/2663624401 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40ad100 con 0x55851d1d71b0 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1e9cc00640 1 -- 172.21.15.114:0/2663624401 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f1ea4010a90 con 0x55851d1d71b0 2024-09-21T07:09:53.469 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55851d4aab90 con 0x55851d1d71b0 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1e9cc00640 1 -- 172.21.15.114:0/2663624401 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f1ea4097020 con 0x55851d1d71b0 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1ec0c0d740 10 client.29293 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55851d4c5950 con 0x55851d1d71b0 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1e9cc00640 1 -- 172.21.15.114:0/2663624401 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55851d4aab90 con 0x55851d1d71b0 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1e9cc00640 1 -- 172.21.15.114:0/2663624401 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f1ea40e5510 con 0x55851d1d71b0 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.467+0000 7f1ec0c0d740 10 client.29293 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.468+0000 7f1ec0c0d740 10 client.29293 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.468+0000 7f1ec0c0d740 20 client.29293 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.468+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d1be9f0 0x55851d5ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.470 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.468+0000 7f1ec0c0d740 4 client.29293 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:53.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.468+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55851d51ff00 con 0x55851d1be9f0 2024-09-21T07:09:53.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.468+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d1be9f0 0x55851d5ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.469+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d1be9f0 0x55851d5ccca0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b3f9da0860ce25c2 server_cookie=25408040c646bda7 in_seq=0 out_seq=0 2024-09-21T07:09:53.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.469+0000 7f1e9cc00640 10 client.29293 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:09:53.471 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.470+0000 7f1e9cc00640 1 -- 172.21.15.114:0/2663624401 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55851d51ff00 con 0x55851d1be9f0 2024-09-21T07:09:53.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.470+0000 7f1e9cc00640 10 client.29293 handle_command_reply: tid=0 2024-09-21T07:09:53.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.470+0000 7f1ec0c0d740 1 client.29293 shutdown 2024-09-21T07:09:53.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.470+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 msgr2=0x55851d5baf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.470+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5baf70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40aac00 tx=0x7f1ea4000bc0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.472 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 shutdown_connections 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d1be9f0 0x55851d5ccca0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d200070 0x55851d5c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1df390 0x55851d5bb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/2663624401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5baf70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1e9cc00640 0 client.29293 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 >> 172.21.15.114:0/2663624401 conn(0x55851d1f4460 msgr2=0x55851d5a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 shutdown_connections 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/2663624401 wait complete. 2024-09-21T07:09:53.473 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.471+0000 7f1ec0c0d740 20 client.29293 trim_cache size 0 max 16384 2024-09-21T07:09:53.499 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.500+0000 7f1ec0c0d740 1 Processor -- start 2024-09-21T07:09:53.501 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.500+0000 7f1ec0c0d740 1 -- start start 2024-09-21T07:09:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.500+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.500+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.500+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d200070 0x55851d5c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41aedf0 con 0x55851d1df390 2024-09-21T07:09:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41b0930 con 0x55851d1d71b0 2024-09-21T07:09:53.502 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41af4c0 con 0x55851d200070 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb2a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5bc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:59050/0 (socket says 172.21.15.114:59050) 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 -- 172.21.15.114:0/933983876 learned_addr learned my addr 172.21.15.114:0/933983876 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d200070 0x55851d5c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 -- 172.21.15.114:0/933983876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d200070 msgr2=0x55851d5c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.503 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d200070 0x55851d5c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 -- 172.21.15.114:0/933983876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 msgr2=0x55851d5c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.501+0000 7f1eb3400640 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.502+0000 7f1eb3400640 1 -- 172.21.15.114:0/933983876 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55851d5c77d0 con 0x55851d1d71b0 2024-09-21T07:09:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.502+0000 7f1eb3400640 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5bc9a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40097b0 tx=0x7f1ea4018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abeaa5a4c499cc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.504 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.502+0000 7f1e9f400640 1 -- 172.21.15.114:0/933983876 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40ad2e0 con 0x55851d1d71b0 2024-09-21T07:09:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.502+0000 7f1e9f400640 1 -- 172.21.15.114:0/933983876 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea40ad4a0 con 0x55851d1d71b0 2024-09-21T07:09:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.502+0000 7f1e9f400640 1 -- 172.21.15.114:0/933983876 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40ad760 con 0x55851d1d71b0 2024-09-21T07:09:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/933983876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 msgr2=0x55851d5bc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5bc9a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40097b0 tx=0x7f1ea4018420 comp rx=0 tx=0).stop 2024-09-21T07:09:53.505 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/933983876 shutdown_connections 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d200070 0x55851d5c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/933983876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d1d71b0 0x55851d5bc9a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/933983876 >> 172.21.15.114:0/933983876 conn(0x55851d1f4460 msgr2=0x55851d5bc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/933983876 shutdown_connections 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.503+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/933983876 wait complete. 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.504+0000 7f1ec0c0d740 1 -- start start 2024-09-21T07:09:53.506 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.504+0000 7f1ec0c0d740 1 Processor -- start 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d200070 0x55851d5cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41b0260 con 0x55851d1df390 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41afb90 con 0x55851d200070 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41aceb0 con 0x55851d1d71b0 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1ec0c0d740 10 client.0 _ll_register_callbacks cb 0x7f1ea4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51772/0 (socket says 172.21.15.114:51772) 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 -- 172.21.15.114:0/4281852067 learned_addr learned my addr 172.21.15.114:0/4281852067 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3e00640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d200070 0x55851d5cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 -- 172.21.15.114:0/4281852067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d200070 msgr2=0x55851d5cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.508 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d200070 0x55851d5cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 -- 172.21.15.114:0/4281852067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 msgr2=0x55851d5cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb3400640 1 -- 172.21.15.114:0/4281852067 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55851d636440 con 0x55851d1d71b0 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.505+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1eb3400640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5cbce0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40ad690 tx=0x7f1ea4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95fa71b48296a10f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1e9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1e9cc00640 1 -- 172.21.15.114:0/4281852067 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea40ad760 con 0x55851d1d71b0 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1e9cc00640 1 -- 172.21.15.114:0/4281852067 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f1ea40b6f20 con 0x55851d1d71b0 2024-09-21T07:09:53.509 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ea40ad4a0 con 0x55851d1d71b0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1e9cc00640 1 -- 172.21.15.114:0/4281852067 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f1ea40ad2e0 con 0x55851d1d71b0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1ec0c0d740 10 client.29299 fetch_fsmap learned FSMap version 60 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.506+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55851d636440 con 0x55851d1d71b0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1e9cc00640 1 -- 172.21.15.114:0/4281852067 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea40ad4a0 con 0x55851d1d71b0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1e9cc00640 1 -- 172.21.15.114:0/4281852067 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7f1ea40acbf0 con 0x55851d1d71b0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1ec0c0d740 10 client.29299 fetch_fsmap finished waiting for FSMap version 60 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1ec0c0d740 10 client.29299 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1ec0c0d740 20 client.29299 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d193da0 0x55851d639a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1ec0c0d740 4 client.29299 mds_command: new command op to 24727 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 --> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55851d51b9b0 con 0x55851d193da0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.507+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d193da0 0x55851d639a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1eb2a00640 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d193da0 0x55851d639a70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c09449c58ca0b99e server_cookie=4d10e3bdeb6d64a in_seq=0 out_seq=0 2024-09-21T07:09:53.511 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1e9cc00640 10 client.29299 ms_handle_connect on v2:172.21.15.193:6836/1013898612 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1e9cc00640 1 -- 172.21.15.114:0/4281852067 <== mds.3 v2:172.21.15.193:6836/1013898612 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55851d51b9b0 con 0x55851d193da0 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1e9cc00640 10 client.29299 handle_command_reply: tid=0 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1ec0c0d740 1 client.29299 shutdown 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 msgr2=0x55851d5cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.508+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5cbce0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40ad690 tx=0x7f1ea4000bc0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 shutdown_connections 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:6836/1013898612,v1:172.21.15.193:6837/1013898612] conn(0x55851d193da0 0x55851d639a70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55851d200070 0x55851d5cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55851d1df390 0x55851d5cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 --2- 172.21.15.114:0/4281852067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55851d1d71b0 0x55851d5cbce0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1e9cc00640 0 client.29299 ms_handle_reset on v2:172.21.15.193:6836/1013898612 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 >> 172.21.15.114:0/4281852067 conn(0x55851d1f4460 msgr2=0x55851d5bcbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 shutdown_connections 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 1 -- 172.21.15.114:0/4281852067 wait complete. 2024-09-21T07:09:53.512 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.509+0000 7f1ec0c0d740 20 client.29299 trim_cache size 0 max 16384 2024-09-21T07:09:53.513 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:09:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e980b6680 msgr2=0x7f1e980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e980b6680 0x7f1e980b8b90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005fd0 tx=0x7f1eac009500 comp rx=0 tx=0).stop 2024-09-21T07:09:53.513 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 msgr2=0x7f1eb419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb419c6e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e9f0 tx=0x7f1ea400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 shutdown_connections 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f1eb4109e10 0x7f1eb41a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1eb41094c0 0x7f1eb419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f1e980b6680 0x7f1e980b8b90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 --2- 172.21.15.114:0/4111125944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4103b80 0x7f1eb419c6e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 >> 172.21.15.114:0/4111125944 conn(0x7f1eb40ff590 msgr2=0x7f1eb4105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.510+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 shutdown_connections 2024-09-21T07:09:53.514 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:53.511+0000 7f1eba000640 1 -- 172.21.15.114:0/4111125944 wait complete. 2024-09-21T07:09:53.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:53 smithi183 ceph-mon[34219]: pgmap v884: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.5 MiB/s wr, 619 op/s 2024-09-21T07:09:53.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:53 smithi193 ceph-mon[34601]: pgmap v884: 129 pgs: 129 active+clean; 3.1 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 5.5 MiB/s wr, 619 op/s 2024-09-21T07:09:54.757 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 -- 172.21.15.114:0/639122595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 msgr2=0x7f182c110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 --2- 172.21.15.114:0/639122595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c110470 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1824009ca0 tx=0x7f182402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 -- 172.21.15.114:0/639122595 shutdown_connections 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 --2- 172.21.15.114:0/639122595 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c110470 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 --2- 172.21.15.114:0/639122595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f182c1092f0 0x7f182c109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 --2- 172.21.15.114:0/639122595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 0x7f182c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 -- 172.21.15.114:0/639122595 >> 172.21.15.114:0/639122595 conn(0x7f182c0ff440 msgr2=0x7f182c101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 -- 172.21.15.114:0/639122595 shutdown_connections 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.493+0000 7f182be00640 1 -- 172.21.15.114:0/639122595 wait complete. 2024-09-21T07:09:54.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.494+0000 7f182be00640 1 Processor -- start 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 -- start start 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 0x7f182c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f182c1092f0 0x7f182c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c1a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f182c113950 con 0x7f182c109c40 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f182c1137d0 con 0x7f182c1039b0 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f182c113ad0 con 0x7f182c1092f0 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c1a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c1a1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54670/0 (socket says 172.21.15.114:54670) 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 0x7f182c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182b400640 1 -- 172.21.15.114:0/2963253837 learned_addr learned my addr 172.21.15.114:0/2963253837 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.495+0000 7f182a000640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f182c1092f0 0x7f182c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:54.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f182aa00640 1 -- 172.21.15.114:0/2963253837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f182c1092f0 msgr2=0x7f182c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f182aa00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f182c1092f0 0x7f182c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f182aa00640 1 -- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 msgr2=0x7f182c1a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f182aa00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c1a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f182aa00640 1 -- 172.21.15.114:0/2963253837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c1a2430 con 0x7f182c1039b0 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f182aa00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 0x7f182c19c4c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f181c00e9f0 tx=0x7f181c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.496+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c00ccc0 con 0x7f182c1039b0 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.497+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182c1a2720 con 0x7f182c1039b0 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:09:54.866 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":61,"btime":"2024-09-21T07:09:54:234933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:09:53.402763+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.497+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f181c0108e0 con 0x7f182c1039b0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.497+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c018880 con 0x7f182c1039b0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.497+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182c1a8e80 con 0x7f182c1039b0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.498+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182c103dc0 con 0x7f182c1039b0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.500+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f181c010430 con 0x7f182c1039b0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.502+0000 7f1823400640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18100b6680 0x7f18100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.502+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f181c0e8310 con 0x7f182c1039b0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.502+0000 7f182a000640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18100b6680 0x7f18100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.503+0000 7f182a000640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18100b6680 0x7f18100b8b90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18180095f0 tx=0x7f1818005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:54.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.504+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f181c0b2810 con 0x7f182c1039b0 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.740+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f182c19d5f0 con 0x7f182c1039b0 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.741+0000 7f1823400640 1 -- 172.21.15.114:0/2963253837 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11990 (secure 0 0 0) 0x7f181c0b7750 con 0x7f182c1039b0 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 61 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18100b6680 msgr2=0x7f18100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18100b6680 0x7f18100b8b90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f18180095f0 tx=0x7f1818005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 msgr2=0x7f182c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 0x7f182c19c4c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f181c00e9f0 tx=0x7f181c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 shutdown_connections 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f182c109c40 0x7f182c1a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f182c1092f0 0x7f182c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18100b6680 0x7f18100b8b90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 --2- 172.21.15.114:0/2963253837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f182c1039b0 0x7f182c19c4c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.745+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 >> 172.21.15.114:0/2963253837 conn(0x7f182c0ff440 msgr2=0x7f182c108920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:54.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.746+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 shutdown_connections 2024-09-21T07:09:54.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:54.746+0000 7f182be00640 1 -- 172.21.15.114:0/2963253837 wait complete. 2024-09-21T07:09:55.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:55 smithi114 ceph-mon[30654]: pgmap v885: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.8 MiB/s wr, 665 op/s 2024-09-21T07:09:55.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:55 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:55.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:55 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2963253837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:55.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:55 smithi183 ceph-mon[34219]: pgmap v885: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.8 MiB/s wr, 665 op/s 2024-09-21T07:09:55.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:55 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:55.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:55 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2963253837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:55.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:55 smithi193 ceph-mon[34601]: pgmap v885: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.8 MiB/s wr, 665 op/s 2024-09-21T07:09:55.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:55 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:09:55.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:55 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2963253837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:09:57.471 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:57 smithi114 ceph-mon[30654]: pgmap v886: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-09-21T07:09:57.515 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24761 quiesce db --cancel --all' 2024-09-21T07:09:57.516 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- timeout 105 ceph tell mds.24761 quiesce db --cancel --all 2024-09-21T07:09:57.547 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:57.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:57 smithi183 ceph-mon[34219]: pgmap v886: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-09-21T07:09:57.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:57 smithi193 ceph-mon[34601]: pgmap v886: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.1 MiB/s wr, 469 op/s 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 -- 172.21.15.114:0/2050393041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 msgr2=0x7fcbd0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/2050393041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 0x7fcbd0109700 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009900 tx=0x7fcbbc02a3f0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 -- 172.21.15.114:0/2050393041 shutdown_connections 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/2050393041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0109c40 0x7fcbd0110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/2050393041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 0x7fcbd0109700 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/2050393041 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 -- 172.21.15.114:0/2050393041 >> 172.21.15.114:0/2050393041 conn(0x7fcbd00ff440 msgr2=0x7fcbd0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 -- 172.21.15.114:0/2050393041 shutdown_connections 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.302+0000 7fcbcfe00640 1 -- 172.21.15.114:0/2050393041 wait complete. 2024-09-21T07:09:58.353 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 Processor -- start 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 -- start start 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 0x7fcbd0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0109c40 0x7fcbd0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0113950 con 0x7fcbd01039b0 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01137d0 con 0x7fcbd0109c40 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.303+0000 7fcbcfe00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0113ad0 con 0x7fcbd01092f0 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbce000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 0x7fcbd0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44468/0 (socket says 172.21.15.114:44468) 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 -- 172.21.15.114:0/3994423378 learned_addr learned my addr 172.21.15.114:0/3994423378 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcf400640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0109c40 0x7fcbd0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.354 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 -- 172.21.15.114:0/3994423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 msgr2=0x7fcbd0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 0x7fcbd0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 -- 172.21.15.114:0/3994423378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0109c40 msgr2=0x7fcbd0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0109c40 0x7fcbd0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 -- 172.21.15.114:0/3994423378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd01a85c0 con 0x7fcbd01039b0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.304+0000 7fcbcea00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0075710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e9f0 tx=0x7fcbc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.305+0000 7fcbc7400640 1 -- 172.21.15.114:0/3994423378 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000cd80 con 0x7fcbd01039b0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.305+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd01a88b0 con 0x7fcbd01039b0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.305+0000 7fcbc7400640 1 -- 172.21.15.114:0/3994423378 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc00108e0 con 0x7fcbd01039b0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.305+0000 7fcbc7400640 1 -- 172.21.15.114:0/3994423378 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0018980 con 0x7fcbd01039b0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.305+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd01a8e10 con 0x7fcbd01039b0 2024-09-21T07:09:58.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.308+0000 7fcbdc1d7740 1 Processor -- start 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.308+0000 7fcbdc1d7740 1 -- start start 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973771d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733a21b0 0x55e973772270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e9737727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e9736e8fb0 con 0x55e9733cb070 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e9736d83e0 con 0x55e9733aa390 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e9736dc1f0 con 0x55e9733a21b0 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbc7400640 1 -- 172.21.15.114:0/3994423378 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fcbc0010430 con 0x7fcbd01039b0 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973771d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973771d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44474/0 (socket says 172.21.15.114:44474) 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbcf400640 1 -- 172.21.15.114:0/2799152696 learned_addr learned my addr 172.21.15.114:0/2799152696 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbce000640 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e9737727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbcea00640 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733a21b0 0x55e973772270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.362 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbcf400640 1 -- 172.21.15.114:0/2799152696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733a21b0 msgr2=0x55e973772270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.309+0000 7fcbcf400640 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733a21b0 0x55e973772270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbcf400640 1 -- 172.21.15.114:0/2799152696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 msgr2=0x55e9737727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbcf400640 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e9737727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbcf400640 1 -- 172.21.15.114:0/2799152696 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9736a2cd0 con 0x55e9733cb070 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbcf400640 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973771d30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009fd0 tx=0x7fcbc80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91cf66f547e1c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbbb400640 1 -- 172.21.15.114:0/2799152696 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8004d20 con 0x55e9733cb070 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbbb400640 1 -- 172.21.15.114:0/2799152696 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc8004050 con 0x55e9733cb070 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.310+0000 7fcbbb400640 1 -- 172.21.15.114:0/2799152696 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8005240 con 0x55e9733cb070 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbc7400640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb40b6680 0x7fcbb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/2799152696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 msgr2=0x55e973771d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973771d30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009fd0 tx=0x7fcbc80118d0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbcea00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb40b6680 0x7fcbb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/2799152696 shutdown_connections 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e9737727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733a21b0 0x55e973772270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbc7400640 1 -- 172.21.15.114:0/3994423378 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fcbc00e7e80 con 0x7fcbd01039b0 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/2799152696 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973771d30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/2799152696 >> 172.21.15.114:0/2799152696 conn(0x55e9733bf460 msgr2=0x55e97376fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/2799152696 shutdown_connections 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.311+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/2799152696 wait complete. 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.312+0000 7fcbdc1d7740 1 -- start start 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.312+0000 7fcbcea00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb40b6680 0x7fcbb40b8b90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55e9737709b0 tx=0x7fcbc0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.312+0000 7fcbdc1d7740 1 Processor -- start 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733a21b0 0x55e97377d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97377ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973789830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55e973539700 con 0x55e9733cb070 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e973673580 con 0x55e9733a21b0 2024-09-21T07:09:58.364 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e973780510 con 0x55e9733aa390 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbdc1d7740 10 client.0 _ll_register_callbacks cb 0x55e973683470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97377ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973789830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97377ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34508/0 (socket says 172.21.15.114:34508) 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 -- 172.21.15.114:0/1602328000 learned_addr learned my addr 172.21.15.114:0/1602328000 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbcf400640 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733a21b0 0x55e97377d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 -- 172.21.15.114:0/1602328000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733a21b0 msgr2=0x55e97377d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733a21b0 0x55e97377d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 -- 172.21.15.114:0/1602328000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 msgr2=0x55e973789830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973789830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.313+0000 7fcbce000640 1 -- 172.21.15.114:0/1602328000 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e973436650 con 0x55e9733aa390 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbce000640 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97377ded0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcbd006b990 tx=0x7fcbbc0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35181f48b1379e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:58.365 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbafe00640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbafe00640 1 -- 172.21.15.114:0/1602328000 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc035040 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e973675b90 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbafe00640 1 -- 172.21.15.114:0/1602328000 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fcbbc005450 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbafe00640 1 -- 172.21.15.114:0/1602328000 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcbbc04f050 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbdc1d7740 10 client.29317 fetch_fsmap learned FSMap version 61 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.314+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e973690950 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbafe00640 1 -- 172.21.15.114:0/1602328000 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e973675b90 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbafe00640 1 -- 172.21.15.114:0/1602328000 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fcbbc0037f0 con 0x55e9733aa390 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbdc1d7740 10 client.29317 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbdc1d7740 10 client.29317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbdc1d7740 20 client.29317 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e9733899f0 0x55e97378f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbdc1d7740 4 client.29317 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.315+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9736eaf00 con 0x55e9733899f0 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.316+0000 7fcbcf400640 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e9733899f0 0x55e97378f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.316+0000 7fcbcf400640 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e9733899f0 0x55e97378f920 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d2cdbec638404a1 server_cookie=b38767e62f517fd3 in_seq=0 out_seq=0 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.316+0000 7fcbafe00640 10 client.29317 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbafe00640 1 -- 172.21.15.114:0/1602328000 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9736eaf00 con 0x55e9733899f0 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbafe00640 10 client.29317 handle_command_reply: tid=0 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 client.29317 shutdown 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 msgr2=0x55e97377ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97377ded0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcbd006b990 tx=0x7fcbbc0043b0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 shutdown_connections 2024-09-21T07:09:58.367 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e9733899f0 0x55e97378f920 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733cb070 0x55e973789830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97377ded0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbafe00640 0 client.29317 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/1602328000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733a21b0 0x55e97377d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 >> 172.21.15.114:0/1602328000 conn(0x55e9733bf460 msgr2=0x55e973774f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 shutdown_connections 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.317+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/1602328000 wait complete. 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.318+0000 7fcbdc1d7740 20 client.29317 trim_cache size 0 max 16384 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 Processor -- start 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 -- start start 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973778f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97376fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733cb070 0x55e97377a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01b3010 con 0x55e9733a21b0 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01b4b80 con 0x55e9733cb070 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01b3710 con 0x55e9733aa390 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973778f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973778f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44492/0 (socket says 172.21.15.114:44492) 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbcf400640 1 -- 172.21.15.114:0/790367120 learned_addr learned my addr 172.21.15.114:0/790367120 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.332+0000 7fcbce000640 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97376fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733cb070 0x55e97377a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 -- 172.21.15.114:0/790367120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 msgr2=0x55e97376fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97376fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 -- 172.21.15.114:0/790367120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733cb070 msgr2=0x55e97377a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733cb070 0x55e97377a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 -- 172.21.15.114:0/790367120 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e973769dc0 con 0x55e9733a21b0 2024-09-21T07:09:58.369 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbcf400640 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973778f20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8006010 tx=0x7fcbc8004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48606e5ac459867f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbbaa00640 1 -- 172.21.15.114:0/790367120 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8034070 con 0x55e9733a21b0 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbbaa00640 1 -- 172.21.15.114:0/790367120 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc8004ca0 con 0x55e9733a21b0 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.333+0000 7fcbbaa00640 1 -- 172.21.15.114:0/790367120 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80053e0 con 0x55e9733a21b0 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/790367120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 msgr2=0x55e973778f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973778f20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8006010 tx=0x7fcbc8004890 comp rx=0 tx=0).stop 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/790367120 shutdown_connections 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733cb070 0x55e97377a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733aa390 0x55e97376fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/790367120 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973778f20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/790367120 >> 172.21.15.114:0/790367120 conn(0x55e9733bf460 msgr2=0x55e97376f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/790367120 shutdown_connections 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.334+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/790367120 wait complete. 2024-09-21T07:09:58.370 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 -- start start 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 Processor -- start 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973785c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e973780750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733cb070 0x55e973780c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01b44b0 con 0x55e9733a21b0 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01b3de0 con 0x55e9733aa390 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01b1010 con 0x55e9733cb070 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.335+0000 7fcbdc1d7740 10 client.0 _ll_register_callbacks cb 0x7fcbbc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973785c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973785c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44500/0 (socket says 172.21.15.114:44500) 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 -- 172.21.15.114:0/3922010725 learned_addr learned my addr 172.21.15.114:0/3922010725 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbce000640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e973780750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733cb070 0x55e973780c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.371 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 -- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733cb070 msgr2=0x55e973780c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733cb070 0x55e973780c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 -- 172.21.15.114:0/3922010725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 msgr2=0x55e973780750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:58.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e973780750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 -- 172.21.15.114:0/3922010725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9737811d0 con 0x55e9733a21b0 2024-09-21T07:09:58.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbcf400640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973785c70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8000f80 tx=0x7fcbc8011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d54f5a1944156fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:58.372 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.336+0000 7fcbafe00640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:09:58.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbafe00640 1 -- 172.21.15.114:0/3922010725 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8034050 con 0x55e9733a21b0 2024-09-21T07:09:58.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbafe00640 1 -- 172.21.15.114:0/3922010725 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fcbc80041f0 con 0x55e9733a21b0 2024-09-21T07:09:58.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc8004ca0 con 0x55e9733a21b0 2024-09-21T07:09:58.373 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbafe00640 1 -- 172.21.15.114:0/3922010725 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcbc80043f0 con 0x55e9733a21b0 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbdc1d7740 10 client.18669 fetch_fsmap learned FSMap version 61 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55e9737faf60 con 0x55e9733a21b0 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbafe00640 1 -- 172.21.15.114:0/3922010725 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc8004ca0 con 0x55e9733a21b0 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.337+0000 7fcbafe00640 1 -- 172.21.15.114:0/3922010725 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fcbc8026040 con 0x55e9733a21b0 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbdc1d7740 10 client.18669 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbdc1d7740 10 client.18669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbdc1d7740 20 client.18669 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e97335eda0 0x55e9737fd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbdc1d7740 4 client.18669 mds_command: new command op to 24742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:09:58.374 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 --> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9736e69b0 con 0x55e97335eda0 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbce000640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e97335eda0 0x55e9737fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbce000640 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e97335eda0 0x55e9737fd400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e6c8f4500b1f4dfd server_cookie=3785794dd9a8b6cc in_seq=0 out_seq=0 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.338+0000 7fcbafe00640 10 client.18669 ms_handle_connect on v2:172.21.15.193:6838/3226153327 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbafe00640 1 -- 172.21.15.114:0/3922010725 <== mds.2 v2:172.21.15.193:6838/3226153327 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9736e69b0 con 0x55e97335eda0 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbafe00640 10 client.18669 handle_command_reply: tid=0 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 client.18669 shutdown 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 msgr2=0x55e973785c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973785c70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8000f80 tx=0x7fcbc8011e10 comp rx=0 tx=0).stop 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 shutdown_connections 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:6838/3226153327,v1:172.21.15.193:6839/3226153327] conn(0x55e97335eda0 0x55e9737fd400 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e9733cb070 0x55e973780c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e9733aa390 0x55e973780750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 --2- 172.21.15.114:0/3922010725 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55e9733a21b0 0x55e973785c70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.375 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbafe00640 0 client.18669 ms_handle_reset on v2:172.21.15.193:6838/3226153327 2024-09-21T07:09:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 >> 172.21.15.114:0/3922010725 conn(0x55e9733bf460 msgr2=0x55e97376e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 shutdown_connections 2024-09-21T07:09:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 1 -- 172.21.15.114:0/3922010725 wait complete. 2024-09-21T07:09:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.339+0000 7fcbdc1d7740 20 client.18669 trim_cache size 0 max 16384 2024-09-21T07:09:58.376 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:09:58.376 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb40b6680 msgr2=0x7fcbb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb40b6680 0x7fcbb40b8b90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55e9737709b0 tx=0x7fcbc0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 msgr2=0x7fcbd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0075710 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e9f0 tx=0x7fcbc000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 shutdown_connections 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fcbb40b6680 0x7fcbb40b8b90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbd0109c40 0x7fcbd0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbd01092f0 0x7fcbd0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 --2- 172.21.15.114:0/3994423378 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fcbd01039b0 0x7fcbd0075710 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 >> 172.21.15.114:0/3994423378 conn(0x7fcbd00ff440 msgr2=0x7fcbd0101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 shutdown_connections 2024-09-21T07:09:58.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:58.340+0000 7fcbcfe00640 1 -- 172.21.15.114:0/3994423378 wait complete. 2024-09-21T07:09:58.609 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:09:59.146 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T07:09:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 -- 172.21.15.114:0/1851151339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd81410fe60 msgr2=0x7fd8141122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 --2- 172.21.15.114:0/1851151339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd81410fe60 0x7fd8141122c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009ca0 tx=0x7fd80c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 -- 172.21.15.114:0/1851151339 shutdown_connections 2024-09-21T07:09:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 --2- 172.21.15.114:0/1851151339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd81410fe60 0x7fd8141122c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 --2- 172.21.15.114:0/1851151339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd814076100 0x7fd81410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 --2- 172.21.15.114:0/1851151339 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8140757b0 0x7fd814075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 -- 172.21.15.114:0/1851151339 >> 172.21.15.114:0/1851151339 conn(0x7fd8140ff330 msgr2=0x7fd814101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 -- 172.21.15.114:0/1851151339 shutdown_connections 2024-09-21T07:09:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.247+0000 7fd819200640 1 -- 172.21.15.114:0/1851151339 wait complete. 2024-09-21T07:09:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.248+0000 7fd819200640 1 Processor -- start 2024-09-21T07:09:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 -- start start 2024-09-21T07:09:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8140757b0 0x7fd81419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 0x7fd81419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.251 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81410fe60 0x7fd8141a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd814115a50 con 0x7fd8140757b0 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141158d0 con 0x7fd814076100 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd819200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd814115bd0 con 0x7fd81410fe60 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd812a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 0x7fd81419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd813400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8140757b0 0x7fd81419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd813e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81410fe60 0x7fd8141a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd812a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 0x7fd81419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42690/0 (socket says 172.21.15.114:42690) 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.249+0000 7fd812a00640 1 -- 172.21.15.114:0/3061563357 learned_addr learned my addr 172.21.15.114:0/3061563357 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.250+0000 7fd812a00640 1 -- 172.21.15.114:0/3061563357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81410fe60 msgr2=0x7fd8141a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.250+0000 7fd812a00640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81410fe60 0x7fd8141a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.250+0000 7fd812a00640 1 -- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8140757b0 msgr2=0x7fd81419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.250+0000 7fd812a00640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8140757b0 0x7fd81419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.250+0000 7fd812a00640 1 -- 172.21.15.114:0/3061563357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8141a2340 con 0x7fd814076100 2024-09-21T07:09:59.252 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.250+0000 7fd812a00640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 0x7fd81419c9b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e9c0 tx=0x7fd80000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.254+0000 7fd80be00640 1 -- 172.21.15.114:0/3061563357 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80000ccc0 con 0x7fd814076100 2024-09-21T07:09:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.254+0000 7fd80be00640 1 -- 172.21.15.114:0/3061563357 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8000054d0 con 0x7fd814076100 2024-09-21T07:09:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.254+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8141a2630 con 0x7fd814076100 2024-09-21T07:09:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.255+0000 7fd80be00640 1 -- 172.21.15.114:0/3061563357 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800010430 con 0x7fd814076100 2024-09-21T07:09:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.255+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8141a8dc0 con 0x7fd814076100 2024-09-21T07:09:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.258+0000 7fd80be00640 1 -- 172.21.15.114:0/3061563357 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd800005020 con 0x7fd814076100 2024-09-21T07:09:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.258+0000 7fd82181d740 1 Processor -- start 2024-09-21T07:09:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.258+0000 7fd82181d740 1 -- start start 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd82181d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c700e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd82181d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c701350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd82181d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c701890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd82181d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd0c6766b0 con 0x55bd0c33b4d0 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd82181d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0c665ae0 con 0x55bd0c3332f0 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd82181d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd0c6698f0 con 0x55bd0c35c1b0 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd812a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c701890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c701350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c700e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd812a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c701890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44530/0 (socket says 172.21.15.114:44530) 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c701350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42698/0 (socket says 172.21.15.114:42698) 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c700e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34542/0 (socket says 172.21.15.114:34542) 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813400640 1 -- 172.21.15.114:0/2762488625 learned_addr learned my addr 172.21.15.114:0/2762488625 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 -- 172.21.15.114:0/2762488625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 msgr2=0x55bd0c701350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c701350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 -- 172.21.15.114:0/2762488625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 msgr2=0x55bd0c701890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c701890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813e00640 1 -- 172.21.15.114:0/2762488625 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0c632780 con 0x55bd0c35c1b0 2024-09-21T07:09:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.259+0000 7fd813400640 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c701350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:59.263 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd813e00640 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c700e10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02ade0 tx=0x7fd80c02f410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37866376d1bf61a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd80be00640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7f80b6700 0x7fd7f80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd808c00640 1 -- 172.21.15.114:0/2762488625 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c031cf0 con 0x55bd0c35c1b0 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd813400640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7f80b6700 0x7fd7f80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd80be00640 1 -- 172.21.15.114:0/3061563357 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fd8000e7e30 con 0x7fd814076100 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd808c00640 1 -- 172.21.15.114:0/2762488625 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c036070 con 0x55bd0c35c1b0 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.260+0000 7fd808c00640 1 -- 172.21.15.114:0/2762488625 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004040 con 0x55bd0c35c1b0 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.261+0000 7fd82181d740 1 -- 172.21.15.114:0/2762488625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 msgr2=0x55bd0c700e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.261+0000 7fd813400640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7f80b6700 0x7fd7f80b8c10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55bd0c6ffa90 tx=0x7fd804005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.261+0000 7fd82181d740 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c700e10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02ade0 tx=0x7fd80c02f410 comp rx=0 tx=0).stop 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 -- 172.21.15.114:0/2762488625 shutdown_connections 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c701890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c701350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 --2- 172.21.15.114:0/2762488625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c700e10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.264 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 -- 172.21.15.114:0/2762488625 >> 172.21.15.114:0/2762488625 conn(0x55bd0c350300 msgr2=0x55bd0c6fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 -- 172.21.15.114:0/2762488625 shutdown_connections 2024-09-21T07:09:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 -- 172.21.15.114:0/2762488625 wait complete. 2024-09-21T07:09:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 -- start start 2024-09-21T07:09:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.262+0000 7fd82181d740 1 Processor -- start 2024-09-21T07:09:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c70c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c7186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55bd0c59dab0 con 0x55bd0c33b4d0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0c60c570 con 0x55bd0c3332f0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bd0c70f3b0 con 0x55bd0c35c1b0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd82181d740 10 client.0 _ll_register_callbacks cb 0x55bd0c61b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd812a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd812a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44540/0 (socket says 172.21.15.114:44540) 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd812a00640 1 -- 172.21.15.114:0/2141440849 learned_addr learned my addr 172.21.15.114:0/2141440849 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd813e00640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c7186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.263+0000 7fd813400640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c70c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd812a00640 1 -- 172.21.15.114:0/2141440849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 msgr2=0x55bd0c7186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd812a00640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c7186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd812a00640 1 -- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 msgr2=0x55bd0c70c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd812a00640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c70c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd812a00640 1 -- 172.21.15.114:0/2141440849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0c3c7790 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd812a00640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70c9c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c6a0 tx=0x7fd800009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ae72573ef02aa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd7fd600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd7fd600640 1 -- 172.21.15.114:0/2141440849 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000ad620 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd7fd600640 1 -- 172.21.15.114:0/2141440849 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fd8000a91a0 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.264+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80c036070 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd7fd600640 1 -- 172.21.15.114:0/2141440849 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd80000ce60 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 10 client.18681 fetch_fsmap learned FSMap version 61 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55bd0c440620 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd7fd600640 1 -- 172.21.15.114:0/2141440849 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c036070 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd7fd600640 1 -- 172.21.15.114:0/2141440849 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fd8000a93a0 con 0x55bd0c33b4d0 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 10 client.18681 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 10 client.18681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 20 client.18681 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c31ad80 0x55bd0c71e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 4 client.18681 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd0c678600 con 0x55bd0c31ad80 2024-09-21T07:09:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.265+0000 7fd813400640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c31ad80 0x55bd0c71e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.266+0000 7fd7fd600640 10 client.18681 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.266+0000 7fd813400640 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c31ad80 0x55bd0c71e3e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ffbfc87f11262d6 server_cookie=6cdd10ac977152a in_seq=0 out_seq=0 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd7fd600640 1 -- 172.21.15.114:0/2141440849 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd0c678600 con 0x55bd0c31ad80 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd7fd600640 10 client.18681 handle_command_reply: tid=0 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 client.18681 shutdown 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 msgr2=0x55bd0c70c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70c9c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c6a0 tx=0x7fd800009310 comp rx=0 tx=0).stop 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 shutdown_connections 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c31ad80 0x55bd0c71e3e0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c35c1b0 0x55bd0c7186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70c9c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd7fd600640 0 client.18681 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 --2- 172.21.15.114:0/2141440849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c3332f0 0x55bd0c70c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 >> 172.21.15.114:0/2141440849 conn(0x55bd0c350300 msgr2=0x55bd0c701e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 shutdown_connections 2024-09-21T07:09:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.267+0000 7fd82181d740 1 -- 172.21.15.114:0/2141440849 wait complete. 2024-09-21T07:09:59.272 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.268+0000 7fd82181d740 20 client.18681 trim_cache size 0 max 16384 2024-09-21T07:09:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 Processor -- start 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 -- start start 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c6fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 0x55bd0c723b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141b2e30 con 0x55bd0c33b4d0 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141b49a0 con 0x55bd0c35c1b0 2024-09-21T07:09:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd82181d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141b3530 con 0x55bd0c3332f0 2024-09-21T07:09:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd812a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.283+0000 7fd813e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 0x55bd0c723b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 0x55bd0c723b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42710/0 (socket says 172.21.15.114:42710) 2024-09-21T07:09:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 -- 172.21.15.114:0/818219717 learned_addr learned my addr 172.21.15.114:0/818219717 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813400640 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c6fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 -- 172.21.15.114:0/818219717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 msgr2=0x55bd0c6fdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c6fdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 -- 172.21.15.114:0/818219717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 msgr2=0x55bd0c70e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 -- 172.21.15.114:0/818219717 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0c703110 con 0x55bd0c35c1b0 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd812a00640 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.284+0000 7fd813e00640 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 0x55bd0c723b50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02f500 tx=0x7fd80c003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=150ed8dd2f445fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.285+0000 7fd7ffe00640 1 -- 172.21.15.114:0/818219717 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c052020 con 0x55bd0c35c1b0 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.285+0000 7fd7ffe00640 1 -- 172.21.15.114:0/818219717 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c036040 con 0x55bd0c35c1b0 2024-09-21T07:09:59.287 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.285+0000 7fd7ffe00640 1 -- 172.21.15.114:0/818219717 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c02fa80 con 0x55bd0c35c1b0 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.285+0000 7fd82181d740 1 -- 172.21.15.114:0/818219717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 msgr2=0x55bd0c723b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.285+0000 7fd82181d740 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 0x55bd0c723b50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd80c02f500 tx=0x7fd80c003b80 comp rx=0 tx=0).stop 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 -- 172.21.15.114:0/818219717 shutdown_connections 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c35c1b0 0x55bd0c723b50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c33b4d0 0x55bd0c70e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 --2- 172.21.15.114:0/818219717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c6fdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.289 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 -- 172.21.15.114:0/818219717 >> 172.21.15.114:0/818219717 conn(0x55bd0c350300 msgr2=0x55bd0c70d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 -- 172.21.15.114:0/818219717 shutdown_connections 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 -- 172.21.15.114:0/818219717 wait complete. 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 -- start start 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.286+0000 7fd82181d740 1 Processor -- start 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c712080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c33b4d0 0x55bd0c712620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c35c1b0 0x55bd0c780d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141b42d0 con 0x55bd0c35c1b0 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141b3c00 con 0x55bd0c33b4d0 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd8141b0e30 con 0x55bd0c3332f0 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd82181d740 10 client.0 _ll_register_callbacks cb 0x7fd8000a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c712080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd812a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c33b4d0 0x55bd0c712620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c712080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34570/0 (socket says 172.21.15.114:34570) 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 -- 172.21.15.114:0/2864873513 learned_addr learned my addr 172.21.15.114:0/2864873513 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813e00640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c35c1b0 0x55bd0c780d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 -- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c33b4d0 msgr2=0x55bd0c712620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c33b4d0 0x55bd0c712620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 -- 172.21.15.114:0/2864873513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c35c1b0 msgr2=0x55bd0c780d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c35c1b0 0x55bd0c780d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd813400640 1 -- 172.21.15.114:0/2864873513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0c781400 con 0x55bd0c3332f0 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.287+0000 7fd812a00640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c33b4d0 0x55bd0c712620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd813400640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c712080 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55bd0c71eaf0 tx=0x7fd8040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b87b0b424cdaff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd7fd600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd7fd600640 1 -- 172.21.15.114:0/2864873513 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804005020 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd80c036040 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd7fd600640 1 -- 172.21.15.114:0/2864873513 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fd8040051c0 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd7fd600640 1 -- 172.21.15.114:0/2864873513 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd80403b050 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd82181d740 10 client.29341 fetch_fsmap learned FSMap version 61 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.288+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55bd0c788880 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd7fd600640 1 -- 172.21.15.114:0/2864873513 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c036040 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd7fd600640 1 -- 172.21.15.114:0/2864873513 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fd80402f070 con 0x55bd0c3332f0 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd82181d740 10 client.29341 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd82181d740 10 client.29341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd82181d740 20 client.29341 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd82181d740 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c2f01c0 0x55bd0c78b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd82181d740 4 client.29341 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T07:09:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd0c6740b0 con 0x55bd0c2f01c0 2024-09-21T07:09:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.289+0000 7fd812a00640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c2f01c0 0x55bd0c78b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.290+0000 7fd7fd600640 10 client.29341 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.290+0000 7fd812a00640 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c2f01c0 0x55bd0c78b070 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=583a94a87b02934e server_cookie=47ae002cc264227c in_seq=0 out_seq=0 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd7fd600640 1 -- 172.21.15.114:0/2864873513 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd0c6740b0 con 0x55bd0c2f01c0 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd7fd600640 10 client.29341 handle_command_reply: tid=0 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 client.29341 shutdown 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 msgr2=0x55bd0c712080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c712080 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55bd0c71eaf0 tx=0x7fd8040046c0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 shutdown_connections 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55bd0c35c1b0 0x55bd0c780d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0c33b4d0 0x55bd0c712620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55bd0c2f01c0 0x55bd0c78b070 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 --2- 172.21.15.114:0/2864873513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bd0c3332f0 0x55bd0c712080 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 >> 172.21.15.114:0/2864873513 conn(0x55bd0c350300 msgr2=0x55bd0c6fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd7fd600640 0 client.29341 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 shutdown_connections 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 1 -- 172.21.15.114:0/2864873513 wait complete. 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.311+0000 7fd82181d740 20 client.29341 trim_cache size 0 max 16384 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:{ 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "message": "", 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout: "return_code": 0 2024-09-21T07:09:59.314 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stdout:} 2024-09-21T07:09:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7f80b6700 msgr2=0x7fd7f80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7f80b6700 0x7fd7f80b8c10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55bd0c6ffa90 tx=0x7fd804005f90 comp rx=0 tx=0).stop 2024-09-21T07:09:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 msgr2=0x7fd81419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 0x7fd81419c9b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e9c0 tx=0x7fd80000ee80 comp rx=0 tx=0).stop 2024-09-21T07:09:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 shutdown_connections 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81410fe60 0x7fd8141a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd7f80b6700 0x7fd7f80b8c10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd814076100 0x7fd81419c9b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 --2- 172.21.15.114:0/3061563357 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd8140757b0 0x7fd81419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 >> 172.21.15.114:0/3061563357 conn(0x7fd8140ff330 msgr2=0x7fd814109970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 shutdown_connections 2024-09-21T07:09:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi114.front.sepia.ceph.com.smithi114.stderr:2024-09-21T07:09:59.312+0000 7fd819200640 1 -- 172.21.15.114:0/3061563357 wait complete. 2024-09-21T07:09:59.450 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:09:59 smithi114 ceph-mon[30654]: pgmap v887: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-09-21T07:09:59.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:09:59 smithi183 ceph-mon[34219]: pgmap v887: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-09-21T07:09:59.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:09:59 smithi193 ceph-mon[34601]: pgmap v887: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 3.7 MiB/s wr, 433 op/s 2024-09-21T07:09:59.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 -- 172.21.15.114:0/4069970195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010b6d0 msgr2=0x7f57e010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 --2- 172.21.15.114:0/4069970195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010b6d0 0x7f57e010bae0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009900 tx=0x7f57cc02a410 comp rx=0 tx=0).stop 2024-09-21T07:09:59.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 -- 172.21.15.114:0/4069970195 shutdown_connections 2024-09-21T07:09:59.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 --2- 172.21.15.114:0/4069970195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e010c020 0x7f57e0112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 --2- 172.21.15.114:0/4069970195 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010b6d0 0x7f57e010bae0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 --2- 172.21.15.114:0/4069970195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57e0105d90 0x7f57e01061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 -- 172.21.15.114:0/4069970195 >> 172.21.15.114:0/4069970195 conn(0x7f57e0101860 msgr2=0x7f57e0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 -- 172.21.15.114:0/4069970195 shutdown_connections 2024-09-21T07:09:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.822+0000 7f57e6400640 1 -- 172.21.15.114:0/4069970195 wait complete. 2024-09-21T07:09:59.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.823+0000 7f57e6400640 1 Processor -- start 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 -- start start 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 0x7f57e019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57e010b6d0 0x7f57e019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010c020 0x7f57e01a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0115d30 con 0x7f57e010c020 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0115bb0 con 0x7f57e010b6d0 2024-09-21T07:09:59.826 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57e6400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0115eb0 con 0x7f57e0105d90 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 0x7f57e019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010c020 0x7f57e01a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57df400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 0x7f57e019c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34590/0 (socket says 172.21.15.114:34590) 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57df400640 1 -- 172.21.15.114:0/393155691 learned_addr learned my addr 172.21.15.114:0/393155691 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.824+0000 7f57dea00640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57e010b6d0 0x7f57e019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57df400640 1 -- 172.21.15.114:0/393155691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57e010b6d0 msgr2=0x7f57e019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57df400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57e010b6d0 0x7f57e019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57df400640 1 -- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010c020 msgr2=0x7f57e01a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57df400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010c020 0x7f57e01a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57df400640 1 -- 172.21.15.114:0/393155691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e01a2630 con 0x7f57e0105d90 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57dfe00640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010c020 0x7f57e01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.825+0000 7f57df400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 0x7f57e019c6c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e9f0 tx=0x7f57d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.826+0000 7f57d7e00640 1 -- 172.21.15.114:0/393155691 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000cd80 con 0x7f57e0105d90 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.826+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e01a2920 con 0x7f57e0105d90 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.826+0000 7f57d7e00640 1 -- 172.21.15.114:0/393155691 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d00108e0 con 0x7f57e0105d90 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.826+0000 7f57d7e00640 1 -- 172.21.15.114:0/393155691 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0018980 con 0x7f57e0105d90 2024-09-21T07:09:59.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.826+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01a9080 con 0x7f57e0105d90 2024-09-21T07:09:59.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 Processor -- start 2024-09-21T07:09:59.830 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 -- start start 2024-09-21T07:09:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b292290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b2927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b292d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a73b209ff0 con 0x55a73aeec2d0 2024-09-21T07:09:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a73b1f9420 con 0x55a73aec3410 2024-09-21T07:09:59.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.829+0000 7f57ed036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a73b1fd230 con 0x55a73aecb5f0 2024-09-21T07:09:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b292290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b292290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44586/0 (socket says 172.21.15.114:44586) 2024-09-21T07:09:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 -- 172.21.15.114:0/1977355895 learned_addr learned my addr 172.21.15.114:0/1977355895 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dea00640 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b292d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57d7e00640 1 -- 172.21.15.114:0/393155691 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f57d0010430 con 0x7f57e0105d90 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 -- 172.21.15.114:0/1977355895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 msgr2=0x55a73b292d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b292d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 -- 172.21.15.114:0/1977355895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 msgr2=0x55a73b2927d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b2927d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.830+0000 7f57dfe00640 1 -- 172.21.15.114:0/1977355895 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73af578b0 con 0x55a73aeec2d0 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.831+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b292290 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f57d80117e0 tx=0x7f57d8011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d39325a574e95f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.831+0000 7f57d4c00640 1 -- 172.21.15.114:0/1977355895 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8011310 con 0x55a73aeec2d0 2024-09-21T07:09:59.833 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.831+0000 7f57d4c00640 1 -- 172.21.15.114:0/1977355895 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d8003e20 con 0x55a73aeec2d0 2024-09-21T07:09:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.831+0000 7f57d4c00640 1 -- 172.21.15.114:0/1977355895 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8004490 con 0x55a73aeec2d0 2024-09-21T07:09:59.834 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57d7e00640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f57c40b6680 0x7f57c40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57d7e00640 1 -- 172.21.15.114:0/393155691 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f57d00e7e70 con 0x7f57e0105d90 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57df400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f57c40b6680 0x7f57c40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 -- 172.21.15.114:0/1977355895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 msgr2=0x55a73b292290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b292290 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f57d80117e0 tx=0x7f57d8011ca0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 -- 172.21.15.114:0/1977355895 shutdown_connections 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b292d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b2927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 --2- 172.21.15.114:0/1977355895 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b292290 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.832+0000 7f57ed036740 1 -- 172.21.15.114:0/1977355895 >> 172.21.15.114:0/1977355895 conn(0x55a73aee06c0 msgr2=0x55a73b290180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.833+0000 7f57ed036740 1 -- 172.21.15.114:0/1977355895 shutdown_connections 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.833+0000 7f57ed036740 1 -- 172.21.15.114:0/1977355895 wait complete. 2024-09-21T07:09:59.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.833+0000 7f57df400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f57c40b6680 0x7f57c40b8b90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55a73b290f10 tx=0x7f57d00aa020 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.833+0000 7f57ed036740 1 -- start start 2024-09-21T07:09:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 Processor -- start 2024-09-21T07:09:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b29dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b2a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a73b05a810 con 0x55a73aeec2d0 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a73b194550 con 0x55a73aec3410 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a73b2a0a70 con 0x55a73aecb5f0 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.834+0000 7f57ed036740 10 client.0 _ll_register_callbacks cb 0x55a73b1a4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b29dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b29dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34608/0 (socket says 172.21.15.114:34608) 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 -- 172.21.15.114:0/1769665575 learned_addr learned my addr 172.21.15.114:0/1769665575 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b2a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.837 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 -- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 msgr2=0x55a73b29d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 -- 172.21.15.114:0/1769665575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 msgr2=0x55a73b2a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b2a9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dea00640 1 -- 172.21.15.114:0/1769665575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73b186200 con 0x55a73aecb5f0 2024-09-21T07:09:59.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.835+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57dea00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b29dea0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f57e006ca80 tx=0x7f57cc0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff1a10a8141a103 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57c9600640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57c9600640 1 -- 172.21.15.114:0/1769665575 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc035040 con 0x55a73aecb5f0 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57c9600640 1 -- 172.21.15.114:0/1769665575 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f57cc005450 con 0x55a73aecb5f0 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d8003e20 con 0x55a73aecb5f0 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57c9600640 1 -- 172.21.15.114:0/1769665575 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f57cc04f050 con 0x55a73aecb5f0 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57ed036740 10 client.29353 fetch_fsmap learned FSMap version 61 2024-09-21T07:09:59.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.836+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a73b2ac740 con 0x55a73aecb5f0 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57c9600640 1 -- 172.21.15.114:0/1769665575 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d8003e20 con 0x55a73aecb5f0 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57c9600640 1 -- 172.21.15.114:0/1769665575 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f57cc003860 con 0x55a73aecb5f0 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57ed036740 10 client.29353 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57ed036740 10 client.29353 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57ed036740 20 client.29353 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57ed036740 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73aeaac50 0x55a73b2afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57ed036740 4 client.29353 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.837+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a73b20bf40 con 0x55a73aeaac50 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.838+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73aeaac50 0x55a73b2afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.838+0000 7f57dfe00640 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73aeaac50 0x55a73b2afc60 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbaae70333a165cc server_cookie=5a31b70583081b34 in_seq=0 out_seq=0 2024-09-21T07:09:59.840 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.838+0000 7f57c9600640 10 client.29353 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.839+0000 7f57c9600640 1 -- 172.21.15.114:0/1769665575 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a73b20bf40 con 0x55a73aeaac50 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.839+0000 7f57c9600640 10 client.29353 handle_command_reply: tid=0 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.839+0000 7f57ed036740 1 client.29353 shutdown 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 msgr2=0x55a73b29dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b29dea0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f57e006ca80 tx=0x7f57cc0043b0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 shutdown_connections 2024-09-21T07:09:59.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b2a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73aeaac50 0x55a73b2afc60 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b29dea0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 --2- 172.21.15.114:0/1769665575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57c9600640 0 client.29353 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 >> 172.21.15.114:0/1769665575 conn(0x55a73aee06c0 msgr2=0x55a73b295470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.840+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 shutdown_connections 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.841+0000 7f57ed036740 1 -- 172.21.15.114:0/1769665575 wait complete. 2024-09-21T07:09:59.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.841+0000 7f57ed036740 20 client.29353 trim_cache size 0 max 16384 2024-09-21T07:09:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 Processor -- start 2024-09-21T07:09:59.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 -- start start 2024-09-21T07:09:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b2a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b29ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b3180 con 0x55a73aeec2d0 2024-09-21T07:09:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b4cf0 con 0x55a73aec3410 2024-09-21T07:09:59.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.859+0000 7f57ed036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b3880 con 0x55a73aecb5f0 2024-09-21T07:09:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b29ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b29ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44598/0 (socket says 172.21.15.114:44598) 2024-09-21T07:09:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 -- 172.21.15.114:0/3096031056 learned_addr learned my addr 172.21.15.114:0/3096031056 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dea00640 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b2a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.862 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 -- 172.21.15.114:0/3096031056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 msgr2=0x55a73b2a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b2a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 -- 172.21.15.114:0/3096031056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 msgr2=0x55a73b29f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dfe00640 1 -- 172.21.15.114:0/3096031056 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73b2985c0 con 0x55a73aeec2d0 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.860+0000 7f57dea00640 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b2a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.861+0000 7f57dfe00640 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b29ab50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f57d8003be0 tx=0x7f57d80040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1a80df0d9e0dda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.861+0000 7f57cbe00640 1 -- 172.21.15.114:0/3096031056 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8025040 con 0x55a73aeec2d0 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.861+0000 7f57cbe00640 1 -- 172.21.15.114:0/3096031056 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d8005730 con 0x55a73aeec2d0 2024-09-21T07:09:59.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.861+0000 7f57cbe00640 1 -- 172.21.15.114:0/3096031056 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d802d410 con 0x55a73aeec2d0 2024-09-21T07:09:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 -- 172.21.15.114:0/3096031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 msgr2=0x55a73b29ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b29ab50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f57d8003be0 tx=0x7f57d80040d0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 -- 172.21.15.114:0/3096031056 shutdown_connections 2024-09-21T07:09:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aeec2d0 0x55a73b29ab50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aecb5f0 0x55a73b2a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 --2- 172.21.15.114:0/3096031056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aec3410 0x55a73b29f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 -- 172.21.15.114:0/3096031056 >> 172.21.15.114:0/3096031056 conn(0x55a73aee06c0 msgr2=0x55a73b29ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 -- 172.21.15.114:0/3096031056 shutdown_connections 2024-09-21T07:09:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.862+0000 7f57ed036740 1 -- 172.21.15.114:0/3096031056 wait complete. 2024-09-21T07:09:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 -- start start 2024-09-21T07:09:59.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 Processor -- start 2024-09-21T07:09:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 0x55a73b2a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 0x55a73b2a6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aeec2d0 0x55a73b2a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.866 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b4620 con 0x55a73aec3410 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b3f50 con 0x55a73aecb5f0 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f57e01b1180 con 0x55a73aeec2d0 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.863+0000 7f57ed036740 10 client.0 _ll_register_callbacks cb 0x7f57cc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 0x55a73b2a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 0x55a73b2a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 0x55a73b2a6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42770/0 (socket says 172.21.15.114:42770) 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dfe00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 0x55a73b2a5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44600/0 (socket says 172.21.15.114:44600) 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 -- 172.21.15.114:0/131161926 learned_addr learned my addr 172.21.15.114:0/131161926 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dfe00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aeec2d0 0x55a73b2a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 -- 172.21.15.114:0/131161926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aeec2d0 msgr2=0x55a73b2a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aeec2d0 0x55a73b2a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 -- 172.21.15.114:0/131161926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 msgr2=0x55a73b2a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 0x55a73b2a5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 -- 172.21.15.114:0/131161926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73b2a1430 con 0x55a73aecb5f0 2024-09-21T07:09:59.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dfe00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 0x55a73b2a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:09:59.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57dea00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 0x55a73b2a6540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f57cc006010 tx=0x7f57cc004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17b57244b414f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:09:59.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.864+0000 7f57c9600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:09:59.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.865+0000 7f57c9600640 1 -- 172.21.15.114:0/131161926 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc058060 con 0x55a73aecb5f0 2024-09-21T07:09:59.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.865+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d8005730 con 0x55a73aecb5f0 2024-09-21T07:09:59.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.865+0000 7f57c9600640 1 -- 172.21.15.114:0/131161926 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f57cc031520 con 0x55a73aecb5f0 2024-09-21T07:09:59.869 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.865+0000 7f57c9600640 1 -- 172.21.15.114:0/131161926 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f57cc035070 con 0x55a73aecb5f0 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.865+0000 7f57ed036740 10 client.29399 fetch_fsmap learned FSMap version 61 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.865+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a73b31b010 con 0x55a73aecb5f0 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57c9600640 1 -- 172.21.15.114:0/131161926 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d8005730 con 0x55a73aecb5f0 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57c9600640 1 -- 172.21.15.114:0/131161926 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f57cc0319a0 con 0x55a73aecb5f0 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57ed036740 10 client.29399 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57ed036740 10 client.29399 resolve_mds: validated gid 24761 aka daemon mds.b 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57ed036740 20 client.29399 populate_metadata read hostname 'smithi114' 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57ed036740 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73ae80000 0x55a73b31d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57ed036740 4 client.29399 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55a73b2079f0 con 0x55a73ae80000 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.866+0000 7f57dfe00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73ae80000 0x55a73b31d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.867+0000 7f57dfe00640 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73ae80000 0x55a73b31d750 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268cf3d2c583dd30 server_cookie=c5a0657ab44e601d in_seq=0 out_seq=0 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.867+0000 7f57c9600640 10 client.29399 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.867+0000 7f57c9600640 1 -- 172.21.15.114:0/131161926 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55a73b2079f0 con 0x55a73ae80000 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.867+0000 7f57c9600640 10 client.29399 handle_command_reply: tid=0 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.867+0000 7f57ed036740 1 client.29399 shutdown 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 msgr2=0x55a73b2a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 0x55a73b2a6540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f57cc006010 tx=0x7f57cc004420 comp rx=0 tx=0).stop 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 shutdown_connections 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a73ae80000 0x55a73b31d750 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a73aeec2d0 0x55a73b2a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a73aecb5f0 0x55a73b2a6540 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 --2- 172.21.15.114:0/131161926 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a73aec3410 0x55a73b2a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57c9600640 0 client.29399 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 >> 172.21.15.114:0/131161926 conn(0x55a73aee06c0 msgr2=0x55a73b29ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 shutdown_connections 2024-09-21T07:09:59.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 1 -- 172.21.15.114:0/131161926 wait complete. 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.868+0000 7f57ed036740 20 client.29399 trim_cache size 0 max 16384 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 61, 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stdout: "leader": 24761, 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stdout: "set_version": 60, 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stdout: "sets": {} 2024-09-21T07:09:59.872 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f57c40b6680 msgr2=0x7f57c40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f57c40b6680 0x7f57c40b8b90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55a73b290f10 tx=0x7f57d00aa020 comp rx=0 tx=0).stop 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 msgr2=0x7f57e019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 0x7f57e019c6c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e9f0 tx=0x7f57d000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 shutdown_connections 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f57e010c020 0x7f57e01a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f57c40b6680 0x7f57c40b8b90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f57e010b6d0 0x7f57e019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 --2- 172.21.15.114:0/393155691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e0105d90 0x7f57e019c6c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 >> 172.21.15.114:0/393155691 conn(0x7f57e0101860 msgr2=0x7f57e010ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 shutdown_connections 2024-09-21T07:09:59.873 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:09:59.869+0000 7f57e6400640 1 -- 172.21.15.114:0/393155691 wait complete. 2024-09-21T07:10:00.239 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:00.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:00 smithi114 ceph-mon[30654]: overall HEALTH_OK 2024-09-21T07:10:00.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:00 smithi183 ceph-mon[34219]: overall HEALTH_OK 2024-09-21T07:10:00.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:00 smithi193 ceph-mon[34601]: overall HEALTH_OK 2024-09-21T07:10:00.937 INFO:tasks.quiescer:done joining Quiescers 2024-09-21T07:10:00.937 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T07:10:00.946 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T07:10:00.957 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T07:10:00.992 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T07:10:00.992 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T07:10:00.992 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T07:10:00.992 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T07:10:01.071 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T07:10:01.132 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 -- 172.21.15.114:0/4116921239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109250 msgr2=0x7fee7c109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 --2- 172.21.15.114:0/4116921239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109250 0x7fee7c109660 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fee68009900 tx=0x7fee6802a410 comp rx=0 tx=0).stop 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 -- 172.21.15.114:0/4116921239 shutdown_connections 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 --2- 172.21.15.114:0/4116921239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109ba0 0x7fee7c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 --2- 172.21.15.114:0/4116921239 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109250 0x7fee7c109660 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 --2- 172.21.15.114:0/4116921239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 0x7fee7c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 -- 172.21.15.114:0/4116921239 >> 172.21.15.114:0/4116921239 conn(0x7fee7c0ff320 msgr2=0x7fee7c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 -- 172.21.15.114:0/4116921239 shutdown_connections 2024-09-21T07:10:01.357 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.355+0000 7fee81600640 1 -- 172.21.15.114:0/4116921239 wait complete. 2024-09-21T07:10:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.356+0000 7fee81600640 1 Processor -- start 2024-09-21T07:10:01.358 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.356+0000 7fee81600640 1 -- start start 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee81600640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 0x7fee7c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee81600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109250 0x7fee7c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee81600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 0x7fee7c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee81600640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c1138b0 con 0x7fee7c109ba0 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee81600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c113730 con 0x7fee7c109250 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee81600640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c113a30 con 0x7fee7c103920 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee80c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 0x7fee7c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee7be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 0x7fee7c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee80c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 0x7fee7c1a1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44620/0 (socket says 172.21.15.114:44620) 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee80c00640 1 -- 172.21.15.114:0/4250631279 learned_addr learned my addr 172.21.15.114:0/4250631279 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.359 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.357+0000 7fee7b400640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109250 0x7fee7c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee80c00640 1 -- 172.21.15.114:0/4250631279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 msgr2=0x7fee7c19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee80c00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 0x7fee7c19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.360 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee80c00640 1 -- 172.21.15.114:0/4250631279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109250 msgr2=0x7fee7c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee80c00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109250 0x7fee7c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee80c00640 1 -- 172.21.15.114:0/4250631279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c1a23e0 con 0x7fee7c109ba0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee7be00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 0x7fee7c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee7b400640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109250 0x7fee7c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee80c00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 0x7fee7c1a1ce0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fee7400eb10 tx=0x7fee7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74004b20 con 0x7fee7c109ba0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c1a26d0 con 0x7fee7c109ba0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee74004cc0 con 0x7fee7c109ba0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74010670 con 0x7fee7c109ba0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.358+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c1a8e60 con 0x7fee7c109ba0 2024-09-21T07:10:01.361 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.360+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c103d30 con 0x7fee7c109ba0 2024-09-21T07:10:01.363 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.362+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fee74005000 con 0x7fee7c109ba0 2024-09-21T07:10:01.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.364+0000 7fee78c00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee600b6680 0x7fee600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.365+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fee740e8300 con 0x7fee7c109ba0 2024-09-21T07:10:01.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.365+0000 7fee7be00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee600b6680 0x7fee600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.366 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.365+0000 7fee7be00640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee600b6680 0x7fee600b8b90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009fc0 tx=0x7fee6c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.368 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.367+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee74056710 con 0x7fee7c109ba0 2024-09-21T07:10:01.439 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:01 smithi114 ceph-mon[30654]: pgmap v888: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.9 MiB/s wr, 349 op/s 2024-09-21T07:10:01.585 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.583+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fee7c10f680 con 0x7fee600b6680 2024-09-21T07:10:01.594 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.592+0000 7fee78c00640 1 -- 172.21.15.114:0/4250631279 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fee7c10f680 con 0x7fee600b6680 2024-09-21T07:10:01.594 INFO:teuthology.orchestra.run.smithi114.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T06:54:00", "first": "2024-09-21T06:55:00", "last": "2024-09-21T07:10:00", "last_pruned": "2024-09-21T07:10:00", "created_count": 16, "pruned_count": 9, "active": true} 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee600b6680 msgr2=0x7fee600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee600b6680 0x7fee600b8b90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009fc0 tx=0x7fee6c005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 msgr2=0x7fee7c1a1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 0x7fee7c1a1ce0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fee7400eb10 tx=0x7fee7400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 shutdown_connections 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fee7c109ba0 0x7fee7c1a1ce0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fee600b6680 0x7fee600b8b90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee7c109250 0x7fee7c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 --2- 172.21.15.114:0/4250631279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee7c103920 0x7fee7c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 >> 172.21.15.114:0/4250631279 conn(0x7fee7c0ff320 msgr2=0x7fee7c1067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 shutdown_connections 2024-09-21T07:10:01.597 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.595+0000 7fee81600640 1 -- 172.21.15.114:0/4250631279 wait complete. 2024-09-21T07:10:01.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:01 smithi183 ceph-mon[34219]: pgmap v888: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.9 MiB/s wr, 349 op/s 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 -- 172.21.15.114:0/4037231644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e50105d90 msgr2=0x7f7e501061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 --2- 172.21.15.114:0/4037231644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e50105d90 0x7f7e501061a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009900 tx=0x7f7e4002a410 comp rx=0 tx=0).stop 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 -- 172.21.15.114:0/4037231644 shutdown_connections 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 --2- 172.21.15.114:0/4037231644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e5010c020 0x7f7e50112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 --2- 172.21.15.114:0/4037231644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010b6d0 0x7f7e5010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 --2- 172.21.15.114:0/4037231644 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e50105d90 0x7f7e501061a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 -- 172.21.15.114:0/4037231644 >> 172.21.15.114:0/4037231644 conn(0x7f7e50101860 msgr2=0x7f7e50103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 -- 172.21.15.114:0/4037231644 shutdown_connections 2024-09-21T07:10:01.622 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.620+0000 7f7e56400640 1 -- 172.21.15.114:0/4037231644 wait complete. 2024-09-21T07:10:01.624 DEBUG:teuthology.orchestra.run.smithi114:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T07:10:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.621+0000 7f7e56400640 1 Processor -- start 2024-09-21T07:10:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.621+0000 7f7e56400640 1 -- start start 2024-09-21T07:10:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e56400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e50105d90 0x7f7e5019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.625 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e56400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 0x7f7e5019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e56400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010c020 0x7f7e501a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e56400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50115d30 con 0x7f7e5010b6d0 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e56400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50115bb0 con 0x7f7e5010c020 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e56400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50115eb0 con 0x7f7e50105d90 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e4ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 0x7f7e5019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e50105d90 0x7f7e5019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e4ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 0x7f7e5019cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44640/0 (socket says 172.21.15.114:44640) 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e4ea00640 1 -- 172.21.15.114:0/1876444845 learned_addr learned my addr 172.21.15.114:0/1876444845 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010c020 0x7f7e501a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.622+0000 7f7e4ea00640 1 -- 172.21.15.114:0/1876444845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e50105d90 msgr2=0x7f7e5019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e50105d90 0x7f7e5019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e4ea00640 1 -- 172.21.15.114:0/1876444845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010c020 msgr2=0x7f7e501a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010c020 0x7f7e501a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.626 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e4ea00640 1 -- 172.21.15.114:0/1876444845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e501a2650 con 0x7f7e5010b6d0 2024-09-21T07:10:01.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 0x7f7e5019cc80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00e9f0 tx=0x7f7e3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e47e00640 1 -- 172.21.15.114:0/1876444845 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00ccc0 con 0x7f7e5010b6d0 2024-09-21T07:10:01.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e47e00640 1 -- 172.21.15.114:0/1876444845 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e3c0108e0 con 0x7f7e5010b6d0 2024-09-21T07:10:01.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e501a2940 con 0x7f7e5010b6d0 2024-09-21T07:10:01.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.623+0000 7f7e47e00640 1 -- 172.21.15.114:0/1876444845 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c018860 con 0x7f7e5010b6d0 2024-09-21T07:10:01.627 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.624+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501a90a0 con 0x7f7e5010b6d0 2024-09-21T07:10:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.627+0000 7f7e5d036740 1 Processor -- start 2024-09-21T07:10:01.629 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 -- start start 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e47e00640 1 -- 172.21.15.114:0/1876444845 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7e3c010430 con 0x7f7e5010b6d0 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b073440da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b0734412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b073079390 0x55b073441820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b0733b7fb0 con 0x55b073079390 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0733a73e0 con 0x55b07309a070 2024-09-21T07:10:01.630 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.628+0000 7f7e5d036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b0733ab1f0 con 0x55b0730711b0 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b073079390 0x55b073441820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b073440da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b0734412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b073440da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42810/0 (socket says 172.21.15.114:42810) 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4fe00640 1 -- 172.21.15.114:0/2196870741 learned_addr learned my addr 172.21.15.114:0/2196870741 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b0734412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34686/0 (socket says 172.21.15.114:34686) 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 -- 172.21.15.114:0/2196870741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 msgr2=0x55b073440da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b073440da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 -- 172.21.15.114:0/2196870741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b073079390 msgr2=0x55b073441820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.631 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b073079390 0x55b073441820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.629+0000 7f7e4f400640 1 -- 172.21.15.114:0/2196870741 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b073371cd0 con 0x55b0730711b0 2024-09-21T07:10:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b0734412e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7e4002aa00 tx=0x7f7e40004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d49c7ffd8c7b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e47e00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7e340b6680 0x7f7e340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e47e00640 1 -- 172.21.15.114:0/1876444845 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f7e3c0e7dd0 con 0x7f7e5010b6d0 2024-09-21T07:10:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e44c00640 1 -- 172.21.15.114:0/2196870741 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40035040 con 0x55b0730711b0 2024-09-21T07:10:01.632 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e44c00640 1 -- 172.21.15.114:0/2196870741 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e400054f0 con 0x55b0730711b0 2024-09-21T07:10:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7e340b6680 0x7f7e340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.630+0000 7f7e44c00640 1 -- 172.21.15.114:0/2196870741 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400035f0 con 0x55b0730711b0 2024-09-21T07:10:01.633 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 -- 172.21.15.114:0/2196870741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 msgr2=0x55b0734412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b0734412e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7e4002aa00 tx=0x7f7e40004040 comp rx=0 tx=0).stop 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7e340b6680 0x7f7e340b8b90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7e480116b0 tx=0x7f7e48009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 -- 172.21.15.114:0/2196870741 shutdown_connections 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b073079390 0x55b073441820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b0734412e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2196870741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b073440da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 -- 172.21.15.114:0/2196870741 >> 172.21.15.114:0/2196870741 conn(0x55b07308e460 msgr2=0x55b07343ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 -- 172.21.15.114:0/2196870741 shutdown_connections 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.631+0000 7f7e5d036740 1 -- 172.21.15.114:0/2196870741 wait complete. 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.632+0000 7f7e5d036740 1 -- start start 2024-09-21T07:10:01.634 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.632+0000 7f7e5d036740 1 Processor -- start 2024-09-21T07:10:01.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 0x55b07344c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07344ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.635 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b073458810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55b073208700 con 0x55b07309a070 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b073342580 con 0x55b0730711b0 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b07344f520 con 0x55b073079390 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e5d036740 10 client.0 _ll_register_callbacks cb 0x55b073352470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 0x55b07344c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 0x55b07344c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42818/0 (socket says 172.21.15.114:42818) 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e4ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07344ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e4f400640 1 -- 172.21.15.114:0/1661627306 learned_addr learned my addr 172.21.15.114:0/1661627306 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.633+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b073458810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4f400640 1 -- 172.21.15.114:0/1661627306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 msgr2=0x55b07344ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4f400640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07344ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4f400640 1 -- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 msgr2=0x55b073458810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.636 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4f400640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b073458810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4f400640 1 -- 172.21.15.114:0/1661627306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b073105650 con 0x55b0730711b0 2024-09-21T07:10:01.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b073458810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:10:01.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e4f400640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 0x55b07344c540 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7e40005ab0 tx=0x7f7e40009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd950f6733fb8c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e39600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e39600640 1 -- 172.21.15.114:0/1661627306 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40053060 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b073344b90 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.634+0000 7f7e39600640 1 -- 172.21.15.114:0/1661627306 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f7e400035f0 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e39600640 1 -- 172.21.15.114:0/1661627306 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7e40035050 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 10 client.29417 fetch_fsmap learned FSMap version 61 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55b07335f950 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e39600640 1 -- 172.21.15.114:0/1661627306 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b073344b90 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e39600640 1 -- 172.21.15.114:0/1661627306 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f7e40031880 con 0x55b0730711b0 2024-09-21T07:10:01.638 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 10 client.29417 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 10 client.29417 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 20 client.29417 populate_metadata read hostname 'smithi114' 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b0730589f0 0x55b07345e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 4 client.29417 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.635+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0733b9f00 con 0x55b0730589f0 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.636+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b0730589f0 0x55b07345e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.637+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b0730589f0 0x55b07345e4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0da2eb764fe897 server_cookie=360a7c7dfcbd3127 in_seq=0 out_seq=0 2024-09-21T07:10:01.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.637+0000 7f7e39600640 10 client.29417 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:10:01.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e39600640 1 -- 172.21.15.114:0/1661627306 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0733b9f00 con 0x55b0730589f0 2024-09-21T07:10:01.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e39600640 10 client.29417 handle_command_reply: tid=0 2024-09-21T07:10:01.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 client.29417 shutdown 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 msgr2=0x55b07344c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 0x55b07344c540 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7e40005ab0 tx=0x7f7e40009260 comp rx=0 tx=0).stop 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 shutdown_connections 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b0730589f0 0x55b07345e4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b073458810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07344ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 --2- 172.21.15.114:0/1661627306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b0730711b0 0x55b07344c540 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 >> 172.21.15.114:0/1661627306 conn(0x55b07308e460 msgr2=0x55b073443f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e39600640 0 client.29417 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 shutdown_connections 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.638+0000 7f7e5d036740 1 -- 172.21.15.114:0/1661627306 wait complete. 2024-09-21T07:10:01.642 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.639+0000 7f7e5d036740 20 client.29417 trim_cache size 0 max 16384 2024-09-21T07:10:01.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.652+0000 7f7e5d036740 1 Processor -- start 2024-09-21T07:10:01.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.652+0000 7f7e5d036740 1 -- start start 2024-09-21T07:10:01.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b073442790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.654 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b073079390 0x55b073458780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b07344b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e5d036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b3180 con 0x55b07309a070 2024-09-21T07:10:01.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e5d036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b4cf0 con 0x55b073079390 2024-09-21T07:10:01.655 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e5d036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b3880 con 0x55b0730711b0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b073442790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b073442790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34704/0 (socket says 172.21.15.114:34704) 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e4f400640 1 -- 172.21.15.114:0/252860124 learned_addr learned my addr 172.21.15.114:0/252860124 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b073079390 0x55b073458780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.653+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b07344b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4f400640 1 -- 172.21.15.114:0/252860124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b073079390 msgr2=0x55b073458780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4f400640 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b073079390 0x55b073458780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4f400640 1 -- 172.21.15.114:0/252860124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 msgr2=0x55b07344b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4f400640 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b07344b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4f400640 1 -- 172.21.15.114:0/252860124 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b073443090 con 0x55b0730711b0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b07344b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e4f400640 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b073442790 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e40004a80 tx=0x7f7e40004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2521a21e71af21b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e3be00640 1 -- 172.21.15.114:0/252860124 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40053070 con 0x55b0730711b0 2024-09-21T07:10:01.656 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e3be00640 1 -- 172.21.15.114:0/252860124 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e40005850 con 0x55b0730711b0 2024-09-21T07:10:01.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.654+0000 7f7e3be00640 1 -- 172.21.15.114:0/252860124 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40003e10 con 0x55b0730711b0 2024-09-21T07:10:01.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 -- 172.21.15.114:0/252860124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 msgr2=0x55b073442790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b073442790 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e40004a80 tx=0x7f7e40004700 comp rx=0 tx=0).stop 2024-09-21T07:10:01.657 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 -- 172.21.15.114:0/252860124 shutdown_connections 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b07309a070 0x55b07344b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b073079390 0x55b073458780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 --2- 172.21.15.114:0/252860124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b0730711b0 0x55b073442790 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 -- 172.21.15.114:0/252860124 >> 172.21.15.114:0/252860124 conn(0x55b07308e460 msgr2=0x55b07344adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 -- 172.21.15.114:0/252860124 shutdown_connections 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.655+0000 7f7e5d036740 1 -- 172.21.15.114:0/252860124 wait complete. 2024-09-21T07:10:01.658 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.656+0000 7f7e5d036740 1 -- start start 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.656+0000 7f7e5d036740 1 Processor -- start 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 0x55b07345f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07345ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b0734c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b4620 con 0x55b0730711b0 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b3f50 con 0x55b07309a070 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501b1180 con 0x55b073079390 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e5d036740 10 client.0 _ll_register_callbacks cb 0x7f7e40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 0x55b07345f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b0734c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.659 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e4ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07345ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e4f400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 0x55b07345f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44672/0 (socket says 172.21.15.114:44672) 2024-09-21T07:10:01.660 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.657+0000 7f7e4f400640 1 -- 172.21.15.114:0/2592570416 learned_addr learned my addr 172.21.15.114:0/2592570416 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4f400640 1 -- 172.21.15.114:0/2592570416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 msgr2=0x55b07345ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07345ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4f400640 1 -- 172.21.15.114:0/2592570416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 msgr2=0x55b0734c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b0734c7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4f400640 1 -- 172.21.15.114:0/2592570416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0734c8170 con 0x55b0730711b0 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07345ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4fe00640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b0734c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e4f400640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 0x55b07345f970 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7e40006010 tx=0x7f7e40003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad33d4189686976f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.658+0000 7f7e39600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:10:01.661 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e39600640 1 -- 172.21.15.114:0/2592570416 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40053030 con 0x55b0730711b0 2024-09-21T07:10:01.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e39600640 1 -- 172.21.15.114:0/2592570416 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f7e40003bd0 con 0x55b0730711b0 2024-09-21T07:10:01.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e40005850 con 0x55b0730711b0 2024-09-21T07:10:01.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e39600640 1 -- 172.21.15.114:0/2592570416 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7e40003dd0 con 0x55b0730711b0 2024-09-21T07:10:01.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e5d036740 10 client.18729 fetch_fsmap learned FSMap version 61 2024-09-21T07:10:01.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55b0734c9300 con 0x55b0730711b0 2024-09-21T07:10:01.662 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.659+0000 7f7e39600640 1 -- 172.21.15.114:0/2592570416 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e40005850 con 0x55b0730711b0 2024-09-21T07:10:01.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:01 smithi193 ceph-mon[34601]: pgmap v888: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 2.9 MiB/s wr, 349 op/s 2024-09-21T07:10:01.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e39600640 1 -- 172.21.15.114:0/2592570416 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f7e4005d030 con 0x55b0730711b0 2024-09-21T07:10:01.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e5d036740 10 client.18729 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:10:01.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e5d036740 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-09-21T07:10:01.663 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e5d036740 20 client.18729 populate_metadata read hostname 'smithi114' 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b07302dda0 0x55b0734cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e5d036740 4 client.18729 mds_command: new command op to 24760 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 --> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0733b59b0 con 0x55b07302dda0 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.660+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b07302dda0 0x55b0734cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.661+0000 7f7e4ea00640 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b07302dda0 0x55b0734cb820 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28771b7b257e8f5c server_cookie=222744fcf7a80e77 in_seq=0 out_seq=0 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.661+0000 7f7e39600640 10 client.18729 ms_handle_connect on v2:172.21.15.114:6840/2055898271 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.661+0000 7f7e39600640 1 -- 172.21.15.114:0/2592570416 <== mds.1 v2:172.21.15.114:6840/2055898271 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0733b59b0 con 0x55b07302dda0 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.661+0000 7f7e39600640 10 client.18729 handle_command_reply: tid=0 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.661+0000 7f7e5d036740 1 client.18729 shutdown 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 msgr2=0x55b07345f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.664 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 0x55b07345f970 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7e40006010 tx=0x7f7e40003820 comp rx=0 tx=0).stop 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 shutdown_connections 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:6840/2055898271,v1:172.21.15.114:6841/2055898271] conn(0x55b07302dda0 0x55b0734cb820 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b07309a070 0x55b0734c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b073079390 0x55b07345ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e39600640 0 client.18729 ms_handle_reset on v2:172.21.15.114:6840/2055898271 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 --2- 172.21.15.114:0/2592570416 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55b0730711b0 0x55b07345f970 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.662+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 >> 172.21.15.114:0/2592570416 conn(0x55b07308e460 msgr2=0x55b073442560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.665 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.663+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 shutdown_connections 2024-09-21T07:10:01.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.663+0000 7f7e5d036740 1 -- 172.21.15.114:0/2592570416 wait complete. 2024-09-21T07:10:01.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.663+0000 7f7e5d036740 20 client.18729 trim_cache size 0 max 16384 2024-09-21T07:10:01.666 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:10:01.666 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7e340b6680 msgr2=0x7f7e340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7e340b6680 0x7f7e340b8b90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7e480116b0 tx=0x7f7e48009500 comp rx=0 tx=0).stop 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 msgr2=0x7f7e5019cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 0x7f7e5019cc80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00e9f0 tx=0x7f7e3c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 shutdown_connections 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f7e340b6680 0x7f7e340b8b90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010c020 0x7f7e501a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f7e5010b6d0 0x7f7e5019cc80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 --2- 172.21.15.114:0/1876444845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e50105d90 0x7f7e5019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 >> 172.21.15.114:0/1876444845 conn(0x7f7e50101860 msgr2=0x7f7e5010ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 shutdown_connections 2024-09-21T07:10:01.667 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.664+0000 7f7e56400640 1 -- 172.21.15.114:0/1876444845 wait complete. 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 -- 172.21.15.114:0/1375949575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94109b90 msgr2=0x7fea941103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 --2- 172.21.15.114:0/1375949575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94109b90 0x7fea941103c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009be0 tx=0x7fea8c02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 -- 172.21.15.114:0/1375949575 shutdown_connections 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 --2- 172.21.15.114:0/1375949575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94109b90 0x7fea941103c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 --2- 172.21.15.114:0/1375949575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea94109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 --2- 172.21.15.114:0/1375949575 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea94103910 0x7fea94103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 -- 172.21.15.114:0/1375949575 >> 172.21.15.114:0/1375949575 conn(0x7fea940ff330 msgr2=0x7fea94101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 -- 172.21.15.114:0/1375949575 shutdown_connections 2024-09-21T07:10:01.868 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.866+0000 7fea99c00640 1 -- 172.21.15.114:0/1375949575 wait complete. 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.867+0000 7fea99c00640 1 Processor -- start 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.867+0000 7fea99c00640 1 -- start start 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94103910 0x7fea9419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea9419c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea94109b90 0x7fea941a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fea94113850 con 0x7fea94109b90 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea941136d0 con 0x7fea94109240 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea941139d0 con 0x7fea94103910 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94103910 0x7fea9419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea9419c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea99200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea94109b90 0x7fea941a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea9419c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42856/0 (socket says 172.21.15.114:42856) 2024-09-21T07:10:01.870 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 -- 172.21.15.114:0/3072583947 learned_addr learned my addr 172.21.15.114:0/3072583947 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94103910 0x7fea9419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34730/0 (socket says 172.21.15.114:34730) 2024-09-21T07:10:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 -- 172.21.15.114:0/3072583947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94103910 msgr2=0x7fea9419c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94103910 0x7fea9419c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 -- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea94109b90 msgr2=0x7fea941a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea94109b90 0x7fea941a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:01.871 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.868+0000 7fea93400640 1 -- 172.21.15.114:0/3072583947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea941a2360 con 0x7fea94109240 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.869+0000 7fea93400640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea9419c9d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fea8000e9f0 tx=0x7fea8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.869+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8000ccc0 con 0x7fea94109240 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.869+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea800108e0 con 0x7fea94109240 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.869+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea941a2650 con 0x7fea94109240 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.869+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80018860 con 0x7fea94109240 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.869+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea941a8de0 con 0x7fea94109240 2024-09-21T07:10:01.872 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.870+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea94103d20 con 0x7fea94109240 2024-09-21T07:10:01.875 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.873+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fea80010430 con 0x7fea94109240 2024-09-21T07:10:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.875+0000 7fea90c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea780b6680 0x7fea780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.875+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fea800e82c0 con 0x7fea94109240 2024-09-21T07:10:01.876 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.875+0000 7fea93e00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea780b6680 0x7fea780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:01.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.876+0000 7fea93e00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea780b6680 0x7fea780b8b90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fea84009530 tx=0x7fea84009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:01.877 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:01.876+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fea800b27c0 con 0x7fea94109240 2024-09-21T07:10:02.098 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.096+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 --> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fea9410f690 con 0x7fea780b6680 2024-09-21T07:10:02.106 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.105+0000 7fea90c00640 1 -- 172.21.15.114:0/3072583947 <== mgr.14223 v2:172.21.15.114:6800/1914683713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fea9410f690 con 0x7fea780b6680 2024-09-21T07:10:02.106 INFO:teuthology.orchestra.run.smithi114.stdout:/ 1m 6m3h 2024-09-21T07:10:02.109 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea780b6680 msgr2=0x7fea780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea780b6680 0x7fea780b8b90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fea84009530 tx=0x7fea84009260 comp rx=0 tx=0).stop 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 msgr2=0x7fea9419c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea9419c9d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fea8000e9f0 tx=0x7fea8000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 shutdown_connections 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fea94109b90 0x7fea941a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fea780b6680 0x7fea780b8b90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea94109240 0x7fea9419c9d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 --2- 172.21.15.114:0/3072583947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea94103910 0x7fea9419c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 >> 172.21.15.114:0/3072583947 conn(0x7fea940ff330 msgr2=0x7fea941088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 shutdown_connections 2024-09-21T07:10:02.110 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:02.108+0000 7fea99c00640 1 -- 172.21.15.114:0/3072583947 wait complete. 2024-09-21T07:10:02.134 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T07:10:02.143 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T07:10:02.599 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T07:10:02.969 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:03.463 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:03 smithi114 ceph-mon[30654]: from='client.18711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:03.463 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:03 smithi114 ceph-mon[30654]: from='client.29429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:03.463 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:03 smithi114 ceph-mon[30654]: pgmap v889: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-09-21T07:10:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:03 smithi183 ceph-mon[34219]: from='client.18711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:03.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:03 smithi183 ceph-mon[34219]: from='client.29429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:03.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:03 smithi183 ceph-mon[34219]: pgmap v889: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-09-21T07:10:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:03 smithi193 ceph-mon[34601]: from='client.18711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:03 smithi193 ceph-mon[34601]: from='client.29429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T07:10:03.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:03 smithi193 ceph-mon[34601]: pgmap v889: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-09-21T07:10:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 -- 172.21.15.114:0/3415431061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 msgr2=0x7fc87c10bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 --2- 172.21.15.114:0/3415431061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 0x7fc87c10bad0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc864009860 tx=0x7fc86402a340 comp rx=0 tx=0).stop 2024-09-21T07:10:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 -- 172.21.15.114:0/3415431061 shutdown_connections 2024-09-21T07:10:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 --2- 172.21.15.114:0/3415431061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc87c10c010 0x7fc87c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 --2- 172.21.15.114:0/3415431061 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 0x7fc87c10bad0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.814 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 --2- 172.21.15.114:0/3415431061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 -- 172.21.15.114:0/3415431061 >> 172.21.15.114:0/3415431061 conn(0x7fc87c101810 msgr2=0x7fc87c103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 -- 172.21.15.114:0/3415431061 shutdown_connections 2024-09-21T07:10:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.812+0000 7fc882400640 1 -- 172.21.15.114:0/3415431061 wait complete. 2024-09-21T07:10:03.815 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.813+0000 7fc882400640 1 Processor -- start 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.813+0000 7fc882400640 1 -- start start 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc882400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc882400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 0x7fc87c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc882400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc87c10c010 0x7fc87c1a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc882400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c115cd0 con 0x7fc87c10b6c0 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc882400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c115b50 con 0x7fc87c10c010 2024-09-21T07:10:03.816 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc882400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c115e50 con 0x7fc87c105d80 2024-09-21T07:10:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc872a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 0x7fc87c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc87b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc87b400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34744/0 (socket says 172.21.15.114:34744) 2024-09-21T07:10:03.817 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc87b400640 1 -- 172.21.15.114:0/1558592024 learned_addr learned my addr 172.21.15.114:0/1558592024 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.814+0000 7fc87be00640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc87c10c010 0x7fc87c1a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc87b400640 1 -- 172.21.15.114:0/1558592024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc87c10c010 msgr2=0x7fc87c1a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc87b400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc87c10c010 0x7fc87c1a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc87b400640 1 -- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 msgr2=0x7fc87c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc87b400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 0x7fc87c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc87b400640 1 -- 172.21.15.114:0/1558592024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c1a2650 con 0x7fc87c105d80 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc87b400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c19c6e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00e9c0 tx=0x7fc86c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc878c00640 1 -- 172.21.15.114:0/1558592024 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00cd80 con 0x7fc87c105d80 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c1a2940 con 0x7fc87c105d80 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc878c00640 1 -- 172.21.15.114:0/1558592024 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c0035f0 con 0x7fc87c105d80 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.815+0000 7fc878c00640 1 -- 172.21.15.114:0/1558592024 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c010690 con 0x7fc87c105d80 2024-09-21T07:10:03.818 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.816+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c1a90a0 con 0x7fc87c105d80 2024-09-21T07:10:03.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 Processor -- start 2024-09-21T07:10:03.820 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 -- start start 2024-09-21T07:10:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f40ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f41620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5eb7fb0 con 0x55a8d5b711b0 2024-09-21T07:10:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5ea73e0 con 0x55a8d5b79390 2024-09-21T07:10:03.821 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.819+0000 7fc8891f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5eab1f0 con 0x55a8d5b9a070 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f40ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc872a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f41620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f40ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:34760/0 (socket says 172.21.15.114:34760) 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc878c00640 1 -- 172.21.15.114:0/1558592024 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fc86c003fb0 con 0x7fc87c105d80 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 -- 172.21.15.114:0/451905920 learned_addr learned my addr 172.21.15.114:0/451905920 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc872a00640 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f41620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42888/0 (socket says 172.21.15.114:42888) 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87b400640 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 -- 172.21.15.114:0/451905920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 msgr2=0x55a8d5f41620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f41620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 -- 172.21.15.114:0/451905920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 msgr2=0x55a8d5f410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.822 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 -- 172.21.15.114:0/451905920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d5e71cd0 con 0x55a8d5b9a070 2024-09-21T07:10:03.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87b400640 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:10:03.823 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.820+0000 7fc87be00640 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f40ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc874011440 tx=0x7fc874011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a93fcd4092946d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc870c00640 1 -- 172.21.15.114:0/451905920 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874003c30 con 0x55a8d5b9a070 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc870c00640 1 -- 172.21.15.114:0/451905920 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc874003dd0 con 0x55a8d5b9a070 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc870c00640 1 -- 172.21.15.114:0/451905920 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87400fa20 con 0x55a8d5b9a070 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc878c00640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8600b6680 0x7fc8600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc878c00640 1 -- 172.21.15.114:0/1558592024 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fc86c0e7dd0 con 0x7fc87c105d80 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc8891f7740 1 -- 172.21.15.114:0/451905920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 msgr2=0x55a8d5f40ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.824 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc8891f7740 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f40ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc874011440 tx=0x7fc874011900 comp rx=0 tx=0).stop 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.821+0000 7fc87b400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8600b6680 0x7fc8600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 -- 172.21.15.114:0/451905920 shutdown_connections 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f41620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 --2- 172.21.15.114:0/451905920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f40ba0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 -- 172.21.15.114:0/451905920 >> 172.21.15.114:0/451905920 conn(0x55a8d5b8e460 msgr2=0x55a8d5f3ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 -- 172.21.15.114:0/451905920 shutdown_connections 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 -- 172.21.15.114:0/451905920 wait complete. 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc87b400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8600b6680 0x7fc8600b8b90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55a8d5f3f820 tx=0x7fc86c003e70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.822+0000 7fc8891f7740 1 -- start start 2024-09-21T07:10:03.825 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 Processor -- start 2024-09-21T07:10:03.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 0x55a8d5f4c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b79390 0x55a8d5f4cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f58670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5d08700 con 0x55a8d5b79390 2024-09-21T07:10:03.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5e42580 con 0x55a8d5b711b0 2024-09-21T07:10:03.827 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a8d5f4f380 con 0x55a8d5b9a070 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.823+0000 7fc8891f7740 10 client.0 _ll_register_callbacks cb 0x55a8d5e52470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc872a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b79390 0x55a8d5f4cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 0x55a8d5f4c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 0x55a8d5f4c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:42894/0 (socket says 172.21.15.114:42894) 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 -- 172.21.15.114:0/3757690473 learned_addr learned my addr 172.21.15.114:0/3757690473 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f58670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 -- 172.21.15.114:0/3757690473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 msgr2=0x55a8d5f58670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f58670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 -- 172.21.15.114:0/3757690473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b79390 msgr2=0x55a8d5f4cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b79390 0x55a8d5f4cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 -- 172.21.15.114:0/3757690473 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d5c05650 con 0x55a8d5b711b0 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc872a00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b79390 0x55a8d5f4cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc87be00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 0x55a8d5f4c7f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc874009aa0 tx=0x7fc87400f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7cbf29d3e1e27df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:03.828 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.824+0000 7fc869600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc869600640 1 -- 172.21.15.114:0/3757690473 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87400f9d0 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8d5e44b90 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc869600640 1 -- 172.21.15.114:0/3757690473 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fc87400fb70 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc869600640 1 -- 172.21.15.114:0/3757690473 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc874035050 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc8891f7740 10 client.29441 fetch_fsmap learned FSMap version 61 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a8d5e5f950 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.825+0000 7fc869600640 1 -- 172.21.15.114:0/3757690473 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a8d5e44b90 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc869600640 1 -- 172.21.15.114:0/3757690473 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fc874025040 con 0x55a8d5b711b0 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc8891f7740 10 client.29441 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc8891f7740 10 client.29441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc8891f7740 20 client.29441 populate_metadata read hostname 'smithi114' 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc8891f7740 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b589f0 0x55a8d5f5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc8891f7740 4 client.29441 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:10:03.829 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d5eb9f00 con 0x55a8d5b589f0 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.826+0000 7fc872a00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b589f0 0x55a8d5f5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.827+0000 7fc872a00640 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b589f0 0x55a8d5f5e580 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50b037597f77b12 server_cookie=7ca0156071385779 in_seq=0 out_seq=0 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.827+0000 7fc869600640 10 client.29441 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc869600640 1 -- 172.21.15.114:0/3757690473 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8d5eb9f00 con 0x55a8d5b589f0 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc869600640 10 client.29441 handle_command_reply: tid=0 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 client.29441 shutdown 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 msgr2=0x55a8d5f4c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 0x55a8d5f4c7f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc874009aa0 tx=0x7fc87400f680 comp rx=0 tx=0).stop 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 shutdown_connections 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b589f0 0x55a8d5f5e580 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b9a070 0x55a8d5f58670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b79390 0x55a8d5f4cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc869600640 0 client.29441 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 --2- 172.21.15.114:0/3757690473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b711b0 0x55a8d5f4c7f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.831 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 >> 172.21.15.114:0/3757690473 conn(0x55a8d5b8e460 msgr2=0x55a8d5f43d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:03.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 shutdown_connections 2024-09-21T07:10:03.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 1 -- 172.21.15.114:0/3757690473 wait complete. 2024-09-21T07:10:03.832 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.828+0000 7fc8891f7740 20 client.29441 trim_cache size 0 max 16384 2024-09-21T07:10:03.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.842+0000 7fc8891f7740 1 Processor -- start 2024-09-21T07:10:03.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.842+0000 7fc8891f7740 1 -- start start 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b711b0 0x55a8d5f46380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f58570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 0x55a8d5f497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc8891f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b31a0 con 0x55a8d5b9a070 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc8891f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b4d10 con 0x55a8d5b79390 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc8891f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b38a0 con 0x55a8d5b711b0 2024-09-21T07:10:03.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 0x55a8d5f497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 0x55a8d5f497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44736/0 (socket says 172.21.15.114:44736) 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 -- 172.21.15.114:0/2633441961 learned_addr learned my addr 172.21.15.114:0/2633441961 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc872a00640 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f58570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b711b0 0x55a8d5f46380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 -- 172.21.15.114:0/2633441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b711b0 msgr2=0x55a8d5f46380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b711b0 0x55a8d5f46380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 -- 172.21.15.114:0/2633441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 msgr2=0x55a8d5f58570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f58570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc87be00640 1 -- 172.21.15.114:0/2633441961 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d5f433b0 con 0x55a8d5b9a070 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.843+0000 7fc872a00640 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f58570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:03.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.844+0000 7fc87be00640 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 0x55a8d5f497e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc874004df0 tx=0x7fc874000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf26c3d617889a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:03.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.844+0000 7fc86be00640 1 -- 172.21.15.114:0/2633441961 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874004620 con 0x55a8d5b9a070 2024-09-21T07:10:03.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.844+0000 7fc86be00640 1 -- 172.21.15.114:0/2633441961 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8740047c0 con 0x55a8d5b9a070 2024-09-21T07:10:03.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.844+0000 7fc86be00640 1 -- 172.21.15.114:0/2633441961 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874004260 con 0x55a8d5b9a070 2024-09-21T07:10:03.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 -- 172.21.15.114:0/2633441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 msgr2=0x55a8d5f497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 0x55a8d5f497e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc874004df0 tx=0x7fc874000f40 comp rx=0 tx=0).stop 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 -- 172.21.15.114:0/2633441961 shutdown_connections 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b9a070 0x55a8d5f497e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b79390 0x55a8d5f58570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2633441961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b711b0 0x55a8d5f46380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 -- 172.21.15.114:0/2633441961 >> 172.21.15.114:0/2633441961 conn(0x55a8d5b8e460 msgr2=0x55a8d5f45c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 -- 172.21.15.114:0/2633441961 shutdown_connections 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 -- 172.21.15.114:0/2633441961 wait complete. 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.845+0000 7fc8891f7740 1 -- start start 2024-09-21T07:10:03.848 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 Processor -- start 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f50770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b79390 0x55a8d5f50cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b9a070 0x55a8d5f511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b4640 con 0x55a8d5b711b0 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b3f70 con 0x55a8d5b9a070 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c1b11a0 con 0x55a8d5b79390 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc8891f7740 10 client.0 _ll_register_callbacks cb 0x7fc874004e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc87be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f50770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc87be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f50770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:44752/0 (socket says 172.21.15.114:44752) 2024-09-21T07:10:03.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.846+0000 7fc87be00640 1 -- 172.21.15.114:0/2483584516 learned_addr learned my addr 172.21.15.114:0/2483584516 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc872a00640 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b79390 0x55a8d5f50cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc87be00640 1 -- 172.21.15.114:0/2483584516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b79390 msgr2=0x55a8d5f50cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc87be00640 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b79390 0x55a8d5f50cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc87be00640 1 -- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b9a070 msgr2=0x55a8d5f511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc87be00640 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b9a070 0x55a8d5f511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc87be00640 1 -- 172.21.15.114:0/2483584516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d5fc7f80 con 0x55a8d5b711b0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc87be00640 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f50770 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc874005d70 tx=0x7fc874009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ba5196e79f9c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc869600640 10 client.0 ms_handle_connect on v2:172.21.15.114:3300/0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc869600640 1 -- 172.21.15.114:0/2483584516 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87403d070 con 0x55a8d5b711b0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc869600640 1 -- 172.21.15.114:0/2483584516 <== mon.0 v2:172.21.15.114:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7fc874002030 con 0x55a8d5b711b0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.847+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8740047c0 con 0x55a8d5b711b0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc869600640 1 -- 172.21.15.114:0/2483584516 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc87400f5b0 con 0x55a8d5b711b0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 10 client.18750 fetch_fsmap learned FSMap version 61 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a8d5fc91c0 con 0x55a8d5b711b0 2024-09-21T07:10:03.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc869600640 1 -- 172.21.15.114:0/2483584516 <== mon.0 v2:172.21.15.114:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8740047c0 con 0x55a8d5b711b0 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc869600640 1 -- 172.21.15.114:0/2483584516 <== mon.0 v2:172.21.15.114:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fc874004090 con 0x55a8d5b711b0 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 10 client.18750 fetch_fsmap finished waiting for FSMap version 61 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 20 client.18750 populate_metadata read hostname 'smithi114' 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b2dda0 0x55a8d5fcb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 4 client.18750 mds_command: new command op to 24761 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T07:10:03.852 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.848+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 --> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8d5eb59b0 con 0x55a8d5b2dda0 2024-09-21T07:10:03.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.849+0000 7fc872a00640 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b2dda0 0x55a8d5fcb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:03.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.849+0000 7fc869600640 10 client.18750 ms_handle_connect on v2:172.21.15.183:6838/42004963 2024-09-21T07:10:03.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.849+0000 7fc872a00640 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b2dda0 0x55a8d5fcb7c0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a6a889e26ebd6e server_cookie=b5badfd828930409 in_seq=0 out_seq=0 2024-09-21T07:10:03.853 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.849+0000 7fc869600640 1 -- 172.21.15.114:0/2483584516 <== mds.0 v2:172.21.15.183:6838/42004963 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8d5eb59b0 con 0x55a8d5b2dda0 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.849+0000 7fc869600640 10 client.18750 handle_command_reply: tid=0 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.849+0000 7fc8891f7740 1 client.18750 shutdown 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 msgr2=0x55a8d5f50770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f50770 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc874005d70 tx=0x7fc874009fd0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 shutdown_connections 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a8d5b9a070 0x55a8d5f511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.183:6838/42004963,v1:172.21.15.183:6839/42004963] conn(0x55a8d5b2dda0 0x55a8d5fcb7c0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8d5b79390 0x55a8d5f50cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 --2- 172.21.15.114:0/2483584516 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x55a8d5b711b0 0x55a8d5f50770 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 >> 172.21.15.114:0/2483584516 conn(0x55a8d5b8e460 msgr2=0x55a8d5f41bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc869600640 0 client.18750 ms_handle_reset on v2:172.21.15.183:6838/42004963 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 shutdown_connections 2024-09-21T07:10:03.854 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 1 -- 172.21.15.114:0/2483584516 wait complete. 2024-09-21T07:10:03.855 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.850+0000 7fc8891f7740 20 client.18750 trim_cache size 0 max 16384 2024-09-21T07:10:03.855 INFO:teuthology.orchestra.run.smithi114.stdout:[] 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8600b6680 msgr2=0x7fc8600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8600b6680 0x7fc8600b8b90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x55a8d5f3f820 tx=0x7fc86c003e70 comp rx=0 tx=0).stop 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 msgr2=0x7fc87c19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c19c6e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00e9c0 tx=0x7fc86c00edc0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 shutdown_connections 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fc8600b6680 0x7fc8600b8b90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc87c10c010 0x7fc87c1a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fc87c10b6c0 0x7fc87c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 --2- 172.21.15.114:0/1558592024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc87c105d80 0x7fc87c19c6e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 >> 172.21.15.114:0/1558592024 conn(0x7fc87c101810 msgr2=0x7fc87c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 shutdown_connections 2024-09-21T07:10:03.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:03.851+0000 7fc882400640 1 -- 172.21.15.114:0/1558592024 wait complete. 2024-09-21T07:10:04.593 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T07:10:04.594 INFO:tasks.fwd_scrub:done joining 2024-09-21T07:10:04.594 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T07:10:04.609 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:10:04.841 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 -- 172.21.15.114:0/1015396102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 msgr2=0x7f560c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 --2- 172.21.15.114:0/1015396102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c104cf0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009900 tx=0x7f55fc02a410 comp rx=0 tx=0).stop 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 -- 172.21.15.114:0/1015396102 shutdown_connections 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 --2- 172.21.15.114:0/1015396102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c10dd40 0x7f560c110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 --2- 172.21.15.114:0/1015396102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c105230 0x7f560c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 --2- 172.21.15.114:0/1015396102 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c104cf0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 -- 172.21.15.114:0/1015396102 >> 172.21.15.114:0/1015396102 conn(0x7f560c0fe350 msgr2=0x7f560c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 -- 172.21.15.114:0/1015396102 shutdown_connections 2024-09-21T07:10:04.842 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.840+0000 7f5613000640 1 -- 172.21.15.114:0/1015396102 wait complete. 2024-09-21T07:10:04.843 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 Processor -- start 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 -- start start 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c105230 0x7f560c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c10dd40 0x7f560c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f560c113a10 con 0x7f560c1048e0 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f560c113890 con 0x7f560c10dd40 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.841+0000 7f5613000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f560c113b90 con 0x7f560c105230 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54134/0 (socket says 172.21.15.114:54134) 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 -- 172.21.15.114:0/587677995 learned_addr learned my addr 172.21.15.114:0/587677995 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560b400640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c105230 0x7f560c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f5611000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c10dd40 0x7f560c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 -- 172.21.15.114:0/587677995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c105230 msgr2=0x7f560c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c105230 0x7f560c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.844 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 -- 172.21.15.114:0/587677995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c10dd40 msgr2=0x7f560c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:04.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c10dd40 0x7f560c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:04.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 -- 172.21.15.114:0/587677995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1a03a0 con 0x7f560c1048e0 2024-09-21T07:10:04.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.842+0000 7f560be00640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c19a470 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005ab0 tx=0x7f55fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:04.845 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.843+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc03f070 con 0x7f560c1048e0 2024-09-21T07:10:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.843+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc031b60 con 0x7f560c1048e0 2024-09-21T07:10:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.843+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1a0630 con 0x7f560c1048e0 2024-09-21T07:10:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.843+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc035030 con 0x7f560c1048e0 2024-09-21T07:10:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.843+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1a6e20 con 0x7f560c1048e0 2024-09-21T07:10:04.846 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.844+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c104cf0 con 0x7f560c1048e0 2024-09-21T07:10:04.847 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.846+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f55fc042070 con 0x7f560c1048e0 2024-09-21T07:10:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.847+0000 7f5608c00640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6680 0x7f55f00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.848+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f55fc108a30 con 0x7f560c1048e0 2024-09-21T07:10:04.849 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.848+0000 7f560b400640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6680 0x7f55f00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:04.850 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.848+0000 7f560b400640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6680 0x7f55f00b8b90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f560c10f300 tx=0x7f55f8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:04.851 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:04.850+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55fc0d2f30 con 0x7f560c1048e0 2024-09-21T07:10:05.042 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.041+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f560c1a7080 con 0x7f560c1048e0 2024-09-21T07:10:05.044 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.042+0000 7f5608c00640 1 -- 172.21.15.114:0/587677995 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25465 (secure 0 0 0) 0x7f55fc0d7e70 con 0x7f560c1048e0 2024-09-21T07:10:05.045 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:10:05.045 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":105,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T07:09:15.000071+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":24,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":24,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":24,"length":1}]},{"pool":4,"snaps":[{"begin":12,"length":1},{"begin":24,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6680 msgr2=0x7f55f00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6680 0x7f55f00b8b90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f560c10f300 tx=0x7f55f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 msgr2=0x7f560c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c19a470 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005ab0 tx=0x7f55fc003c10 comp rx=0 tx=0).stop 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 shutdown_connections 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f55f00b6680 0x7f55f00b8b90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f560c10dd40 0x7f560c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.047 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f560c105230 0x7f560c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 --2- 172.21.15.114:0/587677995 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f560c1048e0 0x7f560c19a470 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 >> 172.21.15.114:0/587677995 conn(0x7f560c0fe350 msgr2=0x7f560c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:05.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 shutdown_connections 2024-09-21T07:10:05.048 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.045+0000 7f5613000640 1 -- 172.21.15.114:0/587677995 wait complete. 2024-09-21T07:10:05.070 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T06:45:37.711233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T06:50:42.695843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T06:50:45.269171+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T06:50:54.559397+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T07:10:05.070 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 -- 172.21.15.114:0/1937073413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff244105cd0 msgr2=0x7ff2441060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 --2- 172.21.15.114:0/1937073413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff244105cd0 0x7ff2441060e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff238009900 tx=0x7ff23802a400 comp rx=0 tx=0).stop 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 -- 172.21.15.114:0/1937073413 shutdown_connections 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 --2- 172.21.15.114:0/1937073413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff24410bf50 0x7ff244112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 --2- 172.21.15.114:0/1937073413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24410b600 0x7ff24410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 --2- 172.21.15.114:0/1937073413 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff244105cd0 0x7ff2441060e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 -- 172.21.15.114:0/1937073413 >> 172.21.15.114:0/1937073413 conn(0x7ff2441016f0 msgr2=0x7ff244103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 -- 172.21.15.114:0/1937073413 shutdown_connections 2024-09-21T07:10:05.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.293+0000 7ff24be00640 1 -- 172.21.15.114:0/1937073413 wait complete. 2024-09-21T07:10:05.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 Processor -- start 2024-09-21T07:10:05.297 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 -- start start 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff244105cd0 0x7ff24419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 0x7ff24419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24410bf50 0x7ff2441a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7ff244115c60 con 0x7ff24410b600 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff244115ae0 con 0x7ff244105cd0 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.294+0000 7ff24be00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff244115de0 con 0x7ff24410bf50 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 0x7ff24419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 0x7ff24419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54152/0 (socket says 172.21.15.114:54152) 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 -- 172.21.15.114:0/72239467 learned_addr learned my addr 172.21.15.114:0/72239467 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff249400640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff244105cd0 0x7ff24419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff249e00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24410bf50 0x7ff2441a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 -- 172.21.15.114:0/72239467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24410bf50 msgr2=0x7ff2441a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.298 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24410bf50 0x7ff2441a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 -- 172.21.15.114:0/72239467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff244105cd0 msgr2=0x7ff24419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff244105cd0 0x7ff24419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 -- 172.21.15.114:0/72239467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2441a2530 con 0x7ff24410b600 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.295+0000 7ff248a00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 0x7ff24419cba0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff23000e970 tx=0x7ff23000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.296+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23000cd80 con 0x7ff24410b600 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.296+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2300035f0 con 0x7ff24410b600 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.296+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2441a2820 con 0x7ff24410b600 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.296+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230010690 con 0x7ff24410b600 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.296+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2441a8fb0 con 0x7ff24410b600 2024-09-21T07:10:05.299 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.297+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff244106f40 con 0x7ff24410b600 2024-09-21T07:10:05.300 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.299+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7ff230003aa0 con 0x7ff24410b600 2024-09-21T07:10:05.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.301+0000 7ff242000640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff22c0b6700 0x7ff22c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.301+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7ff2300e7dd0 con 0x7ff24410b600 2024-09-21T07:10:05.302 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.301+0000 7ff249400640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff22c0b6700 0x7ff22c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.303 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.302+0000 7ff249400640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff22c0b6700 0x7ff22c0b8c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff238005ab0 tx=0x7ff238005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:05.305 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.303+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2300adf50 con 0x7ff24410b600 2024-09-21T07:10:05.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.488+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff24419d720 con 0x7ff24410b600 2024-09-21T07:10:05.490 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.489+0000 7ff242000640 1 -- 172.21.15.114:0/72239467 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7ff2300b2330 con 0x7ff24410b600 2024-09-21T07:10:05.490 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 1 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff22c0b6700 msgr2=0x7ff22c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff22c0b6700 0x7ff22c0b8c10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff238005ab0 tx=0x7ff238005a20 comp rx=0 tx=0).stop 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 msgr2=0x7ff24419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 0x7ff24419cba0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff23000e970 tx=0x7ff23000ee30 comp rx=0 tx=0).stop 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 shutdown_connections 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24410bf50 0x7ff2441a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7ff24410b600 0x7ff24419cba0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7ff22c0b6700 0x7ff22c0b8c10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 --2- 172.21.15.114:0/72239467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff244105cd0 0x7ff24419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.493 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 >> 172.21.15.114:0/72239467 conn(0x7ff2441016f0 msgr2=0x7ff244103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:05.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 shutdown_connections 2024-09-21T07:10:05.494 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.491+0000 7ff24be00640 1 -- 172.21.15.114:0/72239467 wait complete. 2024-09-21T07:10:05.516 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:05 smithi114 ceph-mon[30654]: pgmap v890: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-09-21T07:10:05.516 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:05 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/587677995' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:05.518 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T07:10:05.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:05 smithi183 ceph-mon[34219]: pgmap v890: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-09-21T07:10:05.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:05 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/587677995' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:05.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:05 smithi193 ceph-mon[34601]: pgmap v890: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-09-21T07:10:05.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:05 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/587677995' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 -- 172.21.15.114:0/2260597176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c103ab0 msgr2=0x7f8c1c103ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 --2- 172.21.15.114:0/2260597176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c103ec0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009900 tx=0x7f8c1002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 -- 172.21.15.114:0/2260597176 shutdown_connections 2024-09-21T07:10:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 --2- 172.21.15.114:0/2260597176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c109d30 0x7f8c1c110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 --2- 172.21.15.114:0/2260597176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.740 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 --2- 172.21.15.114:0/2260597176 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c103ec0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 -- 172.21.15.114:0/2260597176 >> 172.21.15.114:0/2260597176 conn(0x7f8c1c0ff4d0 msgr2=0x7f8c1c101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 -- 172.21.15.114:0/2260597176 shutdown_connections 2024-09-21T07:10:05.741 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.738+0000 7f8c24400640 1 -- 172.21.15.114:0/2260597176 wait complete. 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.739+0000 7f8c24400640 1 Processor -- start 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.739+0000 7f8c24400640 1 -- start start 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c24400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c24400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c24400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c109d30 0x7f8c1c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c24400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c1139f0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c24400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c113870 con 0x7f8c1c103ab0 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c24400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1c113b70 con 0x7f8c1c109d30 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c21000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c21a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c21000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54168/0 (socket says 172.21.15.114:54168) 2024-09-21T07:10:05.743 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c21000640 1 -- 172.21.15.114:0/2128221369 learned_addr learned my addr 172.21.15.114:0/2128221369 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c22400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c109d30 0x7f8c1c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c21000640 1 -- 172.21.15.114:0/2128221369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c109d30 msgr2=0x7f8c1c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.740+0000 7f8c21000640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c109d30 0x7f8c1c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c21000640 1 -- 172.21.15.114:0/2128221369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c103ab0 msgr2=0x7f8c1c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c21000640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c21000640 1 -- 172.21.15.114:0/2128221369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1c1a86e0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c22400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c109d30 0x7f8c1c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c21a00640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c21000640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c075c50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800e9c0 tx=0x7f8c0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800ccc0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c080054d0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1c1a89d0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.744 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08010430 con 0x7f8c1c1093e0 2024-09-21T07:10:05.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.741+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1c1a8ed0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.745 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.742+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1c103ec0 con 0x7f8c1c1093e0 2024-09-21T07:10:05.747 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.745+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f8c08005020 con 0x7f8c1c1093e0 2024-09-21T07:10:05.748 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.747+0000 7f8c1a000640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c040b66b0 0x7f8c040b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.747+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f8c080e7f10 con 0x7f8c1c1093e0 2024-09-21T07:10:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.747+0000 7f8c21a00640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c040b66b0 0x7f8c040b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:05.749 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.748+0000 7f8c21a00640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c040b66b0 0x7f8c040b8bc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c10000c00 tx=0x7f8c10031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:05.752 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.751+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8c080ae030 con 0x7f8c1c1093e0 2024-09-21T07:10:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.939+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c1c02e640 con 0x7f8c1c1093e0 2024-09-21T07:10:05.941 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.940+0000 7f8c1a000640 1 -- 172.21.15.114:0/2128221369 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f8c080b2410 con 0x7f8c1c1093e0 2024-09-21T07:10:05.941 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c040b66b0 msgr2=0x7f8c040b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c040b66b0 0x7f8c040b8bc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8c10000c00 tx=0x7f8c10031400 comp rx=0 tx=0).stop 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 msgr2=0x7f8c1c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c075c50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800e9c0 tx=0x7f8c0800ee80 comp rx=0 tx=0).stop 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 shutdown_connections 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c1c109d30 0x7f8c1c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f8c1c1093e0 0x7f8c1c075c50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f8c040b66b0 0x7f8c040b8bc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 --2- 172.21.15.114:0/2128221369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c1c103ab0 0x7f8c1c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 >> 172.21.15.114:0/2128221369 conn(0x7f8c1c0ff4d0 msgr2=0x7f8c1c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 shutdown_connections 2024-09-21T07:10:05.944 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:05.942+0000 7f8c24400640 1 -- 172.21.15.114:0/2128221369 wait complete. 2024-09-21T07:10:05.967 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T07:10:06.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 -- 172.21.15.114:0/1291374447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 msgr2=0x7f381c1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 --2- 172.21.15.114:0/1291374447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 0x7f381c1125b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f3814009ca0 tx=0x7f381402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 -- 172.21.15.114:0/1291374447 shutdown_connections 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 --2- 172.21.15.114:0/1291374447 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 0x7f381c1125b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 --2- 172.21.15.114:0/1291374447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 --2- 172.21.15.114:0/1291374447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f381c105b00 0x7f381c105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 -- 172.21.15.114:0/1291374447 >> 172.21.15.114:0/1291374447 conn(0x7f381c101520 msgr2=0x7f381c103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 -- 172.21.15.114:0/1291374447 shutdown_connections 2024-09-21T07:10:06.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.186+0000 7f381b200640 1 -- 172.21.15.114:0/1291374447 wait complete. 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.187+0000 7f381b200640 1 Processor -- start 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 -- start start 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f381c105b00 0x7f381c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 0x7f381c1a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f381c115a40 con 0x7f381c10bd80 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f381c1158c0 con 0x7f381c10b430 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381b200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f381c115bc0 con 0x7f381c105b00 2024-09-21T07:10:06.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f3819400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f3819e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f381c105b00 0x7f381c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f3819400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c19c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:38670/0 (socket says 172.21.15.114:38670) 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f381a800640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 0x7f381c1a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.188+0000 7f3819400640 1 -- 172.21.15.114:0/2886781832 learned_addr learned my addr 172.21.15.114:0/2886781832 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3819400640 1 -- 172.21.15.114:0/2886781832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f381c105b00 msgr2=0x7f381c19c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3819400640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f381c105b00 0x7f381c19c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3819400640 1 -- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 msgr2=0x7f381c1a1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3819400640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 0x7f381c1a1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3819400640 1 -- 172.21.15.114:0/2886781832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c1a2330 con 0x7f381c10b430 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3819400640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c19c9a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f380800e9f0 tx=0x7f380800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:06.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380800ccc0 con 0x7f381c10b430 2024-09-21T07:10:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c1a2620 con 0x7f381c10b430 2024-09-21T07:10:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c1a8db0 con 0x7f381c10b430 2024-09-21T07:10:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080108e0 con 0x7f381c10b430 2024-09-21T07:10:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.189+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080187a0 con 0x7f381c10b430 2024-09-21T07:10:06.193 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.190+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c105f10 con 0x7f381c10b430 2024-09-21T07:10:06.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.193+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f3808010430 con 0x7f381c10b430 2024-09-21T07:10:06.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.195+0000 7f3812a00640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f38000b6700 0x7f38000b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.195+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f38080e7f70 con 0x7f381c10b430 2024-09-21T07:10:06.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.195+0000 7f3819e00640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f38000b6700 0x7f38000b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.196+0000 7f3819e00640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f38000b6700 0x7f38000b8c10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f380c009bd0 tx=0x7f380c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:06.198 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.196+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38080b2470 con 0x7f381c10b430 2024-09-21T07:10:06.390 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.388+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f381c02e640 con 0x7f381c10b430 2024-09-21T07:10:06.391 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.389+0000 7f3812a00640 1 -- 172.21.15.114:0/2886781832 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f38080b73b0 con 0x7f381c10b430 2024-09-21T07:10:06.391 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 64 2024-09-21T07:10:06.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.391+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f38000b6700 msgr2=0x7f38000b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.393 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.391+0000 7f381b200640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f38000b6700 0x7f38000b8c10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f380c009bd0 tx=0x7f380c005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.391+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 msgr2=0x7f381c19c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.391+0000 7f381b200640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c19c9a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f380800e9f0 tx=0x7f380800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 shutdown_connections 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f381c10bd80 0x7f381c1a1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f38000b6700 0x7f38000b8c10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f381c10b430 0x7f381c19c9a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 --2- 172.21.15.114:0/2886781832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f381c105b00 0x7f381c19c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 >> 172.21.15.114:0/2886781832 conn(0x7f381c101520 msgr2=0x7f381c103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 shutdown_connections 2024-09-21T07:10:06.394 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.392+0000 7f381b200640 1 -- 172.21.15.114:0/2886781832 wait complete. 2024-09-21T07:10:06.415 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T07:10:06.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:06 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/72239467' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:10:06.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:06 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2128221369' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:10:06.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:06 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/72239467' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:10:06.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:06 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2128221369' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 -- 172.21.15.114:0/2253630716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 msgr2=0x7f36781060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 --2- 172.21.15.114:0/2253630716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f36781060e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3668009900 tx=0x7f366802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 -- 172.21.15.114:0/2253630716 shutdown_connections 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 --2- 172.21.15.114:0/2253630716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f367810bf50 0x7f3678112780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 --2- 172.21.15.114:0/2253630716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f367810b600 0x7f367810ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 --2- 172.21.15.114:0/2253630716 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f36781060e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 -- 172.21.15.114:0/2253630716 >> 172.21.15.114:0/2253630716 conn(0x7f3678101710 msgr2=0x7f3678103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 -- 172.21.15.114:0/2253630716 shutdown_connections 2024-09-21T07:10:06.637 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.635+0000 7f367ea00640 1 -- 172.21.15.114:0/2253630716 wait complete. 2024-09-21T07:10:06.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 Processor -- start 2024-09-21T07:10:06.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 -- start start 2024-09-21T07:10:06.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f367819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.639 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f367810b600 0x7f367819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f367810bf50 0x7f36781a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f3678115c60 con 0x7f3678105cd0 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3678115ae0 con 0x7f367810b600 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.636+0000 7f367ea00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3678115de0 con 0x7f367810bf50 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f367819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f367819c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54202/0 (socket says 172.21.15.114:54202) 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 -- 172.21.15.114:0/3636396161 learned_addr learned my addr 172.21.15.114:0/3636396161 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f366f400640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f367810b600 0x7f367819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f367ca00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f367810bf50 0x7f36781a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 -- 172.21.15.114:0/3636396161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f367810bf50 msgr2=0x7f36781a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f367810bf50 0x7f36781a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 -- 172.21.15.114:0/3636396161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f367810b600 msgr2=0x7f367819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f367810b600 0x7f367819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.640 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 -- 172.21.15.114:0/3636396161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36781a2530 con 0x7f3678105cd0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.637+0000 7f3677e00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f367819c600 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3668005ab0 tx=0x7f3668003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.638+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366803f070 con 0x7f3678105cd0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.638+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3668031b60 con 0x7f3678105cd0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.638+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36781a27c0 con 0x7f3678105cd0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.638+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668035030 con 0x7f3678105cd0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.638+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36781a8fb0 con 0x7f3678105cd0 2024-09-21T07:10:06.641 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.639+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36781060e0 con 0x7f3678105cd0 2024-09-21T07:10:06.643 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.641+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f3668042070 con 0x7f3678105cd0 2024-09-21T07:10:06.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.643+0000 7f3675600640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f365c0b6700 0x7f365c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:06.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.643+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f36681089a0 con 0x7f3678105cd0 2024-09-21T07:10:06.644 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.643+0000 7f366f400640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f365c0b6700 0x7f365c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:06.645 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.644+0000 7f366f400640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f365c0b6700 0x7f365c0b8c10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f367810b050 tx=0x7f3660005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:06.646 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.645+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f36680d2ea0 con 0x7f3678105cd0 2024-09-21T07:10:06.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:06 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/72239467' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T07:10:06.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:06 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2128221369' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T07:10:06.835 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.833+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f367810dd80 con 0x7f3678105cd0 2024-09-21T07:10:06.836 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.834+0000 7f3675600640 1 -- 172.21.15.114:0/3636396161 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7f36680d7de0 con 0x7f3678105cd0 2024-09-21T07:10:06.836 INFO:teuthology.orchestra.run.smithi114.stdout:pg_num: 32 2024-09-21T07:10:06.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f365c0b6700 msgr2=0x7f365c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.838 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f365c0b6700 0x7f365c0b8c10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f367810b050 tx=0x7f3660005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 msgr2=0x7f367819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f367819c600 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3668005ab0 tx=0x7f3668003c10 comp rx=0 tx=0).stop 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 shutdown_connections 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f367810bf50 0x7f36781a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f365c0b6700 0x7f365c0b8c10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f367810b600 0x7f367819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 --2- 172.21.15.114:0/3636396161 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f3678105cd0 0x7f367819c600 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 >> 172.21.15.114:0/3636396161 conn(0x7f3678101710 msgr2=0x7f3678103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 shutdown_connections 2024-09-21T07:10:06.839 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:06.837+0000 7f367ea00640 1 -- 172.21.15.114:0/3636396161 wait complete. 2024-09-21T07:10:06.863 DEBUG:teuthology.orchestra.run.smithi114:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 -- 172.21.15.114:0/1000820978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 msgr2=0x7f6c68104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 --2- 172.21.15.114:0/1000820978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 0x7f6c68104cf0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009900 tx=0x7f6c5802a400 comp rx=0 tx=0).stop 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 -- 172.21.15.114:0/1000820978 shutdown_connections 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 --2- 172.21.15.114:0/1000820978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c6810dd40 0x7f6c68110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 --2- 172.21.15.114:0/1000820978 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c68105230 0x7f6c6810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 --2- 172.21.15.114:0/1000820978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 0x7f6c68104cf0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 -- 172.21.15.114:0/1000820978 >> 172.21.15.114:0/1000820978 conn(0x7f6c680fe350 msgr2=0x7f6c681007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 -- 172.21.15.114:0/1000820978 shutdown_connections 2024-09-21T07:10:07.086 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.084+0000 7f6c6d400640 1 -- 172.21.15.114:0/1000820978 wait complete. 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.085+0000 7f6c6d400640 1 Processor -- start 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.085+0000 7f6c6d400640 1 -- start start 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.085+0000 7f6c6d400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 0x7f6c6819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c6d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c68105230 0x7f6c6819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c6d400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 0x7f6c681a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c6d400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681139c0 con 0x7f6c6810dd40 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c6d400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68113840 con 0x7f6c68105230 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c6d400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68113b40 con 0x7f6c681048e0 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 0x7f6c681a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 0x7f6c681a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:54214/0 (socket says 172.21.15.114:54214) 2024-09-21T07:10:07.088 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c66800640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 0x7f6c6819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 -- 172.21.15.114:0/2999179893 learned_addr learned my addr 172.21.15.114:0/2999179893 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c5de00640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c68105230 0x7f6c6819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 -- 172.21.15.114:0/2999179893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 msgr2=0x7f6c6819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 0x7f6c6819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 -- 172.21.15.114:0/2999179893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c68105230 msgr2=0x7f6c6819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c68105230 0x7f6c6819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.086+0000 7f6c67200640 1 -- 172.21.15.114:0/2999179893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c681a2550 con 0x7f6c6810dd40 2024-09-21T07:10:07.089 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.087+0000 7f6c67200640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 0x7f6c681a1e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000eb10 tx=0x7f6c6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.087+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60004b20 con 0x7f6c6810dd40 2024-09-21T07:10:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.087+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c60004cc0 con 0x7f6c6810dd40 2024-09-21T07:10:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.087+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c681a2840 con 0x7f6c6810dd40 2024-09-21T07:10:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.087+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60010670 con 0x7f6c6810dd40 2024-09-21T07:10:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.087+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c681a8fd0 con 0x7f6c6810dd40 2024-09-21T07:10:07.090 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.088+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c68104cf0 con 0x7f6c6810dd40 2024-09-21T07:10:07.093 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.092+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f6c60005000 con 0x7f6c6810dd40 2024-09-21T07:10:07.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.093+0000 7f6c5fe00640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c4c0b6680 0x7f6c4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:07.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.093+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9490+0+0 (secure 0 0 0) 0x7f6c600e82b0 con 0x7f6c6810dd40 2024-09-21T07:10:07.095 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.094+0000 7f6c66800640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c4c0b6680 0x7f6c4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:07.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.094+0000 7f6c66800640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c4c0b6680 0x7f6c4c0b8b90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6c58031440 tx=0x7f6c58009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:07.096 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.095+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6c60056710 con 0x7f6c6810dd40 2024-09-21T07:10:07.336 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.334+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f6c6819d740 con 0x7f6c6810dd40 2024-09-21T07:10:07.344 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.343+0000 7f6c5fe00640 1 -- 172.21.15.114:0/2999179893 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+834062 (secure 0 0 0) 0x7f6c600ae3d0 con 0x7f6c6810dd40 2024-09-21T07:10:07.355 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:10:07.355 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:10:07.355 INFO:teuthology.orchestra.run.smithi114.stdout: "epoch": 25, 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "active_gid": 14223, 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "active_name": "x", 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "active_addrs": { 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:6800", 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 1914683713 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v1", 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:6801", 2024-09-21T07:10:07.356 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 1914683713 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "active_addr": "172.21.15.114:6801/1914683713", 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "active_change": "2024-09-21T06:43:29.388120+0000", 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "available": true, 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "standbys": [ 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "gid": 24157, 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "z", 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr_features": 4540719139924082687, 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: "available_modules": [ 2024-09-21T07:10:07.357 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "alerts", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.358 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.359 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.360 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.361 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.362 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.363 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_destination": { 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_destination", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.364 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_from_name": { 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_from_name", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "Ceph", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email From: name", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.365 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_host": { 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_host", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP server", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.366 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_password": { 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_password", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Password to authenticate with", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_port": { 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_port", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "465", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP port", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.368 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_sender": { 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_sender", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_ssl": { 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_ssl", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_user": { 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_user", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "User to authenticate as", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.372 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "balancer", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "active": { 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "active", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.373 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_time": { 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_time", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0000", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.374 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_weekday": { 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_weekday", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-21T07:10:07.375 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_max_iterations": { 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "25", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "250", 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.376 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_metrics": { 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_metrics", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.377 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_step": { 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_step", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.5", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0.001", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "0.999", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:10:07.378 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "end_time": { 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_time", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2359", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:10:07.379 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "end_weekday": { 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_weekday", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:10:07.380 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.381 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.382 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.383 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.384 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.385 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "min_score": { 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_score", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:10:07.386 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "upmap", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "crush-compat", 2024-09-21T07:10:07.387 INFO:teuthology.orchestra.run.smithi114.stdout: "none", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "read", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap-read" 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Balancer mode", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_ids": { 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_ids", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.388 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.389 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.390 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_deviation": { 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_deviation", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.391 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_optimizations": { 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.392 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm", 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.393 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_down_multiplier": { 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_down_multiplier", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_refresh_rate": { 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_refresh_rate", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "20", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_starting_port": { 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_starting_port", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4721", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_ptrace": { 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_ptrace", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.397 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_interval": { 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_interval", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.7", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm_log_destination": { 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm_log_destination", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "file", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "file,syslog", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "syslog" 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:10:07.400 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "cgroups_split": { 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cgroups_split", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.401 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "config_checks_enabled": { 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_checks_enabled", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.402 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "config_dashboard": { 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_dashboard", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.403 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_alertmanager": { 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_alertmanager", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.404 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_base": { 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_base", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_elasticsearch": { 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "elasticsearch container image", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.406 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_grafana": { 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_grafana", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.407 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_haproxy": { 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_haproxy", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "HAproxy container image", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_agent": { 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_collector": { 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_query": { 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger query container image", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_keepalived": { 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_keepalived", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Keepalived container image", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.412 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_loki": { 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_loki", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Loki container image", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.413 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nginx": { 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nginx", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nginx container image", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.414 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_node_exporter": { 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_node_exporter", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nvmeof": { 2024-09-21T07:10:07.415 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nvmeof", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nvme-of container image", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:10:07.416 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_prometheus": { 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_prometheus", 2024-09-21T07:10:07.417 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.418 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_promtail": { 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_promtail", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Promtail container image", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.419 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_samba": { 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_samba", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.420 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_samba_metrics": { 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.421 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_snmp_gateway": { 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "container_init": { 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_init", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.423 INFO:teuthology.orchestra.run.smithi114.stdout: "daemon_cache_timeout": { 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:10:07.424 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "900", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.425 INFO:teuthology.orchestra.run.smithi114.stdout: "default_registry": { 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_registry", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "device_cache_timeout": { 2024-09-21T07:10:07.426 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_cache_timeout", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1800", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "device_enhanced_scan": { 2024-09-21T07:10:07.427 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_enhanced_scan", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "facts_cache_timeout": { 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "facts_cache_timeout", 2024-09-21T07:10:07.428 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "grafana_dashboards_path": { 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.429 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "host_check_interval": { 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "host_check_interval", 2024-09-21T07:10:07.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "hw_monitoring": { 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hw_monitoring", 2024-09-21T07:10:07.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "inventory_list_all": { 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "inventory_list_all", 2024-09-21T07:10:07.432 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.433 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.434 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "log_refresh_metadata": { 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_refresh_metadata", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:10:07.435 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:10:07.436 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.437 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.438 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.439 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "*", 2024-09-21T07:10:07.440 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "max_count_per_host": { 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_count_per_host", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.441 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "max_osd_draining_count": { 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_osd_draining_count", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.442 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "migration_current": { 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "migration_current", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.443 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "internal - do not modify", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.444 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "root", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm-package", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "root" 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "oob_default_addr": { 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "oob_default_addr", 2024-09-21T07:10:07.445 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "169.254.1.1", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "prometheus_alerts_path": { 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:10:07.446 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_insecure": { 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_insecure", 2024-09-21T07:10:07.447 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_password": { 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_password", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.448 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_url": { 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_url", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.449 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_username": { 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_username", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.450 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "secure_monitoring_stack": { 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "service_discovery_port": { 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "service_discovery_port", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.452 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8765", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.453 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_config_file": { 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_config_file", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.454 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.455 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_keepalive_interval": { 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "use_agent": { 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_agent", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.457 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "use_repo_digest": { 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_repo_digest", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.458 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_failed_host_check": { 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.459 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_daemons": { 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.460 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_hosts": { 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.461 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crash", 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.462 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.463 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.464 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.465 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.466 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "retain_interval": { 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retain_interval", 2024-09-21T07:10:07.467 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "31536000", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.468 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_recent_interval": { 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_recent_interval", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.469 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1209600", 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.470 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dashboard", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.471 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.472 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.473 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.474 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.476 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.477 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:10:07.478 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:10:07.479 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.480 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.481 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.482 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:10:07.483 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:10:07.484 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:10:07.485 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_URL": { 2024-09-21T07:10:07.486 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:10:07.487 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:10:07.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:10:07.489 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.490 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.491 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.492 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "[]", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.493 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "{}", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.494 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.498 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.499 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.500 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.501 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.502 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.503 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.504 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.505 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.506 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "45", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.507 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.508 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.509 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.510 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.511 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.512 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.513 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.514 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.515 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "cross_origin_url": { 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cross_origin_url", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.516 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "crt_file": { 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crt_file", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "debug": { 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "debug", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.518 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "jwt_token_ttl": { 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "jwt_token_ttl", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "28800", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.519 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.520 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.521 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.522 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.523 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.524 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "motd": { 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "motd", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "The message of the day", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.526 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.527 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8080", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.530 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.531 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl_server_port": { 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl_server_port", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8443", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.532 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "sso_oauth2": { 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sso_oauth2", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "redirect", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect" 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.534 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "url_prefix": { 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url_prefix", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.536 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_monitoring": { 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_monitoring", 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.537 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "monitor device health metrics", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.538 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.539 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.541 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.542 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "mark_out_threshold": { 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mark_out_threshold", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2419200", 2024-09-21T07:10:07.543 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_name": { 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_name", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "device_health_metrics", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.544 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "retention_period": { 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retention_period", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15552000", 2024-09-21T07:10:07.545 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_frequency": { 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_frequency", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-21T07:10:07.546 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "self_heal": { 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "self_heal", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.547 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.548 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.549 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_threshold": { 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_threshold", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7257600", 2024-09-21T07:10:07.550 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "diskprediction_local", 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.551 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.552 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.553 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.554 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.555 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "predict_interval": { 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predict_interval", 2024-09-21T07:10:07.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "predictor_model": { 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predictor_model", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.557 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "prophetstor", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:07.558 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.559 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.560 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "influx", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": false, 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "batch_size": { 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "batch_size", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5000", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:10:07.561 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "database": { 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "database", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "ceph", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:10:07.562 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": { 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hostname", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:10:07.563 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "30", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "5", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.564 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.565 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.566 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.567 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.568 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.569 INFO:teuthology.orchestra.run.smithi114.stdout: "password": { 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "password", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "port": { 2024-09-21T07:10:07.570 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "port", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8086", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server port", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.571 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.572 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "false", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "threads": { 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threads", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "32", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "username": { 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "username", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.575 INFO:teuthology.orchestra.run.smithi114.stdout: "verify_ssl": { 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "verify_ssl", 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "true", 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.578 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "insights", 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.579 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.580 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.581 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.582 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.583 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.584 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "iostat", 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.585 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.586 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.587 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.588 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.589 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.590 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.591 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "k8sevents", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_event_retention_days": { 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ceph_event_retention_days", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.592 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "config_check_secs": { 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_check_secs", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "10", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.593 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.594 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.595 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.596 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.597 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.598 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "localpool", 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "failure_domain": { 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "failure_domain", 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.600 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "host", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.601 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.602 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.603 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.604 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.605 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "min_size": { 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_size", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.606 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "num_rep": { 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "num_rep", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-21T07:10:07.607 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_num": { 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_num", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "128", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.608 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "prefix": { 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prefix", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.609 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.610 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "subtree": { 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "subtree", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "rack", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.611 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mds_autoscaler", 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.612 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.613 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.614 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.615 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.616 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.617 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.618 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirroring", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.619 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.620 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.621 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.622 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.623 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.624 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "nfs", 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.625 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.626 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.628 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.631 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "fail_fs": { 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "fail_fs", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:10:07.632 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.633 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.634 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.636 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.637 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator": { 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "rook", 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: "test_orchestrator" 2024-09-21T07:10:07.638 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Orchestrator backend", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.639 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_perf_query", 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.640 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.641 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.642 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.643 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.644 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.645 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_support", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.647 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.648 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.649 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.650 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.651 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.652 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_autoscaler", 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.653 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.654 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.655 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.656 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.657 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.658 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "threshold": { 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threshold", 2024-09-21T07:10:07.660 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1.0", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "scaling threshold", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.661 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "progress", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_pg_recovery_event": { 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:10:07.662 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.663 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.664 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.665 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.666 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.667 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.668 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "max_completed_events": { 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_completed_events", 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "50", 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.669 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.670 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.671 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus", 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "cache": { 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cache", 2024-09-21T07:10:07.672 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "exclude_perf_counters": { 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "exclude_perf_counters", 2024-09-21T07:10:07.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.674 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.675 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.676 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.677 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.678 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools": { 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.679 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "300", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_interval": { 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_interval", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15.0", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.682 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "9283", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.684 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "stale_cache_strategy": { 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stale_cache_strategy", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "log", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "default", 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "default", 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "error" 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-21T07:10:07.687 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.688 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.689 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.691 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.692 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.693 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_snap_create": { 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.694 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.695 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.696 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "trash_purge_schedule": { 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "trash_purge_schedule", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.697 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "restful", 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_auth": { 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_auth", 2024-09-21T07:10:07.698 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.699 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.702 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.703 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.704 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "max_requests": { 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_requests", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:07.705 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.706 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.707 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.708 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rgw", 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.709 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.710 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.711 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.712 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.713 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:10:07.714 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.716 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rook", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.717 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.718 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.719 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.720 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.721 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.722 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "storage_class": { 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "storage_class", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "local", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:10:07.723 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "selftest", 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.724 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.725 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.726 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.727 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.728 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "roption1": { 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption1", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.729 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "roption2": { 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption2", 2024-09-21T07:10:07.730 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "xyz", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption1": { 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption1", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.731 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption2": { 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption2", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption3": { 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption3", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption4": { 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption4", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.734 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption5": { 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption5", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.735 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption6": { 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption6", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption7": { 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption7", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.737 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "42", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.738 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "testkey": { 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testkey", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.739 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "testlkey": { 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testlkey", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.740 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "testnewline": { 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testnewline", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.741 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smb", 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.742 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "internal_store_backend": { 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "internal_store_backend", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.743 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.744 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.745 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.746 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.748 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "update_orchestration": { 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "update_orchestration", 2024-09-21T07:10:07.749 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_m_granularity": { 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_m_granularity", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.751 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "dump_on_update": { 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dump_on_update", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.752 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.753 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.754 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.755 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.756 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.757 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.758 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stats", 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.759 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.760 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.761 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.762 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.763 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.764 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.765 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "status", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.766 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.767 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.769 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telegraf", 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "address": { 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "address", 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.772 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15", 2024-09-21T07:10:07.773 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.774 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.775 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.776 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.777 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.778 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.779 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.780 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.781 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telemetry", 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_basic": { 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_basic", 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.782 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_crash": { 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_crash", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.783 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_device": { 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_device", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.784 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_ident": { 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_ident", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.785 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_perf": { 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_perf", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.786 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "contact": { 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "contact", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.787 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "description": { 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "description", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.788 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "device_url": { 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_url", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.789 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.790 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "24", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "8", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.791 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "last_opt_revision": { 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "last_opt_revision", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.792 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard": { 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.793 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard_description": { 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard_description", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.794 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.795 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.796 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.797 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.798 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.799 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "organization": { 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "organization", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.800 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "proxy": { 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "proxy", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.801 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.802 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "url": { 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.803 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "test_orchestrator", 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.804 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.805 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.806 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.807 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.808 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.809 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.810 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.811 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.812 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.813 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.814 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.815 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_clones": { 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_clones", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.816 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "periodic_async_work": { 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "periodic_async_work", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.817 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Periodically check for async work", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_delay": { 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.818 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.819 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.820 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix", 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.821 INFO:teuthology.orchestra.run.smithi114.stdout: "discovery_interval": { 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "discovery_interval", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "100", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.822 INFO:teuthology.orchestra.run.smithi114.stdout: "identifier": { 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "identifier", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.823 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.824 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.825 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.826 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.827 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.828 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.829 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_host": { 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_host", 2024-09-21T07:10:07.830 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_port": { 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_port", 2024-09-21T07:10:07.831 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10051", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_sender": { 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_sender", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.832 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.833 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "gid": 24158, 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "y", 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "mgr_features": 4540719139924082687, 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "available_modules": [ 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "alerts", 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:07.834 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.835 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.836 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.837 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.838 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.839 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_destination": { 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_destination", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.840 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_from_name": { 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_from_name", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.841 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "Ceph", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email From: name", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_host": { 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_host", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.842 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP server", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_password": { 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_password", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.843 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Password to authenticate with", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_port": { 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_port", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.844 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "465", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP port", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_sender": { 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_sender", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.845 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_ssl": { 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_ssl", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.846 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_user": { 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_user", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.847 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "User to authenticate as", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.848 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "balancer", 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.849 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "active": { 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "active", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.850 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_time": { 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_time", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0000", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.851 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_weekday": { 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_weekday", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.852 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_max_iterations": { 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "25", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "250", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.853 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_metrics": { 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_metrics", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.854 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_step": { 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_step", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.5", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0.001", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "0.999", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.855 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "end_time": { 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_time", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2359", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.856 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "end_weekday": { 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_weekday", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.857 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.858 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.859 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.860 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.861 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.862 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "min_score": { 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_score", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.863 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "upmap", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "crush-compat", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "none", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "read", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap", 2024-09-21T07:10:07.864 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap-read" 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Balancer mode", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_ids": { 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_ids", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.865 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.866 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.867 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_deviation": { 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_deviation", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.868 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_optimizations": { 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.869 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm", 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_down_multiplier": { 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_down_multiplier", 2024-09-21T07:10:07.870 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_refresh_rate": { 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_refresh_rate", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.871 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "20", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_starting_port": { 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_starting_port", 2024-09-21T07:10:07.872 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4721", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_ptrace": { 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_ptrace", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.873 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_interval": { 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_interval", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.874 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:07.875 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.7", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm_log_destination": { 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm_log_destination", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.876 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "file", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "file,syslog", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "syslog" 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.877 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "cgroups_split": { 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cgroups_split", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.878 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "config_checks_enabled": { 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_checks_enabled", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.879 INFO:teuthology.orchestra.run.smithi114.stdout: "config_dashboard": { 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_dashboard", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_alertmanager": { 2024-09-21T07:10:07.880 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_alertmanager", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_base": { 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_base", 2024-09-21T07:10:07.881 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_elasticsearch": { 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:10:07.882 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "elasticsearch container image", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_grafana": { 2024-09-21T07:10:07.883 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_grafana", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_haproxy": { 2024-09-21T07:10:07.884 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_haproxy", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "HAproxy container image", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_agent": { 2024-09-21T07:10:07.885 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_collector": { 2024-09-21T07:10:07.886 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_query": { 2024-09-21T07:10:07.887 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger query container image", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_keepalived": { 2024-09-21T07:10:07.888 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_keepalived", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Keepalived container image", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_loki": { 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_loki", 2024-09-21T07:10:07.889 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Loki container image", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nginx": { 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nginx", 2024-09-21T07:10:07.890 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nginx container image", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_node_exporter": { 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_node_exporter", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.891 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nvmeof": { 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nvmeof", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.892 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nvme-of container image", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.893 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_prometheus": { 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_prometheus", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.894 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_promtail": { 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_promtail", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.895 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Promtail container image", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_samba": { 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_samba", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.896 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_samba_metrics": { 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.897 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_snmp_gateway": { 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.898 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "container_init": { 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_init", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.899 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "daemon_cache_timeout": { 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.900 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.901 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "900", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "default_registry": { 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_registry", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.902 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "device_cache_timeout": { 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_cache_timeout", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1800", 2024-09-21T07:10:07.903 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "device_enhanced_scan": { 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_enhanced_scan", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.904 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "facts_cache_timeout": { 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "facts_cache_timeout", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.905 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "grafana_dashboards_path": { 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.906 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "host_check_interval": { 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "host_check_interval", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:07.907 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "hw_monitoring": { 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hw_monitoring", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.908 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "inventory_list_all": { 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "inventory_list_all", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.909 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.910 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.911 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "log_refresh_metadata": { 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_refresh_metadata", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.912 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.913 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.914 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.915 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.916 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "*", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.917 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "max_count_per_host": { 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_count_per_host", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.918 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "max_osd_draining_count": { 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_osd_draining_count", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.919 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "migration_current": { 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "migration_current", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.920 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "internal - do not modify", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "root", 2024-09-21T07:10:07.921 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm-package", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "root" 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "oob_default_addr": { 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "oob_default_addr", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.922 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "169.254.1.1", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "prometheus_alerts_path": { 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.923 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_insecure": { 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_insecure", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.924 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_password": { 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_password", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.925 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_url": { 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_url", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.926 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_username": { 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_username", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.927 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "secure_monitoring_stack": { 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.928 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "service_discovery_port": { 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "service_discovery_port", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8765", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.929 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.930 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_config_file": { 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_config_file", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.931 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.932 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_keepalive_interval": { 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.933 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "use_agent": { 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_agent", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.934 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "use_repo_digest": { 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_repo_digest", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.935 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_failed_host_check": { 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.936 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_daemons": { 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.937 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_hosts": { 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.938 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crash", 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.939 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.940 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.941 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.942 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.943 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "retain_interval": { 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retain_interval", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.944 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "31536000", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.945 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_recent_interval": { 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_recent_interval", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.946 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1209600", 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dashboard", 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:07.947 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.949 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.950 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.951 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.953 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.954 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:10:07.955 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:10:07.956 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:10:07.957 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:10:07.958 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:10:07.959 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.960 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.961 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.962 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_URL": { 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.963 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.964 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.965 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.966 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.967 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.968 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "[]", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.969 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "{}", 2024-09-21T07:10:07.970 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.971 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.972 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.973 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.974 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.975 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.976 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.977 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.978 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.979 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.980 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.981 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.982 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8", 2024-09-21T07:10:07.983 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "45", 2024-09-21T07:10:07.984 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.985 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:07.986 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.987 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.988 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.989 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.990 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:07.991 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:07.992 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "cross_origin_url": { 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cross_origin_url", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.993 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "crt_file": { 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crt_file", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.994 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "debug": { 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "debug", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.995 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "jwt_token_ttl": { 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "jwt_token_ttl", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "28800", 2024-09-21T07:10:07.996 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.997 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:07.998 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:07.999 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.000 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.001 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.002 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "motd": { 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "motd", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.003 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "The message of the day", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.004 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.005 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8080", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.006 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.007 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.008 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl_server_port": { 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl_server_port", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8443", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.009 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "sso_oauth2": { 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sso_oauth2", 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.010 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "redirect", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.011 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect" 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:08.012 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "url_prefix": { 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url_prefix", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.013 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.014 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_monitoring": { 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_monitoring", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "monitor device health metrics", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.015 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.016 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.017 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.018 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.019 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "mark_out_threshold": { 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mark_out_threshold", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.020 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2419200", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_name": { 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_name", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.021 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "device_health_metrics", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "retention_period": { 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retention_period", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.022 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15552000", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_frequency": { 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_frequency", 2024-09-21T07:10:08.023 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "self_heal": { 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "self_heal", 2024-09-21T07:10:08.024 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.025 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.026 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_threshold": { 2024-09-21T07:10:08.027 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_threshold", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7257600", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.028 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "diskprediction_local", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.029 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.030 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.031 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.032 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.033 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "predict_interval": { 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predict_interval", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.034 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "predictor_model": { 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predictor_model", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "prophetstor", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.035 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.036 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.037 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "influx", 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": false, 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "batch_size": { 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "batch_size", 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.038 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5000", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "database": { 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "database", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.039 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "ceph", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": { 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hostname", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.040 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.041 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "30", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "5", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.042 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.043 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.044 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.045 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.046 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "password": { 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "password", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.047 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "port": { 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "port", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8086", 2024-09-21T07:10:08.048 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server port", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.049 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.050 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "false", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "threads": { 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threads", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.051 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "32", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "username": { 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "username", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.052 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "verify_ssl": { 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "verify_ssl", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.053 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "true", 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "insights", 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.054 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.055 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.056 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.057 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.058 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.059 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.060 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "iostat", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.061 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.062 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.063 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.064 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.065 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.066 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "k8sevents", 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.067 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_event_retention_days": { 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ceph_event_retention_days", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.068 INFO:teuthology.orchestra.run.smithi114.stdout: "config_check_secs": { 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_check_secs", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "10", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.069 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.070 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.071 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.072 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.073 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.074 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.075 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "localpool", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "failure_domain": { 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "failure_domain", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "host", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.076 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.077 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.078 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.079 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.080 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.081 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "min_size": { 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_size", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.082 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "num_rep": { 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "num_rep", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.083 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_num": { 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_num", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "128", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.084 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "prefix": { 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prefix", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:10:08.085 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.086 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "subtree": { 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "subtree", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "rack", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:10:08.087 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mds_autoscaler", 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.088 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.089 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.090 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.091 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.092 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.093 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirroring", 2024-09-21T07:10:08.094 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.095 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.096 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.097 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.098 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.099 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.100 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "nfs", 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.101 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.102 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.103 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.104 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.105 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.106 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.107 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "fail_fs": { 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "fail_fs", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.108 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.109 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.110 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.111 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.112 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.113 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator": { 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "rook", 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "test_orchestrator" 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.114 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Orchestrator backend", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.115 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_perf_query", 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.116 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.117 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.118 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.119 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.120 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.121 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_support", 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.122 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.123 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.124 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.125 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.126 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.127 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.128 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_autoscaler", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.129 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.130 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.131 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.132 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.133 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.134 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.135 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "threshold": { 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threshold", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1.0", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "scaling threshold", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:10:08.136 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "progress", 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_pg_recovery_event": { 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.137 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.138 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.139 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.140 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.141 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.142 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.143 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "max_completed_events": { 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_completed_events", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "50", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.144 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.145 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.146 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus", 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.147 INFO:teuthology.orchestra.run.smithi114.stdout: "cache": { 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cache", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.148 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "exclude_perf_counters": { 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "exclude_perf_counters", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.149 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.150 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.151 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.152 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.153 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.154 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools": { 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.155 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "300", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_interval": { 2024-09-21T07:10:08.156 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_interval", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15.0", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:08.157 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.158 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "9283", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.159 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "stale_cache_strategy": { 2024-09-21T07:10:08.160 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stale_cache_strategy", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "log", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-21T07:10:08.161 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "default", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "default", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "error" 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.162 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.163 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.164 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.165 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.166 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.167 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.168 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_snap_create": { 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.169 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.170 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.171 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "trash_purge_schedule": { 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "trash_purge_schedule", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.172 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "restful", 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.173 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_auth": { 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_auth", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.174 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.175 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.176 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.177 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.178 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.179 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.180 INFO:teuthology.orchestra.run.smithi114.stdout: "max_requests": { 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_requests", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.181 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.182 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.183 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.184 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rgw", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.185 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.186 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.187 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.188 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.189 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.189 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.190 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.191 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.192 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rook", 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.193 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.194 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.195 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.196 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.197 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.198 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "storage_class": { 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "storage_class", 2024-09-21T07:10:08.199 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "local", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.200 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "selftest", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.201 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.202 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.203 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.204 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.205 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "roption1": { 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption1", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.206 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "roption2": { 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption2", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "xyz", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.207 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption1": { 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption1", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.208 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption2": { 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption2", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.209 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption3": { 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption3", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.210 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption4": { 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption4", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.211 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption5": { 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption5", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.212 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption6": { 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption6", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.213 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption7": { 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption7", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "42", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.214 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.215 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "testkey": { 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testkey", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.216 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "testlkey": { 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testlkey", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.217 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "testnewline": { 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testnewline", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.218 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smb", 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "internal_store_backend": { 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "internal_store_backend", 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.219 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.220 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.221 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.222 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.223 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.224 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.225 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "update_orchestration": { 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "update_orchestration", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.226 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.227 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_m_granularity": { 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_m_granularity", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.228 INFO:teuthology.orchestra.run.smithi114.stdout: "dump_on_update": { 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dump_on_update", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.229 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.230 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.231 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.232 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.233 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.234 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.235 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stats", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.236 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.237 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.238 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.239 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.240 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.241 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "status", 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.242 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.243 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.244 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.245 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.246 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.247 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telegraf", 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.248 INFO:teuthology.orchestra.run.smithi114.stdout: "address": { 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "address", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.249 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.250 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.251 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.252 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.253 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.254 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.255 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.256 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telemetry", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_basic": { 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_basic", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:10:08.257 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_crash": { 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_crash", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:10:08.258 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_device": { 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_device", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.259 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_ident": { 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_ident", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.260 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_perf": { 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_perf", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.261 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "contact": { 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "contact", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.262 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "description": { 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "description", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.263 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "device_url": { 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_url", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.264 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.265 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "24", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "8", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.266 INFO:teuthology.orchestra.run.smithi114.stdout: "last_opt_revision": { 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "last_opt_revision", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.267 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard": { 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.268 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard_description": { 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard_description", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.269 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.270 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.271 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.272 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.273 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.274 INFO:teuthology.orchestra.run.smithi114.stdout: "organization": { 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "organization", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "proxy": { 2024-09-21T07:10:08.275 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "proxy", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.276 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "url": { 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url", 2024-09-21T07:10:08.277 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.278 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "test_orchestrator", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.279 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.280 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.281 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.282 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.283 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.284 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.285 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.286 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.287 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.288 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.289 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_clones": { 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_clones", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.290 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "periodic_async_work": { 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "periodic_async_work", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.291 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Periodically check for async work", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_delay": { 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.292 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.293 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.294 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix", 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.295 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "discovery_interval": { 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "discovery_interval", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "100", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.296 INFO:teuthology.orchestra.run.smithi114.stdout: "identifier": { 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "identifier", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.297 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.298 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.299 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.300 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.301 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.302 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.303 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_host": { 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_host", 2024-09-21T07:10:08.304 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_port": { 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_port", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.305 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10051", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_sender": { 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_sender", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.306 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.307 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "modules": [ 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "dashboard", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "iostat", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "nfs", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "restful", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "snap_schedule" 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "available_modules": [ 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "alerts", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.308 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.309 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.310 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.311 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.312 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.313 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.314 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_destination": { 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_destination", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.315 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_from_name": { 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_from_name", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "Ceph", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Email From: name", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_host": { 2024-09-21T07:10:08.316 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_host", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP server", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_password": { 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_password", 2024-09-21T07:10:08.317 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Password to authenticate with", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_port": { 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_port", 2024-09-21T07:10:08.318 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "465", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP port", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_sender": { 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_sender", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.319 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_ssl": { 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_ssl", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.320 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "smtp_user": { 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smtp_user", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.321 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "User to authenticate as", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.322 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "balancer", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.323 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "active": { 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "active", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.324 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_time": { 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_time", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0000", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.325 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "begin_weekday": { 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "begin_weekday", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.326 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_max_iterations": { 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "25", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "250", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.327 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_metrics": { 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_metrics", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.328 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "crush_compat_step": { 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crush_compat_step", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.5", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0.001", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "0.999", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.329 INFO:teuthology.orchestra.run.smithi114.stdout: "end_time": { 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_time", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2359", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.330 INFO:teuthology.orchestra.run.smithi114.stdout: "end_weekday": { 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "end_weekday", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "0", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "6", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.331 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.332 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.333 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.334 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.335 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "min_score": { 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_score", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.336 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.337 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "upmap", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "crush-compat", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "none", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "read", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap-read" 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Balancer mode", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_ids": { 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_ids", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.339 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_deviation": { 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_deviation", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "upmap_max_optimizations": { 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.343 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm", 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_down_multiplier": { 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_down_multiplier", 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-21T07:10:08.344 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_refresh_rate": { 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_refresh_rate", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "20", 2024-09-21T07:10:08.345 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "agent_starting_port": { 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "agent_starting_port", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4721", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.346 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_ptrace": { 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_ptrace", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.347 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_interval": { 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_interval", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.348 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0.7", 2024-09-21T07:10:08.349 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm_log_destination": { 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cephadm_log_destination", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.350 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "file", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "file,syslog", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "syslog" 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "cgroups_split": { 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cgroups_split", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.351 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "config_checks_enabled": { 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_checks_enabled", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.352 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "config_dashboard": { 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_dashboard", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.353 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_alertmanager": { 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_alertmanager", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.354 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_base": { 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_base", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.355 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_elasticsearch": { 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.356 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "elasticsearch container image", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_grafana": { 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_grafana", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.357 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_haproxy": { 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_haproxy", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.358 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "HAproxy container image", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_agent": { 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.359 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_collector": { 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:10:08.360 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_jaeger_query": { 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.361 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Jaeger query container image", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_keepalived": { 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_keepalived", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.362 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Keepalived container image", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_loki": { 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_loki", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.363 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Loki container image", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nginx": { 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nginx", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.364 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nginx container image", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_node_exporter": { 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_node_exporter", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.365 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_nvmeof": { 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_nvmeof", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.366 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Nvme-of container image", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.367 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_prometheus": { 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_prometheus", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.368 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Prometheus container image", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_promtail": { 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_promtail", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.369 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Promtail container image", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_samba": { 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_samba", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.370 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_samba_metrics": { 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.371 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "container_image_snmp_gateway": { 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.372 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "container_init": { 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "container_init", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.373 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "daemon_cache_timeout": { 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.374 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "900", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.375 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "default_registry": { 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "default_registry", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "docker.io", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:10:08.376 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "device_cache_timeout": { 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_cache_timeout", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1800", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:10:08.377 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "device_enhanced_scan": { 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_enhanced_scan", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.378 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "facts_cache_timeout": { 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "facts_cache_timeout", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.379 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "grafana_dashboards_path": { 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.380 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "host_check_interval": { 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "host_check_interval", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.381 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "hw_monitoring": { 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hw_monitoring", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:10:08.382 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "inventory_list_all": { 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "inventory_list_all", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.383 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.384 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "log_refresh_metadata": { 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_refresh_metadata", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.385 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.386 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.387 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.388 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.389 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.390 INFO:teuthology.orchestra.run.smithi114.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "*", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "max_count_per_host": { 2024-09-21T07:10:08.391 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_count_per_host", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "max_osd_draining_count": { 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_osd_draining_count", 2024-09-21T07:10:08.392 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "migration_current": { 2024-09-21T07:10:08.393 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "migration_current", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "internal - do not modify", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "mode": { 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mode", 2024-09-21T07:10:08.394 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "root", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm-package", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "root" 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.395 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "oob_default_addr": { 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "oob_default_addr", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "169.254.1.1", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.396 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "prometheus_alerts_path": { 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.397 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_insecure": { 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_insecure", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.398 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_password": { 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_password", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.399 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_url": { 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_url", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "registry_username": { 2024-09-21T07:10:08.400 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "registry_username", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "secure_monitoring_stack": { 2024-09-21T07:10:08.401 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "service_discovery_port": { 2024-09-21T07:10:08.402 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "service_discovery_port", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8765", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.403 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_config_file": { 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_config_file", 2024-09-21T07:10:08.404 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.405 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "ssh_keepalive_interval": { 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.406 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "use_agent": { 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_agent", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.407 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "use_repo_digest": { 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "use_repo_digest", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.408 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_failed_host_check": { 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.409 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_daemons": { 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.410 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_on_stray_hosts": { 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.411 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crash", 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.412 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.413 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.414 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.415 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.416 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.417 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "retain_interval": { 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retain_interval", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "31536000", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.418 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_recent_interval": { 2024-09-21T07:10:08.419 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_recent_interval", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1209600", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.420 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dashboard", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.421 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.422 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.423 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.424 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.425 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.426 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.427 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.428 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.429 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.430 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.431 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.432 INFO:teuthology.orchestra.run.smithi114.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.433 INFO:teuthology.orchestra.run.smithi114.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:10:08.434 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:10:08.435 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_URL": { 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:10:08.436 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.437 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.438 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.439 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.440 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.441 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.442 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "[]", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.443 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "{}", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.444 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.445 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.446 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.447 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.448 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.449 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.450 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.451 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.452 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.453 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.454 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.455 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.456 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "45", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.457 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.458 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "admin", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.459 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.460 INFO:teuthology.orchestra.run.smithi114.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.461 INFO:teuthology.orchestra.run.smithi114.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.462 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:10:08.463 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:10:08.464 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "cross_origin_url": { 2024-09-21T07:10:08.465 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cross_origin_url", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "crt_file": { 2024-09-21T07:10:08.466 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "crt_file", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "debug": { 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "debug", 2024-09-21T07:10:08.467 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "jwt_token_ttl": { 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "jwt_token_ttl", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.468 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "28800", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.469 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.470 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.471 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.472 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.473 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.474 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "motd": { 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "motd", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.475 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "The message of the day", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.476 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-21T07:10:08.477 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8080", 2024-09-21T07:10:08.478 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.479 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.480 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl_server_port": { 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl_server_port", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8443", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.481 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "sso_oauth2": { 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sso_oauth2", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.482 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "redirect", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.483 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "redirect" 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:08.484 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "url_prefix": { 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url_prefix", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.485 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.486 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_monitoring": { 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_monitoring", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "monitor device health metrics", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.487 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.488 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.489 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.490 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.491 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "mark_out_threshold": { 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mark_out_threshold", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.492 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "2419200", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "pool_name": { 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pool_name", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.493 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "device_health_metrics", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "retention_period": { 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "retention_period", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.494 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15552000", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_frequency": { 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_frequency", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.495 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "self_heal": { 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "self_heal", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.496 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.497 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.498 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "warn_threshold": { 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "warn_threshold", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7257600", 2024-09-21T07:10:08.499 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "diskprediction_local", 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.500 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.501 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.502 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.503 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.504 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "predict_interval": { 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predict_interval", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.505 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "86400", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "predictor_model": { 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "predictor_model", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.506 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "prophetstor", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.507 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "600", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.508 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "influx", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": false, 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:10:08.509 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "batch_size": { 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "batch_size", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5000", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.510 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "database": { 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "database", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "ceph", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.511 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "hostname": { 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "hostname", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.512 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "30", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "5", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.513 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.514 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.515 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.516 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.517 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "password": { 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "password", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.518 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "port": { 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "port", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.519 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "8086", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "InfluxDB server port", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.520 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "ssl": { 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ssl", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.521 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "false", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "threads": { 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threads", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.522 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "32", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "username": { 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "username", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.523 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "verify_ssl": { 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "verify_ssl", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.524 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "true", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "insights", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.525 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.526 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.527 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.528 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.529 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.530 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.531 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "iostat", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.532 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.533 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.534 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.535 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.536 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.537 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "k8sevents", 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_event_retention_days": { 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "ceph_event_retention_days", 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.538 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "7", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "config_check_secs": { 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "config_check_secs", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.539 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "10", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.540 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.541 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.542 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.543 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.544 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.545 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "localpool", 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "failure_domain": { 2024-09-21T07:10:08.546 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "failure_domain", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "host", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.547 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.548 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.549 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.550 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.551 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "min_size": { 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "min_size", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.552 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "num_rep": { 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "num_rep", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.553 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_num": { 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_num", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.554 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "128", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "prefix": { 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prefix", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.555 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.556 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "subtree": { 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "subtree", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "rack", 2024-09-21T07:10:08.557 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mds_autoscaler", 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.558 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.559 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.560 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.561 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.562 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.563 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.564 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirroring", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.565 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.566 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.567 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.568 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.569 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.570 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "nfs", 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.571 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.572 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.573 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.574 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.575 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.576 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-21T07:10:08.577 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "fail_fs": { 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "fail_fs", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.578 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.579 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.580 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.581 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.582 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.583 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator": { 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "orchestrator", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "cephadm", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "rook", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "test_orchestrator" 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Orchestrator backend", 2024-09-21T07:10:08.584 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.585 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_perf_query", 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.586 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.587 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.588 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.589 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.590 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.591 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "osd_support", 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.592 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.593 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.594 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.595 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.597 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.597 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.597 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.597 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.597 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.598 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.599 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.600 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "pg_autoscaler", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.601 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.602 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.603 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.604 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.605 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.606 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.607 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "threshold": { 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "threshold", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "3.0", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1.0", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "scaling threshold", 2024-09-21T07:10:08.608 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "progress", 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_pg_recovery_event": { 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.609 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-21T07:10:08.610 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.611 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.612 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.613 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.614 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.615 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "max_completed_events": { 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_completed_events", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.616 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "50", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "sleep_interval": { 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sleep_interval", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.617 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.618 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "prometheus", 2024-09-21T07:10:08.619 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "cache": { 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "cache", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.620 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "exclude_perf_counters": { 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "exclude_perf_counters", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.621 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.622 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.623 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.624 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.625 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.626 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools": { 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.627 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "300", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.628 INFO:teuthology.orchestra.run.smithi114.stdout: "scrape_interval": { 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "scrape_interval", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15.0", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.629 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "::", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:08.630 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "9283", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.631 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "stale_cache_strategy": { 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stale_cache_strategy", 2024-09-21T07:10:08.632 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "log", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_behaviour": { 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_behaviour", 2024-09-21T07:10:08.633 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "default", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "default", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "error" 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.634 INFO:teuthology.orchestra.run.smithi114.stdout: "standby_error_status_code": { 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "standby_error_status_code", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "400", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "599", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.635 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.636 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.637 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.638 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.639 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.640 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_snap_create": { 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.641 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.642 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.643 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "trash_purge_schedule": { 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "trash_purge_schedule", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.644 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "restful", 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "enable_auth": { 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enable_auth", 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.645 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "key_file": { 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "key_file", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.646 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.647 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.648 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.649 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.650 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.651 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "max_requests": { 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_requests", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "500", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.652 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "server_addr": { 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_addr", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.653 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "server_port": { 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "server_port", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.654 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.655 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rgw", 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.656 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.657 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.658 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.659 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.660 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.661 INFO:teuthology.orchestra.run.smithi114.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "5", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.662 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.663 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rook", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.664 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.665 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.666 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.667 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.668 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.669 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "storage_class": { 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "storage_class", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "local", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.670 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "selftest", 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.671 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.672 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.673 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.674 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.675 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "roption1": { 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption1", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.676 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "roption2": { 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "roption2", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "xyz", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.677 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption1": { 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption1", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.678 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption2": { 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption2", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.679 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption3": { 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption3", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "float", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.680 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption4": { 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption4", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.681 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption5": { 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption5", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.682 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption6": { 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption6", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.683 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "rwoption7": { 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rwoption7", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "1", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "42", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.684 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.685 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "testkey": { 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testkey", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.686 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "testlkey": { 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testlkey", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.687 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "testnewline": { 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "testnewline", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.688 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "smb", 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "internal_store_backend": { 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "internal_store_backend", 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.689 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.690 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.691 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.692 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.693 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.694 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.695 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "update_orchestration": { 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "update_orchestration", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.696 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "allow_m_granularity": { 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "allow_m_granularity", 2024-09-21T07:10:08.697 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "dump_on_update": { 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "dump_on_update", 2024-09-21T07:10:08.698 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.699 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.700 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.701 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.702 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.703 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.704 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "stats", 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.705 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.706 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.707 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.708 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.709 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.710 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.711 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "status", 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.712 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.713 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.714 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.715 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.716 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.717 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telegraf", 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "address": { 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "address", 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.718 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.719 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "15", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.720 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.721 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.722 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.723 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.724 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.725 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "telemetry", 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_basic": { 2024-09-21T07:10:08.726 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_basic", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_crash": { 2024-09-21T07:10:08.727 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_crash", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_device": { 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_device", 2024-09-21T07:10:08.728 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_ident": { 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_ident", 2024-09-21T07:10:08.729 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "channel_perf": { 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "channel_perf", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.730 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "contact": { 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "contact", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.731 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "description": { 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "description", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.732 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "device_url": { 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "device_url", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.733 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "enabled": { 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "enabled", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.734 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "24", 2024-09-21T07:10:08.735 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "8", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "last_opt_revision": { 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "last_opt_revision", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "1", 2024-09-21T07:10:08.736 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard": { 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.737 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "leaderboard_description": { 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "leaderboard_description", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.738 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.739 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.740 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.741 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.742 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.743 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "organization": { 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "organization", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.744 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "proxy": { 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "proxy", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.745 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.746 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "url": { 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "url", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.747 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "test_orchestrator", 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.748 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.749 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.750 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.751 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.752 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.753 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.754 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.755 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.756 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.757 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.758 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.759 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "max_concurrent_clones": { 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "max_concurrent_clones", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "4", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.760 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "periodic_async_work": { 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "periodic_async_work", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Periodically check for async work", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.761 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_delay": { 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.762 INFO:teuthology.orchestra.run.smithi114.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "True", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.763 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.764 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "can_run": true, 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "error_string": "", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "module_options": { 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "discovery_interval": { 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "discovery_interval", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "uint", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "100", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.765 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "identifier": { 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "identifier", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.766 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "interval": { 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "interval", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "secs", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "60", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.767 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "log_level": { 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_level", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.768 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster": { 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.769 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_cluster_level": { 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_cluster_level", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.770 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "info", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [ 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "critical", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "debug", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "error", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "info", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "warning" 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.771 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "log_to_file": { 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "log_to_file", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "bool", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "False", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.772 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "sqlite3_killpoint": { 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "dev", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 1, 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "0", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.773 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_host": { 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_host", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.774 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_port": { 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_port", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "int", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "10051", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.775 INFO:teuthology.orchestra.run.smithi114.stdout: "zabbix_sender": { 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "zabbix_sender", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "str", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "level": "advanced", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "flags": 0, 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "min": "", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "max": "", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "enum_allowed": [], 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "desc": "", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "long_desc": "", 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "tags": [], 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: "see_also": [] 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.776 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "services": { 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "dashboard": "https://172.21.15.114:8443/" 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "always_on_modules": { 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "octopus": [ 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-21T07:10:08.777 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "pacific": [ 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-21T07:10:08.778 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "quincy": [ 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.779 INFO:teuthology.orchestra.run.smithi114.stdout: "reef": [ 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: ], 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "squid": [ 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "balancer", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "crash", 2024-09-21T07:10:08.780 INFO:teuthology.orchestra.run.smithi114.stdout: "devicehealth", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "orchestrator", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "pg_autoscaler", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "progress", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "rbd_support", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "status", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "telemetry", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "volumes" 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "last_failure_osd_epoch": 5, 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "active_clients": [ 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "devicehealth", 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:08.781 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 117743516 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "libcephsqlite", 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 1996042013 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.782 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "rbd_support", 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3447237743 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-21T07:10:08.783 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 177384431 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "snap_schedule", 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3261987584 2024-09-21T07:10:08.784 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: "name": "volumes", 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: "addrvec": [ 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: { 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: "type": "v2", 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: "addr": "172.21.15.114:0", 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: "nonce": 3970362811 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout: ] 2024-09-21T07:10:08.785 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:10:08.786 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:07 smithi193 ceph-mon[34601]: pgmap v891: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 265 KiB/s wr, 59 op/s 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:07 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2886781832' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:07 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3636396161' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:08 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2999179893' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:07 smithi183 ceph-mon[34219]: pgmap v891: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 265 KiB/s wr, 59 op/s 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:07 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2886781832' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:07 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3636396161' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:10:08.787 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:08 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2999179893' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c4c0b6680 msgr2=0x7f6c4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c4c0b6680 0x7f6c4c0b8b90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6c58031440 tx=0x7f6c58009560 comp rx=0 tx=0).stop 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 msgr2=0x7f6c681a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 0x7f6c681a1e50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000eb10 tx=0x7f6c6000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 shutdown_connections 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f6c6810dd40 0x7f6c681a1e50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f6c4c0b6680 0x7f6c4c0b8b90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c68105230 0x7f6c6819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:08.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 --2- 172.21.15.114:0/2999179893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c681048e0 0x7f6c6819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 >> 172.21.15.114:0/2999179893 conn(0x7f6c680fe350 msgr2=0x7f6c6810f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 shutdown_connections 2024-09-21T07:10:08.789 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:07.356+0000 7f6c6d400640 1 -- 172.21.15.114:0/2999179893 wait complete. 2024-09-21T07:10:08.795 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T07:10:08.795 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T07:10:08.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:07 smithi114 ceph-mon[30654]: pgmap v891: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 265 KiB/s wr, 59 op/s 2024-09-21T07:10:08.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:07 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2886781832' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T07:10:08.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:07 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3636396161' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-21T07:10:08.796 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:08 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2999179893' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T07:10:09.176 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:09.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:09 smithi114 ceph-mon[30654]: pgmap v892: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 114 KiB/s wr, 42 op/s 2024-09-21T07:10:09.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:09 smithi183 ceph-mon[34219]: pgmap v892: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 114 KiB/s wr, 42 op/s 2024-09-21T07:10:09.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:09 smithi193 ceph-mon[34601]: pgmap v892: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 114 KiB/s wr, 42 op/s 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 981, 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1008, 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 547801, 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 926700, 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 39, 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 8, 2024-09-21T07:10:09.941 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.340561087, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.090510265, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.084718050, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.138408234, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1006, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.425626560, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000423088 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 301, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000070125, 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000232 2024-09-21T07:10:09.942 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 4825, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 4113, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 2176086, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 7479663, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 10, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 6, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.796353310, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.387441481, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.349660413, 2024-09-21T07:10:09.943 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.103885637, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 4114, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 1.287365547, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000312923 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5571, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001600071, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000287 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 589632 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.944 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 4055, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 5084, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 2356603, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 2555177, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 20, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 8, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 1.969805074, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 1.543850682, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.400255396, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.033072057, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5081, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 2.013199232, 2024-09-21T07:10:09.945 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000396221 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3500, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.001493172, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000426 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1162736, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 228224 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T07:10:09.946 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2458, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 3.460081075, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001407681 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 252, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.026710214, 2024-09-21T07:10:09.947 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000105992 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 1256, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 1232, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1232, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 34.340831714, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.027874051 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 24, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 5, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 2, 2024-09-21T07:10:09.948 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 19, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 1, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 360, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 46, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 304, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 10, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 294, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 160, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 58, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 58, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 19, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 4753, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 4115, 2024-09-21T07:10:09.949 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 23, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 5, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 190, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 4961, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 5, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 157, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 5, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 143, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 1388, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 1, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 1017, 2024-09-21T07:10:09.950 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 1486476457, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 7, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 1388, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 36, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 36, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 1352, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 743, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 979, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 3, 2024-09-21T07:10:09.951 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 1399, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 528, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 104, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 852 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 6, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 7, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 2, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:09.952 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 197, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 83, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 114, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 15, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 1656, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 8, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 3, 2024-09-21T07:10:09.953 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_write_failed": 0 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 2324, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 2324, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-21T07:10:09.954 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 5, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 5, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 9058017, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 0, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 948, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 14.032063773, 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.014801755 2024-09-21T07:10:09.955 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 0 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 166, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 400, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 234, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 37, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 46, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 9, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 360, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 660, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 300, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 58, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 327, 2024-09-21T07:10:09.956 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 269, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 126220, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 334311424 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 1805, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 1256, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 253, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 35, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 111, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 29.061335132, 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.261813830 2024-09-21T07:10:09.957 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 13, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.019080981, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001467767 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 114, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.496166670, 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.004352339 2024-09-21T07:10:09.958 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 508, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 1.446272801, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.002846993 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000115166, 2024-09-21T07:10:09.959 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000115166 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.960 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-21T07:10:09.961 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.962 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 141, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.424858826, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.003013183 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 33, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.029298785, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000887841 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 195, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 2.083792149, 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.010686113 2024-09-21T07:10:09.963 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.964 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.965 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.966 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.967 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 116, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.779911204, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.006723372 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.968 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 2, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 2, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 227, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 113, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 928, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:09.969 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:09.970 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T07:10:09.971 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 832326, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 313, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 9648, 2024-09-21T07:10:09.972 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 196, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 852447, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 36465, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-21T07:10:09.973 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 2831, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 5017756, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 27, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 2831, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2831, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 28.536373367, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.010079962 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2831, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 3025 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 2831, 2024-09-21T07:10:09.974 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 1418, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 1413, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 0, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 10, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 23, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 971, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 87, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 2, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 1, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 181, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-21T07:10:09.975 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1395, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 20, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-21T07:10:09.976 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 105, 2024-09-21T07:10:09.977 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 18, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 12, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 12, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 0, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 265, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 14, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 56 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 1, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-21T07:10:09.978 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 5, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 170, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 83, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 9857, 2024-09-21T07:10:09.979 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 4215808, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 9857, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 9857, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 4215808, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T07:10:09.980 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 2831, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 5026955, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2831, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 5026955, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.981 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 2831, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 2831, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 2831, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2831, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.982 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 83, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 9287, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 83, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:09.983 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 23, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 9287, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x565423d836c0": { 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 4592, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 2324, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 4863713, 2024-09-21T07:10:09.984 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 2324, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 948, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 4859121, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "trackedop": { 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 1713 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:09.985 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:10:10.939 INFO:tasks.check_counter:Daemon mds.a mds.exported=5 2024-09-21T07:10:10.940 INFO:tasks.check_counter:Daemon mds.a mds.imported=5 2024-09-21T07:10:10.940 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=7 2024-09-21T07:10:10.940 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:10.940 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:10.940 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T07:10:10.940 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T07:10:11.301 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:11 smithi114 ceph-mon[30654]: pgmap v893: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 68 KiB/s wr, 36 op/s 2024-09-21T07:10:11.338 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:11.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:11 smithi183 ceph-mon[34219]: pgmap v893: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 68 KiB/s wr, 36 op/s 2024-09-21T07:10:11.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:11 smithi193 ceph-mon[34601]: pgmap v893: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 68 KiB/s wr, 36 op/s 2024-09-21T07:10:12.295 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:10:12.295 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:12.295 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 324, 2024-09-21T07:10:12.295 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 319, 2024-09-21T07:10:12.295 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 867232, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 197040, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 0, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.096986276, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.056193211, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.047888062, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 318, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.064927251, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000204173 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:12.296 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 867232, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 197040 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 244, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 320, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 203680, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-09-21T07:10:12.297 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.045923363, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.037793812, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.003725599, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 242, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.036183182, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000149517 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:10:12.298 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 203680 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 4, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 4, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 2944, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 448, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 2, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.003481134, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.000079605, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.003319258, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:12.299 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.005542537, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.001847512 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 0, 2024-09-21T07:10:12.300 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:12.301 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:12.302 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 149735, 2024-09-21T07:10:12.303 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 30, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T07:10:12.304 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 968, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 2, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 326, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 821429, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 326, 2024-09-21T07:10:12.305 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 326, 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 821429, 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:12.306 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:10:12.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:12 smithi114 ceph-mon[30654]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T07:10:12.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:12 smithi183 ceph-mon[34219]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T07:10:12.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:12 smithi193 ceph-mon[34601]: osdmap e106: 12 total, 12 up, 12 in 2024-09-21T07:10:13.198 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T07:10:13.198 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T07:10:13.198 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T07:10:13.198 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T07:10:13.198 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T07:10:13.198 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T07:10:13.198 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T07:10:13.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:13 smithi193 ceph-mon[34601]: pgmap v895: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 70 KiB/s wr, 38 op/s 2024-09-21T07:10:13.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:13 smithi114 ceph-mon[30654]: pgmap v895: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 70 KiB/s wr, 38 op/s 2024-09-21T07:10:13.572 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:13.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:13 smithi183 ceph-mon[34219]: pgmap v895: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 70 KiB/s wr, 38 op/s 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 1055, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 1055, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 355481, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 334855, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 7, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.242523012, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.122556075, 2024-09-21T07:10:14.335 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.044810285, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.053830946, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 1053, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.233547996, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000221792 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:14.336 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 2578, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2539, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 3846321, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 898635, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 3, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.385002485, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.172703164, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.115117565, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.089545190, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 2538, 2024-09-21T07:10:14.337 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.332151057, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000130871 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 1129520, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 199296 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 9, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 246, 2024-09-21T07:10:14.338 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 4612, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 589701, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 7, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 4, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.043212578, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.031816602, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.007942931, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000103224, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 245, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.062947923, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000256930 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:14.339 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 588736 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 1, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-MDSRank": { 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T07:10:14.340 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 5269, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.222270808, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000042184 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "queue_len": 0, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "complete_latency": { 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "mds": { 2024-09-21T07:10:14.341 INFO:teuthology.orchestra.run.smithi114.stdout: "request": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "reply": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "reply_latency": { 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_reply": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "forward": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_commit": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_split": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_merge": 0, 2024-09-21T07:10:14.342 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes": 575, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_top": 219, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_bottom": 356, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pin_tail": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_pinned": 261, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_expired": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "inodes_with_caps": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "caps": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "subtrees": 23, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_hit": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_forward": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_discover": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:14.343 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "traverse_lock": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "load_cent": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "q": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "exported": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "exported_inodes": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "imported": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "imported_inodes": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rfiles": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rbytes": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "root_rsnaps": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:10:14.344 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "scrub_file_inodes": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps": 0, 2024-09-21T07:10:14.345 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_cap_release": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "process_request_cap_release": 0 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_cache": { 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_try_discover": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_send_discover": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "dir_handle_discover": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_created": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_enqueued": 0, 2024-09-21T07:10:14.346 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "strays_migrated": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_started": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "recovery_completed": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_exportdir": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_flush": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:14.347 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_started": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "uninline_write_failed": 0 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_log": { 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "evlrg": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "evadd": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "evex": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "evtrm": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "ev": 1173, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "evexg": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "evexd": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "segadd": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "segex": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "segtrm": 0, 2024-09-21T07:10:14.348 INFO:teuthology.orchestra.run.smithi114.stdout: "seg": 3, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "segmjr": 2, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "segexg": 0, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "segexd": 0, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "expos": 4194304, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "wrpos": 6353255, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "rdpos": 6352681, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "jlat": { 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "replayed": 1173 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_mem": { 2024-09-21T07:10:14.349 INFO:teuthology.orchestra.run.smithi114.stdout: "ino": 456, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "ino+": 622, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "ino-": 166, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "dir": 32, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "dir+": 35, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "dir-": 3, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "dn": 575, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "dn+": 627, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "dn-": 52, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "cap": 0, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "cap+": 0, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "cap-": 0, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "rss": 119148, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "heap": 319565824 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.350 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_server": { 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_client_request": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_request": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_client_session": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "handle_peer_request": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "req_create_latency": { 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getattr_latency": { 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.351 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "req_link_latency": { 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookup_latency": { 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.352 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupino_latency": { 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupname_latency": { 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.353 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "req_lssnap_latency": { 2024-09-21T07:10:14.354 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mkdir_latency": { 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.355 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mknod_latency": { 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "req_mksnap_latency": { 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "req_open_latency": { 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.356 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "req_readdir_latency": { 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rename_latency": { 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.357 INFO:teuthology.orchestra.run.smithi114.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmdir_latency": { 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.358 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setattr_latency": { 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.359 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setlayout_latency": { 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.360 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "req_setxattr_latency": { 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "req_symlink_latency": { 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "req_unlink_latency": { 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.361 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_sessions": { 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "session_count": 2, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "session_add": 3, 2024-09-21T07:10:14.362 INFO:teuthology.orchestra.run.smithi114.stdout: "session_remove": 1, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_open": 2, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "sessions_stale": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "total_load": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "average_load": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "avg_session_uptime": 2207, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:14.363 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:14.364 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T07:10:14.365 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 215008, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 509, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:14.366 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 9648, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 196, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 1526367, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 15174, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.367 INFO:teuthology.orchestra.run.smithi114.stdout: "objecter": { 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_active": 0, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_laggy": 0, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send": 3255, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_send_bytes": 21, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_resend": 3, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_reply": 3255, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_latency": { 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3255, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 58.823709464, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.018071800 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "op_inflight": 0, 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "oplen_avg": { 2024-09-21T07:10:14.368 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 3255, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 3256 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "op": 3255, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "op_r": 3255, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "op_w": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "op_rmw": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "op_pg": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_stat": 2028, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_create": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_read": 1225, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_write": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writefull": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_writesame": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_append": 0, 2024-09-21T07:10:14.369 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_zero": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_truncate": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_delete": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_mapext": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_getxattr": 1, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_setxattr": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_call": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_watch": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_notify": 0, 2024-09-21T07:10:14.370 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "osdop_other": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_active": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_send": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_resend": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "linger_ping": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_active": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_send": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "poolop_resend": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_active": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_send": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "poolstat_resend": 0, 2024-09-21T07:10:14.371 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_active": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_send": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "statfs_resend": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "command_active": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "command_send": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "command_resend": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "map_epoch": 106, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "map_full": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "map_inc": 19, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_sessions": 6, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_open": 8, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_session_close": 2, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_laggy": 0, 2024-09-21T07:10:14.372 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_wr": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_rd": 2, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_del": 0 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "oft": { 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_objs": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_updates": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "omap_total_removes": 0 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "purge_queue": { 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executing_high_water": 0, 2024-09-21T07:10:14.373 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed_ops": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_executed": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 3638, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 3856696, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 3638, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:14.374 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 3638, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 3856696, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:14.375 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 3255, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 2158956, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 3255, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 2158956, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 1024, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:14.376 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 3255, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 3255, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 3255, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 3255, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:14.377 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.378 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-write_buf_throttle-0x559e634596c0": { 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 3758096384, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 0, 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:14.379 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "trackedop": { 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: "slow_ops_count": 0 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:14.380 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:10:15.448 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T07:10:15.448 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T07:10:15.448 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T07:10:15.448 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:15.448 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:15.448 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T07:10:15.449 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T07:10:15.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:15 smithi114 ceph-mon[30654]: pgmap v896: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-09-21T07:10:15.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:15 smithi183 ceph-mon[34219]: pgmap v896: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-09-21T07:10:15.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:15 smithi193 ceph-mon[34601]: pgmap v896: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-09-21T07:10:15.821 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:16.581 INFO:teuthology.orchestra.run.smithi114.stdout:{ 2024-09-21T07:10:16.581 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 326, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 321, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 867776, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 198320, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 0, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.100027315, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.061554577, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.045747069, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 320, 2024-09-21T07:10:16.582 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.046762388, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000146132 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 867776, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 198320 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:16.583 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 6, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 248, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 3264, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 204480, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 4, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 2, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.047462462, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.038349828, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.005376340, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 247, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.050119026, 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000202911 2024-09-21T07:10:16.584 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 204480 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_messages": 0, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages": 2, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_bytes": 0, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_bytes": 0, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_created_connections": 2, 2024-09-21T07:10:16.585 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_active_connections": 1, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_total_time": 0.000833456, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_recv_time": 0.000704673, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.586 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "cct": { 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "total_workers": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "unhealthy_workers": 0 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "mempool": { 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:16.587 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:16.588 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_bytes": 149735, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_anon_items": 30, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:16.589 INFO:teuthology.orchestra.run.smithi114.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_bytes": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_bytes": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "pgmap_items": 0, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_bytes": 968, 2024-09-21T07:10:16.590 INFO:teuthology.orchestra.run.smithi114.stdout: "mds_co_items": 2, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_1_items": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "unittest_2_items": 0 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: }, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "val": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "max": 104857600, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "get_started": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "get": 328, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "get_sum": 821689, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "get_or_fail_success": 328, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "take": 0, 2024-09-21T07:10:16.591 INFO:teuthology.orchestra.run.smithi114.stdout: "take_sum": 0, 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: "put": 328, 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: "put_sum": 821689, 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: "wait": { 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgcount": 0, 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: "sum": 0.000000000, 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: "avgtime": 0.000000000 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout: } 2024-09-21T07:10:16.592 INFO:teuthology.orchestra.run.smithi114.stdout:} 2024-09-21T07:10:17.408 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T07:10:17.408 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T07:10:17.408 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T07:10:17.408 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T07:10:17.408 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T07:10:17.408 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T07:10:17.409 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T07:10:17.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:17 smithi114 ceph-mon[30654]: pgmap v897: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:10:17.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:17 smithi183 ceph-mon[34219]: pgmap v897: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:10:17.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:17 smithi193 ceph-mon[34601]: pgmap v897: 129 pgs: 4 active+clean+snaptrim, 14 active+clean+snaptrim_wait, 111 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:10:17.747 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6439, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 7179, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2684019, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 16211424, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 307, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.695842132, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.286037730, 2024-09-21T07:10:18.621 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.330169801, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.076400903, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7183, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.494626449, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000208078 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6602, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001331110, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000201 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:18.622 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4093, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4442, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2773830, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 7608245, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 13, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.402405026, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.191493086, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.177786535, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.056972920, 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:18.623 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4445, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.272152089, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000511170 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3118, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000685558, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000219 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1181664, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 232448 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 11289, 2024-09-21T07:10:18.624 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 11926, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4618476, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 7923522, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 44, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 10, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.992831646, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.431275431, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.523931903, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.153993436, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11915, 2024-09-21T07:10:18.625 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.863420988, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000072465 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 13254, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003058276, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000230 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 545680, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 808864 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-21T07:10:18.626 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3747, 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.436134938, 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000917036 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:18.627 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 100, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004148836, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041488 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 2038, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 1996, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1996, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 154.045420533, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.077177064 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 42, 2024-09-21T07:10:18.628 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 8, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 3, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 631, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 3, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 120, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 49, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 57, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 14, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 48, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 1160, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 36, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 36, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 14, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 4694, 2024-09-21T07:10:18.629 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 4109, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 42, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 3, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 51, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 502, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 24, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 1140, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 2, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 164, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 1575, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:18.630 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 1014, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 1454800519, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 7, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 1575, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 231, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 15, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 216, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 1344, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 12, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 860, 2024-09-21T07:10:18.631 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 1504, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 2128, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 1166, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 24, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 181 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 6, 2024-09-21T07:10:18.632 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 37, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 50, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 24, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 26, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 15, 2024-09-21T07:10:18.633 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 1713, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 15, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 24, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 3, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 3452, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 3438, 2024-09-21T07:10:18.634 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 3438, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 14, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 11, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 75, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 74, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 74, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 1, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 1, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 1, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 16688312, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 16801122, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-21T07:10:18.635 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1747, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 42.809457760, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.024504555 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 124, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1274, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1150, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 43, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 105, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 62, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 120, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 1590, 2024-09-21T07:10:18.636 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1470, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 36, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 1112, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 1076, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 138804, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 373043200 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 2237, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 12, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 2038, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 110, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 25, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1020, 2024-09-21T07:10:18.637 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.178152028, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003115835 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 185, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.055169160, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000298211 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-21T07:10:18.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 26, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.041648548, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001601867 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 141, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 148.692234829, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 1.054554856 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-21T07:10:18.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000236628, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000236628 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:18.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004495274, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000280954 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 37, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.294317291, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.007954521 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-21T07:10:18.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.663500130, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.041468758 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 405, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.082644203, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000204059 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.642 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 41, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.073549189, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001793882 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 49, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.239395429, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004885621 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.643 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012874094, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006437047 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.602110515, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.066901168 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.644 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001637664, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000818832 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.645 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.027398159, 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002739815 2024-09-21T07:10:18.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 36, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.076057392, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002112705 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.647 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 4, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 47, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 23, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 951, 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:18.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:18.649 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:18.650 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 314852, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 191, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:18.651 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9888, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 202, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 524889, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 5644, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:18.652 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 7041, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 16707419, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 37, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 7041, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7041, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 385.276168356, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.054718955 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-21T07:10:18.653 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7041, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 11456 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 7041, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 1625, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 5416, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 560, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 2069, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 24, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 1756, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 191, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-21T07:10:18.654 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 3, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 577, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1601, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 3130, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-21T07:10:18.655 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-21T07:10:18.656 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 106, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 28, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 1465, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 22, 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 58 2024-09-21T07:10:18.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 5, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 60, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 24, 2024-09-21T07:10:18.658 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 21817, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 8106257, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 21817, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 21817, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 8106257, 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:18.659 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:18.660 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 7041, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 16709879, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 7041, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 16709879, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.661 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 7041, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 7041, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 7041, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 7041, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.662 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 24, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2600, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 24, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 9, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2600, 2024-09-21T07:10:18.663 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5631561af450": { 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3452, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 12606818, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3452, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:18.664 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1747, 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12606818, 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 1515 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:18.665 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-21T07:10:19.540 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:19 smithi183 ceph-mon[34219]: pgmap v898: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:10:19.581 INFO:tasks.check_counter:Daemon mds.b mds.exported=24 2024-09-21T07:10:19.581 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-09-21T07:10:19.581 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-21T07:10:19.581 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-09-21T07:10:19.581 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-09-21T07:10:19.581 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T07:10:19.581 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T07:10:19.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:19 smithi193 ceph-mon[34601]: pgmap v898: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:10:19.815 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:19 smithi114 ceph-mon[30654]: pgmap v898: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-09-21T07:10:19.839 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:10:21.030 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3464, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3465, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5834607, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 1101241, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.323304210, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.128656911, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.076002260, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.100012790, 2024-09-21T07:10:21.031 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3463, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.142346068, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041104 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:21.032 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 387, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 348, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1131948, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 203370, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.057915478, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.026334594, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.042864637, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000137506, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 347, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.029961532, 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000086344 2024-09-21T07:10:21.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1131392, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 202672 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 7, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 248, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3532, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 591422, 2024-09-21T07:10:21.034 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.019361666, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.013552770, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.003772341, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000037513, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 247, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021815420, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000088321 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 591104 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:21.036 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5824, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.149957525, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000025748 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.037 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-21T07:10:21.038 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 487, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 4, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 483, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 482, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 10, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-21T07:10:21.039 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-21T07:10:21.040 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-21T07:10:21.041 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:21.042 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:10:21.043 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 2324, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-21T07:10:21.044 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 5, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 9058017, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 9057443, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.045 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 2324 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 149, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 389, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 240, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 20, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 28, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 8, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 487, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 649, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 162, 2024-09-21T07:10:21.046 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 119368, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 319569920 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.047 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-21T07:10:21.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.051 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.052 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.053 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-21T07:10:21.054 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.055 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.056 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2493, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-21T07:10:21.059 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:21.060 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:21.061 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 190190, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 267, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:21.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9888, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 202, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 837452, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 35825, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:21.063 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3466, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3466, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3466, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 30.384168819, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.008766349 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-21T07:10:21.064 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3466, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3467 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3466, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3466, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2124, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1340, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-21T07:10:21.065 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-21T07:10:21.066 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-21T07:10:21.067 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 107, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 20, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 4, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-21T07:10:21.068 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-21T07:10:21.069 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3854, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 6600623, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3854, 2024-09-21T07:10:21.070 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3854, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 6600623, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:21.071 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3466, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4864053, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3466, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4864053, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3466, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3466, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3466, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3466, 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:21.073 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:21.074 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5593dc2736c0": { 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:21.075 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:21.076 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-21T07:10:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:21 smithi114 ceph-mon[30654]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T07:10:21.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:21 smithi114 ceph-mon[30654]: pgmap v900: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-21T07:10:21.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:21 smithi183 ceph-mon[34219]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T07:10:21.364 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:21 smithi183 ceph-mon[34219]: pgmap v900: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-21T07:10:21.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:21 smithi193 ceph-mon[34601]: osdmap e107: 12 total, 12 up, 12 in 2024-09-21T07:10:21.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:21 smithi193 ceph-mon[34601]: pgmap v900: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-09-21T07:10:22.164 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T07:10:22.164 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T07:10:22.165 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-21T07:10:22.165 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:22.165 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:22.165 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T07:10:22.165 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T07:10:22.413 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:10:23.014 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3424, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3017, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2760380, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 4126293, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 38, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 8, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.364489424, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.174096291, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.173360538, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.035343897, 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:23.015 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3016, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.165783744, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000054968 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3326, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000648720, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000195 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1148384, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 220752 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:23.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 3892, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4505, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1975977, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 2669419, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 16, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.347152860, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.164078963, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.156785971, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.028731190, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4504, 2024-09-21T07:10:23.017 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.385423818, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000085573 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3547, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001073621, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000302 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 4688, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 504656 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1943, 2024-09-21T07:10:23.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2130, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1211155, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 964856, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 14, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.161890511, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.068093265, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.047665820, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.037005989, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2128, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.152439865, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000071635 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:23.019 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 576, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000114491, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000198 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 4176, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 18288 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:23.020 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2327, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.808585324, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000777217 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 401, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.023424585, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000058415 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 657, 2024-09-21T07:10:23.021 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 653, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 653, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 12.602581123, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.019299511 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 4, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 27, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 2, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 456, 2024-09-21T07:10:23.022 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 67, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 371, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 18, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 359, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 54, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 290, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 290, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 24, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 5688, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 5584, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 9, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 42, 2024-09-21T07:10:23.023 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 1327, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 1, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 51, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 8, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 379, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 3106, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 1017, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 1486476457, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 7, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 3106, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:23.024 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 3106, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 351, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 681, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 690, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 140, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 45, 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 505 2024-09-21T07:10:23.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 8, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 7, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 120, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 120, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-21T07:10:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 15, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 3570, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 1, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 4, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 1173, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1173, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 3, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 3, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-21T07:10:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 6353255, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 372, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 11.428240029, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.030721075 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 459, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 630, 2024-09-21T07:10:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 171, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 41, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 44, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 3, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 459, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 635, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 176, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 290, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 485, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 195, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 130032, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 335429632 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 748, 2024-09-21T07:10:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 657, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 29, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 16, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 117, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5.997033456, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.051256696 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 121, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4.377492952, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.036177627 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 140, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.881700485, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.013440717 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 134, 2024-09-21T07:10:23.036 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.073754902, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000550409 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012113332, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000576825 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 117, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.253661676, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002168048 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:23.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.038 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.041 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.006824320, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002274773 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.042 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-21T07:10:23.043 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 132, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 66, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 954, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:23.044 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:23.045 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:23.046 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 309483, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 477, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9888, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 202, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:23.047 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 1580384, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 17584, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 4047, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 2366451, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-09-21T07:10:23.048 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 4047, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4047, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 50.745620041, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.012539070 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4047, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4200 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 4047, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3149, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 898, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-21T07:10:23.049 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 43, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 415, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 93, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 1, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 253, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 3106, 2024-09-21T07:10:23.050 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-21T07:10:23.051 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 107, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 20, 2024-09-21T07:10:23.052 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 200, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 59 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-21T07:10:23.053 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 5, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 240, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 120, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 9255, 2024-09-21T07:10:23.054 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5110684, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 9255, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 9255, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5110684, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:23.055 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4047, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2379747, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4047, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2379747, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.056 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4047, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4047, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4047, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4047, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 120, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 13296, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:23.058 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 120, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 43, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 13296, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55c775c9b5f0": { 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:23.059 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1173, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2158951, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1173, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 372, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2158951, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:23.060 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-21T07:10:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 3858 2024-09-21T07:10:23.061 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:23.061 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-21T07:10:23.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:23 smithi183 ceph-mon[34219]: pgmap v901: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-21T07:10:23.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:23 smithi114 ceph-mon[30654]: pgmap v901: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-21T07:10:23.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:23 smithi193 ceph-mon[34601]: pgmap v901: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-21T07:10:23.856 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-09-21T07:10:23.856 INFO:tasks.check_counter:Daemon mds.h mds.imported=8 2024-09-21T07:10:23.856 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-21T07:10:23.856 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:23.856 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:23.856 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T07:10:23.856 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T07:10:24.102 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T07:10:24.637 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-21T07:10:24.637 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:24.637 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 393, 2024-09-21T07:10:24.637 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 352, 2024-09-21T07:10:24.637 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1135986, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 205364, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.062332387, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.028129883, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.045553327, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000017032, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 352, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.029887694, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000084908 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.638 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1135152, 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 205040 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:24.639 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 931, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1177, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 228962, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 798193, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.118553538, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.052292424, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.021374120, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.034603567, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1175, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.056354468, 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000047961 2024-09-21T07:10:24.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 502448 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2465, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2467, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2771880, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 783928, 2024-09-21T07:10:24.641 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.290207439, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.122125254, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.064371229, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.084377848, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2465, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.112073874, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000045466 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.642 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5839, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.172843255, 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000029601 2024-09-21T07:10:24.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-21T07:10:24.644 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 498, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 70, 2024-09-21T07:10:24.645 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 428, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 398, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 23, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:24.646 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:24.647 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:10:24.648 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-21T07:10:24.649 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-21T07:10:24.650 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:10:24.651 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1698, 2024-09-21T07:10:24.652 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 3, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 6242035, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 6241461, 2024-09-21T07:10:24.653 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 1698 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 485, 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 488, 2024-09-21T07:10:24.654 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 3, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 32, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 34, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 2, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 498, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 498, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 120556, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 319569920 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.655 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.656 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.657 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.659 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-21T07:10:24.660 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-21T07:10:24.661 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.662 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.662 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.662 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.663 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.664 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.666 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.667 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.668 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.669 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:24.670 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-09-21T07:10:24.671 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2497, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:24.672 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:24.673 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:24.674 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 208674, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 533, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:24.675 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9888, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 202, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 1532160, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 16497, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-21T07:10:24.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3396, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3396, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3396, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 12.108668976, 2024-09-21T07:10:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003565568 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3396, 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3397 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3396, 2024-09-21T07:10:24.678 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3396, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2038, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1356, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-21T07:10:24.679 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-21T07:10:24.680 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-21T07:10:24.681 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 107, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 20, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-21T07:10:24.682 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:10:24.683 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3785, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 3773461, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3785, 2024-09-21T07:10:24.684 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3785, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3773461, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:24.685 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3396, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2047736, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3396, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2047736, 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:24.687 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3396, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3396, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3396, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3396, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.688 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x562b474ad2b0": { 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-21T07:10:24.690 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-21T07:10:24.691 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-21T07:10:24.692 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-21T07:10:24.693 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-21T07:10:25.339 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T07:10:25.339 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T07:10:25.339 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T07:10:25.339 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:25.339 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:25.339 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T07:10:25.339 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T07:10:25.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:25 smithi114 ceph-mon[30654]: pgmap v902: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-21T07:10:25.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:25 smithi183 ceph-mon[34219]: pgmap v902: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-21T07:10:25.646 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:10:25.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:25 smithi193 ceph-mon[34601]: pgmap v902: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-21T07:10:26.447 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 5671, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 6948, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2125938, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 11316452, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 50, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 8, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.550607543, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.275977362, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.196906389, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.072191602, 2024-09-21T07:10:26.448 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6944, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.602012897, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000086695 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3645, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001504823, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000412 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 736 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.449 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3509, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 4117, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1988098, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 6382851, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 10, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.338842894, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.159188832, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.100064325, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.059255262, 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:26.450 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4116, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.334292499, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000081217 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1835, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000392505, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000213 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 505040 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:26.451 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 5554, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 6953, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4040603, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 3600235, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 8, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.532723807, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.268295737, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.261849650, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.031055130, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6954, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.455073070, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000065440 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.452 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 8820, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001878924, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000213 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1157888, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 229696 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T07:10:26.453 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4236, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3.944787689, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000931252 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 70, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.004014737, 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000057353 2024-09-21T07:10:26.454 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 982, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 742, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 742, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 7.060717308, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009515791 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 240, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 4, 2024-09-21T07:10:26.455 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 19, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 1, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 686, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 49, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 627, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 10, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 616, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 172, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 410, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 410, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 54, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 8616, 2024-09-21T07:10:26.456 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 8327, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 209, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 4, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 2258, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 2, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 164, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 10, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 550, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 4985, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:26.457 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1017, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1486476457, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 7, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 4985, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 244, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 244, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 4741, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 16, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 1277, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 1343, 2024-09-21T07:10:26.458 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 1, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 1918, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 375, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 23, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 89 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 38, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 6, 2024-09-21T07:10:26.459 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 14, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 14, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 15, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 5615, 2024-09-21T07:10:26.460 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 2, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 1, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 1715, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T07:10:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 1715, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 4, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 4, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 16373562, 2024-09-21T07:10:26.462 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1151, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 55.336000165, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.048076455 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 671, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 849, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 178, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 107, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 114, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 7, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 686, 2024-09-21T07:10:26.463 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 877, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 191, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 410, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 740, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 330, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 131516, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 337526784 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 1025, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 44, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 982, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 140, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 89, 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T07:10:26.464 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 20, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.032699764, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001634988 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 343, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6.497713943, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.018943772 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T07:10:26.465 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 231, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.355631713, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001539531 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T07:10:26.466 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:26.467 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.468 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.469 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 101, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.109295681, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001082135 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 29, 2024-09-21T07:10:26.470 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.026203386, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000903565 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 7, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.017418881, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.002488411 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:26.471 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.472 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.004722615, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.004722615 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.473 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.474 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 10, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.017031325, 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001703132 2024-09-21T07:10:26.475 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T07:10:26.476 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 85, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 42, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 1710, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:26.477 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:26.478 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:26.479 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 322789, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 623, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T07:10:26.480 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9648, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 196, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 2464146, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 28541, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:26.481 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 6411, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 12329048, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 33, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 6411, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6411, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 85.714693560, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.013369941 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.482 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 6411, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6559 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 6411, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 4997, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 1414, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 8, 2024-09-21T07:10:26.483 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 1159, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 71, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 43, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 4985, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T07:10:26.484 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 1, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T07:10:26.485 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 107, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 20, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T07:10:26.486 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 199, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 8, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 55 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 10, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 5, 2024-09-21T07:10:26.487 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 28, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 14, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 14730, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 6852216, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 14730, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 14730, 2024-09-21T07:10:26.488 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6852216, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:26.489 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6411, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 12331006, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6411, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12331006, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:26.490 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 6411, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 6411, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 6411, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6411, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 14, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2022, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 14, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 8, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2022, 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:26.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5624efadd2b0": { 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 574, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1715, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 12179258, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1715, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:26.493 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1151, 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12178684, 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 5980 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:26.494 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T07:10:27.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:27 smithi193 ceph-mon[34601]: pgmap v903: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:10:27.489 INFO:tasks.check_counter:Daemon mds.c mds.exported=2 2024-09-21T07:10:27.489 INFO:tasks.check_counter:Daemon mds.c mds.imported=10 2024-09-21T07:10:27.489 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-21T07:10:27.489 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:27.489 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:27.489 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T07:10:27.489 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T07:10:27.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:27 smithi114 ceph-mon[30654]: pgmap v903: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:10:27.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:27 smithi183 ceph-mon[34219]: pgmap v903: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-21T07:10:27.737 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:10:28.280 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T07:10:28.280 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1661, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1622, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 5739372, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 608608, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 1, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.227129246, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.096008033, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.081594391, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.048168175, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1621, 2024-09-21T07:10:28.281 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.095080618, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000058655 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1131824, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 203472 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:28.282 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1741, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1983, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4693250, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1056494, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.189755793, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.078499952, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.042193448, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.057333960, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1981, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.111061746, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000056063 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.283 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 503744 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 503, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 503, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4323932, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 159137, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-09-21T07:10:28.284 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.057657263, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.023861595, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.015880839, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.014172988, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 502, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.027941969, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000055661 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T07:10:28.285 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5910, 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.192330325, 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000032543 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.286 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.287 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 165, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 102, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 63, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-21T07:10:28.288 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 13, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 4, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-21T07:10:28.289 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1014, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1454831056, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 7, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:28.290 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-09-21T07:10:28.291 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-21T07:10:28.292 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T07:10:28.293 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 3454, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T07:10:28.294 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 2, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 72, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 16688312, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 16835206, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 16835158, 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T07:10:28.295 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 3454 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 110, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 1248, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 1138, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 16, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 70, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 54, 2024-09-21T07:10:28.296 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 165, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 1301, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 1136, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 121428, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 322121728 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-09-21T07:10:28.297 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.298 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.299 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.300 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.301 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.302 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T07:10:28.303 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.304 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.305 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.306 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.307 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.308 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.309 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 4, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2498, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.310 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:28.311 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:28.312 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 194536, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 170, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:28.313 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9648, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 196, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 448671, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 4541, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:28.314 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3511, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 26, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 4, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3511, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3511, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 60.630073137, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.017268605 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.315 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3511, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3512 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3511, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 3511, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 2142, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1366, 2024-09-21T07:10:28.316 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 2, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:28.317 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T07:10:28.318 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 107, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 20, 2024-09-21T07:10:28.319 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 5, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 6, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 1, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T07:10:28.320 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:28.321 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3901, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 14382805, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3901, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3901, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 14382805, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.322 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3511, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 12640912, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3511, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12640912, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.323 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3511, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3511, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3511, 2024-09-21T07:10:28.324 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3511, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x564be50a3450": { 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:28.326 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.327 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:28.328 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T07:10:28.328 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-21T07:10:28.328 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:28.328 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T07:10:29.057 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T07:10:29.057 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T07:10:29.057 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-21T07:10:29.057 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:29.057 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:29.057 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T07:10:29.058 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T07:10:29.305 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:10:29.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:29 smithi193 ceph-mon[34601]: pgmap v904: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T07:10:29.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:29 smithi114 ceph-mon[30654]: pgmap v904: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T07:10:29.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:29 smithi183 ceph-mon[34219]: pgmap v904: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T07:10:29.867 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T07:10:29.867 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:29.867 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3003, 2024-09-21T07:10:29.867 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2980, 2024-09-21T07:10:29.867 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1603524, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 3585649, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 40, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.297987291, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.122821717, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.077446808, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.069032411, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2978, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.218687739, 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000073434 2024-09-21T07:10:29.868 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 284, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000053645, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000188 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 4189, 2024-09-21T07:10:29.869 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 5169, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1977907, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 3003500, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 18, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.378602227, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.204231168, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.174635847, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.021139461, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5167, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.487828863, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000094412 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.870 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3955, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001340331, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000338 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 506464 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3089, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2685, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 2764043, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1495400, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-09-21T07:10:29.871 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 7, 2024-09-21T07:10:29.872 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.299627142, 2024-09-21T07:10:29.873 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.138641358, 2024-09-21T07:10:29.873 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.160820024, 2024-09-21T07:10:29.873 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.022555365, 2024-09-21T07:10:29.873 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2685, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.168525250, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000062765 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3356, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000756891, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000225 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1165136, 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 230832 2024-09-21T07:10:29.874 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2791, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.740684729, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000981972 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:29.875 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 18, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000652314, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000036239 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 838, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 831, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 831, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 29.614444725, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.035637117 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T07:10:29.876 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 7, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 27, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 2, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 492, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 83, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 409, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 373, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 268, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 268, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 24, 2024-09-21T07:10:29.877 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 6719, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 6676, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 7, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 6, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 1665, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 7, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 276, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:29.878 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 3607, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 1017, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 1486476457, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 7, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 3607, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 3607, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 1050, 2024-09-21T07:10:29.879 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 957, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 1526, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 377, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 14, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 25 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 8, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 9, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-21T07:10:29.880 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 3, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 3, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 15, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 4054, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T07:10:29.881 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 2, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 1698, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 1698, 2024-09-21T07:10:29.882 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 3, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 3, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 6242035, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 683, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 6.544739720, 2024-09-21T07:10:29.883 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.009582342 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 494, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 497, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 3, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 41, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 44, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 3, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 492, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 507, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 15, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 268, 2024-09-21T07:10:29.884 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 508, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 240, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 128604, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 334385152 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 890, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 838, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 25, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 3, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.003370713, 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.003370713 2024-09-21T07:10:29.885 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 494, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 8.975766726, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.018169568 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.886 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 218, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 12.680050154, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.058165367 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.887 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.888 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T07:10:29.889 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 94, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 7.942687150, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.084496671 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 21, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.009928041, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000472763 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.891 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.892 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.893 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.894 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.002641941, 2024-09-21T07:10:29.895 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000880647 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 0, 2024-09-21T07:10:29.896 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 68, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 34, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 1711, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:29.897 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:29.898 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:29.899 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 322142, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 391, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9648, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 196, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:29.900 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 1641935, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 17904, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 4492, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 2243363, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 28, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 4492, 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T07:10:29.901 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4492, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 30.369535382, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.006760804 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4492, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4611 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 4492, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 3608, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 884, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-09-21T07:10:29.902 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 684, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 61, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 20, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 3607, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:29.903 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T07:10:29.904 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 107, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 20, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T07:10:29.905 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 173, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 35 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 6, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 3, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 6, 2024-09-21T07:10:29.906 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 3, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 10277, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 5433385, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 10277, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 10277, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 5433385, 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:29.907 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 4492, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2243810, 2024-09-21T07:10:29.908 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 4492, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2243810, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:29.909 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 4492, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 4492, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 4492, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 4492, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:29.910 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 447, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 447, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5577341211e0": { 2024-09-21T07:10:29.911 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1698, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 2047731, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1698, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 683, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2047731, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:29.912 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.913 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:29.913 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T07:10:29.913 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 4321 2024-09-21T07:10:29.913 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:29.913 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T07:10:30.683 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T07:10:30.683 INFO:tasks.check_counter:Daemon mds.i mds.imported=7 2024-09-21T07:10:30.683 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-21T07:10:30.683 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:30.683 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:30.683 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T07:10:30.683 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T07:10:30.930 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T07:10:31.334 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:31 smithi193 ceph-mon[34601]: pgmap v905: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-21T07:10:31.485 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1489, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1448, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 1503552, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 554852, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 5, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 1, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.207568688, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.089761385, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.077739392, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.041050890, 2024-09-21T07:10:31.486 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1448, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.101419232, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000070040 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 1135968, 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 206960 2024-09-21T07:10:31.487 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1124, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1372, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 4474776, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 951696, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.146257700, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.067470240, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.028907088, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.038749607, 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:31.488 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1370, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.066244532, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000048353 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 594832 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T07:10:31.489 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1326, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1327, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 8340875, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 421351, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 4, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.148279422, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.060199626, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.037710897, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.039534635, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1325, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.078350412, 2024-09-21T07:10:31.490 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000059132 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-09-21T07:10:31.491 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5996, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.176346876, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000029410 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.492 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-09-21T07:10:31.493 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 687, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 175, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 512, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 437, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 54, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-09-21T07:10:31.494 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-09-21T07:10:31.495 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-09-21T07:10:31.496 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.497 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-09-21T07:10:31.498 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_started": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_succeeded": 0, 2024-09-21T07:10:31.499 INFO:teuthology.orchestra.run.smithi193.stdout: "uninline_write_failed": 0 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 1714, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 4, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-09-21T07:10:31.500 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 16372988, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 16369216, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 1714 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 654, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 824, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 170, 2024-09-21T07:10:31.501 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 99, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 105, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 6, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 687, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 851, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 164, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 121636, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 320618496 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-09-21T07:10:31.502 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.503 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.504 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.505 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-09-21T07:10:31.506 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.507 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:31 smithi183 ceph-mon[34219]: pgmap v905: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-21T07:10:31.507 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.507 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.507 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-09-21T07:10:31.507 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-09-21T07:10:31.508 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.509 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.510 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.511 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.512 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.513 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 2, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-09-21T07:10:31.514 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2497, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-09-21T07:10:31.515 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T07:10:31.516 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-09-21T07:10:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 227575, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 830, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 9648, 2024-09-21T07:10:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 196, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 2266339, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 24454, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-09-21T07:10:31.519 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3541, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3541, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3541, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 40.515383462, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.011441791 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3541, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3542 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3541, 2024-09-21T07:10:31.520 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 3541, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 2112, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 1427, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-09-21T07:10:31.521 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-09-21T07:10:31.522 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-09-21T07:10:31.523 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 107, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 20, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 6, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 8, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 2, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-09-21T07:10:31.524 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T07:10:31.525 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 3935, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 13942137, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 3935, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3935, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 13942137, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.526 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3541, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 12194215, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3541, 2024-09-21T07:10:31.527 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 12194215, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:31.528 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3541, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3541, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3541, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3541, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:31.529 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.530 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x559bd8ecf380": { 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-09-21T07:10:31.531 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-09-21T07:10:31.532 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-21T07:10:31.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:31 smithi114 ceph-mon[30654]: pgmap v905: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 12 op/s 2024-09-21T07:10:32.204 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T07:10:32.333 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T07:10:32.334 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-21T07:10:32.334 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-21T07:10:32.334 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T07:10:32.334 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T07:10:32.357 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T07:10:32.357 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T07:10:32.357 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.412 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.413 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:32 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.416 DEBUG:teuthology.orchestra.run.smithi114:> set -ex 2024-09-21T07:10:32.416 DEBUG:teuthology.orchestra.run.smithi114:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:10:32.494 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T07:10:32.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:32 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:32 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T07:10:32.905 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: pgmap v906: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: Unable to set osd_memory_target on smithi193 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:10:33.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:33 smithi114 ceph-mon[30654]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.572+0000 7f4c0ce00640 1 -- 172.21.15.114:0/2721956402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 msgr2=0x7f4c08112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.572+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/2721956402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c08112840 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009ca0 tx=0x7f4c0002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 -- 172.21.15.114:0/2721956402 shutdown_connections 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/2721956402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c08112840 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/2721956402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c0810b6c0 0x7f4c0810bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/2721956402 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c08105d80 0x7f4c08106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 -- 172.21.15.114:0/2721956402 >> 172.21.15.114:0/2721956402 conn(0x7f4c081017f0 msgr2=0x7f4c08103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 -- 172.21.15.114:0/2721956402 shutdown_connections 2024-09-21T07:10:33.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.573+0000 7f4c0ce00640 1 -- 172.21.15.114:0/2721956402 wait complete. 2024-09-21T07:10:33.576 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 Processor -- start 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 -- start start 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c08105d80 0x7f4c0819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c0810b6c0 0x7f4c0819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c081a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08115d20 con 0x7f4c0810b6c0 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08115ba0 con 0x7f4c0810c010 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4c0ce00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08115ea0 con 0x7f4c08105d80 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.574+0000 7f4bfd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c0810b6c0 0x7f4c0819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4bfd400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c0810b6c0 0x7f4c0819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:39864/0 (socket says 172.21.15.114:39864) 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4bfd400640 1 -- 172.21.15.114:0/1241224434 learned_addr learned my addr 172.21.15.114:0/1241224434 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c081a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c05e00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c08105d80 0x7f4c0819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 -- 172.21.15.114:0/1241224434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c08105d80 msgr2=0x7f4c0819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.577 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c08105d80 0x7f4c0819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 -- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c0810b6c0 msgr2=0x7f4c0819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c0810b6c0 0x7f4c0819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 -- 172.21.15.114:0/1241224434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c081a25a0 con 0x7f4c0810c010 2024-09-21T07:10:33.578 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.575+0000 7f4c06800640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c081a1e70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4c00005e30 tx=0x7f4c00030dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.576+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00003e00 con 0x7f4c0810c010 2024-09-21T07:10:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.576+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00040070 con 0x7f4c0810c010 2024-09-21T07:10:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.576+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c081a2830 con 0x7f4c0810c010 2024-09-21T07:10:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.576+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00045a20 con 0x7f4c0810c010 2024-09-21T07:10:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.576+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a8ff0 con 0x7f4c0810c010 2024-09-21T07:10:33.579 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.577+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08106190 con 0x7f4c0810c010 2024-09-21T07:10:33.581 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.579+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4c00045bc0 con 0x7f4c0810c010 2024-09-21T07:10:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.580+0000 7f4bff400640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4bec0b6700 0x7f4bec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.581+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9490+0+0 (secure 0 0 0) 0x7f4c00108980 con 0x7f4c0810c010 2024-09-21T07:10:33.582 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.581+0000 7f4c05e00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4bec0b6700 0x7f4bec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:33.583 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.581+0000 7f4c05e00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4bec0b6700 0x7f4bec0b8c10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009bd0 tx=0x7f4bf8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:33.586 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.585+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c000ceaa0 con 0x7f4c0810c010 2024-09-21T07:10:33.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-21T07:10:33.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:10:33.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: pgmap v906: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-09-21T07:10:33.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T07:10:33.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: Unable to set osd_memory_target on smithi193 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-21T07:10:33.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:10:33.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:10:33.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:33 smithi183 ceph-mon[34219]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:10:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-21T07:10:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T07:10:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: pgmap v906: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.6 KiB/s wr, 12 op/s 2024-09-21T07:10:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-09-21T07:10:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: Unable to set osd_memory_target on smithi193 to 184593203: error parsing value: Value '184593203' is below minimum 939524096 2024-09-21T07:10:33.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T07:10:33.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T07:10:33.663 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:33 smithi193 ceph-mon[34601]: from='mgr.14223 ' entity='mgr.x' 2024-09-21T07:10:33.787 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.786+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4c0810de00 con 0x7f4c0810c010 2024-09-21T07:10:33.788 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.786+0000 7f4bff400640 1 -- 172.21.15.114:0/1241224434 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v107) ==== 81+0+890 (secure 0 0 0) 0x7f4c000d2e80 con 0x7f4c0810c010 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2966269937 2024-09-22T06:43:29.387989+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3575030312 2024-09-22T06:41:54.185935+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/1583737438 2024-09-22T06:42:25.331257+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/3364907862 2024-09-22T06:41:54.185935+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/150287858 2024-09-22T06:41:54.185935+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/150287858 2024-09-22T06:41:54.185935+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/830030899 2024-09-22T06:42:25.331257+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/830030899 2024-09-22T06:42:25.331257+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/673737563 2024-09-22T06:42:25.331257+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/791492062 2024-09-22T06:41:54.185935+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2622463004 2024-09-22T06:43:29.387989+0000 2024-09-21T07:10:33.789 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/2480842130 2024-09-22T06:42:25.331257+0000 2024-09-21T07:10:33.790 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:0/200663390 2024-09-22T06:43:29.387989+0000 2024-09-21T07:10:33.790 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6800/8263225 2024-09-22T06:43:29.387989+0000 2024-09-21T07:10:33.790 INFO:teuthology.orchestra.run.smithi114.stdout:172.21.15.114:6801/8263225 2024-09-22T06:43:29.387989+0000 2024-09-21T07:10:33.790 INFO:teuthology.orchestra.run.smithi114.stderr:listed 15 entries 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4bec0b6700 msgr2=0x7f4bec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4bec0b6700 0x7f4bec0b8c10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8009bd0 tx=0x7f4bf8005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 msgr2=0x7f4c081a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c081a1e70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4c00005e30 tx=0x7f4c00030dc0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 shutdown_connections 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f4bec0b6700 0x7f4bec0b8c10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c0810c010 0x7f4c081a1e70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4c0810b6c0 0x7f4c0819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 --2- 172.21.15.114:0/1241224434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c08105d80 0x7f4c0819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:33.791 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 >> 172.21.15.114:0/1241224434 conn(0x7f4c081017f0 msgr2=0x7f4c08108c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:33.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 shutdown_connections 2024-09-21T07:10:33.792 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:33.789+0000 7f4c0ce00640 1 -- 172.21.15.114:0/1241224434 wait complete. 2024-09-21T07:10:34.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:34 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1241224434' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:34.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:34 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1241224434' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:34.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:34 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1241224434' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T07:10:34.743 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:10:34.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:34.743 DEBUG:teuthology.orchestra.run.smithi114:> sudo umount /tmp/cephfs 2024-09-21T07:10:35.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T07:10:35.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:35.046 DEBUG:teuthology.orchestra.run.smithi114:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-21T07:10:35.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:35.138 DEBUG:teuthology.orchestra.run.smithi114:> ip netns list 2024-09-21T07:10:35.228 INFO:teuthology.orchestra.run.smithi114.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-21T07:10:35.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:35.229 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-21T07:10:35.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:10:35.287 DEBUG:teuthology.orchestra.run.smithi114:> sudo ip link delete ceph-brx 2024-09-21T07:10:35.315 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:35 smithi114 ceph-mon[30654]: pgmap v907: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-21T07:10:35.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:35 smithi183 ceph-mon[34219]: pgmap v907: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-21T07:10:35.638 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T07:10:35.650 INFO:tasks.cephadm:Teardown begin 2024-09-21T07:10:35.650 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:10:35.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:10:35.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:35 smithi193 ceph-mon[34601]: pgmap v907: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-21T07:10:36.054 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:36.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 -- 172.21.15.114:0/2604270026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 msgr2=0x7fe98c103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:36.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 --2- 172.21.15.114:0/2604270026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c103fb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe980009900 tx=0x7fe98002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.671 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 -- 172.21.15.114:0/2604270026 shutdown_connections 2024-09-21T07:10:36.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 --2- 172.21.15.114:0/2604270026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe98c109e30 0x7fe98c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 --2- 172.21.15.114:0/2604270026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe98c1094e0 0x7fe98c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 --2- 172.21.15.114:0/2604270026 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c103fb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 -- 172.21.15.114:0/2604270026 >> 172.21.15.114:0/2604270026 conn(0x7fe98c0ff5f0 msgr2=0x7fe98c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:36.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 -- 172.21.15.114:0/2604270026 shutdown_connections 2024-09-21T07:10:36.672 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.669+0000 7fe994000640 1 -- 172.21.15.114:0/2604270026 wait complete. 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.670+0000 7fe994000640 1 Processor -- start 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.670+0000 7fe994000640 1 -- start start 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe994000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe994000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe98c1094e0 0x7fe98c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe994000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe98c109e30 0x7fe98c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe994000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113b40 con 0x7fe98c103ba0 2024-09-21T07:10:36.674 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe994000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c1139c0 con 0x7fe98c1094e0 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe994000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe98c113cc0 con 0x7fe98c109e30 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48390/0 (socket says 172.21.15.114:48390) 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 -- 172.21.15.114:0/3794472655 learned_addr learned my addr 172.21.15.114:0/3794472655 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe992000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe98c109e30 0x7fe98c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe990c00640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe98c1094e0 0x7fe98c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 -- 172.21.15.114:0/3794472655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe98c109e30 msgr2=0x7fe98c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe98c109e30 0x7fe98c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 -- 172.21.15.114:0/3794472655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe98c1094e0 msgr2=0x7fe98c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe98c1094e0 0x7fe98c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.671+0000 7fe991600640 1 -- 172.21.15.114:0/3794472655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98c1a87b0 con 0x7fe98c103ba0 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.672+0000 7fe991600640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c075710 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe980005ab0 tx=0x7fe980003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:36.675 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.672+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98003f070 con 0x7fe98c103ba0 2024-09-21T07:10:36.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.672+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe980031d60 con 0x7fe98c103ba0 2024-09-21T07:10:36.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.672+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98c1a8a40 con 0x7fe98c103ba0 2024-09-21T07:10:36.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.672+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980035030 con 0x7fe98c103ba0 2024-09-21T07:10:36.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.672+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98c1a8f20 con 0x7fe98c103ba0 2024-09-21T07:10:36.676 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.673+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98c103fb0 con 0x7fe98c103ba0 2024-09-21T07:10:36.677 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.676+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fe980042070 con 0x7fe98c103ba0 2024-09-21T07:10:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.677+0000 7fe98a000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9740b6680 0x7fe9740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.677+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9490+0+0 (secure 0 0 0) 0x7fe980108410 con 0x7fe98c103ba0 2024-09-21T07:10:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.677+0000 7fe990c00640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9740b6680 0x7fe9740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:36.679 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.678+0000 7fe990c00640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9740b6680 0x7fe9740b8b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe98c108ff0 tx=0x7fe97800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:36.683 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.681+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9800d2910 con 0x7fe98c103ba0 2024-09-21T07:10:36.909 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.908+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe98c02e640 con 0x7fe98c103ba0 2024-09-21T07:10:36.910 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.909+0000 7fe98a000640 1 -- 172.21.15.114:0/3794472655 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11990 (secure 0 0 0) 0x7fe9800d7850 con 0x7fe98c103ba0 2024-09-21T07:10:36.911 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:10:36.911 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":61,"btime":"2024-09-21T07:09:54:234933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:09:53.402763+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:10:36.911 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 61 2024-09-21T07:10:36.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9740b6680 msgr2=0x7fe9740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:36.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9740b6680 0x7fe9740b8b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe98c108ff0 tx=0x7fe97800b390 comp rx=0 tx=0).stop 2024-09-21T07:10:36.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 msgr2=0x7fe98c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:36.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c075710 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe980005ab0 tx=0x7fe980003cd0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 shutdown_connections 2024-09-21T07:10:36.913 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe98c109e30 0x7fe98c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fe9740b6680 0x7fe9740b8b90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe98c1094e0 0x7fe98c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 --2- 172.21.15.114:0/3794472655 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fe98c103ba0 0x7fe98c075710 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:36.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.911+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 >> 172.21.15.114:0/3794472655 conn(0x7fe98c0ff5f0 msgr2=0x7fe98c101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:36.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.912+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 shutdown_connections 2024-09-21T07:10:36.914 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:36.912+0000 7fe994000640 1 -- 172.21.15.114:0/3794472655 wait complete. 2024-09-21T07:10:37.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:37 smithi114 ceph-mon[30654]: pgmap v908: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-21T07:10:37.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:37 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3794472655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:37.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:37 smithi183 ceph-mon[34219]: pgmap v908: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-21T07:10:37.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:37 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3794472655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:37 smithi193 ceph-mon[34601]: pgmap v908: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-09-21T07:10:37.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:37 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3794472655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:37.799 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T07:10:38.170 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 -- 172.21.15.114:0/87715237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c06c0e0 msgr2=0x7fd48c06c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 --2- 172.21.15.114:0/87715237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c06c0e0 0x7fd48c06c4f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd478009900 tx=0x7fd47802a410 comp rx=0 tx=0).stop 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 -- 172.21.15.114:0/87715237 shutdown_connections 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 --2- 172.21.15.114:0/87715237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c110040 0x7fd48c112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 --2- 172.21.15.114:0/87715237 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c06c0e0 0x7fd48c06c4f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 --2- 172.21.15.114:0/87715237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c1046a0 0x7fd48c06bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 -- 172.21.15.114:0/87715237 >> 172.21.15.114:0/87715237 conn(0x7fd48c0799a0 msgr2=0x7fd48c079db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:38.795 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 -- 172.21.15.114:0/87715237 shutdown_connections 2024-09-21T07:10:38.796 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.793+0000 7fd493c00640 1 -- 172.21.15.114:0/87715237 wait complete. 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 Processor -- start 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 -- start start 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c06c0e0 0x7fd48c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 0x7fd48c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c110040 0x7fd48c1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c115ce0 con 0x7fd48c110040 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c115b60 con 0x7fd48c1046a0 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.794+0000 7fd493c00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c115e60 con 0x7fd48c06c0e0 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 0x7fd48c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd491c00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c110040 0x7fd48c1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 0x7fd48c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:48794/0 (socket says 172.21.15.114:48794) 2024-09-21T07:10:38.797 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd491200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c06c0e0 0x7fd48c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 -- 172.21.15.114:0/245566487 learned_addr learned my addr 172.21.15.114:0/245566487 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 -- 172.21.15.114:0/245566487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c06c0e0 msgr2=0x7fd48c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c06c0e0 0x7fd48c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 -- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c110040 msgr2=0x7fd48c1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c110040 0x7fd48c1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd48be00640 1 -- 172.21.15.114:0/245566487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1a2600 con 0x7fd48c1046a0 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd491200640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c06c0e0 0x7fd48c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:38.798 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.795+0000 7fd491c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c110040 0x7fd48c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.796+0000 7fd48be00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 0x7fd48c19cc30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd47802a930 tx=0x7fd478031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.796+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47803f030 con 0x7fd48c1046a0 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.796+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1a2890 con 0x7fd48c1046a0 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.796+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd478004490 con 0x7fd48c1046a0 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.796+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478035070 con 0x7fd48c1046a0 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.796+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a9050 con 0x7fd48c1046a0 2024-09-21T07:10:38.799 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.797+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c06c4f0 con 0x7fd48c1046a0 2024-09-21T07:10:38.801 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.800+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fd478042030 con 0x7fd48c1046a0 2024-09-21T07:10:38.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.801+0000 7fd489600640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd4740b6680 0x7fd4740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:38.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.801+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9490+0+0 (secure 0 0 0) 0x7fd478108220 con 0x7fd48c1046a0 2024-09-21T07:10:38.803 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.802+0000 7fd491200640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd4740b6680 0x7fd4740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:38.804 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.802+0000 7fd491200640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd4740b6680 0x7fd4740b8b90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd480005fd0 tx=0x7fd480009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:38.805 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:38.803+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4780d2720 con 0x7fd48c1046a0 2024-09-21T07:10:39.028 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.026+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd48c19d7c0 con 0x7fd48c1046a0 2024-09-21T07:10:39.029 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.027+0000 7fd489600640 1 -- 172.21.15.114:0/245566487 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11990 (secure 0 0 0) 0x7fd4780d7660 con 0x7fd48c1046a0 2024-09-21T07:10:39.030 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:10:39.030 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":61,"btime":"2024-09-21T07:09:54:234933+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14685,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6835/3420506424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6834","nonce":3420506424},{"type":"v1","addr":"172.21.15.114:6835","nonce":3420506424}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14697,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.114:6837/1280617419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6836","nonce":1280617419},{"type":"v1","addr":"172.21.15.114:6837","nonce":1280617419}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T06:50:48.407210+0000","modified":"2024-09-21T07:09:53.402763+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24761,"mds_1":24760,"mds_2":24742,"mds_3":24727,"mds_4":24725},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14742":{"gid":14742,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.114:6839/244481807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6838","nonce":244481807},{"type":"v1","addr":"172.21.15.114:6839","nonce":244481807}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24667":{"gid":24667,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6833/2086656361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":2086656361},{"type":"v1","addr":"172.21.15.193:6833","nonce":2086656361}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24668":{"gid":24668,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6833/919969847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":919969847},{"type":"v1","addr":"172.21.15.183:6833","nonce":919969847}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2729823804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2729823804},{"type":"v1","addr":"172.21.15.183:6835","nonce":2729823804}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24715":{"gid":24715,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2057077471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2057077471},{"type":"v1","addr":"172.21.15.193:6835","nonce":2057077471}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24725":{"gid":24725,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6837/841299119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":841299119},{"type":"v1","addr":"172.21.15.183:6837","nonce":841299119}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24727":{"gid":24727,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":22,"addr":"172.21.15.193:6837/1013898612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1013898612},{"type":"v1","addr":"172.21.15.193:6837","nonce":1013898612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24742":{"gid":24742,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":20,"addr":"172.21.15.193:6839/3226153327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3226153327},{"type":"v1","addr":"172.21.15.193:6839","nonce":3226153327}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24760":{"gid":24760,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":18,"addr":"172.21.15.114:6841/2055898271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6840","nonce":2055898271},{"type":"v1","addr":"172.21.15.114:6841","nonce":2055898271}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24761":{"gid":24761,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.183:6839/42004963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":42004963},{"type":"v1","addr":"172.21.15.183:6839","nonce":42004963}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24761,"qdb_cluster":[24761,24760,24742,24727,24725]},"id":1}]} 2024-09-21T07:10:39.030 INFO:teuthology.orchestra.run.smithi114.stderr:dumped fsmap epoch 61 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd4740b6680 msgr2=0x7fd4740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd4740b6680 0x7fd4740b8b90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd480005fd0 tx=0x7fd480009500 comp rx=0 tx=0).stop 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 msgr2=0x7fd48c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 0x7fd48c19cc30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd47802a930 tx=0x7fd478031a20 comp rx=0 tx=0).stop 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 shutdown_connections 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fd48c110040 0x7fd48c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:39.032 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fd4740b6680 0x7fd4740b8b90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:39.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd48c1046a0 0x7fd48c19cc30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:39.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 --2- 172.21.15.114:0/245566487 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd48c06c0e0 0x7fd48c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:39.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 >> 172.21.15.114:0/245566487 conn(0x7fd48c0799a0 msgr2=0x7fd48c1014f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:39.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 shutdown_connections 2024-09-21T07:10:39.033 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:39.030+0000 7fd493c00640 1 -- 172.21.15.114:0/245566487 wait complete. 2024-09-21T07:10:39.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:39 smithi114 ceph-mon[30654]: pgmap v909: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:10:39.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:39 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/245566487' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:39.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:39 smithi183 ceph-mon[34219]: pgmap v909: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:10:39.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:39 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/245566487' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:39.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:39 smithi193 ceph-mon[34601]: pgmap v909: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-09-21T07:10:39.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:39 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/245566487' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T07:10:39.833 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T07:10:40.204 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:40.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 -- 172.21.15.114:0/2045831511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 msgr2=0x7fbc0010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:40.856 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 --2- 172.21.15.114:0/2045831511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0010d660 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009900 tx=0x7fbbec02a410 comp rx=0 tx=0).stop 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 -- 172.21.15.114:0/2045831511 shutdown_connections 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 --2- 172.21.15.114:0/2045831511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc0010dc20 0x7fbc00110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 --2- 172.21.15.114:0/2045831511 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0010d660 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 --2- 172.21.15.114:0/2045831511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc001047c0 0x7fbc00104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 -- 172.21.15.114:0/2045831511 >> 172.21.15.114:0/2045831511 conn(0x7fbc000fe2c0 msgr2=0x7fbc00100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 -- 172.21.15.114:0/2045831511 shutdown_connections 2024-09-21T07:10:40.857 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.854+0000 7fbc05200640 1 -- 172.21.15.114:0/2045831511 wait complete. 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.855+0000 7fbc05200640 1 Processor -- start 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 -- start start 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc001047c0 0x7fbc0019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0010dc20 0x7fbc001a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00113880 con 0x7fbc00105110 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00113700 con 0x7fbc001047c0 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbc05200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00113a00 con 0x7fbc0010dc20 2024-09-21T07:10:40.858 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbbfea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:40.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbbfea00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48424/0 (socket says 172.21.15.114:48424) 2024-09-21T07:10:40.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbbfea00640 1 -- 172.21.15.114:0/2433879388 learned_addr learned my addr 172.21.15.114:0/2433879388 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:40.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbbff400640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc001047c0 0x7fbc0019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:40.859 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.856+0000 7fbbffe00640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0010dc20 0x7fbc001a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbfea00640 1 -- 172.21.15.114:0/2433879388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0010dc20 msgr2=0x7fbc001a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbfea00640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0010dc20 0x7fbc001a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbfea00640 1 -- 172.21.15.114:0/2433879388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc001047c0 msgr2=0x7fbc0019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbfea00640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc001047c0 0x7fbc0019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbfea00640 1 -- 172.21.15.114:0/2433879388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc001a2500 con 0x7fbc00105110 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbff400640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc001047c0 0x7fbc0019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbffe00640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0010dc20 0x7fbc001a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.857+0000 7fbbfea00640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0019cb30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbec02a930 tx=0x7fbbec031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.858+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec03f030 con 0x7fbc00105110 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.858+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc001a2790 con 0x7fbc00105110 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.858+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbec004490 con 0x7fbc00105110 2024-09-21T07:10:40.860 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.858+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec035070 con 0x7fbc00105110 2024-09-21T07:10:40.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.858+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc001a8f50 con 0x7fbc00105110 2024-09-21T07:10:40.861 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.859+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0006b560 con 0x7fbc00105110 2024-09-21T07:10:40.863 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.861+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7fbbec042030 con 0x7fbc00105110 2024-09-21T07:10:40.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.863+0000 7fbbf7e00640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbbe40b6680 0x7fbbe40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:40.864 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.863+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9490+0+0 (secure 0 0 0) 0x7fbbec108220 con 0x7fbc00105110 2024-09-21T07:10:40.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.863+0000 7fbbff400640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbbe40b6680 0x7fbbe40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:40.865 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.864+0000 7fbbff400640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbbe40b6680 0x7fbbe40b8b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009fc0 tx=0x7fbbf0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:40.867 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:40.865+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbbec0d2720 con 0x7fbc00105110 2024-09-21T07:10:41.058 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.056+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc001a9120 con 0x7fbc00105110 2024-09-21T07:10:41.060 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.058+0000 7fbbf7e00640 1 -- 172.21.15.114:0/2433879388 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+25465 (secure 0 0 0) 0x7fbbec0d7660 con 0x7fbc00105110 2024-09-21T07:10:41.061 INFO:teuthology.orchestra.run.smithi114.stdout: 2024-09-21T07:10:41.061 INFO:teuthology.orchestra.run.smithi114.stdout:{"epoch":107,"fsid":"5d75913a-77e4-11ef-baf5-efdc52797490","created":"2024-09-21T06:41:28.662923+0000","modified":"2024-09-21T07:10:20.025462+0000","last_up_change":"2024-09-21T06:49:51.182641+0000","last_in_change":"2024-09-21T06:49:27.225875+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T06:45:37.711233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T06:50:42.695843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T06:50:45.269171+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T06:50:54.559397+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89d9168b-a941-4432-a508-6ab83fc6148c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6802","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6803","nonce":4089006446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6804","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6805","nonce":4089006446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6808","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6809","nonce":4089006446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6806","nonce":4089006446},{"type":"v1","addr":"172.21.15.114:6807","nonce":4089006446}]},"public_addr":"172.21.15.114:6803/4089006446","cluster_addr":"172.21.15.114:6805/4089006446","heartbeat_back_addr":"172.21.15.114:6809/4089006446","heartbeat_front_addr":"172.21.15.114:6807/4089006446","state":["exists","up"]},{"osd":1,"uuid":"a9bba525-a4a1-4988-8288-4a9023f07724","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6801","nonce":427115039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6803","nonce":427115039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6807","nonce":427115039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":427115039},{"type":"v1","addr":"172.21.15.183:6805","nonce":427115039}]},"public_addr":"172.21.15.183:6801/427115039","cluster_addr":"172.21.15.183:6803/427115039","heartbeat_back_addr":"172.21.15.183:6807/427115039","heartbeat_front_addr":"172.21.15.183:6805/427115039","state":["exists","up"]},{"osd":2,"uuid":"8bc9357c-f3a5-4370-954a-e1654bab5e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6801","nonce":1721723108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6803","nonce":1721723108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6807","nonce":1721723108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1721723108},{"type":"v1","addr":"172.21.15.193:6805","nonce":1721723108}]},"public_addr":"172.21.15.193:6801/1721723108","cluster_addr":"172.21.15.193:6803/1721723108","heartbeat_back_addr":"172.21.15.193:6807/1721723108","heartbeat_front_addr":"172.21.15.193:6805/1721723108","state":["exists","up"]},{"osd":3,"uuid":"36ee2356-5e1f-491f-bdf5-d9bd22956581","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6810","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6811","nonce":2440023671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6812","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6813","nonce":2440023671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6816","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6817","nonce":2440023671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6814","nonce":2440023671},{"type":"v1","addr":"172.21.15.114:6815","nonce":2440023671}]},"public_addr":"172.21.15.114:6811/2440023671","cluster_addr":"172.21.15.114:6813/2440023671","heartbeat_back_addr":"172.21.15.114:6817/2440023671","heartbeat_front_addr":"172.21.15.114:6815/2440023671","state":["exists","up"]},{"osd":4,"uuid":"b7db874e-11e1-44a2-9f0f-3a1f8738356a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6809","nonce":204187293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6811","nonce":204187293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6815","nonce":204187293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":204187293},{"type":"v1","addr":"172.21.15.183:6813","nonce":204187293}]},"public_addr":"172.21.15.183:6809/204187293","cluster_addr":"172.21.15.183:6811/204187293","heartbeat_back_addr":"172.21.15.183:6815/204187293","heartbeat_front_addr":"172.21.15.183:6813/204187293","state":["exists","up"]},{"osd":5,"uuid":"0798fc1c-9adf-42ec-b897-385b1be1bd71","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6809","nonce":3650564948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6811","nonce":3650564948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6815","nonce":3650564948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3650564948},{"type":"v1","addr":"172.21.15.193:6813","nonce":3650564948}]},"public_addr":"172.21.15.193:6809/3650564948","cluster_addr":"172.21.15.193:6811/3650564948","heartbeat_back_addr":"172.21.15.193:6815/3650564948","heartbeat_front_addr":"172.21.15.193:6813/3650564948","state":["exists","up"]},{"osd":6,"uuid":"6d41daa7-6623-4431-9bbb-fe54c1329dc9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6818","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6819","nonce":1058136039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6820","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6821","nonce":1058136039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6824","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6825","nonce":1058136039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6822","nonce":1058136039},{"type":"v1","addr":"172.21.15.114:6823","nonce":1058136039}]},"public_addr":"172.21.15.114:6819/1058136039","cluster_addr":"172.21.15.114:6821/1058136039","heartbeat_back_addr":"172.21.15.114:6825/1058136039","heartbeat_front_addr":"172.21.15.114:6823/1058136039","state":["exists","up"]},{"osd":7,"uuid":"88b7c23c-2407-4f8e-895e-5349e4fb3083","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6817","nonce":3203682724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6819","nonce":3203682724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6823","nonce":3203682724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3203682724},{"type":"v1","addr":"172.21.15.183:6821","nonce":3203682724}]},"public_addr":"172.21.15.183:6817/3203682724","cluster_addr":"172.21.15.183:6819/3203682724","heartbeat_back_addr":"172.21.15.183:6823/3203682724","heartbeat_front_addr":"172.21.15.183:6821/3203682724","state":["exists","up"]},{"osd":8,"uuid":"ca4001c9-2aa5-4258-9263-2ed19f7dde20","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6817","nonce":2425758688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6819","nonce":2425758688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6823","nonce":2425758688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2425758688},{"type":"v1","addr":"172.21.15.193:6821","nonce":2425758688}]},"public_addr":"172.21.15.193:6817/2425758688","cluster_addr":"172.21.15.193:6819/2425758688","heartbeat_back_addr":"172.21.15.193:6823/2425758688","heartbeat_front_addr":"172.21.15.193:6821/2425758688","state":["exists","up"]},{"osd":9,"uuid":"76a2c023-5c26-46da-9de8-7d42a9b048df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6826","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6827","nonce":903490275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6828","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6829","nonce":903490275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6832","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6833","nonce":903490275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.114:6830","nonce":903490275},{"type":"v1","addr":"172.21.15.114:6831","nonce":903490275}]},"public_addr":"172.21.15.114:6827/903490275","cluster_addr":"172.21.15.114:6829/903490275","heartbeat_back_addr":"172.21.15.114:6833/903490275","heartbeat_front_addr":"172.21.15.114:6831/903490275","state":["exists","up"]},{"osd":10,"uuid":"5cf450ee-20ef-4871-afbf-3f9dc1d0c45c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6825","nonce":4150013196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6827","nonce":4150013196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6831","nonce":4150013196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":4150013196},{"type":"v1","addr":"172.21.15.183:6829","nonce":4150013196}]},"public_addr":"172.21.15.183:6825/4150013196","cluster_addr":"172.21.15.183:6827/4150013196","heartbeat_back_addr":"172.21.15.183:6831/4150013196","heartbeat_front_addr":"172.21.15.183:6829/4150013196","state":["exists","up"]},{"osd":11,"uuid":"0bdaf146-f4e6-4675-834d-1d130fa491ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6825","nonce":3118065190}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6827","nonce":3118065190}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6831","nonce":3118065190}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":3118065190},{"type":"v1","addr":"172.21.15.193:6829","nonce":3118065190}]},"public_addr":"172.21.15.193:6825/3118065190","cluster_addr":"172.21.15.193:6827/3118065190","heartbeat_back_addr":"172.21.15.193:6831/3118065190","heartbeat_front_addr":"172.21.15.193:6829/3118065190","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:44:43.018214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:10.127579+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:45:35.244023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:05.931627+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:33.441072+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:46:59.072351+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:30.212925+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:47:57.503740+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:24.494063+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:48:55.142053+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:23.449854+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T06:49:48.939665+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]},{"pgid":"4.a","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.114:0/2966269937":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/3575030312":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/1583737438":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/3364907862":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6801/150287858":"2024-09-22T06:41:54.185935+0000","172.21.15.114:6800/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:6801/830030899":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/673737563":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/791492062":"2024-09-22T06:41:54.185935+0000","172.21.15.114:0/2622463004":"2024-09-22T06:43:29.387989+0000","172.21.15.114:0/2480842130":"2024-09-22T06:42:25.331257+0000","172.21.15.114:0/200663390":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6800/8263225":"2024-09-22T06:43:29.387989+0000","172.21.15.114:6801/8263225":"2024-09-22T06:43:29.387989+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":26,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:10:41.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbbe40b6680 msgr2=0x7fbbe40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbbe40b6680 0x7fbbe40b8b90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009fc0 tx=0x7fbbf0005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:41.063 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 msgr2=0x7fbc0019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0019cb30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbbec02a930 tx=0x7fbbec031a20 comp rx=0 tx=0).stop 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 shutdown_connections 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc0010dc20 0x7fbc001a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.061+0000 7fbc05200640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7fbc00105110 0x7fbc0019cb30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.062+0000 7fbc05200640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7fbbe40b6680 0x7fbbe40b8b90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.062+0000 7fbc05200640 1 --2- 172.21.15.114:0/2433879388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc001047c0 0x7fbc0019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.062+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 >> 172.21.15.114:0/2433879388 conn(0x7fbc000fe2c0 msgr2=0x7fbc001024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.062+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 shutdown_connections 2024-09-21T07:10:41.064 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:41.062+0000 7fbc05200640 1 -- 172.21.15.114:0/2433879388 wait complete. 2024-09-21T07:10:41.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:41 smithi114 ceph-mon[30654]: pgmap v910: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-21T07:10:41.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:41 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/2433879388' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:41.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:41 smithi183 ceph-mon[34219]: pgmap v910: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-21T07:10:41.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:41 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/2433879388' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:41.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:41 smithi193 ceph-mon[34601]: pgmap v910: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-09-21T07:10:41.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:41 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/2433879388' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T07:10:41.970 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T07:10:42.340 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:42.478 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:42 smithi114 ceph-mon[30654]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:10:42.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:42 smithi183 ceph-mon[34219]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:10:42.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:42 smithi193 ceph-mon[34601]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T07:10:43.017 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 -- 172.21.15.114:0/201022152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c9810b6c0 msgr2=0x7f9c9810bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 --2- 172.21.15.114:0/201022152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c9810b6c0 0x7f9c9810bad0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009900 tx=0x7f9c8002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 -- 172.21.15.114:0/201022152 shutdown_connections 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 --2- 172.21.15.114:0/201022152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c9810c010 0x7f9c98112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 --2- 172.21.15.114:0/201022152 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c9810b6c0 0x7f9c9810bad0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 --2- 172.21.15.114:0/201022152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c98105d80 0x7f9c98106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 -- 172.21.15.114:0/201022152 >> 172.21.15.114:0/201022152 conn(0x7f9c98101810 msgr2=0x7f9c98103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 -- 172.21.15.114:0/201022152 shutdown_connections 2024-09-21T07:10:43.018 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.015+0000 7f9c9e200640 1 -- 172.21.15.114:0/201022152 wait complete. 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.016+0000 7f9c9e200640 1 Processor -- start 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.016+0000 7f9c9e200640 1 -- start start 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c98105d80 0x7f9c9819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9810b6c0 0x7f9c9819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c9e200640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c9810c010 0x7f9c981a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c9e200640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98115cd0 con 0x7f9c98105d80 2024-09-21T07:10:43.019 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c9e200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98115b50 con 0x7f9c9810b6c0 2024-09-21T07:10:43.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c9e200640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98115e50 con 0x7f9c9810c010 2024-09-21T07:10:43.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c98105d80 0x7f9c9819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c8ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9810b6c0 0x7f9c9819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c97400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c98105d80 0x7f9c9819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:48446/0 (socket says 172.21.15.114:48446) 2024-09-21T07:10:43.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c97400640 1 -- 172.21.15.114:0/822670372 learned_addr learned my addr 172.21.15.114:0/822670372 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:43.020 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.017+0000 7f9c97e00640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c9810c010 0x7f9c981a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c8ea00640 1 -- 172.21.15.114:0/822670372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c9810c010 msgr2=0x7f9c981a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c8ea00640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c9810c010 0x7f9c981a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c8ea00640 1 -- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c98105d80 msgr2=0x7f9c9819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c8ea00640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c98105d80 0x7f9c9819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c8ea00640 1 -- 172.21.15.114:0/822670372 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c981a2630 con 0x7f9c9810b6c0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c8ea00640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9810b6c0 0x7f9c9819cc60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9c8002a910 tx=0x7f9c80031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.018+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8003f070 con 0x7f9c9810b6c0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.019+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c981a28c0 con 0x7f9c9810b6c0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.019+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c80004490 con 0x7f9c9810b6c0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.019+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80035070 con 0x7f9c9810b6c0 2024-09-21T07:10:43.021 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.019+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981a9080 con 0x7f9c9810b6c0 2024-09-21T07:10:43.022 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.020+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c98106190 con 0x7f9c9810b6c0 2024-09-21T07:10:43.024 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.022+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f9c8003b460 con 0x7f9c9810b6c0 2024-09-21T07:10:43.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.024+0000 7f9c94c00640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:43.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.025+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9490+0+0 (secure 0 0 0) 0x7f9c80108160 con 0x7f9c9810b6c0 2024-09-21T07:10:43.026 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.025+0000 7f9c97400640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:43.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.025+0000 7f9c97400640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9c880095f0 tx=0x7f9c88005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:43.027 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.026+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c800d2660 con 0x7f9c9810b6c0 2024-09-21T07:10:43.253 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.252+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9c9819d7f0 con 0x7f9c9810b6c0 2024-09-21T07:10:43.265 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.263+0000 7f9c94c00640 1 -- 172.21.15.114:0/822670372 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7f9c800d75a0 con 0x7f9c9810b6c0 2024-09-21T07:10:43.265 INFO:teuthology.orchestra.run.smithi114.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:10:43.268 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c7c0b6680 msgr2=0x7f9c7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9c880095f0 tx=0x7f9c88005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9810b6c0 msgr2=0x7f9c9819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9810b6c0 0x7f9c9819cc60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9c8002a910 tx=0x7f9c80031a20 comp rx=0 tx=0).stop 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 shutdown_connections 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c9810c010 0x7f9c981a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f9c7c0b6680 0x7f9c7c0b8b90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.269 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9810b6c0 0x7f9c9819cc60 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 --2- 172.21.15.114:0/822670372 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f9c98105d80 0x7f9c9819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 >> 172.21.15.114:0/822670372 conn(0x7f9c98101810 msgr2=0x7f9c98103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 shutdown_connections 2024-09-21T07:10:43.270 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:43.267+0000 7f9c9e200640 1 -- 172.21.15.114:0/822670372 wait complete. 2024-09-21T07:10:43.495 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:43 smithi114 ceph-mon[30654]: pgmap v911: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T07:10:43.495 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:43 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a[30650]: 2024-09-21T07:10:43.253+0000 7fa3ee800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:10:43.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-mon[34601]: pgmap v911: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c[64237]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c[64237]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-mds[64241]: -- 172.21.15.193:0/2689076466 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56430372f860 con 0x5643078d7180 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c[64237]: starting mds.c at 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f[63968]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f[63968]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-mds[63972]: -- 172.21.15.193:0/1096725418 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbb0e57860 con 0x55cbb4fff600 2024-09-21T07:10:43.564 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f[63968]: starting mds.f at 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i[64509]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i[64509]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-mds[64513]: -- 172.21.15.193:0/1928438524 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56040cf87860 con 0x56041112f180 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i[64509]: starting mds.i at 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l[63691]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l[63691]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-mds[63695]: -- 172.21.15.193:0/680281436 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c985ef860 con 0x556c9c797180 2024-09-21T07:10:43.565 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:10:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l[63691]: starting mds.l at 2024-09-21T07:10:43.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-mon[34219]: pgmap v911: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-mds[63084]: -- 172.21.15.183:0/3726267857 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b82df3860 con 0x559b86f9b180 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: starting mds.b at 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k[62263]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k[62263]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-mds[62267]: -- 172.21.15.183:0/51287010 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbd6f6d860 con 0x55cbdb115a80 2024-09-21T07:10:43.615 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k[62263]: starting mds.k at 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e[62543]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e[62543]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-mds[62547]: -- 172.21.15.183:0/3232091951 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea33b5b860 con 0x55ea37d03180 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e[62543]: starting mds.e at 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h[62812]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h[62812]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-mds[62816]: -- 172.21.15.183:0/3136431288 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560563291860 con 0x560567439180 2024-09-21T07:10:43.616 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:10:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h[62812]: starting mds.h at 2024-09-21T07:10:43.798 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:10:43 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g[72394]: ignoring --setuser ceph since I am not root 2024-09-21T07:10:43.798 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:10:43 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g[72394]: ignoring --setgroup ceph since I am not root 2024-09-21T07:10:43.798 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:10:43 smithi114 ceph-mds[72399]: -- 172.21.15.114:0/1394044794 <== mon.0 v2:172.21.15.114:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e77ca0d860 con 0x55e780bb5180 2024-09-21T07:10:43.798 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:10:43 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g[72394]: starting mds.g at 2024-09-21T07:10:44.171 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:10:44.547 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:44.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/822670372' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:10:44.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:10:44.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T07:10:44.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:10:44.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T07:10:44.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T07:10:44.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:44 smithi114 ceph-mon[30654]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T07:10:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/822670372' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:10:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:10:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T07:10:44.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:10:44.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T07:10:44.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T07:10:44.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:44 smithi183 ceph-mon[34219]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/822670372' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: osdmap e108: 12 total, 12 up, 12 in 2024-09-21T07:10:44.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:44 smithi193 ceph-mon[34601]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 -- 172.21.15.114:0/80407113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 msgr2=0x7f184c10fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 --2- 172.21.15.114:0/80407113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c10fbc0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1838009860 tx=0x7f183802a340 comp rx=0 tx=0).stop 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 -- 172.21.15.114:0/80407113 shutdown_connections 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 --2- 172.21.15.114:0/80407113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c110100 0x7f184c112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 --2- 172.21.15.114:0/80407113 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c10fbc0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 --2- 172.21.15.114:0/80407113 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 0x7f184c075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 -- 172.21.15.114:0/80407113 >> 172.21.15.114:0/80407113 conn(0x7f184c0ff5f0 msgr2=0x7f184c101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 -- 172.21.15.114:0/80407113 shutdown_connections 2024-09-21T07:10:45.188 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.186+0000 7f1852000640 1 -- 172.21.15.114:0/80407113 wait complete. 2024-09-21T07:10:45.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.187+0000 7f1852000640 1 Processor -- start 2024-09-21T07:10:45.189 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.187+0000 7f1852000640 1 -- start start 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f1852000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 0x7f184c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f1852000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f1852000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c110100 0x7f184c1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f1852000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f184c115cb0 con 0x7f184c0757b0 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f1852000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f184c115b30 con 0x7f184c110100 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f1852000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f184c115e30 con 0x7f184c076100 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 0x7f184c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184aa00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184b400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 0x7f184c19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60622/0 (socket says 172.21.15.114:60622) 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184b400640 1 -- 172.21.15.114:0/1652557662 learned_addr learned my addr 172.21.15.114:0/1652557662 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:45.190 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184be00640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c110100 0x7f184c1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.191 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184aa00640 1 -- 172.21.15.114:0/1652557662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c110100 msgr2=0x7f184c1a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184aa00640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c110100 0x7f184c1a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184aa00640 1 -- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 msgr2=0x7f184c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184aa00640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 0x7f184c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.188+0000 7f184aa00640 1 -- 172.21.15.114:0/1652557662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a25b0 con 0x7f184c076100 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.189+0000 7f184aa00640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c19cbe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1838035040 tx=0x7f1838032dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.189+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838040020 con 0x7f184c076100 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.189+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1838030450 con 0x7f184c076100 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.189+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838037040 con 0x7f184c076100 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.189+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a2840 con 0x7f184c076100 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.189+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a9000 con 0x7f184c076100 2024-09-21T07:10:45.192 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.190+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c076cc0 con 0x7f184c076100 2024-09-21T07:10:45.194 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.192+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f1838043020 con 0x7f184c076100 2024-09-21T07:10:45.195 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.194+0000 7f1843e00640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18300b6680 0x7f18300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.194+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9832+0+0 (secure 0 0 0) 0x7f1838107e70 con 0x7f184c076100 2024-09-21T07:10:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.194+0000 7f184b400640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18300b6680 0x7f18300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:45.196 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.195+0000 7f184b400640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18300b6680 0x7f18300b8b90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f183c005c80 tx=0x7f183c009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:45.197 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.196+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f183804a020 con 0x7f184c076100 2024-09-21T07:10:45.424 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:45.423+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f184c1a91d0 con 0x7f184c076100 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: pgmap v913: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6833/4009310041,v1:172.21.15.183:6835/4009310041] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6832/2049743010,v1:172.21.15.183:6834/2049743010] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6832/57820135,v1:172.21.15.193:6834/57820135] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6833/4220620095,v1:172.21.15.193:6835/4220620095] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6837/936790089,v1:172.21.15.183:6839/936790089] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.183:6836/1253692040,v1:172.21.15.183:6838/1253692040] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6836/1850619971,v1:172.21.15.193:6838/1850619971] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.193:6837/1512432145,v1:172.21.15.193:6839/1512432145] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6838/2366990963,v1:172.21.15.114:6840/2366990963] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: mds.? [v2:172.21.15.114:6839/838804738,v1:172.21.15.114:6841/838804738] up:boot 2024-09-21T07:10:45.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:10:45.567 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:45 smithi114 ceph-mon[30654]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: pgmap v913: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6833/4009310041,v1:172.21.15.183:6835/4009310041] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6832/2049743010,v1:172.21.15.183:6834/2049743010] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6832/57820135,v1:172.21.15.193:6834/57820135] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6833/4220620095,v1:172.21.15.193:6835/4220620095] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6837/936790089,v1:172.21.15.183:6839/936790089] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.183:6836/1253692040,v1:172.21.15.183:6838/1253692040] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6836/1850619971,v1:172.21.15.193:6838/1850619971] up:boot 2024-09-21T07:10:45.614 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.193:6837/1512432145,v1:172.21.15.193:6839/1512432145] up:boot 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6838/2366990963,v1:172.21.15.114:6840/2366990963] up:boot 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: mds.? [v2:172.21.15.114:6839/838804738,v1:172.21.15.114:6841/838804738] up:boot 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:10:45.615 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:45 smithi193 ceph-mon[34601]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: pgmap v913: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 55 KiB/s wr, 16 op/s 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6833/4009310041,v1:172.21.15.183:6835/4009310041] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6832/2049743010,v1:172.21.15.183:6834/2049743010] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6832/57820135,v1:172.21.15.193:6834/57820135] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6833/4220620095,v1:172.21.15.193:6835/4220620095] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6837/936790089,v1:172.21.15.183:6839/936790089] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.183:6836/1253692040,v1:172.21.15.183:6838/1253692040] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6836/1850619971,v1:172.21.15.193:6838/1850619971] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.193:6837/1512432145,v1:172.21.15.193:6839/1512432145] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6838/2366990963,v1:172.21.15.114:6840/2366990963] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: mds.? [v2:172.21.15.114:6839/838804738,v1:172.21.15.114:6841/838804738] up:boot 2024-09-21T07:10:45.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T07:10:45.617 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:45 smithi183 ceph-mon[34219]: from='mgr.14223 172.21.15.114:0/2427315312' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T07:10:46.281 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.280+0000 7f1843e00640 1 -- 172.21.15.114:0/1652557662 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) ==== 104+0+0 (secure 0 0 0) 0x7f18380d2220 con 0x7f184c076100 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18300b6680 msgr2=0x7f18300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18300b6680 0x7f18300b8b90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f183c005c80 tx=0x7f183c009260 comp rx=0 tx=0).stop 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 msgr2=0x7f184c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c19cbe0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1838035040 tx=0x7f1838032dc0 comp rx=0 tx=0).stop 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 shutdown_connections 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f18300b6680 0x7f18300b8b90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:46.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f184c110100 0x7f184c1a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:46.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f184c076100 0x7f184c19cbe0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:46.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 --2- 172.21.15.114:0/1652557662 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f184c0757b0 0x7f184c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:46.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 >> 172.21.15.114:0/1652557662 conn(0x7f184c0ff5f0 msgr2=0x7f184c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:46.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.282+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 shutdown_connections 2024-09-21T07:10:46.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:46.283+0000 7f1852000640 1 -- 172.21.15.114:0/1652557662 wait complete. 2024-09-21T07:10:46.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:46 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/1652557662' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:10:46.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:46 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:10:46.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:46 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/1652557662' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:10:46.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:46 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:10:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:46 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/1652557662' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:10:46.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:46 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T07:10:47.199 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:10:47.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:47 smithi183 ceph-mon[34219]: pgmap v914: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-09-21T07:10:47.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:47 smithi183 ceph-mon[34219]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T07:10:47.535 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:47 smithi183 ceph-mon[34219]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T07:10:47.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:47 smithi183 ceph-mon[34219]: Cluster is now healthy 2024-09-21T07:10:47.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:47 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:10:47.536 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:47 smithi183 ceph-mon[34219]: fsmap 12 up:standby 2024-09-21T07:10:47.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:47 smithi114 ceph-mon[30654]: pgmap v914: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-09-21T07:10:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:47 smithi114 ceph-mon[30654]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T07:10:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:47 smithi114 ceph-mon[30654]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T07:10:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:47 smithi114 ceph-mon[30654]: Cluster is now healthy 2024-09-21T07:10:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:47 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:10:47.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:47 smithi114 ceph-mon[30654]: fsmap 12 up:standby 2024-09-21T07:10:47.575 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:47 smithi193 ceph-mon[34601]: pgmap v914: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-09-21T07:10:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:47 smithi193 ceph-mon[34601]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T07:10:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:47 smithi193 ceph-mon[34601]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T07:10:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:47 smithi193 ceph-mon[34601]: Cluster is now healthy 2024-09-21T07:10:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:47 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T07:10:47.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:47 smithi193 ceph-mon[34601]: fsmap 12 up:standby 2024-09-21T07:10:48.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 -- 172.21.15.114:0/2223206439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c10be40 msgr2=0x7f0e5c112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:48.377 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 --2- 172.21.15.114:0/2223206439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c10be40 0x7f0e5c112670 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009ca0 tx=0x7f0e5402a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 -- 172.21.15.114:0/2223206439 shutdown_connections 2024-09-21T07:10:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 --2- 172.21.15.114:0/2223206439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c10be40 0x7f0e5c112670 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 --2- 172.21.15.114:0/2223206439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e5c10b4f0 0x7f0e5c10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 --2- 172.21.15.114:0/2223206439 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 -- 172.21.15.114:0/2223206439 >> 172.21.15.114:0/2223206439 conn(0x7f0e5c101680 msgr2=0x7f0e5c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:48.378 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 -- 172.21.15.114:0/2223206439 shutdown_connections 2024-09-21T07:10:48.379 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.375+0000 7f0e62000640 1 -- 172.21.15.114:0/2223206439 wait complete. 2024-09-21T07:10:48.381 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.376+0000 7f0e62000640 1 Processor -- start 2024-09-21T07:10:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.376+0000 7f0e62000640 1 -- start start 2024-09-21T07:10:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e62000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e62000640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e5c10b4f0 0x7f0e5c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e62000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5c10be40 0x7f0e5c1a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e62000640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c115b50 con 0x7f0e5c10be40 2024-09-21T07:10:48.382 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e62000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c1159d0 con 0x7f0e5c105bb0 2024-09-21T07:10:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e62000640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0e5c115cd0 con 0x7f0e5c10b4f0 2024-09-21T07:10:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e60c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e60c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.114:52870/0 (socket says 172.21.15.114:52870) 2024-09-21T07:10:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e60c00640 1 -- 172.21.15.114:0/146443298 learned_addr learned my addr 172.21.15.114:0/146443298 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e53e00640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e5c10b4f0 0x7f0e5c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:48.383 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.377+0000 7f0e61600640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5c10be40 0x7f0e5c1a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e60c00640 1 -- 172.21.15.114:0/146443298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e5c10b4f0 msgr2=0x7f0e5c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e60c00640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e5c10b4f0 0x7f0e5c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e60c00640 1 -- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5c10be40 msgr2=0x7f0e5c1a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e60c00640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5c10be40 0x7f0e5c1a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e60c00640 1 -- 172.21.15.114:0/146443298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c1a2430 con 0x7f0e5c105bb0 2024-09-21T07:10:48.384 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e60c00640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c19c4c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00ea60 tx=0x7f0e4c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00cde0 con 0x7f0e5c105bb0 2024-09-21T07:10:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4c0035f0 con 0x7f0e5c105bb0 2024-09-21T07:10:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c010690 con 0x7f0e5c105bb0 2024-09-21T07:10:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e5c1a2720 con 0x7f0e5c105bb0 2024-09-21T07:10:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.378+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5c1a8e80 con 0x7f0e5c105bb0 2024-09-21T07:10:48.385 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.379+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5c105fc0 con 0x7f0e5c105bb0 2024-09-21T07:10:48.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.382+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f0e4c0037a0 con 0x7f0e5c105bb0 2024-09-21T07:10:48.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.383+0000 7f0e5a000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0e400b6700 0x7f0e400b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:48.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.383+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9832+0+0 (secure 0 0 0) 0x7f0e4c0e81b0 con 0x7f0e5c105bb0 2024-09-21T07:10:48.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.383+0000 7f0e53e00640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0e400b6700 0x7f0e400b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:48.387 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.384+0000 7f0e53e00640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0e400b6700 0x7f0e400b8c10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c104d80 tx=0x7f0e44005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:48.388 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.385+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0e4c0ae180 con 0x7f0e5c105bb0 2024-09-21T07:10:48.575 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:48.574+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0e5c19d5f0 con 0x7f0e5c105bb0 2024-09-21T07:10:49.279 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.277+0000 7f0e5a000640 1 -- 172.21.15.114:0/146443298 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f0e5c19d5f0 con 0x7f0e5c105bb0 2024-09-21T07:10:49.279 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:10:49.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0e400b6700 msgr2=0x7f0e400b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0e400b6700 0x7f0e400b8c10 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c104d80 tx=0x7f0e44005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 msgr2=0x7f0e5c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c19c4c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00ea60 tx=0x7f0e4c00ef20 comp rx=0 tx=0).stop 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 shutdown_connections 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f0e5c10be40 0x7f0e5c1a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0e5c10b4f0 0x7f0e5c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f0e400b6700 0x7f0e400b8c10 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 --2- 172.21.15.114:0/146443298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e5c105bb0 0x7f0e5c19c4c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 >> 172.21.15.114:0/146443298 conn(0x7f0e5c101680 msgr2=0x7f0e5c10aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:49.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 shutdown_connections 2024-09-21T07:10:49.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:49.282+0000 7f0e62000640 1 -- 172.21.15.114:0/146443298 wait complete. 2024-09-21T07:10:49.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:49 smithi114 ceph-mon[30654]: pgmap v915: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-21T07:10:49.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:49 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/146443298' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:49.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:49 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:49.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:49 smithi183 ceph-mon[34219]: pgmap v915: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-21T07:10:49.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:49 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/146443298' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:49.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:49 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:49 smithi193 ceph-mon[34601]: pgmap v915: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 59 KiB/s wr, 13 op/s 2024-09-21T07:10:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:49 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/146443298' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:49.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:49 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:50.142 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:10:50.513 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:50.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:50 smithi114 ceph-mon[30654]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:50.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:50 smithi114 ceph-mon[30654]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T07:10:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:50 smithi183 ceph-mon[34219]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:50.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:50 smithi183 ceph-mon[34219]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T07:10:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:50 smithi193 ceph-mon[34601]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:50.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:50 smithi193 ceph-mon[34601]: osdmap e109: 12 total, 12 up, 12 in 2024-09-21T07:10:51.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 -- 172.21.15.114:0/998982561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4874109c20 msgr2=0x7f4874110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:51.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 --2- 172.21.15.114:0/998982561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4874109c20 0x7f4874110450 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ca0 tx=0x7f486c02a8c0 comp rx=0 tx=0).stop 2024-09-21T07:10:51.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 -- 172.21.15.114:0/998982561 shutdown_connections 2024-09-21T07:10:51.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 --2- 172.21.15.114:0/998982561 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f4874109c20 0x7f4874110450 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:51.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 --2- 172.21.15.114:0/998982561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48741092d0 0x7f48741096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:51.128 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 --2- 172.21.15.114:0/998982561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4874103990 0x7f4874103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:51.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 -- 172.21.15.114:0/998982561 >> 172.21.15.114:0/998982561 conn(0x7f48740ff3e0 msgr2=0x7f4874101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:51.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 -- 172.21.15.114:0/998982561 shutdown_connections 2024-09-21T07:10:51.129 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.126+0000 7f4873e00640 1 -- 172.21.15.114:0/998982561 wait complete. 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.127+0000 7f4873e00640 1 Processor -- start 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.127+0000 7f4873e00640 1 -- start start 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873e00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4874103990 0x7f487419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873e00640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 0x7f487419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4874109c20 0x7f48741a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873e00640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f48741138e0 con 0x7f48741092d0 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4874113760 con 0x7f4874109c20 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873e00640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4874113a60 con 0x7f4874103990 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 0x7f487419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4874103990 0x7f487419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 0x7f487419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60654/0 (socket says 172.21.15.114:60654) 2024-09-21T07:10:51.130 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872a00640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4874103990 0x7f487419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.114:51098/0 (socket says 172.21.15.114:51098) 2024-09-21T07:10:51.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 -- 172.21.15.114:0/3350001247 learned_addr learned my addr 172.21.15.114:0/3350001247 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:51.131 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4873400640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4874109c20 0x7f48741a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 -- 172.21.15.114:0/3350001247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4874103990 msgr2=0x7f487419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4874103990 0x7f487419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 -- 172.21.15.114:0/3350001247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4874109c20 msgr2=0x7f48741a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4874109c20 0x7f48741a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.128+0000 7f4872000640 1 -- 172.21.15.114:0/3350001247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48741a24c0 con 0x7f48741092d0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.129+0000 7f4872000640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 0x7f487419caf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f486000e9c0 tx=0x7f486000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.129+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486000ccc0 con 0x7f48741092d0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.129+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48600054d0 con 0x7f48741092d0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.129+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48741a27b0 con 0x7f48741092d0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.129+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860010430 con 0x7f48741092d0 2024-09-21T07:10:51.132 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.129+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48741a8f10 con 0x7f48741092d0 2024-09-21T07:10:51.133 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.130+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4874103da0 con 0x7f48741092d0 2024-09-21T07:10:51.134 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.133+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f4860005000 con 0x7f48741092d0 2024-09-21T07:10:51.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.135+0000 7f486b400640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48580b6680 0x7f48580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:51.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.136+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9299+0+0 (secure 0 0 0) 0x7f48600e7f40 con 0x7f48741092d0 2024-09-21T07:10:51.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.136+0000 7f4872a00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48580b6680 0x7f48580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:51.138 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.136+0000 7f4872a00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48580b6680 0x7f48580b8b90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4864009bd0 tx=0x7f4864005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:51.139 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.138+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4860056710 con 0x7f48741092d0 2024-09-21T07:10:51.330 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:51.328+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f487419d680 con 0x7f48741092d0 2024-09-21T07:10:51.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:51 smithi114 ceph-mon[30654]: pgmap v917: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:51.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:51 smithi183 ceph-mon[34219]: pgmap v917: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:51.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:51 smithi193 ceph-mon[34601]: pgmap v917: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:52.301 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.299+0000 7f486b400640 1 -- 172.21.15.114:0/3350001247 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7f487419d680 con 0x7f48741092d0 2024-09-21T07:10:52.301 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data' removed 2024-09-21T07:10:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48580b6680 msgr2=0x7f48580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48580b6680 0x7f48580b8b90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4864009bd0 tx=0x7f4864005f90 comp rx=0 tx=0).stop 2024-09-21T07:10:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 msgr2=0x7f487419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 0x7f487419caf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f486000e9c0 tx=0x7f486000ee80 comp rx=0 tx=0).stop 2024-09-21T07:10:52.306 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 shutdown_connections 2024-09-21T07:10:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f48580b6680 0x7f48580b8b90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4874109c20 0x7f48741a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.307 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f48741092d0 0x7f487419caf0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 --2- 172.21.15.114:0/3350001247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4874103990 0x7f487419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:52.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 >> 172.21.15.114:0/3350001247 conn(0x7f48740ff3e0 msgr2=0x7f4874107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:52.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.304+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 shutdown_connections 2024-09-21T07:10:52.308 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:52.305+0000 7f4873e00640 1 -- 172.21.15.114:0/3350001247 wait complete. 2024-09-21T07:10:52.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:52 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3350001247' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:52.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:52 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3350001247' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:52.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:52 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3350001247' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:53.243 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T07:10:53.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:53 smithi114 ceph-mon[30654]: pgmap v918: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:53.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:53 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3350001247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:53.565 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:53 smithi114 ceph-mon[30654]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T07:10:53.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:53 smithi183 ceph-mon[34219]: pgmap v918: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:53.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:53 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3350001247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:53.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:53 smithi183 ceph-mon[34219]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T07:10:53.618 INFO:teuthology.orchestra.run.smithi114.stderr:Inferring config /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T07:10:53.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:53 smithi193 ceph-mon[34601]: pgmap v918: 97 pgs: 97 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:53.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:53 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3350001247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:53.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:53 smithi193 ceph-mon[34601]: osdmap e110: 12 total, 12 up, 12 in 2024-09-21T07:10:54.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 -- 172.21.15.114:0/2042598630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 msgr2=0x7f7078104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:54.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 --2- 172.21.15.114:0/2042598630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078104de0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7068009900 tx=0x7f706802a400 comp rx=0 tx=0).stop 2024-09-21T07:10:54.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 -- 172.21.15.114:0/2042598630 shutdown_connections 2024-09-21T07:10:54.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 --2- 172.21.15.114:0/2042598630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f707810de30 0x7f7078110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:54.283 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 --2- 172.21.15.114:0/2042598630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7078105320 0x7f707810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:54.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 --2- 172.21.15.114:0/2042598630 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078104de0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:54.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 -- 172.21.15.114:0/2042598630 >> 172.21.15.114:0/2042598630 conn(0x7f7078079740 msgr2=0x7f7078079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:54.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 -- 172.21.15.114:0/2042598630 shutdown_connections 2024-09-21T07:10:54.284 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.281+0000 7f707e400640 1 -- 172.21.15.114:0/2042598630 wait complete. 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.282+0000 7f707e400640 1 Processor -- start 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 -- start start 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7078105320 0x7f7078102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f707810de30 0x7f7078103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 -- --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_getmap magic: 0 -- 0x7f7078113a90 con 0x7f70781049d0 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7078113910 con 0x7f7078105320 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f707e400640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7078113c10 con 0x7f707810de30 2024-09-21T07:10:54.285 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f7077400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:54.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f7077400640 1 --2- >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.114:3300/0 says I am v2:172.21.15.114:60672/0 (socket says 172.21.15.114:60672) 2024-09-21T07:10:54.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f7077400640 1 -- 172.21.15.114:0/3012171229 learned_addr learned my addr 172.21.15.114:0/3012171229 (peer_addr_for_me v2:172.21.15.114:0/0) 2024-09-21T07:10:54.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.283+0000 7f706fe00640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7078105320 0x7f7078102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:54.286 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077e00640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f707810de30 0x7f7078103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077400640 1 -- 172.21.15.114:0/3012171229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f707810de30 msgr2=0x7f7078103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f707810de30 0x7f7078103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077400640 1 -- 172.21.15.114:0/3012171229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7078105320 msgr2=0x7f7078102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7078105320 0x7f7078102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077400640 1 -- 172.21.15.114:0/3012171229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70781a8780 con 0x7f70781049d0 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077e00640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f707810de30 0x7f7078103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7077400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078102710 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7068032fd0 tx=0x7f7068004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706803f030 con 0x7f70781049d0 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7068004490 con 0x7f70781049d0 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.284+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70781a8a10 con 0x7f70781049d0 2024-09-21T07:10:54.287 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.285+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068035070 con 0x7f70781049d0 2024-09-21T07:10:54.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.285+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70781a8ef0 con 0x7f70781049d0 2024-09-21T07:10:54.288 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.286+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707806c240 con 0x7f70781049d0 2024-09-21T07:10:54.289 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.288+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 4 ==== mgrmap(e 25) ==== 154644+0+0 (secure 0 0 0) 0x7f7068042030 con 0x7f70781049d0 2024-09-21T07:10:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.289+0000 7f7074c00640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f705c0b6680 0x7f705c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:10:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.290+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8711+0+0 (secure 0 0 0) 0x7f7068107db0 con 0x7f70781049d0 2024-09-21T07:10:54.291 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.290+0000 7f706fe00640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f705c0b6680 0x7f705c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:10:54.292 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.290+0000 7f706fe00640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f705c0b6680 0x7f705c0b8b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f707807a010 tx=0x7f706000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:10:54.295 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.294+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f706810c070 con 0x7f70781049d0 2024-09-21T07:10:54.485 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:54.483+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 --> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f707802e640 con 0x7f70781049d0 2024-09-21T07:10:55.311 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.309+0000 7f7074c00640 1 -- 172.21.15.114:0/3012171229 <== mon.0 v2:172.21.15.114:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v111) ==== 178+0+0 (secure 0 0 0) 0x7f70680ce1e0 con 0x7f70781049d0 2024-09-21T07:10:55.311 INFO:teuthology.orchestra.run.smithi114.stderr:pool 'cephfs_data_ec' removed 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f705c0b6680 msgr2=0x7f705c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f705c0b6680 0x7f705c0b8b90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f707807a010 tx=0x7f706000b390 comp rx=0 tx=0).stop 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 msgr2=0x7f7078102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078102710 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7068032fd0 tx=0x7f7068004270 comp rx=0 tx=0).stop 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 shutdown_connections 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f707810de30 0x7f7078103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:6800/1914683713,v1:172.21.15.114:6801/1914683713] conn(0x7f705c0b6680 0x7f705c0b8b90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7078105320 0x7f7078102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 --2- 172.21.15.114:0/3012171229 >> [v2:172.21.15.114:3300/0,v1:172.21.15.114:6789/0] conn(0x7f70781049d0 0x7f7078102710 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:10:55.317 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.315+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 >> 172.21.15.114:0/3012171229 conn(0x7f7078079740 msgr2=0x7f70780ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:10:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.316+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 shutdown_connections 2024-09-21T07:10:55.318 INFO:teuthology.orchestra.run.smithi114.stderr:2024-09-21T07:10:55.316+0000 7f707e400640 1 -- 172.21.15.114:0/3012171229 wait complete. 2024-09-21T07:10:55.414 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:55 smithi193 ceph-mon[34601]: pgmap v920: 33 pgs: 33 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:55.414 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:55 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3012171229' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:55.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:55 smithi114 ceph-mon[30654]: pgmap v920: 33 pgs: 33 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:55.566 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:55 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3012171229' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:55.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:55 smithi183 ceph-mon[34219]: pgmap v920: 33 pgs: 33 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T07:10:55.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:55 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3012171229' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T07:10:56.150 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:10:56.229 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:10:56.270 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:10:56.328 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T07:10:56.328 DEBUG:teuthology.orchestra.run.smithi114:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:10:56.392 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:10:56.412 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T07:10:56.427 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T07:10:56.428 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T07:10:56.428 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a 2024-09-21T07:10:56.475 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:56 smithi114 ceph-mon[30654]: from='client.? 172.21.15.114:0/3012171229' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:56.476 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:56 smithi114 ceph-mon[30654]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T07:10:56.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:56 smithi183 ceph-mon[34219]: from='client.? 172.21.15.114:0/3012171229' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:56.614 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:56 smithi183 ceph-mon[34219]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T07:10:56.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:56 smithi193 ceph-mon[34601]: from='client.? 172.21.15.114:0/3012171229' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T07:10:56.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:10:56 smithi193 ceph-mon[34601]: osdmap e111: 12 total, 12 up, 12 in 2024-09-21T07:10:56.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:56 smithi114 systemd[1]: Stopping Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:10:57.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:56 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a[30650]: 2024-09-21T07:10:56.917+0000 7fa3f5600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:10:57.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:56 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a[30650]: 2024-09-21T07:10:56.917+0000 7fa3f5600640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T07:10:57.319 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:57 smithi114 podman[121412]: 2024-09-21 07:10:57.155442222 +0000 UTC m=+0.422694519 container died 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:10:57.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:57 smithi114 podman[121412]: 2024-09-21 07:10:57.691630786 +0000 UTC m=+0.958883079 container cleanup 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T07:10:57.816 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:57 smithi114 bash[121412]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a 2024-09-21T07:10:58.316 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:58 smithi114 podman[121424]: 2024-09-21 07:10:58.002026157 +0000 UTC m=+0.825799855 container remove 1743bb6a5e1aaca11bddfb8c316c905d42d3496927c9605c3a0bc9acbd167075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-21T07:10:58.629 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T07:10:59.066 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:58 smithi114 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T07:10:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:58 smithi114 systemd[1]: Stopped Ceph mon.a for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T07:10:59.067 INFO:journalctl@ceph.mon.a.smithi114.stdout:Sep 21 07:10:58 smithi114 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.a.service: Consumed 36.337s CPU time. 2024-09-21T07:10:59.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:10:59.119 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T07:10:59.119 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T07:10:59.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.b 2024-09-21T07:10:59.615 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:10:59 smithi183 systemd[1]: Stopping Ceph mon.b for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:01.317 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T07:11:00.892+0000 7f54aa000640 -1 client.14235: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-21T07:11:01.317 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Exception in thread Thread-21: 2024-09-21T07:11:01.317 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: Traceback (most recent call last): 2024-09-21T07:11:01.317 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self.run() 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T07:11:00.894+0000 7f54aa000640 -1 client.14235: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self.function(*self.args, **self.kwargs) 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: self.prune_snapshots(sched) 2024-09-21T07:11:01.318 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-21T07:11:02.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:11:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-b[34215]: 2024-09-21T07:11:01.745+0000 7f5d822ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:11:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-b[34215]: 2024-09-21T07:11:01.745+0000 7f5d822ed640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T07:11:02.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:11:01 smithi183 podman[67463]: 2024-09-21 07:11:01.89935148 +0000 UTC m=+1.679096166 container died bd48b0f8fac8a931b76fddb6cfb1827e11109abd0a7d58971c3791bf94c46c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:11:05.114 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:11:04 smithi183 podman[67463]: 2024-09-21 07:11:04.939856837 +0000 UTC m=+4.719601518 container cleanup bd48b0f8fac8a931b76fddb6cfb1827e11109abd0a7d58971c3791bf94c46c86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T07:11:05.115 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 21 07:11:04 smithi183 bash[67463]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-b 2024-09-21T07:11:05.433 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T07:11:05.482 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:05.482 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T07:11:05.482 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T07:11:05.482 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.c 2024-09-21T07:11:05.912 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:11:05 smithi193 systemd[1]: Stopping Ceph mon.c for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:06.389 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:11:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c[34597]: 2024-09-21T07:11:06.100+0000 7f6fa9f18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:06.389 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:11:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c[34597]: 2024-09-21T07:11:06.100+0000 7f6fa9f18640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T07:11:06.662 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:11:06 smithi193 podman[69025]: 2024-09-21 07:11:06.387939276 +0000 UTC m=+0.662792249 container died 5930fe7c63e7ff4829d3af8df98e1b76d0e57c11641eea8cec5cf0b7f6195e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T07:11:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:11:08 smithi193 podman[69025]: 2024-09-21 07:11:08.986982156 +0000 UTC m=+3.261835126 container cleanup 5930fe7c63e7ff4829d3af8df98e1b76d0e57c11641eea8cec5cf0b7f6195e0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T07:11:09.162 INFO:journalctl@ceph.mon.c.smithi193.stdout:Sep 21 07:11:08 smithi193 bash[69025]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mon-c 2024-09-21T07:11:09.411 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T07:11:09.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:09.464 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T07:11:09.464 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T07:11:09.464 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x 2024-09-21T07:11:09.815 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:09 smithi114 systemd[1]: Stopping Ceph mgr.x for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:10.315 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T07:11:09.952+0000 7f556aa00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:10.316 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x[30874]: 2024-09-21T07:11:09.952+0000 7f556aa00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T07:11:10.316 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:10 smithi114 podman[121529]: 2024-09-21 07:11:10.055342915 +0000 UTC m=+0.249229402 container died 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T07:11:10.815 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:10 smithi114 podman[121529]: 2024-09-21 07:11:10.589641576 +0000 UTC m=+0.783528066 container cleanup 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T07:11:10.816 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:10 smithi114 bash[121529]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x 2024-09-21T07:11:11.110 INFO:journalctl@ceph.mgr.x.smithi114.stdout:Sep 21 07:11:10 smithi114 podman[121541]: 2024-09-21 07:11:10.84976947 +0000 UTC m=+0.767187019 container remove 65183b67ba2418c682371ffb7d8cbcce488e597bebfc7c8ec4946c8b9bd2f7d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-x, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T07:11:11.118 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T07:11:11.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:11.193 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T07:11:11.193 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T07:11:11.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.y 2024-09-21T07:11:11.538 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 07:11:11 smithi183 systemd[1]: Stopping Ceph mgr.y for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:11.838 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 07:11:11 smithi183 podman[67586]: 2024-09-21 07:11:11.533787021 +0000 UTC m=+0.152658250 container died 09d6116226142296e4d5eda55146a25ff22081e64b1b3e0799c14e771d426a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:11:12.114 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 07:11:11 smithi183 podman[67586]: 2024-09-21 07:11:11.944389389 +0000 UTC m=+0.563260603 container cleanup 09d6116226142296e4d5eda55146a25ff22081e64b1b3e0799c14e771d426a5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T07:11:12.114 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 21 07:11:11 smithi183 bash[67586]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-y 2024-09-21T07:11:12.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T07:11:12.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:12.410 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T07:11:12.410 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T07:11:12.410 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.z 2024-09-21T07:11:12.748 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:12 smithi193 systemd[1]: Stopping Ceph mgr.z for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:13.061 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:12 smithi193 podman[69151]: 2024-09-21 07:11:12.747529295 +0000 UTC m=+0.147977633 container died f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T07:11:13.412 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 podman[69151]: 2024-09-21 07:11:13.167388748 +0000 UTC m=+0.567837088 container cleanup f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T07:11:13.412 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 bash[69151]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z 2024-09-21T07:11:13.640 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T07:11:13.663 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 podman[69163]: 2024-09-21 07:11:13.411138249 +0000 UTC m=+0.666172921 container remove f152bc6190f309bc23f946883ac7b886363ea02adc358e61f3742d1908886091 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mgr-z, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T07:11:13.663 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-21T07:11:13.663 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.z.service: Failed with result 'exit-code'. 2024-09-21T07:11:13.663 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 systemd[1]: Stopped Ceph mgr.z for 5d75913a-77e4-11ef-baf5-efdc52797490. 2024-09-21T07:11:13.663 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Sep 21 07:11:13 smithi193 systemd[1]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mgr.z.service: Consumed 7.496s CPU time. 2024-09-21T07:11:13.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:13.674 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T07:11:13.675 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T07:11:13.675 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.0 2024-09-21T07:11:14.066 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:13 smithi114 systemd[1]: Stopping Ceph osd.0 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:14.565 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0[47237]: 2024-09-21T07:11:14.089+0000 7fe98b400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:14.566 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0[47237]: 2024-09-21T07:11:14.089+0000 7fe98b400640 -1 osd.0 111 *** Got signal Terminated *** 2024-09-21T07:11:14.566 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0[47237]: 2024-09-21T07:11:14.089+0000 7fe98b400640 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:11:20.315 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:20 smithi114 podman[121649]: 2024-09-21 07:11:20.254876277 +0000 UTC m=+6.296006193 container died c8c3ebd2f308130c2f430a7b7dd1b7940aefaf2d093a01a3b50d4900ae2dadaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T07:11:23.127 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:23 smithi114 podman[121649]: 2024-09-21 07:11:23.12074796 +0000 UTC m=+9.161877875 container cleanup c8c3ebd2f308130c2f430a7b7dd1b7940aefaf2d093a01a3b50d4900ae2dadaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T07:11:23.128 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:23 smithi114 bash[121649]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0 2024-09-21T07:11:23.815 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:23 smithi114 podman[121669]: 2024-09-21 07:11:23.347485783 +0000 UTC m=+2.977606343 container remove c8c3ebd2f308130c2f430a7b7dd1b7940aefaf2d093a01a3b50d4900ae2dadaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T07:11:24.186 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:23 smithi114 podman[121754]: 2024-09-21 07:11:23.791823824 +0000 UTC m=+0.027434552 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:11:24.186 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:23 smithi114 podman[121754]: 2024-09-21 07:11:23.906855359 +0000 UTC m=+0.142466077 container create 3cb93d2586213c26c3c746fb95f2c9054379c3d74f5b20b9a66e5992e44cd43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T07:11:24.565 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:24 smithi114 podman[121754]: 2024-09-21 07:11:24.184792626 +0000 UTC m=+0.420403356 container init 3cb93d2586213c26c3c746fb95f2c9054379c3d74f5b20b9a66e5992e44cd43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T07:11:24.566 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:24 smithi114 podman[121754]: 2024-09-21 07:11:24.191575006 +0000 UTC m=+0.427185726 container start 3cb93d2586213c26c3c746fb95f2c9054379c3d74f5b20b9a66e5992e44cd43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True) 2024-09-21T07:11:24.566 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:24 smithi114 podman[121754]: 2024-09-21 07:11:24.293222585 +0000 UTC m=+0.528833315 container attach 3cb93d2586213c26c3c746fb95f2c9054379c3d74f5b20b9a66e5992e44cd43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:11:25.065 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:24 smithi114 podman[121777]: 2024-09-21 07:11:24.714186098 +0000 UTC m=+0.026573990 container died 3cb93d2586213c26c3c746fb95f2c9054379c3d74f5b20b9a66e5992e44cd43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T07:11:25.515 INFO:journalctl@ceph.osd.0.smithi114.stdout:Sep 21 07:11:25 smithi114 podman[121777]: 2024-09-21 07:11:25.513979639 +0000 UTC m=+0.826367524 container remove 3cb93d2586213c26c3c746fb95f2c9054379c3d74f5b20b9a66e5992e44cd43f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T07:11:25.546 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T07:11:25.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:25.622 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T07:11:25.622 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T07:11:25.622 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.1 2024-09-21T07:11:25.935 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:25 smithi183 systemd[1]: Stopping Ceph osd.1 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:26.364 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1[38912]: 2024-09-21T07:11:25.933+0000 7fbf02779640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:26.364 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1[38912]: 2024-09-21T07:11:25.933+0000 7fbf02779640 -1 osd.1 111 *** Got signal Terminated *** 2024-09-21T07:11:26.364 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1[38912]: 2024-09-21T07:11:25.933+0000 7fbf02779640 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:11:32.061 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:31 smithi183 podman[67723]: 2024-09-21 07:11:31.686806647 +0000 UTC m=+5.861147789 container died 5ba97e2bf3af35bc365bf23d9ea75c61b2c463efe7cf1c7be999e142b4f13644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T07:11:32.315 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:32 smithi183 podman[67723]: 2024-09-21 07:11:32.167112681 +0000 UTC m=+6.341453826 container cleanup 5ba97e2bf3af35bc365bf23d9ea75c61b2c463efe7cf1c7be999e142b4f13644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T07:11:32.315 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:32 smithi183 bash[67723]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1 2024-09-21T07:11:32.614 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:32 smithi183 podman[67738]: 2024-09-21 07:11:32.477526974 +0000 UTC m=+0.785878824 container remove 5ba97e2bf3af35bc365bf23d9ea75c61b2c463efe7cf1c7be999e142b4f13644 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:11:33.114 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:32 smithi183 podman[67826]: 2024-09-21 07:11:32.733280553 +0000 UTC m=+0.015853447 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:11:33.115 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:32 smithi183 podman[67826]: 2024-09-21 07:11:32.870192767 +0000 UTC m=+0.152765660 container create 3001d98747d4b5197f5aefbb84ac883f2101d3eeda414fcab1c9a282f0559b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-21T07:11:33.416 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:33 smithi183 podman[67826]: 2024-09-21 07:11:33.114526908 +0000 UTC m=+0.397099806 container init 3001d98747d4b5197f5aefbb84ac883f2101d3eeda414fcab1c9a282f0559b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:11:33.417 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:33 smithi183 podman[67826]: 2024-09-21 07:11:33.117959899 +0000 UTC m=+0.400532802 container start 3001d98747d4b5197f5aefbb84ac883f2101d3eeda414fcab1c9a282f0559b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T07:11:33.417 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:33 smithi183 podman[67826]: 2024-09-21 07:11:33.256414802 +0000 UTC m=+0.538987699 container attach 3001d98747d4b5197f5aefbb84ac883f2101d3eeda414fcab1c9a282f0559b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T07:11:33.689 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 21 07:11:33 smithi183 podman[67826]: 2024-09-21 07:11:33.416294457 +0000 UTC m=+0.698867360 container died 3001d98747d4b5197f5aefbb84ac883f2101d3eeda414fcab1c9a282f0559b5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:11:34.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T07:11:34.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:34.180 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T07:11:34.180 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T07:11:34.180 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.2 2024-09-21T07:11:34.490 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:34 smithi193 systemd[1]: Stopping Ceph osd.2 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:34.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2[39650]: 2024-09-21T07:11:34.488+0000 7f06b8d26640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:34.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2[39650]: 2024-09-21T07:11:34.488+0000 7f06b8d26640 -1 osd.2 111 *** Got signal Terminated *** 2024-09-21T07:11:34.912 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2[39650]: 2024-09-21T07:11:34.488+0000 7f06b8d26640 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:11:40.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:40 smithi193 podman[69290]: 2024-09-21 07:11:40.143162119 +0000 UTC m=+5.771267097 container died 123bb3f3a6c5d7864ebd72e1a5339cfeb687d86a33dfbfaa8ad8bd2e4f5e6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T07:11:40.838 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:40 smithi193 podman[69290]: 2024-09-21 07:11:40.53822854 +0000 UTC m=+6.166333528 container cleanup 123bb3f3a6c5d7864ebd72e1a5339cfeb687d86a33dfbfaa8ad8bd2e4f5e6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=) 2024-09-21T07:11:40.838 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:40 smithi193 bash[69290]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2 2024-09-21T07:11:41.148 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:40 smithi193 podman[69305]: 2024-09-21 07:11:40.837390722 +0000 UTC m=+0.692615663 container remove 123bb3f3a6c5d7864ebd72e1a5339cfeb687d86a33dfbfaa8ad8bd2e4f5e6a26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T07:11:41.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:41 smithi193 podman[69397]: 2024-09-21 07:11:41.049662622 +0000 UTC m=+0.015948032 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:11:41.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:41 smithi193 podman[69397]: 2024-09-21 07:11:41.174682359 +0000 UTC m=+0.140967769 container create 7644c932a00e7796ef5f5f5270bf0c1e1fda6c105f5bb60c5208adc7b021ab2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T07:11:41.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:41 smithi193 podman[69397]: 2024-09-21 07:11:41.385894096 +0000 UTC m=+0.352179508 container init 7644c932a00e7796ef5f5f5270bf0c1e1fda6c105f5bb60c5208adc7b021ab2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T07:11:41.412 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:41 smithi193 podman[69397]: 2024-09-21 07:11:41.389082035 +0000 UTC m=+0.355367445 container start 7644c932a00e7796ef5f5f5270bf0c1e1fda6c105f5bb60c5208adc7b021ab2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T07:11:41.716 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:41 smithi193 podman[69397]: 2024-09-21 07:11:41.51949049 +0000 UTC m=+0.485775908 container attach 7644c932a00e7796ef5f5f5270bf0c1e1fda6c105f5bb60c5208adc7b021ab2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T07:11:41.816 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:41 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:41.434+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:42.007 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:41.819+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:42.007 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:41.920+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:42.008 INFO:journalctl@ceph.osd.2.smithi193.stdout:Sep 21 07:11:41 smithi193 podman[69397]: 2024-09-21 07:11:41.715597388 +0000 UTC m=+0.681882814 container died 7644c932a00e7796ef5f5f5270bf0c1e1fda6c105f5bb60c5208adc7b021ab2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T07:11:42.343 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T07:11:42.383 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:42.383 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T07:11:42.384 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T07:11:42.384 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.3 2024-09-21T07:11:42.815 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:42 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:42.436+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:42.815 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:42 smithi114 systemd[1]: Stopping Ceph osd.3 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:42.877 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:42.803+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:43.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:42.875+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:43.315 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:42 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:42.827+0000 7fd8b1400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:43.315 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:42 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:42.827+0000 7fd8b1400640 -1 osd.3 111 *** Got signal Terminated *** 2024-09-21T07:11:43.315 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:42 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:42.827+0000 7fd8b1400640 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:11:43.736 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:43 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:43.393+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:43.912 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:43.769+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:44.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:43 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:43.735+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:43.914+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:44.761 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:44 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:44.419+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:45.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:44 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:44.760+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:45.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:44.769+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:45.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:44.926+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:45.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:45.314+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:45.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:45.306+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:45.791 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:45 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:45.370+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:46.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:45 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:45.789+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:46.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:45.728+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:46.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:45.969+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:46.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:46.306+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:46.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:46.333+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:46.787 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:46 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:46.364+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:46.972 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:46.718+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:47.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:46 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:46.785+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:47.315 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:46.971+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:47.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:47.258+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:47.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:47.313+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:47.816 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3[53649]: 2024-09-21T07:11:47.398+0000 7fd8ace00640 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.378397+0000 front 2024-09-21T07:11:18.378325+0000 (oldest deadline 2024-09-21T07:11:41.278381+0000) 2024-09-21T07:11:47.974 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:47.719+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:48.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:47 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:47.812+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:48.359 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:47.972+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:48.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:48.282+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:48.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:48.358+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:48.991 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:48.670+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:49.211 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:48 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:48.857+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:49.376 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:48.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:49.554 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:49 smithi114 podman[121857]: 2024-09-21 07:11:49.210421466 +0000 UTC m=+6.545051582 container died 076a25ec4d1bdce88f445a42c613b3741593cc2b525104bf69d4b72962e7dfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T07:11:49.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:49.252+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:49.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:49.372+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:49.815 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:49 smithi114 podman[121857]: 2024-09-21 07:11:49.766050299 +0000 UTC m=+7.100680415 container cleanup 076a25ec4d1bdce88f445a42c613b3741593cc2b525104bf69d4b72962e7dfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-21T07:11:49.815 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:49 smithi114 bash[121857]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3 2024-09-21T07:11:49.943 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:49.686+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:50.315 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:49 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:49.893+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:50.316 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:49 smithi114 podman[121875]: 2024-09-21 07:11:49.984619761 +0000 UTC m=+0.766143623 container remove 076a25ec4d1bdce88f445a42c613b3741593cc2b525104bf69d4b72962e7dfb7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T07:11:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:49.942+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:50.515 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:50 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:50.234+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:50.722 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:50.418+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:50.816 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:50 smithi114 podman[121959]: 2024-09-21 07:11:50.411995965 +0000 UTC m=+0.026628951 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:11:50.816 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:50 smithi114 podman[121959]: 2024-09-21 07:11:50.527151148 +0000 UTC m=+0.141784112 container create 5456d3730bfe0702404c4e890961b772c9f708367b3762b4f5b440e43dd4710a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T07:11:50.986 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:50.720+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:51.087 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:50 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:50.933+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:51.087 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:50 smithi114 podman[121959]: 2024-09-21 07:11:50.824478503 +0000 UTC m=+0.439111491 container init 5456d3730bfe0702404c4e890961b772c9f708367b3762b4f5b440e43dd4710a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:11:51.087 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:50 smithi114 podman[121959]: 2024-09-21 07:11:50.831126897 +0000 UTC m=+0.445759862 container start 5456d3730bfe0702404c4e890961b772c9f708367b3762b4f5b440e43dd4710a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T07:11:51.087 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:50 smithi114 podman[121959]: 2024-09-21 07:11:50.929574986 +0000 UTC m=+0.544207962 container attach 5456d3730bfe0702404c4e890961b772c9f708367b3762b4f5b440e43dd4710a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T07:11:51.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:50.985+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:51.484 INFO:journalctl@ceph.osd.3.smithi114.stdout:Sep 21 07:11:51 smithi114 podman[121982]: 2024-09-21 07:11:51.126296813 +0000 UTC m=+0.026243358 container died 5456d3730bfe0702404c4e890961b772c9f708367b3762b4f5b440e43dd4710a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-21T07:11:51.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:51 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:51.253+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:51.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:51.406+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:51.933 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T07:11:52.008 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:11:52.008 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T07:11:52.008 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T07:11:52.009 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.4 2024-09-21T07:11:52.014 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:51.687+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:52.315 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:52 smithi183 systemd[1]: Stopping Ceph osd.4 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:11:52.315 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:52 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:52.239+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:52.315 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:51 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:51.979+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:52.371 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:52.013+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:52.614 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:52 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:52.313+0000 7f1049d62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:11:52.615 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:52 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:52.313+0000 7f1049d62640 -1 osd.4 111 *** Got signal Terminated *** 2024-09-21T07:11:52.615 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:52 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:52.313+0000 7f1049d62640 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:11:52.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:52.370+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:53.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:52.695+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:53.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:52.695+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:53.315 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:52 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:52.957+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:53.316 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:52 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:52.816+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:53.386 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:53.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:53.614 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:53 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:53.278+0000 7f1042345640 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.656338+0000 front 2024-09-21T07:11:29.656454+0000 (oldest deadline 2024-09-21T07:11:52.556486+0000) 2024-09-21T07:11:53.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:53 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:53.200+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:53.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:53.385+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:53.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:53.385+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:54.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:53.971+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:54.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:53.728+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:54.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:53.728+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:54.315 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:53 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:53.825+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:54.315 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:54 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:53.999+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:54.614 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:54 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:54.291+0000 7f1042345640 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.656338+0000 front 2024-09-21T07:11:29.656454+0000 (oldest deadline 2024-09-21T07:11:52.556486+0000) 2024-09-21T07:11:54.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:54 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:54.203+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:54.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:54.399+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:54.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:54.399+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:55.002 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:54.952+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:55.002 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:54.729+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:55.003 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:54.729+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:55.315 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:54 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:54.830+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:55.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:55.023+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:55.614 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:55 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:55.334+0000 7f1042345640 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.656338+0000 front 2024-09-21T07:11:29.656454+0000 (oldest deadline 2024-09-21T07:11:52.556486+0000) 2024-09-21T07:11:55.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:55 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:55.228+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:55.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:55 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:55.228+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:11:55.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:55 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:55.428+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:55.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:55 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:55.428+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:56.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:55 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:55.940+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:56.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:55 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:55.737+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:56.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:55 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:55.737+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:56.315 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:55.817+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:56.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:55 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:55.994+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:56.614 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:56 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4[45023]: 2024-09-21T07:11:56.353+0000 7f1042345640 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.656338+0000 front 2024-09-21T07:11:29.656454+0000 (oldest deadline 2024-09-21T07:11:52.556486+0000) 2024-09-21T07:11:56.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:56 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:56.197+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:56.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:56 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:56.197+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:11:56.754 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:56.431+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:56.754 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:56.431+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:57.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:56.897+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:57.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:56.897+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:11:57.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:56.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:57.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:56.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:57.315 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:56 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:56.817+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:57.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:56 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:56.953+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:57.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:56 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:56.953+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:11:57.615 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:11:57 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:11:57.488+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:11:57.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:57 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:57.245+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:57.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:57 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:57.245+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:11:57.776 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:57.446+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:57.777 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:57.446+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:58.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:57 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:57.801+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:58.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:57 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:57.930+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:58.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:57 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:57.930+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:11:58.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:57.865+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:58.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:57.865+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:11:58.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:57.775+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:58.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:57.775+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:58.364 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:58 smithi183 podman[67953]: 2024-09-21 07:11:58.088301869 +0000 UTC m=+5.899536668 container died 13f76befa6635d5e44dc9f668a6c4ed893553565e24e09adba715891435b2c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T07:11:58.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:58 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:58.284+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:58.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:58 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:58.284+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:11:58.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:58.398+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:58.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:58.398+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:58.783 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:58 smithi183 podman[67953]: 2024-09-21 07:11:58.496989106 +0000 UTC m=+6.308223894 container cleanup 13f76befa6635d5e44dc9f668a6c4ed893553565e24e09adba715891435b2c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:11:58.783 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:58 smithi183 bash[67953]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4 2024-09-21T07:11:58.784 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:11:58 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:11:58.519+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:11:59.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:58 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:58.762+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:11:59.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:58 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:58.955+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:11:59.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:58 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:58.955+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:11:59.093 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:58 smithi183 podman[67968]: 2024-09-21 07:11:58.782581833 +0000 UTC m=+0.694457220 container remove 13f76befa6635d5e44dc9f668a6c4ed893553565e24e09adba715891435b2c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T07:11:59.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:58.742+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:11:59.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:58.742+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:11:59.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:58.885+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:11:59.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:58 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:58.885+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:11:59.364 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:59 smithi183 podman[68055]: 2024-09-21 07:11:58.994524936 +0000 UTC m=+0.016150599 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:11:59.365 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:59 smithi183 podman[68055]: 2024-09-21 07:11:59.124550531 +0000 UTC m=+0.146176224 container create 1391ff74d81c358dc3bce7cfda5b8bdcbb52fe128925dd60b62c2984205fa93b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T07:11:59.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:59 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:59.265+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:11:59.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:11:59 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:11:59.265+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:11:59.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:59.381+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:11:59.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:11:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:11:59.381+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:11:59.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:11:59 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:11:59.530+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:11:59.865 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:59 smithi183 podman[68055]: 2024-09-21 07:11:59.410073274 +0000 UTC m=+0.431698926 container init 1391ff74d81c358dc3bce7cfda5b8bdcbb52fe128925dd60b62c2984205fa93b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:11:59.865 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:59 smithi183 podman[68055]: 2024-09-21 07:11:59.413379633 +0000 UTC m=+0.435005282 container start 1391ff74d81c358dc3bce7cfda5b8bdcbb52fe128925dd60b62c2984205fa93b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-21T07:11:59.865 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:59 smithi183 podman[68055]: 2024-09-21 07:11:59.51051168 +0000 UTC m=+0.532137345 container attach 1391ff74d81c358dc3bce7cfda5b8bdcbb52fe128925dd60b62c2984205fa93b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:11:59.865 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 21 07:11:59 smithi183 podman[68055]: 2024-09-21 07:11:59.602204651 +0000 UTC m=+0.623830303 container died 1391ff74d81c358dc3bce7cfda5b8bdcbb52fe128925dd60b62c2984205fa93b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T07:12:00.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:11:59 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:11:59.789+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:00.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:59 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:59.943+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:00.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:11:59 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:11:59.943+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:00.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:59.930+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:00.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:11:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:11:59.930+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:00.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:59.781+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:00.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:11:59 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:11:59.781+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:00.274 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:00 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:00.220+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:00.274 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:00 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:00.220+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:00.302 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T07:12:00.347 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:00.347 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T07:12:00.348 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T07:12:00.348 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.5 2024-09-21T07:12:00.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:00.374+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:12:00.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:00.374+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:12:00.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:00 smithi193 systemd[1]: Stopping Ceph osd.5 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:00.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:00 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:00.523+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:00.928 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:00.799+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:00.928 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:00.799+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:00.928 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:00.667+0000 7feab381c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:00.928 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:00.667+0000 7feab381c640 -1 osd.5 111 *** Got signal Terminated *** 2024-09-21T07:12:00.928 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:00.667+0000 7feab381c640 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:01.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:00.768+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:01.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:00.954+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:01.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:00.954+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:01.340 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:00.926+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:01.341 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:00 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:00.926+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:01.568 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:01.177+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:01.568 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:01.177+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:01.568 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:01.178+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:01.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:01.339+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:12:01.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:01.339+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:12:01.827 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:01.566+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:01.827 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:01 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:01.566+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:02.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:01 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:01.741+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:02.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:01 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:01.741+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:02.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:01 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:01.936+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:02.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:01 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:01.936+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:02.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:01.896+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:02.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:01.896+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:02.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:01.774+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:02.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:01.774+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:02.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:01 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:01.774+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:02.602 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:02 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:02.130+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:02.602 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:02 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:02.130+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:02.602 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:02 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:02.130+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:02.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:02.361+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:12:02.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:02.361+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:12:02.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:02 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:02.601+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:02.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:02 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:02.601+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:03.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:02.746+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:03.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:02.746+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:03.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:02.901+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:03.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:02 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:02.901+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:03.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:02.799+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:03.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:02.799+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:03.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:02.799+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:03.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:02.892+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:03.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:02 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:02.892+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:03.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:03 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:03.170+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:03.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:03 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:03.170+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:03.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:03 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:03.170+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:03.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:03.326+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:12:03.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:03.326+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:12:04.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:03 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:03.710+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:04.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:03 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:03.710+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:04.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:03 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:03.862+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:04.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:03 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:03.862+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:04.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:03 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:03.630+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:04.115 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:03 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:03.630+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:04.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:03.913+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:04.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:03.913+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:04.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:03.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:04.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:03.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:04.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:03 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:03.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:04.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:04 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:04.174+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:04.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:04 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:04.174+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:04.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:04 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:04.174+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:04.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:04.357+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:12:04.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:04.357+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:12:04.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:04.357+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:38.538398+0000 front 2024-09-21T07:11:38.538424+0000 (oldest deadline 2024-09-21T07:12:03.838670+0000) 2024-09-21T07:12:05.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:04 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:04.734+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:05.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:04 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:04.734+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:05.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:04 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:04.851+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:05.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:04 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:04.851+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:05.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:04 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:04.680+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:05.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:04 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:04.680+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:05.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:04.911+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:05.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:04.911+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:05.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:04.911+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:05.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:04.732+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:05.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:04.732+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:05.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:04 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:04.732+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:05.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:05 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:05.139+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:05.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:05 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:05.139+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:05.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:05 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:05.139+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:05.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:05.374+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:18.734723+0000 front 2024-09-21T07:11:18.734677+0000 (oldest deadline 2024-09-21T07:11:44.634755+0000) 2024-09-21T07:12:05.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:05.374+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:28.036529+0000 front 2024-09-21T07:11:28.036732+0000 (oldest deadline 2024-09-21T07:11:52.736649+0000) 2024-09-21T07:12:05.662 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5[45968]: 2024-09-21T07:12:05.374+0000 7feaafe07640 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:38.538398+0000 front 2024-09-21T07:11:38.538424+0000 (oldest deadline 2024-09-21T07:12:03.838670+0000) 2024-09-21T07:12:06.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:05 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:05.704+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:06.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:05 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:05.704+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:06.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:05 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:05.835+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:06.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:05 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:05.835+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:06.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:05 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:05.835+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:06.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:05 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:05.641+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:06.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:05 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:05.641+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:06.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:05.697+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:06.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:05.697+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:06.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:05.697+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:06.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:05.936+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:06.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:05.936+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:06.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:05 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:05.936+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:06.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:06 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:06.161+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:06.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:06 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:06.161+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:06.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:06 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:06.161+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:06.748 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:06 smithi193 podman[69512]: 2024-09-21 07:12:06.492700049 +0000 UTC m=+5.947176260 container died fccf002e6bfe8b7a3ae920fec5ba67b73cc7cca45329c41d808451a9e109e184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T07:12:07.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:06.711+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:07.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:06.711+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:07.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:06.840+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:07.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:06.840+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:07.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:06 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:06.840+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:07.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:06 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:06.667+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:07.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:06 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:06.667+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:07.146 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:06 smithi193 podman[69512]: 2024-09-21 07:12:06.876565248 +0000 UTC m=+6.331041447 container cleanup fccf002e6bfe8b7a3ae920fec5ba67b73cc7cca45329c41d808451a9e109e184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:12:07.146 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:06 smithi193 bash[69512]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5 2024-09-21T07:12:07.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:06.747+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:07.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:06.747+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:07.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:06.747+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:07.147 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:06.983+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:07.147 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:06.983+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:07.147 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:06 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:06.983+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:07.412 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69529]: 2024-09-21 07:12:07.145014749 +0000 UTC m=+0.647273626 container remove fccf002e6bfe8b7a3ae920fec5ba67b73cc7cca45329c41d808451a9e109e184 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:12:07.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:07 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:07.211+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:07.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:07 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:07.211+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:07.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:07 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:07.211+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:07.733 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69615]: 2024-09-21 07:12:07.354719709 +0000 UTC m=+0.015907643 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:07.733 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69615]: 2024-09-21 07:12:07.479514961 +0000 UTC m=+0.140702923 container create 74facc5d5bb57d3b8a99df419b52daa161099abe68beeba053eca4406a218013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-21T07:12:07.733 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69615]: 2024-09-21 07:12:07.732252779 +0000 UTC m=+0.393440695 container init 74facc5d5bb57d3b8a99df419b52daa161099abe68beeba053eca4406a218013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T07:12:08.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:07 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:07.710+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:08.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:07 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:07.710+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:08.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:07 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:07.806+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:08.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:07 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:07.806+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:08.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:07 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:07.806+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:08.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:07 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:07.617+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:08.115 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:07 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:07.617+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:08.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:07 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:07.977+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:08.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:07 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:07.977+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:08.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:07 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:07.977+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:08.162 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69615]: 2024-09-21 07:12:07.735535465 +0000 UTC m=+0.396723387 container start 74facc5d5bb57d3b8a99df419b52daa161099abe68beeba053eca4406a218013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T07:12:08.162 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69615]: 2024-09-21 07:12:07.840629317 +0000 UTC m=+0.501817248 container attach 74facc5d5bb57d3b8a99df419b52daa161099abe68beeba053eca4406a218013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T07:12:08.163 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 21 07:12:07 smithi193 podman[69615]: 2024-09-21 07:12:07.930492296 +0000 UTC m=+0.591680223 container died 74facc5d5bb57d3b8a99df419b52daa161099abe68beeba053eca4406a218013 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T07:12:08.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:07 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:07.775+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:08.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:07 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:07.775+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:08.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:07 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:07.775+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:08.591 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:08 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:08.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:08.591 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:08 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:08.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:08.591 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:08 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:08.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:08.607 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T07:12:08.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:08.649 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T07:12:08.650 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T07:12:08.650 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.6 2024-09-21T07:12:08.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:08 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:08.590+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:08.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:08 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:08.590+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:08.912 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:08 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:08.748+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:08.912 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:08 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:08.748+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:08.912 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:08 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:08.748+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:09.058 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:08 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:08.823+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:09.058 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:08 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:08.823+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:09.058 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:08 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:08.823+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:09.058 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:08 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:08.711+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:09.058 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:08 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:08.711+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:09.059 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:08 smithi114 systemd[1]: Stopping Ceph osd.6 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:09.059 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:09.056+0000 7fd762e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:09.059 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:09.056+0000 7fd762e00640 -1 osd.6 111 *** Got signal Terminated *** 2024-09-21T07:12:09.059 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:09.056+0000 7fd762e00640 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:09.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:09 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:09.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:09.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:09 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:09.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:09.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:09 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:09.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:09.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:09 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:09.157+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:09.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:09 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:09.157+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:09.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:09 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:09.157+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:10.037 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:09 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:09.738+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:10.037 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:09 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:09.738+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:10.037 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:09 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:09.738+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:10.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:09.704+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:10.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:09.704+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:10.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:09.864+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:10.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:09.864+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:10.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:09 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:09.864+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:10.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:09 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:09.616+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:10.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:09 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:09.616+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:10.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:10.035+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:10.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:10.035+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:10.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:10.035+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:10.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:10.138+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:10.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:10.138+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:10.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:10.138+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:10.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:10.138+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:11.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:10.766+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:11.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:10.766+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:11.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:10.766+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:11.022 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:10 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:10.766+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:11.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:10.693+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:11.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:10.693+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:11.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:10.853+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:11.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:10.853+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:11.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:10.853+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:11.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:10.853+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:11.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:10.613+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:11.114 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:10 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:10.613+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:11.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:11.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:11.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:11.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:11.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:11.020+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:11.567 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:11.134+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:11.568 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:11.134+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:11.568 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:11.134+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:11.568 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:11.134+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:11.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:11.566+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:11.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:11.566+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:11.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:11 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:11.566+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:12.064 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:11.808+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:12.065 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:11.808+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:12.065 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:11.808+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:12.065 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:11 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:11.808+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:12.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:11.699+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:12.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:11.699+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:12.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:11.699+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:47.032632+0000 front 2024-09-21T07:11:47.032704+0000 (oldest deadline 2024-09-21T07:12:11.633362+0000) 2024-09-21T07:12:12.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:11.872+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:12.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:11.872+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:12.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:11.872+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:12.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:11 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:11.872+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:12.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:12.061+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:12.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:12.061+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:12.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:12.061+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:12.548 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:12.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:12.548 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:12.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:12.548 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:12.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:12.549 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:12.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:12.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:12.546+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:12.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:12.546+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:12.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:12 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:12.546+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:12.684+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:12.684+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:12.684+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:47.032632+0000 front 2024-09-21T07:11:47.032704+0000 (oldest deadline 2024-09-21T07:12:11.633362+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:12.891+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:12.891+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:12.891+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:13.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:12.891+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:13.026+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:13.026+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:13.026+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:12.841+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:12.841+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:12.841+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:13.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:12 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:12.841+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:13.531 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:13.140+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:13.532 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:13.140+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:13.532 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:13.140+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:13.532 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:13.140+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:13.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:13.529+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:13.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:13.529+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:13.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:13 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:13.529+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:14.065 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:13.703+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.430177+0000 front 2024-09-21T07:11:26.429907+0000 (oldest deadline 2024-09-21T07:11:52.330003+0000) 2024-09-21T07:12:14.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:13.703+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.031210+0000 front 2024-09-21T07:11:37.031356+0000 (oldest deadline 2024-09-21T07:12:01.731497+0000) 2024-09-21T07:12:14.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6[59826]: 2024-09-21T07:12:13.703+0000 7fd759800640 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:47.032632+0000 front 2024-09-21T07:11:47.032704+0000 (oldest deadline 2024-09-21T07:12:11.633362+0000) 2024-09-21T07:12:14.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:13.844+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:14.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:13.844+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:14.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:13.844+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:14.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:13 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:13.844+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:14.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:14.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:14.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:13.836+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:13.836+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:13.836+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:14.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:13 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:13.836+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:14.493 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:14.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:14.493 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:14.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:14.493 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:14.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:14.493 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:14.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:14.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:14.491+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:14.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:14.491+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:14.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:14 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:14.491+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:15.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:14.864+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:15.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:14.864+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:15.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:14.864+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:15.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:14 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:14.864+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:15.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:15.006+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:15.164 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:15.006+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:15.164 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:15.006+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:15.315 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:14.829+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:15.315 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:14.829+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:15.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:14.829+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:15.316 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:14 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:14.829+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:15.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:15.109+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:15.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:15.109+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:15.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:15.109+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:15.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:15.109+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:15.751 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:15 smithi114 podman[122062]: 2024-09-21 07:12:15.400495905 +0000 UTC m=+6.464317282 container died 11c68249b0bb2ed9b7f7607a4f61e49f4c7f75d3bb40213e8317802b1f79b90c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T07:12:15.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:15.494+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:15.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:15.494+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:15.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:15 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:15.494+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:16.004 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:15 smithi114 podman[122062]: 2024-09-21 07:12:15.99855996 +0000 UTC m=+7.062381377 container cleanup 11c68249b0bb2ed9b7f7607a4f61e49f4c7f75d3bb40213e8317802b1f79b90c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T07:12:16.004 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:15 smithi114 bash[122062]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6 2024-09-21T07:12:16.004 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:15 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:15.836+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:16.004 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:15 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:15.836+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:16.004 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:15 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:15.836+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:16.004 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:15 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:15.836+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:16.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:16.025+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:16.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:16.025+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:16.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:16.025+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:16.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:15.854+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:16.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:15.854+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:16.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:15.854+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:16.163 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:15 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:15.854+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:16.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:16.063+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:16.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:16.063+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:16.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:16.063+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:16.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:16.063+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:16.565 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:16 smithi114 podman[122079]: 2024-09-21 07:12:16.258296349 +0000 UTC m=+0.826507187 container remove 11c68249b0bb2ed9b7f7607a4f61e49f4c7f75d3bb40213e8317802b1f79b90c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:12:16.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:16.504+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:16.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:16.504+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:16.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:16 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:16.504+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:17.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:16 smithi114 podman[122166]: 2024-09-21 07:12:16.713933086 +0000 UTC m=+0.027404412 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:17.066 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:16 smithi114 podman[122166]: 2024-09-21 07:12:16.859170299 +0000 UTC m=+0.172641602 container create 52f0a1d8fd32071800c118b42a8ad68d6167656e7e2c747248deb0d4e8f19be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T07:12:17.066 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:16.805+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:17.066 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:16.805+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:17.066 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:16.805+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:17.066 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:16 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:16.805+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:17.067 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:16 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:16.811+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:17.067 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:16 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:16.811+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:17.067 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:16 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:16.811+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:17.067 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:16 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:16.811+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:17.348 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:17 smithi114 podman[122166]: 2024-09-21 07:12:17.088283492 +0000 UTC m=+0.401754799 container init 52f0a1d8fd32071800c118b42a8ad68d6167656e7e2c747248deb0d4e8f19be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-21T07:12:17.349 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:17 smithi114 podman[122166]: 2024-09-21 07:12:17.095098228 +0000 UTC m=+0.408569536 container start 52f0a1d8fd32071800c118b42a8ad68d6167656e7e2c747248deb0d4e8f19be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:12:17.349 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:17 smithi114 podman[122166]: 2024-09-21 07:12:17.186846203 +0000 UTC m=+0.500317521 container attach 52f0a1d8fd32071800c118b42a8ad68d6167656e7e2c747248deb0d4e8f19be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-21T07:12:17.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:17.095+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:17.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:17.095+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:17.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:17.095+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:17.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:17.095+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:17.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:17.063+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:17.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:17.063+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:17.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:17.063+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:17.648 INFO:journalctl@ceph.osd.6.smithi114.stdout:Sep 21 07:12:17 smithi114 podman[122166]: 2024-09-21 07:12:17.34755423 +0000 UTC m=+0.661025549 container died 52f0a1d8fd32071800c118b42a8ad68d6167656e7e2c747248deb0d4e8f19be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:12:17.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:17.497+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:17.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:17.498+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:17.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:17 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:17.498+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:17.991 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:17 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:17.779+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:17.992 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:17 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:17.779+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:17.992 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:17 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:17.779+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:17.992 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:17 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:17.779+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:18.021 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T07:12:18.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:18.095 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T07:12:18.096 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T07:12:18.096 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.7 2024-09-21T07:12:18.102 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:17.789+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:18.102 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:17.789+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:18.102 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:17.789+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:18.102 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:17 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:17.789+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:18.365 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 systemd[1]: Stopping Ceph osd.7 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:18.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:18.059+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:18.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:18.059+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:18.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:18.059+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:18.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:18.059+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:18.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:18.100+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:18.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:18.100+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:18.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:18.100+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:18.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:18.416+0000 7fd05a80a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:18.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:18.416+0000 7fd05a80a640 -1 osd.7 111 *** Got signal Terminated *** 2024-09-21T07:12:18.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:18.416+0000 7fd05a80a640 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:18.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:18.494+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:18.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:18.494+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:18.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:18.494+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:19.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:18 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:18.754+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:19.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:18 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:18.754+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:19.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:18 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:18.754+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:19.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:18 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:18.754+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:19.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:18 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:18.754+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:19.128 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:18.780+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:19.128 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:18.780+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:19.128 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:18.780+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:19.128 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:18 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:18.780+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:19.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:19.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:19.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:19.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:19.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:19.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:19.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:19.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:19.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:19.126+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:19.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:19.126+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:19.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:19.126+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:19.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:19.126+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:19.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:19.525+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:19.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:19.525+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:19.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:19 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:19.525+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:20.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:19.715+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:20.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:19.715+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:20.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:19.715+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:20.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:19.715+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:20.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:19 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:19.715+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:20.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:19.731+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:20.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:19.731+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:20.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:19.731+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:20.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:19 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:19.731+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:20.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:20.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:20.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:20.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:20.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:20.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:20.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:20.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:20.515 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:20.128+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:20.515 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:20.128+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:20.515 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:20.128+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:20.515 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:20.128+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:20.515 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:20.128+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:20.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:20.513+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:20.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:20.513+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:20.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:20.513+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:20.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:20.513+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:56.420285+0000 front 2024-09-21T07:11:56.420250+0000 (oldest deadline 2024-09-21T07:12:19.920510+0000) 2024-09-21T07:12:21.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:20 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:20.749+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:21.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:20 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:20.749+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:21.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:20 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:20.749+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:21.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:20 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:20.749+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:21.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:20 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:20.749+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:21.099 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:20.717+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:21.099 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:20.717+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:21.099 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:20.717+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:21.099 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:20 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:20.717+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:21.414 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:21.097+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:21.414 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:21.097+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:21.414 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:21.097+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:21.414 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:21.097+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:21.520 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:21.120+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:21.520 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:21.120+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:21.520 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:21.120+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:21.520 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:21.120+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:21.520 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:21.120+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:21.788 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:21.518+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:21.788 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:21.518+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:21.788 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:21.518+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:21.788 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:21 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:21.518+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:56.420285+0000 front 2024-09-21T07:11:56.420250+0000 (oldest deadline 2024-09-21T07:12:19.920510+0000) 2024-09-21T07:12:22.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:21 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:21.789+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:22.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:21 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:21.789+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:22.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:21 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:21.789+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:22.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:21 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:21.789+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:22.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:21 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:21.789+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:22.143 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:21.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:22.143 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:21.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:22.143 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:21.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:22.144 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:21 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:21.753+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:22.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:22.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:22.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:22.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:22.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:22.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:22.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:22.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:22.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:22.107+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:22.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:22.141+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:22.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:22.141+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:22.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:22.141+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:22.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:22.141+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:22.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:22.527+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:30.915782+0000 front 2024-09-21T07:11:30.915802+0000 (oldest deadline 2024-09-21T07:11:56.815997+0000) 2024-09-21T07:12:22.864 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:22.527+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.917155+0000 front 2024-09-21T07:11:37.917228+0000 (oldest deadline 2024-09-21T07:12:00.817097+0000) 2024-09-21T07:12:22.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:22.527+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.919212+0000 front 2024-09-21T07:11:45.919279+0000 (oldest deadline 2024-09-21T07:12:11.119469+0000) 2024-09-21T07:12:22.865 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:22 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7[51527]: 2024-09-21T07:12:22.527+0000 7fd056df5640 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:56.420285+0000 front 2024-09-21T07:11:56.420250+0000 (oldest deadline 2024-09-21T07:12:19.920510+0000) 2024-09-21T07:12:23.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:22 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:22.750+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:23.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:22 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:22.750+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:23.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:22 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:22.750+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:23.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:22 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:22.750+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:23.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:22 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:22.750+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:23.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:22.756+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:23.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:22.756+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:23.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:22.756+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:23.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:22 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:22.756+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:23.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:23.137+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:23.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:23.137+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:23.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:23.137+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:23.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:23.137+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:23.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:23 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:23.137+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:23.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:23.190+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:23.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:23.190+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:23.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:23.190+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:23.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:23.190+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:24.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:23.734+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:24.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:23.734+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:24.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:23.734+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:24.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:23.734+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:24.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:23 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:23.734+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:24.153 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:23.794+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:24.153 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:23.794+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:24.153 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:23.794+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:24.154 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:23 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:23.794+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:24.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:24.151+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:24.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:24.151+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:24.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:24.151+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:24.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:24.151+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:24.615 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:24 smithi183 podman[68159]: 2024-09-21 07:12:24.337387602 +0000 UTC m=+6.048328876 container died d4002361a87093b609d09a75c300af188604fbfaa66145e90eca9d5d2ff50983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T07:12:24.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:24.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:24.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:24.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:24.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:24.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:24.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:24.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:24.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:24.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:25.044 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:24 smithi183 podman[68159]: 2024-09-21 07:12:24.7417301 +0000 UTC m=+6.452671374 container cleanup d4002361a87093b609d09a75c300af188604fbfaa66145e90eca9d5d2ff50983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T07:12:25.044 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:24 smithi183 bash[68159]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7 2024-09-21T07:12:25.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:24 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:24.701+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:25.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:24 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:24.701+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:25.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:24 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:24.701+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:25.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:24 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:24.701+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:25.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:24 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:24.701+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:25.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:24.833+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:25.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:24.833+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:25.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:24.833+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:25.146 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:24 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:24.833+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:25.351 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68186]: 2024-09-21 07:12:25.043320896 +0000 UTC m=+0.700589128 container remove d4002361a87093b609d09a75c300af188604fbfaa66145e90eca9d5d2ff50983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T07:12:25.351 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:25.152+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:25.352 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:25.152+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:25.352 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:25.152+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:25.352 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:25.152+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:25.352 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:25 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:25.152+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:25.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:25.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:25.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:25.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:25.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:25.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:25.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:25.144+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:25.615 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68276]: 2024-09-21 07:12:25.252936047 +0000 UTC m=+0.016021196 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:25.615 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68276]: 2024-09-21 07:12:25.377466239 +0000 UTC m=+0.140551424 container create 58dfc8eee63602815fb82a1b70c8480fdb4509b38557e53842aaf09184e377e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:12:26.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:25 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:25.673+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:26.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:25 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:25.673+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:26.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:25 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:25.673+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:26.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:25 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:25.673+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:26.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:25 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:25.673+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:26.092 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68276]: 2024-09-21 07:12:25.613682761 +0000 UTC m=+0.376767907 container init 58dfc8eee63602815fb82a1b70c8480fdb4509b38557e53842aaf09184e377e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T07:12:26.093 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68276]: 2024-09-21 07:12:25.617319007 +0000 UTC m=+0.380404156 container start 58dfc8eee63602815fb82a1b70c8480fdb4509b38557e53842aaf09184e377e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T07:12:26.093 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68276]: 2024-09-21 07:12:25.730178577 +0000 UTC m=+0.493263726 container attach 58dfc8eee63602815fb82a1b70c8480fdb4509b38557e53842aaf09184e377e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:12:26.093 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 21 07:12:25 smithi183 podman[68276]: 2024-09-21 07:12:25.800138271 +0000 UTC m=+0.563223411 container died 58dfc8eee63602815fb82a1b70c8480fdb4509b38557e53842aaf09184e377e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-21T07:12:26.131 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:25.874+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:26.132 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:25.874+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:26.132 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:25.874+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:26.132 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:25 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:25.874+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:26.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:26 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:26.179+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:26.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:26 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:26.179+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:26.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:26 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:26.179+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:26.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:26 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:26.179+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:26.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:26 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:26.179+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:26.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:26.129+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:26.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:26.129+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:26.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:26.129+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:26.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:26.129+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:26.431 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T07:12:26.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:26.477 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T07:12:26.477 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T07:12:26.477 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.8 2024-09-21T07:12:26.807 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 systemd[1]: Stopping Ceph osd.8 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:27.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:26.634+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:27.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:26.634+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:27.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:26.634+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:27.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:26.634+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:27.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:26 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:26.634+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:27.148 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.805+0000 7f98e143f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:27.148 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.805+0000 7f98e143f640 -1 osd.8 111 *** Got signal Terminated *** 2024-09-21T07:12:27.148 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.805+0000 7f98e143f640 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:27.148 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.885+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:27.148 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.885+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:27.149 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.885+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:27.149 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:26 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:26.885+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:27.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:27.146+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:27.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:27.146+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:27.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:27.146+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:27.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:27.146+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:27.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:27 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:27.186+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:27.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:27 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:27.186+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:27.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:27 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:27.186+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:27.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:27 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:27.186+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:27.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:27 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:27.186+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:28.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:27 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:27.640+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:28.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:27 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:27.640+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:28.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:27 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:27.640+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:28.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:27 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:27.640+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:28.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:27 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:27.640+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:28.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:27.888+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:28.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:27.888+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:28.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:27.888+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:28.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:27 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:27.888+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:28.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:28.203+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:28.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:28.203+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:28.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:28.203+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:28.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:28.203+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:28.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:28.203+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:28.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:28.175+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:28.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:28.175+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:28.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:28.175+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:28.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:28.175+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:28.662 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:28.175+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:29.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:28 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:28.657+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:29.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:28 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:28.657+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:29.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:28 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:28.657+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:29.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:28 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:28.657+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:29.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:28 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:28.657+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:29.150 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:28.853+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:29.150 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:28.853+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:29.151 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:28.853+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:29.151 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:28 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:28.853+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:29.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:29.148+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:29.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:29.148+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:29.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:29.148+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:29.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:29.148+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:29.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:29.148+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:29.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:29 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:29.188+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:29.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:29 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:29.188+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:29.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:29 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:29.188+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:29.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:29 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:29.188+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:29.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:29 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:29.188+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:30.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:29 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:29.629+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:30.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:29 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:29.629+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:30.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:29 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:29.629+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:30.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:29 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:29.629+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:30.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:29 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:29.629+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:30.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:29 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:29.629+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:30.137 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:29.844+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:30.138 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:29.844+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:30.138 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:29.844+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:30.138 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:29.844+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:30.138 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:29 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:29.844+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.164661+0000 front 2024-09-21T07:12:04.164626+0000 (oldest deadline 2024-09-21T07:12:29.464719+0000) 2024-09-21T07:12:30.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:30.135+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:30.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:30.135+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:30.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:30.135+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:30.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:30.135+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:30.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:30.135+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:30.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:30 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:30.164+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:30.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:30 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:30.164+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:30.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:30 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:30.164+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:30.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:30 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:30.164+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:30.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:30 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:30.164+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:30.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:30 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:30.164+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:31.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:30.589+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:31.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:30.589+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:31.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:30.589+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:31.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:30.589+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:31.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:30.589+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:31.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:30 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:30.589+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:31.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:30.887+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:15.758376+0000 front 2024-09-21T07:11:15.758443+0000 (oldest deadline 2024-09-21T07:11:41.058392+0000) 2024-09-21T07:12:31.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:30.887+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.360283+0000 front 2024-09-21T07:11:27.360325+0000 (oldest deadline 2024-09-21T07:11:52.060371+0000) 2024-09-21T07:12:31.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:30.887+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.261693+0000 front 2024-09-21T07:11:37.261751+0000 (oldest deadline 2024-09-21T07:12:01.361855+0000) 2024-09-21T07:12:31.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:30.887+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:46.662927+0000 front 2024-09-21T07:11:46.663065+0000 (oldest deadline 2024-09-21T07:12:10.762748+0000) 2024-09-21T07:12:31.162 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:30 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8[52577]: 2024-09-21T07:12:30.887+0000 7f98dda2a640 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.164661+0000 front 2024-09-21T07:12:04.164626+0000 (oldest deadline 2024-09-21T07:12:29.464719+0000) 2024-09-21T07:12:31.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:31 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:31.093+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:31.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:31 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:31.093+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:31.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:31 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:31.093+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:31.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:31 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:31.093+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:31.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:31 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:31.093+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:31.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:31 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:31.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:31.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:31 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:31.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:31.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:31 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:31.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:31.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:31 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:31.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:31.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:31 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:31.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:31.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:31 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:31.123+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:32.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:31.599+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:32.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:31.599+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:32.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:31.599+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:32.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:31.599+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:32.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:31.599+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:32.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:31 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:31.599+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:32.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:32.108+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:32.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:32.108+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:32.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:32.108+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:32.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:32.108+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:32.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:32.108+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:32.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:32 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:32.108+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:32.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:32.140+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:32.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:32.140+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:32.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:32.140+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:32.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:32.140+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:32.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:32.140+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:32.814 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:32 smithi193 podman[69730]: 2024-09-21 07:12:32.499771753 +0000 UTC m=+5.817297973 container died e312607183155104e8d8983a849b8b739f8e275c7c803395d4c00f1c14c909a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:12:33.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:32.573+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:33.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:32.573+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:33.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:32.573+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:33.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:32.573+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:33.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:32.573+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:33.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:32 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:32.573+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:33.111 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:32 smithi193 podman[69730]: 2024-09-21 07:12:32.922347818 +0000 UTC m=+6.239874025 container cleanup e312607183155104e8d8983a849b8b739f8e275c7c803395d4c00f1c14c909a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T07:12:33.111 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:32 smithi193 bash[69730]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8 2024-09-21T07:12:33.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:33 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:33.068+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:33.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:33 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:33.068+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:33.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:33 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:33.068+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:33.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:33 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:33.068+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:33.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:33 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:33.068+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:33.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:33 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:33.068+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:33.412 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69748]: 2024-09-21 07:12:33.115427124 +0000 UTC m=+0.616116511 container remove e312607183155104e8d8983a849b8b739f8e275c7c803395d4c00f1c14c909a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T07:12:33.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:33 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:33.109+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:33.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:33 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:33.109+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:33.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:33 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:33.109+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:33.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:33 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:33.109+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:33.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:33 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:33.109+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:33.870 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69836]: 2024-09-21 07:12:33.437020562 +0000 UTC m=+0.015509777 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:33.870 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69836]: 2024-09-21 07:12:33.541443647 +0000 UTC m=+0.119932856 container create ff5930efaba9669ad467fd33773cb90e0f16ac9018571d53b95ceb387610df43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T07:12:33.870 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69836]: 2024-09-21 07:12:33.768894769 +0000 UTC m=+0.347383976 container init ff5930efaba9669ad467fd33773cb90e0f16ac9018571d53b95ceb387610df43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T07:12:33.870 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69836]: 2024-09-21 07:12:33.772086904 +0000 UTC m=+0.350576110 container start ff5930efaba9669ad467fd33773cb90e0f16ac9018571d53b95ceb387610df43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:12:33.870 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69836]: 2024-09-21 07:12:33.86888074 +0000 UTC m=+0.447369954 container attach ff5930efaba9669ad467fd33773cb90e0f16ac9018571d53b95ceb387610df43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T07:12:34.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:33 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:33.595+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:34.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:33 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:33.595+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:34.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:33 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:33.595+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:34.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:33 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:33.595+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:34.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:33 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:33.595+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:34.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:33 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:33.595+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:34.134 INFO:journalctl@ceph.osd.8.smithi193.stdout:Sep 21 07:12:33 smithi193 podman[69836]: 2024-09-21 07:12:33.958706053 +0000 UTC m=+0.537195264 container died ff5930efaba9669ad467fd33773cb90e0f16ac9018571d53b95ceb387610df43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:12:34.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:34.131+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:34.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:34.131+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:34.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:34.131+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:34.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:34.131+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:34.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:34 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:34.131+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:34.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:34 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:34.114+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:34.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:34 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:34.114+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:34.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:34 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:34.114+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:34.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:34 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:34.114+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:34.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:34 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:34.114+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:34.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:34 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:34.114+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:34.691 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T07:12:34.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:34.733 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T07:12:34.734 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T07:12:34.734 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.9 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:34.644+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:34.644+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:34.644+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:34.644+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:34.644+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:34.644+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:35.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:34 smithi114 systemd[1]: Stopping Ceph osd.9 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:35.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:35 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:35.091+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:35.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:35 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:35.091+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:35.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:35 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:35.091+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:35.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:35 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:35.091+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:35.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:35 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:35.091+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:35.565 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.180+0000 7f4540200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:35.565 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.180+0000 7f4540200640 -1 osd.9 111 *** Got signal Terminated *** 2024-09-21T07:12:35.566 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.180+0000 7f4540200640 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:35.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:35.142+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:35.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:35.142+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:35.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:35.142+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:35.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:35.142+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:35.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:35.142+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:35.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:35 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:35.142+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:36.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.675+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:36.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.675+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:36.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.675+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:36.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.675+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:36.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.675+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:36.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:35 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:35.675+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:36.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:36.112+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:36.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:36.112+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:36.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:36.112+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:36.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:36.112+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:36.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:36.112+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:36.163 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:36.112+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:36.484 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:36 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:36.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:36.484 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:36 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:36.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:36.484 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:36 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:36.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:36.484 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:36 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:36.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:36.484 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:36 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:36.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:36.485 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:36 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:36.147+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:37.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:36 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:36.707+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:37.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:36 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:36.707+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:37.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:36 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:36.707+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:37.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:36 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:36.707+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:37.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:36 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:36.707+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:37.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:36 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:36.707+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:37.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:37 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:37.110+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:37.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:37 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:37.110+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:37.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:37 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:37.110+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:37.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:37 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:37.110+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:37.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:37 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:37.110+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:37.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:37 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:37.110+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:37.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:37 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:37.127+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:37.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:37 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:37.127+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:37.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:37 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:37.127+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:37.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:37 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:37.127+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:37.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:37 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:37.127+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:37.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:37 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:37.127+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:38.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:37 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:37.738+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:38.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:37 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:37.738+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:38.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:37 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:37.738+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:38.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:37 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:37.738+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:38.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:37 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:37.738+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:38.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:37 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:37.738+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:38.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:38 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:38.075+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:38.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:38 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:38.075+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:38.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:38 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:38.075+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:38.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:38 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:38.075+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:38.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:38 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:38.075+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:38.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:38 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:38.075+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:38.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:38 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:38.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:38.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:38 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:38.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:38.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:38 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:38.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:38.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:38 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:38.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:38.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:38 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:38.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:38.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:38 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:38.155+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:39.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:38 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:38.691+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:39.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:38 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:38.691+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:39.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:38 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:38.691+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:39.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:38 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:38.691+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:39.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:38 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:38.691+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:39.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:38 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:38.691+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:39.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:39 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:39.113+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:39.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:39 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:39.030+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:39.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:39 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:39.030+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:39.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:39 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:39.030+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:39.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:39 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:39.030+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:39.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:39 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:39.030+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:39.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:39 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:39.030+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:40.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:39 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:39.723+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:16.712091+0000 front 2024-09-21T07:11:16.712129+0000 (oldest deadline 2024-09-21T07:11:43.112768+0000) 2024-09-21T07:12:40.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:39 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:39.723+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:27.814717+0000 front 2024-09-21T07:11:27.814257+0000 (oldest deadline 2024-09-21T07:11:56.615047+0000) 2024-09-21T07:12:40.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:39 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:39.723+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.615715+0000 front 2024-09-21T07:11:36.615674+0000 (oldest deadline 2024-09-21T07:12:05.416357+0000) 2024-09-21T07:12:40.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:39 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:39.723+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:45.417085+0000 front 2024-09-21T07:11:45.417051+0000 (oldest deadline 2024-09-21T07:12:10.117151+0000) 2024-09-21T07:12:40.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:39 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:39.723+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.818492+0000 front 2024-09-21T07:11:54.818538+0000 (oldest deadline 2024-09-21T07:12:18.318645+0000) 2024-09-21T07:12:40.066 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:39 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9[66071]: 2024-09-21T07:12:39.723+0000 7f453bc00640 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.120856+0000 front 2024-09-21T07:12:04.120669+0000 (oldest deadline 2024-09-21T07:12:29.420329+0000) 2024-09-21T07:12:40.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:40.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:40.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:40.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:40.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:40.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:40.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:40 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:40.077+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:40.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:40.003+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:40.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:40.003+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:40.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:40.003+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:40.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:40.003+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:40.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:40.003+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:40.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:40.003+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:41.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:41.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:41.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:41.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:41.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:41.365 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:41.366 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:41 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:41.087+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:41.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:41.044+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:41.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:41.044+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:41.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:41.044+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:41.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:41.044+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:41.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:41.044+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:41.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:41 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:41.044+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:42.065 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:41 smithi114 podman[122267]: 2024-09-21 07:12:41.76726077 +0000 UTC m=+6.748233382 container died 964bbf005dda6e89b9bdc3ce9c6ffc933ed4104a2b4b6822cb31c6eb04adb564 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:12:42.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:42.056+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:42.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:42.056+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:42.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:42.056+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:42.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:42.056+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:42.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:42.056+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:42.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:42 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:42.056+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:42.536 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:42 smithi114 podman[122267]: 2024-09-21 07:12:42.300119539 +0000 UTC m=+7.281092149 container cleanup 964bbf005dda6e89b9bdc3ce9c6ffc933ed4104a2b4b6822cb31c6eb04adb564 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:12:42.536 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:42 smithi114 bash[122267]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9 2024-09-21T07:12:42.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:42.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:42.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:42.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:42.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:42.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:42.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:42 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:42.136+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:42.815 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:42 smithi114 podman[122285]: 2024-09-21 07:12:42.534474822 +0000 UTC m=+0.733403693 container remove 964bbf005dda6e89b9bdc3ce9c6ffc933ed4104a2b4b6822cb31c6eb04adb564 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:12:43.313 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:43.041+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:43.313 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:43.041+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:43.313 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:43.041+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:43.313 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:43.041+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:43.313 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:43.041+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:43.314 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:43 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:43.041+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:43.317 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:42 smithi114 podman[122369]: 2024-09-21 07:12:42.894421456 +0000 UTC m=+0.026806498 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:43.318 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:42 smithi114 podman[122369]: 2024-09-21 07:12:42.993610461 +0000 UTC m=+0.125995475 container create 692f190b2419961fd45fbac9476c8ec76960f891dd724417200fcda13ef63da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T07:12:43.318 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:43 smithi114 podman[122369]: 2024-09-21 07:12:43.221736665 +0000 UTC m=+0.354121696 container init 692f190b2419961fd45fbac9476c8ec76960f891dd724417200fcda13ef63da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:12:43.318 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:43 smithi114 podman[122369]: 2024-09-21 07:12:43.228678509 +0000 UTC m=+0.361063532 container start 692f190b2419961fd45fbac9476c8ec76960f891dd724417200fcda13ef63da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T07:12:43.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:43.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:43.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:43.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:43.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:43.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:43.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:43 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:43.162+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:43.815 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:43 smithi114 podman[122369]: 2024-09-21 07:12:43.354788218 +0000 UTC m=+0.487173251 container attach 692f190b2419961fd45fbac9476c8ec76960f891dd724417200fcda13ef63da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T07:12:43.816 INFO:journalctl@ceph.osd.9.smithi114.stdout:Sep 21 07:12:43 smithi114 podman[122369]: 2024-09-21 07:12:43.490230339 +0000 UTC m=+0.622615366 container died 692f190b2419961fd45fbac9476c8ec76960f891dd724417200fcda13ef63da7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:12:44.201 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T07:12:44.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:44.278 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T07:12:44.278 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T07:12:44.278 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.10 2024-09-21T07:12:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:44.001+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:44.001+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:44.001+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:44.001+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:44.001+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:44.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:44.001+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:44.604 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:44.604 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:44.604 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:44.605 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:44.605 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:44.605 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:44.605 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.125+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:44.605 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 systemd[1]: Stopping Ceph osd.10 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:44.864 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.602+0000 7f68ebb05640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:44.864 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.602+0000 7f68ebb05640 -1 osd.10 111 *** Got signal Terminated *** 2024-09-21T07:12:44.865 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:44 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:44.602+0000 7f68ebb05640 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:45.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:45.046+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:45.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:45.046+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:45.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:45.046+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:45.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:45.046+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:45.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:45.046+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:45.408 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:45 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:45.046+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:45.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:45.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:45.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:45.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:45.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:45.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:45.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:45 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:45.133+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:46.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:46.048+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:46.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:46.048+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:46.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:46.048+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:46.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:46.048+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:46.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:46.048+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:46.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:46 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:46.048+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:46.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:46.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:46.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:46.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:46.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:46.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:46.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:46 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:46.118+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:47.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:47.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:47.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:47.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:47.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:47.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:47.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:47.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:47.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:47.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:47.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:47 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:47.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:47.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:47.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:47.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:47.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:47.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:47.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:47.616 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:47 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:47.131+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:48.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:48.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:48.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:48.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:48.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:48.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:48.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:48.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:48.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:48.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:48.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:48 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:48.049+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:48.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:48.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:48.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:48.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:48.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:48.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:48.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:48 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:48.168+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:49.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:49.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:49.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:49.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:49.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:49.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:49.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:49 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:49.029+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:49.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:14.705389+0000 front 2024-09-21T07:11:14.705421+0000 (oldest deadline 2024-09-21T07:11:45.305759+0000) 2024-09-21T07:12:49.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:29.907187+0000 front 2024-09-21T07:11:29.907232+0000 (oldest deadline 2024-09-21T07:11:54.607311+0000) 2024-09-21T07:12:49.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:37.508309+0000 front 2024-09-21T07:11:37.508335+0000 (oldest deadline 2024-09-21T07:12:00.408420+0000) 2024-09-21T07:12:49.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6814 osd.3 since back 2024-09-21T07:11:44.410061+0000 front 2024-09-21T07:11:44.410041+0000 (oldest deadline 2024-09-21T07:12:09.709918+0000) 2024-09-21T07:12:49.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:54.911253+0000 front 2024-09-21T07:11:54.911278+0000 (oldest deadline 2024-09-21T07:12:19.611320+0000) 2024-09-21T07:12:49.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:04.312250+0000 front 2024-09-21T07:12:04.312109+0000 (oldest deadline 2024-09-21T07:12:29.612087+0000) 2024-09-21T07:12:49.615 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:49 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10[58219]: 2024-09-21T07:12:49.204+0000 7f68e80f0640 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:13.713069+0000 front 2024-09-21T07:12:13.713109+0000 (oldest deadline 2024-09-21T07:12:39.013041+0000) 2024-09-21T07:12:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:50.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:50.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:50 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:50.074+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:50.614 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:50 smithi183 podman[68378]: 2024-09-21 07:12:50.356941349 +0000 UTC m=+5.878268285 container died d1a381e9df5aaf7567c5b886b37f033f5089b0858f9a049f9b777bb6f8f11bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2) 2024-09-21T07:12:51.021 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:50 smithi183 podman[68378]: 2024-09-21 07:12:50.726918819 +0000 UTC m=+6.248245716 container cleanup d1a381e9df5aaf7567c5b886b37f033f5089b0858f9a049f9b777bb6f8f11bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T07:12:51.021 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:50 smithi183 bash[68378]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10 2024-09-21T07:12:51.364 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68396]: 2024-09-21 07:12:51.020062977 +0000 UTC m=+0.663838717 container remove d1a381e9df5aaf7567c5b886b37f033f5089b0858f9a049f9b777bb6f8f11bed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T07:12:51.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:51.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:51.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:51.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:51.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:51.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:51.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.031+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:51.619 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68489]: 2024-09-21 07:12:51.27099718 +0000 UTC m=+0.016105936 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:51.620 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68489]: 2024-09-21 07:12:51.370653984 +0000 UTC m=+0.115762734 container create f65d0f0809cfe45d06038a4060559e27296e8e72f9aca3f1a0f9bf1d1d968f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T07:12:51.620 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68489]: 2024-09-21 07:12:51.615085416 +0000 UTC m=+0.360194167 container init f65d0f0809cfe45d06038a4060559e27296e8e72f9aca3f1a0f9bf1d1d968f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T07:12:52.110 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68489]: 2024-09-21 07:12:51.618679735 +0000 UTC m=+0.363788491 container start f65d0f0809cfe45d06038a4060559e27296e8e72f9aca3f1a0f9bf1d1d968f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:12:52.110 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68489]: 2024-09-21 07:12:51.731912533 +0000 UTC m=+0.477021283 container attach f65d0f0809cfe45d06038a4060559e27296e8e72f9aca3f1a0f9bf1d1d968f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T07:12:52.110 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 21 07:12:51 smithi183 podman[68489]: 2024-09-21 07:12:51.823018815 +0000 UTC m=+0.568127571 container died f65d0f0809cfe45d06038a4060559e27296e8e72f9aca3f1a0f9bf1d1d968f3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T07:12:52.393 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:52.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:51 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:51.990+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:52.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:12:52.432 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T07:12:52.432 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T07:12:52.432 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.11 2024-09-21T07:12:52.718 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:52 smithi193 systemd[1]: Stopping Ceph osd.11 for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:12:52.719 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:52.716+0000 7f8c34665640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:12:52.719 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:52.716+0000 7f8c34665640 -1 osd.11 111 *** Got signal Terminated *** 2024-09-21T07:12:52.719 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:52 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:52.716+0000 7f8c34665640 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T07:12:53.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:53.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:53.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:53.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:53.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:53.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:53.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:53 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:53.015+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:54.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.014+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:55.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:54 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:54.991+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:56.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:56.413 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:56 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:56.038+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-21T07:12:29.567889+0000 front 2024-09-21T07:12:29.567868+0000 (oldest deadline 2024-09-21T07:12:55.467841+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6806 osd.0 since back 2024-09-21T07:11:17.754895+0000 front 2024-09-21T07:11:17.754863+0000 (oldest deadline 2024-09-21T07:11:41.254752+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-21T07:11:26.857975+0000 front 2024-09-21T07:11:26.857801+0000 (oldest deadline 2024-09-21T07:11:56.858387+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-09-21T07:11:36.858823+0000 front 2024-09-21T07:11:36.859036+0000 (oldest deadline 2024-09-21T07:12:04.459401+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-21T07:11:53.662035+0000 front 2024-09-21T07:11:53.661992+0000 (oldest deadline 2024-09-21T07:12:18.961889+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-09-21T07:12:03.464089+0000 front 2024-09-21T07:12:03.463973+0000 (oldest deadline 2024-09-21T07:12:28.163927+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.114:6822 osd.6 since back 2024-09-21T07:12:11.065053+0000 front 2024-09-21T07:12:11.065000+0000 (oldest deadline 2024-09-21T07:12:35.764737+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-21T07:12:23.166896+0000 front 2024-09-21T07:12:23.166950+0000 (oldest deadline 2024-09-21T07:12:48.466890+0000) 2024-09-21T07:12:57.412 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:57 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11[59240]: 2024-09-21T07:12:57.072+0000 7f8c2cc48640 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-09-21T07:12:29.567889+0000 front 2024-09-21T07:12:29.567868+0000 (oldest deadline 2024-09-21T07:12:55.467841+0000) 2024-09-21T07:12:58.852 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:58 smithi193 podman[69951]: 2024-09-21 07:12:58.55623278 +0000 UTC m=+5.930969002 container died 8708f1fef542cdab9a335c582b5e0827b41192ac6eed3390b9f80452a8f58f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T07:12:59.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:58 smithi193 podman[69951]: 2024-09-21 07:12:58.958094048 +0000 UTC m=+6.332830257 container cleanup 8708f1fef542cdab9a335c582b5e0827b41192ac6eed3390b9f80452a8f58f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:12:59.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:58 smithi193 bash[69951]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11 2024-09-21T07:12:59.494 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[69966]: 2024-09-21 07:12:59.201116126 +0000 UTC m=+0.637617503 container remove 8708f1fef542cdab9a335c582b5e0827b41192ac6eed3390b9f80452a8f58f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:12:59.809 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[70052]: 2024-09-21 07:12:59.493537259 +0000 UTC m=+0.098608881 container create 828c5563513903b9f2ed1ef7e5d833c76c9bf472d7f85f8d6cb3e1f30e22efc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T07:12:59.810 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[70052]: 2024-09-21 07:12:59.410720084 +0000 UTC m=+0.015774937 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T07:12:59.810 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[70052]: 2024-09-21 07:12:59.713143365 +0000 UTC m=+0.318198209 container init 828c5563513903b9f2ed1ef7e5d833c76c9bf472d7f85f8d6cb3e1f30e22efc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T07:12:59.810 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[70052]: 2024-09-21 07:12:59.716546659 +0000 UTC m=+0.321601504 container start 828c5563513903b9f2ed1ef7e5d833c76c9bf472d7f85f8d6cb3e1f30e22efc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T07:12:59.810 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[70052]: 2024-09-21 07:12:59.808507279 +0000 UTC m=+0.413562143 container attach 828c5563513903b9f2ed1ef7e5d833c76c9bf472d7f85f8d6cb3e1f30e22efc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T07:13:00.162 INFO:journalctl@ceph.osd.11.smithi193.stdout:Sep 21 07:12:59 smithi193 podman[70052]: 2024-09-21 07:12:59.913013305 +0000 UTC m=+0.518068150 container died 828c5563513903b9f2ed1ef7e5d833c76c9bf472d7f85f8d6cb3e1f30e22efc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-21T07:13:00.492 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T07:13:00.530 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:00.530 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T07:13:00.530 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T07:13:00.531 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.a 2024-09-21T07:13:00.939 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 21 07:13:00 smithi114 systemd[1]: Stopping Ceph mds.a for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:01.315 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 21 07:13:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-a[72694]: 2024-09-21T07:13:00.937+0000 7f2bdf000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:01.316 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 21 07:13:00 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-a[72694]: 2024-09-21T07:13:00.937+0000 7f2bdf000640 -1 mds.a *** got signal Terminated *** 2024-09-21T07:13:03.815 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 21 07:13:03 smithi114 podman[122475]: 2024-09-21 07:13:03.558015172 +0000 UTC m=+2.736728251 container died c674d312586806a57dcd8aa7d9581afb0f2894632033d0d45189d12d5f0b57b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:13:04.315 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 21 07:13:04 smithi114 podman[122475]: 2024-09-21 07:13:04.025995067 +0000 UTC m=+3.204708145 container cleanup c674d312586806a57dcd8aa7d9581afb0f2894632033d0d45189d12d5f0b57b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-21T07:13:04.315 INFO:journalctl@ceph.mds.a.smithi114.stdout:Sep 21 07:13:04 smithi114 bash[122475]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-a 2024-09-21T07:13:04.514 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T07:13:04.586 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:04.586 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T07:13:04.586 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T07:13:04.586 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.d 2024-09-21T07:13:04.951 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:04 smithi114 systemd[1]: Stopping Ceph mds.d for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:05.316 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:04 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-d[71665]: 2024-09-21T07:13:04.949+0000 7f952b800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:05.316 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:04 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-d[71665]: 2024-09-21T07:13:04.949+0000 7f952b800640 -1 mds.d *** got signal Terminated *** 2024-09-21T07:13:09.236 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:08 smithi114 podman[122595]: 2024-09-21 07:13:08.96098336 +0000 UTC m=+4.108670011 container died 8dc8f81b75996e704e1f4cf57895437e899e618a5ba2a20afad266c26ccc9175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-21T07:13:09.565 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:09 smithi114 podman[122595]: 2024-09-21 07:13:09.446571294 +0000 UTC m=+4.594257943 container cleanup 8dc8f81b75996e704e1f4cf57895437e899e618a5ba2a20afad266c26ccc9175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-d, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T07:13:09.566 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:09 smithi114 bash[122595]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-d 2024-09-21T07:13:09.919 INFO:journalctl@ceph.mds.d.smithi114.stdout:Sep 21 07:13:09 smithi114 podman[122607]: 2024-09-21 07:13:09.656562705 +0000 UTC m=+0.681034577 container remove 8dc8f81b75996e704e1f4cf57895437e899e618a5ba2a20afad266c26ccc9175 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T07:13:09.943 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T07:13:10.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:10.014 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T07:13:10.014 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T07:13:10.014 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.g 2024-09-21T07:13:10.315 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:10 smithi114 systemd[1]: Stopping Ceph mds.g for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:10.815 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g[72394]: 2024-09-21T07:13:10.443+0000 7f14fb400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:10.815 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:10 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g[72394]: 2024-09-21T07:13:10.443+0000 7f14fb400640 -1 mds.g *** got signal Terminated *** 2024-09-21T07:13:11.815 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:11 smithi114 podman[122715]: 2024-09-21 07:13:11.544966613 +0000 UTC m=+1.263710552 container died c6a6a9c4caf1c431fa54d0570077c98d25d821b7f2b83da06a8e554764a8abe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T07:13:12.136 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:11 smithi114 podman[122715]: 2024-09-21 07:13:11.967459282 +0000 UTC m=+1.686203217 container cleanup c6a6a9c4caf1c431fa54d0570077c98d25d821b7f2b83da06a8e554764a8abe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T07:13:12.136 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:11 smithi114 bash[122715]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g 2024-09-21T07:13:12.386 INFO:journalctl@ceph.mds.g.smithi114.stdout:Sep 21 07:13:12 smithi114 podman[122727]: 2024-09-21 07:13:12.135139295 +0000 UTC m=+0.583432471 container remove c6a6a9c4caf1c431fa54d0570077c98d25d821b7f2b83da06a8e554764a8abe4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:13:12.392 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T07:13:12.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:12.460 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T07:13:12.460 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T07:13:12.460 DEBUG:teuthology.orchestra.run.smithi114:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.j 2024-09-21T07:13:12.816 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 07:13:12 smithi114 systemd[1]: Stopping Ceph mds.j for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:13.316 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 07:13:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j[71288]: 2024-09-21T07:13:12.840+0000 7f876fc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:13.316 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 07:13:12 smithi114 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j[71288]: 2024-09-21T07:13:12.840+0000 7f876fc00640 -1 mds.j *** got signal Terminated *** 2024-09-21T07:13:17.166 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 07:13:16 smithi114 podman[122835]: 2024-09-21 07:13:16.864067057 +0000 UTC m=+4.133720693 container died 1704241ca44d3fe22f6f0f5af51f7e43772e177efd02ef9eaf5f11d2489f66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T07:13:17.531 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 07:13:17 smithi114 podman[122835]: 2024-09-21 07:13:17.270852455 +0000 UTC m=+4.540506091 container cleanup 1704241ca44d3fe22f6f0f5af51f7e43772e177efd02ef9eaf5f11d2489f66a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-21T07:13:17.532 INFO:journalctl@ceph.mds.j.smithi114.stdout:Sep 21 07:13:17 smithi114 bash[122835]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-j 2024-09-21T07:13:17.705 DEBUG:teuthology.orchestra.run.smithi114:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T07:13:17.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:17.773 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T07:13:17.773 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T07:13:17.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.b 2024-09-21T07:13:18.094 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:13:17 smithi183 systemd[1]: Stopping Ceph mds.b for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:18.364 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:13:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: 2024-09-21T07:13:18.093+0000 7fda378e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:18.364 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:13:18 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b[63080]: 2024-09-21T07:13:18.093+0000 7fda378e9640 -1 mds.b *** got signal Terminated *** 2024-09-21T07:13:19.661 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:13:19 smithi183 podman[68594]: 2024-09-21 07:13:19.394637381 +0000 UTC m=+1.430542827 container died 6ed66db88e2214c16687189ca1fc3c030ae7d863f33adcf3995b4c934630acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:13:20.020 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:13:19 smithi183 podman[68594]: 2024-09-21 07:13:19.768093496 +0000 UTC m=+1.803998941 container cleanup 6ed66db88e2214c16687189ca1fc3c030ae7d863f33adcf3995b4c934630acdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T07:13:20.020 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 21 07:13:19 smithi183 bash[68594]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-b 2024-09-21T07:13:20.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T07:13:20.237 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:20.237 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T07:13:20.238 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T07:13:20.238 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.e 2024-09-21T07:13:20.573 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:13:20 smithi183 systemd[1]: Stopping Ceph mds.e for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:20.864 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:13:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e[62543]: 2024-09-21T07:13:20.571+0000 7f0c90f18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:20.864 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:13:20 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e[62543]: 2024-09-21T07:13:20.571+0000 7f0c90f18640 -1 mds.e *** got signal Terminated *** 2024-09-21T07:13:23.674 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:13:23 smithi183 podman[68713]: 2024-09-21 07:13:23.369273259 +0000 UTC m=+2.916202583 container died 86959222bd3a838916a0cc93e5a88cea0daa4bbab4b69e0fb7b9f3991eb2a1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T07:13:24.015 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:13:23 smithi183 podman[68713]: 2024-09-21 07:13:23.771733579 +0000 UTC m=+3.318662901 container cleanup 86959222bd3a838916a0cc93e5a88cea0daa4bbab4b69e0fb7b9f3991eb2a1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T07:13:24.015 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 21 07:13:23 smithi183 bash[68713]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-e 2024-09-21T07:13:24.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T07:13:24.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:24.201 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T07:13:24.201 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T07:13:24.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.h 2024-09-21T07:13:24.615 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:13:24 smithi183 systemd[1]: Stopping Ceph mds.h for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:24.615 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:13:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h[62812]: 2024-09-21T07:13:24.525+0000 7f210ace9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:24.615 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:13:24 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h[62812]: 2024-09-21T07:13:24.525+0000 7f210ace9640 -1 mds.h *** got signal Terminated *** 2024-09-21T07:13:27.702 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:13:27 smithi183 podman[68845]: 2024-09-21 07:13:27.368401757 +0000 UTC m=+2.947838827 container died 2f8e30bc628063275d4d2edb5efa35ea92a48f1b3542c16f989e785241377686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T07:13:28.011 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:13:27 smithi183 podman[68845]: 2024-09-21 07:13:27.800533556 +0000 UTC m=+3.379970626 container cleanup 2f8e30bc628063275d4d2edb5efa35ea92a48f1b3542c16f989e785241377686 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T07:13:28.011 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 21 07:13:27 smithi183 bash[68845]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-h 2024-09-21T07:13:28.229 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T07:13:28.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:28.262 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T07:13:28.262 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T07:13:28.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.k 2024-09-21T07:13:28.589 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:28 smithi183 systemd[1]: Stopping Ceph mds.k for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:28.864 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k[62263]: 2024-09-21T07:13:28.586+0000 7f272d868640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:28.864 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:28 smithi183 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k[62263]: 2024-09-21T07:13:28.586+0000 7f272d868640 -1 mds.k *** got signal Terminated *** 2024-09-21T07:13:31.681 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:31 smithi183 podman[68965]: 2024-09-21 07:13:31.414569179 +0000 UTC m=+2.931807429 container died 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T07:13:31.932 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:31 smithi183 podman[68965]: 2024-09-21 07:13:31.753771452 +0000 UTC m=+3.271009696 container cleanup 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T07:13:31.932 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:31 smithi183 bash[68965]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k 2024-09-21T07:13:31.932 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 21 07:13:31 smithi183 podman[68978]: 2024-09-21 07:13:31.929789842 +0000 UTC m=+0.505287282 container remove 9d4f9f8b7951332f6708f6094a78562fc93b07cbc42db64705906b3eb37076ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-21T07:13:32.185 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T07:13:32.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:32.218 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T07:13:32.218 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T07:13:32.218 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.c 2024-09-21T07:13:32.542 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:13:32 smithi193 systemd[1]: Stopping Ceph mds.c for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:32.903 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:13:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c[64237]: 2024-09-21T07:13:32.540+0000 7f3ed3a03640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:32.904 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:13:32 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c[64237]: 2024-09-21T07:13:32.540+0000 7f3ed3a03640 -1 mds.c *** got signal Terminated *** 2024-09-21T07:13:35.662 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:13:35 smithi193 podman[70154]: 2024-09-21 07:13:35.37631388 +0000 UTC m=+2.962234083 container died a12693ffa40c1e6f8df24f40ef9d7e8325db36546b81b7e7cef8bc45aef111ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T07:13:36.009 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:13:35 smithi193 podman[70154]: 2024-09-21 07:13:35.79879229 +0000 UTC m=+3.384712493 container cleanup a12693ffa40c1e6f8df24f40ef9d7e8325db36546b81b7e7cef8bc45aef111ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T07:13:36.010 INFO:journalctl@ceph.mds.c.smithi193.stdout:Sep 21 07:13:35 smithi193 bash[70154]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-c 2024-09-21T07:13:36.234 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T07:13:36.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:36.271 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T07:13:36.271 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T07:13:36.271 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.f 2024-09-21T07:13:36.596 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:13:36 smithi193 systemd[1]: Stopping Ceph mds.f for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:36.911 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:13:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f[63968]: 2024-09-21T07:13:36.594+0000 7f0ddeedc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:36.912 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:13:36 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f[63968]: 2024-09-21T07:13:36.594+0000 7f0ddeedc640 -1 mds.f *** got signal Terminated *** 2024-09-21T07:13:39.662 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:13:39 smithi193 podman[70288]: 2024-09-21 07:13:39.380524448 +0000 UTC m=+2.891298725 container died bc6d2fcb8cb37a04b6e5011dfa36c9367a83cd927bbe172f9790dde1aa3f2ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T07:13:40.022 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:13:39 smithi193 podman[70288]: 2024-09-21 07:13:39.75293637 +0000 UTC m=+3.263710646 container cleanup bc6d2fcb8cb37a04b6e5011dfa36c9367a83cd927bbe172f9790dde1aa3f2ee1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T07:13:40.022 INFO:journalctl@ceph.mds.f.smithi193.stdout:Sep 21 07:13:39 smithi193 bash[70288]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-f 2024-09-21T07:13:40.183 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T07:13:40.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:40.217 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T07:13:40.217 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T07:13:40.217 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.i 2024-09-21T07:13:40.662 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:13:40 smithi193 systemd[1]: Stopping Ceph mds.i for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:40.662 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:13:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i[64509]: 2024-09-21T07:13:40.523+0000 7f3814770640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:40.662 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:13:40 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i[64509]: 2024-09-21T07:13:40.523+0000 7f3814770640 -1 mds.i *** got signal Terminated *** 2024-09-21T07:13:43.662 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:13:43 smithi193 podman[70410]: 2024-09-21 07:13:43.39254913 +0000 UTC m=+2.959485806 container died 44628a12ac797a6731c5687c4c59ee11e584dc486b8528a84b8e437dd2c5e882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T07:13:44.026 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:13:43 smithi193 podman[70410]: 2024-09-21 07:13:43.77346781 +0000 UTC m=+3.340404485 container cleanup 44628a12ac797a6731c5687c4c59ee11e584dc486b8528a84b8e437dd2c5e882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T07:13:44.026 INFO:journalctl@ceph.mds.i.smithi193.stdout:Sep 21 07:13:43 smithi193 bash[70410]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-i 2024-09-21T07:13:44.195 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T07:13:44.227 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:44.228 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T07:13:44.228 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T07:13:44.228 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.l 2024-09-21T07:13:44.662 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:13:44 smithi193 systemd[1]: Stopping Ceph mds.l for 5d75913a-77e4-11ef-baf5-efdc52797490... 2024-09-21T07:13:44.662 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:13:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l[63691]: 2024-09-21T07:13:44.527+0000 7ff5c5dcc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T07:13:44.662 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:13:44 smithi193 ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l[63691]: 2024-09-21T07:13:44.527+0000 7ff5c5dcc640 -1 mds.l *** got signal Terminated *** 2024-09-21T07:13:47.662 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:13:47 smithi193 podman[70532]: 2024-09-21 07:13:47.395205662 +0000 UTC m=+2.955317485 container died e091ad4273b9875cd2b8f7afc24efafe3f2fa044809154e244b339ee3e3bd44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T07:13:48.021 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:13:47 smithi193 podman[70532]: 2024-09-21 07:13:47.760999866 +0000 UTC m=+3.321111692 container cleanup e091ad4273b9875cd2b8f7afc24efafe3f2fa044809154e244b339ee3e3bd44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T07:13:48.022 INFO:journalctl@ceph.mds.l.smithi193.stdout:Sep 21 07:13:47 smithi193 bash[70532]: ceph-5d75913a-77e4-11ef-baf5-efdc52797490-mds-l 2024-09-21T07:13:48.186 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d75913a-77e4-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T07:13:48.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T07:13:48.219 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T07:13:48.219 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T07:13:48.509 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T07:13:52.383 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T07:13:52.586 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T07:13:55.152 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T07:13:55.354 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T07:13:57.907 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:13:57.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:13:58.006 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T07:13:58.034 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T07:13:58.036 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi114/crash 2024-09-21T07:13:58.036 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash -- . 2024-09-21T07:13:58.104 INFO:teuthology.orchestra.run.smithi114.stderr:tar: /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T07:13:58.104 INFO:teuthology.orchestra.run.smithi114.stderr:tar: Error is not recoverable: exiting now 2024-09-21T07:13:58.107 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi183/crash 2024-09-21T07:13:58.108 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash -- . 2024-09-21T07:13:58.160 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T07:13:58.160 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-21T07:13:58.162 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi193/crash 2024-09-21T07:13:58.162 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash -- . 2024-09-21T07:13:58.208 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T07:13:58.208 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-21T07:13:58.209 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T07:13:58.209 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T07:13:58.323 INFO:teuthology.orchestra.run.smithi114.stdout:2024-09-21T06:57:01.267164+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-09-21T07:13:58.324 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-21T07:13:58.324 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[SEC\]' /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T07:13:58.409 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[ERR\]' /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T07:13:58.490 DEBUG:teuthology.orchestra.run.smithi114:> sudo egrep '\[WRN\]' /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T07:13:58.570 INFO:teuthology.orchestra.run.smithi114.stdout:2024-09-21T06:57:01.267164+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal) see mds.b log and `damage ls` output for details 2024-09-21T07:13:58.571 INFO:tasks.cephadm:Compressing logs... 2024-09-21T07:13:58.571 DEBUG:teuthology.orchestra.run.smithi114:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:13:58.575 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:13:58.578 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:13:58.613 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T07:13:58.614 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T07:13:58.614 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T07:13:58.614 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log 2024-09-21T07:13:58.616 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T07:13:58.620 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T07:13:58.620 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T07:13:58.620 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T07:13:58.620 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T07:13:58.620 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T07:13:58.622 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T07:13:58.622 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T07:13:58.622 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T07:13:58.623 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T07:13:58.650 INFO:teuthology.orchestra.run.smithi114.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T07:13:58.650 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-21T07:13:58.659 INFO:teuthology.orchestra.run.smithi114.stderr:gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T07:13:58.660 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T07:13:58.660 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T07:13:58.661 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T07:13:58.662 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T07:13:58.663 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T07:13:58.664 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T07:13:58.664 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T07:13:58.665 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T07:13:58.666 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log 2024-09-21T07:13:58.667 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T07:13:58.667 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T07:13:58.668 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T07:13:58.669 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T07:13:58.670 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T07:13:58.671 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.3.log: 87.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T07:13:58.683 INFO:teuthology.orchestra.run.smithi114.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-21T07:13:58.685 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.2% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T07:13:58.730 INFO:teuthology.orchestra.run.smithi114.stderr: 94.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T07:13:58.789 INFO:teuthology.orchestra.run.smithi193.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T07:13:58.798 INFO:teuthology.orchestra.run.smithi114.stderr: 94.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T07:13:58.876 INFO:teuthology.orchestra.run.smithi193.stderr: 90.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T07:13:58.892 INFO:teuthology.orchestra.run.smithi114.stderr: 84.5% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T07:13:58.920 INFO:teuthology.orchestra.run.smithi114.stderr: 90.2% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T07:13:59.061 INFO:teuthology.orchestra.run.smithi114.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T07:13:59.135 INFO:teuthology.orchestra.run.smithi114.stderr: 91.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T07:13:59.152 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 90.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T07:13:59.173 INFO:teuthology.orchestra.run.smithi183.stderr: 82.2% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T07:13:59.246 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T07:13:59.392 INFO:teuthology.orchestra.run.smithi193.stderr: 93.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T07:13:59.445 INFO:teuthology.orchestra.run.smithi193.stderr: 90.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T07:13:59.715 INFO:teuthology.orchestra.run.smithi183.stderr: 92.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T07:13:59.772 INFO:teuthology.orchestra.run.smithi193.stderr: 87.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T07:13:59.862 INFO:teuthology.orchestra.run.smithi183.stderr: 87.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T07:14:00.054 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T07:14:00.414 INFO:teuthology.orchestra.run.smithi114.stderr: 93.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T07:14:01.983 INFO:teuthology.orchestra.run.smithi193.stderr: 94.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T07:14:02.149 INFO:teuthology.orchestra.run.smithi183.stderr: 93.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T07:14:02.267 INFO:teuthology.orchestra.run.smithi114.stderr: 90.4% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T07:14:02.812 INFO:teuthology.orchestra.run.smithi193.stderr: 93.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T07:14:04.195 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T07:14:06.981 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T07:14:12.092 INFO:teuthology.orchestra.run.smithi183.stderr: 91.5% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T07:14:22.138 INFO:teuthology.orchestra.run.smithi114.stderr: 90.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T07:14:23.629 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T07:14:50.282 INFO:teuthology.orchestra.run.smithi183.stderr: 94.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T07:14:59.457 INFO:teuthology.orchestra.run.smithi193.stderr: 95.1% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T07:15:03.146 INFO:teuthology.orchestra.run.smithi193.stderr: 95.0% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T07:15:34.240 INFO:teuthology.orchestra.run.smithi114.stderr: 94.3% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T07:18:38.918 INFO:teuthology.orchestra.run.smithi193.stderr: 90.7% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T07:18:40.430 INFO:teuthology.orchestra.run.smithi183.stderr: 90.5% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T07:18:45.078 INFO:teuthology.orchestra.run.smithi183.stderr: 90.6% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T07:19:49.321 INFO:teuthology.orchestra.run.smithi183.stderr: 90.5% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T07:20:17.194 INFO:teuthology.orchestra.run.smithi183.stderr: 90.7% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T07:20:17.196 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-21T07:20:17.196 INFO:teuthology.orchestra.run.smithi183.stderr:real 6m18.603s 2024-09-21T07:20:17.196 INFO:teuthology.orchestra.run.smithi183.stderr:user 5m18.386s 2024-09-21T07:20:17.196 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m11.807s 2024-09-21T07:20:34.530 INFO:teuthology.orchestra.run.smithi193.stderr: 90.9% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T07:21:10.051 INFO:teuthology.orchestra.run.smithi193.stderr: 90.5% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T07:22:05.320 INFO:teuthology.orchestra.run.smithi193.stderr: 90.6% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T07:22:05.322 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-09-21T07:22:05.322 INFO:teuthology.orchestra.run.smithi193.stderr:real 8m6.730s 2024-09-21T07:22:05.322 INFO:teuthology.orchestra.run.smithi193.stderr:user 5m57.292s 2024-09-21T07:22:05.322 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m13.673s 2024-09-21T07:23:39.097 INFO:teuthology.orchestra.run.smithi114.stderr: 90.7% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T07:24:04.734 INFO:teuthology.orchestra.run.smithi114.stderr: 90.7% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T07:24:22.448 INFO:teuthology.orchestra.run.smithi114.stderr: 90.6% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T07:25:20.424 INFO:teuthology.orchestra.run.smithi114.stderr: 90.5% -- replaced with /var/log/ceph/5d75913a-77e4-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T07:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr: 2024-09-21T07:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:real 11m21.827s 2024-09-21T07:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:user 6m31.086s 2024-09-21T07:25:20.431 INFO:teuthology.orchestra.run.smithi114.stderr:sys 0m43.651s 2024-09-21T07:25:20.432 INFO:tasks.cephadm:Archiving logs... 2024-09-21T07:25:20.433 DEBUG:teuthology.misc:Transferring archived files from smithi114:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi114/log 2024-09-21T07:25:20.434 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:26:25.029 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi183/log 2024-09-21T07:26:25.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:27:18.175 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi193/log 2024-09-21T07:27:18.175 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:28:16.809 INFO:tasks.cephadm:Removing cluster... 2024-09-21T07:28:16.809 DEBUG:teuthology.orchestra.run.smithi114:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --force 2024-09-21T07:28:17.331 INFO:teuthology.orchestra.run.smithi114.stdout:Deleting cluster with fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T07:28:19.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --force 2024-09-21T07:28:19.378 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T07:28:20.028 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d75913a-77e4-11ef-baf5-efdc52797490 --force 2024-09-21T07:28:20.244 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 5d75913a-77e4-11ef-baf5-efdc52797490 2024-09-21T07:28:20.937 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T07:28:20.938 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T07:28:20.991 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T07:28:21.011 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T07:28:21.027 INFO:tasks.cephadm:Teardown complete 2024-09-21T07:28:21.028 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T07:28:21.045 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T07:28:21.045 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:28:21.050 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:28:21.054 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:28:21.170 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T07:28:21.181 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T07:28:21.181 DEBUG:teuthology.orchestra.run.smithi114:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:28:21.185 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:28:21.188 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:28:21.202 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:28:21.202 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:28:21.215 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:28:21.215 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-21T07:28:21.215 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 +53us[ +97us] +/- 54ms 2024-09-21T07:28:21.215 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -138us[ -138us] +/- 64ms 2024-09-21T07:28:21.215 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +160us[ +204us] +/- 69ms 2024-09-21T07:28:21.215 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:28:21.216 INFO:teuthology.orchestra.run.smithi114.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:28:21.217 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:28:21.217 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-21T07:28:21.217 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 +46us[ +58us] +/- 54ms 2024-09-21T07:28:21.218 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 -183us[ -170us] +/- 64ms 2024-09-21T07:28:21.218 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +150us[ +163us] +/- 69ms 2024-09-21T07:28:21.218 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:28:21.322 INFO:teuthology.orchestra.run.smithi114.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:28:21.322 INFO:teuthology.orchestra.run.smithi114.stdout:=============================================================================== 2024-09-21T07:28:21.322 INFO:teuthology.orchestra.run.smithi114.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +26us[ +20us] +/- 54ms 2024-09-21T07:28:21.322 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -175us[ -175us] +/- 64ms 2024-09-21T07:28:21.322 INFO:teuthology.orchestra.run.smithi114.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +140us[ +140us] +/- 69ms 2024-09-21T07:28:21.322 INFO:teuthology.orchestra.run.smithi114.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:28:21.324 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T07:28:21.334 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T07:28:21.335 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T07:28:21.361 DEBUG:teuthology.orchestra.run.smithi114:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:28:21.605 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:28:21.698 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:28:21.788 DEBUG:teuthology.orchestra.run.smithi114:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T07:28:21.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:21.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T07:28:21.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:21.882 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T07:28:21.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:21.910 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T07:28:21.921 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T07:28:21.962 INFO:teuthology.task.internal:Duration was 3273.832784 seconds 2024-09-21T07:28:21.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T07:28:22.012 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T07:28:22.012 DEBUG:teuthology.orchestra.run.smithi114:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:28:22.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:28:22.019 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:28:22.083 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:28:22.106 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:28:22.124 INFO:teuthology.orchestra.run.smithi114.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:28:22.666 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T07:28:22.666 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi114.front.sepia.ceph.com 2024-09-21T07:28:22.667 DEBUG:teuthology.orchestra.run.smithi114:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:28:22.722 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-21T07:28:22.722 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:28:22.755 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-21T07:28:22.756 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:28:22.793 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T07:28:22.793 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:28:22.797 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:28:22.807 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:28:22.869 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T07:28:22.869 DEBUG:teuthology.orchestra.run.smithi114:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:28:22.873 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:28:22.879 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:28:24.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T07:28:24.442 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T07:28:24.442 DEBUG:teuthology.orchestra.run.smithi114:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:28:24.500 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:28:24.548 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:28:24.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T07:28:24.591 DEBUG:teuthology.orchestra.run.smithi114:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:24.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:24.598 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:24.630 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-21T07:28:24.663 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-21T07:28:24.684 INFO:teuthology.orchestra.run.smithi114.stdout:kernel.core_pattern = core 2024-09-21T07:28:24.715 DEBUG:teuthology.orchestra.run.smithi114:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:24.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:24.748 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:24.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:24.766 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:24.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:24.784 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T07:28:24.796 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T07:28:24.797 DEBUG:teuthology.misc:Transferring archived files from smithi114:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi114 2024-09-21T07:28:24.798 DEBUG:teuthology.orchestra.run.smithi114:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:28:24.899 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi183 2024-09-21T07:28:24.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:28:24.940 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913559/remote/smithi193 2024-09-21T07:28:24.940 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:28:24.981 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T07:28:24.981 DEBUG:teuthology.orchestra.run.smithi114:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:28:24.985 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:28:24.987 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:28:25.033 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T07:28:25.045 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T07:28:25.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T07:28:25.113 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T07:28:25.113 DEBUG:teuthology.orchestra.run.smithi114:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:28:25.118 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:28:25.120 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:28:25.136 INFO:teuthology.orchestra.run.smithi183.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:28 /home/ubuntu/cephtest 2024-09-21T07:28:25.137 INFO:teuthology.orchestra.run.smithi193.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:28 /home/ubuntu/cephtest 2024-09-21T07:28:25.148 INFO:teuthology.orchestra.run.smithi114.stdout: 393253 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:28 /home/ubuntu/cephtest 2024-09-21T07:28:25.151 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T07:28:25.162 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T07:28:25.292 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 3273.832783937454 failure_reason: '"2024-09-21T06:57:01.267164+0000 mds.b (mds.0) 19 : cluster [WRN] Scrub error on inode 0x10000000007 (/volumes/qa/sv_1/17dcb9ad-b76e-4c95-912f-73d77a25c644/postgres/data/pg_wal) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-09-21T07:28:25.292 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:28:25.388 INFO:teuthology.run:FAIL